Numerical Analysis

The process of using a black box program to automatically differentiate every line of a source code using repeated applications of the derivative chain rule. The result is a transformed program that accomplishes the original computation and computes derivatives as well.

This is extremely useful when dealing with legacy programs, as one can quickly ascertain a gradient vector or even a Jacobian Matrix without resorting to finite difference techniques, or the tedium of manual differentiation of a 60000 line program.

AD codes like ADIFOR or ADOL-C operate in both forward and backward modes. The backward mode requires more memory, but is faster.