Jacobian-Vector Products (JVP)

Functions for computing Jacobian-vector products efficiently without forming the full Jacobian matrix.

Mathematical Background

The JVP computes J(x) * v where J(x) is the Jacobian of function f at point x and v is a direction vector. This is computed using finite difference approximations:

Forward: J(x) * v ≈ (f(x + h*v) - f(x)) / h
Central: J(x) * v ≈ (f(x + h*v) - f(x - h*v)) / (2h)

where h is the step size and v is the direction vector.

Step Size

Because f is evaluated at x + h*v, the step h has units of [x]/[v]: it is not a step in x and it is not dimensionless. The step used is

h = max(relstep * norm(x), absstep) * dir / norm(v)

so that the perturbation h*v is a relstep relative change of x, floored at absstep, and the computed JVP is invariant (up to that floor) to rescaling of v. This matters for matrix-free Krylov solvers, where v is a unit-norm basis vector and norm(x) may be large: a step that ignores norm(x) is stuck at the absstep floor and the JVP loses accuracy proportionally to norm(x).

Performance Benefits

JVP functions are particularly efficient when you only need directional derivatives:

Function evaluations: Only 2 function evaluations (vs O(n) for full Jacobian)
Forward differences: 2 function evaluations, O(h) accuracy
Central differences: 2 function evaluations, O(h²) accuracy
Memory efficient: No need to store the full Jacobian matrix

Use Cases

JVP is particularly useful for:

Optimization: Computing directional derivatives along search directions
Sparse directions: When v has few non-zero entries
Memory constraints: Avoiding storage of large Jacobian matrices
Newton methods: Computing Newton steps J⁻¹ * v iteratively

Limitations

Complex step: JVP does not currently support complex step differentiation (Val(:complex))
In-place functions: For in-place function evaluation, ensure proper cache sizing

Functions

FiniteDiff.finite_difference_jvp — Function

FiniteDiff.finite_difference_jvp(
    f,
    x      :: AbstractArray{<:Number},
    v      :: AbstractArray{<:Number},
    fdtype :: Type{T1}                = Val{:forward},
    f_in                              = nothing;
    relstep = default_relstep(fdtype, eltype(x)),
    absstep = relstep)

Compute the Jacobian-vector product J(x) * v using finite differences.

This function computes the directional derivative of f at x in direction v without explicitly forming the Jacobian matrix. This is more efficient than computing the full Jacobian when only J*v is needed.

Arguments

f: Function to differentiate (vector→vector map)
x::AbstractArray{<:Number}: Point at which to evaluate the Jacobian
v::AbstractArray{<:Number}: Direction vector for the product
fdtype::Type{T1}=Val{:forward}: Finite difference method (:forward, :central)
f_in=nothing: Pre-computed f(x) value (if available)

Keyword Arguments

relstep: Relative step size (default: method-dependent optimal value)
absstep=relstep: Absolute step size fallback

The step is ϵ = max(relstep*norm(x), absstep)*dir/norm(v), i.e. relstep and absstep set the size of the perturbation ϵ*v relative to x, and the result is unchanged (up to the absstep floor) if v is rescaled.

Returns

Vector J(x) * v representing the Jacobian-vector product

Examples

f(x) = [x[1]^2 + x[2], x[1] * x[2], x[2]^3]
x = [1.0, 2.0]
v = [1.0, 0.0]  # Direction vector
jvp = finite_difference_jvp(f, x, v)  # Directional derivative

Mathematical Background

The JVP is computed using the finite difference approximation:

Forward: J(x) * v ≈ (f(x + h*v) - f(x)) / h
Central: J(x) * v ≈ (f(x + h*v) - f(x - h*v)) / (2h)

where h is the step size and v is the direction vector.

Notes

Requires only 2 function evaluations (vs O(n) for full Jacobian)
Forward differences: 2 function evaluations, O(h) accuracy
Central differences: 2 function evaluations, O(h²) accuracy
Particularly efficient when v is sparse or when only one directional derivative is needed

source

FiniteDiff.finite_difference_jvp(
    f,
    x,
    v,
    cache::JVPCache;
    relstep = default_relstep(fdtype, eltype(x)),
    absstep = relstep)

Cached.

source

FiniteDiff.finite_difference_jvp! — Function

finite_difference_jvp!(
    jvp        :: AbstractArray{<:Number},
    f,
    x          :: AbstractArray{<:Number},
    v          :: AbstractArray{<:Number},
    fdtype     :: Type{T1}          = Val{:forward},
    returntype :: Type{T2}          = eltype(x),
    f_in       :: Union{T2,Nothing} = nothing;
    relstep = default_relstep(fdtype, eltype(x)),
    absstep = relstep)

Cache-less.

source

FiniteDiff.finite_difference_jvp!(
    jvp   :: AbstractArray{<:Number},
    f,
    x     :: AbstractArray{<:Number},
    v     :: AbstractArray{<:Number},
    cache :: JVPCache;
    relstep = default_relstep(fdtype, eltype(x)),
    absstep = relstep,
    dir     = true)

Cached.

source

Cache

FiniteDiff.JVPCache — Type

JVPCache{X1, FX1, FDType}

Cache structure for Jacobian-vector product (JVP) computations.

Stores temporary arrays needed for efficient JVP computation without repeated allocations. The JVP computes J(x) * v where J(x) is the Jacobian of function f at point x and v is a vector.

Fields

x1::X1: Temporary array for perturbed input values
fx1::FX1: Temporary array for function evaluations

source