Matthew Litwin

The Gradient

Now that we have a good dual basis for linear functionals on our tangent space, we can use it to express an important functional - the gradient of a function defined on our surface.

The Gradient

Consider a function F({ui})F(\{u_i\}) on our surface. We can look at the rate of change AiA_i of FF along the direction of our tangent vectors eie_i, and form the linear functional ff where

f(ei)=Ai=∂F({ui})∂uif(e_i) = A_i = \frac{\partial F(\{u_i\})}{\partial u_i}

Since ff is a linear function, we can express it in the dual basis. ff is called the gradient of FF, written ∇F\nabla F, and then

∇F=∑iAiei=∑i∂F({ui})∂uiei\nabla F = \sum_i A_ie^i = \sum_i \frac{\partial F(\{u_i\})}{\partial u_i} e^i

and now for any vector v=∑ixiejv = \sum_i x^i e_j in our tangent space, the change in ff when you displace by vv is

f(v)=∇F⋅v=∑ijAixj(ei⋅ej)=∑ijAixjδji=∑iAixif(v) = \nabla F \cdot v = \sum_{ij}A_i x^j (e^i \cdot e_j) = \sum_{ij}A_i x^j \delta^i_j = \sum_i A_i x^i

∇F⋅v=∑iAixi\nabla F \cdot v = \sum_i A_i x^i

This is how much FF changes with an (infinitesimal) displacement by a vector vv in the tangent space.

So for any unit vector vv, ∇F⋅v\nabla F \cdot v is the rate of change of FF along vv.

A Geometric Interpretation of the Gradient

This is the Directional derivative variously defined. For our purposes here we will define it for any nonzero vector vv, putting the normalization into the definition.

∇vF=∇F⋅v∣v∣\nabla_v F = \nabla F \cdot \frac{v}{|v|}

With this definition, for all λ≠0\lambda \neq 0,

∇vF=∇λvF\nabla_v F = \nabla_{\lambda v} F

and thus for any set of {λv0}\{\lambda v_0\} we can pick a λ\lambda and thus a v=λv0v = \lambda v_0 with

λ=∇v0F∣v0∣  ⟹  ∣v∣=∇vF\lambda = \frac{\nabla_{v_0} F}{|v_0|} \implies |v| = \nabla_v F

Now we can consider set D\mathcal{D} of all vectors such vectors vv in the tangent space such that

∣∇F⋅v∣∣v∣=∣v∣\frac{|\nabla F \cdot v|}{|v|} = |v|

For ∇F\nabla F, we also have

∇F⋅∇F∣∇F∣=∇F⋅∇F∣∇F∣=∣∇F∣2∣∇F∣=∣∇F∣\frac{\nabla F \cdot \nabla F}{|\nabla F|} = \frac{\nabla F \cdot \nabla F}{|\nabla F|} = \frac{|\nabla F|^2}{|\nabla F|} = |\nabla F|

So ∇F\nabla F is in D\mathcal{D}

But also ∇F\nabla F is the vector of largest length in D\mathcal{D}. To prove this we will need the Cauchy-Schwarz Inequality

u⋅v=⟨u,v⟩≤∣u∣∣v∣u \cdot v = \langle u,v \rangle \leq |u||v|

A Proof of the Cauchy-Schwarz Inequality

Form a quadratic function p(t)p(t)

p(t)=⟨tu+v,tu+v⟩=t2⟨u,u⟩+2t⟨u,v⟩+⟨v,v⟩p(t) = \langle tu+v,tu+v \rangle = t^2\langle u,u \rangle + 2t\langle u,v \rangle + \langle v,v \rangle

Since our quadratic form ⟨,⟩\langle ,\rangle is never negative, we have

t2⟨u,u⟩+2t⟨u,v⟩+⟨v,v⟩≥0 t^2\langle u,u \rangle + 2t\langle u,v \rangle + \langle v,v \rangle \geq 0

This is a parabola that has a minimum at p′(t)=0p'(t)=0

t=−⟨u,v⟩⟨u,u⟩t = -\frac{\langle u,v \rangle} {\langle u,u \rangle}

so plugging in this tt we have

⟨u,v⟩2⟨u,u⟩−2⟨u,v⟩2⟨u,u⟩+⟨v,v⟩≥0\frac{\langle u,v \rangle^2}{\langle u,u \rangle} -2 \frac{\langle u,v \rangle^2}{\langle u,u \rangle} + {\langle v,v \rangle} \geq 0

Multiplying by ⟨u,u⟩\langle u,u \rangle and simplifying and rearranging

⟨u,u⟩⟨v,v⟩≥⟨u,v⟩2\langle u,u \rangle \langle v,v \rangle \geq {\langle u,v \rangle}^2

Taking square roots and reversing the order

⟨u,v⟩≤⟨u,u⟩⟨v,v⟩=∣u∣∣v∣\langle u,v \rangle \leq \sqrt{\langle u,u \rangle} \sqrt{\langle v,v \rangle} = |u||v|

The steepest direction

And now we can consider any vv with

∇F⋅v∣v∣=∣v∣\frac{\nabla F \cdot v}{|v|} = |v|

and then apply Cauchy-Schwarz

∣v∣2=∇F⋅v≤∣∇F∣∣v∣|v|^2 = \nabla F \cdot v \leq |\nabla F||v|

and so

∣v∣≤∣∇F∣|v| \leq |\nabla F|

Two Interpretations of the Gradient

Now we have two ways to think about the gradient of a function FF.

In the dual space, ∇F\nabla F is a linear functional that describes the infinitesimal change of FF according to the partial derivatives of FF along the tangent basis vectors eie_i.

In the tangent space, ∇F\nabla F is a vector in the direction of the steepest change of FF, whose length is equal to the rate of change of FF in that direction.

Next Up

The Einstein summation convention.