### Lagrange Multipliers

```Math 452 - Advanced Calculus II
Maxima, Minima, Manifolds and Lagrange Multipliers
Definition 1. Let D be a compact set of Rn . We say that the function
f : D → R has a local maximum (respectively, local minimum) on D
at the point p ∈ D if and only if there exists an open ball B ⊂ D centered
at p such that f (x) ≤ f (p) [respectively, f (x) ≥ f (p) for all points x ∈ B.
Recall the well-known result from single-variable calculus that if the
differentiable function f : R → R has a local maximum or local minimum
at p ∈ R, then f (p) = 0.
Lemma 1. Let S ⊂ Rn , and ϕ : R → S be a differentiable curve with
ϕ(0) = a. If f is a differentiable real-valued function defined on some open
set containing S, and f has a local maximum (or local minimum) on S at
a, then the gradient vector ∇f (a) is orthogonal to the velocity vector ϕ (0).
Proof. On the blackboard.
Corollary 1. If U is an open set of Rn and a ∈ U is a point at which the
differentiable function f : U → R has a local maximum or local minimum,
then ∇f (a) = 0.
Definition 2. A set M ⊂ Rn is said to have a k-dimensional tangent
plane at the point a ∈ M if the union of all tangent lines to differentiable
curves on M passing through a is a k-dimensional plane.
Definition 3. The projection mapping πi : Rn → Rn−1 is defined by
removing the ith coordinate:
πi (x1 , . . . , xn ) = (x1 , . . . , xi , . . . , xn )
= (x1 , . . . , xi−1 , xi+1 , . . . , xn ) ∈ Rn−1 .
1
1
Single-constraint Optimization
Definition 4. The set P ⊂ Rn is called an (n − 1)-dimensional patch
if and only if for some integer i, 1 ≤ i ≤ n, there exists a differentiable
function h : U → R for U ⊂ Rn−1 open, such that
P = {x ∈ Rn : πi (x) ∈ U
and
xi = h(πi (x)) }.
NOTE: The concept of an (n − 1)-dimensional patch is equivalent to
having a permutation xi1 , . . . , xin of the coordinates x1 , . . . , xn and a differentiable function h : U → R on an open set U ⊂ Rn−1 such that:
P = {x ∈ Rn : (xi1 , . . . , xin−1 ) ∈ U
and xin = h(xi1 , . . . , xin−1 ) }
Definition 5. The set M ⊂ Rn is called an (n−1)-dimensional manifold
if and only if each point a ∈ M lies in an open subset U ⊂ Rn such that
U ∩ M is an (n − 1)-dimensional patch.
Theorem 1. If M is an (n − 1)-dimensional manifold in Rn , then at each
of its points M has an (n − 1)-dimensional tangent plane.
Proof. On the blackboard.
Theorem (Implicit Function Theorem). Let g : Rn → R be continuously
differentiable and suppose that g(a) = 0 and Dn g(a) = 0. Then there exists a
neighborhood U of a and a differentiable function f : V → R, with V ⊂ Rn−1
a neighborhood of (a1 , . . . , an−1 ), such that
U ∩ g −1 (0) = {x ∈ Rn : (x1 , . . . , xn−1 ) ∈ V
and
xn = f (x1 , . . . , xn1 )}.
Theorem 2. Suppose that g : Rn → R is continuously differentiable. If M
is the set of all points x ∈ S = g −1 (0) at which ∇g(x) = 0, then M is an
(n − 1)-manifold. Given a ∈ M , the gradient vector ∇g(a) is orthogonal to
the tangent plane to M at a.
Proof. On the blackboard.
2
Theorem 3. Suppose g : Rn → R is continuously differentiable and let
M be the set of points x ∈ Rn at which g(x) = 0 and ∇g(x) = 0. If the
differentiable function f : Rn → R attains a local maximum or minimum on
M at the point a ∈ M , then
∇f (a) = λ∇g(a)
for some number λ, denoted as the Lagrange multiplier.
Proof. On the blackboard.
2
Multiple-constraint Optimization
Definition 6. The set P ⊂ Rn is called a k-dimensional patch if and
only if there exists a permutation xi1 , . . . , xin of x1 , . . . , xn , and differentiable
function h : U → Rn−k for U ⊂ Rk open, such that
P = {x ∈ Rn : (xi1 , . . . , xik ) ∈ U
and
(xik+1 , . . . , xin ) = h(xi1 , . . . , xik ) }
Definition 7. The set M ⊂ Rn is called a k-dimensional manifold if and
only if each point a ∈ M lies in an open subset U ⊂ Rn such that U ∩ M is
a k-dimensional patch.
Theorem 4. If M is an k-dimensional manifold in Rn then, at each of its
points, M has a k-dimensional tangent plane.
Proof. On the blackboard.
Theorem (Implicit Mapping Theorem). Let g : Rn → Rm (m < n) be a
continuously differentiable map. Suppose that g(a) = 0 and that the rank of
the derivative matrix g (a) is m. Then there exists a permutation xi1 , . . . , xin
of the coordinates in Rn , an open set U ⊂ Rn containing a, an open subset
V ⊂ Rn−m containing b = πn−m (ai1 , . . . , ain ), and a differentiable mapping
h : V → Rm such that each point x ∈ U lies on S = g−1 (0) if and only if
(xi1 , . . . , xin−m ) ∈ V and
(xin−m+1 , . . . , xin ) = h(xi1 , . . . , xin−m ).
3
Theorem 5. Suppose that g : Rn → Rm is continuously differentiable. If
M is the set of all points x ∈ S = g−1 (0) for which the rank of g (x) is
m, then M is an (n − m)-manifold. Given a ∈ M , the gradient vectors
∇g1 (a), . . . , ∇gm (a) are all orthogonal to the tangent plane to M at a.
Theorem 6. Suppose g : Rn → Rm (m < n) is continuously differentiable
and let M be the set of points x ∈ Rn such that g(x) = 0 and the gradient
vectors ∇g1 (a), . . . , ∇gm (a) are linearly independent. If the differentiable
function f : Rn → R attains a local maximum or minimum on M at the
point a ∈ M , then there exist real numbers λ1 , . . . , λm (called Lagrange
multipliers) such that:
∇f (a) = λ1 ∇g1 (a) + . . . + λm ∇gm (a)
4
```