Multivariable Calculus

1. Partial derivatives and the gradient

Nothing new is needed to compute a partial derivative — it is exactly the single-variable machinery of the differentiation note, applied to a function of one variable at a time while the other is frozen. What genuinely is new, and is the subject of Section 2, is what it means for $f$ to be differentiable as a function of both variables together.

2. Differentiability: why partial derivatives are not enough

3. The multivariable chain rule

Taking $f=F$, $x(t)=t$, $y(t)=g(t)$ for an implicitly defined $y=g(x)$ solving $F(x,y)=0$ recovers exactly the informal computation of implicit differentiation in the differentiation note (Section 10): $F_x+F_y\,g'(x)=0$, so $g'(x)=-F_x/F_y$ wherever $F_y\neq0$ — now derived from this section's rigorous chain rule rather than asserted. Section 8 below supplies the missing piece that note deferred: why a differentiable $g$ solving $F(x,y)=0$ exists near a point with $F_y\neq0$ in the first place.

4. Directional derivatives and steepest ascent

By the Cauchy–Schwarz inequality, $D_{\mathbf u}f(a,b)=\nabla f(a,b)\cdot\mathbf u\leq\|\nabla f(a,b)\|\,\|\mathbf u\|=\|\nabla f(a,b)\|$, with equality iff $\mathbf u$ points in the same direction as $\nabla f(a,b)$ (the Cauchy–Schwarz equality case). So among all directions, the gradient's own direction is exactly the one maximizing the instantaneous rate of increase, and that maximum rate is $\|\nabla f(a,b)\|$ itself — the fact that makes $-\nabla f$ the universal choice of descent direction in gradient-based optimization.

5. Clairaut's theorem: equality of mixed partials

The proof is two applications of the one-variable MVT in each order, then a squeeze — the same double-MVT pattern used again in Section 6's proof of Taylor's theorem below, and structurally the same "compute two ways, equate, take a limit" strategy as the Cauchy Mean Value Theorem's role in L'Hôpital's rule (differentiation note, Section 8).

6. Taylor's theorem in two variables and the Hessian

The proof technique — restrict to the line segment joining the two points and invoke the one-variable theorem — is completely general and reduces essentially every multivariable differential fact used in this note back to single-variable calculus applied along a cleverly chosen path; Section 9's proof of the Lagrange multiplier condition uses exactly the same reduction, restricting to a constraint curve instead of a line.

7. Critical points and the second-derivative test

Points with $\nabla f=\mathbf0$ (or where $f$ fails to be differentiable) are critical points; as in the one-variable case, being critical is necessary but not sufficient for an extremum — classifying which is the job of the following test.

7.1 The test

7.2 Classification via Hessian eigenvalues

$H(a,b)$ is symmetric (Clairaut, Section 5), so by the spectral theorem (eigenvalues note, Section 6) it has two real eigenvalues $\lambda_1,\lambda_2$ with $\lambda_1\lambda_2=\det H=D$ and $\lambda_1+\lambda_2=\operatorname{tr}H=f_{xx}+f_{yy}$. $D>0$ means $\lambda_1,\lambda_2$ share a sign; $f_{xx}>0$ in that case forces both positive (a positive-definite Hessian — the quadratic form $Q$ from Section 7.1 is exactly the Rayleigh-quotient-flavored object of that note's Section 8), giving a local min, and both negative gives a local max. $D<0$ means $\lambda_1,\lambda_2$ have opposite signs — the Hessian is indefinite, curving up in one eigendirection and down in the other, precisely the saddle geometry in Figure 2. The sign conditions on $D$ and $f_{xx}$ in Section 7.1 are, in this light, just a determinant-and-trace-only way of reading off the eigenvalues' signs without computing them.

8. The Implicit Function Theorem

$F_y(a,b)\neq0$ is not a technicality: it is exactly the condition ruling out a vertical tangent to the curve $F=0$ at $(a,b)$ (where no function $y=g(x)$ could locally describe the curve at all, e.g. the circle $x^2+y^2=1$ at $(1,0)$, where $F_y=2y=0$ — precisely where the earlier informal formula $y'=-F_x/F_y$ from Section 3 blows up).

9. Lagrange multipliers

Geometrically: the constraint curve's tangent direction at $(a,b)$ is $(1,h'(a))$, and $g_x(a,b)+g_y(a,b)h'(a)=0$ says this tangent is orthogonal to $\nabla g(a,b)$ (as it must be — $g$ is constant along its own level curve, Section 4). The proof shows $f$'s directional derivative along that same tangent must vanish at an extremum, forcing $\nabla f(a,b)$ orthogonal to the tangent too — and two vectors both orthogonal to the same direction in the plane are parallel to each other.

10. Multiple integrals and vector calculus, stated without proof

The remaining classical machinery of multivariable calculus is integral rather than differential, and proving it rigorously (measure-theoretic Fubini, the change-of-variables formula via a Riemann-sum argument in $\mathbb R^n$, Green's theorem via a decomposition into rectangles) is a substantially larger undertaking than this note's scope. The statements are recorded here for reference, each a direct higher-dimensional generalization of a single-variable fact already proved in the integration note.

(See Marsden & Tromba, 2012, Ch. 6 & 8, for full statements and proofs, including the divergence and Stokes' theorems in three dimensions of which Green's theorem is the planar special case.)

11. Computation

The figures above are generated by multivariable-calculus/generate_figures.py. The snippet below verifies Clairaut's theorem numerically (finite-difference mixed partials computed in both orders) and checks the second-derivative test's predictions against a direct perturbation scan around each critical point.

import numpy as np

def f(x, y):
    return np.sin(x) * y**2 + x**2 * np.cos(y)

h = 1e-4
def fxy(x, y):
    return (f(x+h,y+h) - f(x+h,y-h) - f(x-h,y+h) + f(x-h,y-h)) / (4*h*h)
def fyx(x, y):
    return (f(x+h,y+h) - f(x-h,y+h) - f(x+h,y-h) + f(x-h,y-h)) / (4*h*h)

print(f"f_xy(0.7,-0.3) = {fxy(0.7,-0.3):.8f}")
print(f"f_yx(0.7,-0.3) = {fyx(0.7,-0.3):.8f}")

cases = {
    "x^2+xy+y^2":    (lambda x,y: x**2+x*y+y**2,   2, 2, 1),
    "-(x^2+xy+y^2)": (lambda x,y: -(x**2+x*y+y**2), -2, -2, -1),
    "x^2-y^2":       (lambda x,y: x**2-y**2,        2, -2, 0),
}
for name, (fn, fxx, fyy, fxy_) in cases.items():
    D = fxx*fyy - fxy_**2
    thetas = np.linspace(0, 2*np.pi, 200)
    vals = np.array([fn(0.05*np.cos(t), 0.05*np.sin(t)) - fn(0,0) for t in thetas])
    actual = "local min" if np.all(vals > 0) else "local max" if np.all(vals < 0) else "saddle"
    print(f"f={name:16s} D={D:5.1f} f_xx={fxx:3d}  scan-confirmed={actual}")

Actual output:

f_xy(0.7,-0.3) = -0.04517702
f_yx(0.7,-0.3) = -0.04517702

f=x^2+xy+y^2       D=  3.0 f_xx=  2  scan-confirmed=local min
f=-(x^2+xy+y^2)    D=  3.0 f_xx= -2  scan-confirmed=local max
f=x^2-y^2          D= -4.0 f_xx=  2  scan-confirmed=saddle

The two finite-difference mixed partials agree to 8 decimal places, confirming Clairaut's theorem numerically for a function with no special symmetry; and a fine scan of $f$ around each critical point (sampling $200$ directions on a small circle) confirms every classification the second-derivative test predicts, exactly as the contour figure shows geometrically.

12. Common pitfalls

13. Connections

• Differentiation. Nearly every proof here reduces to that note's one-variable machinery along a line or a slice: Fermat's theorem and the Mean Value Theorem power Sections 2, 5, 7, and 8; Taylor's theorem with Lagrange remainder is the engine of Section 6's proof; the chain rule (Section 3) is the direct two-variable generalization of that note's Section 4.
• Continuity and limits. The Intermediate Value Theorem and Extreme Value Theorem are used, respectively, in the Implicit Function Theorem's existence argument (Section 8) and the second-derivative test's closing compactness step (Section 7.1).
• Eigenvalues. Section 7.2 classifies a critical point by the sign of $D=\det H$, exactly the sign of the product of the Hessian's (necessarily real, by the spectral theorem) eigenvalues — the Rayleigh-quotient machinery of that note's Section 8 is what makes "positive-definite Hessian" a precise, checkable condition rather than a slogan.
• Integration. Section 10's Fubini and change-of-variables theorems are direct higher-dimensional generalizations of that note's iterated-computation and substitution results; Green's theorem is the planar Fundamental Theorem of Calculus, a boundary integral standing in for evaluating an antiderivative at two endpoints.
• Optimization (machine learning). Section 4's steepest-ascent property of the gradient is the entire justification for gradient descent moving along $-\nabla f$, proved convergent for convex, smooth objectives in that note's Section 4; Section 7's second-derivative test, generalized to $n$ dimensions via the same Hessian-eigenvalue logic, is exactly the tool that note's Section 10 uses to tell a saddle from a genuine minimum.

14. References

• Rudin, W. (1976). Principles of Mathematical Analysis (3rd ed.). McGraw-Hill.
• Apostol, T. M. (1974). Mathematical Analysis (2nd ed.). Addison-Wesley.
• Marsden, J. E., & Tromba, A. J. (2012). Vector Calculus (6th ed.). W. H. Freeman.
• Hubbard, J. H., & Hubbard, B. B. (2015). Vector Calculus, Linear Algebra, and Differential Forms (5th ed.). Matrix Editions.