KKT Conditions Explained with Worked Examples

For minimising subject to , the KKT conditions at a minimum are , , and complementary slackness : a constraint either is active or has zero multiplier.

Write 2x or 2*x to multiply, ^ for powers, and ; between simultaneous equations. You can also type factor, expand, derivative of, integrate … from a to b, limit … as x->0 or 15% of 80.

Try an example

Before you start

This page is easier if you are comfortable with:

What KKT conditions means

Minimise subject to , i.e. . Stationarity: , , so ; complementary slackness with the active constraint gives , , and the minimum value is 2. The multiplier says that relaxing the constraint by lowers the optimum by about (a shadow price).

Minimise subject to (): and , so . Minimise subject to (): the unconstrained minimum 3 violates the constraint, so the constraint is active, , and gives , with minimum value 4. If the unconstrained minimum were feasible, would be 0.

With only equality constraints the conditions reduce to Lagrange multipliers (Lagrange multipliers); for convex problems (convex functions) the KKT conditions are sufficient as well as necessary. They are the basis of LP duality (linear programming duality) and support vector machines; the stationarity equation uses the gradient.

What is it used for?

The KKT conditions describe the balance of forces at the best point of a constrained problem.

How to use kkt conditions

  1. Write the problem as minimise subject to .
  2. Write stationarity .
  3. Decide which constraints are active (); others have .
  4. Solve and check and feasibility.

Graph

Circle x² + y² = 2 touching the constraint line x + y = 2 at (1, 1)minimum (1, 1), λ = 2-2-1.5-1-0.500.511.522.5-2-1012Circle x² + y² = 2 touching the constraint line x + y = 2 at (1, 1)xyGraph with x from -2 to 2.5 and y from -2 to 2.5. Shows x² + y² = 2 (73); x + y = 2 (2); point minimum (1, 1), λ = 2 at (1, 1).

The level curve just touches the constraint where the gradients are parallel.

Worked examples

These examples are worked by hand, step by step, so you can compare your own working.

Example 1: x² + y² with x + y ≥ 2

Find the minimum point, multiplier and value.

Worked example
Answer
x=y=1, λ=2, f=2x=y=1,\ \lambda=2,\ f=2
  1. Stationarity
  2. Active constraint

Example 2: x² with x ≥ 1

Find the multiplier at the minimum.

Worked example
Answer
22
  1. 2x−λ=0 at x = 1

Example 3: (x − 3)² with x ≤ 1

Find the multiplier and the minimum value.

Worked example
Answer
λ=4, f=4\lambda=4,\ f=4
  1. 2(x−3)+λ = 0 at x = 1

Special cases

A few situations come up again and again. Knowing them saves time and helps you spot answers that cannot be right.

min x² s.t. x ≥ 1

x = 1, multiplier λ = 2

2x − λ = 0 at x = 1.

min x² + y² s.t. x + y = 2

x = y = 1, λ = 2

2x = λ.

Complementary slackness

λ(x − 1) = 0

Active constraint or λ = 0.

Value

f(1, 1) = 2

The constrained minimum.

Word problems

At a constrained minimum of a convex problem there are multipliers λᵢ ≥ 0 with ∇f + Σ λᵢ∇gᵢ = 0 (stationarity), primal feasibility, and complementary slackness λᵢgᵢ = 0.

Inequality constraint

Minimise x² subject to x ≥ 1. Find x and λ.

  1. Unconstrained minx = 0 infeasible
  2. Constraint activex = 1
  3. Stationarity2x = λ, so λ = 2
  4. Check: λ ≥ 0 ✓x = 1, λ = 2.

Equality constraint

Minimise x² + y² subject to x + y = 2.

  1. ∇f(2x, 2y) = λ(1, 1)
  2. x = y1
  3. Check: x + y = 2 ✓x = y = 1.

Value Higher

Find the minimum value.

  1. 1² + 1²
  2. Check: ✓2.

Inactive constraint Higher

Minimise (x − 3)² subject to x ≥ 1. Is the constraint active?

  1. Unconstrained minx = 3 ≥ 1
  2. Check: λ = 0 ✓No, λ = 0.

Common mistakes

Negative multiplier

λ = −2 ✗

λ ≥ 0 for ≥ constraints (in this form) ✓

Sign convention matters.

Forgetting complementary slackness

λ ≠ 0 and constraint slack ✗

λ(g) = 0 ✓

One of them must be zero.

Checking only stationarity

solved ∇f = 0 ✗

include constraints ✓

Check feasibility.

Applying to non-convex problems as sufficient

KKT guarantees optimum ✗

only necessary in general ✓

Convexity makes it sufficient.

Exam tips

  1. Stationarity.
  2. Feasibility.
  3. λ ≥ 0.
  4. Complementary slackness.
  5. Convex ⇒ sufficient.

Practice questions

Try each one on paper, then open the answer. Use the solver above to see the full working.

1. λ from 2x = λ at x = 1 Foundation

2

2. Minimum of x² + y² with x + y = 4: x Foundation

2

3. Minimum value of x² + y² with x + y = 4 Foundation

8

4. λ for that problem (2x) Higher

4

5. Min of x² s.t. x ≥ 3: value Higher

9

6. λ for x ≥ 3 Higher

6

Practice generator

Unlimited fresh questions with instant marking. Choose a level, work it out on paper, then type your answer.

Solve: …

Key terms

KKT conditions
Optimality conditions with inequality constraints.
Multiplier
Weight on a constraint.
Active constraint
Holds with equality.
Complementary slackness
λg = 0.
Feasible
Satisfies all constraints.
Lagrangian
f + Σ λᵢgᵢ.

Frequently asked questions

What are the KKT conditions?

Stationarity, feasibility, λ ≥ 0 and complementary slackness.

When are they sufficient?

For convex problems.

What is complementary slackness?

Each multiplier times its constraint is zero.

What is an active constraint?

One holding with equality.

Which page covers convexity?

convex functions.

Which page covers LP duality?

linear programming duality.

More in Numerical methods and optimisation and the formula sheets.