Linear Programming Learning Notes (2) The Simplex Method

All the resources come from Linear Programming: Foundations and Extensions by Professor Robert J. Vanderbei.
Explore the link below for further information:
LP Book Resources

Part 1

Basic Theory: The Simplex Method and Duality

Chapter 2 The Simplex Method

An Example

maximize        5x1+4x2+3x3
s.t.                   2x1+3x2+x3≤5
                        4x1+x2+2x3≤11
                        3x1+4x2+2x3≤8
                                     x1,x2,x3≥0            (2.1)

Step1: Rewrite with Slack Variables

maximize        ζ=5x1+4x2+3x3
s.t.                   w1=5−2x1−3x2−x3
                        w2=11−4x1−x2−2x3
                        w3=8−x1−4x2−2x3
                        x1,x2,x3,w1,w2,w3≥0     (2.2)

Step2: Iteration

(2.2) is a so-called Feasible Dicionary Solution.

1. To start the iterative process, we need an initial feasible solution. Here, we simply set all the original variables to zero and determine the slack variables using the defining equations:
   x1=0,x2=0,x3=0,w1=5,w2=11,w3=8
2. Improve ζ by increasing the decision variables with the biggest positive coefficients, but satisfying the constraints.
   Here we increase x1 since its coefficient is biggest 5. Satisfying the w1,w2,w3constraints, we finally get x1≤5/2, so our new, improved solution then is :
   x1=5/2,x2=0,x3=0,w1=0,w2=1,w3=1/2
3. To proceed, we need to have one group of variables that were initially zero and we had the rest explicitly expressed in terms of these. The only thing we need to do is to swap the roles of x1and w1, by using the equation for w1 in (2.2) to solve for x1:
   x1=52−12w1−32x2−12x3

4. Perform row operations on the equations in (2.2):
   ζ=12.5−2.5w1−3.5x2+0.5x3−−−−−−−−−−−−−−−−−−−−−−−−−−
   x1=2.5−0.5w1−1.5x2−0.5x3
   w2=1+2w1+5x2
   w3=0.5+1.5w1+0.5x2−0.5x3               (2.3)

5. Repeat 2-4 steps until all the coefficients of the variables in ζ are negative. The final result is :
   ζ=13−w1−3x2−w3−−−−−−−−−−−−−−−−−−−
   x1=22−2w1−2x2+x3
   w2=1+2w1+5x2
   x3=1+3w1+x2−2w3
   with x1=2,x2=0,x3=1,w1=0,w2=1,w3=0               (2.4)

Theory

1. Dictionaries, Bases, Etc.
The systems of equations(2.2),(2.3)and (2.4) are dictionaries. With the exception of ζ, the variables that appear on the left (dependent variables) are called basic variables. Those on the right (independent variables) are called nonbasic variables. The solutions we have obtained by setting the nonbasic variables to zero are called basic feasible solutions.

2. The Simplex Method

• Standard Form:

  maximize        ∑nj=1cjxj
  s.t.                   ∑nj=1aijxj≤bi,i=1,2,...,m
                                          xj≥0,j=1,2,...,n

• Rewrite with Slack Variables:
  maximize        ζ=∑nj=1cjxj
  s.t.                   wi=b−∑nj=1aijxj≤bi,i=1,2,...,m

• Interwine the slack variables with the originals:
  (x1,...,xn,w1,...,wm)=(x1,...,xn,xn+1,...,xn+m)

• Iterations of the Simplex Method:
  Let B denote the collection of indices from {1,2,...,n+m} corresponding to the basic variables, and let N denote that to the nonbasic ones. Initially, we have N={1,2,...,n} and B={n+1,n+2,...,n+m}, but this changes after iterations.

  1. Pick one variable to transfer it from basic to nonbasic. This one is called the entering variable. It is chosen with the aim of increasing ζ: that is, one whose coefficient is positive. There are several selection criteria, here it is sufficed to say that we usually pick an index k from {j∈N:cj¯¯¯>0} having the largest coefficient. The bar put over the coefficient indicate that they change as the algorithm progresses.
  2. Determine the corresponded leaving variable, the one goes from basic to nonbasic. It is chosen to preserve nonnegativity of the current basic variables.
     We need to ensure that each of the variables remains nonnegative. Hence, we require that:
     b¯i−a¯ikxk=≥0,i∈B
     so, we get:
     xk=mini∈B:a¯ik>0b¯i/a¯ik
     The indice of the leaving variable l is pick l from {i∈B:b¯i/a¯ik is minimal}.
     It will change from one dictionary to the next, which is called a pivot. Particular rules that make the entering variable choice unambiguous are called pivot rules.
  3. Repeat step 1 and 2 until there is no coefficients of nonbasic variables in ζ is positive.

  3. Initializtion
  The right-hand sides of the dictionary being all nonnegative ensures that the initial dictionary is feasible, but this is not always the case. We handle this difficulty by introducing an auxiliary problem for which
  (1) A feasible dictionary is easy to find and
  (2) The optimal dictionary provides a feasible dictionary for the original problem.
  The auxiliary problem is :
  maximize        −x0
  s.t.                   ∑nj=1aijxj−x0≤bi,i=1,2,...,m
                                          xj≥0,j=0,1,2,...,n
  It is best to illustrate how to obtain a feasible dictionary with an example:
  maximize        −2x1−2x2
  s.t.                   −x1+x2≤−1
                          −x1−2x2≤−2
                                         x2≤1
                                     x1,x2≥0
  The auxiliary problem is
  maximize        −x0
  s.t.                   −x1+x2−x0≤−1
                          −x1−2x2−x0≤−2
                                         x2−x0≤1
                                     x0,x1,x2≥0
  The initial infeasible dictionary:
  ζ=−x0−−−−−−−
  w1=−1+x1−x2+x0
  w2=−2+x1+2x2+x0
  w3=1−x2+x0
  To convert it into a feasible one, all we need to do is one pivot with variable x0 entering and the “most infeasible variable”, w2, leaving the basis: (This is called Phase I)
  ζ=−2+x1+2x2−w2−−−−−−−−−−−−−−−−−−−
  w1=1−3x2+w2
  x0=2−x1−2x2+w2
  w3=3−x1−3x2+w2
  Using the steps introduced section before, we get the optimal dictionary: (This is called Phase II)
  ζ=0−x0−−−−−−−−−
  x2=0.33−0.33w1+0.33w2
  x1=1.33−x0+0.67w1+0.33w2
  w3=0.67+x0+0.33w1−0.33w2
  We now drop x0 from the equations and reintroduce the original objective function:
  ζ=−2x1−x2=−3−w1−w2
  Hence, the starting feasible dictionary for the original problem is :
  ζ=−2x1−x2=−3−w1−w2−−−−−−−−−−−−−−−−−−−−−−−−−
  x2=0.33−0.33w1+0.33w2
  x1=1.33+0.67w1+0.33w2
  w3=0.67+0.33w1−0.33w2
  4. Unboundedness
  If all the ratios a¯ik/b¯i are non positive, then the entering variable can be increased indefinitely to produce an arbitrarily large objective value. In such situations, we say that the problem is unbounded. For example:
  ζ=5+x3−x1−−−−−−−−−−−−−
  x2=5+2x3−3x1
  x4=7−4x1
  x5=x1
  It is unbounded.