Optimization and Objective Function

Introduction to Optimization Course Notes for CO 250/CM 340 Fall 2012

c Department of Combinatorics and Optimization University of Waterloo

August 27, 2012

2

Contents
1 Introduction 1.1 An example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.1.1 1.1.2 1.2 1.3 1.2.1 1.3.1 1.3.2 1.4 1.4.1 1.4.2 1.5 1.5.1 1.5.2 1.6 1.6.1 1.6.2 1.6.3 1.7 1.8 2 The formulation . . . . . . . . . . . . . . . . . . . . . . . . . . . . Correctness . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Multiperiod models . . . . . . . . . . . . . . . . . . . . . . . . . . . Assignment problem . . . . . . . . . . . . . . . . . . . . . . . . . . Knapsack problem . . . . . . . . . . . . . . . . . . . . . . . . . . . Shortest path problem . . . . . . . . . . . . . . . . . . . . . . . . . Minimum cost perfect matching . . . . . . . . . . . . . . . . . . . . Minimum cost perfect matching . . . . . . . . . . . . . . . . . . . . Shortest path problem . . . . . . . . . . . . . . . . . . . . . . . . . Pricing a Tech Gadget . . . . . . . . . . . . . . . . . . . . . . . . . Finding a closest point feasible in an LP . . . . . . . . . . . . . . . . Finding a “central” feasible solution of an LP . . . . . . . . . . . . . 7 8 9 10 12 13 21 22 24 35 35 37 40 40 42 48 48 50 50 52 53 55 55 56

Linear programs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Integer programs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Optimization problems on graphs . . . . . . . . . . . . . . . . . . . . . . . .

Integer programs, continued . . . . . . . . . . . . . . . . . . . . . . . . . .

Non linear programs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Overview of the course . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Further reading and notes . . . . . . . . . . . . . . . . . . . . . . . . . . .

Solving linear programs 2.1 Possible outcomes . . . . . . . . . . . . . . . . . . . . . . . . . . .

Bibliography: [1] N. Alon, D. Moshkovitz, and S. Safra. Algorithmic construction of sets for k-restrictions. ACM Transactions on Algorithms, 2(2):153–177, 2006. [2] A. Ben-Tal, L. El Ghaoui, and A. Nemirovski. Robust Optimization. Princeton Series in Applied Mathematics. Princeton University Press, Princeton, NJ, 2009. [3] M. J. Best. Portfolio Optimization. Chapman & Hall/CRC Finance Series. CRC Press, Boca Raton, FL, 2010. [4] S. Boyd and L. Vandenberghe. Convex Optimization. Cambridge University Press, Cambridge, 2004. [5] V. Chv´ tal. Linear Programming. A Series of Books in the Mathematical Sciences. W. H. a Freeman and Company, New York, 1983. [6] W. J. Cook, W. H. Cunningham, W. R. Pulleyblank, and A. Schrijver. Combinatorial Optimization. Wiley-Interscience Series in Discrete Mathematics and Optimization. John Wiley & Sons Inc., New York, 1998. [7] G. Cornu´ jols and R. T¨ t¨ nc¨ . Optimization Methods in Finance. Mathematics, Finance and e uu u Risk. Cambridge University Press, Cambridge, 2007. [8] G. B. Dantzig. Linear Programming and Extensions. Princeton University Press, Princeton, N.J., 1963. [9] E. W. Dijkstra. A note on two problems in connexion with graphs. Numerische Mathematik, 1:269–271, 1959. [10] I. Dinur and S. Safra. The importance of being biased. In Proceedings, ACM Symposium on Theory of Computing, pages 33–42, 2002. [11] U. Feige. A threshold of ln n for approximating set cover. J. ACM, 45(4):634–652, 1998. [12] M. L. Fredman and R. E. Tarjan. Fibonacci heaps and their uses in improved network optimization algorithms. Journal of the ACM, 34(3):596–615, July 1987. [13] G. H. Golub and C. F. Van Loan. Matrix Computations. Johns Hopkins Studies in the Mathematical Sciences. Johns Hopkins University Press, Baltimore, MD, third edition, 1996. 261 262 BIBLIOGRAPHY [14] N. Karmarkar. A new polynomial-time algorithm for linear programming. Combinatorica, 4(4):373–395, 1984. [15] L. G. Khachiyan. A polynomial algorithm in linear programming. Dokl. Akad. Nauk SSSR, 244(5):1093–1096, 1979. [16] S. Khot and O. Regev. Vertex cover might be hard to approximate to within 2-epsilon. J. Comput. Syst. Sci., 74(3):335–349, 2008. [17] R. D. C. Monteiro, I. Adler, and M. G. C. Resende. A polynomial time primal-dual affine scaling algorithm for linear and convex quadratic programming and its power series extension. Mathematics of Operations Research, 15:191–214, 1990. [18] A. L. Peressini, F. E. Sullivan, and J. J. Uhl. The Mathematics of Nonlinear Programming. Undergraduate Texts in Mathematics. Springer-Verlag, New York, 1988. [19] M. L. Pinedo. Scheduling. Springer, New York, third edition, 2008. Theory, algorithms, and systems, With 1 CD-ROM (Windows, Macintosh and UNIX). [20] R. Raz and S. Safra. A sub-constant error-probability low-degree test, and a sub-constant errorprobability pcp characterization of np. In Proceedings, ACM Symposium on Theory of Computing, pages 475–484, 1997. [21] A. Schrijver. Theory of Linear and Integer Programming. Wiley-Interscience Series in Discrete Mathematics. John Wiley & Sons Ltd., Chichester, 1986. [22] R. J. Vanderbei. Linear Programming. International Series in Operations Research & Management Science, 37. Kluwer Academic Publishers, Boston, MA, second edition, 2001. [23] V. V. Vazirani. Approximation Algorithms. Springer, 2001. [24] D. P. Williamson and D. Shmoys. The Design of Approximation Algorithms. Cambridge University Press, 2011. [25] W. L. Winston. Operations Research, Applications and Algorithms. Thomson Learning, 2004. [26] L. A. Wolsey. Integer Programming. Wiley-Interscience Series in Discrete Mathematics and Optimization. John Wiley & Sons Inc., New York, 1998. [27] Y. Ye. Interior Point Algorithms. Wiley-Interscience Series in Discrete Mathematics and Optimization. John Wiley & Sons Inc., New York, 1997. Index st-cut, 43 active constraints, 106 adjacent vertices, 35 affine function, 12, 234 Alternating path, 154 Augmenting path, 155 auxiliary linear program, 93, 94 variables, 93 basic feasible solution, 71 basic solution, 70 basis, 70 feasible, 71 bipartite, 38 branch & bound, 227 branch & bound nodes, 228 branch & bound tree, 229, 230 branch and cut, 213 branching on a variable, 228 canonical form, 72 certificate of infeasibility, 58 certificate of unboundedness, 60 Complementary Slackness Theorem, 181 cone, 185 cone of tight constraints, 186 convex hull, 214 convex set, 104 Cycle, 156 Dantzig’s rule, 89 decision variable, 24 deficient set, 146 depth-first search, 230 differentiability, 247 dual, 123, 141 dual Simplex algorithm, 231 edge, 35 edge cover, 46 epigraph of a function, 238 equivalent linear programs, 64 Exposed vertex, 155 extreme point, 106 Farkas’ Lemma, 58 feasible region, 102 Fermat’s Last Theorem, 236 free variable, 114 Fundamental Theorem of LP, 56, 97 Fundamental Theorem of LP (SEF), 97 gradient, 247 Graph Connected, 156 graph, 35 halfspace, 103 Hall’s Theorem, 146 hyperplane, 103 incident, 35 input-output systems, 205 integer program, 21 IP, 22 mixed, 22 pure, 22 Karush-Kuhn-Tucker Theorem, 248, 251 Lagrangian, 251 263 264 largest coefficient rule, 89 largest improvement rule, 89 law of diminishing marginal returns, 205 Leaf, 156 level set, 240 lexicographic rule, 114 linear constraint, 12 linear program, 12 feasible, 55 infeasible, 55 LP, 12 unbounded, 55 Markowitz model, 54 Matching Maximum, 155 matching, 38 neighbours, 146 non linear program, 48 non linear program:NLP, 48 optimal value, 55 path, 37 perfect matching, 38 pivot rules, 114 polyhedron, polyhedra, 104 portfolio optimization, 54 potential function, 258 pruning, 230 reducible, 91 robust optimization, 54 shadow prices, 203, 205 Simplex algorithm, 85 Slater point, 248 smallest subscript rule, 85, 97 solution feasible, 55 optimal, 55 standard equality form (SEF), 63 steepest edge rule, 89 Strong Duality Theorem, 175 subgradient, 242 supporting halfspace, 243 tight constraint, 106 Tree, 156 value, objective function value, 55 variable basic, 70 free, 64 non-basic, 70 slack, 66 vertex, 35 vertex cover, 46, 127 INDEX Weak Duality Theorem, 123, 141, 168