Lecture Outlines and Supplementary Material, CO367/CM442 Winter'11

This webpage contains files used during the lectures. Additional material/links are also included. Be aware that this webpage is continually being polished/changed.

CO367/CM442 is about Nonlinear Optimization and concentrates on Convex Optimization. In Winter'11 we are using the text Convex Optimization – Boyd and Vandenberghe. This book and the slides for the class lectures are available online. (This also includes videos of lectures given by Stephen Boyd in 2008. In addition, the old review sessions are useful.)
This course aims to cover parts (not all) of the eleven chapters from the text.
There will be six assignments that account for 40% of the final grade. These are due in class by 1:30PM on the due date. Late homework will not be accepted. You must work on your assignments on your own. Some of the assignments will use CVX. Please download and install this software.
There will be one midterm (20%) and a final exam (40%).

Lectures Date Subjects Covered Lecture and Supplementary Information
Lecture 29-34 (M-F)2. Mar. 21-Apr 1 constrained optimization lecture notes and primal-dual interior point methods
  • Announcements:
  • topics
    • Newton's Method; analystic centers; proof of exactness on linear equations
    • barrier method; barrier applied to LP to yield primal-dual interior point method.
Week 11 starts
Lecture 26-28 M-F. Mar. 14-18 Unconstrained Optimization ( pdf file); additional lecture notes; constrained optimization lecture notes
  • topics
    • descent methods, gradient descent, sufficient decrease
Week 10 starts
Lecture 23-25 M-F. Mar. 7-11 Unconstrained Optimization ( pdf file); additional lecture notes
  • topics
    • descent methods, gradient descent, sufficient decrease
Week 9 starts
Lecture 20-22 M-F. Feb. 28 - Mar. 4 Duality cont.... ( pdf file);
(FYI only: Numerical Linear Algebra background ( pdf file));
Unconstrained Optimization ( pdf file);
  • topics
    • KKT conditions/ strong duality/
    • perturbation and sensitivity analaysis
    • duality and problem reformulations
    • semidefinite programming, SDP (dual/constaint qualification/application MC)
    • descent methods, gradient descent, sufficient decrease
Week 8 starts
Lecture 17-19 M-F. Feb. 14-18 Duality cont.... ( pdf file)
  • Announcements:
    • TA/Jerry is back and will hold an extended office hour on Tues, Feb 15.
    • KKT conditions/ strong duality/
    • Using KKT to derive the spectral resolution of a symmetric matrix A
    • Using KKT to derive lower/upper bounds on eigenvalues based on the trace of A and of (A*A).
Week 7 starts
Lecture 14-16 M-F Feb. 7-11 Duality ( pdf file) continued
  • Announcements: For the midterm (Wed. Feb 16, 2011, 7-9PM, MC4058):
    • You may find the first two chapters of this book useful: The mathematics of nonlinear programming By Anthony L. Peressini, Francis E. Sullivan, J. Jerry Uhl, available on google scholar. In particular, please try the problems.
    • An outline of the sections to study for the midterm in our text:
      1. 2.1,2.2,2.3 (pages 21-38)
        2.5-2.5.2 (pages 46-51)
      2. 3.1 (page 67)
        3.1.3-3.1.7 (pages 69-77)
        3.2-3.2.4 (pages 79-87)
      3. 4.2.1-4.2.3 (pages 136-142)
        4.3 (pages 146-148)
        4.4 (pages 152-156)
      4. 5.1 (pages 215-220)
        5.2 (pages 223-226)
Week 6 starts
Lecture 13-14 W-F Feb. 2-4 Duality ( pdf file) Sections in text: 5.1 Lagrangian (space where Lagrange multipliers lie); Lagrange dual function and lower bounds; Examples: linear least squares, LP, partitioning (max-cut), minimum volume covering ellipsoid.
Lecture 12 M. Jan. 31 Convex Programs ( pdf file) Sections in text: 4.6, 4.7
  • For CVX: you can change the path on the windows environment using e.g. path = %path%;N:\co367.d\cvx
    you also need a startup file after starting MATLAB, e.g. startupcvx.m file for the appropriate path additions.
  • semidefinite programming; eigenvalue minimization; portfolio optimization (Pareto points)
Week 5 starts
Lecture 11 F. Jan. 28 Convex Programs ( pdf file) Pages in notes: 4-9 to 4-27.
Lecture 10 W. Jan. 26 Convex Programs ( pdf file) Pages in notes: 4-1 to 4-9. (Voronoi diagrams from assignment, also for your interest only: finitely generated/polyhedral cones)
quadratic programming, least squares including: bounding variance, LP with random costs, Markowitz portfolio opt. (there are many sources for info. on portfolio opt., e.g. this with conic opt.)
Lecture 9 M. Jan. 24 Convex Optimization Problems ( pdf file) Sections in text: 4.1,4.2.-4.2.4 (read 4.2.5), 4.3 download CVX
(gunzip and tar xvf OR unzip/xwinzip to get the cvx directory and follow the other installation instructions) create the startupcvx.m file for the appropriate path additions.
Try the cvx/matlab command quickstart.
Examples of Rockafellar-Pshenichni condition
Equivalent convex problems;
Week 4 starts
Lecture 8 F. Jan. 21 ( perspective, conjugate functions, generalized inequalities slides 3-6 to 3-24 and 3-31; Sections in text 3.1 to 3.2, basic properties in 3.3,3.4,3.6)
  • Announcements: Please note that we cannot handout solutions to the assignments. You will get comments on what went wrong. You should try and correct your errors yourself. If you still have questions, then please come see the instructor and/or the TA for the course.
  • Try MATLAB file Aoneseps.m, using file runeps.m. Can you explain the roundoff error?
  • ( These old review sessions from EE364a 2009 may be useful. In particular: Review session 1 is on: examples of convex sets; affine/convex;conic hulls; preserving convexity; convex cones. Review session 2 is on: convex functions; determining convexity; preserving convex functions; conjugate functions. )
  • optimization problems in standard form
    convex optimization problems
    Rockafellar-Pshenichni optimality condition (nonnegative directional derivatives)
Lecture 6-7 M-W Jan. 17-19 Convex functions cont... ( pdf file) Sections in text: 3.1.6 to 3.3.2, (and read 3.5 and 3.6) Examples of convex functions and matlab file for plots;
further examples of characterizing convex functions;
epigraph and sublevel set; Jensen's inequality;
operations that preserve convexity;
(read text: perspective, conjugate functions, log-convex and log-concave functions, convexity with respect to generalized inequalities)
Week 3 starts
Lecture 5 F. Jan. 14 Convex functions ( pdf file pgs 3-1to3-5);
Sections in text: 3.1.1 to 3.1.5 		Examples of convex functions;
Restriction to a line;
Lecture 4 W. Jan. 12 convex sets cont... Dual Generalized Inequalities (pdf file), pgs 2-16 to 2-23.
Lecture 3 M. Jan. 10 Chapter 2, Convex sets cont... ( pdf file, pgs 2-9 to 2-15)
Week 2 starts
Lecture 2 F. Jan. 7 Chapter 2, Convex Sets ( pdf file), pgs 2-1 to 2-8 Affine and convex sets; convex cones; hyperplanes and halfspaces; norm balls and norm cones
Lecture 1 W. Jan. 5 Introduction to CO367/CM442 ( pdf file); Chapter 1, pages 1-16 in text. Structure of Class
Math. Opt.; Least Squares; Nonliear Opt; Convex Opt;
CVX
Week 1 starts