Math 409-509, Optimization (Spring 2011)
Lectures: Mon and Wed between 12:30-13:45 at CAS B26

Instructor: Emre Mengi Office: SCI 114 E-mail: emengi@ku.edu.tr Phone: 338-1658 Office Hours: Mon 11:15-12:15 Wed 14:00-15:00 (or by appointment) Teaching Assistant: Rumi Melih Pelen Office: SCI 156 E-mail: mpelen@ku.edu.tr Office Hours: Mon  15:00-17:00

Course Description: Optimization problems arise in various fields ranging from economics to physics and in our daily lives. Airlines arrange their schedules to maximize their profit subject to constraints imposed by limited resources such as number of crew-members and planes. Ray of light follows a path to minimize the travel time.  Two main ingredients of an optimization problem are an objective function which we want to minimize or maximize, a set of constraints which determines the set of allowable points over which the objective function must be minimized or maximized. The first part of this course will focus on unconstrained optimization problems in the absence of constraints. Constrained optimization problems will be considered in the second part. For both cases we will derive the optimality conditions (i.e. conditions that distinguish an optimal point from an ordinary point), introduce numerical algorithms to locate points satisfying the optimality conditions and analyze the convergence properties of the numerical algorithms. (See the course syllabus for issues such as grading and the formats of the exams.) Announcements: You can see your finals on June 9th, Thursday between 12:50-13:20. Please first have a look at the actual final and solutions.

---

Textbooks:

• Numerical Optimization - Class Notes by Philip E. Gill and Margaret H. Wright  (This is the main textbook. You can obtain a copy from the Xerox room in the student center.)

• Numerical Optimization 2nd Edition by Jorge Nocedal and Stephen J. Wright  (On several occasions we will also depend on this book. This book will be available at the reserve desk in the library.)

Resources:

• Lecture Notes
• Lecture 3 - Derivative of a Vector-valued Function, First Order Optimality Conditions
• Lecture 4 - Second Order Optimality Conditions
• Lecture 5 - Eigenvalue Characterization for Positive Definiteness
• Lecture 6 - Root Finding, Newton's Method for Univariate Functions, Sequences and their Convergence
• Lecture 7 - Newton's Method for Multivariate Functions
• Lecture 8 - Convergence of Newton's Method
• Lecture 9 - Line Search Algorithms
• Lecture 10 - Inexact Line Search
• Lecture 11 - Modified Newton's Method
• Lecture 12 - Quasi Newton Methods - Part I
• Lecture 13 - Quasi Newton Methods - Part II
• Lecture 14 - Convergence of Line Search Algorithms
• Lecture 15 - Nonlinear Optimization with Equality-Constraints (Part I)
• Lecture 16 - Nonlinear Optimization with Equality-Constraints (Part II)
• Lecture 17 - Nonlinear Optimization with Equality-Constraints (Part III)
• Lecture 18 - Nonlinear Optimization with Equality-Constraints (Part IV)
• Lecture 19 - Nonlinear Optimization with Inequality Constraints (Part I)
• Lecture 20 - Nonlinear Optimization with Inequality Constraints (Part II)
• Lecture 21 - Nonlinear Optimization with Inequality Constraints (Part III)
• Lectures 22,23 - Sensitivity of Constrained Optimization Problems and Duality for Linear Programs
• Lecture 24 - Primal-Dual Interior Point Methods for Linear Programs
• Lecture 25 - Practical Aspects of Primal-Dual Interior Point Methods for Linear Programs
• Lecture 26 - Penalty Function Methods for Nonlinear Programs
• Lecture 27 - Logarithmic Barrier Method 
• Some Additional Examples Regarding Linear Programs and Duality

• An Introductory Matlab Manual

• Past Exams (that I handed out at UC San Diego in Spring 2009)
• midterm 1
• midterm 2
• final

---

Homeworks: All homeworks are due by 14:00 on the indicated dates below. Homework 1 - Solutions Homework 2 - Solutions m-files: Newton.m, Newton_backtrack.m, Newton_optimize.m, linesearch.m, Rosenbrock.m, plotcont.m, contourro.m Homework 3 - Solutions m-files: linesearch_wolfe.m,  lszoom.m,  inside.m,  cubic_interp.m Homework 4 - Solutions m-file: Newton_backtrack.m Homework 5 Homework 6 (will not be collected) Midterm: The midterm was held during the regular lecture hour on April 25, Monday. Final: The final will take place on June 2nd, Thursday between 9:00-12:00 at SCI Z08.

Exams and Solutions: midterm --- solutions final --- solutions Important Enrollment Dates and Holidays: (Holidays are underlined.) February 14th, First Day of Classes February 16th-18th, Add-drops April 4th-8th, Spring Break April 15th, Last Day to Withdraw May 19th (Thr), Youth and Sports Day  May 27th, Last Day of Classes June 1st-14th, Finals