 |
|
||||||||||
Dynamic Programming and Optimal Control
Dynamic Programming and Optimal Control
4 credit points
Start: 28.09.11/
End: 21.12.11
Frequency: Annually, fall semester
Lecturer
Assistants
Nico Hübel,
Mohanarajah Gajamohan
Day/Time/Location
Wednesdays
13:00-16:00, ML H 44
Book
Dynamic Programming and Optimal Control by Dimitri P. Bertsekas, Vol. I, 3rd edition, 2005, 558 pages, hardcover.
Description:
Dynamic Programming Algorithm; Deterministic Systems and Shortest Path Problems; Infinite Horizon Problems; Value/Policy Iteration; Deterministic Continuous-Time Optimal Control; Problems with Perfect State Information.
Book: Dynamic Programming and Optimal Control by Dimitri P. Bertsekas, Vol. I, 3rd edition, 2005, 558 pages, hardcover.
Requirements: Knowledge of advanced calculus, introductory probability theory, and matrix-vector algebra.
Announcements
| Feb 06 | Solutions to the Final Exam are online. |
| Dec 20 | Quiz 2 will take place in ML D 28, 13:15-14:05 on Dec 21 (Wed). There will not be any lectures or recitation on Dec 21. |
| Dec 16 |
Final Examination Details: 26-Jan-2011 (Thu), 09:00-11:30, HIL F 41. Written aids: One A4 sheet of paper (2 pages, handwritten or computer typed) |
| Nov 11 | The recitation session next week, Nov 16, 15:15-16:00, is moved to Nov 22 (Tue), 18:00-18:45, due to the unavailability of both teaching assistants. The venue will remain the same (ML H 44). We apologize for the inconvenience. |
| Oct 30 | Quiz 1 will take place in ML D 28, 13:15-14:05 on Nov 02 (Wed). There will not be any lectures or recitation on Nov 02. |
| Sep 27 | The class fact sheet can be downloaded from here (factSheet2011.pdf). |
|
Aug 29 |
The recommended text book [Dynamic Programming and Optimal Control, Dimitri P. Bertsekas] will be for sale on Oct 05, 15:00 in front of the class room (Price: 120 Chf/ 100 Chf with discount). The books can be also be ordered from the published directly using the following link. http://www.athenasc.com/ordering.html |
|
Aug 03 |
There will be more information about the class Dynamic Programming and Optimal Control (Fall '11) on this page soon. |
Class Facts
| Instructor | Prof. Raffaello D'Andrea |
| Teaching Assistants | Nico Hübel, Mohanarajah Gajamohan |
| Lecture |
Wednesday, 13:15 to 15:00, ML H 44 |
| Exercise | Wednesday, 15:15 to 16:00, ML H 44 |
|
Office hours |
Thursday, 17:00 to 18:00, ML K 37.1 |
|
Exam |
TBA |
|
Grading |
40% quiz/programming exercises, 60% final exam if the grade for quiz and programming exercises is better than the grade in the final exam; 100% final exam otherwise. |
| Only the three best grades from the quiz and the programming exercises will count towards the 40% above. | |
| PhD students will get credits for the class if they pass the class (final grade of 4.0 or higher). | |
|
Repetition |
The final exam is only offered in the session after the course unit. Repetition is only possible after re-enrolling. |
Class Schedule
| # | date | topic | reading |
|
L1 |
Sep 28 |
The Dynamic Programming Algorithm |
1.1 - 1.4 |
|
L2 |
Oct 05 |
The Dynamic Programming Algorithm |
1.1 - 1.4 |
| L3 |
Oct 12 |
The Dynamic Programming Algorithm |
1.1 - 1.4 |
| L4 |
Oct 19 |
Deterministic Systems and the Shortest Path Problem |
2.1 - 2.3 |
| L5 |
Oct 26 |
Deterministic Systems and the Shortest Path Problem |
2.1 - 2.3 |
| Q1 | Nov 02 | Quiz 1: The Dynamic Programming Algorithm (No Lecture) | 1.1 - 1.4 |
| L6 |
Nov 09 |
Infinite Horizon Problems, Value Iteration, Policy Iteration |
7.1 - 7.3 |
| L7 |
Nov 16 |
Infinite Horizon Problems, Value Iteration, Policy Iteration |
7.1 - 7.3 |
|
L8 |
Nov 23 |
Infinite Horizon Problems, Value Iteration, Policy Iteration |
7.1 - 7.3 |
| L9 |
Nov 30 |
Deterministic Continuous-Time Optimal Control |
3.1 - 3.4 |
| L10 | Dec 07 | Deterministic Continuous-Time Optimal Control |
3.1 - 3.4 |
|
L11 |
Dec 14 | Deterministic Continuous-Time Optimal Control |
3.1 - 3.4 |
| Q2 | Dec 21 | Quiz 2: Deterministic Continuous-Time Optimal Control (No Lecture) | 3.1 - 3.4 |
Quizzes and Programming Exercises
During the semester, there will be graded quizzes and programming exercises, which can be used to improve the final grade for the course (see "grading"). Quizzes will take place at the beginning of the lecture and will test the student's understanding of the corresponding topic. The programming exercises will require the student to apply the lecture material.
Up to three students can work together on the programming exercises. If they do, they have to hand in one solution per group and will all receive the same grade.
| # | type | topic | dates | download |
| Q1 | Quiz | The Dynamic Programming Algorithm |
Nov 02 |
Solutions Results |
| P1 | Programming | Deterministic Systems and the Shortest Path Problem |
Nov 02 (issued) Nov 16 (due) |
ProgEx1_template ProgEx1.pdf Results |
| P2 | Programming | Value/Policy Iteration |
Nov 23 (issued) Dec 07 (due) |
ProgEx2 ProgEx2_FAQ.txt Errata.txt Results |
| Q2 | Quiz | Deterministic Continuous-Time Optimal Control |
Dec 21 |
Solutions Results |
Past Quizzes and Exams
| type | download |
| Quiz 2010 (The Dynamic Programming Algorithm) | Quiz & Solutions |
| Quiz 2010 (Deterministic Continuous-Time Optimal Control) | Quiz & Solutions |
| Final Exam 2010 | Final 2010 & Solutions |
| Quiz 2009 (The Dynamic Programming Algorithm) | Quiz & Solutions |
| Quiz 2009 (Deterministic Continuous-Time Optimal Control) | Quiz & Solutions |
| Final Exam 2009 | Final 2009 & Solutions |
| Midterm Exam 2008 | Midterm 2008 & Solutions |
| Final Exam 2008 | Final 2008 & Solutions |
Problem Sets
We will make sets of problems and solutions available online for the chapters covered in the lecture. It is the student's responsibility to solve the problems and understand their solutions. The TAs will answer questions in office hours and some of the problems might be covered during the exercises.
| # |
topic |
download |
|
1 |
The Dynamic Programming Algorithm | ProblemSet1.pdf |
| 2 | Deterministic Systems and the Shortest Path Problem | ProblemSet2.pdf |
| 3 | Infinite Horizon Problems, Value Iteration, Policy Iteration |
ProblemSet3.pdf script_P73c.m |
| 4 | Deterministic Continuous-Time Optimal Control | ProblemSet4.pdf |
Additional Reading
- Tutorial on Viscosity Solutions to the HJB equation.
- Additional material for those interested in the history of the Maximum Principle from a mathematical perspective.
- Optional Reading: http://spectrum.ieee.org/geek-life/profiles/2010-medal-of-honor-winner-andrew-j-viterbi : Some nice insights into the development of the Viterbi algorithm.
Wichtiger Hinweis:
Diese Website wird in älteren Versionen von Netscape ohne
graphische Elemente dargestellt. Die Funktionalität der
Website ist aber trotzdem gewährleistet. Wenn Sie diese
Website regelmässig benutzen, empfehlen wir Ihnen, auf
Ihrem Computer einen aktuellen Browser zu installieren. Weitere
Informationen finden Sie auf
folgender
Seite.
Important Note:
The content in this site is accessible to any browser or
Internet device, however, some graphics will display correctly
only in the newer versions of Netscape. To get the most out of
our site we suggest you upgrade to a newer browser.
More
information
