摘要

Dynamic programming is a very general optimization method for sequential decision making, with many practical applications in engineering design, operations research, optimal resource allocation, automatic control, dynamic planning, economics and finance, and combinatorial optimization. Among optimization methods, it has the broadest range of applications:  deterministic, stochastic, discrete, and continuous problems.  However, it suffers from the curse of dimensionality: an exponential growth of computational requirements as the problem size increases.

This has led to extensive work over the last twenty years on the methodology of neuro-dynamic programming/reinforcement learning, which is based on various types of approximations and simulation, and can deal with problems of very large size. One key idea is to construct off-line, using simulation, an (approximate) scoring function which is used in real-time to rank decisions at any system state that may arise. This is much like what is done in computer chess and computer backgammon, where positions are evaluated by means of a scoring function, and the move that leads to the position with the best score is chosen. Another important idea is to use simulation and/or heuristics to compute on-line the values of an approximate scoring function. This talk will overview these methodologies, emphasizing on-line methods. A followup talk will emphasize off-line methods, and discuss recent extensions to Monte-Carlo methods for approximate solution of large systems of equations arising in broader scientific computation contexts.

演讲人简历

Dimitri P. Bertsekas received his undergraduate degree in engineering from the National Technical University of Athens, Greece, and his Ph.D. from the Massachusetts Institute of Technology.

Dr. Bertsekas has held faculty positions with the Engineering-Economic Systems Dept., Stanford University (1971-1974) and the Electrical Engineering Dept. of the University of Illinois, Urbana  (1974-1979). Since 1979 he has been teaching at the Electrical Engineering and Computer Science Department of the Massachusetts Institute of Technology (M.I.T.), where he is currently McAfee Professor of Engineering. He consults regularly with private industry and has held editorial positions in several journals. His research at  M.I.T. spans several fields, including optimization, control, large-scale computation, and data communication networks, and is closely  tied to his teaching and book authoring activities. He has written numerous research papers, and fourteen books, several of which are used as textbooks in MIT classes.

Professor Bertsekas was awarded the INFORMS 1997 Prize for Research  Excellence in the Interface Between Operations Research and Computer  Science for his book "Neuro-Dynamic Programming" (co-authored with John Tsitsiklis), the 2000 Greek National Award for Operations  Research, and the 2001 ACC John R. Ragazzini Education Award. In  2001, he was elected to the United States National Academy of  Engineering.

Dr. Bertsekas' recent books are "Dynamic Programming and Optimal Control: 3rd Edition" (2007), "Introduction to Probability: 2nd Edition" (2008), and Convex Optimization Theory (2009), all published by Athena Scientific.

Besides his professional activities, Professor Bertsekas is interested in travel, portrait, and landscape photography. His pictures have been exhibited on several occasions at M.I.T., and can also be accessed from his www site.