简介: |
In control and optimization, we usually choose actions based on a system’s current state. In control of modern engineering systems, however, we often meet the situation where actions can only be taken after an event occurs. With the Markov model, such an event can be defined as a set of state transitions that share some common features. Apparently, an event contains some information about the future, which is very important for performance optimization because, after all, optimization is achieved by predicting the future. The talk consists of three parts:
1) Illustrate the aforementioned limitation of Markov decision process (MDPs) by a simple example. We define event-based policies and introduce the event-based learning and optimization approach. We show that in same cases the event- based policies may perform better than the state-based policies because the former utilizes the future information captured by the events.
2) We provide a solution to the event-based optimization problem. Gradient-based and policy iteration based approaches are discussed. In many cases, the number of event-based policies may scale to the system size, and computation can be saved by aggregation.
Xi-Ren Cao received the M.S. and Ph.D. degrees from Harvard University, in 1981 and 1984, respectively, where he was a research fellow from 1984 to 1986. He then worked as a principal and consultant engineer/engineering manager at Digital Equipment Corporation, U.S.A, until October 1993. Since then, he is a Professor of the Hong Kong University of Science and Technology (HKUST), Hong Kong, China. He is the director of the Center for Networking at HKUST. He held visiting positions at Harvard University, University of Massachusetts at Amherst, AT&T Labs, University of Maryland at College Park, University of Notre Dame, Tsinghua University, University of Science and Technology of China, and other universities.
Dr. Cao owns three patents in data and tele-communications and published two books: “Realization Probabilities - the Dynamics of Queuing Systems,” Springer Verlag, 1994, and “Perturbation Analysis of Discrete-Event Dynamic Systems,” Kluwer Academic Publishers, 1991 (co-authored with Y. C. Ho). He received the Outstanding Transactions Paper Award from the IEEE Control System Society in 1987 and the Outstanding Publication Award from the Institution of Management Science in 1990. He is a Fellow of IEEE, Chairman of the IEEE Fellow Evaluation Committee of the IEEE Control Systems Society, Editor-in-Chief of Discrete Event Dynamic Systems: Theory and Applications, Associate Editor at Large of IEEE Transactions of Automatic Control, member of the Board of Governors of IEEE Control Systems Society, member of IFAC Technical Board, and chairman of IFAC Coordinating Committee on Systems and Signals. He has been served as associate editor of a number of international journals and chairman of a few technical committees of international professional societies. His current research areas include discrete event dynamic systems, stochastic learning and optimization theory, performance analysis of communication systems, and signal processing.
|
