[UAI] NEW BOOK on relational Markov decision processes, reinforcement learning, and decision-theoretic planning
Dear colleagues, Learning and acting in large, probabilistic, relational worlds is investigated in research fields such as machine learning, intelligent agents, knowledge representation and optimization. This research has been ongoing for many years, and below follows an *announcement* for the first *book* that describes and surveys these developments in a unified manner. It starts with 'learning sequential decision making problems under uncertainty' and surveys important developments related to knowledge representation, generalization and abstraction. The core of the book is a detailed and complete study of 'relational representations' in this field. In addition to introducing several new methodological, technical and algorithmic advances, the book contains complete surveys of relational reinforcement learning, first-order decision-theoretic planning, and matters related to world models, hierarchies and knowledge transfer. The book provides a complete and self-contained reference work, and is aimed at (PhD) students and researchers working on matters related to learning and acting in large, probabilistic, (relational) worlds. Best regards, Martijn van Otterlo -- Book Announcement -- THE LOGIC OF ADAPTIVE BEHAVIOR: Knowledge Representation and Algorithms for Adaptive Sequential Decision Making under Uncertainty in First-Order and Relational Domains by Martijn van Otterlo 2009 -- IOS Press, Amsterdam, Berlin, Oxford, Tokyo, Washington D.C. ISBN 978-1-58603-969-1 Hardcover, 500+pp, 800++refs (also available in electronic version) --- See for more information: http://www.iospress.nl/html/9781586039691.php --- Preface and TOC: (Online) http://www.booksonline.iospress.nl/Content/View.aspx?piid=11738 Contents: Chapter 1: Introduction / Chapter 2: Markov decision processes: concepts and algorithms / Chapter 3: Generalization and abstraction in MDPs / Chapter 4: Reasoning, learning and acting in first-order worlds / Chapter 5: Model-free algorithms for relational MDPs / Chapter 6: Model-based algorithms for relational MDPs / Chapter 7: Sapience, models and hierarchy / Chapter 8: Conclusions and future directions - Back Cover text: - Learning and reasoning in large, structured, probabilistic worlds is at the heart of artificial intelligence. Markov decision processes have become the de facto standard in modeling and solving sequential decision making problems under uncertainty. Many efficient reinforcement learning and dynamic programming techniques exist that can solve such problems. Until recently, the representational state-of-the-art in this field was based on propositional representations. However, it is hard to imagine a truly general, intelligent system that does not conceive of the world in terms of objects and their properties and relations to other objects. To this end, this book studies lifting Markov decision processes, reinforcement learning and dynamic programming to the first-order (or, relational) setting. Based on an extensive analysis of propositional representations and techniques, a methodological translation is constructed from the propositional to the relational setting. Furthermore, this book provides a thorough and complete description of the state-of-the-art, it surveys vital, related historical developments and it contains extensive descriptions of several new model-free and model-based solution techniques. ---------- Dr. Ir. Martijn van Otterlo Department of Computer Science Katholieke Universiteit Leuven Celestijnenlaan 200A B3001 Heverlee, Belgium. TeLePhOnE ... +32 +16 32 7741 WeB ... http://www.cs.kuleuven.be/~martijn eMaIl ... Martijn.vanOtterlo at cs.kuleuven.be -- Disclaimer: http://www.kuleuven.be/cwis/email_disclaimer.htm ___ uai mailing list uai@ENGR.ORST.EDU https://secure.engr.oregonstate.edu/mailman/listinfo/uai
[UAI] Relational RL Survey
Dear reader, A comprehensive survey of relational reinforcement learning is available from my webpage: "A Survey of Reinforcement Learning in Relational Domains". M. van Otterlo -- TR-CTIT-05-31 - (70pp) CTIT Technical Report Series ISSN 1381-3625 Abstract. Reinforcement learning has developed into a primary approach for learning control strategies for autonomous agents. However, most of the work has focused on the algorithmic aspect, i.e. various ways of computing value functions and policies. Usually the representational aspects were limited to the use of attribute-value or propositional languages to describe states, actions etc. A recent direction -- under the general name of relational reinforcement learning -- is concerned with upgrading the representation of reinforcement learning methods to the first-order case, being able to speak, reason and learn about objects and relations between objects. This survey aims at presenting an introduction to this new field, starting from the classical reinforcement learning framework. We will describe the main motivations and challenges, and give a comprehensive survey of methods that have been proposed in the literature. The aim is to give a complete survey of the available literature, of the underlying motivations and of the implications of the new methods for learning in large, relational and probabilistic environments. Work is underway to provide an updated version soon. Any comments, suggestions, and pointers to (new) work that does not yet appear in this survey will be greatly appreciated. Regards, Martijn van Otterlo. http://www.cs.utwente.nl/~otterlo/ ___ uai mailing list uai@ENGR.ORST.EDU https://secure.engr.oregonstate.edu/mailman/listinfo/uai
[UAI] Workshop on Rich Representations for Reinforcement Learning
(Apologies for multiple postings) Second Call for Papers: Workshop on Rich Representations for Reinforcement Learning Date: August 7th, 2005 in conjunction with ICML'05, Bonn, Germany Web Site: http://www.cs.waikato.ac.nz/~kurtd/rrfrl/ Featuring a.o. Invited Talks of * Roni Khardon * Satinder Singh * ... Overview: Reinforcement learning (RL) has developed into a primary approach to learning control strategies for autonomous agents. The majority of RL work has focused on propositional or attribute-value representations of states and actions, simple temporal models of action, and memoryless policy representations. Many problem domains, however, are not easily represented under these assumptions. This has led to recent work that studies the use of richer representations in RL to overcome some of these traditional limitations. This includes for example: relational reinforcement learning, where states, actions and learned policies have relational representations; richer temporal representations of action, such as options; richer policy representations that incorporate internal state, such as MAXQ hierarchies; and the recently introduced predictive state representations where the state of a system is represented in terms of the predictions of future observations. The main topic of the workshop will be the application of these (and possibly other) rich representational formats, the relationships among them, and their benefits (or drawbacks) for reinforcement learning. There have been a number of previous workshops that focus on individual representational items noted above. The goal of this workshop is mainly to promote interaction between researchers in the various representational aspects of RL. There is a high diversity of rich representations and possible approaches, many of which may mutually benefit one another. This workshop will give researchers the chance to consider such benefits and highlight some of the key challenges that remain. Given the co-location of ICML with ILP this year, we expect attendees from both conferences to participate in the workshop as the topic intersects with interests of both, in particular the incorporation of relational and logical representations into RL. Some example topics/issues that could be addressed include: * New algorithms for exploiting rich representations to the fullest. When is it possible to design algorithms for rich representations by reduction to traditional techniques? * When and how does reinforcement learning benefit from rich representations? Specific real-world successes and failures are of particular interest. * What is the influence of rich representations on the (re-)usability of reinforcement learning results, or transfer learning (for example through goal parameterization)? * Should the introduction of rich representations in reinforcement learning be accompanied by different learning goals (such as policy-optimality) to keep the learning problems feasible? * How should we evaluate new algorithms for rich representations? Specific benchmarks that exhibit the weaknesses and benefits of various representational features are of particular interest. * How can RL benefit from/contribute to existing models and techniques used for (decision-theoretic) planning and agents that already use richer representations, but lack learning? Submissions Format: Potential speakers should submit a paper of a maximum of 6 pages in the ICML paper format. We encourage smaller contributions or summaries of on-going work, one page abstracts, and position papers on the topics relevant to the workshop. To supply the panel planned at the end of the workshop with discussion topics, we ask each potential presenter and participant to propose, in advance, a provocative question or claim, with the emphasis on provocative. We will use the resulting pool of questions, possibly anonymously, to stimulate discussion as needed. The papers and provocative questions or claims should be sent by email to [EMAIL PROTECTED] We will assume that your questions can be attributed to you unless you request anonymity. Important Dates: April 1 Paper submission deadline April 22 Notification of acceptance May 13 Final paper deadline August 7 Workshop date Organizing Committee: Kurt Driessens: University of Waikato, Hamilton, New Zealand Alan Fern: Oregon State University, Corvallis, U.S.A. Martijn van Otterlo: University of Twente, The Netherlands Program Committee: Robert Givan: Purdue University, U.S.A. Roni Khardon: Tufts University, U.S.A. Ron Parr: Duke University, U.S.A. Sridhar Mahadevan:University of Massachusetts, U.S.A. Satinder Singh: University of Michigan, U.S.A. Prasad Tadepalli: Oregon State University, U.S.A. -- --- Martijn van Otterlo Tel:(31)(53) 489 4111 TKI, Dept. of Computer Science, Fax:
[UAI] Benelearn 2005 -- Call for Participation
+ Please distribute as widely as possible. Apologies for crossposting. +++ Call for Participation Benelearn 2005 Annual Machine Learning Conference of Belgium and the Netherlands http://hmi.ewi.utwente.nl/conference/benelearn2005 Februari 17 - 18, 2005 University of Twente, Enschede, The Netherlands Early Registration deadline: Februari 1st With support from NWO, SIKS, CTIT and Senter-IOP +++ Benelearn is the annual machine learning conference of Belgium and the Netherlands. It serves as a forum to exchange ideas and present recent work. Technical Program includes: 2 invited talks: - Probabilistic Logic Learning and Reasoning by K. Kersting (Machine Learning Lab, University of Freiburg, Germany) - Challenges in Multimodal Processing by S. Bengio (IDIAP, Martigny, Switserland) List of accepted papers: Amplifying the Block Matrix Structure for Spectral Clustering Igor Fischer and Jan Poland Saarland University, Germany IDSIA, Manno (Lugano), Switzerland Evolving Neural Networks for Forest Fire Control Marco Wiering, Filippo Mignogna and Bernard Maassen Utrecht University, The Netherlands Experiments with Relational Neural Networks Werner Uwents and Hendrik Blockeel Catholic University of Leuven, Belgium Assessment of SVM Reliability for Microarrays Data Analysis Andrea Malossini, Enrico Blanzieri and Raymond T. Ng University of Trento, Italy University of British Columbia, Canada Speaker Prediction based on Head Orientations Rutger Rienks, Ronald Poppe and Mannes Poel University of Twente, The Netherlands Strong Asymptotic Assertions for Discrete MDL in Regression and Classification Jan Poland and Marcus Hutter IDSIA, Manno (Lugano), Switzerland Detecting Deviation in Multinomially Distributed Data Jan Peter Patist Vrije Universiteit Amsterdam, The Netherlands Monotone Constraints in Frequent Tree Mining Jeroen De Knijf and Ad Feelders Utrecht University, The Netherlands Best-Response Play in Partially Observable Card Games Frans Oliehoek, Matthijs Spaan and Nikos Vlassis University of Amsterdam, The Netherlands Master Algorithms for Active Experts Problems based on Increasing Loss Values Jan Poland and Marcus Hutter IDSIA, Manno (Lugano), Switzerland Maximizing Expected Utility in Coevolutionary Search Edwin de Jong Utrecht University, The Netherlands Reliability yields Information Gain Ida Sprinkhuizen-Kuyper, Evgueni Smirnov and Georgi Nalbantov University of Maastricht, The Netherlands Erasmus University Rotterdam, The Netherlands Facial Expression Analysis using Multi-layer Perceptrons Michal Sindlar and Marco Wiering Utrecht University, The Netherlands Reinforcement Learning using Optimistic Process Filtered Models Funlade Sunmola and Jeremy Wyatt University of Birmingham, United Kingdom The registration form and further information can be found at the conference website: http://hmi.ewi.utwente.nl/conference/benelearn2005 The Organizing Comittee: Martijn van Otterlo (University of Twente, the Netherlands) Mannes Poel (University of Twente, the Netherlands) Anton Nijholt (University of Twente, the Netherlands) Sponsors: Netherlands Organisation for Scientific Research (NWO) http://www.nwo.nl Dutch research school for Information and Knowledge Systems (SIKS) http://www.siks.nl/ Centre for Telematics and Information Technology (CTIT) SRO NICE. http://www.ctit.utwente.nl/research/sro/nice/index.html Senter-IOP http://www.senter.nl/asp/page.asp?alias=iop ___ uai mailing list uai@ENGR.ORST.EDU https://secure.engr.oregonstate.edu/mailman/listinfo/uai
