Adaptive Computation

Prof. Dr. Luc De Raedt

Co-organizers : Dr. Stefan Kramer, Dipl.-Inf. Kristian Kersting

Thursday 11-13 o'clock, Room: SR 02-017, Bulding 052

Adaptive computation is concerned with procedures that adapt themselves over time with experience. As such adaptive computation is strongly related to the field of Machine Learning. In this course, we will study a few topics in adaptive computation.

In the first part, we will give an introduction to Reinforcement Learning. Reinforcement learning addresses the problem of learning control strategies for autonomous agents by trial and error. It is of special interest, because it may serve as a simple, but general model of intelligent behavior. The basis of this part of the course will be the textbook by Sutton and Barto (see below). One famous application of reinforcement learning is G. Tesauro's TD-gammon program that plays backgammon at world-champion level and that was developing its strategies using techniques of adaptive computation.

In the second part, we will talk about Bayesian networks. Bayesian networks are directed acyclic graphs defining the dependencies (and above all, the "independencies") among random variables. A Bayesian network thus describes the probability distribution governing this set. Bayesian networks are an extremely popular knowledge representation framework. They combine logical with probabilistic aspects and they have been used in a wide variety of expert systems, for prediction, for diagnosis, etc. Many applications in e.g. medicine, robotics, and biology exist. Stuart Russell once stated that Bayesian networks are "the best thing since sliced bread".

The third part of the course will possibly deal with learning and adaptation in richer, more expressive representation languages, such as first-order logic.The sub-field of Machine Learning concerned with this topic is called Inductive Logic Programming (ILP). ILP is the one area of Machine Learning that explicitly addresses the role of prior knowledge in learning. For instance, it is possible to specify background knowledge that may be used to explain observations. In this part, we will use our own slides for the lectures.

The fourth part of the course will possibly deal with genetic algorithms, which are inspired on Darwin's theory of evolution. Genetic algorithms exploit Darwin's principle as a search procedure or to optimize certain functions or behaviour.

The following books will be used in the course:

R. S. Sutton, A. G. Barto."Reinforcement Learning: An Introduction". The MIT Press, Cambridge, MA, 1998.

and possibly

F. V. Jensen."An Introduction to Bayesian Networks". Springer Verlag, Berlin, Heidelberg, New York, 1996.

No prior knowledge on Artificial Intelligence or Machine Learning is required to follow this course. The course will be partly taught in English, but students can ask questions in German.

• Exercise sheet 1 (20.12.2001)

• Bayesian Networks:
  • Introduction & Modelling [intro_modelling_BN.ps.gz]
  • Inference (Bucket Elimination, Junction Tree) [inference_BN.ps.gz]
  • Inference (Junction Tree) & Hidden Markov Models [inference_HMM_BN.ps.gz]
  • Learning (Parameter Estimation) [learning_BN.ps.gz]
• Genetic Algorithms:
  • Introduction [gas1.ppt.gz]
  • GAs in Problem Solving [gas2.ppt.gz]
  • GAs in Scientific Models [gas3.ppt.gz]
  • Theory of GAs [gas4.ppt.gz]