• Prof Sylvie Thiebaux (computer science)

Planning is one main areas of AI. The most basic problem in planning consists in generating a course of action -- a plan, or a policy -- enabling an agent to achieve given goals in its environment, starting from a known initial state. Planners are general purpose solvers that take as input the description of the problem to solve (this describes the actions available to the agent and their effects on the environment), and return a plan (or policy) achieving the problem's goal. Planners typically use search or constrained optimisation to solve the problem.

Deep learning has become the method of choice for perception tasks in computer vision or natural language processing. However, whether and how deep learning can learn to solve reasoning tasks such as planning, is very much an open question. Deep learning has mainly been used in reinforcement learning, which enables an agent to learn how to act in its environment by interacting with it and receiving rewards for good behaviour. Unfortunately this requires very large amount of data and CPU, is unsafe, doesn't easily capture and exploit existing knowledge of the problem, and is overall difficult to apply to real world problems other than computer games. Instead, new approaches are needed that combine reasoning and learning.

We have started to tackle this challenge with two previous ANU honours students, now doing their respective PhDs at Berkeley and MIT. We have devised new neural network architectures which exploit the structure of planning representations to learn policies and planning heuristics (cost estimators to guide the search of a planner) from examples of plans. Exploiting the structure of the planning domain reduces significantly the amount of training needed compared to reinforcement learning approaches. Another interesting aspect of the learnt policies is that they generalise to all problems within the same planning domain, including problems of different size. That way, one can learn policies or heuristics from a few examples plans of small problems and apply the result to larger problems from the same planning domain. The results on a range of planning benchmark problems are encouraging.

This has opened many avenues of research, on how to improve the efficiency and power of these architectures, and on verifying and explaining their results. This leads to the research topics below.