This Master’s thesis addresses one of the hardest challenges in time series analysis - controlling or forecasting systems with complex dynamics such as chaos or limit cycles. The core of the thesis will consist of applying a mathematical framework that we developed for ecological time series analysis to a cart-pole control problem.

Motivation

Control systems are ubiquitous in everyday life. For many applications simple linear controllers like PID are sufficient, however there emerge increasingly many tasks requiring more sophisticated control strategies. One of the algorithms addressing this challenge is model predictive control (MPC). It can find complex control scenarios but requires an accurate system model. The classical system identification is often very time-consuming and requires expert knowledge of the system. Machine learning tools can facilitate this process significantly.
However, even with machine learning tools the dynamics of the system might be difficult to forecast. For instance, chaotic systems such as those seen in ecological populations, meteorology or robotics are notably difficult to model and predict.

Background and previous works

Control problem: In this thesis we want to focus on performing swing-up with a cart-pole robot (see the animation). With its unstable equilibrium and double target (pole and cart position) it is a well-studied, but still challenging system and serves as a common benchmark for control algorithms. In our lab we have both cartpole simulator and physical cartpole robot. We have also successfully implemented MPC based on accurate ODEs describing cartpole dynamics. The control quality is however far from perfect and gets even worse while using neural network as a system model. Its improvement with new methods will be a visible outcome of your thesis. There exists also an opportunity to test the same methods on our car simulator and a wheeled robot.

Mathematical framework: In collaboration with the ecology department, we developed a theory in which the dynamics of chaotic models are used to evaluate when the training is possible in such systems. We found that training models for short term predictions was feasible but suffered from low accuracy, while training on long-term predictions provides accurate parameter estimates but requires a very good initial guess. In current works we are trying to use short term predictions to give accurate guesses and long-term predictions to fine-tune the parameters.

Approach
Our goal is to combine the two frameworks. The cart pole system’s parameters have to be inferred so that the controller can make predictions and decide the best control strategy. A good model would allow the controller to make accurate predictions within a long-term horizon and thus give better control signals. Your job is therefore to use the theory developed for time series models and apply it to the control framework.

Tasks

• Familiarize yourself with the basic theory, which includes:
◦ Chaos and limit cycles: from dynamical systems theory
◦ Loss functions: from machine learning
• Familiarize yourself with the cart-pole set-up
◦ Run simulations of the environment
◦ Train the model with the current methods
• Train the cart-pole model
◦ Use one timestep predictions to define a loss function and train on it
◦ Iteratively expand to multiple timesteps
◦ Measure the evolution of the training loss
• Evaluate the training perfomance.
◦ Test the model on a cart-pole set-up: evaluate the control
◦ Compute the loss function on the test data
◦ Compute and visualize the loss function for other parameters
• If there is time, we can extend this method:
◦ Using deep neural networks for the controller
◦ To other problems in robotics or control

Your profile

• Interest in machine learning and robotics.
• Programming experience.
• Willingness to learn concepts from mathematics
• Hands-on mentality

Remarks

Given that this is a proof-of-concept project, the roadmap is open to changes. If you want to develop your own ideas or emphasize some of the aspects we will be glad to alter the project outline.

You do not need to be a hardcore mathematician just the willingness to understand the intuition behind some mathematical quantities.

Supervision

The project will take place between the Grewe Lab and the Sensors Research Group. The supervisors will be Dr. Pau Vilimelis Aceituno and Marcin Paluch.