DeepMind, the AI subsidiary of Google parent Alphabet, has once again successfully applied its algorithms to a difficult scientific problem. In cooperation with the Swiss Plasma Center of the Swiss Federal Institute of Technology in Lausanne (EPFL), the UK-based company has now trained its deep reinforcement learning AI to control ultra-hot plasma in an experimental nuclear fusion reactor.
Fourth state: plasma
However, controlling nuclear fusion in reactors is technically very difficult. The central problem: Atomic nuclei repel each other. Bringing them together in a reactor is only possible at extremely high temperatures, often reaching hundreds of millions of degrees — hotter than it is at the center of the sun. At these temperatures, matter is neither solid nor liquid nor gaseous. Instead, it transitions into a fourth state called plasma. It's a simmering, extremely hot particle soup.
The task now is to hold the plasma together in a reactor long enough to draw energy from it. Inside the sun, the plasma is held together by gravity. On Earth, physicists and chemists have to use a variety of tricks. In a magnet-based reactor, a so-called tokamak, the plasma is trapped in a magnetic cage. Horizontal conductor loops enclose the donut-shaped reactor vessel, in the middle of which there is an elongated solenoid magnet that induces a ring-shaped current in the reactor and thus further heats the plasma. By controlling the current in the coils, the surrounding magnetic fields can be shaped. This prevents the plasma from touching the walls of the reactor vessel, which could cool it down and damage the reactor at the same time.
But controlling the plasma requires constant monitoring and changing of the magnetic field, which previous controllers manage more or less well. The DeepMind team therefore trained its reinforcement learning algorithm precisely on this in a simulation. Training with reinforcement learning should enable a software agent to behave in any situation in such a way that it pursues its goal as stringently as possible – without explicitly writing down all possible actions for all possible situations. This basically works with trial and error: He tries an action, if it is effective, it is more likely to be selected next time. If it is not effective, your probability of selection is reduced. After a sufficient number of attempts, the agent learns an optimal "policy".
(Image:
UKAEA
)
Exercise on the virtual reactor
After the AI had learned to control and adequately change the shape of the plasma using magnetic impulses in the virtual reactor, an experiment began. In a small experimental reactor, the EPFL researchers let the algorithm take control, the Variable Configuration Tokamak (TCV) located at the university. The result: It could be seen that the AI was perfectly capable of controlling the real reactor without additional fine-tuning by humans. However, the attempt was very short due to the system: the AI controlled the plasma for a total of two seconds – that's how long the TCV can run before it overheats.
In order for the control to work, the neural network of the DeepMind AI records 90 different measured values around 10,000 times per second. These show, among other things, the current shape and position of the plasma. The voltage of the 19 magnets of the TCV is then configured accordingly. The resulting feedback loop works much faster than the previously used PID controller. Above all, however, it is much more flexible, because it enables a wide variety of plasma configurations to be generated and controlled with just one control structure.
To speed up the system, the AI was split into two neural networks. In the simulation, a large network first learned how to control the reactor. These capabilities were then transferred to a smaller, faster network that controls the reactor itself (the so-called "critic/actor" model). However, because the simulation used to train the software becomes too imprecise in certain parameter ranges, the control software excludes such "uncertain areas" as a precaution.
AI to better experiment with plasma controls
Nevertheless, the success is impressive. "It's an incredibly powerful technique," says Jonathan Citrin of the Dutch Institute for Fundamental Energy Research, who is familiar with the study. "It's an important first step in a very exciting direction." The research team believes that using AI to control plasma could also make it easier to experiment with different conditions in such reactors. This in turn helps to better understand the process and potentially speed up the development of commercial nuclear fusion reactors.
Research is progressing, but only slowly. Only recently, researchers using the experimental fusion reactor JET showed that they can generate a self-sustaining fusion reaction with deuterium and tritium. However, to create a fusion reaction in a tokamak that puts out more energy than was put into it, a large reactor like ITER with superconducting magnets is needed. However, it will not be ready until 2035 at the earliest. It remains to be seen whether private companies, which are also working on smaller fusion reactors and in some cases are pursuing completely different physical concepts, are actually faster. So far, the race is still completely open.
However, the joy of experimentation could increase. "With these types of control systems, we can take a risk that would otherwise be problematic," says Ambrogio Fasoli, director of the Swiss Plasma Center and chairman of the Eurofusion consortium. Human operators are often unwilling to stress the plasma beyond certain limits. "There are events that we absolutely must avoid because they damage the hardware." However, if you are sure that you have a control system that does not overload the technology despite being pushed to its limits, you could also explore completely new possibilities. "We could speed up the research."
