Today at the 2018 Machine Learning for Healthcare conference at Stanford University, Shah and researcher Gregory Yauney will present a self-learning artificial intelligence model that undertakes that balancing act. Trained with real patient data, their AI model predicts individual dosing regimens that shrink tumors while minimizing side effects for a deadly form of brain cancer.

Currently doctors typically choose how to dose a patient using protocols based on animal studies and past clinical trials, some from the 1950s or earlier. Shah figured there was room for improvement. He and Yauney decided to focus on glioblastoma, a type of brain cancer, because it is treated aggressively with large doses chemotherapy and radiation therapy to shrink the tumors as quickly as possible. That also means the treatment can make patients very, very sick.

“Our goal was to reduce toxicity and dosing for patients who unfortunately have this disease,” says Shah.

It wasn’t an easy goal to achieve. First, they approached it as a classification problem that needed a supervised neural network. That didn’t work. “The search data is too large, and even humans don’t know the right answer,” says Shah. Next, they thought maybe it was an automation problem: the system just needed to be told what to do. It wasn’t. That system simply threw drugs at the problem, suggesting more doses than seemed appropriate.

Ultimately, they turned to reinforcement learning (RL), a technique in which an AI agent learns through trial and error to favor a certain behavior in order to maximize a reward. That’s how Google’s DeepMind beats so many games, for example.

But in this case, the researchers tweaked the RL model, creating what they call an “unorthodox approach.” Instead of pointing the AI agent toward a single goal, like winning a game of Go or parking a car, they programmed it to strike a balance between shrinking a tumor as much as possible and reducing the size and number of doses by action-derived rewards.

The model was trained with clinical trial data from 50 glioblastoma patients. The AI conducted about 20,000 trial-and-error test runs on simulated versions of those patients. Whenever the model initiated a dose, it would be rewarded if that dose shrunk the overall tumor size. Yet if the model chose to administer the full set of doses, it was penalized, encouraging the model to choose fewer, smaller doses when possible.

The researchers then tested the model on 50 new simulated patients. Compared to treatment regimens currently used by doctors, the regimens designed by the self-taught AI achieved significant tumor reduction while lowering the frequency and size of drug doses.

The technique has yet to be tested in real patients. The researchers are currently in discussions with regulatory agencies and academic hospitals about selecting clinical trial sites to try out the AI. Shah imagines doctors and patients might use it as a recommendation for possible dosing options, though final decisions for the foreseeable future would remain in the doctor’s hands.