T1D Resources

Diabetes-Related Reinforcement Learning (DRRL)

Reinforcement learning for insulin dosing and glucose control: simulators, starter code, and prior work to get a project off the ground.

Overview

T1D is a chronic disease in which the body can no longer produce insulin, the hormone that regulates blood glucose. People with T1D must supply insulin themselves, either through injections or an insulin pump. Many pumps run a control algorithm that delivers insulin automatically, but mostly only the small, steady amounts needed throughout the day, known as basal insulin. Larger doses given for meals or to correct high blood glucose, known as boluses, are still largely left to the patient.

This leaves the most consequential part of managing the condition to the person with T1D, and mistakes have real costs: blood glucose that runs too high (hyperglycemia) or too low (hypoglycemia). My main interest is offline reinforcement learning for insulin control, which learns dosing policies from logged data rather than trial and error on real patients.

Where DRRL fits

It can be hard to place this work within the broader landscape of CGM- and insulin-based models and decision making. On one side is the growing body of work that uses CGM data for general downstream tasks, such as the foundation model GlucoFM. On the other are insulin-dosing RL problems outside diabetes, such as insulin delivery in the ICU. I think diabetes-related reinforcement learning (DRRL) should be treated as a class of its own, and it is the primary focus of my research. Work outside DRRL is still worth following, since many of its ideas carry over to these goals.

Code & getting started

  • simglucose Python

    Open-source Python implementation of the FDA-accepted UVA/Padova 2008 T1D simulator, with a Gym-style interface for RL.

Datasets

For public continuous glucose monitoring (CGM) data, start with Glucose-ML, which collects public CGM datasets for research.

Related work

  1. Offline reinforcement learning for safer blood glucose control in people with type 1 diabetes Harry Emerson, Matthew Guy, Ryan McConville. Journal of Biomedical Informatics, 2023.
  2. Deep Reinforcement Learning for Closed-Loop Blood Glucose Control Ian Fox, Joyce Lee, Rodica Pop-Busui, Jenna Wiens. Machine Learning for Healthcare (MLHC), 2020.