Academic Research

Lirio is a science-focused company, using evidence-based approaches to creating solutions to nudge individuals to take action for better health. Our team of PhDs collaborate through the Behavioral Reinforcement Learning Lab (BReLL). Their expertise: behavioral science, machine learning, computer science, mathematics, and more.

Lirio’s Behavioral Reinforcement Learning Lab (BReLL) aims to re-shape the current understanding of how behavior change and AI come together to address some of the most stubbornly persistent challenges in healthcare: patient engagement, care gaps, and improved health outcomes.

Selected, published research of Lirio team members can be found below. If you have questions for our researchers, please email .

Featured Research

How Are Machine Learning and Artificial Intelligence Used in Digital Behavior Change Interventions? A Scoping Review

Bucher, A., Blazek, E. S., & Symons, C. (2024). Mayo Clinic Proceedings: Digital Health.
arXiv

A review of 3,000 peer-reviewed studies identified just 32 that showcase the use of artificial intelligence in digital behavior change interventions. The 23 AI-DBCIs identified use a variety of machine learning approaches including conversational AI, reinforcement learning, and classical ML algorithms. The paper identifies opportunities to advance research on the use of AI to change real world behaviors including generating evidence about which techniques may be most suitable for specific behavioral domains.

https://www.mcpdigitalhealth.org/article/S2949-7612(24)00042-7/fulltext

Increasing Entropy to Boost Policy Gradient Performance on Personalization Tasks

Starnes, A., Dereventsov, A., & Webster, C. (2023)
2023 IEEE International Conference on Data Mining Workshops

Often reinforcement learning agents select only a few actions even though there are many options available and some of them are better choices. We examine methods that encourage an agent to make more diverse choices while maintaining its performance.

doi.org/10.48550/arXiv.2310.05324

Gaussian Smoothing Gradient Descent for Minimizing High-Dimensional Non-Convex Functions

Starnes, A., Dereventsov, A., & Webster, C. (2023)
arXiv

Fluctuations in a function often cause algorithms to become trapped and unable to find the minimum. One way to reduce these fluctuations is to smooth them out, which increases the likelihood that the minimum can be found. We numerically show that smoothing can outperform traditional optimization techniques and mathematically show how smoothing impacts the speed at which the minimum can be found.

https://arxiv.org/abs/2311.00521

Gaussian Smoothing Stochastic Gradient Descent (GSmoothSGD)

Starnes, A. & Webster, C. (2023)
arXiv

We continue where our previous smoothing work left off and apply it to the machine learning setting. Mathematically, we show that smoothing can find the optimal model at least as fast as traditional methods. We also propose applying smoothing to a well known optimization method (SVRG) in order to combine the benefits of these two methods.

Responses to a COVID-19 Vaccination Intervention: Qualitative Analysis of 17k Unsolicited SMS Replies

Blazek, E.S., West, A.B., & Bucher, A. (2023)
Health Psychology

Understanding why some people choose to not get vaccinated against COVID-19 continues to be important. In this study of more than 17,000 text messages sent in reply to Lirio’s COVID-19 vaccination Precision Nudging intervention, Lirio’s behavioral scientists coded 22 types of reply. Insights from these unsolicited replies enhance the ability to identify appropriate intervention techniques that can promote COVID-19 vaccination.

https://doi.org/10.1037/hea0001297

The Patient Experience of the Future is Personalized: Using Technology to Scale an N of 1 Approach

Bucher, A. (2023)
Journal of Patient Experience

Personalized experiences are more effective at creating sustained behavior change. Digitally enabled personalized outreach can improve patient’s experience by providing relevant, meaningful calls to action at a time when labor-intensive human-to-human personalization is challenged by systemic health staffing shortages. Strategic use of digital tools to engage patients and supplement human-to-human care scale personalization to the benefit of patient and provider experience.

doi/10.1177/23743735231167975

Persona-Based Conversational AI: State of the Art and Challenges

Liu, J., Symons, C., & Vatsavai, R. R. (2022)
2022 IEEE International Conference on Data Mining Workshops

Conversational AI has become an increasingly prominent and practical application of machine learning. However, existing conversational AI techniques suffer from various limitations such as a lack of well-developed methods for incorporating auxiliary information that could help a model understand conversational context better. We explore how persona-based information could help improve the quality of response generation in conversations.

doi: 10.1109/ICDMW58026.2022.00129

Additional Research
Barriers to COVID-19 Vaccination in a Troop of Fleet Antiterrorism Security Team Marines: Observational Study

Blazek, E.S., & Bucher, A. (2024)
JMIR Formative Research

To effectively drive vaccination behavior within a population subgroup, we need to understand the behavioral determinants prominent for that group. We conducted this observational study to understand why Fleet Antiterrorism Security Marines said they would decline the COVID-19 vaccine when it was first made available to military personnel in 2021.

https://formative.jmir.org/2024/1/e50181

On the Unreasonable Efficiency of State Space Clustering in Personalization Tasks

A. Dereventsov, R. R. Vatsavai, & C. G. Webster (2022)

2021 IEEE International Conference on Data Mining Workshops

In this effort we consider a reinforcement learning (RL) technique for solving personalization tasks with complex reward signals. Numerical examples demonstrate the efficiency of different RL procedures and are used to illustrate that this technique accelerates the agent’s ability to learn and does not restrict the agent’s performance.

doi: 10.1109/ICDMW53433.2021.00097

An Adaptive Stochastic Gradient-free Approach for High-dimensional Blackbox Optimization

Dereventsov, A., Webster, C. G., & Daws, J. (2020)

Proceedings of the International Conference on Computational Intelligence

We propose a novel adaptive stochastic gradient-free (ASGF) approach for solving high-dimensional nonconvex optimization problems based on function evaluations. We employ a directional Gaussian smoothing of the target function that generates a surrogate of the gradient and assists in avoiding bad local optima by utilizing nonlocal information of the loss landscape. Applying a deterministic quadrature scheme results in a massively scalable technique that is sample-efficient and achieves spectral accuracy.

arXiv:2006.10887

Leveraging Behavioral Science and Artificial Intelligence to Support Mental Health in the Workplace: A Pilot Study

West, A. B., Guo, Y. V., & Bucher, A. (2023)
Frontiers in Psychology

In a pilot study using Precision Nudging compared to a control to encourage employee engagement with the employee assistance program (EAP), results suggest that using behavioral science and artificial intelligence can improve employee usage of EAP, specifically with the intention of exploring mental health and stress management resources, compared to benchmark rates of 5% per year.

https://doi.org/10.3389/fpsyt.2023.1219229

Modeling Non-deterministic Human Behaviors in Discrete Food Choices

A. Starnes, A. Dereventsov, E. S. Blazek, & F. Phillips (2022)

2022 IEEE International Conference on Data Mining Workshops

We establish a non-deterministic model that predicts a user’s food preferences from their demographic information. Our simulator is based on NHANES dataset and domain expert knowledge in the form of established behavioral studies.

doi: 10.1109/ICDMW58026.2022.00131

Offline Policy Comparison Under Limited Historical Agent-environment Interactions

Dereventsov, A., Daws, J., & Webster, C. G. (2022)

We address the challenge of policy evaluation in real-world applications of reinforcement  learning systems where the available historical data is limited due to ethical, practical, or security considerations.

arXiv:2106.03934

Examining Policy Entropy of Reinforcement Learning Agents for Personalization Tasks

Dereventsov, A., Starnes, A., & Webster, C. G. (2022)

We demonstrate that Policy Optimization agents often possess low-entropy policies during training, which in practice results in agents prioritizing certain actions and avoiding others. Conversely, we also show that Q-Learning agents are far less susceptible to such behavior and generally maintain high-entropy policies throughout training, which is often preferable in real-world applications.

arXiv:2211.11869v2

Feasibility of a Reinforcement Learning–Enabled Digital Health Intervention to Promote Mammograms: Retrospective, Single-Arm, Observational Study

Bucher, A., Blazek, E.S., & West, A.B. (2022)

In a retrospective single-arm observational study, Lirio’s Precision Nudging for Mammography reached almost 140,000 women significantly overdue for recommended breast cancer screening. Data suggested that Precision Nudging has equitable outreach, with proportional responses across age groups, race, household income levels, and educational attainment.

doi: 10.2196/42343

If We Build It Right, They Will Come: Driving Health Outcomes With Precision Nudging

Delaney, S., & Bucher, A. (2022)

The Behavioral Economics Guide 2022

Lirio matches behavioral techniques to best support the desired target behaviors. In this paper, learn how Lirio’s designers leverage behavioral economics “nudges” as an engagement strategy that allows them to deliver in-depth behavior change techniques within the intervention.

Simulated Contextual Bandits for Personalization Tasks from Recommendation Datasets

Dereventsov, A., & Bibin, A. (2022)

2022 IEEE International Conference on Data Mining Workshops

We propose a method for generating simulated contextual bandit environments for personalization tasks from recommendation datasets like MovieLens, Netflix, Last.fm, Million Song, etc. The obtained environments can be used to develop methods for solving personalization tasks, algorithm benchmarking, model simulation, and more.

doi: 10.1109/ICDMW58026.2022.00127

An Economic Impact Model for Estimating the Value to Health Systems of a Digital Intervention for Diabetes Primary Care: Development and Usefulness Study

Powers, B. & Bucher, A. (2022)

What is the value of a primary care visit for a patient with diabetes to a health system? In this paper, Lirio shares a model leveraging publicly available national data that allows health systems to calculate an economic impact for primary care in patients with Type 1 and Type 2 diabetes by adding the payer mix for their population.

https://doi.org/10.2196/37745

Personalized Digital Health Communications to Increase COVID-19 Vaccination in Underserved Populations: A Double Diamond Approach to Behavioral Design

Ford, K. L., West, A. B., Bucher, A., & Osborn, C. Y. (2022)

In this paper, learn about how Lirio’s behavioral designers used the Double Diamond model to intentionally incorporate research from diverse populations to deliver health equity for COVID-19 vaccination.

https://doi.org/10.3389/fdgth.2022.831093