Welcome to Geek’s Corner! Lirio believes strongly in advancing the science of behavior change AI, and that includes sharing a collection of Lirio blog posts that are a bit more technical in nature. Want to know more? Contact Lirio’s Chief Evangelist, Patrick Hunt, who can put you in contact with one of our subject matter experts.
Behavior change can be appropriately cast as a real-world reinforcement learning problem. The challenges involved in solving this problem effectively in the healthcare setting go well beyond the fact that existing tools are inadequate. Even the latest machine learning research has not addressed the full problem or incorporated many of the critical confounding factors that exist in this real-world setting. The AI team at Lirio focuses on advancing the state of the art in real-world reinforcement learning to cope with some of these challenges. For example, we developed the Adaptive Stochastic Gradient-Free (ASGF) optimization strategy to address many of the challenges present in solving real-world reinforcement learning (RL) optimization tasks.
This blog post discusses another advancement, Limited Data Estimator, which compares different approaches to applying reinforcement learning in challenging data environments.
A New Approach
Lirio’s Behavioral Reinforcement Learning Lab (BReLL) recently published a paper describing a new approach for comparing reinforcement learning policies using limited historical data. The paper explains why existing methods for comparing policies are not effective when the amount of data is extremely sparse, as is typically the case in applications of our behavior change AI platform.
The paper presents a new measure called the Limited Data Estimator (LDE) which shows “that with high confidence the LDE correctly compares target policies even when only a small number of agent-environment interactions are available,” and demonstrates its effectiveness compared to existing policy comparison methods on several learning tasks.
Why Does This Matter?
Healthcare outcomes are inordinately affected by people’s often irrational behaviors. Lirio works to promote healthy, medically recommended behaviors, but executing behavior change AI is difficult for several reasons. We need to advance the state of reinforcement learning so we can apply it more effectively in real-world scenarios, including healthcare settings that encourage patients to adopt healthy behaviors.
Dramatic Improvement Needed Offline
Unlike image and object recognition, video games, robotics, and other types of machine learning, there are still opportunities for dramatic improvement in representation and optimization in newer subfields like real-world behavioral reinforcement learning. In addition, human health behaviors are not very well understood, so much so that any kind of simulated data cannot guarantee the accuracy required for training a model.
Navigating Balancing in Healthcare
When a system interacts with people in a health-related setting, several unique challenges arise. Practical considerations can limit the number of interactions, and ethical considerations can limit exploration. When it comes to the latter issue, consider that a common challenge related to the exploration-exploitation trade-off in reinforcement learning is the issue of balancing.
Balancing refers to providing the best-known intervention guiding health outcomes compared to the risk of providing sub-optimal interventions while continuing to explore more optimal interventions that the system simply hasn’t discovered yet. The ethical questions surrounding such choices have not played a role in earlier reinforcement learning applications, yet they are core issues when interacting with patients about their health-related behaviors.
An Evolving Opportunity
The Behavioral Reinforcement Learning Lab (BReLL) at Lirio is committed to using AI to scale behavioral interventions to positively impact people’s lives. We will continue to innovate not just on our unique and proprietary solutions, but on methods that advance the efficacy and scope of real-world applications of machine learning.
Learn More from Lirio
To discover more about Lirio’s approach to behavior change AI and real-world lessons learned, join us for our next webinar: “The AI You Don’t Have: Scaling Personalized Behavioral Interventions,” Thursday, September 9 at 1:00 p.m. ET.
Following the interest and engagement from the in-booth session at HIMSS21, this webinar will allow for more exploration and discussion on Lirio’s behavior change AI platform and our novel applications in improving equity and access to care.
Follow Lirio on Facebook: Facebook.com/lirio.llc, LinkedIn: LinkedIn.com/company/lirio, and Twitter: @Lirio_LLC.
Other readers viewed:
What we learned about the Future of Healthcare at HIMSS21
Lirio Launches Hyper-Personalized Vaccination Journey to Encourage Vaccine Adoption
Lirio Research: Introducing a New Form of Machine Learning Optimization