CLEO reports progress on effort to enhance Guided Pathways with genAI

A recently released progress report titled “Generative AI: Opportunities to enhance CLEO’s Guided Pathways” has shared Community Legal Education Ontario’s insights regarding its efforts to develop a genAI narrative prototype and its ongoing testing, achievements, obstacles, and future. 

Through Guided Pathways, available on the Steps to Justice website, CLEO lets users answer free online interviews to complete legal forms, draft letters, and know their next steps. These question-and-answer-style interviews seek to deconstruct court forms and other legal documents and simplify complicated legal processes. 

According to CLEO, the report represents a milestone in its three-year research project delving into how genAI can safely and meaningfully make legal narratives more complete, maintain user control and trust, and ultimately promote access to justice for self-represented litigants. 

“Our goal is to explore genAI responsibly, transparently, and in collaboration with community and justice sector partners,” CLEO said in a news release. “We want to ensure our innovations truly support the people who rely on CLEO for accessible, high-quality public legal information.” 

CLEO added that its recent progress builds upon the foundation set in its October 2024 discussion paper, which explored how genAI can help Guided Pathways users improve their legal narratives, access real-time support, and comprehend how to proceed with their legal matters. 

Report’s contents

CLEO’s report highlighted the potential of its genAI narrative prototype, developed in collaboration with researchers at McGill University Faculty of Law and the Montreal Cyberjustice Laboratory, in helping users generate more coherent and compelling narratives with the structured information already collected via Guided Pathways. 

The report also found that CLEO’s responsible AI approach, currently under development, reflected evolving best practices in privacy and its design principles of ease of use, accessibility, privacy, security, and long-term sustainability. 

According to CLEO’s report, the team’s progress over the project’s first year included: 

• adjusting genAI prompts for better narrative organization and persuasiveness 
• setting a research and evaluation framework, which included a benchmarking process and structured scoring rubrics 
• performing over 300 controlled tests and varied-input experiments for Guided Pathways concerning family law emergency motions and Small Claims Court scenarios 
• testing a three-part suggestion framework, which included a claim checker, a story checker, and an evidence checker 
• adding a ‘Highlight Changes’ feature to identify genAI edits and ensure transparency for users 
• designing a preview of a genAI-powered chatbot, which offers 24/7 user support to answer common technical questions 

The report shared that the project team will next work on: 

• finalizing prompt sets and the suggestion framework for Guided Pathways regarding family law emergency motions, Small Claims Court scenarios, and peace bonds 
• enhancing the evaluation framework 
• evaluating privacy-enhancing technology within the system 
• scaling the narrative assistant across more Guided Pathways 
• engaging community partners and other stakeholders