Horizon Patients Can Now Ask an AI. Safety Experts Urge Caution

Patients in Horizon Health Network's cardiac, surgical and maternity programs no longer have to wait for the clinic to open to ask whether a symptom is normal. They can now ask an AI.

Horizon has launched Seamless Answers, a conversational AI feature in the SeamlessMD app, for patients in those programs, the health network announced on Oct. 22. Patients can ask questions such as "When can I remove my dressing?" or "Is this symptom normal?" and get an instant reply.

The launch comes with open questions. Horizon's release does not say how the answers will be checked for accuracy, and the usage numbers come from Horizon and the vendor. Patient-safety groups and researchers have warned about relying on chatbots for health questions.

"This new feature allows them to ask questions and get sound information anytime, including outside of normal business hours when they may not be able to contact their care provider or team," said Jennifer Sheils, Horizon's vice-president of strategy and transformation and chief innovation officer.

Only approved answers

Seamless Answers pulls information only from the education library inside the SeamlessMD app, Horizon says. When a question falls outside that approved content, patients are told to contact their care team.

SeamlessMD, a Toronto company, launched the feature in May. It describes it as retrieval-augmented generation: the AI answers only from each health system's approved patient education library, cites its source and links back to the page it used. The company says it has clinical guardrails and works in several languages.

"Technology and AI is evolving rapidly, and people increasingly expect it to help make everyday tasks simpler and more accessible, including when it comes to their health care," said Dr. Joshua Liu, SeamlessMD's CEO and co-founder. "Horizon is one of the first organizations meeting that expectation with an AI tool that is safe, appropriate and useful for patients."

The risks

ECRI, an independent U.S. patient-safety organization, ranked the misuse of AI chatbots in health care No. 1 on its list of health technology hazards for 2026. Chatbots "are programmed to sound confident and to always provide an answer to satisfy the user, even when the answer isn't reliable," ECRI said. It urged health systems to set up AI governance committees and regularly audit their AI tools.

ECRI's warning was aimed at general-purpose chatbots such as ChatGPT. Seamless Answers is built differently: it answers from a closed library. But SeamlessMD's claim that this approach "prevents hallucination" is the company's own. Neither Horizon's release nor SeamlessMD's launch announcement points to an independent test of how accurate the answers are.

The World Health Organization warned in its 2024 guidance on AI models in health of "documented risks of producing false, inaccurate, biased, or incomplete statements, which could harm people using such information in making health decisions." It also flagged "automation bias," where errors get overlooked or "difficult choices are improperly delegated" to the AI. The WHO called for independent audits, with published results, when such tools are rolled out at scale.

In a randomized study of nearly 1,300 people published in Nature Medicine in February, people who used general-purpose AI chatbots to decide what to do about doctor-written medical scenarios did no better than people using an internet search or their own judgment. The study did not test closed-library tools like Seamless Answers.

"Patients need to be aware that asking a large language model about their symptoms can be dangerous, giving wrong diagnoses and failing to recognise when urgent help is needed," said Dr. Rebecca Payne, a GP and the study's lead medical practitioner, in a University of Oxford release.

'Complement, not replace'

The New Brunswick Medical Society has said AI can help patients and doctors, but with limits. Its president, Dr. Carl Boucher, made the comments in a September statement to the Telegraph-Journal about AI health tools in general, not Seamless Answers.

AI "should complement, not replace the expertise, clinical judgment, and human connection that are central to quality medical care," Boucher said. He also warned patients to "exercise caution when sharing personal health information with AI platforms."

The Canadian Medical Protective Association advises doctors to check a tool's privacy and security, whether patients have consented to how their data is used, and whether Health Canada has approved it. It warns that "all AI models will have some inherent bias."

Privacy

SeamlessMD says Canadian patient data is stored in Microsoft Azure's Canadian data centres and "never leaves the country." It lists compliance with several privacy laws, including Ontario's PHIPA, but the list does not name New Brunswick's Personal Health Information Privacy and Access Act.

Horizon's release does not mention privacy, a privacy impact assessment, or how patients' questions are stored or reviewed.

When patients ask

In its first group of hospitals, SeamlessMD says, 55 per cent of questions came outside business hours: 26 per cent overnight and 22 per cent on weekends.

The most common topics were activity and mobility (24 per cent), pain and medication (15 per cent), wound care (8 per cent) and getting ready for surgery (6 per cent).

Unverified numbers

Horizon says 94 per cent of patients have found Seamless Answers useful so far. It did not say how many have used it, how the figure was measured or whether the answers have been checked for accuracy. The timing figures above are SeamlessMD's, from its first group of hospitals, not Horizon's.

More than 7,700 Horizon patients have signed up for the SeamlessMD app since it launched there in 2023. Horizon is working to bring it to other areas of care.

Both health networks

Horizon says Vitalité Health Network has also put Seamless Answers in place, making New Brunswick the first province in Atlantic Canada to offer it in every health region. Vitalité had not posted its own announcement as of Oct. 23.

The two networks and the province expanded SeamlessMD to maternal care across New Brunswick in March, for families during pregnancy and after birth.

Sources

• Horizon Health Network: Patients using SeamlessMD app now have access to reliable, clinician-approved AI support
• SeamlessMD: SeamlessMD launches Seamless Answers conversational AI
• Vitalité Health Network: A maternal health support tool to assist families during pregnancy and after birth
• SeamlessMD: SeamlessMD expands to maternal care across all of New Brunswick
• ECRI: Misuse of AI chatbots tops annual list of health technology hazards
• World Health Organization: WHO releases AI ethics and governance guidance for large multi-modal models
• Nature Medicine: Reliability of LLMs as medical assistants for the general public: a randomized preregistered study
• University of Oxford: New study warns of risks in AI chatbots giving medical advice
• Telegraph-Journal: Doctors weigh in on AI's growing role in N.B. health system
• Canadian Medical Protective Association: AI in medical practice
• SeamlessMD: Security, Privacy & Compliance FAQs