MIL-OSI Economics: Meet four developers leading the way with AI agents

Source: Microsoft

Headline: Meet four developers leading the way with AI agents

Cancer care has made huge progress in recent years. But some cases don’t fit the tried-and-true treatments. These patients’ cases are sent to a “tumor board,” a team that includes such specialists as a radiologist, a pathologist, the oncologist, a surgeon and others who bring their expertise together to provide the best care.

Tumor boards are “high-stakes, high-cost meetings for high-risk patients,” says Timothy Keyes, a data scientist at Stanford Health Care and a combined MD and PhD candidate in cancer biology and biomedical informatics at Stanford University School of Medicine.

As a medical student, Keyes helped attending oncologists prepare cases to be presented to the tumor board. “They take a ton of time to prep,” he says. Information must be culled from many different sources, from electronic health records to imaging scans to medical literature, and it might not be easy to find. It all has to be summarized for presentation to the tumor board.

Now, a new tool developed by Microsoft is enabling Stanford data scientists and developers to build and test AI agents to help alleviate this administrative burden and speed up the workflow for tumor board preparation. Microsoft’s new healthcare agent orchestrator is now available to others in the Agent Catalog in Azure AI Foundry.

The healthcare agent orchestrator has helped the Stanford team build and test autonomous AI agents that consult disparate data sources and collaborate on tasks that might otherwise take hours – building a chronological patient timeline, synthesizing current literature, referencing treatment guidelines, sourcing clinical trials and generating reports – using clinically grounded knowledge to deliver accurate and reliable results. Stanford Health Care is still testing its application of the healthcare agent orchestrator in a research setting and has not yet put it to real-time clinical use.

All the agents work with Microsoft 365 Copilot so that busy clinicians don’t have to spend precious time onboarding to use the agents – they can simply type what they want in natural language in apps like Teams or Word without having to add another application to their workflow. Stanford Health Care is just one institution putting Microsoft’s healthcare agent orchestrator in Foundry through its paces.

Agents can get past the fragmentation of data that come from clinician notes, notes from the staff that deals with insurance, notes from nurses, images such as CT scans that are very different from pathology slides, and more, Keyes says.

“It’s really hard to get a chat model to do this,” he says. But agents can focus on a specialized task, with the healthcare agent orchestrator directing requests to the appropriate agent. Getting started is really easy. Stanford Health Care set up the initial agents from Azure AI Foundry Agent Catalog and deployed into Microsoft Teams for testing in about 10 minutes, Keyes says.

The data organizer brings in clinical notes, labs, medications and genomic data, all of which come in different formats, and structures the information into a succinct abstract, with citations so the clinician can quickly verify it or go to see the relevant section in depth.

Keyes recalls being with other medical trainees and his attending physician asking for a radiology report in the electronic health record. “And it’s like, click, click, click, click, click, click – 100 different clicks versus ‘oh, it’s right here in front of me.’’’ When he checked the agent’s citations against the actual notes, they were correct.

The radiology agent reads radiology images using the leading specialized AI models on Azure AI Foundry, and the pathology agent analyzes the whole-slide images and provides relevant pathology findings. Another agent identifies which clinical trials the patient is eligible for.

The medical research agent uses reasoning models to search over scientific papers on cancer, again giving links for quick retrieval of the full documents.

At the end of the process, a report creation agent summarizes the key components of the patient’s case to be discussed at the tumor board, turning it into a Word document or PowerPoint.

Preparing a single patient’s case for a tumor board could take Keyes several hours; in testing, AI agents might make the work 10 times faster, he says. Stanford Health Care has more than a dozen tumor boards serving about 4,000 patients, so the time savings would multiply quickly.

“The agents will enable the work to be done easier, faster and more efficiently, which really matters when you’re talking about meetings with 10 clinicians in them, where time is really precious,” Keyes says. Time is precious, too, for the patients.

“I think in a lot of industries when they think agentic, they get very excited about, ‘it’s going to work very autonomously. It’s going to be making decisions, and I can just look at what it’s doing every once in a while.’ That is not really what we’re envisioning. We do want the clinicians in charge of a patient’s care. We always want them to be able to check.”

“I would be excited at the idea of AI helping my doctors to be the best version of themselves and to liberate them from some of the time-consuming components of documentation so they can spend more time with me the patient,” he says.