IPM Take
The debate over clinical AI is still often framed as a question about whether algorithms will eventually become useful enough to enter medical practice.
The new data suggest that framing is already behind reality.
More than 950,000 US clinicians submitted 271 million questions to OpenEvidence between January 2024 and June 2026, and the most common questions were not diagnostic puzzles designed to test whether an AI could identify the right disease. Clinicians were asking what treatment to use, how to manage medications, what guidelines recommend and how to handle clinical problems that often fell outside their own specialty.
Cardiology sat at the centre of that behaviour. It was the most common medical topic among physician queries, even though cardiologists themselves used the platform less heavily than several other specialties.
That creates a different AI policy problem from the one most benchmarking studies have been designed to solve.
If an oncologist, surgeon or family physician is asking an AI platform about anticoagulation, heart failure therapy or cardiovascular drug safety, the relevant test is not whether the model can pass a cardiology examination. It is whether the answer is current, evidence-based, appropriately caveated and safe enough to influence a real treatment decision.
The study does not tell us whether the answers were correct or whether AI improved patient outcomes. It tells us something that arguably has to come first: what clinicians are actually using these systems for.
Governance now has to catch up with behaviour.
Executive Summary
A cross-sectional study published in JAMA examined clinician use of OpenEvidence, an AI-enabled clinical information platform that synthesises responses to free-text medical questions and provides citations to scientific literature and professional guidance.
Between January 2024 and June 2026, 952,008 US clinicians submitted approximately 271 million queries, including 556,200 physicians. Monthly active use increased from just 355 clinicians in January 2024 to more than 703,000 by June 2026, while monthly query volume rose from approximately 6,000 to 27.8 million.
Use was not concentrated only among physicians. Nurse practitioners and physician assistants each submitted an average of 44.5 queries per active user-month, compared with 37.1 among physicians.
Career stage mattered as well. Physicians within ten years of medical-school graduation accounted for a substantially larger share of queries than their representation in the workforce, while residents and fellows were also disproportionately heavy users.
Most importantly, clinicians were using the platform for clinical decisions rather than primarily for diagnosis. Across physicians, 51.9% of queries concerned clinical decision-making, with treatment and management accounting for 33.1%, while diagnosis represented 13.9%.
Cardiology was the single most common medical topic among physician queries, accounting for 8.6%, ahead of infectious disease and oncology/hematology. It was also particularly prominent among pharmacists, representing 11.3% of their queries.
Physicians frequently asked questions outside their own specialties. Depending on specialty, between 35% and 88% of queries concerned another clinical field.
Among cardiology professionals specifically, treatment and management accounted for 28.1% of queries, followed by medication information, disease information, tests and imaging, literature searches and risk or prognosis. Cardiologists were also among the specialties most likely to ask specifically about clinical guidelines.
The study describes use, not clinical performance. It did not assess whether AI responses were accurate, whether clinicians followed them or whether patient outcomes changed.
That limitation is also the central policy message: clinical AI evaluation may need to shift toward the treatment, medication and cross-specialty questions that clinicians are already asking in practice.
Why it matters
- HTA bodies: Clinical AI may increasingly function as an evidence-intermediation technology rather than a conventional diagnostic device. Assessment frameworks will need to consider accuracy, source quality, generalisability, updating frequency and effects on clinical decisions, while determining whether these tools improve outcomes or simply change how clinicians retrieve information.
- Payers: AI evidence platforms could reduce the time clinicians spend searching guidelines and literature, potentially improving efficiency and consistency of care. But widespread utilisation does not prove value. Payers and health systems will need evidence on whether AI-supported decisions reduce unnecessary testing, inappropriate prescribing, referral delays or other measurable costs.
- Industry / innovation partners: The findings change the competitive benchmark for clinical AI. Products will increasingly be judged not only on whether they can answer medical questions, but whether they can reliably support complex treatment decisions across specialty boundaries, surface trustworthy evidence and communicate uncertainty in a way clinicians can safely use.
For years, one of the favourite ways to test medical artificial intelligence has been to give it an examination.
Researchers have asked large language models board-style questions, diagnostic vignettes and multiple-choice clinical problems, then compared the answers with physician performance or official answer keys. Those studies have been useful for establishing that increasingly sophisticated models can retrieve and organise a remarkable amount of medical knowledge, but they have also created an artificial picture of what clinical AI looks like once a doctor opens a laptop in the middle of an ordinary working day.
The questions clinicians actually ask appear to be different.
A new study in JAMA analysed activity on OpenEvidence, an AI-enabled clinical information platform available to verified US healthcare professionals, and captured an extraordinary volume of real-world use. Over two and a half years, more than 950,000 clinicians submitted approximately 271 million questions, providing one of the clearest pictures yet of how AI-based medical information tools are being incorporated into clinical work.
What clinicians wanted most was not help proving that they could recognise a diagnosis.
They wanted help deciding what to do next.
More than half of the queries analysed concerned clinical decision-making, and treatment and management alone accounted for roughly one third of physician questions. Diagnosis, by comparison, represented less than 14%.
That distinction has major implications for the way clinical AI is evaluated.
A model can perform extremely well when asked to identify the correct answer from four possible diagnoses and still struggle when presented with a patient taking multiple medications, several chronic conditions and an incomplete evidence base. Treatment decisions require more than recall. They require interpretation of guidelines, contraindications, competing risks, patient characteristics and evidence that may not point neatly in one direction.
Those are the questions clinicians are already bringing to AI.
Cardiology provides an especially revealing example because cardiovascular medicine frequently intersects with almost every other specialty. Anticoagulation matters before surgery. QT prolongation matters in psychiatry and oncology. Heart failure medications overlap with kidney disease and diabetes. Cardiovascular risk alters decisions in primary care, endocrinology and rheumatology, while imaging and biomarker results routinely spill beyond cardiology departments.
In the JAMA analysis, cardiology was the most common medical topic among physician queries overall, accounting for 8.6%, despite cardiologists themselves using the platform at a below-average rate compared with several other specialties.
That apparent contradiction tells a more interesting story than raw adoption numbers.
Cardiology knowledge is increasingly being accessed by clinicians who are not cardiologists.
Internal-medicine physicians, family doctors, emergency clinicians, surgeons, pulmonologists and others were among those asking cardiovascular questions. Across the study more generally, physicians frequently queried topics outside their own specialty, with the proportion ranging from 35% among oncologists and hematologists to 88% among emergency physicians.
AI is therefore beginning to occupy a space that has traditionally been filled by several different mechanisms: searching a guideline, opening a reference database, asking a colleague, requesting a curbside consultation or formally referring to another specialist.
The study cannot tell us which of those behaviours AI is replacing.
That uncertainty matters.
If a clinician uses AI to refresh knowledge before speaking with a cardiologist, the technology may improve the quality of the eventual consultation. If AI instead becomes a substitute for specialist input in a complex case, the safety implications are very different.
The same ambiguity applies to timing. The researchers could see what clinicians asked, but not whether a query was entered while the patient was sitting in front of them, during preparation for a clinic session or later as part of general learning.
That means the study should not be interpreted as evidence that hundreds of thousands of clinicians are simply following AI-generated instructions at the bedside.
It does show that AI has moved far beyond occasional experimentation.
Monthly active users increased from only 355 at the beginning of 2024 to more than 703,000 by June 2026. By that point, more than 92% of active users were returning rather than first-time users, while average query intensity remained high.
That pattern suggests AI-supported evidence retrieval is becoming habitual for a substantial proportion of users.
The generational divide is equally revealing. Physicians within ten years of medical-school graduation accounted for a much greater share of queries than their representation in the national workforce, while residents and fellows were also heavy users. Clinicians with fewer than ten years in practice averaged 43.4 queries per active month compared with 29.9 among those with at least two decades of practice.
That could represent familiarity with digital interfaces, differences in clinical confidence or a broader shift in how medical knowledge is being managed.
For much of modern medical education, professional competence has been built around acquiring a large internal store of knowledge and then periodically updating it through journals, conferences, guidelines and continuing education. AI creates a different model in which some of that burden moves from recall toward retrieval.
A clinician may not remember every recommendation in the latest heart-failure guideline because a system can retrieve and synthesise it in seconds.
That does not necessarily reduce the importance of expertise.
It may increase it.
The easier information becomes to obtain, the more important it becomes to recognise when that information is incomplete, inapplicable or wrong.
This is especially relevant in cardiology, where guideline recommendations can depend on details such as ejection fraction, renal function, blood pressure, rhythm status, comorbidities and previous therapies. A superficially plausible answer that overlooks one of those variables could produce a very different clinical decision.
Research evaluating large language models already suggests why this distinction matters. Models can perform impressively on conventional multiple-choice cardiology questions while losing substantial accuracy when questions become open-ended or when important clinical details are modified. High examination scores therefore do not automatically establish robust clinical reasoning.
Yet examination-style performance remains one of the most visible ways medical AI is compared.
The new real-world data suggest that evaluation needs to become much less comfortable.
If clinicians are asking treatment questions, AI should be tested on messy treatment questions. If they are asking about drug interactions across specialties, evaluations should reproduce those cross-disciplinary situations. If they are relying on AI to retrieve guidelines, testing should examine whether recommendations remain current after a guideline changes and whether the system distinguishes strong recommendations from weak evidence.
And if clinicians are asking AI about patients outside their own area of expertise, evaluators should pay particular attention to overconfidence, because users may be least able to identify an incorrect answer precisely when they are relying on the system most heavily.
The OpenEvidence study itself also deserves scrutiny.
The analysis concerned a single commercial platform whose design is specifically focused on evidence synthesis and whose access to medical-publishing content may differ from other AI products. The results therefore should not be assumed to represent all generative AI tools used by clinicians.
Several study authors were also employed by OpenEvidence, including its chief medical officer, and the company provided the underlying data and participated in analysis. Those relationships were disclosed in the JAMA publication and do not invalidate the dataset, but they make independent replication across other platforms particularly important.
There is another unusual methodological layer: AI was used to study AI use.
Researchers drew a stratified random sample of 38.8 million queries and used a large language model to classify their clinical purpose and medical topic. That classification was checked against clinician reviewers, with agreement comparable to agreement between clinicians themselves, but some uncertainty remains because the categories were inferred rather than manually assigned to millions of questions.
Even with those limitations, the scale of the findings makes the broader message difficult to dismiss.
Clinical AI is no longer waiting outside the health system for permission to become relevant.
Clinicians are already using it to navigate a volume of medical evidence that has become impossible for any individual to retain, and they are often using it precisely where medicine becomes most complicated: choosing therapies, managing medications and crossing the boundaries between specialties.
The next phase of clinical AI policy therefore cannot be built entirely around hypothetical future adoption or polished benchmark scores.
Health systems need to know how these tools affect actual decisions, whether clinicians verify their answers, when AI complements specialist consultation and when it substitutes for it, how errors propagate and whether the overall result is better care.
Those questions are considerably harder than asking an AI to pass an exam.
They are also the questions that now matter.
Clinical AI has already moved from answering medical trivia to sitting beside real treatment decisions. The evidence and governance frameworks now need to move with it.
