_edited.png)
Medical AI, clinical decision-making, and patient autonomy: from assisted decision-making to algorithmic co-clinic.
- May 9
- 10 min read
Medical artificial intelligence is often presented as a precision technology: it identifies patterns, supports diagnoses, anticipates risks, organizes clinical information, and promises to alleviate overburdened healthcare systems. This view is correct, but incomplete. The most important bioethical problem is not just whether AI is technically reliable. It is whether, upon entering the consultation, it silently transforms the way doctors and patients make decisions.
Tuan Pham's article, "Ethical and legal considerations in healthcare AI: innovation and policy for safe and fair use ," published in 2025 in Royal Society Open Science , offers a solid starting point for this discussion. Pham argues that AI in healthcare should respect autonomy, beneficence, non-maleficence, justice, transparency, and accountability, emphasizing that patients should understand when and how AI influences diagnosis or treatment (Pham, 2025). This position is prudent and necessary: AI should support clinical decision-making, not replace human judgment; it should enhance care, not obscure responsibility.
But there is a deeper issue. The main risk of medical AI is not just taking information away from the patient. It is subtly taking away their authorship of the decision . The patient may continue to sign consent forms and listen to explanations, but cease to recognize the decision as truly their own, because the recommendation is mediated by a technical authority that is difficult to challenge.
Google DeepMind's AI Co-Clinician
The discussion became even more concrete with the announcement of Google DeepMind's AI Co-Clinician in April 2026. The initiative proposes a "triadic care" model: patient, AI, and physician. AI is not presented as a replacement for the professional, but as a support system for the clinical journey, under the authority of the responsible physician (Karthikesalingam, Natarajan and Kohli, 2026).
This development is bioethically crucial. AI is no longer just in the background and is getting closer to the clinical encounter itself. According to Google DeepMind's description, AI Co-Clinician was explored on two levels: supporting the clinician, through evidence synthesis and decision support; and supporting the patient, through multimodal interactions in telemedicine simulations with real-time audio and video (Karthikesalingam, Natarajan and Kohli, 2026).
The company describes safety assessments focused on errors of commission and omission, as well as promising results in realistic primary care questions and open-ended medication-related tasks (Karthikesalingam, Natarajan and Kohli, 2026). However, this data should be interpreted with caution. This is a research initiative, not a clinically validated product for widespread use. Independent studies, peer review, prospective validation, real-world assessment, and impact analysis on vulnerable populations are needed.
Most importantly for bioethics, it's not just the system's performance. It's the shift in category. The AI Co-Clinician shows that medical AI is moving from a support tool to a supervised clinical presence . It can gather information, structure symptoms, observe signs, suggest questions, support clinical reasoning, and influence communication.
This confirms the thesis of the third clinical voice. If AI speaks to the patient, observes the patient, and guides part of the interaction, then traditional informed consent is no longer sufficient. The patient must know what role the AI played, what data it collected, what recommendations it produced, who reviewed the result, and how they can request an alternative human evaluation.
From autonomy as choice to autonomy as authorship.
Classical bioethics understands autonomy as the capacity to act with intention, understanding, and without undue coercion (Beauchamp and Childress, 2019). This definition remains essential. However, medical AI introduces a new difficulty: coercion may no longer appear as direct imposition, but as a decisional architecture .
When an algorithmic system suggests a diagnosis, calculates a risk, prioritizes a patient, or recommends a treatment, the decision is no longer made solely between doctor and patient. A third voice emerges: the statistical voice of the machine. This voice can be useful, but it can also reorganize the consultation, make certain options more visible, reduce alternatives, and create an appearance of technical inevitability.
Mittelstadt et al. demonstrated that algorithmic systems raise ethical problems that go far beyond privacy, including opacity, bias, discrimination, accountability, and effects on autonomy (Mittelstadt et al., 2016). In medicine, these problems become especially sensitive because clinical decision-making occurs in situations of vulnerability, fear, dependence, and knowledge asymmetry.
Therefore, the question should no longer be simply: "Was the patient informed that AI was used?". The more demanding question is: Did the patient understand the weight of AI in the decision, was able to question it, had access to alternatives, and maintained the decision as an expression of their own will?
The merit and the limits of transparency.
Pham is right to argue that autonomy requires clear communication about the function, accuracy, limitations, and uncertainties of AI systems used in healthcare (Pham, 2025). The World Health Organization has followed the same guidance, arguing that medical AI should only be implemented with respect for human rights, inclusion, safety, accountability, and ethical governance (World Health Organization, 2021; World Health Organization, 2024). The European Union's AI Act also treats many medical AI systems as high-risk systems, subject to increased requirements for risk management, data quality, human oversight, and transparency (European Commission, 2024).
However, transparency alone may be insufficient. Telling the patient that "AI was used" does not guarantee autonomy. It may even create a false sense of ethical compliance. The crucial question is what was explained: Did the AI merely organize information? Did it suggest hypotheses? Did it influence the diagnosis? Did it recommend a treatment? Was it reviewed by a professional? Were there alternatives?
Joshua Hatherley questions precisely the idea that simply disclosing the use of machine learning is always sufficient or morally decisive (Hatherley, 2025). Cohen and Slottje, in turn, show that AI forces us to rethink informed consent, because what matters is knowing when the use of AI is material to the patient's decision and how it should be integrated into the shared decision-making process (Cohen and Slottje, 2024).
This leads to a fundamental distinction: disclosure is not explanation; explanation is not understanding; understanding is not autonomy; and autonomy is not yet authorship . Authorship requires that the patient recognize the decision as compatible with their values, their history, their acceptable risk, and their conception of a good life.
Algorithmic paternalism
Medicine has always had a knowledge asymmetry: the doctor knows more than the patient about diagnosis and treatment. AI adds a new asymmetry. Often, even the doctor himself does not fully understand the inner workings of the system he uses. Thus, a chain of trust is formed: the patient trusts the doctor; the doctor trusts the system; the institution trusts the provider; the provider trusts the data.
This chain can be useful, but it can also make moral responsibility diffuse. Malta and Lamy describe this risk as algorithmic paternalism , that is, a situation in which AI limits the patient's active participation through low transparency, over-reliance, and insufficient human oversight (Malta and Lamy, 2025).
The patient is not coerced; he is guided. He is not silenced; he is framed. He is not formally deprived of choice; he is presented with options already organized by a technical rationality that he can hardly re-establish. This is perhaps one of the most subtle threats to contemporary autonomy.
The case studied by Obermeyer et al. is particularly illustrative. The authors demonstrated that an algorithm used in population health management in the United States exhibited racial bias because it used healthcare costs as an indirect indicator of clinical need, reproducing pre-existing inequalities in access to care (Obermeyer et al., 2019). The problem was not merely technical. It was moral: historical inequalities were transformed into seemingly neutral criteria.
Trust is not obedience.
Trust is essential in medicine. But trust is not the same as obedience. Jones, Thornton, and Wyatt show that clinicians distinguish between trusting a system and that system being truly trustworthy (Jones, Thornton, and Wyatt, 2023). This difference is crucial.
A system may be used because it is integrated into the hospital, because it saves time, because it is recommended by management, or because it seems more objective than human judgment. But none of that guarantees that it is clinically appropriate, ethically fair, or transparent for the patient.
In the doctor-patient relationship, the risk is that trust in AI will turn into epistemic obedience. The patient accepts the recommendation not because they understand it, but because they feel it would be irrational to contradict the machine. The decision seems free, but it has been shaped by an almost unquestionable technical authority.
The third voice in the consultation
The central argument of this article is that medical AI should be understood as a third clinical voice . It is not merely an invisible tool, nor a substitute for the physician. It is an epistemic presence that participates in the decision-making process.
This third voice recognizes patterns, calculates probabilities, and organizes information. But it doesn't experience the consequences of the decision. It doesn't feel fear, hope, shame, faith, fatigue, or ambivalence. It can improve medicine, but it doesn't replace the human depth of the clinical relationship.
Dahlin shows that autonomy in human-AI encounters must be considered relationally: humans grant and withdraw autonomy from systems according to context, use, and the trust placed in them (Dahlin, 2024). In medicine, this idea is fundamental. The problem is not only whether AI "decides." It is about how doctors and patients will behave in the face of a system that appears to participate in the decision.
The ethics of medical AI must therefore ask: who structured the decision? Who defined the options? Who assigned weight to the risks? Who made one alternative more visible than another? Who answered the patient's questions? Who assumes ultimate responsibility?
Towards a bioethics of clinical authorship
Autonomy, in the context of medical AI, should be rethought as clinical authorship . This authorship has four main requirements.
First, it requires sufficient intelligibility . The patient doesn't need to understand the technical details of the model, but should be able to tell if AI was used for diagnosis, prognosis, screening, treatment, communication, or administrative management.
Secondly, it requires contestability . Algorithmic recommendations must be open to questioning by both doctors and patients. A system that cannot be challenged becomes a closed clinical authority.
Third, it requires a plurality of alternatives . The AI recommendation should not appear as the only rational path. The patient must be aware of clinically acceptable alternative options.
Fourth, it demands identifiable accountability . AI cannot create a gray area where the doctor blames the system, the hospital blames the provider, and the patient is left without a moral interlocutor. Accountability must remain human, clinically and institutionally auditable.
Practical implications
For hospitals, ethics committees, and healthcare institutions, the practical application is clear. It's not enough to say that AI is ethical. Concrete rules need to be created to protect patient autonomy.
Any clinical AI system should be classified according to its impact on decision-making: administrative, informative, recommendatory, communicational, assisted decision-making, or critical decision-making. The greater the impact, the greater the need for explanation, documentation, and human supervision.
Informed consent must distinguish between AI used for internal efficiency, AI used to influence diagnosis or treatment, and AI that interacts directly with the patient. Whenever direct interaction exists, there must be a specific conversational consent regime: the patient must know that they are communicating with AI, what the purpose of the interaction is, that the AI does not replace the doctor, that they can interrupt or refuse the interaction, and that relevant information will be reviewed by a human professional.
The most important practical question should not simply be: “Does this system work?”. It should be: Does this system improve the patient's ability to participate in the decision, or does it only improve the efficiency with which the decision is presented to them?
Conclusion
Medical AI can profoundly improve healthcare. It can make medicine faster, more accurate, accessible, and personalized. But its ethical legitimacy depends on a fundamental condition: the technology must serve the clinical relationship, and not the clinical relationship serve the technology.
Pham is right to argue that AI should be used with transparency, fairness, human oversight, and responsibility (Pham, 2025). But bioethics must go further. Patient autonomy is not protected solely through information. It is protected by ensuring that the patient remains the author of the decision that affects their body, their health, their suffering, and their future.
Google DeepMind's AI Co-Clinician demonstrates that this debate is no longer futuristic. AI is beginning to enter the consultation space itself. It can listen, observe, speak, synthesize evidence, and support decision-making. This evolution can be extraordinarily beneficial. But the closer AI gets to the patient, the greater the demand for intelligibility, contestability, human oversight, and accountability must be.
The medicine of the future will not be ethically superior because it is more automated. It will be superior if it uses artificial intelligence to make clinical decision-making more understandable, fairer, more humane, and more truly shared. The challenge is not choosing between doctor and machine. It is preventing the patient from disappearing in the space between them.
References
Alelyani, T. (2025) 'A validated framework for responsible AI in healthcare autonomous systems', Scientific Reports , 15, 44432.
Beauchamp, TL and Childress, JF (2019) Principles of Biomedical Ethics . 8th edn. Oxford: Oxford University Press.
Cohen, IG and Slottje, A. (2024) 'Artificial intelligence and the law of informed consent', in Solaiman, B. and Cohen, IG (eds.) Research Handbook on Health, AI and the Law . Cheltenham: Edward Elgar.
Dahlin, E. (2024) 'And say the AI responded? Dancing around “autonomy” in AI/human encounters', Social Studies of Science , 54(1).
European Commission (2024) 'Artificial Intelligence in healthcare'. European Commission.
Hatherley, J. (2025) 'Are clinicians ethically obligated to disclose their use of medical machine learning systems to patients?', Journal of Medical Ethics , 51(8), pp. 567–573.
Jones, C., Thornton, J. and Wyatt, J.C. (2023) 'Artificial intelligence and clinical decision support: clinicians' perspectives on trust, trustworthiness, and liability', Medical Law Review , 31(4), pp. 501–520.
Karthikesalingam, A., Natarajan, V. and Kohli, P. (2026) 'Enabling a new model for healthcare with AI co-clinician', Google DeepMind Blog , 30 April.
Karthikesalingam, A., Natarajan, V. and Kohli, P. (2026) 'AI co-clinician: researching the path toward AI-augmented care', Google DeepMind Blog , 30 April. Available at: https://deepmind.google/blog/ai-co-clinician/
Malta, KC and Lamy, M. (2025) 'The challenges of patient autonomy in the face of the use of artificial intelligence in health', Cadernos Ibero-Americanos de Direito Sanitário , 14(4), pp. 28–52.
Mittelstadt, B. D., Allo, P., Taddeo, M., Wachter, S. and Floridi, L. (2016) 'The ethics of algorithms: Mapping the debate', Big Data & Society , 3(2).
Obermeyer, Z., Powers, B., Vogeli, C. and Mullainathan, S. (2019) 'Dissecting racial bias in an algorithm used to manage the health of populations', Science , 366(6464), pp. 447–453.
Pham, T. (2025) 'Ethical and legal considerations in healthcare AI: innovation and policy for safe and fair use', Royal Society Open Science , 12(5), 241873.
UNESCO (2021) Recommendation on the Ethics of Artificial Intelligence . Paris: UNESCO.
World Health Organization (2021) Ethics and governance of artificial intelligence for health: WHO guidance . Geneva: World Health Organization.
World Health Organization (2024) Ethics and governance of artificial intelligence for health: guidance on large multi-modal models . Geneva: World Health Organization.
Comments