Bias in AI Datasets

Integrity issues in Generative Artificial Intelligence (AI) results negatively impact patient outcomes (Miliard, 2025).  Types of bias in AI datasets include:

• Algorithmic bias and stereotyping:
  • Training data must necessarily reflect historical and systemic inequalities.
  • Needs of minority patients are historically underestimated.
  • English data is overrepresented and downplays non-English views.
• Interaction bias (Hanna, 2025):
  • User engagement with AI can impact the system’s performance and impartiality.
  • Users are not aware that the AI system is less reliable for some populations.
• Automation bias:
  • People tend to trust that information a computer provides is accurate (Nguyen, 2025).
  • Blindly believing AI results can mislead the clinician's decision-making.

AI bias can be mitigated by using information literacy and critical appraisal just like you would with any other information source. Guides to avoiding misinformation, such as University of Chicago’s SIFT method, can sharpen your digital literacy skills.  CASP checklists are a great place to start when critically appraising biomedical information.

If you would like further assistance with using Generative AI for research, contact the Library at hsl@geisinger.edu.