Research

Trustworthy AI in healthcare, NLP for the Sustainable Development Goals, and — now — how to evaluate agents.

1. How do you attribute an agent's failure to the step that caused it, rather than grading only the final state?
2. What does a simulated world need to contain for an eval result to transfer to production behavior?
3. What makes a scenario cheap enough to write that engineers write one when they find a bug, and expensive enough to be right that the suite can be trusted?
4. What has to be true of a preference signal collected from real usage for training on it to improve the product rather than the metric?
5. What is the right unit of an eval scenario for systems that both build automations and repair them when the world changes?
6. Which properties of a benchmark predict that improvements on it will show up for real users?

Updated Sep 2026. I revise this list twice a year.

Also on Google Scholar.

• 2023
  Sailing the Data Sea to Advance Research on the Sustainable Development Goals

  Spezzatti A, Kheradmand E, Gupta K, Peras M, Zaminpeyma R

  Argues that progress on the SDGs is bottlenecked by data discoverability, and sets out the design and ethics of a global catalog to fix it.

  The Ethics of Artificial Intelligence for the Sustainable Development Goals, Springer · DOI
• 2022
  To explain or not to explain? — Artificial intelligence explainability in clinical decision support systems

  Amann J, Vetter D, Blomberg SN, Christensen HC, Coffee M, Gerke S, … Spezzatti A, … (Z-Inspection initiative)

  Whether a clinical AI should explain itself depends on validation level, deployment context and who is deciding — explainability is not always the right answer.

  PLOS Digital Health · PDF · DOI
• 2022
  Assessing trustworthy AI in times of COVID-19: deep learning for predicting a multiregional score conveying the degree of lung compromise

  Allahabadi H, Amann J, Balot I, Beretta A, Binkley C, Bozenhard J, … Spezzatti A, … Zicari RV

  A post-hoc trustworthiness assessment of a deployed chest-X-ray severity model in Brescia, applying the EU HLEG framework under pandemic pressure.

  IEEE Transactions on Technology and Society · DOI
• 2022
  Note: Leveraging artificial intelligence to build a data catalog and support research on the sustainable development goals

  Spezzatti A, Kheradmand E, Gupta K, Peras M, Zaminpeyma R

  Presents the SDG Data Catalog system: NLP over open-access literature to build a global database of datasets, metadata and research networks.

  ACM SIGCAS/SIGCHI COMPASS · DOI
• 2021
  Co-design of a trustworthy AI system in healthcare: deep learning based skin lesion classifier

  Zicari RV, Ahmed S, Amann J, Braun SA, Brodersen J, … Spezzatti A, … Wurth R

  Shows how an ethically-aligned co-design process shapes a skin-lesion classifier from the earliest design phase, with a multidisciplinary team.

  Frontiers in Human Dynamics · PDF · DOI
• 2021
  On assessing trustworthy AI in healthcare: Machine learning as a supportive tool to recognize cardiac arrest in emergency calls

  Zicari RV, Brusseau J, Blomberg SN, Christensen HC, Coffee M, … Spezzatti A, … Amann J, et al.

  Applies the EU trustworthy-AI guidelines to a live emergency-call cardiac-arrest detector in Copenhagen and surfaces the trade-offs an interdisciplinary team actually faces.

  Frontiers in Human Dynamics · PDF · DOI
• 2020
  Hidden in Plain Sight: Building a Global Sustainable Development Data Catalogue

  Hodson J, Spezzatti A

  Describes mining millions of open-access papers to build an open, extensible catalogue of SDG-relevant datasets.

  ICT Analysis and Applications, Springer · DOI

• 2018 — 2021
  SDG Data Catalog

  Led the UN SDG Data Catalog at the AI for Good Foundation: an NLP extraction pipeline (BiLSTM-CRF, fine-tuned BERT) over millions of academic papers that maps datasets, research, and organizations across the Sustainable Development Goals, reaching over 92% recall on dataset discovery. Secured $300K+ in Microsoft funding and was presented at SIGKDD 2019.

  Code · AI4Good
• 2020 — 2023
  Z-Inspection

  Co-developed a process, grounded in applied ethics and the EU High-Level Expert Group definition of trustworthy AI, for assessing whether an AI system can be trusted in practice. Applied it to four production healthcare AI systems with clinicians, ethicists, and legal scholars; the assessments became four peer-reviewed publications.

  z-inspection.org
• 2024
  MicroBio LLM

  Fine-tuned Mistral and LLaMA on k-mer DNA sequences for metagenomic classification, reaching 82% family-level taxonomic accuracy by treating genomic sequences as a language.

  Code
• 2019
  Maia — Music Generation

  Trained LSTM and Transformer architectures from scratch on tokenized MIDI corpora at UC Berkeley, with sparse attention for compute efficiency. The accompanying write-up became one of the most-read Towards Data Science pieces on music generation.

  Code · Article
• 2020 — 2021
  DE&I Tracker

  Monthly public tracking of workforce diversity across 1.3M+ US organizations, built with the AI4Good Foundation.

  Code · AI4Good
• 2019
  SciKid-Learn

  An interactive tool that adapts student learning to cognitive preferences and learning styles. Presented at SIGKDD 2019.

  Code · Paper

1. Apr 2021
   SDG Data Catalog funded by Microsoft

   AI4Good / Microsoft
2. Jun 2019
   MEng alumnus Andy Spezzatti speaks at SIGKDD 2019

   Berkeley Fung Institute