Code on computer screen
News

The Open‑Source AI Accessibility Checker Holding LLMs Accountable

3 minute read
Michelle Hawley avatar
By
SAVED
GAAD Foundation and ServiceNow launch AIMAC, an open‑source framework that uses axe‑core to score LLMs’ HTML output for accessibility — and publishes scores.

It's Global Accessibility Awareness Day (GAAD), and the GAAD Foundation, partnered with ServiceNow, have unveiled the AI Model Accessibility Checker (AIMAC). This open‑source framework is designed to evaluate how well coding‑focused large language models (LLMs) produce accessible HTML code.

By providing a standardized accessibility score and public leaderboard, AIMAC pushes LLM providers to make inclusive code standards a core part of their development lifecycle, rather than an afterthought. This initiative represents a major step forward for enterprises looking to integrate accessibility into their broader enterprise AI initiatives and AI governance strategies.

Introducing the AI Model Accessibility Checker (AIMAC)

AIMAC is an extensible evaluation framework that:

  • Sends prompts to LLMs and analyzes returned HTML against the axe‑core accessibility engine.
  • Generates a comparative score, weighted by severity of issues, so teams can benchmark performance.
  • Supports fully customizable prompts, making it adaptable to different use cases — from design and layout to semantic structure.

Open‑Source, Extensible Framework

“Accessibility must be a foundational requirement as AI reshapes our digital future,” said Joe Devon, GAAD co‑founder. “With AI adoption accelerating, there’s a risk of the industry becoming a ‘winner takes all’ space dominated by a handful of companies. If accessibility isn’t prioritized, people with disabilities risk being systematically excluded from AI’s transformative potential."

AIMAC, according to Devon, addresses this risk by embedding accessibility as a baseline standard into AI innovation. 

Partnership and Support

ServiceNow hosts AIMAC on GitHub and provides critical executive sponsorship and cross‑functional partnerships.

“Accessibility should never be an afterthought. It must be embedded into every phase of the product development lifecycle,” said Eamon McErlean, VP and global head of accessibility at ServiceNow. While the tech industry has made progress, added McErlean, accessibility has been an afterthought for far too long.

"We can’t let history repeat itself with AI," he said. "That’s why I’m proud to launch AIMAC with Joe — a trusted advocate, expert and ServiceNow collaborator — as we join forces to champion inclusive innovation and ensure AI experiences are equitable from the start.”

Related Article: A Practical Guide to AI Governance and Embedding Ethics in AI Solutions

How to Define 'Accessible Code' 

"We want LLM providers to treat accessible code generation as a measurable benchmark, just like code correctness, performance and security," said Devon.

To do this, he explained, they created a public leaderboard that scores models on how accessibly they general HTML. "We prompt each model to generate pages, then pass the output through an automated evaluation layer built on top of axe‑core, a widely adopted accessibility testing engine. Each issue is weighted by severity to produce a single accessibility score."     

This system is not just for benchmarking, according to Devon. "It is trivial for any model provider to integrate axe‑core into their training or evaluation pipelines. This allows them to generate massive volumes of synthetic HTML with and without known accessibility violations, which can be used not only for testing but also to improve training data and model behavior."

AIMAC offers a share, standardized framework that makes this process repeatable and comparable. Without a system like this, said Devon, accessibility will likely go on the back burner, if not ignored entirely. "Our goal is to make accessible code generation both measurable and improvable, so that better defaults become the norm. While we are currently focused on static HTML, we intend to expand the scope to component libraries, frameworks and mobile platforms.”

Why AIMAC Is Open Source

According to Devon, the decision to open-source AIMAC was two-fold: to encourage competition among LLMs and also provide a shared foundation any organization can use to evaluate how accessibly their models generate code and improve those systems accordingly. 

"To make the evaluations meaningful, we designed our prompts to be neutral and representative," said Devon. "We avoided injecting accessibility hints or biasing the models toward good behavior. The goal is to measure how accessibly a model performs by default, then improve from there."

In the AI space, he continued, especially in enterprise settings, there's a strong understanding that internal evaluations are essential. "Any serious team developing or deploying LLMs already has testing infrastructure in place to track model performance. In the accessibility space, this kind of infrastructure is less common. Not due to a lack of interest, but because many teams have less experience with LLMs and have not yet had access to concrete examples or testing frameworks tailored to accessibility concerns."

Learning Opportunities

By going open source, according to Devon, they hope to bridge that gap, giving AI and accessibility teams a practice and low-friction way to evaluate and improve accessible code generation. 

AIMAC's Impact and Next Steps

  • Benchmarking and Transparency: Companies can now publicly demonstrate their commitment to accessibility best practices in AI development.
  • Community‑Driven Enhancements: As an open‑source project, AIMAC invites contributions that will expand its scope to new platforms and frameworks.
  • Broader Industry Adoption: By embedding accessibility as a baseline requirement, AIMAC paves the way for more inclusive digital experiences and strengthens AI ethics and compliance across sectors.

For developers, researchers, and enterprise teams interested in driving inclusive AI innovation, AIMAC is available now on GitHub.

About the Author
Michelle Hawley

Michelle Hawley is an experienced journalist who specializes in reporting on the impact of technology on society. As editorial director at Simpler Media Group, she oversees the day-to-day operations of VKTR, covering the world of enterprise AI and managing a network of contributing writers. She's also the host of CMSWire's CMO Circle and co-host of CMSWire's CX Decoded. With an MFA in creative writing and background in both news and marketing, she offers unique insights on the topics of tech disruption, corporate responsibility, changing AI legislation and more. She currently resides in Pennsylvania with her husband and two dogs. Connect with Michelle Hawley:

Main image: Phanithi on Adobe Stock
Featured Research