Skip to content
Contact

AI vendor transparency index

AI vendor transparency index: methods

Scope, reference standards, the search protocol, the limitations and the corrections route for the AI vendor transparency index.

AI vendor transparency index: methods

Version 1. Rubric fixed and published before any entry was scored. Search protocol applied and all sources accessed 22 September 2026.

Scope

This assessment covers what each vendor publishes about the validity and fairness of its scores. It does not assess the tool, the model, the underlying science or the company.

A vendor whose instrument is strong and whose public documentation is limited will show few criteria as documented, and no assessment of that instrument is made or implied. The reverse also holds. Each cell records a quoted public source with its URL and access date, or records the level as Absent.

Reference standards

The eight criteria are derived from the sections of a bias and measurement-equivalence audit: identification, sample, grouping variables tested, invariance testing, items flagged for differential item functioning and their disposition, partial invariance, adverse impact, reliability by group, disposition and sign-off, limitations, and next audit.

Each criterion also maps to a technical-documentation item of Annex IV of the EU AI Act, and rests on a chapter of the Standards for Educational and Psychological Testing (AERA, APA and NCME). Chapters are cited by title.

Search protocol

Applied identically to each vendor.

  1. The vendor domain: science, research, validity, technical, trust, legal, compliance and resources sections, and the site search.
  2. Documents linked from those pages, including documents that require contact details. Where a document requires contact details it is recorded as existing, as requiring contact details, and as not read.
  3. Published bias-audit material, including any New York City Local Law 144 summary, which that law requires to be posted publicly.
  4. One general web search per vendor, on the vendor name with each of: technical manual, validity, adverse impact, bias audit, reliability.
  5. Sources behind a login, information obtained in a sales conversation or through personal contact, and descriptions by third parties are excluded.

Absent records the outcome of this search on 22 September 2026.

Limitations

  • The assessment covers seventeen vendors, those meeting the inclusion criteria published alongside it. The criteria and the decision on every candidate considered are stated in full, so the boundary of this edition can be checked rather than taken on trust.
  • Three cells rest on sources that could not be fully examined, and are marked provisional on the entry concerned.
  • Published pages change. Each cell carries the date on which it was read.
  • Absent is not a finding of non-compliance. The EU AI Act obligations for stand-alone high-risk systems apply from 2 December 2027. A vendor without United States exposure is not subject to Local Law 144.
  • No weighted total is calculated and entries are listed alphabetically.

Corrections

Where a source was not reached by the search, where a document has since been published, or where a cell misreports what a page states, the cell is re-scored.

Write to milos@centerforpsychology.me. Every correction is answered within ten working days. Anyone may write, including the vendors described here.

A correction that changes a level is applied to the entry and to the published data, and the entry states what changed and when. The previous level is retained. A correction that is not accepted receives a reasoned answer. Nothing is quietly amended.

Author

Dr Milos Kankaras. PhD in social sciences, Tilburg University, on measurement equivalence in cross-cultural survey research. Previously OECD, where he led the Study on Social and Emotional Skills across ten participating sites, and Cedefop, Eurofound and UNESCO.