Research
Independent analysis of public assessment data, with the measurement work shown rather than assumed.
Most published comparisons of educational and psychological data rest on an assumption that is almost never tested: that the thing being measured means the same thing in every group being compared. When that assumption fails, the comparison is not merely imprecise. It is measuring different constructs in different groups and reporting the gap between them as though it were a difference in level. The research published here tests that assumption before making the comparison, and reports the answer even when the answer is that the comparison cannot be made.
That is the whole editorial position. It is also why this page sits apart from the Center’s services: what follows is published free, in full, and is not an advertisement for anything.
Published series
PISA 2022 country notes
Nine independent analyses of public PISA 2022 microdata, one per world region. Each note leads with the question that is peculiar to its own country rather than applying a template: Latvia’s language of instruction, Germany’s tracking, Brazil’s regional gap, the seven parallel curriculum systems inside the United Arab Emirates. Two of the nine report that the comparison they tested is not supported, and withhold the figures rather than print them with a caveat.
Further series are added here as they are published.
How this research is done
Four commitments hold across everything on this page, and they are the reason these analyses can be checked rather than merely believed.
Uncertainty is reported in full
Where achievement is imputed rather than measured, estimates are pooled across all plausible values by Rubin’s rules, and sampling variance is computed from the replicate weights the data provider ships for the purpose. Omitting either understates uncertainty, usually by a factor of two or more, and a great deal of published commentary does omit them.
Known figures are reproduced before new ones are presented
Every analysis of a public dataset begins by regenerating statistics the data provider has already published for that population. If those do not come out right, nothing downstream is trustworthy, and a reader should not have to take that on faith.
Comparability is tested, not assumed
Any questionnaire construct compared across groups is put through a staged configural, metric, scalar and strict invariance cascade first, and the verdict is printed whether or not it is convenient.
Null and negative results are published
An analysis concluding that the intended comparison cannot be supported is published as readily as one that produces a finding. This is the commitment that costs something, and it is the one that separates research from marketing.
Areas of work
The research programme runs along five lines. Pages are added under each as work is published.
Measurement equivalence and cross-cultural comparability
Whether an instrument measures the same construct across languages, countries and groups, which is the precondition for every cross-national comparison rather than a refinement of one.
Large-scale assessment
Secondary analysis of international assessment microdata, including PISA, PIAAC and TIMSS, at national and subnational level.
Social and emotional skills
Measurement and policy analysis of the non-cognitive domain, where instrument quality varies far more than the literature admits.
Instrument construction, validation and norming
Scale development from construct blueprint through pilot to normative reference data.
Survey methodology
Questionnaire design, response processes, and the planned-missingness designs that modern large-scale surveys increasingly rely on.
The published record
The peer-reviewed work sits with the journals. The full list is on ORCID, Google Scholar and ResearchGate, covering twenty years of international assessment work with the OECD, UNESCO and Eurofound.
Commissioning this work for your organisation
The analyses on this page are published free. The equivalent work on your own data, at a fixed scope and a fixed price, is consulting delivered by AdriaMont Consulting DOO. Write to milos@adriamont.me.
If what you need is instrument design, validation, or psychometric support for a study you are running, that is the Center’s own work and it is described under services.
About
Dr Milos Kankaras is a psychometrician and policy analyst with more than twenty years of international large-scale assessment work, including with the OECD, UNESCO and Eurofound. His published specialism is measurement equivalence and cross-cultural comparability. He holds a PhD in social sciences from Tilburg University.
Published by the Center for Psychology, Podgorica. The Center for Psychology and the AdriaMont Institute are both operated by AdriaMont Consulting DOO, Cetinjski put 36, 81100 Podgorica, Montenegro. Commissioning enquiries go to milos@adriamont.me.
Dr Milos Kankaras | milos@centerforpsychology.me | miloskankaras.com | centerforpsychology.me | ORCID | LinkedIn