Diagnostic Psychometrics

Introductions

W. Jake Thompson

Hex logo for the measr R package.

Conceptual foundations

  • Traditional assessments and psychometric models measure an overall skill or ability
  • Assume a continuous latent trait

A normal distribution with images of Taylor Swift from each era overlayed.

Traditional methods

  • The output is a weak ordering of teams due to error in estimates
    • Confident Red (Taylor’s Version) is the best
    • Not confident which is second best (Speak Now (Taylor’s Version), folklore, Lover)
  • Limited in the types of questions that can be answered.
    • Why is Taylor Swift (debut) so low?
    • What aspects do each album demonstrate mastery or competency of?
    • How much skill is “enough” to be competent?

Music example

  • Rather than measuring overall musical knowledge, we can break music down into set of skills or attributes
    • Songwriting
    • Production
    • Vocals
    • Cohesion

Four circles representing the 4 attributes. The bottom half of each circle is sharded dark, and the top half is light, to indicate there are two categories for each attribute.

  • Attributes are categorical, often dichotomous (e.g., proficient vs. non-proficient)

Diagnostic classification models

  • DCMs place individuals into groups according to proficiency of multiple attributes
songwriting production vocals cohesion
Xmark Xmark Xmark Check
Xmark Check Check Check