From the archives: About measurement
Originally posted on July 29, 2009
This year my colleagues and I are celebrating the achievement of a 25-year effort to create a fully-automated developmental assessment system for texts. The system is deeply grounded in Piagetian and neo-Piagetian theory and research along with a great deal of study related to testing and measurement. During the first 10 years of my work developing Lectical Assessments, I spent a fair bit of time explaining how testing works. Gradually, I came to understand that the science behind our assessments was less important to clients than their practical benefits, so I shifted my focus away from academics toward practical application. Recently, a question about testing from a colleague sent me on a search through the archives of my original blog. Realizing that these articles contained important information that might be useful to at least a few of my readers, I decided to rescue some of them. In this article, I attempt to illustrate how measurements evolve and advance science. It’s particularly pertinent as we celebrate our own measurement milestone.
The story of how measurement permits scientific advance can be illustrated through any number of examples. One such example involves the evolution of the measurement of temperature and its effects on our understanding of the molecular structure of elemental substances like lead.
The tale begins with an assortment of semi-mythical early scientists, who agreed in their observations that lead only melts when it is very hot — much hotter than the temperature at which ice melts, and quite a bit cooler than the temperature at which iron melts. These observations, made repeatedly, resulted in the hypothesis that lead melts at a particular temperature.
To test this theory it was necessary to develop a standard for measuring temperature. A variety of early thermometers were developed and implemented. Partly because these early temperature-measuring devices were poorly calibrated, and partly because different temperature-measuring devices employed different scales, the temperature at which lead melted seemed to vary from device to device and context to context.
Scientists divided into a number of ‘camps.’ One group argued that there were multiple pathways toward melting, which explained why the melting seemed to occur at different temperatures. Another group argued that the melting of lead could not be understood apart from the context in which the melting occurs. Only when a measure of temperature had been adequately developed and widely accepted did it become possible to observe that lead consistently melts at about 327º C.
Armed with this knowledge, scientists asked what it is about lead that causes it to melt at this particular temperature. They then developed hypotheses about the factors contributing to this phenomenon, observing that changes in altitude or air pressure seemed to result in small differences in its melting temperature. So, context did seem to play a role! In order to observe these differences more accurately, the measurement of temperature was further refined. The resulting observations provided information that ultimately contributed to an understanding of lead’s and other elements’ molecular structure.
While parts of this story are fictional, it’s true that the thermometer has greatly contributed to our understanding of the properties of a variety of substances. Interestingly, the thermometer, like all other measures, emerged from what were originally qualitative observations about the effects of different amounts of heat that were quantified over time. The value of the thermometer, as we all know, extends far beyond its use as a measure of the melting temperature of lead. The thermometer is a measure of temperature in general, meaning that it can be employed to measure temperature in an almost limitless range of substances and contexts. It is this generality, in the end, that makes it possible to investigate the impact of context on the melting temperature of a substance, or to compare the relative melting temperatures of a range of elemental substances. This generality (or context-independence) is one of the primary features of a good measure.
Good measurement requires (1) the identification of a unidimensional, content and context-independent trait (temperature, length, time); (2) a system for assessing the amount of the trait; (3) determinations of the reliability and validity of the measurements; and finally (4) the calibration of a measure. A good thermometer has all of the qualities of a good measure. It is a well-calibrated instrument that can be employed to accurately and reliably measure a general, unidimensional trait across a wide range of contexts.
It was this perspective on measurement that first inspired me to try to develop a good general measure of the developmental dimension (hierarchical complexity). To read more about how this way of thinking relates to the Lectical Assessment System (LAS), read About Measurement on the LecticaLive site. Pay special attention to the list of things we can do with the LAS.
Related articles
