What PISA measures. What we measure.

PISA is the international test that’s used to compare the effectiveness of education internationally.

What is being measured?

There are three sources of difficulty in the story. It’s Lectical phase is 10c — the third phase of four in level 10. Also, the story is challenging to interpret because it’s written to be a bit ambiguous. I had to read it twice in order to appreciate the subtlety of the author’s message. And it is set on the water in a rural setting, so there’s lots of language that would be new to many students. How well a student will comprehend this story hinges on their level of understanding — where they are currently performing on the Lectical Scale — and how much they know about living on the water in a rural setting. Assuming they understand the content of the story, comprehension also depends on how good students are at decoding its somewhat ambiguous message.

If you don’t know what I mean by Lectical Level, this video is for you!

Real world problems

Now, let’s consider what it means to present a real-world problem. When we at Lectica use this term, we usually mean that the problem is ill-structured (like the world), without a “correct” answer. (We don’t even talk about correctness.) The challenges we present to learners reveal the current level of their understandings — there is always room for growth. We think of development as a process of learning to make “better and better mistakes.” This is a VERY different mindset from the “right or wrong” mindset nurtured by conventional standardized tests.

How knowledge is applied

Let’s go back to the story about the woman and the panther. After they read the story, test-takers are asked to respond to a series of multiple choice and written response questions. In one written response question they are asked, “What does the story suggest was the woman’s reason for feeding the panther?”

  • She felt sorry for it.
  • Because she knew what it felt like to be hungry.
  • Because she’s a compassionate person.
  • To help it live. (p. 77)

What is reading comprehension?

All of these observations lead me back to the question, “What is reading comprehension?” PISA developers define reading comprehension in terms of understanding and interpretation, and Lectical assessments measure the sophistication of students’ understanding and interpretation. It looks like our definitions are at least very similar.

  • First, we have to ask students to show us how they’re thinking. This means asking for verbal responses that include both judgments and justifications for those judgments.
  • Second, the questions we ask need to be more open-ended. Life is rarely about finding right answers. It’s about finding increasingly adequate answers. We need to prepare students for that reality.
  • Third, we need to engage in the careful, painstaking study of how students construct meanings over time.

--

--

Award-winning educator, scholar, & consultant, Dr. Theo Dawson, discusses a wide range of topics related to learning and development.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Theo Dawson

Award-winning educator, scholar, & consultant, Dr. Theo Dawson, discusses a wide range of topics related to learning and development.