Mattias Winnberg’s PhD thesis: What are we really measuring when students use spreadsheets in mathematics tests?

What is your thesis mainly about?

“My thesis, Students’ mathematical digital competence in assessment: Validity considerations in the instrumental use of spreadsheet tools, examines what needs to be considered when digital tools are used in mathematics tests, such as national exams and international studies like PISA (Programme for International Student Assessment) where students’ knowledge is compared globally.

A central question is what is being measured: students’ mathematical knowledge, their ability to use the digital tool, or a combination of both. If students are not sufficiently familiar with the tool, there is a risk that results are influenced more by technical difficulties than by their mathematical competence.

In the thesis, I therefore investigate how digital tools—particularly spreadsheets—are used in mathematics tasks and what this means for how we interpret test results.”

Why did you choose to focus on this topic?

“When I worked as a teacher, I noticed something that often surprised me. Many students were used to employ computers in everyday life, but when they were asked to use digital tools in mathematics, it suddenly became much more difficult. That made me start wondering: why is that? How are digital tools actually used in mathematics, both in teaching and in tests? And could it be done in a better way?

My curiosity deepened further a few years later when I worked with the PISA study and contributed to developing digital mathematics tasks. That gave me insight into how such tasks are constructed and how important their design is. In international studies, where students’ knowledge is compared globally, it is particularly important that tasks function fairly and provide equal conditions for all students. I therefore wanted to better understand the difficulties students may encounter and how these affect their results.”

Can you say something about your findings?

“One important finding is that we cannot assume students to know how to use digital tools, not even for simple functions. In one of my studies, students solved mathematical problems using spreadsheets. Even though they had used similar programs before, many struggled with technical aspects, such as selecting cells or entering formulas. At the same time, they were often able to explain how they intended to solve the problem mathematically. In other words, they knew what they wanted to do but had difficulties doing it in the tool itself.

Another finding concerns how students are prepared to use digital tools in PISA. Training mainly focuses on technical procedures—following steps, clicking in the right places, and entering the correct inputs. Students receive less practice in understanding how the tools relate to the mathematics—why they are doing what they are doing and what it actually means. This can result in test outcomes that do not fully reflect the mathematical knowledge that the test is intended to measure.”

What was the most enjoyable part of your study?

“One of the most rewarding aspects of working on the thesis has been meeting other researchers and PhD students, both in Sweden and internationally. I have also appreciated the variety in the work. In some studies, I met and interviewed students, while in others I analyzed large international datasets. The combination of close interaction with students and work with extensive data has made the research both exciting and enriching.”

What do you hope your thesis will contribute?

“I hope my research can contribute to a better understanding of both the opportunities and challenges of using digital tools in assessment. Today, such tools are increasingly used in tests, for example, in international studies like PISA. It is therefore important that students are given opportunities to practice using these tools in meaningful ways already in their regular instruction.”

What are your plans going forward? Has your thesis influenced you in any way?

“I would like to continue researching digital tools in national tests and in international assessments. Working on the thesis has really opened my eyes to how much interesting research can be done using large-scale educational datasets. I will also continue teaching in teacher education, where I focus on quantitative research methods.”

Download and read Mattias’s thesis

Students’ mathematical digital competence in assessment: Validity considerations in the instrumental use of spreadsheet tools