Process data refers to data obtained from respondents interacting with a computer-based assessment item. It is recorded in the computer log files and often comes in the form of sequences of events with time stamps.
For instance, in the Programme for International Student Assessment (PISA) in 2012, students were assessed on problem-solving competence via a set of interactive test items. The test is designed to measure individuals’ capacity to engage in cognitive processing to understand and resolve problem situations where a method of solution is not immediately obvious (OECD, 2013). Through exploring the problem situation and interacting with the computer-based environment, students can uncover pieces of information that would be useful for solving the problem in hand. In addition to their explicit responses to the test questions, students’ behavioral data (e.g., the log of their interactions with the scenario), are also recorded to make sense of their progress and strategies towards solving the problems. For the PISA problem-solving items, a student’s explicit responses to each question are the product data, and the series of clicks and entries incurred during his or her interactions with the questions, as well as the time stamps associated with each action, are the process data.
Below is an example of process data collected from PISA 2012 ticket items. More examples can be found here.
On the Ticket item, students are prompted to interact with an automated ticket machine in a train station. A simulated touch-screen ticket machine is presented to the students, where they can click on different buttons to select which type of ticket to purchase (i.e., full or concession, train or subway, daily or fixed trips, and number of rides), view the prices for the selected ticket type, and make or cancel the purchase. They are also given text descriptions of the machine and different types of tickets. As the task for this problem, students are given a scenario, and based on the scenario, they need to find and purchase a ticket on the machine with the lowest price. The video below provides an demonstration of the Ticket problem, as well as the process data collected as the student explores different options on the ticket machine. The contents on the left in the video are simplified version of process data.
Source: OECD released PISA 2012 items