ASSET is a skill-based test that measures students’ conceptual understanding and benchmarks the performance of schools nationally with violating insights and reports. Students get feedback on their areas of weakness, and teachers on concepts their students are not understanding. ASSET is offered in English, Mathematics and Science (for classes 3-10), Social Studies (for classes 5-9) and Hindi (for classes 4-8). The elapsing of each ASSET test is between 45 and 70 minutes.
ASSET is offered in online and pen-and-paper modes, predominantly in India and the UAE. While the test is taken entirely online in the UAE, most schools in India opt for the pen-and-paper version of the test.[1] The pen-and-paper version of ASSET is an entirely multiple-choice test.
How can we know if a particular ASSET test paper is too long? (In the online version of the test, it is easy to determine the time taken by students for any question or if students faced time difficulty in completing the test. But how can this be unswayable for the pen-and-paper test?[2])
We should note, at this point, that ASSET is a low-stakes test. This ways that its purpose is to provide feedback, and there are no significant consequences – either to a student’s performance in school or any other reward or punishment – to a student’s performance in ASSET. Further, since all the students of the same matriculation receive the same paper in a subject, small variations in paper difficulty or paper length do not matter much as every student faces the same wholesomeness or disadvantage. Yet, it is unchangingly useful to analyse aspects like paper length, as this can help modernize future papers and moreover help understand student thinking better.
Skip or missed questions: Every question has four options – A, B, C and D of which only one is correct. If a student marks increasingly than one option, that is invalid (and the student is marked wrong). If a student does not select any option for a question, that is a ‘missed question’ or a ‘skip’, as we should refer to it in this article.
How much do students skip questions? In the 21 years that ASSET has been conducted, students have skipped questions 1.73% of the time! But the pattern in these skips is both interesting and revealing. For example, the table unelevated shows that while questions are skipped 5.9% in matriculation 3 English paper, they are skipped only 0.6% of the time in matriculation 6 Social Studies paper.
Why do students skip questions? We can try and hypothesise why students skip questions. Clearly, if students do not reach a question – either considering they somehow ‘miss’ it or run out of time surpassing reaching a question, they will skip it. They may skip a question that they find difficult, troublemaking or tedious. Note that these 3 terms have specific meanings – ‘difficult’ ways that the percentage of students answering correctly is low; ‘confusing’ ways that the students are not worldly-wise to select between the options (due to which the percentage of students choosing the variegated options is similar) and ‘tedious’ could refer to questions that are either too long or have procedures that may be lengthy or mechanical. In this article, we shall see if the first two relate to skipping.
Interesting patterns were observed related to students skipping questions:
1. Question Skipping is a Paper Characteristic: Data shows that if a paper is given to students in variegated batches or to students in variegated regions, the pattern of skipping, surprisingly, is very similar. As an example, see the graphs below, which show the same ASSET paper given to students in India and the Middle East. Though the percentage of students unquestionably skipping questions is different, the shape of the graph is scrutinizingly identical. This ‘skipping graph’ is scrutinizingly like a fingerprint uniquely identifying a paper.
2. Younger children tend to skip questions more: We once got an indication of this from Icon 1. However, it is interesting to trammels the percentage of students who skip a large number of questions. The table unelevated shows the percentage of children who do not wordplay 40% of the paper or more, and the trends are very similar to what we see in Icon 1.
3. Skipping is linked to the subject: The language papers like English and Hindi have the highest extent of skipping, and Social Studies has the least. Mathematics and Science fall in between. We cannot conclusively explain the reason for this trend, though the trend itself is very clear. A multiple regression run of the variegated factors found that without the matriculation level, subject was most predictive of the extent of skipping.
4. Difficult questions or plane troublemaking questions, are not skipped much more: It seems likely that if a question is difficult or confusing, students would skip such a question more. But surprisingly, this turns out not to be true.
5. The strongest predictor of skipping is the position of a question in a test: The final finding is moreover a surprise. The weightier predictor of the extent of skip seems to be not a foible of the question at all, but merely its position in the question paper! Wideness all papers, skip percentages increase scrutinizingly continuously from question 1 till the last question. In some papers, the rate of increase is low (Figure 5) while it is higher in others (Figure 1). In the icon below, the untried line represents the stereotype slope of the increase.
Determining paper lengthiness from question skip data:
The previous wringer shows that every paper seems to have a ‘natural slope’ and a ‘natural question skipping rate’. Plane wideness over 20 or 30 papers, which typically represent students over 10-15 years, the skipping pattern of English Matriculation 3 papers remains very steady and never resembles the skipping pattern of, say, a Science Matriculation 7 paper.
Furthermore, the skipping rate seems to increase fairly steadily, and in specimen a paper is too long, we can expect an unusual increase towards the end of the paper. (We pinpoint the start of the paper as the first 75% of the questions and the ‘end’ as the remaining.)
In order to compare this quantitatively, we summate the stereotype of the 20 weightier papers, specified as those papers that had the lowest percentage of skips towards the end of the paper. The table shows unelevated compares the stereotype characteristics of the top 20 papers with the current paper. However, since all past papers are shown, long papers can hands be identified, as shown below.
Summary and Conclusion
The procedure described whilom can be used for any multiple-choice paper-and-pen test to provide an indication if the paper is too long. We used this technique with one of the past Central Teacher Eligibility Test (CTET) papers. The wringer suggested that plane though the paper had other issues, the paper was not too long for the time provided.
___________
[1] Though schools took the test online during Covid (students usually take the test from home), return to school was moreover marked by a return-to-paper for ASSET!
[2] The rest of this vendible focuses only on the paper-and-pen version of the ASSET test.
The post Using Student Skipped Responses to Detect if a Test Paper is too Long appeared first on EI blog.
