Computer adaptive tests (CAT) are assessments that are tailored to the individual proficiency of each student being administered the test. This usually results in a testing experience that is neither too easy nor too difficult for students.
The mathematics and reading Standards of Learning (SOL) tests for students in grades 3-8 utilize computer adaptive tests.
A Computer Adaptive Test (CAT) is an assessment that is customized for every student based on how the student responds to the questions. This is in contrast to the traditional test in which all students who take a particular version of the test respond to the same test questions.
Computer Adaptive Testing is similar to traditional testing in many ways, but there are also some differences. This table highlights some of the characteristics that a Mathematics SOL CAT and a traditional SOL test have in common and some of the characteristics that are different. Explanations are included for clarity.
| Test Characteristics | Same | Different | Explanation |
|---|---|---|---|
| Content assessed | X | Each test format assesses the Standards of Learning (SOL) for a particular course. | |
| Question types/format | X | Each test format has multiple-choice questions and technology-enhanced items (TEI). The number of TEI can range from 10% to 30% as the test adapts to the performance level of the individual student. | |
| Number of test questions | X | For a particular course, the number of test items on a CAT may be different than the number of items on a traditional SOL test. However, all students taking a CAT for a particular course are administered the same number of items. As well, all students taking a traditional form of the test are administered the same number of items. See the SOL Test Blueprint for each mathematics test on the Virginia Department of Education website for additional information. | |
| Test questions presented to each student | X | A CAT is customized for each student. In a traditional test, all students administered the same test form are presented with the same questions. | |
| Presentation of content | X | Mathematics test questions are generally presented in order of reporting category. Each test format continues to do this. | |
| Time to complete the test | X | SOL tests are untimed in both formats. Students should be afforded as much time as needed to complete the test, but are generally expected to complete the test in one day. School divisions have the option of administering the grades 3, 4, and 5 SOL Mathematics tests over two days; however, school divisions are strongly encouraged to consider the changes that have been implemented in the online elementary school SOL tests when finalizing their test administration plans. | |
| Allowable test manipulatives | X | Each test format allows students access to the same test manipulatives such as a calculator or scratch paper. The SOL Test Examiner's Manual contains a complete list for each SOL test | |
| Online tools within TestNav | X | Online tools are available in each format within the testing software such as the choice eliminator and highlighter. The SOL Test Examiner's Manual contains a complete list for each SOL test. Practice test items are available in the TestNav application to provide an introduction to online navigation and online tools. | |
| Navigation through the test | X | In a mathematics SOL CAT, questions must be answered in the order they are presented. Each Question must be answered Test Characteristics Same Different Explanation completely before the student can move on to the next question. A student cannot skip any questions, nor return to a previously answered question. This is in contrast to the traditional test, where students can answer questions in a section of the test in any order. For specific CAT navigation instructions, refer to the Introduction to TestNav 8: Multiple-Choice/Technology-Enhanced Item Tests document on the Virginia Department of Education website. | |
| Special test accommodations | X | The special test accommodations provided for each test format are the same, but the manner in which some test accommodations are delivered to students may vary for a CAT. The SOL Test Examiner's Manual provides instructions. | |
| Scaled scores | X | Scaled scores remain on a scale of 0 to 600, with 400 indicating pass/proficient and 500 indicating pass/advanced for both test formats. Scaled scores for a CAT are computed using the number of questions answered correctly, as well as the difficulty level of the questions answered correctly. |
Passage-Based Reading Computer Adaptive Testing is similar to traditional testing in many ways, but there are also some differences. This table highlights some of the characteristics that a Passage-Based Reading SOL CAT and a traditional SOL test have in common and some of the characteristics that are different. Explanations are included for clarity.
| Test Characteristics | Same | Different | Explanation |
|---|---|---|---|
| Content assessed | X | Each test form assesses the Standards of Learning (SOL) for a particular course. | |
| Question types/format | X | Each test format has multiple-choice questions and technology-enhanced items (TEI). The number of TEI can range from 5% to 30% as the test adapts to the performance level of the individual student. | |
| Number of test questions | X | For a particular course, the number of test items and passages on a Passage-Based Reading CAT may be different from the number of items on a traditional Reading SOL test. However, all students taking a Passage-Based Reading CAT for a particular course are administered the same number of items. The Reading CAT for grades 3 through 8 will have four passages and item sets that contribute toward a student's score, and one field test passage and item set that is being tried out for future use. Each passage and item set for grades 3 and 4 includes a single passage and accompanying items. Students in grades 5 through 8 may receive a paired passage and accompanying items as one of the five total passage and item sets of the Reading CAT. All students taking a traditional form of the reading test are administered the same number of items for a particular course, and there are typically six to seven passages and item sets on the traditional test form. See the SOL Test Blueprints for each reading test on the Virginia Department of Education website for additional information. | |
| Test questions and passages presented to each student | X | A Passage-Based Reading CAT is customized for each student. In a traditional test, all students who take a particular version of the test respond to the same test questions. | |
| Time to complete the test | X | SOL tests are untimed in both formats. Students should be afforded as much time as needed to complete the test, but are generally expected to complete the test in one day. School divisions have the option of administering the grades 3, 4, and 5 SOL reading tests over two days; however, school divisions are strongly encouraged to consider the changes that have been implemented in the online elementary school SOL tests when finalizing their test administration plans. | |
| Allowable test manipulatives | X | Each test format allows students access to the same test manipulatives, such as scratch paper. The SOL Examiner's Manual contains a complete list for each SOL test. | |
| Online tools within TestNav 8 | X | Online tools are available in each format within the testing software, such as the choice eliminator and highlighter. The SOL Test Examiner's Manual contains a complete list for each SOL test. Practice test items are available to provide an introduction to online navigation and online tools. | |
| Navigation through the test | X | In a Passage-Based Reading SOL CAT, students must answer each question before proceeding to the next question, and they are not able to skip questions. Once a question is answered, the student is able to navigate back to that question as long as they are in the same passage set of items. Students are not able to return to other passages and test items within the test. This is in contrast to the traditional test, where students can answer questions in a section of the test in any order. For specific CAT navigation instructions, refer to the Introduction to TestNav 8: Multiple-Choice/Technology-Enhanced Item Tests document on the Virginia Department of Education website. | |
| Special test accommodations | X | The special test accommodations provided for each test format are the same, but the manner in which some test accommodations are delivered to students may vary for a Passage-Based Reading CAT. The SOL Test Examiner's Manual provides instructions. | |
| Scaled scores | X | Scaled scores remain on a scale of 0 to 600, with 400 indicating pass/proficient and 500 indicating pass/advanced for both test formats. Scaled scores for a CAT are computed using the number of questions answered correctly, as well as the difficulty level of the questions answered correctly. |
Computer adaptive testing is an assessment format that provides a customized assessment for every student. The computer selects questions and problems that are neither too difficult nor too easy based on the content knowledge demonstrated by each test taker.
When a student takes a computer adaptive test, answers are scored as each question or problem is completed, and the computer selects subsequent items based on the student's performance. Tests begin with a question or problem of moderate difficulty. If a student answers the item correctly, the computer selects a slightly more challenging question or problem as the next item. Conversely, an incorrect or incomplete response results in the selection of a slightly less difficult question or problem.
Scores are determined by the number of questions a student answers correctly and the relative difficulty of the correctly answered items. While the questions and problems on computer adaptive tests vary in difficulty from student to student, all students must answer the same number of items to complete a test. Because student responses on a computer adaptive test are scored in the sequence in which they are presented, students are not able to skip items or return to previous answers.
Yes. During 2013-14, approximately 3,000 students participated in a pilot of computer adaptive testing. Students took computer-adaptive versions of either the Grade 7 Mathematics SOL test or the Grade 8 Reading SOL test.
States employing computer adaptive testing in elementary and secondary schools report increased student engagement and fewer security risks.
No. Computer adaptive testing is a well-established, scientifically validated, and widely used assessment format. Computer adaptive testing is used by professional boards, the military, and other organizations to accurately measure knowledge and ability.
Passage-based computer adaptive testing is part of the Virginia Department of Education's (VDOE) continuing effort to improve the testing experience of students. The grades 3-8 Reading Standards of Learning (SOL) tests are administered in a passage-based computer adaptive format.
A passage-based SOL test is a test that presents a reading passage for the students to read, followed by a number of test questions about the passage. Additional passages and the related questions are presented throughout the test.
A passage-based computer adaptive reading test is an assessment format that provides a customized assessment for every student. The computer selects the passages and questions that are neither too difficult nor too easy based on the content knowledge demonstrated by each test taker. A passage and item set are selected from a collection or "pool" of passages and questions. The questions associated with a passage may have multiple variations of item difficulty. A student's performance on the entire set of questions for a passage will determine the overall level of difficulty for the set of items delivered to the student with the next passage.
Tests begin with a student being administered a passage and an associated set of items that have a medium level of difficulty. Each subsequent passage and set of items administered to a student is selected based on the student's performance on the previous set of questions. This is determined by the overall number of questions answered correctly by the student as well as the difficulty level of those questions. A student who answers more questions correctly in a passage set of items is more likely to get a slightly more difficult set of items in the next passage set. Conversely, if a student answers more questions incorrectly in the passage set, a slightly less difficult set of items will likely be administered in the next passage set.
In passage-based computer adaptive testing, students must answer each question before proceeding to the next question, and they are not able to skip questions. However, once a question is answered, the student is able to navigate back to previous questions within that same passage set of items to review or change an answer. Once the student proceeds to the next passage, the student can no longer go back to a previous passage and its associated questions.