Chapter 6 – Selected Response Tests

Educational Assessment - Review By Brenda Roof
Classroom Assessment – What Teachers Need to Know - W. James Popham

Chapter six focuses on creating and evaluating appropriate selected response assessments. There are five general item-writing commandments for selected and constructed response items. The author also discusses strengths and weaknesses for four types of selected response assessments. Selected item responses if written properly can assess higher level thinking skills. A best practice is to have a content knowledge colleague review your assessments as well as using formulas that can be used to review your assessments.
The five general item-writing commandments are essential to remember when creating selected and constructed response tests. The first commandment is not to provide opaque directions to students regarding how to respond to your assessments. Teachers typically don’t put serious thought into their directions. When a student is introduced to a testing format that is unfamiliar, wordy directions can be a distracter and cause incorrect responses that are unintentional and impair the implications from the assessment. The second commandment is not to employ ambiguous statements for your assessment items. If your questions are unclear and students are unsure of what you mean the questions can be misinterpreted. Again, this could cause a wrong response, when the student knows the answer, but does not know what you are asking. The third commandment is not to provide unintentional clues, regarding the correct response. Students in this case, will come up with a correct response because they were lead to it from the wording of the items. The student may not really know the correct response and now attention to that item is not assessed properly and follow-up may not occur when it would have been beneficial to gain actual learning. An unintentional clue can be as simple as the use of a word like “an” and how it completes the sentence or answer. The fourth commandment is to not employ complex syntax in assessment items. The goal in this case is to use very simple sentences. Too many clauses mess up the flow of the test item and what it is asking. The fifth and final commandment is to not use vocabulary that is more advanced than required or understood by the student. To get a fix on the students status you need to assess what is taught and learned not introduce new material. The first type of selected response tests is binary-choice items. This form of test item is commonly seen as true-false. This form is probably one of the oldest forms as well. There are five guidelines for writing binary-choice items. The first is phrase items so that superficial analysis will lead to wrong answers. By doing this you are trying to get students to think about the test item and present a way for you to assess how much good thinking they can do. The second guidelines is rarely use negative statements, and never use double negatives. It can be tempting to use the word “not” in a true statement, but this will only confuse the question and should be avoided. The third guideline is to only include one concept in each statement. If a test item has a concept in the first part that is true and the second concept is given that is false it makes it difficult for the student to respond correctly. This also leads to false inferences about the students’ true learning. The fourth guideline is importance of balancing. Keeping equal number of true-false responses is important and should be easy to do. The fifth guideline is keep item length similar for both categories being assessed. This guideline like the fourth guideline encourages structure to avoid guessing. If students see two answers are worded longer and that becomes a common pattern, they will begin responding that way, and again you will not get a true assessment of learning.
The second type of selected response assessments is multiple binary-choice items. A multiple binary-choice item is, when a cluster of items is presented, which requires a binary response to each of the items in the cluster. These types of clusters look similar to multiple choice however; they are statement clusters that require a single response for each cluster. Two important guidelines should be used for multiple binary-choice items. The first is separate item clutters clearly from one another. Since students are more familiar with multiple choice, when using binary multiple choice cluster items together, to clearly identify what is clustered together. Use stars or bold each new cluster very clearly. The second guideline is, to make certain that each item fits well with the clusters stem. The part preceding the response is the stem, all items should be linked to the stem in a meaningful way. One large benefit of using multiple binary-choice is if the stem contains new material and the binary-choice depends on the new material, it is certain that the student will need to go beyond recall knowledge to answer the question. Therefore, more intellectually demanding thought will be required, than just an ability to memorize.
The third style of selected response is multiple choices. This form of testing has also been widely used for achievement testing. It is typically used to measure student’s possession of knowledge as well as their ability to engage in higher levels of thinking. There are five guidelines for multiple choice items that should be employed. The first is that the stem should consist of a self-contained question or problem. Therefore, it is important to put as much content in the stem as necessary, to understand what the question item is getting at. The second guideline is to avoid negatively stated stems. Again using “not” may only confuse the testing item. Sometimes, it is even overlooked, so if it must be used use italics or bold the word “not”. The third guideline for multiple choice items is, not to let the length of alternative responses supply unintended clues. Try to keep all responses the same length or at least two short and two long. Distracters should align with the correct response. The fourth guideline is to make sure you scatter your correct responses. If students notice a pattern in your answers they may respond to that instead of your assessment. A good rule of thumb is 25% of your answers should represent the correct answers if they are A B C or D, or however many answer’s you have evenly divided. The fifth guideline is a suggestion to never use “all-of-the-above”. However; you can use “none-of-the-above”. “None-of-the-above” can be used to increase item difficulty. The reason “all-of-the-above” is not a good idea, as the assessment taker may only look at the first response see it is correct and choose it without looking further. To increase an items level of difficulty using “none-of-the-above” will work for test based inferences such as math problem solving. If a problem is displayed and the student must properly solve the answer if they guess something close you will know they did not work the problem correctly.
The fourth selected response type is matching items. Matching items consist of two parallel lists of words or phrases, requiring students to match items with appropriate items on the second list. One side should be premises and the other responses. There are six guidelines to follow for well constructed matching assessments. The first is employing homogenous lists. Each side should be as close to equal as possible, otherwise matching should not be used. The second guideline is to use relatively brief lists and place shorter words or phases to the right. Use about ten or less premise statements or words to cut down on distracters from choosing the correct response. The third guideline would be to use more responses than premises. The use of more responses will decrease the ability to answer by process of elimination. The fourth guideline is to order the responses logically, to avoid unintended clues. If you order the responses logically or alphabetically, you can avoid giving unintended clues. The fifth guideline is to describe the basis for matching and the number of times a response can be used. Students need to clearly understand how they should respond accurately. The more accurate they respond, the more valid your test is for making score based inferences. The sixth and final guideline is to place all premises and responses for an item on one page. Page flipping only creates confusion and leads to wrong responses, as well as, distractions for other assessment takers.
This chapter addressed important rules for constructing selected and constructive response assessments. Also addressed were the four types of selected response tests commonly used. Guidelines for the use of each of these types of selected response assessments were also discussed. By practicing these concepts assessment based validity and reliability of selected response assessments can be increased. Assessment based inferences can also be employed appropriately.