Chapter 4 – Absence of Bias

Educational Assessment - Review By Brenda Roof
Classroom Assessment – What Teachers Need to Know - W. James Popham

Chapter 4 discusses absence-of-bias, the last of the three essential criteria for evaluating educational assessments. Assessment bias is defined as qualities of any assessment instrument that offend or unfairly penalize a group of students because of the students’ gender, race, ethnicity, socioeconomic status, religion or other such group-defining characteristic. Assessment bias occurs when elements of an assessment distort a subgroups performance on the assessment. Bias review panels should be created to review especially high stakes tests. Students who are disabled or English language learners also experience assessment bias, due to the nature of their needs. An understanding of assessment bias is essential for classroom teachers to be aware of.
Assessment-bias occurs when elements of an assessment distort a student’s performance based on personal characteristics of a student. Assessment bias also interferes with test validity. If it distorts student performance then score based inferences can not be used accurately. There are two forms of assessment bias. The first form is offensiveness. Offensiveness occurs when negative stereotypes of certain sub groups are presented in an assessment. Offensiveness can also act as a distracter to these students taking focus off the question and causing a student to respond incorrectly or not fully. The second form of assessment bias is unfair penalization. Unfair penalization occurs when a student’s test performance is distorted due to content that may not be offensive but disadvantages a student’s subgroup. An example of this could be questions aimed at a strong knowledge of the game of football. Girls may tend to not be as familiar with terms used and therefore not do as well on these types of questions. Unfair penalization happens when it is not the students’ ability that leads to poor performance but the students’ characteristics or subgroup membership.
Disparate impact however, does not indicate assessment bias. If disparate impact has an effect on members of a certain ethnicity, gender or religious group it should be scrutinized for bias, but often it is an educational factor that needs to be addressed. Content review of test items is essential, especially for high stakes tests. These types of reviews can help show disparate impact as well as assessment bias. One way to do this is bias review panels. Bias review panels should consist of experts in the subject being reviewed as well as individual exclusively from the subgroup being adversely impacted or potentially adversely impacted. Male and females should also be equally represented. Once the panel is formed assessment bias needs to be clearly defined and explained to the panel. The purpose of the assessment also needs to be clearly defined and explained. The next step for the bias review panel would be a per-item absence-of-bias judgment. A question should be developed that the review panel will ask of each item as they read through the assessment items. They should answer yes or no to the item question. Once the items are tallied the percentage of no judgments per item are calculated. An overall of per item absence-of-bias index can be computed for each item, and then the entire test. In addition to this type of scoring, when a panelist feels a question warrants a “no”, an explanation is also provided in writing. Often an item is discarded on this written basis. Individual item bias review should then be followed by overall absence-of-bias review. A overall question should be created and asked of the whole assessment. The same scoring process is applied and items can be modified to be corrected rather quickly.
Whole bias detection for high stakes tests can be detected through bias review panels as well. However, this is not practical for classroom assessments. Bias detection in the classroom can be prevented by becoming sensitive to the existence of assessment bias and the need to eliminate it. Teachers’ should think seriously about the backgrounds and experiences of the various students in their class. Then try to review each item on every assessment as if the item might offend or unfairly penalize any students. If any items may show bias eliminate them. It may also be helpful if you are unsure and have access to other teachers with background or knowledge of a particular sub group, to have them review your tests or the item you are unsure of.
Assessing students with disabilities can also lead to assessment bias. Federal laws have forced regular teachers to look at how they teach and assess students with disabilities. In 1975 Public Law 94-142 was enacted for states that properly educated students with disabilities to receive federal funds. This law also established the use of “IEP’s” or Individual Education Plans for students with disabilities. IEP’s are federally prescribed documents created by parents, teachers, and specialized service providers about how a child with disabilities should be educated. In 1997 this act was reauthorized and renamed Individuals with disabilities act (IDEA). This reauthorization required state and districts to identify curricular expectations for special education students similar to expectations of all students. These students were also required to be included in assessment programs reported to the public. In January 2002, No Child Left Behind was enacted to provide consequences for states and districts not complying to force them to have to comply or face consequences. This act also intended to improve achievement levels of all students. These improvements have to be demonstrated on State chosen tests linked to the states curricular aims. Over twelve years all schools must meet adequate yearly progress (AYP), which increases in score levels from year to year. No Child Left Behind also requires sub-groups of students to meet AYP targets based on a number chosen by each state. IEP’s are used today to help students meet content standards pursued by all students.
Assessment accommodations are used to address the issue of assessment-bias for disabled students. Accommodations are procedures or practice that allows students with disabilities to have equitable access to instruction and assessment. Assessments do not lower expectations for these students but rather provide a student with a setting similar to the one used during instruction. The accommodations can not alter the nature of the skills or knowledge being assessed. There are four typical accommodation categories; presentation, response setting and timing and scheduling accommodations. Students should be a part of choosing the accommodations that best work for them to ensure their proper use.
English language learners are another diverse group. This group consists of; students whose first language is not English and know little if any, students who are beginning to learn English but could benefit from school instruction, and students who are proficient in English but need additional assistance in academic or social contexts. Another sub group is limited English Proficient (LEP) students who are students who use a language other than English proficiently at home. If students are going to be assessed fairly that are considered ELL or ESL then classifications need to be made consistent within and across states. There are also sparse populations in areas and statistically this needs to be accounted for. There is also instability over time of these students. Mobility is a very common and therefore instruction instability can also be a contributing factor. Lower base-line and cut scores should be considered as language based subjects tend to be more difficult. The movement to isolate these sub-groups makes sense; however, how the scores are analyzed needs to be taken into consideration for students who have a poor understanding of English. When taking a test in English, preventing test bias for these students should be a consideration.
Classroom teachers need to have awareness for assessment bias. They also need to use measures to prevent bias on their assessments whenever possible. On high stakes tests test review panels need to be formed for item analysis as well as assessment analysis. Students with disabilities should have IEP’s written to help accommodate their testing needs for equity during assessments. Students whose first language is not English should be assessed in a subgroup and not penalized for low scores on English written tests.