Providing Evidence for the Reliability and Validity of Performance Assessment
The technical qualities and scoring procedures of performance assessments must meet high standards for reliability and validity. To ensure that sufficient evidence exists for a measure, the following four issues should be addressed:
- 1. Assessment as a Curriculum Event. Externally mandated assessments that bear little, if any, resemblance to subject area domain and pedagogy cannot provide a valid or reliable indication of what a student knows and is able to do. The assessment should reflect what is taught and how it is taught.
- The content of the instrument.
- The length of activities required to complete the assessment.
- The type of activities required to complete the assessment.
- The number of items in the assessment instrument.
- The scoring rubric.
- The scoring criteria are well articulated.
- Teachers are comfortable with having students share in their own evaluation process.
- Statistical Moderation. This approach is used to compare performances across content areas for groups of students who have taken a test at the same point in time.
- Social Moderation. This is a judgmental approach that is built on consensus of raters. The comparability of scores assigned depends substantially on the development of consensus among professionals.
Making an assessment a curriculum event means reconceptualizing it as a series of theoretically and practically coherent learning activities that are structured in such a way that they lead to a single predetermined end. When planning for assessment as a curriculum event, the following factors should be considered:
2. Task Content Alignment with Curriculum. Content alignment between what is tested and what is taught is essential. What is taught should be linked to valued outcomes for students in the district.
3. Scoring and Subsequent Communications with Consumers. In large-scale assessment systems, the scoring and interpretation of performance assessment instruments is akin to a criterion-referenced approach to testing. A student's performance is evaluated by a trained rater who compares the student's responses to multitrait descriptions of performances and then gives the student a single number corresponding to the description that best characterizes the performance. Students are compared directly to scoring criteria and only indirectly to each other.
In the classroom, every student needs feedback when the purpose of performance assessment is diagnosis and monitoring of student progress. Students can be shown how to assess their own performances when:
4. Linking and Comparing Results Over Time. Linking is a generic term that includes a variety of approaches to making results of one assessment comparable to those of another. Two appropriate and manageable approaches to linking in performance assessment include:
How Can Teachers Influence Students' Performances?
Performance assessment is a promising method that is achievable in the classroom. In classrooms, teachers can use data gathered from performance assessment to guide instruction. Performance assessment should interact with instruction that precedes and follows an assessment task.
When using performance assessments, students' performances can be positively influenced by:
- Selecting assessment tasks that are clearly aligned or connected to what has been taught.
- Sharing the scoring criteria for the assessment task with students prior to working on the task.
- Providing students with clear statements of standards and/or several models of acceptable performances before they attempt a task.
- Encouraging students to complete self-assessments of their performances.
- Interpreting students' performances by comparing them to standards that are developmentally appropriate, as well as to other students' performances.
References
Baker, E. L., O'Neill, H. F., Jr., & Linn, R. L. (1993). Policy and validity prospects for performance-based assessments. American Psychologist, 48, 1210-1218.
U.S. Congress, Office of Technology Assessment. (1992, February). Testing in American schools: Asking the right questions. (OTA-SET-519). Washington, DC: U.S. Government Printing Office.
Derived from: Elliot, S. N. (1994). Creating Meaningful Performance Assessments: Fundamental Concepts. Reston, VA: The Council for Exceptional Children. Product #P5059.
ERIC Digest E531
ERIC Identifier: ED381985
Publication Date: 1995-06-00 Author: Elliott, Stephen N. Source: ERIC Clearinghouse on Disabilities and Gifted Education, Reston, VA
This publication was prepared with funding from the National Library of Education (NLE), Office of Educational Research and Improvement, U.S. Department of Education, under contract no. RR93002005. The opinions expressed in this report do not necessarily reflect the positions or policies of NLE, OERI, or the Department of Education.
Authentic Assessment Overview
Portfolio Assessment
Rubrics: An Overview
