March 13th, 2024
By Emi Iwatani, Thao Vo, Jennifer Randall, Janine Jackson, Drew Gitomer, Shelton Daal, Darius Taylor
This article is also published at Digital Promise. To read their blog post, navigate here.
Key Ideas:
- Though learning occurs within a sociocultural context, the assessment field has historically faced challenges in preparing assessment professionals to address cultural responsiveness in their work.
- Challenges range from adequately training scholars on cultural responsiveness to the availability of mentors capable of guiding students with these interests.
- A group of assessment professionals from diverse backgrounds and institutions are collaborating to address these challenges and train a new generation of assessment professionals.
Historically, large-scale assessments such as K-12 state examinations have claimed to provide teachers, students, and parents with objective measures of what students know and can do, without considering possible impacts of culture. More recently, however, some assessment professionals have pointed out that all learning—and consequently assessment—happens within a sociocultural context, and that context should be leveraged to improve both student learning outcomes and the assessment of those outcomes.
If culture is relevant to learning and assessment (as we and many others believe), then training a new generation of assessment professionals who prioritize cultural responsiveness is critical. In a recent gathering that we organized through Digital Promise, the Center for Culturally Responsive Evaluation and Assessment, and the Center for Measurement Justice, a group of approximately 30 early-career and experienced assessment professionals who aspire to be culturally responsive surfaced challenges and created action plans.
Early-career Scholars’ Aspirations and Current Realities
Many scholars interested in culturally responsive assessment strongly desire to positively impact marginalized communities and cultures through their work. In our convening, early-career assessment scholars, including graduate students, shared aspirations such as:
- Including student and teacher voices throughout the entire assessment process, including development
- Creating new forms of placement exams that advance learning and reduce stereotype bias
- Developing curricula that are responsive to the needs of women of color, including necessary skills and specific challenges they may face in the assessment field
- Making assessments more accurate and meaningful for all by broadening participation in the assessment field
Historically, research and careers in educational assessment have focused on the technical aspects of test design, particularly mathematical modeling, and more recently, integrating artificial intelligence into tests. Unfortunately, this poses challenges for aspiring scholars who want to raise different kinds of questions that address culturally responsive goals. The convening identified two challenges related to instruction and mentoring.
Challenge 1: Graduate training rarely includes theory and practice of cultural responsiveness
Most required training for assessment professionals do not include instruction in cultural responsiveness:
- “Measurement curricula are too model-driven. It would be great to incorporate equity, fairness, and cultural responsiveness into the work and stop treating these as a mere requirement taught in other programs.” – Early-career scholar
- “Would love to see cultural responsiveness integrated fully into all coursework, not just ‘othered’ in a standalone course.” – Early-career scholar
Early-career scholars expressed a desire for their graduate training to extend beyond merely highlighting disparities related to educational assessment. They want practical strategies for addressing these issues. Several ideas included incorporating case-based reflective learning throughout coursework and offering specialized seminars exploring cultural influences and context in measurement.
Many experienced scholars emphasized the importance of supporting faculty in teaching topics related to education equity and social justice. It’s not enough to have resources on individual topics; there is a need for comprehensive guidance to help faculty and departments restructure their curriculum cohesively. While there are some articles and book chapters on course restructuring, there is a lack of formal, comprehensive guidance such as collected volumes, discussion guides, and sample lessons and syllabi, which are crucial for system-wide improvement.
Additionally, scholars of color highlighted the importance of graduate coursework in empowering early-career scholars from minoritized groups by teaching them about the social and political dynamics within the profession. This knowledge can help assessment scholars navigate field-specific challenges and leverage their strengths to contribute meaningfully to assessment improvement efforts.
Challenge 2: Uncertainty around mentorship availability for scholars who aspire to be culturally responsive
Early-career and experienced scholars alike mentioned how mentoring is invaluable and essential for supporting assessment professionals to center cultural responsiveness. Yet the challenges abound regarding the availability and sustainability of quality mentorship in this area:
- “Almost everything I’ve learned about these various issues has come from mentorship by people outside of the traditional measurement field—how do we create opportunities for students to have access to communities of mentors?” – Experienced scholar
- “[G]ood mentoring requires time. For historically minoritized students, perhaps, even more time. …Who has time to provide that level of mentorship? We don’t want it simply to be invisible, uncompensated labor—but then it becomes expensive to support.” – Experienced scholar
Assessment Professionals Are Collaborating Across Boundaries to Address These Challenges
While these are crucial challenges, we are grateful and encouraged by the action plans that the convening participants have created. Many involve collaboration across rank, institutions, and organizations, such as designing a mechanism for cross-institutional mentorship and developing training modules to broaden participation. Two action plans are located in Historically Black Colleges and Universities (HBCUs), and one is cross-national. Stay tuned for action plan spotlights in a blog post series curated by the Center for Measurement Justice.
Learn about the guidance that our silenced assessment ancestors and elders provide in our first blog post.
We are grateful to all our colleagues who engaged in the 2023 CREA Assessment Convening, and for funding for Digital Promise through a gift from MacKenzie Scott which sponsored the convening, including travel for emerging scholars. The convening was held as part of the Seventh International Culturally Responsive Evaluation and Assessment Conference and was hosted by the three institutions listed below.
Follow our journey through our organizational websites and mailing lists
- Digital Promise (https://digitalpromise.org/)
- Center for Measurement Justice (https://measurementjustice.org/)
- Center for Culturally Responsive Evaluation and Assessment (https://crea.education.illinois.edu/)
Author Bios
- Emi Iwatani is a Senior Learning Sciences Researcher at Digital Promise
- Thao Vo is a Doctoral Candidate studying Educational Psychology at Washington State University.
- Jennifer Randall is the Dunn Family Professor of Psychometrics and Test Development at the University of Michigan and the President of the Center for Measurement Justice.
- Janine Jackson is a Senior Data Analyst at the Center for Predictive Analytics and the current chair of the Graduate Student Issues Committee for the National Council on Measurement in Education.
- Drew Gitomer is the Rose and Nicholas DeMarzo Chair in Education at the Rutgers University Graduate School of Education and Affiliate Faculty at the Center for Culturally Responsive Evaluation and Assessment at University of Illinois Urbana-Champaign
- Shelton Daal is a Senior Education Research Associate at Digital Promise.
- Darius Taylor is the Director of Equity, Education Innovation & Research at the American Indian Higher Education Consortium.