June 4th, 2024
By Brian Leventhal, Sara Finney, Laura Lambert
Welcome to the fourth installment of our blog series on a collaborative effort organized with Digital Promise and the Center for Culturally Responsive Evaluation and Assessment (CREA). We hosted a Convening at the Seventh International CREA 2023 conference last fall that generated collaborative action plans to help prepare a new generation of assessment professionals who center cultural responsiveness in their work (see our first, second, and third posts.)
In this blog post, one of the action plan groups shares their progress in recruitment, faculty development, and curriculum to support culturally responsive assessment practice.
Overview of our Work
At James Madison University, our comprehensive action plan focuses on: 1) recruitment, 2) faculty development, and 3) curriculum. To support recruitment, we have fostered new and reinvigorated existing relationships with undergraduate institutions that educate diverse student bodies in addition to enhancing our undergraduate summer internship program. To support faculty development, we have partnered with two critical instructional designers to first determine the needs of our faculty and second, based on the needs assessment, create a curated year-long course to build skills to foster equity in our teaching and assessment-related practice. To support our curriculumt, during this professional development course, our faculty are being provided space, time, and resources to plan meaningful changes to our graduate program’s curriculum.
Centering equity in recruitment
Our recruitment efforts have centered on developing networks with faculty in order to have a more systematic approach to recruiting the next generation of measurement professionals with an equity mindset. For example, one of our faculty members recently traveled to Berea College (a work college that supports many low-income and international students) to reinvigorate relationships with faculty. In addition to speaking to students about our field, she also worked with faculty to curate strategies to support more sustained recruitment efforts.
Since 2018, JMU has offered an undergraduate research internship designed for rising seniors or juniors who are interested in graduate study in educational assessment, measurement, or statistics. This internship was expanded in the summer of 2021 to a national offering where interns are able to join the program in person or remotely. From 2021 to 2023, around 45 candidates applied for 2 or 3 internship positions.
The internship consists of 35 hours a week over 7 weeks. Interns learn the basics of educational assessment through an Assessment 101 course, meet with faculty and current students weekly, and complete a research project. The research project offers interns an opportunity to observe and engage in the process of identifying an assessment-related research question, complete an empirical study that is meaningful to them, and present their findings and implications to both faculty and graduate students in the field. Of the 13 interns who have completed the internship program, 7 have pursued and been admitted into graduate programs focused on measurement or quantitative methods, 3 have attended graduate school in other disciplines, and 3 have graduate school decisions still pending.
For 2024, we are planning new workshops on the history of statistics and measurement, frameworks for conducting research (e.g., intersectionality lens in educational measurement), as well as methods-based discussions of study design and the general linear model. Moreover, some of our current graduate students worked with faculty to create a rubric that centers equity in both educational programming and assessment in higher education. This rubric will be shared with interns to showcase the equity-related products produced by graduate students in our program and their impact on the field. Because of our work at CREA, we have developed new working relationships with the Center for Measurement Justice and Brian French (Washington State University – Pullman) to bolster the undergraduate internship. Each will be providing financial assistance to support one intern, which will bring our total to 4 interns for 2024. This pilot program will allow us to gain insights into how we can expand this financially supported program to more students in future years.
Overcoming barriers to faculty engagement
During the 2022-2023 academic year, we employed a very helpful Needs Assessment to identify our faculty’s confidence of learning and engaging in fairness, equity, and social justice in educational assessment and measurement. Using the results, we were able to curate a year-long professional development course to the specific needs of the unit, addressing areas where value was high, but confidence was low. Furthermore, we were able to leverage the results to address perceived and actual barriers to engaging in this critical work.
As a result of completing this course in 2023-2024, our faculty are able to 1) articulate our own positionality for the purpose of increasing self-awareness and confidence as we approach issues of equity and assessment in their teaching practice; 2) engage in peer-to-peer learning with colleagues to support a culture of trust and accountability; 3) increase confidence around teaching topics of equity in the field of assessment and measurement; and 4) integrate content around equity into the measurement, statistics, and research design curriculum.
Time and space for meaningful curriculum changes
During our year-long professional development course, our faculty engaged in conversations about their own individual action plans. There are three broad ways in which faculty plan to change the nature of their courses:
- Infuse topics related to equity throughout the curriculum
- Modify the way they teach topics already within the curriculum, and
- Use critical and equity-focused pedagogical techniques. For example, some faculty plan to implement a critical approach or intersectional approach to quantitative methods topics, contrasting these approaches with the post-positivist framing typically taught.
Alternatively, some faculty are planning to reframe their course using an Open approach or other approaches that center students. To be clear, not all faculty are implementing every approach in all of their courses. However, through continued open dialogue, our program is implementing strategies to make sure students learn through an equity-centered lens at all levels of their graduate studies. We have recently begun to strategize how we can develop a course on equity-focused and culturally responsive assessment practice that would be offered at multiple institutions. We believe there is a growing community of students spread out geographically that would benefit from having a cross-pollinated course.
Moving Forward
As we approach the summer, we are excited to wrap up our professional development course and draft our action plans. Our summer will be busy working with our undergraduate interns and continuing to transform our curriculum. Our faculty have a common understanding that it takes significant time to do this work well. Some faculty will institute changes this upcoming academic year, whereas others will take the following year to work through their changes. We face many challenges as we transform our program. We feel institutional uncertainty with new leadership. We feel political ambiguity from our state government. Yet, we feel pressure from students to begin this work quickly, which fuels us. And, we feel internal motivation. Moreover, we feel program unity. We are motivated by each other’s support, understanding that we will all approach this with varying degrees of success, and we are okay with that.
Step by step, ever forward!
Author Bios
Brian Leventhal is Director of the Assessment and Measurement Ph.D. program and the Psychological Sciences Masters program at James Madison University. [email protected]
Sara Finney is Professor of Graduate Psychology and Associate Director of the Center for Assessment and Research Studies at James Madison University. [email protected]
Laura Lambert is a doctoral student in the Assessment and Measurement Ph.D. program as well as an A&P faculty member within the Integrated Science and Technology program at James Madison University.