Yet some professors involved in the study said it helped take some pressure off instructors.
“On our campus, in particular, we have used the VALUE rubrics as models to launch discussions as we ask faculty to work toward articulating a shared understanding of what it means to be teaching courses that fulfill our distribution requirements,” Alexis D.
Such results suggest that students are getting the mechanics of math and related skills, but not so much the “why,” or when and where to use certain calculations, according to AAC&U.
"In a world awash in data, VALUE generates evidence — evidence that points to what is working well and, critically, where there is room for improvement," AAC&U asserts.
“If the planets had aligned favorably, I would have jumped at the chance to see how Carleton students' work stacks up.
For now, I look forward to hearing more about the VALUE study, including faculty development implications." Terrel Rhodes, vice president of quality, curriculum and assessment and executive director of VALUE at AAC&U, said grades were long thought to be good measures of learning, and they still are — except that they rely heavily on content mastery.
Now, though, Rutz said, “I can better appreciate what the study offers.” She called the VALUE initiative’s strength its design, in that rubrics were taught to faculty members from participating institutions, the material that was rated was coded and distributed among readers, and the ratings were analyzed “with clear awareness of the limitations.” AAC&U cautions, for example, that the preliminary data "are not generalizable beyond the three individual VALUE Collaboratives," and that extrapolating meaning and "making inferences about the quality of learning at the state or national level are entirely inappropriate at this time." Professors “reading genuine student work shows that student products can be assessed outside of the classroom situation in a responsible way, thanks in large measure to qualified readers,” Rutz added.
Better yet, “the data point toward the necessity for considering the assignment as well as the student work itself.” Indeed, that's a point her recent book makes, and part of Carleton's portfolio assessment that has provided, in Rutz's words, successful, iterative faculty development on assignment design.
The report, “On Solid Ground,” includes results from the first two years of AAC&U’s national Valid Assessment of Learning in Undergraduate Education (VALUE) initiative.
It’s something of a portrait of student performance in critical thinking, written communication and quantitative literacy.