APS Observer - Looking Beyond the SAT: Psychological... -
[Cached Version]
Published on: 2/18/2002
Last Visited: 9/8/2004
To find out, the College Board has commissioned two leading researchers, APS Charter and Fellow Members Neal Schmitt, Michigan State University, and Robert Sternberg, Yale University, to explore innovative measures of the abilities needed for college and "to root these new constructs in theory," Everson said.
...
Schmitt and Sternberg, though starting from quite different theoretical bases, "are actually doing complementary work" for the College Board, Camara explained.
...
Schmitt "is developing measures ... very similar to the kind ... used in employment," which examine an individual's biographical data and ability to make situational judgments, both well-recognized predictors of job success widely used in conjunction with cognitive testing.
...
Schmitt's and Sternberg's teams have both issued reports on preliminary work, but their projects still have several years yet to run.
...
Neal Schmitt
...
Neal Schmitt
...
"The basic objective [of] employment testing," Schmitt explained, "is to get information that allows us to predict subsequent job performance. ... In the case of college students, we're trying to predict ... success at the school they go to."
...
Before they could measure factors that predict a broader concept of college success, however, Schmitt and his team first had to define what college success is.
...
"We built our whole model around" the proposition that the 12 dimensions, taken together, "are success in college," Schmitt said.Next, the team constructed two instruments to measure the 12 dimensions in students.On the assumption that "what predicts success is probably similar past experience," one instrument considers biographical information.The other evaluates individuals' situational judgment about problematic or challenging incidents typical of student life and ordinary daily life.
"Biodata is simply a set of questions in multiple choice format that asks individuals to tell us ... their background experiences and interests," Schmitt explained.
...
"I had to go home and milk the cows," Schmitt recalled.
...
"Lots of data" show that subgroup differences "on these measures" are smaller than on the SAT, Schmitt said."We're measuring something different ... social responsibility, motivation, leadership.They're measuring academic knowledge."But because of these instruments' very open-endedness, faking, exaggeration, even outright lying are among "the major problems associated with [using them] in high stakes testing," he continued.Measures that "seem to have some effect" on the propensity to dissemble include a forthright warning statement that lying, if discovered, disqualifies the applicant.Asking for elaboration - not only how many languages a student speaks or books he or she has read, but which languages and books - also tends to lower scores.In addition, the biodata instrument includes what "amounts to a lie scale," Schmitt said.It asks how often students have done something "that we know they can't have done ... For example, 'Have you written software programs in alternate basic language?'... Answering yes means you've lied.It doesn't exist.We put in five or six items like that."
Coaching is another bane of all high-stakes testing."Our answer," Schmitt said, is to "provide the coaching as part of the tool, actually tell them up front ... what we're trying to measure and put everyone on an equal footing, not just those who got coached. ... We'll describe the dimensions that we're trying to measure. ...
...
Despite the SAT's longstanding familiarity and prestige, Schmitt has data showing that minority students consider the test less relevant to college success and less fair than do white students.Minority students, on the other hand, "were more favorably disposed to our measures than the white group was," Schmitt added.
29 online sources [ view sources ]
Profile is not claimed [ this is me ]
