9486 in the collection  

Boeing to Rank Colleges by Measuring Graduates' Job Success

By Paul Basken

There's about to be a new entrant in the
college-ranking business: the Boeing Company.

The Chicago-based aerospace giant has spent the
past year matching internal data from employee
evaluations with information about the colleges
its engineers attended. It has used that
analysis to create a ranking system, which it
plans to unveil in the coming month, that will
show which colleges have produced the workers
it considers most valuable.

With a 160,000-person work force that includes
35,000 engineers worldwide, Boeing may make a
mark where the government and others have not —
raising the possibility that employers could
become a major force for college
accountability.

"We want to have more than just subjective
information" for evaluating the colleges that
Boeing visits to recruit and hire, said Richard
D. Stephens, the company's senior vice
president for human resources and
administration. "We want to have some concrete
facts and data."

In some ways, Boeing's approach to evaluating
colleges is just a more- sophisticated
extension of businesses' long-standing practice
of using historical trends to help them decide
which colleges to favor. In simpler terms, said
James L. Melsa, a retired dean of engineering
at Iowa State University, that has meant
practices such as hiring graduates of the same
colleges that top company executives attended.

Precision Work

But the Boeing model could have even wider
implications, suggesting common ground for
educators and policy makers who have been in a
heated battle over the best ways of bringing
scientific precision to the job of evaluating
college performance.

Mr. Stephens himself was a member of the
Commission on the Future of Higher Education, a
panel formed by U.S. Education Secretary
Margaret Spellings in 2005 to determine "how we
can get the most out of our national investment
in higher education."

As he sat through meeting after meeting of the
commission, listening to long discourses by
education professionals about ways to lower the
cost and improve the effectiveness of college,
Mr. Stephens decided that his company could
take action on its own.

He asked Boeing officials to review detailed
performance records of the company's 35,000
engineers, an exercise that revealed
differences in quality that could be correlated
to workers' alma maters.

Boeing will now use that data for internal
purposes, guiding the company toward colleges
it wants to favor in such areas as recruiting
and hiring. The performance data also will
drive its choices of partners for academic
research and its decisions about which colleges
it will ask to share in the $100-million that
Boeing spends each year on course work and
supplemental training for its employees.

The company also plans to share its findings
with the colleges, in an effort to help them
improve themselves. Within the next few weeks,
when Mr. Stephens sends 150 letters to
engineering deans notifying them of their
colleges' rankings, he also plans to offer them
specific critiques, based on the work records
of their graduates.

"It's really about improving the dialogue on
curriculum, performance, and how we can build a
stronger relationship between the colleges,
universities, and us, because, ultimately,
their students become our employees," the
Boeing executive said.

Receptive Audience

Colleges and their engineering-school deans
appear receptive to the advice, but many want
both Boeing's advice and the rankings kept
private.

"Constructive feedback that allows us to
improve the quality of the education we're
providing to our students, to me, is always a
good thing," said Sarah A. Rajala, dean of
engineering at Mississippi State University and
president of the American Society for
Engineering Education. At the same time, deans
must be free to put the information in its
proper perspective, meaning privately, without
the pressure of a public ranking, she said.

Boeing will treat its findings as confidential,
Mr. Stephens said. The top-ranked colleges,
however, may be less interested in keeping the
news quiet. "We're not under any false
pretenses that whatever we tell people they're
going to keep amongst themselves, for those who
rank at the top of the list," he said.

Self-promotion is especially likely because
some lesser-known institutions will be revealed
as having done an "excellent" job of producing
high-performing Boeing engineers, Mr. Stephens
said, without identifying any such colleges
ahead of their expected notification.

Ranking the nation's engineering schools may
not be entirely without risk. Some colleges
have been accused of making cosmetic or even
detrimental changes in their operations to
bolster their scores on the annual rankings by
U.S. News & World Report, for example, without
improving the actual quality of their programs.

The Boeing rankings could invite similar
attempts to game the numbers, perhaps by
steering students into or away from certain
specialties, said Mr. Melsa, the former dean at
Iowa State.

Any such responses would probably be rare, said
Mr. Melsa, who is also a former president of
the engineering-education society. "I have more
faith in faculty members and deans and
department chairmen than to believe they will
all of a sudden start doing unnatural things
because Boeing either put them at the bottom of
their list, or the top of their list, or didn't
have them on their list," he said.

Definition of Success

The Boeing project also will invite greater
examination of its employee-evaluation
criteria, which Mr. Stephens has described in
general terms as involving both technical and
nontechnical skills. Ideally, the company's
measures will reflect a wide-ranging definition
of worker success and may thereby spark greater
appreciation of well-rounded students, said
Carol Geary Schneider, president of the
Association of American Colleges and
Universities.

The Spellings Commission produced a final
report that emphasized the need to get more
students into college at a lower cost and to
send them back into the world with better job
skills. But, Ms. Schneider said, the panel paid
little or no attention to strategies for
broadening the definition of a successful
college graduate in such areas as critical
thinking, problem solving, and social
responsibility.

Companies such as Boeing can help emphasize
their value to students, Ms. Schneider said.
"Employers are much better able to call
students' attention to the importance of these
skills than their deans are," she said.

Those skills, including the ability to
communicate and work well in teams, are some of
the same attributes Boeing seeks in its
employees, said Mr. Stephens. "We need
engineers who can practically apply what
they've learned in school to help our
customers."

Boeing's definitions of a successful worker
also could fuel the debate among Washington
policy makers over what constitutes a
successful college. Diane Auer Jones, president
of the Washington Campus, a consortium of
university business schools, resigned in May as
U.S. assistant secretary for postsecondary
education, saying she felt her colleagues in
the Bush administration were pressing colleges
too hard to make their curricula fit specific
corporate needs.

"Higher education prepares individuals not just
to be good engineers, but to be good citizens,
good parents, good community members, and to
appreciate those things that enrich our lives,
like art, music and literature," Ms. Jones
said. "Even under the best of circumstances,
Boeing would only be able to measure one aspect
of higher education."

Potential Pitfalls

Mr. Melsa also recognizes the dangers in a
college's focusing too much on its rating from
Boeing. Iowa State trains engineers not only
for Boeing but also for Lockheed Martin, Exxon
Mobil, and John Deere, among other companies.
Graduates need to be trained broadly enough so
they will be able to solve problems not even
anticipated with technologies not yet invented,
he said.

Boeing's system of evaluating its engineers
doesn't typically reward innovators, said Paul
R. Illian, who spent 22 years as an engineer
there, including seven years as an instructor
in employee training and development.

The corporate job-performance criteria often
emphasized the need to ensure that "the average
employee could do an adequate job," said Mr.
Illian, a researcher at Seattle University who
left Boeing in 2001 and has also worked for
more than a decade on designs for human-powered
flight. "The extreme employees, either plus or
minus, struggled with this, whereas the 'Billy
Joe Bob' engineer in the middle flourished
under it," he said.

That's not necessarily bad, he said. As a
passenger on Boeing-built jets, he said, "I was
darn glad to have that approach. I didn't want
to be sitting in Row 37D on an airplane where
they took a lot of chances in the design."

Boeing's approach to evaluating employees also
varied with economic conditions, Mr. Illian
added, raising the possibility of year-to-year
variations in the company's assessment of
academic quality. In calm economic conditions,
it would "reward good team players and people
who really got along." During times of stress,
it would emphasize "those whose skills were
necessary for the organization to keep the
company functioning."

'Inherently Fallacious'?

Rankings have other potential pitfalls, said
Arthur J. Rothkopf, a senior vice president at
the U.S. Chamber of Commerce, who served
alongside Mr. Stephens on the Spellings
Commission. Many college presidents believe
it's "an inherently fallacious concept to start
ranking institutions, on whatever measure,"
said Mr. Rothkopf, a former president of
Lafayette College, who leads the chamber's
programs on education and work-force
development.

As with the U.S. News statistics, employer-
driven ratings could suffer from biases
favoring colleges that already enjoy a strong
reputation, since the top-performing students
tend to enroll at the most-prestigious
institutions, Mr. Rothkopf said.

Smaller colleges might face a further
disadvantage, he said, since they would be
likely to have too few graduates employed at
any single company to generate any statistical
significance in the ratings. On the other hand,
a smaller institution could emerge as an
unheralded star, Mr. Rothkopf said, gaining an
industry-specific reputation such as that
enjoyed by Williams College in producing art-
museum directors.

Boeing is not the only company seeking better
ways of choosing its college partners. Others
include Microsoft, which directs its most
aggressive recruiting toward a group of about
15 colleges, said Warren K. Ashton, the
company's manager of recruiting.

Those relationships, some dating to the 1980s,
involve colleges that offer programs important
to Microsoft and have a track record of
providing high-quality workers, he said.
Microsoft, however, hasn't dropped any colleges
from its priority list over the years, and it
might find a Boeing-type model useful for
bringing more precision to its recruiting
efforts, he said.

Like Boeing, Microsoft seeks a combination of
abilities in potential workers, including
technical abilities, skills in teamwork and
problem solving, and "business savvy," Mr.
Ashton said. But the company is more concerned
with identifying parts of the world that are
most promising for recruiting than in selecting
individual colleges, he said.

Change often threatens those on top. Iowa State
already enjoyed strong recruiting connections
at Boeing, in part because its graduates
include Thornton Arnold Wilson, a former Boeing
chairman and chief executive, and Vance D.
Coffman, a retired chairman and chief executive
at Lockheed Martin.

The alumni preference conferred upon Iowa State
by Mr. Wilson may not have been as
scientifically sound as Boeing's new process,
but it was based on the real-world
understanding among executives that the
colleges they attended do a good job of
producing successful graduates, Mr. Melsa said.

"We like the system" as it existed, admitted
the former dean at Iowa State. "There's nothing
wrong" with it.

For their part, Education Department officials
praise the initiative of Boeing and any other
companies that might help colleges as they
struggle for ways — other than the standardized
tests that many of them oppose — to demonstrate
their worth to students.

Boeing's project undoubtedly costs the company
more time and money than most businesses could
spend, said Cheryl A. Oldham, acting assistant
secretary for postsecondary education. But for
colleges receiving Boeing's employee analysis,
said Ms. Oldham, who was executive director of
the Spellings Commission, "it's exactly what
they would want to know about how well they are
preparing their students to go out into the
business world."




— Paul Basken
Chronicle of Higher Education
2008-09-19
http://chronicle.com/weekly/v55/i04/04a00102.htm

---

INDEX OF OUTRAGES

Pages: 380   
[1] 2 3 4 5 6  Next >>    Last >>