November 19, 2009

Beating a Dead Horse, Part XVIII

Why is it worth thinking about Malcolm Gladwell?

Because Malcolm takes the politically correct conventional wisdom (you can't make useful predictions about people, heredity doesn't matter, just environment and effort, etcetera etcetera) seriously enough to apply it in all sorts of situations where a more prudent hack would shy away, making him the a One-Man Reductio ad Absurdum of fashionable thought.

Malcolm is the mirror image me. I'm always looking for novel ways to poke holes in the ruling discourse, to point out that the ideological emperor has no clothes; and Malcolm's always looking for ways to validate what passes for thought in polite society.

Of course, we end up demonstrating the same thing, as shown by the differing responses we get. Poor Malcolm gets laughed at because he gets so many things wrong, while I get sputtered at because I get so many things right.

With his complaining letter to the New York Times having received a terse thumping at the hands of Steven Pinker, Malcolm Gladwell revisits the question of whether or not draft position is correlated with an NFL quarterback's career on his blog.

Without admitting it, Gladwell seems to have given up former position that NFL achievement "can't be predicted," there's "no connection," etc. etc. He now seems to be saying that, when you take into account the higher pay of higher draft picks, NFL teams aren't economically optimizing their draft picks, which is a wildly different thing. Gladwell blogs:
There’s a second wonderful paper on this general subject by Cade Massey and Richard Thaler—Thaler being, of course, one of the leading lights in behavioral economics—called “The Loser’s Curse.” ... The key here is that all NFL teams operate under a strict salary cap. So a player’s real worth to a team is the extent to which his performance exceeds the average performance of someone making his salary. ... In fact, according to their analysis, the most useful draft picks are in the second round, not the first: that’s where surplus values tend to be highest. ...

It is important to note here that we are talking about relative value. Personnel decisions in the NFL have clear opportunity costs: if you pay $15 million for a quarterback who only gives you $10 million of value, then you hve $5 million less to pay for a good linebacker. As they write: “To be clear, the player taken with the first pick does have the highest expected performance . . . but he also has the highest salary, and in terms of performance per dollar, is less valuable than players taken in the second round.”

In other words, Malcolm is now, effectively, admitting that he was wrong in his New Yorker article and in his snit of a letter to the New York Times.

But he immediately goes on:
What Massey and Thaler are saying, in essence, is that NFL general managers are not rational decision-makers. [Emphasis Gladwell's]

Malcolm uses words in a Manichean black-white way so that he can tell himself he's always right. He's unable to think relativistically, which makes him popular, but means he makes a fool out of himself when he runs into a meticulous thinker like Pinker or Charles Murray.

Potential correlations between draft order (reversed so that correlations are positive) and achievement run from:

-1.00 (perfectly irrational: intentional self-destructiveness: e.g., using your #1 draft pick to announce, "Ruth Bader Ginsburg is our Quarterback of the Future") to

0.00 (perfectly random: drawing of names from a hat) to

+1.00 (perfectly rational and competent, e.g., making Peyton Manning the #1 pick and not picking Ryan Leaf at all).

When Gladwell says that draft order and performance were "not connected," he was saying the correlation was 0.00. Well, you don't need to know much about football to say it's probably not 0.00. You just need to know that when human beings set out to select other human beings, the correlation with the selected humans' accomplishments is usually above 0.00 and below 1.00.

When, however, you add market-derived costs to the equation, so that a first round pick costs more than a second round pick, with each pick priced at what it's seen as being worth, then you don't expect a high correlation. In fact, in a competitive market, the correlation would tend toward zero under perfect rationality. It's like buying stocks: Apple has a better track record at making money in recent years than AIG, but that doesn't mean you'd make more money buying Apple stock today than AIG stock. Public information about Apple and AIG has already been included in the price.

But Malcolm's not buying any of this technical mumbo-jumbo:
That’s why I think its so useful in this particular discussion. Those who believe that draft position is a good predictor of quarterback performance are essentially voting for the good judgment of the people who make draft decisions. And what Berri and Simmons in particular—and Massey and Thaler in general—remind us is that that kind of blind faith in the likes of Matt Millen and Al Davis simply isn’t justified.
Malcolm's critics suffer from "blind faith" ...

What Massey and Thaler are actually saying is that NFL decisionmakers suffer "biases" that cause them to overvalue first round draft choices economically. In other words, they aren't perfectly rational, which I suspect isn't Big News. Thaley and Massey content that if the NFL executives used the full power of the data to predict individual performance, they would make better decisions.

You'll notice, however, that this is more or less the opposite of Gladwell's claim in his New Yorker article that "there is no way to know who will succeed at it and who won't." Thaley and Massey say there are better ways than the ones being used now.

And, of course, Thaley and Massey's findings don't validate Gladwell's analogy in his original article about inability to select good teachers ahead of time. Spending too much on the highest potential jobseekers is not exactly the problem with unionized school systems.

My published articles are archived at iSteve.com -- Steve Sailer

24 comments:

Jonathan said...

A minor quibble: please stop calling Malcolm Gladwell "Malcolm" in your posts. It seems a cheap way of belittling him, thereby detracting from the overall force of your substantive arguments. "Gladwell" should work just fine.

Anonymous said...

"Those who believe that draft position is a good predictor of quarterback performance are essentially voting for the good judgment of the people who make draft decisions. And what Berri and Simmons in particular—and Massey and Thaler in general—remind us is that that kind of blind faith in the likes of Matt Millen and Al Davis simply isn’t justified."

Can someone explain to me how this works?

Because Millen and Davis are notoriously bad at drafting players, there is no real skill in drafting?

I can just imagine Malcolm at my weekly poker table:

"I always lose. There's no skill in this game."

Black Sea said...

Re: Gladwell's claim that teams aren't optimizing their spending because second rounders often provide better value for the money than first rounders, it is worth noting (and perhaps reminding him) that teams can only field 11 players at a time, and can only have 53 players on the active roster.

It's not like with all that extra money, they're going to be putting 15 players on the field at a time.

rob said...

Dammit. He's pretending he discovered constrained budgeting. Oh well, I wouldn't have gotten 30K out of him anyway.

Isn't that whole thinking through the argument something that Gladwell should try to do before he publishes? He must have been dim enough to believe his bs. Instead of thinking things through, he just Blinks.

nooffensebut said...

I think it is interesting that MALCOLM has taken the view opposite in some sense of Stephen Jay Gould’s view. Malcolm says that IQ does not matter over 120. In The Mismeasure of Man, Gould claims that IQ is useful for defining mental retardation (his own son was autistic) and incredible genius, as exemplified by Albert Einstein, but between those extremes, IQ does not matter. I wonder why these fools do not compare notes.

Anonymous said...

Terrific post. I am going to feel groovy all day, thank you.

M. M. said...

Just a thought--or lack thereof: while Gladstone Gander is a lucky gander with curly hair, Gladwell Gander is an increasingly unlucky one with frizzy fluff.

Sorry folks, I just had to.

rob said...

Humorously, choosing the best football team within limits is part of a family of problems called constrained optimization. Had Mr. Gladwell taken linear algebra, or even 2nd year HS algebra, he would have known that. It's too bad that making accurate predictions is racist.

Even if one can't invert a matrix from a standing start, knowing some math let's you tell when people are trying to blow smoke up your ass. John Allen Paulos wrote a book a while ago called Innumeracy a few years back on the subject.


I for one think it is wonderful that Mr. Gladwell could learn so much from geeky math people who will never have the poofy hair to be famous. He needs to do this before he goes public with his thoughts. From the recent discussion with Dr. Pinker, it's pretty obvious that Gladwell invests lots of ego in the ideas he promotes. He's willing to change his mind, but has to go through significant contortions to convince himself that he was always right.

Gladwell, I'll sell you the idea of looking into into how people will continue to think and do foolish things because they've already started. We can call it the sunk cost fallacy. I have a great title for our book: "SINK!"

Anonymous said...

An economic valuation of a player the way you described is BS but you failed to demolish it.


1) Its impossible to know all the ways that the players interact with one another. E.G. A great player may motivate other players to play better which will be difficult to account for.

2) Fans may be more eager to watch if there is a star player, which would add additional value to top players beyond their actual playing skills.

3) Fans may not have a 'linear interest' in watching, so that each additional yard or pass may have increasing marginal economic returns when advertising and ticket sales are included.

4) Better players mean that the season will last longer, so even if there is no additional interest whatsoever, the additional season length will provide additional revenue for the team.

The list could go on and on

Bill said...

One problem with the Thaley and Massey analysis is that they appear to use expectation value. It's a pretty good study and I think they are right that there is a bias. But when your goal is to win a Super Bowl, maybe maybe the average value of a #1 pick isn't so important to you as the small chance of finding a player at the high end of the curve. Maybe they are paying for a chance at the tail of the distribution.

The old saw in sports is that the team that got the best player won the trade. How many second round picks is Peyton Manning worth?

Vernunft said...

I'd like to stake my position as not knowing what Jonathan is talking about.

Acilius said...

"Malcolm is the mirror image me." I love it when you do this. "We are much alike, Mr Bond."

And it's very true. As a popular science writer, Gladwell writes facile pieces in an engaging, often chirpy tone, flattering his audience into thinking that they understand advanced research. Gladwell's favorite idea seems to be that the weightiest problems our society faces are in fact the result of trivial causes. This idea lends itself to happy talk, since it suggests that equally trivial changes might solve those problems. If the metropolitan upper middle classes secretly fear that the future may be a wasteland of ethnic strife and that Richard Nixon may have been right when he said that "Those African countries aren't going to make it in 500 years, you just know they're not," Gladwell can reassure them that all of the struggles we face might be resolved very quickly, very easily.

You are at least as facile and chirpy as Gladwell is. I don't mean that as a put-down; up to a point, those are desirable qualities in a popularizer. Your rhetorical goals seem to be rather different from his. Instead of leaving your audience feeling they can forget about their secret fears of a hate-ridden future, you seem to leave them with the feeling that they are a plucky little band of truth-tellers. You paint a portrait of yourself and others of like mind as innocently going about the business of figuring out how the world works, when suddenly, out of nowhere, you are set upon by the malign forces of orthodoxy and taboo. I don't happen to find this portrait especially convincing, but you do an excellent job of painting it. I'm not at all surprised at the enthusiasm your fans show.

What your devotees seem to wish would vanish is the anger and confrontation that accompanies racial strife. If only everyone accepted the Gospel According to Saints Rushton, Shockley, etc, then the plucky little band would be vindicated and everyone would settle down.

Gladwell's conduct in this little controversy has been a gift to you beyond price. This time, you actually have made a reasonable, well-argued case; Gladwell actually has dug in his heels and tried to support a silly assertion with name-calling. The fact that he did this by putting your name in the NY Times during your fund-raising drive makes it hard for me to entirely suppress the suspicion that the two of you are working together. The fact that it was only on the editorial page does dampen that suspicion, of course.

Much as you have to gain from Gladwell's denunciations of you,
he may have even more to gain from you. If you became a household name, the metropolitan upper-middle class would need a lot more reassurance to quiet its misgivings about the shape of things to come. The easiest way to get that reassurance would be to take a dose of Gladwellian happy-talk.

Matt Millen said...

I very much dispute the ridiculous notion that NFL teams do not economically optimize their draft picks. Malcolm Gladwell is obviously a football ignoramus and a twit.

Middletown Girl said...

One thing I'll say about Pinker, Sailer, and Gladwell. BOYS WILL BE BOYS. All this guy talk about sports is getting ridiculous.

Anonymous said...

Would you shut your trap for just one minute Matt?

-Rod Marinelli

rob said...

Some anon or another,

Prolly not a linear constrained maximization. I don't care a bit about football. Anyone who can't solve a linear optimization problem has zero chance of solving a nonlinear problem.

Truth said...

"Because Millen and Davis are notoriously bad at drafting players, there is no real skill in drafting?"

Interestingly, Al Davis may be the best case against Gladwell (although to be honest, I don't believe he thinks draft position has NO correlation to success).

In the 1970's Al Davis was regarded as a shrewd evaluator of talent, no he's considered a moron. One could make the case that one of two things have happened:

1) He got dumber in this field

2) His competitors got smarter

I think is probably mostly #2 as Davis probably drafts primarily through emotion, like for instance, he picked Mike Mitchell, a safety from the University of Ohio in the second round last year. Mitchell was projected so low, that some of the scouting services did not even have him listed as a possible free agent. I watched Mitchell's tape and he absolutely trucked a few guys and had them carried off. In 1970, that's all you needed to be a good NFL safety, but now, you have to cover very good tight ends as well.

Davis saw a few of those huge hits and figured it would obviously translate to a league with bigger, faster, more talented players and much better offensive coaching; so far it hasn't.

David Davenport said...

...The fact that he did this by putting your name in the NY Times during your fund-raising drive makes it hard for me to entirely suppress the suspicion that the two of you are working together.

Next: a pic of Steve and Mr. Gladsbags, both wearing jogging shorts, on the cover of Newsweek or Time.

none of the above said...

no offense but: I think that comparing of notes will have to wait until Gladwell decides to write a book on seances.

More seriously, it's worth asking why the "IQ doesn't matter beyond a lower bound" idea seems plausible to anyone. I think it's entirely because of stratification by performance.

Let's assume that your performance in any field is a function of a bunch of stuff--IQ, work ethic, organizational skills, likability, luck, connections, etc. Now, look at a bunch of people with similar performance--say, tenured economics professors at solid but not spectacular schools, or journalists working for top-tier newspapers like the Washington Post, NYT, or WSJ.

Each person has a different mix of IQ, work ethic, organizational skills, connections, luck, etc. But their performance is all pretty similar--they see each other as peers. In this set, everything balances out. The smartest people are lazier, less organized, etc., than the dumbest people, simply because we've stratified our sample by success level. (The guys with high IQs and lots of luck and great work ethics and wonderful connections are stars in the University of Chigago or Harvard econ departments, or they're Malcolm Gladwell or Thomas Friedman, and thus are outside our sample. The guys with low IQs and lousy work ethics and bad interpersonal skills and such never made tenure, or are eking out a living at some medium-sized or small newspaper somewhere covering local government corruption.)

Thus, to the extent you're dealing with your peers by success, you'll notice that:

a. There's some lower bound of intelligence, below which you never see anyone. (The same is true for all the other important inputs to success.)

b. Above that, there doesn't seem to be a huge impact for IQ in your peer group.

c. Tragic flaws are really common. You will routinely see people who have all kinds of advantages, but one Achilles heel that keeps them from rising any further--the brilliant researcher who just can't seem to keep himself motivated long enough to write most of his ideas up, the fantastic journalist whose constant battles with alcoholism limit his success.

This is a place where a wider view, outside your peer group, is really valuable.

Anonymous said...

I suspect there are other reasons for the extra-high salaries for first-round picks. The value of a draft pick isn't JUST his value on the field. There's also (free) publicity to consider (how many front-page articles in the sports section of newspapers, for instance, and also how much free publicity for the team from Nike ads), and how well he draws fans to the seats. There is even the competition between the team owners -- one rich white man inferring 'ha ha, I got the best guy on my team and you didn't' to the other.

With all that (rather intangible) value added in, the 'too-high' salaries for first-round picks might be just about right. I suspect NFL team are reasonably smart and rational, despite what we'd like to think.

Anonymous said...

Steve,

In case you missed it, this comment was posted by the author of one of the references cited by Gladwell in his response to Pinker:

http://gladwell.typepad.com/gladwellcom/2009/11/pinker-on-what-the-dog-saw/comments/page/5/#comments

Malcolm, we are the second source that Pinker mentions. I'm responding here because your description of our argument could not be more inaccurate. You write: "Pinker's second source was a blog post, based on four years of data, written by someone who runs a pre-employment testing company, who also failed to appreciate—as far as I can tell (the key part of the blog post is only a paragraph long)—the distinction between aggregate and per-play performance." Actually, the paragraph I think you're referring to explicitly recognizes the issue of aggregate vs "per-play" performance. Here is what we wrote:

To take the most recent decade as an example, when one looks at all the quarterbacks (67 in all) who were drafted by NFL teams from 2000 to 2004, and compares their overall draft position to their statistics in their first four years in the league, it is clear that on balance NFL teams are very accurate in predicting statistical success in the NFL. Organizational psychologists measure the predictive validity of an employee selection technique by quantifying the strength of the relationship between selection measure and job performance; the strength of the association is expressed as a correlation coefficient. For the whole group, the correlation between draft order and passing yardage is very strong (-.73 — the coefficient is negative because the higher a player is drafted, the lower their draft rank). For those concerned that a measure of total productivity such as passing yardage is somewhat correlated with opportunity, we can consider passer efficiency, as measured by QB rating. Only 51 of the 67 quarterbacks drafted attempted a pass in the NFL, a necessary requirement for calculating a QB rating: for this group there was a -.34 correlation between draft position and QB rating. This is still a strong association, and shows a clear, statistically significant correlation between draft order and future statistical success in the NFL.

So it was five years of draft data, not four, but that's not really the point. After showing the incredibly strong correlation between draft order and total production by NFL QBs, we explicitly address the possible objection that total yards passing may not be as appropriate as an "opportunity adjusted" stat such as QB rating, which is as pure a per-play statistic as there is available for measuring QB proficiency. There we show a correlation between draft order and performance that, while not as strong as the total performance correlation, is still striking.

How you get from this to your conclusion to the fact that we "failed to appreciate...the distinction between aggregate and per-play performance" is beyond me. It seems that either you're intentionally misrepresenting the substance of our argument or that you're not aware of the nature of QB rating, which is a per-play performance metric. Either way, your readers are being misled.

The entire blog post is available at: http://blog.criteriacorp.com/blog/bid/7688/Gladwell-s-New-Yorker-Article-on-Hiring

and another earlier post we wrote about the shortcomings in your reasoning can be found here:

http://blog.criteriacorp.com/blog/bid/7662/Malcolm-Gladwell-s-Outliers

And one more thing: your dismissal of Pinker, surely one of the greatest cognitive scientists of our time, as an "IQ fundamentalist," doesn't really accomplish much, except showing how out of touch you are with the vast scientific literature on cognition from which you occasionally cherry pick findings. You are a great writer and essayist, but picking a fight with Pinker on IQ: what are you thinking?

Posted by: J Millet | November 20, 2009 at 06:15 PM

Anonymous said...

Middletown Girl said... "One thing I'll say about Pinker, Sailer, and Gladwell. BOYS WILL BE BOYS. All this guy talk about sports is getting ridiculous."

Well that tea won't make itself will it? Chop! Chop!

Anonymous said...

Another one of Gladwell's blunders:

http://blogs.sun.com/mwaterl/entry/what_not_to_do

frost said...

Here's an analysis from a football statistics guy who strongly disagrees with Gladwell:

http://www.pro-football-reference.com/blog/?p=4740