Objectivity

I was reading an article today about the perceived quality of GM and Ford cars compared to Japanese brands. According to the piece, American cars aren’t as bad as their reputations would have.

It used, as proof, information from JD Power and Assoc. — you’ve probably seen its ratings used in ads for various things. According to JD Power, after Lexus comes Porsche, then Lincoln, Buick, and Cadillac. Three American cars in the top five.

Then it presents “Another view” — that of Consumer Reports, which found most American cars to be average or below in quality, although a lot better than they used to be.

Fine. This sounds like two slightly opposing views on the subject. except for one thing: JD Power makes its money by selling the right to use its ratings. If you see GM, for example, say that such-and-such a car “Ranked #1 in a JD Power and Associates survey” you should also know that GM paid for the right to say that.

In other words, at least part of JD Powers revenue model requires that companies want to post its survey results. Bad rankings mean fewer sales.

In contrast, Consumer Reports accepts no advertising, and has no vested interest in who wins or loses. Its reputation depends on being as close to above reproach as possible.

I’m not saying that JD Power is on the take. But it’s a fact that it does get money from the companies it rates, and that it gets less money if it rates them poorly.

I was doing a story on a local congressman who brings in a lot of federal money for his district. I wanted to see how he ranked on one of those tax-watchdog group’s “pork” lists. I found Citizens Against Government Waste, which had a done of detailed information. Turns out they rank him pretty high as a ‘porker.’

But then I looked a little deeper into the organization’s methods. I took its list of every congressperson and their rankings, and sorted them in Excel. Turns out that with a very few exceptions the organization finds Democrats to be high on pork and Republicans not to be.

Hmm. In some ways conventional wisdom would say that’s the case. But these days GOP congressmen are spending like there’s no tomorrow just as much as Dems. The $230 million “bridge to nowhere” planned for Alaska was from a Republican, and certainly pork.

Ah, but there was the rub. It turns out that CAGW’s definition of “pork” is pretty specific. It has to do not with what the project is, but how the money was appropriated. If a project goes through “established budgetary procedures” then it’s not pork.

That results in a lot of GOP projects, which are just as porky as those of the Dems, not counting against them. For example, just about anything having to do with defense spending — those things are mostly part of the defense appropriations bills. If $1 billion goes to a useless piece of defense hardware that happens to be produced in the district of a key lawmaker, that’s not pork. But if $50,000 is spent providing adoption counseling in a rural Nebraska town, that is.

It’s all about objectivity, and how easy it is to make something sound objective without actually being so.

Of course, it works the other way, too.

When I worked at PC Magazine — known for its extensive and objective product reviews — we got mail all the time accusing us of favoring our advertisers’ products. We didn’t, and we even published out test methodologies to prove it. But people assumed we did. Consumer Reports, obviously, gets around that criticism.

(Only once when I was there did the wall get crossed. We were doing a story on network products — this was in the days when Netware ruled — and Microsoft’s LAN Manager didn’t make the cut. But then we heard from the editor, Michael Miller: Get Microsoft in. It didn’t have to get a good review, but it had to be in the story.)

Even here at the Roanoke Times I occasionally hear it. One person wrote in about a story I did to accuse me of using advertisers as experts. Never mind that no one I quoted in the story worked for an advertiser. (One was from PBS, the other from a local ISP; I had worked with both of them which is why I talked to them.)

What’s it all mean? I dunno. If nothing else, though, I hope it means that people who hear “objective” commentary take a moment to question the commentator, but that doesn’t mean assuming the worst — it means being a bit skeptical before accepting the “truth.”