How product reviews could be better
 

My Two Cents

I just received one of my weekly product review email newsletters. What struck me is that what product reviews are missing are multi-unit, real-world tests. I'd like magazines/e-zines to test not one unit, but five or more (of the same model). Reviewing one unit doesn't help a reader determine if it's a reliable product. Set up five, bench test the hell out of them, then do the review. Of the five, did two stumble after installing lots of software? Did three have hard drive problems after two weeks of continuous useage?

A review about the softness of a the keyboard and the brightness of the screen is great, but I want to know if a new product is more or less likely to break down than a similar product.

That's what e-zines like Tom's Hardware do. I know there are others like them but I read them so rarely that I don't remember where. Perhaps other TFPers can give you pointers.

CNET will often do side-by-side comparisons. Look at the "Editor's Pick" so product that usually are reviewing in a group.

pook is not talking about comparisons. He is talking about having multiple units of the same product, sort of like spot checking to see that the test was not a fluke. I've never seen any review that has done this, but then again, I don't read many reviews unless I'm in the market.

Exactly. Testing a single unit over the cours of a day isn't much of a review. And while speed is important, once you have that PC/laptop on your desk, it's the fastest PC you know of... to you. Unless I'm gaming, I'd rather own a PC that's 10% slower than the rest but 50% more reliable.

Kind of like Car & Driver when they give you an update on the car a year or two or three from now?

In the case of computer equipment, what time frame would be accurate? One week, one month, six months? What environmental conditions would you test under? For a PC, what peripherals would you test with? What software? Instead, use a benchmarking or load tool with "standard" tests?

I find most of my stability information by scouring message forums. Real world testing from lots of people doing various things that a bench won't uncover.

I've found that stability usually increases with a brand name - not always but usually.

Interesting idea, provided the results are realistic and applicable per the type of device.