Can we believe Consumer Reports?

[geepondy]

Toto MS874114SG Carlyle

I've done a lot of reading on toilets the past two days mainly on this forum. In my quest to find a toilet replacement, I also subscribed to Consumer Reports online (I already am a magazine subscriber) this evening. Their toilet recommended products and reviews varied widely from what I read on this forum. For one thing I read sort of mixed reviews here on their number one rated toilet, the Elger Titan, mainly it seems to leak for a lot of users here. For another the first Toto to make the Consumer Reports review, the Carlyle comes in at number 10. Here I get the impression Toto is generally regarded as the Lexus of toilets and the Ultramax perhaps the best of their line. One problem I immediately recognized for the Consumer Reports review on toilets in which I have noticed for other reviewed products as well is that they review a limited number of products from a certain category. Perhaps they never looked at the Toto Ultramax or Ultimate.


Eljer Titan

[sfphoenix75]

One problem with many (most) Consumer Reports evaluations is that the products are not test over time. So, the recommendations end up being a feature-based comparison on what products deliver the best (initial) bang for your buck. How a product might hold up over months and years of use is usually still a question mark in those reviews.

[hj]

Two problems with any evaluation.
1. The tests are done in a laboratory, not in the field.
2. There is a subjective component to any test, since as a practical matter, there may be little actual difference between models.
3. Tests assume every toilet with the same model number is equal, but there can be variances during the manufacture that makes one flush properly, but the next one in line is a dog.
4. I tell customers to buy the toilet that looks the best to them, as long as it is not the HD $39.05 "complete toilet in a box".

[Terry]

Consumer Reports the magazine, at least the part on toilets, is done by two guys in a warehouse.
They aren't plumbers, they might not even be homeowners.
Two guys that look at toasters, lawnmowers, and oh, by the way, why not plumbing stuff while we're at it?

CR, No in-home testing.

CR, No consumer input.

CR, Not trained in plumbing

CR, No long term product reviews on longevity.

Most other places on the web, like this one, will listen to homeowners and plumbers.
There is a lot more information that way.

[jimbo]

I respect Consumer Reports, but they are not immune from error and bias. They do select just a few models, and they do consider price. We all know that you cannot buy a toilet with a good flush for $50. NONE. I did not read this report, but their "best buy" recommendations do include perceived value for the buck. What toilets do they rate as "best buy". How do any of their highly rated toilets list on the Map test?

[geniescience]

FWIW, I have seen on the internet a few studies of toilets done by other groups not CR. One a university study for the Canadian Mortgage and Housing Corp CMHC, a federal agency in Canada. Another by "Protect Yourself", a Quebec not-for-profit agency.

In any study like this, you can spot serious winners and losers. I guess that is the limit of what a Consumer Reports type of study can do.

There were no slam-dunk winners. Toto 's were all in the top half, not bottom.

David