Benchmark Reviews? Busted, more like
WoSblog is internationally renowned for its psychiatric expertise, so I wasn't surprised yesterday when a viewer of the WoS Forum asked me, quite out of the blue, to assess their mental stability.
"I want to know if I'm going barmy", wrote the clearly-distressed reader, whose cause for doubting his very sanity was – of all things – a review of an incredibly expensive office chair.
The viewer's issue was that they felt the review read more like an advertisement, and from the opening lines alone it was difficult to disagree with their assertion. I'll quote the first three paragraphs in full here, because strange things have been happening to the article since WoSblog started to investigate it, and the way things are going they might not be there by the time you come to read this feature:
"Is a chair's purpose merely to furnish a place for us to sit? There are countless directions to discuss this subjective matter, but suffice it to say that a chair's purpose goes beyond a solid foundation for resting our weight. In an ideal chair, the design would be suitable for hours of comfortable sitting. The chair would accommodate healthy posture, and relieve stress from load-bearing joints. Ideally, this chair would be attractive and feature a look as competent as its ability. There is such a chair.
Herman Miller is a company that engineers function and architects fashion. They've dominated the industry with decades of elite furniture, and influenced our perception of what a chair should be. Herman Miller's Aeron chair, one of the few commercial products added into the Museum of Modern Art's permanent collection, was introduced in 1994 and has set the bar immeasurably high for every chair that's come thereafter. It seems fitting then, that Herman Miller, the name behind the world's most admired chair, has returned to improve upon their prestige.
Welcome the Herman Miller Embody chair. Designed specifically for people who work for hours at their computer, Embody is the first work chair that benefits both mind and body. More than just a solution towards minimizing the negative effects of sitting, Herman Miller's Embody chair was also designed to deliver positive effects on the body. In this article Benchmark Reviews takes you on a tour of the Herman Miller Embody ergonomic office chair, and demonstrates how much better sitting all day can feel. "
Originally, the "review" then followed this up with a paragraph of puff about what a super company Herman Miller was. You can see it in the picture below, retrieved from Google Cache (which is the source of all the coloured highlighting, a result of finding the cached page by searching for the text).
Note that the paragraph is presented in exactly the same text formatting as the rest of the review. This will become more interesting shortly.
The entire second page of the "review" read even more like it was lifted straight out of a press release, complete with bullet-point lists of features described in that weird, stilted way that nobody but PR people ever uses – "Creates harmony between people and computers"; "Enables and promotes healthful movement"; "Lets blood and oxygen flow more freely, enhancing ability to stay focused".
There's a very good reason for that, namely that it WAS lifted straight out of a press release. But we're getting ahead of ourselves.
It transpired that the WoS Forum user (who we must, through gritted teeth, refer to as "VinylPusher") had raised his concerns about the "review" in the comments section, and had his posts deleted. In an email to VinylPusher explaining this censorship, the Executive Editor of Benchmark Reviews – one Olin Coles – had given him the following explanation:
"Considering how many people visit our website, I’d like to ensure that my article has not led others into believing it’s an advertisement. The Embody chair was paid for out of my own pocket, with absolutely no sample/product coordination with the merchant. If you would please supply examples of how you perceive my eight-page article as an advertisement, it would be helpful."
The WoS Forum, meanwhile – and particularly those members of it who are professional writers – was busy wholeheartedly agreeing with VinylPusher's assessment of the "review". For page after page, it spews out PR-speak that resembles a critical review about as much as Chris Moyles resembles Megan Fox. Page 4, for example, kicks off with the following assertion:
"Designed by Bill Stumpf (who pioneered the Aeron Chair) and Jeff Weber, the Herman Miller Embody chair goes a step beyond being merely "heath-neutral". Over time, it can actually improve the health of the person sitting in it. Scientific studies have shown that Embody users can experience better circulation, reduced resting heart rates, and less tissue damage around the sitting muscles."
Oddly, the "scientific studies" in question are neither named nor linked from the piece, leaving the reader none the wiser as to their methodology, impartiality or validity. The text continues "Through years of research and development, Bill Stumpf and Jeff Weber incorporated health-beneficial features such as the Pixelated Support System – a dynamic mesh of adjoining seat materials that has been proven to increase circulation and reduce strain on cell tissue", but similarly neglects to identify the source of this "proof".
It being almost impossible to read the review without arriving at the same conclusion as our poor troubled viewer had, I posted a couple of comments myself, politely requesting that in the interests of propriety, Benchmark Reviews really ought to identify the member of Herman Miller's PR department who had clearly provided most of the text. These comments too were swiftly deleted. In the intervening time, however, another WoS Forums user, Ian Osborne, had located the smoking gun.
This Google search for one of the more glaringly obvious paragraphs of PR-speak turned up the source of the text – Herman Miller's own brochure for the chair. (Which I've mirrored here, as their website is painfully slow.) One of my deleted comments was a link to the Google search, which was where things started to get really fun.
Having had my original comments deleted, I posted another one remarking very briefly on their deletion, using a disposable email account since what normally happens in these situations is that your original username and/or address get blocked. The post was unsurprisingly also deleted, but this time the disposable address got an emailed reply, from "Benchmark Reviews Administrator".
I was rather hurt by the attack on my reading skills, especially as I was also pretty sure that there had been no such identification of the sources of the text, nor any way of distinguishing it from the rest of the copy. In fact, I knew there hadn't, because I'd looked for it the first time.
So I went to look at the review again to check, and was surprised to find that it was no longer accessible. Or at least, no longer accessible to me. No matter which browser I tried the link in, I got an error message.
Asking some friends to confirm the problem, I discovered that they had no difficulties at all, which was odd. I then hooked up a laptop to a mobile broadband dongle and got into the site first time, while my normal connection still came up blank. The only reasonable conclusion was that my IP address had been blocked. It's now the following day, and I still can't get into any part of the Benchmark Reviews site with my normal PC, while everyone else I know can see it fine.
So I decided to follow Ian Osborne's lead, and searched for some of the text via Google Cache. It first produced the copy of page 1 that's pictured near the top of this feature, with the "About Herman Miller, Inc" paragraph in its original formatting. Google Cache described that version as "the page as it appeared on 14 Jul 2010 08:25:39 GMT".
(The page now italicizes that paragraph, and appends the line "Company Summary Provided by Herman Miller.")
The results for page 2 were more interesting. Google Cache's header, however, noted that this was a version of the page created at 02:43:10 GMT on July 18, four days later than the page 1 cache. This time, the opening two paragraphs were italicized, but still lacked the "Source: Herman Miller" tag described in such hurt and angry tones by Olin Coles in his email. There was no other explanation for the italics, and no quote marks, which are the traditional way of depicting a quote.
(The version of the page that's current at the time of writing has a slightly different prefix to the one claimed by Mr Coles, and does feature the "Source: Herman Miller" line, but the curious thing is the placing of it. It appears directly after the two paragraphs of text but before the bullet-point puff list about "creating harmony between people and computers" etc – which was previously headed "Herman Miller Features" – so we must assume that Mr Coles wrote those himself.)
It seemed that the only way to get to the bottom of this perplexing mystery was to go straight to the horse's mouth. So I've dropped Benchmark Reviews a line, starting from the premise that if they didn't like "anonymous internet trolls" I should introduce myself properly.
Dear Mr Coles,
Hello. I'm a professional freelance journalist of 20 years' standing (an archive of my work can be found at worldofstuart.co.uk), and a number of people have recently drawn my attention to your review of the Herman Miller Embody chair. I'm investigating some allegations about the review, and wondered if you could clear a couple of things up for me.
I notice that the article has in the last 24 hours been amended to identify some sections as being written by Herman Miller, which were previously not identified as such. Interestingly, you appear to be claiming that the current version is the original, something which is easily disproven via Google Cache.
Are these the only sections of the article not written by you personally? Can you confirm, for example, that the following passage is your own work?
"Designed by Bill Stumpf (who pioneered the Aeron Chair) and Jeff Weber, the Herman Miller Embody chair goes a step beyond being merely "heath-neutral" [sic]. Over time, it can actually improve the health of the person sitting in it. Scientific studies have shown that Embody users can experience better circulation, reduced resting heart rates, and less tissue damage around the sitting muscles. Embody promotes natural alignment in the spine, relieving stress across the entire back no matter how you twist and turn."
If so, can you provide links to these "scientific studies", since presumably you wouldn't have cited them if you didn't read them yourself? Clearly, in the interests of fairness I'd like to have your side of the story before writing my piece. Should I not hear from you, obviously I'll have to proceed on the basis of the evidence available.
Best regards,
Rev. S. Campbell
If I hear anything, I'll be sure to keep you updated. But in the meantime, let's recap the facts that we know for sure:
1. The original "review" did NOT acknowledge that the paragraphs on pages 1 and 2 were provided directly by Herman Miller.
2. The review has been hastily edited to insert some acknowledgements to that effect, but Benchmark Reviews is claiming that they were always there, something which is provably untrue.
3. Anyone voicing doubts about the integrity of the review has – despite the implicit admission that the original version was misleading and didn't acknowledge PR content – had their comments deleted, and received a huffy or openly abusive email.
4. My IP address, seemingly alone in the world, is no longer able to view the Benchmark Reviews website.
5. Olin Coles really, really loves his $1200 chair, which he definitely paid for himself. He even goes so far as to say that "The Embody chair does not come with arm supports by default, which is a good thing, because most times they'll be unnecessary or unwanted."
The arm supports, instead, are a $100 optional extra. (Other extras include nicer cloth for an extra $200, a shiny silver-coloured base for a further $200, and "premium translucent casters" for another $50, taking the price of the fully-kitted chair comfortably over £1000. And that'd be before VAT.)
6. None of the following quotes are identified in the article as being provided by Herman Miller, even after its editing, so they must have been penned by a professional, "fully independent" reviewer doing his best to write a balanced and critical appraisal of an eye-wateringly expensive piece of top-end office furniture in order to give his readers useful purchasing guidance:
"Herman Miller Embody chair cloth textiles utilize spacer-and-knit constructions used in athletic footwear and geo-textiles. These materials are meant to enhance Embody, not simply cover up the chair. Contemporary colors paired with either of two frame colors and three base colors help to simplify choice* and appeal to universal tastes." [Hmm.]
"The Herman Miller Embody chair's Backfit adjustment and seat conform to your unique shape and distribute weight evenly. Embody's shape mimics the spine, providing subtle support along the entire back that shifts with your movements. The Backfit creates a dynamic surface that reacts to your movements every time you shift"
"To promote the flow of heat away from the body that can build up under the legs and back over time, Herman Miller developed the Pixelated Support system. This 4-layer mesh is filled with negative space** which permits a much greater amount of air flow than traditional fabric and frame chairs. Every square inch reacts under your movements. When you shift, your seat shifts. The extremely responsive nature of the Pixelated Support material means Embody users can sit for hours without experiencing uncomfortable heat-buildup."
"By fitting the body form and reducing seated pressure, Embody increases blood circulation and improves the flow of oxygen to help decreases heart rate." ***
"If there was one Herman Miller piece that goes perfectly with Embody, it's the Envelop desk. Equipped with adjustable features such as a tapered extending desktop with tilt support, the Envelope could be considered one half of the ideal Embody set." ****
7. When asked in November 2008 how he selected which products to review, Olin Coles responded by saying this, on the record:
"I used to take anything that manufacturers would offer, back when BmR was starving for donated products. These days, we work more closely with proven manufacturers to help launch their upcoming products".
My emphasis, there. Make your own minds up, chums.
(Special thanks to alert WoSblog associate John X – not pictured – for splendid additional sourcing.)
* In terms of those three options alone – that is, discounting the numerous caster types and optional armrests etc – there are a mere 168 combinations to choose from. Thank goodness they simplified it!
** I have no idea what "negative space" in a chair is. Sorry.
*** The source of these medical claims, like the others in the review, is not identified. I suspect it's Herman Miller Laboratories.
**** Buy more Herman Miller products! In the eight-page review, the words "Herman Miller" appear 71 times, or once every 64 words. The word "Embody" makes 104 appearances, or one every 44 words. For reference, this single paragraph you're reading now is 44 words long.
UPDATES TO THIS AWESOME SAGA:
Update 1 – Olin Coles publishes my home address and phone number on the internet.
Update 2 – the original source of most of the text is revealed to be a furniture retailer.
Update 3 – suddenly, all the ads in the "review" are replaced by ads for said retailer. Numerous hasty edits are also made to the text as disclaimers and qualifiers, to make it appear less like sales blurb.
Update 4 – Olin Coles files numerous DMCA reports to try to suppress the story, succeeding in having WoSblog suspended for most of a day by its idiot hosts.
Update 5 – Olin Coles admits to taking the material from Smart Furniture – revealing most of his previous statements to have been lies – but claims to have had permission.
It's obvious enough from the initial paragraphs that it's an example of co-back-scratching article-writing simply because of how many times it uses the phrase "Herman Miller" and "Herman Miller's Embody Chair". Hoorah for keyword-spamming your way into a positive position on Google.
Coincidentally, some of the unfairly critically-mauled new M.I.A. album (particularly the opening 'The Message') is about this, when the crap-singing young lady had been searching for information on Sri Lanka, and found only pages and pages of "It's really nice! Come visit us! Look at how pretty the sea is!" advertisements. On the one hand "Well, of course, it's capitalism in action you fool!", on the other hand, if searching for information on Sri Lanka, wouldn't some info on its political history arguably be of more importance than page after page targeting western holidaymakers? And isn't that supposed to be what search engines are for, if we go back to the initial CERN days of the intarweb? Eh?
So, yes. Death to Herman Miller and the Herman Miller Embody Chair! Death to them for spamming up the intarweb (and actually, properly, really spamming it up too, not the mistaken "Oh my God you dared contact me to see if I'd like to win some festival tickets, you bastard!" 'spamming' that slack-jawed cultists kick up a childish fuss about. For example).
p.s. "It transpired that the WoS Forum user (who we must, through gritted teeth, refer to as "VinylPusher"" — "Revstu" is a funny first name, eh?
Given the fact that the front page prominently gives my full title and name as "Rev. Stuart Campbell", I don't think "RevStu" does a very good job of concealing my identity…
But if you click on my Handsoap the Defenestrator profile at the World of Warcraft forums you'll see a link to my personal homepage at http://www.thelatecarrygrant.org.uk which also clearly reveals my real name.
But anyway.
Point being?
I keep my VinylPusher and Stefan Holmes (oops, damn) identities separate for purposes of appealing to my sense of invulnerability.
Actually, no. VinylPusher is a nickname I've self-proclaimed for many years. I went by a different nickname on teh interwebs before VinylPusher came along. Sadly, my old sekrit identity is tinged with enormous personal embarrassment. We all do crazy/stupid things as young adults.
I'm fairly certain that some perhaps slightly non-trivial Googling will quickly reveal my real name from my nickname. From there, I have an unnervingly historically persistent internet presence. Mock away, oh mighty mocker.
P.S., an entertaining article you've done here, Stu. For me, at least 🙂
VinylPusher – nobody is mocking you. The previous blog post and forum thread regarding whether or not people should have to use their real names and prove their identities on things like Facebook or forum threads was being harkened to, as you'll see from the "gritted teeth" link and if you check on WoS forum. It was a "retro" argument as a post-script.
Indeed. I'm a 42-year-old man and I was merely conveying my embarrassment at having to refer to someone as "VinylPusher" in a moderately grown-up investigative-journalism piece.
That's a great picture of Megan Fox. made my day, that has.
Sorry to continue so far off the main topic, but, it's a good thing that messrs Woodward and Bernstein didn't mind getting their info from Deep Throat, eh? If they'd insisted upon a real name for fear of looking silly, it'd all have gone a bit Watchmen.
I second the greatness of that Megan Fox photo.
Taken by Steven Shite, apparently. But yeah, people who want videogame forum users to have to display a real human name definitely wouldn't make exceptions for whistleblowers exposing corruption at a Presidential level. Tch.
Can we have this stuff on the appropriate thread (ie the Blizzard one) from now on? Thanks.
I’d like to read Olin Cole’s review of sitting on Megan Fox.
I'd like to read Olin's review whilst Megan Fox was sitting on me.
I'd rather not have anything to do with that rotter Megan Fox. Eugh.
[…] Benchmark Reviews Busted. […]
Besides, who cares about that seat, we all want this one.
link to worldofstuart.excellentcontent.com
[…] Journo 101 So what would you do if a review you wrote that smacked of press release/brochure copy got ‘busted’, and a journo contacted you for more info, to get your side of the […]
I've been writing reviews for Benhmark Reviews for a year or so now, and I think you're trying way, way too hard to find some kind of scandal here.
If you look at any of the reviews on the site (there are hundreds, pick one), you'll notice they follow a standardized format. They all have an "About [company name]" section typically filled with information from the company's web site. Now, I'm just a humble programmer, not a professional journalist, so perhaps I can be forgiven for thinking it would be pretty damn obvious that material in this section was in fact company propaganda and not written by the reviewer. Nonetheless Olin has told all his reviewers that this information will henceforth be italicized and attributed for the folks who really, really need a clue…such as– and I'm just picking an example off the top of my head here– people who throw around accusations of "plaigarism".
Your main beef appears to be that Olin liked his chair too much, leading him to write a review incorporating some Herman Miller claims and a general tone you thought too obsequious. Your privilege, I suppose…now could you please go bother someone else?
I don’t give a crap how much he liked the chair. I’ve never sat in a Herman Miller Embody, for all I know it’s the greatest chair in the world and deserving of infinite acclaim.
What I care about is his dishonesty. He absolutely provably lied about whether the press-release paragraphs were flagged as such – apparently too stupid to realise that the internet keeps track of these things – when all he had to do was say “Oops, yes, that was a mistake, sorry”, rather than trying to cover it up while ranting furiously about “dim-witted internet trolls” and making groundless allegations about “threats”.
(Even now we’re expressly being told that things like the bullet-point list of features including “Creates harmony between people and computers” are Benchmark Reviews’ own work, NOT something taken off a Herman Miller press release. It’s not in italics, after all, and it appears after the bits identified as being Herman Miller-sourced.)
And he certainly seems to be lying about the rest of the review being his own work, given that most of it appears in almost identical form on the Smart Furniture website and they claim ownership of it, while having no reason to lie.
But tell us this – do you think Mr Coles’ reaction has been that of an innocent man? He was simply and politely asked to offer his side of the story and respond to a couple of very fair questions, and he’s reacted in just about the most unprofessional and childish manner possible short of shitting his pants and throwing his rattle out of the pram. Does it seem reasonable to you?
(I mean, for all the rest of us know that’s how he reacts when you forget to put sugar in his coffee at the office, so maybe it doesn’t seem unusual to you that he flies into a foaming-mouthed rage when someone asks him a straightforward and legitimate question.)
Or are you accusing Smart Furniture of stealing Benchmark Reviews’ work and lying about it? Because as we saw with the Guru3D business, that seems to be something that Benchmark Reviews gets pretty mad about, in which case you’d logically be grateful to us for pointing it out – not throwing the world’s greatest hissy fit, desperately trying to suppress all questions about the issue and breaking all sorts of laws by publishing my personal information on the internet.
Or is it just a pretty incredible coincidence that you both happened to write the exact same 3000 or so words about the same chair?
Oh, and I’d never “throw around accusations of plaigarism”, because as a professional journalist I know how to spell “plagiarism”. If you’re just “a humble programmer not a professional journalist”, why are you masquerading as the latter by writing reviews for a commercial website? Are you being paid? Because if you are, I’m afraid I have some bad news for you – you ARE a professional journalist, with all the responsibilities and obligations that position carries.
I found your blog from the Reg, but I’m still a little unclear about what’s going on. So did you prove he was given the chair for free? You would look pretty silly if he had paid for it himself, so make sure to call around. The other things don’t make any sense to me. You keep complaining that he made changes to the article, but most people would call them corrections. I donno, maybe he did take the two sentences without asking, but then again they might have a relationship with the company. All in all, I think I’ve just wasted a lot of time and given both websites a free visit.
I have no idea if he got the chair for free, and don’t care. Whether he paid for it isn’t the issue. The issue is his blatant lying about having written the review himself, when it’s clearly at least three-quarters copied from a Herman Miller press release. I’m afraid if you think the changes made are simple “corrections” you must still believe in the Tooth Fairy.
You already said that 93.6% of the article was his own, so how could 75% be from Herman Miller press release? Either you are bad at math, or you have an axe to grind with the author and you are using this as an excuse to attack him. I mention this because I’m familiar with several other review websites and most of them put something about the company that made the products they test.
Having actually read his article, long-winded and glowing as it was, there was only a small part on the first page and second page that he didn’t write himself. If he bought a chair and wrote a review about it, who cares? It just seems like you are making a lot out of nothing by complaining he didn’t notify the world of his changes. At least he made them, right?
What are you, an idiot? The 93.6% figure referred to what he was CLAIMING as his own, having admitted that the other 6.4% (ie the passages on pages 1 and 2) was Herman Miller press release.
Whoever wrote this must be living in a cave. Many if not most review websites use massive portions of material from the company that supplied the product. I see it all the time, and there’s nothing wrong with it. Take a look at this: link to geekinvaders.com
There clearly is something wrong with it, or Olin Coles wouldn’t have reacted in such a crazed blind panic when someone pointed out he was doing it.