Product Reviews are Broken

[Rogifan]

Great blog post Sent from my iPad

A good example is a 9to5Mac post originally titled '15 new user experience features revealed in early Apple Watch reviews'. But the actual post was just 15 of the most critical/unfavorable comments from various reviews. The author eventually did change 'features' to 'issues' but he left the word 'new' in the title which still makes it confusing. I haven't read all the reviews but according to Cybart's post there have been 21 reviews so far and only 4 were overly critical yet those were the ones that got most of the attention.

I agree with Cybart that reviews are still important but they need to be redefined. Think about what the average consumer (or whomever you think the audience for the product is) would want to know and center your review around that. Keep it objective and don't make the reviewer or the review itself the star (e.g. the Verge didn't need 31 people involved in their Apple Watch review); no need to turn reviews into major productions.

http://www.aboveavalon.com/notes/2015/4/8/product-reviews-are-broken

 

[shk718]

Great blog post Sent from my iPad

A good example is a 9to5Mac post originally titled '15 new user experience features revealed in early Apple Watch reviews'. But the actual post was just 15 of the most critical/unfavorable comments from various reviews. The author eventually did change 'features' to 'issues' but he left the word 'new' in the title which still makes it confusing. I haven't read all the reviews but according to Cybart's post there have been 21 reviews so far and only 4 were overly critical yet those were the ones that got most of the attention.

I agree with Cybart that reviews are still important but they need to be redefined. Think about what the average consumer (or whomever you think the audience for the product is) would want to know and center your review around that. Keep it objective and don't make the reviewer or the review itself the star (e.g. the Verge didn't need 31 people involved in their Apple Watch review); no need to turn reviews into major productions.

http://www.aboveavalon.com/notes/2015/4/8/product-reviews-are-broken

My problem with these "reviewers" is most of them are just bloggers with no journalistic background - they report their pre-conceived notions. The watch will ultimately be judged by us - the buying public - not some 'reviewer'. My favorite is Nilay Patel from the Verge: "I felt ridiculous wearing the Milanese Loop, so I didn’t." So Nilay you felt "ridiculous" wearing that band but not that crazy 80's studded leather cuff you have on your other arm?????? Dude should not be giving fashion advice.

 

[samiznaetekto]

These mostly politically correct "reviews" are nothing compared to what's coming in a couple of weeks and beyond from hindreds of tech sites, blogs, youtube channels, and thousands and thousands of ordinary users. The worst is yet to come!

 

[5105973]

My problem with these "reviewers" is most of them are just bloggers with no journalistic background - they report their pre-conceived notions. The watch will ultimately be judged by us - the buying public - not some 'reviewer'. My favorite is Nilay Patel from the Verge: "I felt ridiculous wearing the Milanese Loop, so I didn’t." So Nilay you felt "ridiculous" wearing that band but not that crazy 80's studded leather cuff you have on your other arm?????? Dude should not be giving fashion advice.

that was pretty funny. And I agree. Even though it's the kind of crazy move I would have made about 20 years ago.

Neither should the women he consulted about the fashion aspects of the watch be talking about fashion if their lame discussion was anything to go by. They sounded like teenagers at the mall, using the word "like" to preface every sentence. Leaving aside their makeup, clothes and accessories, none of them wanted to be bold and make a real prediction about the fashion world's response. The best the one woman could come up with was that the fashion world would accept it if it becomes ubiquitous and popular. Well the "pants on the ground" look is ubiquitous and popular, but that does not make it high fashion. Coco Chanel is rolling in her grave at that editor/blogger's wishy washy conclusion. The masses at times can create trends but that's not the way the fashion world would prefer it.

 

[bjdraw]

Part of the problem is that reviewers go to work for the same reason you do, but on top of that they have egos -- who would write anything for publication if they didn't believe someone else wanted to read it.

The thing to remember is that reviewers don't usually write the headline, assign the score or write the summary without a lot of editorial direction. And that direction isn't usually full of the pure invent of serving the readers and instead is set to get people to link to or click on the article. Writers want to be read and sites want to earn ad revenue.

In other words; ignore the headline, score and summary and read the full article.

PS, I wrote for Engadget for 9 years and reviewed lots and lots of stuff. I officially resigned in December. And yes, I worked closely with Nilay on many occasions before he left for The Verge.

----------

Neither should the women he consulted about the fashion aspects of the watch be talking about fashion if their lame discussion was anything to go by.

That was all about cross promoting that other publication that are owned by the same media company as the Verge. As was the inclusion of the editor from the food site in the notifications part. The text included mentions those people as well.

 

[FrankySavvy]

Apparently a lot of early complaints come from software, which if Apple resolves before launch could really change opinion.

Nlay Patel from the Verge had a lot of problems with the software. Imagine his review if those problems are fixed.

 

[maxsandis]

I expect problems for this version 1 release. I expect the price to come down and version 2 to be much better. Hopefully it will be usable for most things,like keeping time.

 

[KauaiBruce]

It is just like a movie review. Lots of ego. Some movies I know I will love because THAT reviewer hated it. I actually liked the reviews because they all talked about how THEY used them. We can then evaluate how each of those situations work for US.

They helped show me that I DO want it and still want the SS Link.

 

[adonis3k]

Decent review....software laggy...

http://www.theverge.com/a/apple-watch-review

 

[shk718]

that was pretty funny. And I agree. Even though it's the kind of crazy move I would have made about 20 years ago.

Neither should the women he consulted about the fashion aspects of the watch be talking about fashion if their lame discussion was anything to go by. They sounded like teenagers at the mall, using the word "like" to preface every sentence. Leaving aside their makeup, clothes and accessories, none of them wanted to be bold and make a real prediction about the fashion world's response. The best the one woman could come up with was that the fashion world would accept it if it becomes ubiquitous and popular. Well the "pants on the ground" look is ubiquitous and popular, but that does not make it high fashion. Coco Chanel is rolling in her grave at that editor/blogger's wishy washy conclusion. The masses at times can create trends but that's not the way the fashion world would prefer it.

I agree - those women were useless - their "prediction" was - if it becomes popular it will be popular - seriously???? profound! deep! he also says its heavy - um yes the stainless steel is heavy - as it should be and they also have the sport which isn't. great investigative reporting Nilay!!!!!

 

[MH01]

When the iPhone 6 scored excellent reviews all round product reviews were not broken, Apple watch does not do do well, product reviews are broken? Do you know how ridiculous this sounds? Just buy the watch and enjoy it, you have not even used it , nor even seen it yourself and you are in a crusade to discredit the format of the reviews and individuals reviewers . Have you ever thought the Apple watch may not be perfect??? Shocking concept !

Just relax till the 24th and write your own review.

----------

Apparently a lot of early complaints come from software, which if Apple resolves before launch could really change opinion.

Nlay Patel from the Verge had a lot of problems with the software. Imagine his review if those problems are fixed.

Exactly, you can only review what you are given, not the potential a product might have if firmware/software updates fix it.

I'm waiting to judge for Myself tomorrow when I pop into the Apple Store . Will depend if I keep or cancel my pre-order

----------

My problem with these "reviewers" is most of them are just bloggers with no journalistic background - they report their pre-conceived notions. The watch will ultimately be judged by us - the buying public - not some 'reviewer'. My favorite is Nilay Patel from the Verge: "I felt ridiculous wearing the Milanese Loop, so I didn’t." So Nilay you felt "ridiculous" wearing that band but not that crazy 80's studded leather cuff you have on your other arm?????? Dude should not be giving fashion advice.

Ever think that the leather cuff might have some sentimental value to him, given he has worn it over 12 years? Just saying. I doubt it's so he can be cool.

 

[eoblaed]

I expect problems for this version 1 release. I expect the price to come down and version 2 to be much better. Hopefully it will be usable for most things,like keeping time.

Interestingly, time keeping is going to be the least used feature of my Apple Watch for me.

 

[samiznaetekto]

More from Q&A session:

http://www.reddit.com/r/IAmA/comments/31vw1x/im_nilay_patel_editorinchief_of_the_verge_and_ive/

- That first iPhone wasn't fast, for example. but it was always, always responsive. This sometimes isn't responsive, and it hurts the experience.

- the metal case is already getting a little scratched. It feels like it'll be like an iPod in that regard

- So, I suck at getting the screen to activate. It should turn with a natural motion — other people get it just fine — but I'm out here waving my arms like a clown trying to get it to turn on.

- I seem to be terrible at waking the screen. It's supposed to be a quick glance, but I have to like, overact to make it work. It looks like I'm dramatically indicating boredom.

 

[Rogifan]

Part of the problem is that reviewers go to work for the same reason you do, but on top of that they have egos -- who would write anything for publication if they didn't believe someone else wanted to read it.

The thing to remember is that reviewers don't usually write the headline, assign the score or write the summary without a lot of editorial direction. And that direction isn't usually full of the pure invent of serving the readers and instead is set to get people to link to or click on the article. Writers want to be read and sites want to earn ad revenue.

In other words; ignore the headline, score and summary and read the full article.

PS, I wrote for Engadget for 9 years and reviewed lots and lots of stuff. I officially resigned in December. And yes, I worked closely with Nilay on many occasions before he left for The Verge.

---------



That was all about cross promoting that other publication that are owned by the same media company as the Verge. As was the inclusion of the editor from the food site in the notifications part. The text included mentions those people as well.

Can you explain the reason for the Verge needing 31 people to 'review' the Watch? Seemed like more of a production than an actual review.

----------

When the iPhone 6 scored excellent reviews all round product reviews were not broken, Apple watch does not do do well, product reviews are broken? Do you know how ridiculous this sounds? Just buy the watch and enjoy it, you have not even used it , nor even seen it yourself and you are in a crusade to discredit the format of the reviews and individuals reviewers . Have you ever thought the Apple watch may not be perfect??? Shocking concept !

Yeah I think reviews have been broken for a long time. The Watch reviews, especially the Verge review crystallized it for me. You don't need 31 people involved in a review. I don't need your cross promotion from other sites that are part of the Vox family. What also crystallized it for me was 9to5Mac posting a "roundup" of the reviews that picked out only negative comments. Even though most of the reviews on balance weren't that negative 9to5's "roundup" post would make you think the product was a complete failure and everyone hated it.

 

[KPOM]

Exactly, you can only review what you are given, not the potential a product might have if firmware/software updates fix it.

I'm waiting to judge for Myself tomorrow when I pop into the Apple Store . Will depend if I keep or cancel my pre-order

Which will you do first? Schedule the try-on or place the pre-order? If you do the former, you'll likely wind up waiting longer for the pre-order. If you do the latter, you might have to wait longer to try it on.

 

[samiznaetekto]

Can you explain the reason for the Verge needing 31 people to 'review' the Watch? Seemed like more of a production than an actual review.

----------



Yeah I think reviews have been broken for a long time. The Watch reviews, especially the Verge review crystallized it for me. You don't need 31 people involved in a review. I don't need your cross promotion from other sites that are part of the Vox family. What also crystallized it for me was 9to5Mac posting a "roundup" of the reviews that picked out only negative comments. Even though most of the reviews on balance weren't that negative 9to5's "roundup" post would make you think the product was a complete failure and everyone hated it.

It looks the reviews caused a major trauma to OP. Just relax. The worst is yet to come.

 

[bjdraw]

Can you explain the reason for the Verge needing 31 people to 'review' the Watch? Seemed like more of a production than an actual review.

No, never even heard of that many people contributing to a single review. Maybe 10 when there is video involved as that is like 3 or 4 alone. Even if you add 10 more for the cross promotion, that still isn't 31. But they do things differently at the Verge, which is why so many left Engadget to go there in the first place, to go in a new direction.

 

[Rogifan]

No, never even heard of that many people contributing to a single review. Maybe 10 when there is video involved as that is like 3 or 4 alone. Even if you add 10 more for the cross promotion, that still isn't 31. But they do things differently at the Verge, which is why so many left Engadget to go there in the first place, to go in a new direction.

I liked it better when Josh was running the show. Seems way too full of itself, click-baity and too much random non tech stuff maybe more suited to Vox.

 

[bounty1097]

I think Ben Thompson was spot on about the reviews:

The tech guys are griping about a lack of focus and that the watch is too hard to use, while the explicitly non-tech folks (Phelps opened by explaining that she was a self-professed Luddite) are marveling over the features and celebrating that it's just like an iPhone. It's a fascinating contrast that is repeated across all of the reviews that I read.

One that in particular that stood out to me is how most of the non-tech reviewers loved the idea of talking into their watches, while the tech folks thought it was mostly silly. It kind of reminds me of using iPads to take pictures and selfie-sticks: normal folks love it, while non-techies think it's ridiculous. I do think that, counterintuitively, technologists are in some respects more conservative when it comes to new paradigms than normal folks, and the early reviews at least suggest that might be the case with the Watch.

 

[bjdraw]

I honestly have read anything on there since they launched until this review came out. Even then, I was super annoyed by the background. My eyes are bothered by anything but a white background with black text. I usually read Engadget reviews first, but guess Apple refused to provide them a sample with all the turnover -- same thing happened when Josh left Engadget, took awhile before Apple provided review samples of the got new gadget again.

----------

The tech guys are griping about a lack of focus and that the watch is too hard to use, while the explicitly non-tech folks (Phelps opened by explaining that she was a self-professed Luddite) are marveling over the features and celebrating that it's just like an iPhone. It's a fascinating contrast that is repeated across all of the reviews that I read.

Part of it is that techies have much higher expectations.

 

[MH01]

Which will you do first? Schedule the try-on or place the pre-order? If you do the former, you'll likely wind up waiting longer for the pre-order. If you do the latter, you might have to wait longer to try it on.

I'm going to pre-order. Well, I am going to try and see which models will be available with short delivery times scales, some rumours indicated that certain model in EU might not ship for 4-6 weeks, I have about 5 favourites I would be happy to go with