xkcd

by **rwald** » Fri Aug 12, 2011 4:04 am UTC

Title Text: The bug report was marked 'could not reproduce'.

Of course, in practice the bigger problem is one-star reviews saying "I couldn't figure out how to use this app, therefore it must suck."

by **Maxpm** » Fri Aug 12, 2011 4:04 am UTC

Hover text: "The bug was marked 'could not reproduce.'"

It's still better than just "like/dislike." I think.

Also, I wonder how you would test an app like that?

At least you got the thread title correct! Not like Mr. R. "I Don't Need Quotation Marks In My Subject Title" Wald above there ::rolly-eyes::

by **LucasBrown** » Fri Aug 12, 2011 4:06 am UTC

I declare thee the winner in the thread-starting race.

by **Maxpm** » Fri Aug 12, 2011 4:06 am UTC

Second time in a row that's happened to me.

by **rwald** » Fri Aug 12, 2011 4:08 am UTC

Maxpm wrote:Second time in a row that's happened to me.

Sorry about that; keep trying and one day you'll make it, though.

by **madaco** » Fri Aug 12, 2011 4:13 am UTC

This seems to be related to the impossibility of "perfect" (for some seemingly normal definitions of perfect) voting systems.
actually maybe not, because if there was just a different field for each type of quality of the app...
like if there was a stability score, a good features score and an actually works score?
hmmmm....

by **TaylorP** » Fri Aug 12, 2011 4:14 am UTC

This comic is so true. I had the same thing happen to me with Earthqualert AND TyphoonTeller. In the same week too!

Also, OP missed quotes in his post title. However +1 for "title text."

by **Me321** » Fri Aug 12, 2011 4:18 am UTC

I live in tornado county, I want this app. someone make it please.

by **xkcdfan** » Fri Aug 12, 2011 4:21 am UTC

This is why I only read the lowest-star reviews when deciding whether to buy something.

by **Maxpm** » Fri Aug 12, 2011 4:22 am UTC

Really, I think the perfect voting system is none at all. Five-stars is bad, like/dislike is worse.

From the point of someone creating content, what does a four-star rating tell you? Nothing, really. It says, "There's some way I think this can be improved, but I'm not telling you. Na-na-na!" Sure, you can have words to accompany it, but then why do you need the stars in the first place?

The issue is how to rank the items in question. For a site like YouTube, that means figuring out which videos are the "best" and showing them to the user first.

by **wolfticket** » Fri Aug 12, 2011 4:26 am UTC

This is why voting systems need more stars. Ten at least. Out of a hundred would be ideal I reckon.

by **aaronb1138** » Fri Aug 12, 2011 4:34 am UTC

This is exactly why at NewEgg I filter the 1's first then the 2's before going any farther.

If the 1's generally exceed about 10% and have roughly the same complaint without looking suspect, I skip it and find something else.

by **unus vox** » Fri Aug 12, 2011 4:35 am UTC

I didn't get this after staring at for a minute. I kept thinking it was the 1-star rating that was somehow bogus. Then I realized it was the 5-stars that were silly. This is evidence that I need to reorganize my own priorities when it comes to technology.

by **Steax** » Fri Aug 12, 2011 4:40 am UTC

Also a frightening number of people for whom the app didn't work may not... be able to make reviews.

by **silverkitty** » Fri Aug 12, 2011 4:48 am UTC

Steax wrote:Also a frightening number of people for whom the app didn't work may not... be able to make reviews.

Not so much - were such an app to exist, anyone with a tv or radio on will know about a tornado warning, and they can readily check if their phone also alerted them. I've been through scores of tornado warnings in my area (back when I lived in midwest US) without ever having actually seen a tornado. The biggest problem with this app's reality is that it is completely redundant with free services provided by tv and radio stations by federal law (in the US, anyway). On the other hand, as an illustration of the logical extreme of failure of voting systems, it's a good fictional app

by **anderswc** » Fri Aug 12, 2011 4:57 am UTC

Anyone else notice that this app is listed on the App Store (iPhone specific, right?), but the developer's name is DroidCoder2187? Obviously that means this is a WP7 app

by **Sean Quixote** » Fri Aug 12, 2011 4:57 am UTC

Funnily enough, my first thought after reading this was that it could just as easily be a comment on weather predictions as statistical analysis. Or really it's both, since weather predictions are essentially statistical analysis on some level.

by **jpk** » Fri Aug 12, 2011 5:08 am UTC

Of course, in practice the bigger problem is one-star reviews saying "I couldn't figure out how to use this app, therefore it must suck."

Considering that an app is by its nature a product that should be a model of transparent design, yes, that's a reasonable basis for a one-star review. If I can't figure it out, for me it sucks.
On the other hand, if I can't figure it out and a bazillion other people can, I guess it sucks to be me...

Title Text: The bug report was marked 'could not reproduce'.

Did you hear the one about bug tracking at the contraceptive company...

by **Logik&Verstand** » Fri Aug 12, 2011 5:13 am UTC

xkcdfan wrote:This is why I only read the lowest-star reviews when deciding whether to buy something.

I do that too, but I also read the five star reviews. If it's good for some people, it might be good for me too.

by **addams** » Fri Aug 12, 2011 5:44 am UTC

silverkitty wrote:
Steax wrote:Also a frightening number of people for whom the app didn't work may not... be able to make reviews.

Not so much - were such an app to exist, anyone with a tv or radio on will know about a tornado warning, and they can readily check if their phone also alerted them. I've been through scores of tornado warnings in my area (back when I lived in midwest US) without ever having actually seen a tornado. The biggest problem with this app's reality is that it is completely redundant with free services provided by tv and radio stations by federal law (in the US, anyway). On the other hand, as an illustration of the logical extreme of failure of voting systems, it's a good fictional app

I spent one frightening summer in Tornado country. I saw lightening strike the barn. One bolt was visible across all five rods. Then, the lightning hit the house.
Yeah. That is some frightening stuff.
Then, the Locals started talking about winter. 40 degrees below zero?!? I developed an interest in weather.
I know nothing about technologies. I type. Apps? Yes. I have heard of them. I have never used one.

by **Eutychus** » Fri Aug 12, 2011 5:53 am UTC

Another problem, especially with ratings on sites like TripAdvisor, is that people are often more motivated to vent their spleen when they weren't happy with a hotel than to write in with a positive report when things went well.

by **mjc0961** » Fri Aug 12, 2011 6:36 am UTC

That's not really a problem with averaging star ratings as it is a problem with reviewers in general. Any other rating system and it'd still be skewed. 3 likes and 1 dislike, for example. No matter what rating system you have, there will always be people who review improperly (I can't count how many times I've seen product "reviews" on Amazon that give a product one star and then the review is just complaining about the 3rd party seller without mentioning the product at all) which screwed up any overall rating.

by **radstar** » Fri Aug 12, 2011 6:41 am UTC

I honestly laughed out loud at this comic. I'm not sure why this one and not too many of the others (even though I have read them all and love the comic)??? I think that the straightforward comment plus rating showed perfect bitterness. Do ratings really satisfy our resentments this much?

by **addams** » Fri Aug 12, 2011 6:46 am UTC

mjc0961 wrote:That's not really a problem with averaging star ratings as it is a problem with reviewers in general. Any other rating system and it'd still be skewed. 3 likes and 1 dislike, for example. No matter what rating system you have, there will always be people who review improperly which screwed up any overall rating.

Yeah. That is why God created Statitions.
They know how to even the numbers out.

by **Istaro** » Fri Aug 12, 2011 7:31 am UTC

silverkitty wrote:
Steax wrote:Also a frightening number of people for whom the app didn't work may not... be able to make reviews.

Not so much - were such an app to exist, anyone with a tv or radio on will know about a tornado warning, and they can readily check if their phone also alerted them. I've been through scores of tornado warnings in my area (back when I lived in midwest US) without ever having actually seen a tornado. The biggest problem with this app's reality is that it is completely redundant with free services provided by tv and radio stations by federal law (in the US, anyway). On the other hand, as an illustration of the logical extreme of failure of voting systems, it's a good fictional app

Isn't Steax referring to the idea that there may be people for whom the app didn't work but who didn't post reviews because they'd been killed by the tornado?

by **Alltat** » Fri Aug 12, 2011 7:41 am UTC

wolfticket wrote:This is why voting systems need more stars. Ten at least. Out of a hundred would be ideal I reckon.

For user reviews, there's no point in even having five stars. How often do you see anyone actually rating something at 2 or 4? If people bother to write a review, it's usually because they either really like it or really dislike it, resulting in a 1 or 5. An "in between" option of 3 is useful in theory, but how many user reviews do you really get from people who are indifferent?

In general, I've found that the ratio of likes to dislikes usually gives a decent idea of how good something is. If something gets >90% likes, it's usually good, and anything that exceeds 50% dislikes is typically horrible.

by **imakimuret** » Fri Aug 12, 2011 7:45 am UTC

This is why I check the low starred product reviews first on Amazon. Though mjc0961 is correct, the remaining good negative reviews are very helpful.

by **LazyMonk** » Fri Aug 12, 2011 8:06 am UTC

This reminds me of the "Anti-Satellite" spray can from a few years back -- 100% satellite deterrence guaranteed(!!) double refund if you do get hit(!!)

by **rwald** » Fri Aug 12, 2011 8:22 am UTC

TaylorP wrote:Also, OP missed quotes in his post title. However +1 for "title text."

I noticed that after realizing that Maxpm and I had posted at nearly the same time; he included the quotes, while I didn't. I felt it wasn't worth the "Last edited by" tag at the bottom to change it, though.

jpk wrote:
Title Text: The bug report was marked 'could not reproduce'.

Did you hear the one about bug tracking at the contraceptive company...

Yes, we did.

by **Red Hal** » Fri Aug 12, 2011 8:39 am UTC

Alltat wrote:For user reviews, there's no point in even having five stars. How often do you see anyone actually rating something at 2 or 4? If people bother to write a review, it's usually because they either really like it or really dislike it, resulting in a 1 or 5. An "in between" option of 3 is useful in theory, but how many user reviews do you really get from people who are indifferent?

While I agree in principle, in practice what is needed is a way to filter out all comments that contain "lol", "sucks", "fukn", "!!!", "!!!*" or similar.

by **Plasma Man** » Fri Aug 12, 2011 9:03 am UTC

I'm just waiting for the "Eternal Life" app. That would get entirely 5-star reviews.

by **Melde** » Fri Aug 12, 2011 12:01 pm UTC

Me321 wrote:I live in tornado county, I want this app. someone make it please.

If you have an Android phone, I imagine it wouldn't be hard to sign up for an alert system to text/email you when there's a tornado, and then use Tasker to sound the alarm when one arrives.

by **SpringLoaded12** » Fri Aug 12, 2011 12:14 pm UTC

The reverse case can also happen; I'm off to college soon so I was browsing around online for a mini-fridge, and found one with several one-star reviews saying that it had broken within a week or a month, and then one four-star review that explained the mistake the other reviewers had made that caused it to break so fast.

by **muninsfire** » Fri Aug 12, 2011 12:30 pm UTC

Plasma Man wrote:I'm just waiting for the "Eternal Life" app. That would get entirely 5-star reviews.

"Trapped within an endless abyss of pain. Lord Cthulhu has taken over the earth and all within it are tormented by eternal madness. Death--death! I remember death, but I can no longer feel its sweet release, the nepenthe to wash away the torment!"

"1/5"

(Although yeah, this would be an edge case)

by **Tormuse** » Fri Aug 12, 2011 12:31 pm UTC

Three statisticians were in a forest, hunting deer. They spotted one and prepared to fire their rifles at it. The first statistician fired three metres to the left of it. The second statistician fired three metres to the right of it. The third statistician jumped up and said, "We got him! We got him!"

by **psinnott** » Fri Aug 12, 2011 12:50 pm UTC

I think part of the problem is everyone is using ratings differently.My app got a comment

" 1* until App2SD support is added. 1* until App2SD support is added. Shame, because the app is really, really good."

That is very different to the same one star rating with no comment attached.

You can rate films in the app and generally I rate between 2.5 and 4 since I think 5 or 1 should be quite rare but for apps people usually go 1 or 5 except for the people who use 4.

by **EXi13** » Fri Aug 12, 2011 12:58 pm UTC

Really, what we need to be able to do, is have a star voting system on each user's comment.

But then, in case that is flawed, we'll need to rate those star ratings. Once this has been done for, oh... say... 10 or so recursions, we can average out all the star ratings for each original comment and have something else that doesn't tell us exactly what we need to know.

Simple

by **algorerhythms** » Fri Aug 12, 2011 1:02 pm UTC

I hate to play the "Get out of my head Randall" card, but I live in Oklahoma, near where a tornado struck in May. I was watching the local TV station at the time, so I knew about the tornado, but I noticed that the Weatherbug app on my phone didn't warn me that a tornado warning had been issued until the storm had already passed by. After the storm was over, it freaked out and warned me several times, though.

by **ginbot** » Fri Aug 12, 2011 2:21 pm UTC

I live in Alabama and would NOT say this would be a redundant app; you not always listening to the radio, watching TV, or able to hear a siren. That said, in my case it wouldn't help (You can't bring cell phones in the building) ... but you can't hear the siren either (too much concrete, also radios don't work in majority of the building). So, a mass email is the most effective tech for me.

Personally, I read high and low reviews ... and it depends on the product. For hard drives on NewEgg, the low reviews are important for me to gauge the reliability. However, if it something more subjective (hotel, book, movie), a lot of low reviews sound like whining especially if only one person is complaining about it.

And, if I saw 4 reviews ... well, the sample size isn't large enough to make a judgement either way except that it isn't popular -> caveat emptor.

by **The Origamist** » Fri Aug 12, 2011 2:51 pm UTC

I live in Illinois, and back in June there was a situation where this app would have been useful. I live nearby our town's tornado siren, so I'm used to hearing it promptly and loudly when there is a tornado. One storm was so bad that the power went off, but the tornado siren (which has its own generator) never sounded, so I went to sleep assuming that it was just a bad storm. The next day, I found out that a tornado really had gone through, even knocking down a couple of trees nearby (luckily, no one was hurt). The siren hadn't gone off, though, because the tornado technically hadn't "touched down," which in this case means the tip of the cone was hovering ominously fifty feet from the ground instead of two, which meant it could still do a lot of damage. There were a lot of complaints about that one.

Who is online