An inconclusive and meandering thought process on the question - Are Beer Reviews Bullshit?

My favorite science-fiction blog, io9, recently published a fun little article titled Wine Tasting is Bullshit. Here's Why. where they humorously point out a number of studies where wine tasting experts contradict themselves, or they fail to differentiate between red and white wine when white wine has been dyed with food coloring. They point out one review that describes a wine like this, "The overall character is that of a sex loaded scarlet."

One wine maker, wanting to understand how his wine could win accolades at one competition and be ranked incredibly low in another, decided to give multiple glasses of the same wine to a panel of judges. None of the judges rated the wine with the same results, and it's implied that none of them realized it was the same wine. Now, it can be said that wine changes character over the course of minutes once you allow it to breathe in your glass, so there is some level of subjectivity in this experiment. Another wine maker replies that:

'[W]ine ratings are influenced by uncontrolled factors such as the time of day, the number of hours since the taster last ate and the other wines in the lineup.' He also says critics taste too many wines in too short a time. As a result, he says, 'I would expect a taster's rating of the same wine to vary by at least three, four, five points from tasting to tasting.'

The article also points to a study about how price of wine, or the atmosphere where food is consumed will drastically affect your perception of the taste. "Research out of Cornell University's Food and Brand Lab has shown that people will rate food as more enjoyable if it's consumed in the relaxed atmosphere of a fine dining environment, as opposed to a noisy fast food restaurant."

Perhaps my favorite part of the article reads "MIT behavioral economist Coco Krume recently conducted a meta-analysis of the classifiers used in wine reviews, and found that reviewers tend to use "cheap" and "expensive" words differently. Cheap descriptors are used much more frequently, expensive ones more sparingly. Krume even demonstrated that it's possible to guess the price range of a wine based on the words used in its review."

As I was reading this I couldn't help but think about beer reviews. Are beer reviews bullshit? At the end of the article, the sci-fi geeks encourage us to drink beer instead of wine. It was fate, I knew I had to write something about it. Just a forewarning, I don't reach any conclusions, I just talk a lot, and I also don't really care about answering that question.

We have reviewed beer on this site. To fill you in a little, we realized that our beer reviews are some of our least read posts on the blog. As a writer, I actually seriously dislike reading what other people think about beer, I couldn't care less how someone else perceives the nuances of hop complexity in a review. So why do we write them here? I'm not entirely sure, but I have heard a couple people, here and there, say that they went out and bought something because they read one of our posts. Thanks to some of our sharp friends, and to some reactions from brewers, we have drastically changed the way we write our reviews. I'm not at all saying that we're on to something, that we do it correctly, or anything like that. But if beer reviews are our least read posts, then perhaps other people agree with me that they're boring to read. So, ours have become slightly more story oriented, and we no longer rank beers with number.

I don't know if this is true, but I've heard that in the wine world, what one or two expert reviewers say regarding a wine can make or break the sale of that wine. I doubt that any beer judge has the street cred to crush a particular beers market however, I do think that when people see an instagram photo of The Beer Wench drinking an Odell Tree Shaker, lots of other people also then want to drink that beer. There have been times that I have purposefully not bought beer because someone that I respect says that a certain beer is no good. I trust that person, but can certainly disagree with their assessment too. Usually, this trust revolves around me only have $10 to spend, and should I buy this beer with my money. I rarely look at sites like Beer Advocate and Rate Beer to help me determine if I should buy a beer or not but what I think is interesting is that these sites average out the ratings. So, thankfully, if you do go to their site, you don't have to read through all the reviews, all the poetic waxing, about beer. You can see the average number and move on.

To take a short little rabbit trail. My favorite podcast, Radio Lab, had an interesting program called Emergence about "The Wisdom of the Crowd," uninformed guesses by groups of people and their accuracy. Scientist Sir Francis Galton, who founded Eugenics, believing "that only the better classes should vote, rule, or even have children" spent a day at a county fair and watched a competition where uninformed people guessed the weight of an ox. None of the 800 guesses were correct. Galton believed, because none of the people were experts, that if you "added up a lot of of uninformed, unexpert, individual opinions, you would get a very uninformed, unexpert, group opinion." Of course the founder of Eugenics would think this, right? As it turned out, his theory was wrong, because when he averaged all the guesses, the average of the group was only one pound off from the actual weight of the ox.

Studies like this have been repeated and repeated and the results are often similar, the group average was better than any one, individual guess. This research has been used to solve many problems including, as Radio Lab points out, allowing Google to create the algorithms that decide the best  search results when we get online and do a google search."Google isn't deciding that...this is the premiere site...Collectively, all the people who are authoring documents on the web have decided that this is the best site on the web by choosing to link or not link. Basically the intelligence of Google is an emergent property of all these local decisions to link, none of which were made, until recently, with the intent of influencing Google." Anyway, that got me thinking about Beer Advocate and Rate Beer and that perhaps there's more to them than we might realize? Yes, the people who use these sites do gravitate toward higher ABV, more hops, and big rich stouts above the easier drinking beers. Yes, most of them are not "experts" in that they haven't been in any academic training for judging beer. But for what it's worth, maybe there's something about their average score that holds extra weight? The average score tells us something that the individual judge cannot. Perhaps the scores do not actually add up to "a very uninformed, unexpert, group opinion."

I know that brewers go to Beer Advocate to look up what people think about their beers, and sometimes they are disappointed when people dislike what they've made. It's understandable, brewers are artists as much as they are scientists and when you create something you have invested mental and emotional energy. In a post on Beerpulse, Jacob McKean, founder of the anticipated Modern Times Beer (brewery) says "Something has always bothered me about beer industry gatherings: the inevitable negative comments about beer ratings. Put enough brewers in a room together, crack open a few beers, and without fail, someone will bemoan the unfair, unqualified reviews of their beers on ratings sites." He goes on to say that once his brewery opens he will welcome the unbiased "judgments of people who, in many cases, have tried a tremendous amount of craft beer, know a great deal about brewing, and also—crucially—have no investment in whether or not I succeed." He goes on to say that beer competitions are overrated. "[B]ut enough people whose views I trust—many of whom stand to gain a great deal by winning an award—agree with me to make me think that beer competitions are exactly like wine competitions: more or less random."

All this to say that I'm not attempting to reach any conclusion here. Are beer reviews bullshit? I like to lean toward the "probably not" side of things, but I also just don't care. But there are aspects to the whole process that I do care about. As a writer I understand there is much to weigh when writing about a particular beer. Are there things worth considering when I write that will affect what I say? Should I be 100% forthright about everything on my mind when I'm reviewing a beer? Is that honesty or guilt? Is it important to do that? Wouldn't it be boring? When I talk about a beer am I doing it to show off or to fill in our readers? Should some things stay off this site if I don't like a beer? Why waste my time talking about a beer I didn't like? What does it accomplish when I write something, good or bad, about a beer? Do the readers care? Do the brewers care? Is this site just a journal log? These things all go through my head before I write about a beer, and I'm happy to share those thoughts with you.

Perhaps what the article on io9 failed to mention is that taste is subjective. My wife is much more sensitive to hops than I am, bitterness stands out like a sore thumb to her. I am sensitive to certain types of off-flavors. We will each appreciate beer, or dislike a beer for different reasons. So, taste as much beer as you can/want and just enjoy it.

I'd like to finish with a statement from my friend Win Bassett. "As with all things in life, drink what you like, not what someone else says you should like. Read what you like, not what someone else says you should like. Write what you like, not what someone else says you should write. You get the point."

- - -
Stay updated with our email updates and like us on Facebook. We're also on Twitter and Instagram at @focusonthebeer.