If I didn't rate my beers in comparison to similar ones then I'd be able to predict my rating of almost any beer to within one point without even trying it. Some day I'll make a stat that shows each person's bias towards different types of beer. I already made some equations to measure something like this and I was amazed at how consistent some people are within general types of beer. Maybe I'll put the stat on the member info pages and call it "Rater Predictability" [}:)]

each beer should be rated individually on that particular beers merits or drawbacks irregeardless of style. par example, if i have an ipa that juts not up to snuff, i'm going to rate that particular ipa on it's own and not compared to any other ipa i've had. this is, how i feel, it should be done.

quote: Originally posted by Sigmund I rate on a purely hedonistic scale, regardless of "style" - the beers that brings me the greatest happiness with regard to aroma and flavour and mouthfeel (appearance comes in second) get the highest score. I drink beer, I don't drink "styles". Besides, some of the best beers I've had are difficult to place in rigid "styles" - they are "borderline" beers. In Norway I buy and rate every new beer I can get my hands on, as the selection here is very limited. But abroad, with more good beers around than I can handle, I don't search out pale swill lagers. But pale lagers are everywhere - and sometimes I'm very thirsty in a place where they are the only option. Of course I'll rate them, and as they have served as welcome thirstquenchers, I probably give some of them higher scores than they deserve (I'm only human). Just don't expect me to give any drinkable swill a "perfect" score because "that's the style"! Besides, if we don't rate ANY mediocre beers, how can we justify the higher scores for the better beers?

what he said.

quote: Originally posted by Flashpro If I didn't rate my beers in comparison to similar ones then I'd be able to predict my rating of almost any beer to within one point without even trying it. Some day I'll make a stat that shows each person's bias towards different types of beer. I already made some equations to measure something like this and I was amazed at how consistent some people are within general types of beer. Maybe I'll put the stat on the member info pages and call it "Rater Predictability" [}:)]

Flash, you already can see by looking through the top 100 beers, that style gets weighted by default. The top beers are all from predictably "good" styles. If folks rated from 1-5 in each style, then you would lose ALL ability to figure out what their favorites styles are... something that I think would take away from the stats of the site. I don't know why being able to predict someone's rating within X is a bad thing. In the end, both within style and absolute opinons factor into the rating when all is said and done. I'd like to be able to see what a person rated a beer on the ratings page, relative to what their ratings are within that style. I think it would be cool to show the "outliers" from each person's ratings of a beer. In other words, which of the beers that person rates are in the upper 5 percentile within that style (yes, you show the "max" now, but more than one or two often are in that top 5 percentile, as well as the bottom. For example: My range for Saisons is 3.0-4.0 (min-max). The fact that Sly Fox's Saison Vos was a clear standout, as was Flying Fish's Farmhouse Summer (in a "low" sense within that style) is interesting. The same could be done for other styles. I've had 19 Sweet Stouts. The range is 2.7-4.0. The 95% confidence range is 3.02 to 4.45, meaning that anything rated outside that range is a "standout" in my profile: Youngs Double Chocolate Stout (4.5) and Abeelsen Stout (2.7). You'd have to play with the confidence interval, since 95% is pretty restrictive and we're talking about subjective ratings.... but you get the point. (In the case of Pale Lagers, my range goes from 1.0-3.0. The 95% confidence range is from 0.93 to 3.71, so clearly there won't be any outliers high or low in that example with only a 95% confidence interval used.) With such a calculation, you can still get valuable information from any rater within a style regardless of the fact that they like a style better than another and rate that way. If the raters use the full range within any style, you will lose the ability to find out what styles that individual likes more or less... and which styles are generally preferred by drinkers. That's my two cents.

One other point that is maybe not obvious to everyone: if you rate "overall" and confound the within style merit of the beer with your own style preferences (and therefore will never likely rate a pale lager a 5 or a stout a 1), this can easily be "normalized" out over a 1-5 range to get what Flash is after (essentially broader ranges within a style). Mathematically, that's not hard to do. On the contrary, if you rate a beer ONLY within the style (like Flash supports), you cannot mathematically figure out what people's biases are, nor what the pure "enjoyment" value of a beer might be (like Sig mentioned), as the ratings then only have meaning within style (it also can lose those beers that don't fit a style). In other words, the top 100 beers of the site and any cross-style comparisons are completely meaningless if such a rating method is used. Ultimately, that's what Flash is calling apples to oranges, but I like the top 100 list and find value in it, as well as the within style rankings.

quote: Originally posted by eaglefan538 One other point that is maybe not obvious to everyone: if you rate "overall" and confound the within style merit of the beer with your own style preferences (and therefore will never likely rate a pale lager a 5 or a stout a 1), this can easily be "normalized" out over a 1-5 range to get what Flash is after (essentially broader ranges within a style). Mathematically, that's not hard to do. On the contrary, if you rate a beer ONLY within the style (like Flash supports), you cannot mathematically figure out what people's biases are, nor what the pure "enjoyment" value of a beer might be (like Sig mentioned), as the ratings then only have meaning within style (it also can lose those beers that don't fit a style). In other words, the top 100 beers of the site and any cross-style comparisons are completely meaningless if such a rating method is used. Ultimately, that's what Flash is calling apples to oranges, but I like the top 100 list and find value in it, as well as the within style rankings.

Woah, you're really starting to lose me now. The Overall rating is ALWAYS supposed to be regardless of style. You can't rate that the overall in comparison to just that style, well I suppose you could. But the Overall score is precisely the reason why rating the other 4 in comparison with similar beers works! That is the key factor to why rating by style will ALWAYS cause the beers of crappy styles from being the top beers on the site. Therefore you CAN mathematically figure out what peoples' biases are, thanks to this final number that will pull that style average up or down. Let me give some hard facts: Say I rate 10 Pale Lagers, and I do it using my method of comparing similar beers. Using my judgement from years of drinking I would have a clue of what is best, worst and normal for these types of beers, and I'd rate the worst of the Pale Lagers in the 1-2 range for Aroma, Appearance, Mouthfeel and Flavor. I Would give the "better" Pale Lagers scores of 4-5. The result is a distribution of ratings where my average Pale Lager is somewhere around a 3.0 when NOT factoring in the Overall score yet. Of course, it wouldn't be exactly be a 3.0, but theoretically I should have given at least one "1" in each rating aspect (aroma, app., mf, flv) and at least one "5". To not give out any 1s or 5s (for each rating aspect, not the total beer score) would mean that there exists a beer that is better in that aspect, but that is not possible unless you prove it, which you didn't. Unfortunately, if say some day you do find a beer that is better in one aspect then you would have to adjust your previous ratings to fit them within your new scale. That's the way subjective research goes. OK, so now I rate 10 IPAs in the same manner, with at least one 1 and 5 for each aspect. I end up with an average that may be similar to the Pale Lagers. But wait! We haven't factored in the Overall rating aspect. I might give a range of 1 to 2 for the Pale Lagers, and 3 to 4 in IPAs. I therefore made it nearly impossible for my best Pale Lager to be better than my best IPA, and the average for the lagers would probably be lower than the IPAs. Now lets look outside the world of beer. In just about all competitions you will have judging within categories. Just look at dog shows, garden shows, wine competitions etc. They always score relative to the type of specimen. That is the only thing that makes sense for me. And then at the end, because you always have some bias, you can select a "best of show". BeerPal's top dog will always be reasonable thanks to the Overall rating. And finally, check out any beer rating book and you'll see that they judge by comparing similar types of beer. I'm not the only crazy one.

quote: Originally posted by Flashpro <blockquote id="quote"><table width=90% cellpadding=10><tr><td bgcolor=FFFFCC><font size="1" face="Verdana, Arial, Helvetica" id="quote">quote: Originally posted by eaglefan538 And finally, check out any beer rating book and you'll see that they judge by comparing similar types of beer. I'm not the only crazy one.

My point wasn't that your method is crazy - bp.com is just fine with the way things are, imo. I think there is a lot to be said for within style ratings. Again, I factor style into my ratings. However, I don't think that ONE fifth of the rating component (which is what the "overall" rating is on bp.com) does justice to properly weighting the "overall" apples to oranges comparisons - in a pure mathematical sense. Mathematically, you can take absolute ratings (for aroma, mouthfeel, flavor, appearance, whatever is ranked in an "absolute" sense, ie. NOT within style) and normalize them to calculate a wider range (1-5) for a within style scale. So for the purposes of a website, ranking "overall" (independent of style) would work just fine with a simple normalized calculation (all you need is the right distribution model - normal/gaussian??? - and the stats - sigma/mean) showing the 1-5 range score. This is especially an easier way to go, when you figure most people aren't "experts" in every style and aren't going to go back and bump their scores for each beer they've had of any one (or more) style(s). Example: Lambic "absolute" appearances range from 5-8, let's say (I don't know that I agree with that, but...).... Then that range can correspond to a 1-5 by using the right distribution model (normal?) and statistics (mean, sigma). You do this for each category and you can figure out the within style values (calculated) from the "overall" values (no calculation). The latter would be used to create top 100 lists. The normalized values would be used for broader within style ranges, elminating the problem of having all lambics limited to scores of 3-4, for example (but wouldn't change the sort order within any style).

I think we are making this more complicated than it needs to be. I think everyone here, in some sense, both rates to style AND rates to pleasure. Those styles you like will tend to have higher scores in general to those you don't like. But if something stands out wrt style, it probably shows. Just rate 'em as you like 'em.

quote: Originally posted by doulos31 I think we are making this more complicated than it needs to be.

Yes, it is *just* a beer hobby site... at least that's how I use it. However, if you really want to use the "data" and draw conclusions (across style, in particular), it is important to understand what the data represents. Since no one is using beerpal for strict stastical studies, I agree... it doesn't matter. Sorry for the mindstretching. Some engineer you are.

When I rate something (or if I do again), I'll just rate whatever I'm drinking. But I usually qualify make rating by saying whether or not the beer is from my favorite kind or not.