In October of 2017 we published an article called “When a 90 is not a 90” which explored how various review entities that utilize the 100 point rating system stacked up from 2015 through 2017 to see what each looked like for average scoring, most frequent scores and etc. to determine how much score inflation was occurring. This article just reported data collected.
A month later, we published a follow up article which included opinions about each of the sites and the results from the data collection.
In December of 2018, we posted an article revisiting the sites a year later and tracking their scores over that year to see what changes, if any, had occurred. We also included some new sites that weren’t in the previous articles. These sites are ones that didn’t have a 3 year history of scores, ones that had just started scoring reviews the past year or had just come to our attention. This article was a mix of collected data and opinion.
This year, we are following up with data from 2019 (December 2018 through November 2019). This year, one new site is added and one is dropped. This article will be a mix of collected data and opinion.
REVISITING
CIGAR AFICIONADO
Data: The average score over the last 12 months has been 89.03 compared to
88.81 last 
year and 89.12 from the three years prior, so in between the two previous averages. In regards to the most frequent score given over the previous 12 months, the number moved up to 90 when it had been 89 the two previous periods. 89 was the next most frequent score with 88 being the third most frequent. Cigars still need to score a 91 to start differentiating themselves from the pack in their scoring range.
Opinion: Cigar Aficionado is very consistent in their scoring over time and this last year has been no different. They are also one of the few review sources that uses a decently spread out scale. Regardless of what you think of their review process or what they review, you can at least take solace in the fact that they keep the scoring consistent.
BLIND MAN’S PUFF
Data: The average score over the last 12 months has been 91.05 compared to 90.74 last 
year and 90.22 from the three years prior, so a little more than a quarter point increase from last year and just under a full point increase from the 2015 through 2017 period. In regards to the most frequent score given over the previous 12 months, the number is now 90 compared to 91 and 90 from the previous two periods. The score of 91 was the second most frequent given and 89 the third most frequent. Cigars still need to score a 93 to start differentiating themselves from the pack in their scoring range as compared to a 92 from the 2015 through 2017 period.
Opinion: The continual increase in scoring isn’t much of a surprise. They appear to be cycling through new reviewers, and with this I think it is harder for them to maintain a structure of how scoring is actually performed. As I mentioned last year, the site could use a re-centering of the scoring by revisiting how reviewers submit their scores or re-educating the reviewers on the existing system.
CIGAR COOP
Data: The average score over the last 12 months has been 89.96 compared to 90.78 last year and 90.84 from the three years prior, so slightly more than a three quarter point decrease. In regards to the most frequent score given over the previous 12 months, the number is now 90 when it was 91 last year and the previous three years. The score of 91 is now the second highest with 92 being the third. Cigars continue to need to score a 93 to start differentiating themselves from the pack.
Opinion: The site made additional modifications to its scoring system for this year in order to try to drive a lowering of scores which the changes from the previous year didn’t seem to accomplish. I have an opinion on a second reason for the drop in scores, but I’ll save that for my final thoughts. The site still has a pretty narrow scoring range and it would be nice to see that expand.
CIGAR DOJO
Data: The average score over the last 12 months has been 88.56 compared to 88.95 last year and 90.22 from the three years prior, so getting close to a half point decrease from last year and almost a point and three quarters from the previous period. In regards to the most frequent score given over the previous 12 months, the number remains at 89 when it was at 90 during the 2015 to 2017 period. 90 is new the second most frequent score given with 88 and 87 being tied for third. The distribution of scores are a little odd this year as 93 was scored one more than 92, so it’s hard to find the point where scores differentiate each other, especially since there was only one review higher than 93.
Opinion: There have been some more changes in regards to the reviewers that are contributing to the site, which I think helps them be a bit more discerning in their ratings. In speaking with the team over there, the Managing Editor definitely is proactive in managing the scoring system balance. It’s good to see a continued reduction in inflation as last year I mentioned it would be something to keep an eye on to see if the trend continues, which it has.
HALFWHEEL
Data: The average score over the last 12 months has been 87.23 compared to 86.62 last year and 86.69 from the three years prior, so a decent increase of just over half of a point. In regards to the most frequent score given over the previous 12 months, the number 88 is the same as last year and the previous three years. The second (87) and third (89) most frequent scores were quite close to the most frequent with each being just one less than the previous.Cigars now need to score a 92 to start differentiating themselves from the pack which is inline with how things were during the 2015 through 2017 period compared to 93 last year.
Opinion: In regards to the average score, the site has been very consistent over the years, so I was a bit surprised to see the size of the increase. Digging a little deeper, during the last two periods, there were more cigars that were scored low (such as a 50 and some in the 60’s) than in this current period, and those low scores could definitely have pulled the average down in those previous periods. It’s worth keeping an eye on to see how things trend, but if few cigars get handed those real low scores, the average from this year may actually be the norm.
THE CIGAR AUTHORITY
Data: The average score over the last 12 months has been 92.78 compared to 92.22 last year and 91.57 from the three years prior, so an increase of slightly over half of a point from last year and an increase of nearly a point and a quarter over the previous period. In regards to the most frequent score given over the previous 12 months, the number is now 93 as compared to 92 from the previous two periods. The second most frequent score is now 94 with the third being 92. Cigars continue to need to score a 95 to start differentiating themselves from the pack compared to 94 from the 2015 through 2017 period.
Opinion: As a site that uses the reviews to promote selling the cigars from their stores, you can see where a continuing inflation of scores would be beneficial. If this keeps up, we might see an average score of 100 by the year 2031. As mentioned last year, there are two options to combat the inflation which are to re-calibrate the scoring to bring it back down or extend the scoring scale past 100. As silly as pushing past 100 seems, in a culture of those always wanting more, someone is going to do it, and why not Barry?
TINY TIM’S CIGAR WORLD
Data: The average score over the last 12 months has been 92.03 compared to 90.28 last year and 91.69 from the three years prior, so a large increase of a point and a quarter over last year and just over a quarter of a point from the three years prior. In regards to the most frequent score given over the last 12 months, the number is 93 which is the same as last year. The second and third most frequent scores are now 92 and 91 as compared to 91 and 92 previously. 95 was the fourth most frequent score given, so it’s hard to determine where a cigar starts to differentiate itself in this system.
Opinion: Shortly after my original scoring article was published, Tim chose to revamp his scoring system. The results were very interesting as the average score had dropped but the most frequent score given had increased. With the change to the system, I find it really odd that there was such a significant increase this year. Last year I said we’d have to keep an eye on the trend and boy does that ever apply here. I applaud Tim for looking into his system and making the changes he did last year, but it’s already time for another review. This may just be a case of old habits die hard.
CIGAR SMOKE
Data: The average score over the last 12 months has been 90.92 compared to 92.83 last year, so the recent scoring has seen a decrease of nearly two points. In regards to the most frequent score given over the last 12 months, the number is 93 which falls in line with the previous year. The second and third most frequent scores are now 92 and 94 which is opposite of how they were last year, but with only one review difference between them, not much of a change. Cigars still need to score a 95 to start differentiating themselves from the pack.
Opinion: It was good to see such a significant drop in the average scoring over the previous year. Even though the most frequent score is still the same, the range of lower scores being handed out has widened which has helped. One thing to note is that this site has been offline since September, so we’ll have to see if that is something that is temporary or permanent.
STOGIE PRESS
Data: The average score over the last 12 months has been 92.4 compared to 92.48 last year, so quite consistent over the sites history. In regards to the most frequent score given over the last 12 months, the number is 93 which than the 91 it had the previous year. The second and third most frequent scores are 92 and 94 which makes more sense compared to the scattered distribution from last year. Cigars still need to score a 95 to start differentiating themselves from the pack.
Opinion: The comparison between years is a bit tough as the site had just recently started scoring before last years evaluation. With four times as much data this year, we’re getting a better picture at the scoring trends. The site still scores quite high but that seems to be the design of the scoring system. It’s just tough to use the 100 point system with criteria like this because it’s almost as if you’re preying on those that don’t understand how your scores are derived, but have a familiarity with the idea of the 100 point system.
OVERALL
Comparing average scoring of all of the sites between their most recent scoring and last year, five sites increased their average and four sites decreased their average. The increases for Cigar Aficionado, and for the post part halfwheel, I think can just be chalked up to normal variations in scoring over time as they are not too far out of their range, or in halfwheel’s case, have some outliers that could explain the variation. Blind Man’s Puff and The Cigar Authority’s increases just appear to be a trend for them as they continually inflate scoring over time. The increase for Tiny Tim is alarming based on the scoring changes he made two years ago. You have to wonder if the cigars he smoked were just that much better or if he returned to his previous scoring practices. The decrease at Cigar Coop is significant and would like to see if that trend continues and it’s getting close to where I’d like to see the scoring be. The continuing decrease at Cigar Dojo is nice to see as the review team they have now seems to be more discerning along with the management of the scoring being something they stay on top of. The decrease at CigarSmoke was dramatic as the scoring range broadened, but we’ll have to see if the site continues on so we can see if this is a positive trend in the downward direction. The slight decrease at Stogie Review doesn’t lend to much insight as we would need to see another year to see how things shake out.
Comparing most frequent scores given from all of the sites between their most recent scoring and last year, four sites remained the same, three sites increased and two sites decreased. halfwheel and Cigar Dojo don’t really surprise me in regards to maintaining their frequent scores based on where they were in regards to their previous averages, but Tiny Tim and CigarSmoke remaining the same was a bit surprising due to their respective increase and decrease in average. It just turns out that the scores higher (Tiny Tim) and lower (CigarSmoke) than the most frequent saw increases. Blind Man’s Puff decreasing was a bit of a surprise due to their average increasing, but it’s due to the increase in scores of 94 and above. Coop decreasing makes sense and helped attribute to the average dropping. The Cigar Authority increasing doesn’t surprise me at all with the continual inflation happening there. The increase at Stogie Press but decrease of average makes sense since there is much more data this year and it spreads things out a bit more. The increase at Cigar Aficionado is understandable as their quantities of scores at the top of their curve are so close it could move around frequently within a 3 point window.
NEWLY ANALYZED SITE
HOW BOUT THAT CIGAR
Data: This is a site that launched at the beginning of 2019. It appears to be a team of two and the reviews consist of either one or the other reviewer or sometimes both as a team. The average score across 33 reviews is 88.79. There were 7 reviews that received a 91, 5 were given an 89 and 4 were given a 90. With the distribution of scores, it’s still too early to be able to determine where a cigar needs to score to start differentiating themselves from the pack.
Opinion: It’s pretty nice to see the scoring average where it is for a new site. The scoring range seems to be fairly wide as well with scores from the past 12 months ranging from 95 to 72. Time will tell how things trend, but I would say that this is a good start and I look forward to seeing the progression.
REMOVED SITE
HAVE A PREMIUM CIGAR
This sites last review was posted in July, and overall for the last 12 months, they only posted twelve reviews. In an article like this where data is key, there just isn’t enough here to be able to continue on with analyzing this site.
OVERALL COMPARISON
Adding in the newly added site, we can compare them all side by side for their results over the past 12 months. This an interesting look at the majority of review sources that utilize the 100 point system (aside from a few magazines that don’t have scoring available online) and how they stack up against each other in terms of average scores and most frequent scores given.
The sites are ranked from left to right in terms of having lower average scores and lower number for the most frequent score given. The sources that have either given in to inflation or have poor scoring systems are on the right while the sites that have a more consistent or grounded scoring system are on the left.
FINAL THOUGHTS
Circling back around to cover that item I mentioned previously in the article about why I think some sites have seen a lowering of scores, I think it has to do with the quality of cigars released over the last 12 months. When I say quality, I am referring to the quality of flavor and not the quality of construction. In speaking here, internally, at Developing Palates and along with the teams over at Cigar Coop and Cigar Dojo, we all seem to be in agreement that we feel the last 12 months have seen a drop in overall flavor quality from previous years. I can’t say that I know of any reasons why, but the data suggests that to be the case.
In using our own data, which doesn’t use the 100 point system, we can show the change in scoring for each reviewer on Developing Palates to give you an indication of what kind of movement has happened.
I think the end of year lists will be pretty interesting this year as there might be more variation of who makes list compared to some of the previous years.
CIGAR EDITORIAL: WHEN IS A 90 NOT A 90 – 2 YEARS LATER
