Coffee Quality, Pt 2 — How “quality” became a myth

James Harper:

Welcome back to Filter Stories, I'm James Harper, and this is part two of a three-part series on how in Specialty Coffee we got close to creating this concept of universal coffee quality, an understanding of coffee quality shared by almost everybody in the industry. I strongly recommend you listen to part one before you listen to this. Okay, so I consider myself a coffee professional, maybe you are too, and if you're not, let's pretend you are.

Imagine you and I were sat at a bar and talking coffee stuff, and then a non-coffee person, they come along and they strike up conversation with us, and they ask us at some point, what is the best coffee? You and I would probably interpret that question as meaning, which is the highest quality coffee? And you and I would probably be also quite close as to our understanding, our definition of what is a high quality coffee.

From a sensory point of view, we would say it is a coffee that is juicy, clean, sweet, a lovely acidity, distinctive, pleasurable, aromatic notes. And you and I would also be pretty similar as to what we like to drink. All of us would enjoy that type of coffee.

Isn't that a remarkable thing? That stranger at the bar who approached us, that outsider would probably walk away from that conversation thinking, wow, there is no ambiguity as to what is a high quality coffee. It would appear to them that coffee quality is universal.

This is just what is great coffee. It is set in stone. This is one of the remarkable legacies of the 2004 cupping form from the SCAA, also known as the Q form.

It is one of the major reasons we are so aligned in our preferences, in our definitions of quality. It is because for the last 20 years, we have lived in a world shaped by the 2004 cupping form. All the specialty coffees we've drank over the years, most of them will have been graded by some variation of the 2004 cupping form.

And by engaging in a coffee world shaped by this form, we have trained ourselves, perhaps even subconsciously, to enjoy the sorts of coffees that are awarded highly by this form. But what the 2004 cupping form did was just draw a line in the sand for defining quality. And I was curious to know, why that specific line in the sand?

I mean, depending on how this form was created, maybe actually what we would have aligned on and what we like and what we define as quality over these last 20 years would not be juicy, clean and sweet. Maybe it would just be coffees which have really big body. Maybe we'd focus entirely just on balance and downplay the importance of acidity.

Perhaps it would be primarily the absence of bitterness. Why this specific line in the sand? And so in this episode, what we're going to do is rewind to when the Q form was being developed, before we had our specific, deeply permeated shared understanding of what coffee quality is.

This form had to create a system that would define quality coffees. And for me, an important question is, how universal is it? Is a high quality coffee, you know, a situation where you give a high quality cup of coffee to anybody in the world, and they say, wow, that is amazing, exceptional.

Is this a notion of quality that appeals to every living person on earth? Did the forms creators do these mass market consumer surveys to figure out what exactly people love and think is high quality, and then build the form around that? Or is it actually a flavor experience prized by a very small group of people in a very specific time and place in history?

Well, let's look at how the form was actually developed in the late 90s. And I think we can learn so much by going back to the story of George Howell, as he was creating the first ever cup of excellence competition in Brazil. So George Howell, very famous in the coffee world.

We heard from him in part one, he described how back in the 70s, knowing nothing about coffee, you know, we were kids in a sandbox. He was just learning and trying to discover what made specialty coffees unique from commodity coffees. Sweetness.

George Howell:

Sweetness is number one. Acidity. Without that acidity, everything becomes flat.

James Harper:

Many other things too. The roastery and cafes that George started back in the 70s, fast forward to the 90s, he sells his specialty roastery and cafe business to Starbucks. And then he was involved in different projects here and there.

And he saw there was this problem in the specialty world. A classic chicken and egg situation. There wasn't very much specialty coffee in the first place, because producers didn't want to produce it.

Growing specialty, it costs more, but roasters weren't willing to pay more because in their minds, the coffees weren't different enough from what they could get for cheaper. But George found the fix.

George Howell:

It dawned on me finally, well, we can do a competition.

James Harper:

Let's create a competition to find the best coffees grown in a country and then sell them for high prices in an auction. And so George helped to organize the first ever Cup of Excellence in Brazil, where a group of buyers from Asia, Europe, and the States, they all flew into Brazil and made their way into the heart of coffee growing country, Minas Gerais, in the small city of Lavras.

George Howell:

And they'd come in on a Saturday or a Sunday at the hotel, designated hotel, and everything was taken care of, and they would leave the following weekend.

James Harper:

And there they would taste the finest coffees grown in Brazil that year.

George Howell:

So they would spend five days cupping.

James Harper:

And then whittle them down to the top 10.

George Howell:

We start with, let's say, 40 coffees on Monday, right? Monday and Tuesday. Now, after that, you're down to maybe 25 coffees on Wednesday.

Now you recup those 25 again. Then you're down again. And by the time of Friday, you pick the top 10 and you're cupping those.

James Harper:

And then at the end of the process, these final coffees would be auctioned off, and they would go on and continue to achieve record sums of money. Since this first Cup of Excellence in Brazil 25 years ago, the auctions have returned well over $80 million to farmers. The winning coffee in a recent Cup of Excellence in Ethiopia achieved over 800 euros per kilo.

Roasters who buy Cup of Excellence coffees proudly put this Cup of Excellence sticker on their bags to show the world that they are the flagship, the bleeding edge of coffee quality in their community. Simply put, the Cup of Excellence represents what many in the specialty coffee community regard as some of the highest quality coffees in the world. Okay, so now let's look into exactly how these coffees were scored.

How did the judges determine which coffees were the highest quality? So what's interesting is that George brought an early prototype version of what would become the 2004 SCAA cupping form. What we have here is a story of how the prototype form came together and evolved to what would become the final version.

Now I'll make something super, super clear. George is just one of many, many people all working around the same time trialing out developing this form. As Rick Reinhart, the former CEO and Executive Director of the SCAA mentioned to me.

Rick Reinhart:

And it was a pretty well workshopped. There's a lot of input from a lot of different corners, a lot of disagreement. I will say that the final product is largely a reflection of the vision of Ted Lingle with some concessions on his part and then lots of places where he didn't make concessions.

James Harper:

Anyway, the point I'm trying to make is George did not single-handedly create the 2004 cupping form. But I think it's fair to assume in his story, we see the same issues everybody at the time was grappling with. So here we have this prototype form and even in this prototype stage, the form had all those attributes that we saw in part one that make a coffee distinctive.

Boxes where you could score a coffee on its sweetness, on its acidity, on its body. And so this is the starting point for defining quality. Quality was going to be defined by a group of specialty coffee buyers using specific flavor attributes like sweetness, acidity, body that many in the community had zeroed in on in the last 20 years.

Attributes that made a coffee special, different from commodity. Okay, so that's our foundation, but still we have to define what quality is specifically. Let's take the example of acidity, one of the boxes on that form.

What does a 10 out of 10 on acidity actually mean?

George Howell:

And early in the form's development, acidity body were being measured by their intensity.

James Harper:

Acidity was being measured by its intensity only. So a 1 out of 10, you can barely taste the acidity. 10 out of 10, bam, lots of acidity.

And although scoring acidity on its intensity was in the initial thinking when constructing the form, that's not where it eventually landed. Because as we all know intuitively, more acidity doesn't mean better.

George Howell:

But it was not usable in Brazil.

James Harper:

And so as George and all the other judges are trying to decide, what does a 10 out of 10 actually mean when it comes to acidity? George himself is influenced by some wine books that he's reading at the time. And also in particular, what a Japanese coffee buyer is telling him during the cupping.

George Howell:

Back in those days, it was not this scientific silence. You will not talk to anybody to your right or to your left, but keep your thoughts to yourself. We related to each other in whispers and stuff like that, not to make noise.

So he would come to my table. We had multiple tables and comment, oh, this coffee here is really good, right? And he would use the words sensitive acidity over and over.

The ones who really like sensitive acidity. And that is what inspired me then to change the cupping form and separate intensity from quality. You might have a less intense acidity, but the acidity is just right for that fruit.

James Harper:

And so when scoring acidity, the intensity is ditched. And instead, this more free floating definition takes its place. It's now about how good is that acidity?

How appropriate is it? Which, okay, I guess now takes me back to my original question. What does a 10 out of 10 on acidity taste like?

Well, what eventually happened is today on the Q form, you score that box 10 out of 10, if the acidity is bright, clean, complex, well-defined and fitting for the coffee's origin and style. Basically, a high quality coffee has acidity. And what kind of acidity and how much kind of depends on the coffee, but it has to have acidity.

This is what is hard coded into the 2004 cupping form. It becomes the definition of quality. And then that definition of quality gets exported around the world and becomes set in stone almost because of the Q program and the many thousands of coffee professionals who go through it.

But now let me show you why this is actually just hard coding a preference. Why this definition of quality is not universal. It's just what a particular group of people like to drink.

So, quick recap from the beginning of our story. Back in the 60s and 70s and 80s, coffees were these flat, dull, lifeless flavor experiences. Early specialty coffee pioneers, they prized flavor experiences that were the opposite of commodity.

For example, acidity. Acidity in coffee became this easy to identify signal that this coffee is not a commodity coffee. And in the decades following the 70s, we have a growing specialty coffee community talking and tasting and teaching, influencing each other to define specialty coffees as coffees which have flavors that are the opposite from commodity coffees.

Flavors such as acidity. And so it begins to become a shared specialty coffee palette. And then when the first prototypes of the SCAA cupping form are developed in the late 90s, acidity, of course, is part of the scoring criteria.

At first, the form tried to measure it by its intensity. But then as we just heard, that was ditched. And it became how good, appropriate the acidity is for the coffee style.

But all along, there was no question that a good specialty coffee has a certain kind of acidity. And then a definition of acidity makes it into the 2004 form in very clear, authoritative language. It begins to look universal.

Like there's this objective truth about what quality is. But acidity as a way to define coffee quality is not universal. It's just a reflection of the specialty community's palette that shaped the form.

Acidity is just a preference. It is a preference of people who are in the specialty coffee world, but it is a preference. And that goes for most of the rest of the form too.

In fact, when I was speaking with George, what I found so fascinating is that when he was prototyping what would become the 2004 form, he made sure that the definition of quality was a straight up reflection of the preferences of the judges in the room. The final score out of 100 is pure personal preference. He gave me the smoking gun evidence for this.

George Howell:

The scoring was interesting too. Okay, so you'd have the majority of your cuppers, and we had 20 people plus, right, up to 24. And they tend to, many, not all, would go somewhere between 84 and 87 points, right?

And then we would, after every session, after that one hour, we would go to another room to discuss those coffees while the setup was being made for the next round. And we would always start with, okay, how many people scored coffee number one over 90 points? And hands would go up.

Sometimes a lot, sometimes none, or one, right? And then I'd just pick one or two people and say, why, what'd you get, right? And they would describe it.

And, you know, then you'd get people who would say, well, I gave it an 86, but the adjectives they were using was, it was delicious, it was great. And immediately I would go, then why did you mark it 86? Why isn't it more than that, right?

And others would say this particular coffee was really bad, then they'd give it an 84. And I'd say, why? Why didn't you give it 70 something or even less if that's what you thought?

But so you're trying to get people to step out, to take a chance, to express, right? That's what it's all about.

James Harper:

Yeah, and ultimately the number, a representation of how much they like it. Yes, exactly. Exactly.

Look at the wine industry. George treated these quality scores as they were being treated in the wine world, from which the 100 point scoring system was actually borrowed. In the wine world, a wine expert will rate a wine out of 100, how much they like a wine.

And so consumers, if they like the same sort of wines as this expert does, then they could see how much they would enjoy it too. And that's exactly what the Q form is. If you like the same sort of coffees as this group of coffee buyers did 25, 30 odd years ago, then the quality score produced by the Q form, it's great for you.

Just like in the wine world, you can kind of treat the Q form like a wine expert where you share their taste preferences. And so if they rank it highly, you are going to like it too. But that's not how the form is used in the specialty coffee world.

We don't say to people, hey, if you share my preferences, then this form is for you. In all my years in specialty coffee, I've never heard that. Instead, what I hear is we talk about quality as this abstract notion that kind of floats above all of us, even though quality is just the preferences of the specific people who made that form.

What I find so interesting about this story is how we use language. When we use the word quality, it sounds objective. Like it's beyond preference.

It's universal. It feels so much more authoritative, but it's misleading. What if we'd use the word that was closer to the truth instead?

What if instead of using the word quality, we actually use the word preference? Well, then we have to acknowledge that different people like different things. And coming back to the example of acidity, acidity, this cherished marker we cling onto because it's what defines specialty coffee.

It's what makes a coffee different from commodity coffee. We have to acknowledge that other people may not share our preference for that. And the brutal truth is many don't.

Consumers don't necessarily like the sort of coffees that the Q form ranks really highly. Coffee quality as produced by the Q form is the averaged preferences of a group of coffee experts. But these preferences may not be and are often not shared by many consumers.

There is mounds of evidence from academics looking at the wine world and a growing body of evidence of academics looking at the coffee world that proves this point. I even had the privilege of speaking to one of the leading researchers in this field, Professor Helena Hopfer. She has conducted dozens of studies trying to answer questions like, what is quality?

Who judges quality? And to what extent do consumers agree with the experts? And this is what she finds in her research.

Coffee buyers, look, they are the experts. They taste a lot of coffees. They have a lot of experience.

But consumers don't have this level of experience.

Helena Hopfer:

The consumer doesn't have this experience. Maybe it doesn't pay that much attention and might still like this coffee that an expert says isn't good. But it doesn't mean that the consumer is wrong.

Consumer just hasn't had all of these experiences.

James Harper:

Consumers like coffees that experts don't like. And vice versa. Very often, consumers don't like the wines that experts like.

Consumers go to the supermarket. They see a wine which has won an award. It has an image of a gold medal on the bottle.

Helena Hopfer:

And you're like, okay, I go for a gold medal. And you bring it home and it may not be what you're looking for, right? You might not like it.

James Harper:

Now, your position might be that, the consumer is wrong. They need to go down the path that I, as an expert, have gone down on. They need to taste lots and lots of coffees.

They need to train themselves to my level. I mean, for some consumers, absolutely. They're doing it.

They love it. Let me tell you, from what I know about consumers, their enjoyment of coffee has very little to do with the flavors in the cup. And to be brutally honest, for me, sometimes that's also the case.

It's the decor of the shop. It's latte art. And if they are interested in flavor, a high quality flavor for them may not include acidity at all.

I had a fascinating conversation with the sociologist and philosopher, Kenneth Lieberman, who wrote a book, Tasting Coffee, An Inquiry into Objectivity. He did a lot of studies himself, giving specialty coffees to consumers. And this is what he learned.

Kenneth Liberman:

People who are not into these unique coffees, you know, it's more important to them that their favorite newspaper is sitting next to them or that their favorite cup is being used. I mean, they have their routines, but the tongue doesn't play that major a role.

James Harper:

All of which to say, the Q-form and its definition of quality, it's not shared by all consumers or even most consumers. And frankly, most consumers don't even care about flavor in the first place. So to recap, when it comes to definitions of quality between consumers and experts, you can give consumers a coffee that the 2004 form ranks super highly, 90 points, and some or many may not like it.

And that's assuming they even bothered to pay attention to the flavors in the first place. But also, there is a big disparity as well between the experts. The experts themselves also have different preferences.

The 2004 form might force them to converge onto a single definition of quality, but it may not be what they actually personally like to drink. This has become so apparent in the coffee industry that in the last five years or so, the SCA, the Specialty Coffee Association, this is the bedrock of their position. Different people like different things.

Quality is not universal. In fact, quality is not even necessarily about flavor. It's about a coffee's attributes, many of which could have nothing to do with flavor.

For example, is the coffee organic? Here's Peter Giuliano, the chief research officer of the SCA, making this point very clear in a tutorial video.

Peter Giuliano:

To begin with, we must recognize that different individuals, whether coffee producers, tasters, or consumers, will value different attributes differently. Individuals have different needs and expectations, especially in their food and their drink. This is my point.

James Harper:

Quality, as defined in the 2004 cupping form, is not universal. It is just the preferences, averaged out, of the particular people who made the form back in history. Now you might, okay, now you might feel like I've been banging this drum for a while now, and you're thinking to yourself, and so what?

Where's the harm of having a unified understanding of quality, shared by specialty professionals around the world? What's the harm in that? Well, indeed, there are many benefits.

If you run a roastery, it's great. It gives you a way of understanding what kind of coffees are in the market that you might want to buy. It helps you calibrate your team.

And whether you want to use it or not, at least as a definition, it's out there, like a standard. And it's always helpful to know where you stand relative to a standard, especially when the standard can be so clearly defined. And look, for the farmers who had the money to do the very expensive Q certification, they were able to articulate and think about quality just like their buyers are, which is always tremendously helpful in sales.

But what if you are a farmer who did not do the Q? I think there are some cases where the perception that there is this universal standard for quality, even though we insiders know it's not the case, now I've seen some cases where it has been harmful to coffee producers. And I want to tell those stories in the next episode.

I know for so many of you, losing this shared standard for coffee quality feels like a huge loss. Why get rid of it? What I hope to do in the next episode is make you just a little bit less nostalgic and critical of a tool that I know so many of you are so personally invested in.