Black Box: Consumer demand for proof of sustainable impacts spurs new data initiatives

Sustainable certification has grown into a billion-euro industry, but it has a blind spot, which time and again is making schemes and the products carrying their ecolabels easy targets for criticism: There is no or very little quantitative data to document their impact.

When consumers buy FSC or PEFC-certified garden furniture or cocoa products certified by Fairtrade or Rainforest Alliance, they assume the premium they pay will help reduce deforestation, CO2 emissions and increase the number of farmers receiving at least a minimum living wage – but there are rarely any numbers to confirm it.

For Rens Rutten, a Science and Impact Specialist with Rainforest Alliance, this is all very familiar ground. For the last twenty years, she has been searching for ways to learn what works or not in development programs, the last five years for certification schemes.

“In terms of impact, we are approaching certification like a Black Box. We don’t actually know very well how the certification program works on the ground,” she said.

“Until now we have been relying on audit data and in some cases, we have used pilots and try-outs to get more data and insight into what is happening on the level of the certificate holder. We have not succeeded in developing an efficient and scalable approach,” said Rens Rutten.

A Google search for ‘sustainable certification impact’ will show a long history of scientific impact studies and white papers attempting to identify changes relating to the introduction of sustainable certification standards into farming, fisheries and forestry.

Similar studies appear in the Impact Sections of several of the leading schemes’ websites alongside Impact Dashboards flashing non-impact statistics – also known as vanity metrics - such as rise in number of farmers certified, hectares of forest covered by the standard and number of companies participating in the scheme:

“We have regularly commissioned impact studies. Sometimes they show that certification may have contributed to certain improvements, but these studies approach certification as one program, while in fact it consists of a combination of interventions that are implemented in different ways. Therefore, I often find that these studies are not very useful to learn what works or not, and for which type of farmers,” said Rens Rutten.
 

Smart Meters and Farm Intelligence Apps

In a recent update of the Rainforest Alliance Scheme, Rens Rutten and her colleagues are currently working on a pilot project, which may add a range of new mechanisms to improve monitoring capabilities and data gathering. Part of the bundle are so-called Smart Meters with flexible targets set by farmers themselves, which RA hope will improve data gathering and use of the data by the farmers to monitor the changes and to adapt their activities when needed. A new digital platform and a Farm Intelligence App will help farmers and managers make efficient use of the information and make sure impacts - or the lack thereof – is being monitored.

“We are trying to show, what is actually happening on the level of the certificate holder and the farms. We want to support farmers to collecting and using these data, and at the same time it will provide us with more information on the improvements they are getting from working with the standard,” said Rens Rutten.

The pilot scheme will be running for two years with Smart Meters covering 11 topics on both farm and management levels, ranging from irrigation and water use, pesticides and greenhouse gas emissions to social issues such as income, gender, and health and security.   

Measuring impacts and trying to make sense of the data on a mass scale is nothing new, however: Schemes and NGO’s have been struggling with this for more than a decade, but so far none of these projects have produced any scalable and useful data.

Some topics such as gender equality or child labour are extremely complex to measure. Use of fertilizer may differ heavily from country to country depending on the soil, the crop and the climate. And while one scheme may propose one set of indicators, others may have an entirely different focus, making a universal set of claims that clients can use to sell their product difficult if not impossible.

The introduction of standards in one forest area may also lead to intensified illegal harvesting outside the certified forest area. Finally, while a baseline study will give a snapshot of a particular issue – say health & security standards – at a given time, it is impossible to know, to what extent these changes would have happened over time without certification.

What makes Rens Rutten believe this project will produce anything useful data, when so many attempts have failed? Rainforest Alliance is not imposing any targets, she explains. The main purpose of the Smart Meters is for farmers and management to be able to measure their own progress according to the individual indicator, such as for instance water use, and use that data to adapt their actions towards continuous improvement.

“This approach brings the ownership and the use of the data to the farms and farm groups and will generate more credible data because these data are in the first place meant to be used for their own learning cycle. However, we are for the first time introducing a common methodology, which will hopefully show a fuller picture of the impacts our certificate holders are producing;” said Rens Rutten.

At Preferred by Nature, Market & Development Director Adam Grant says too little attention has been paid to impacts, but that projects like Rainforest Alliance’s pilot and other current impact initiatives are evidence of a new willingness to deliver the data that businesses and consumers are asking for.

“Not knowing the real impact of what we do is to some extent like working with a blindfold. We might have an idea that we are pushing suppliers and businesses in the right direction, but at the end of the day, we lack the data to prove there is any real impact,” said Adam Grant.

Preferred by Nature recently launched the Carbon Footprint Management Standard (CFM), which allows businesses to accurately measure their carbon impact and develop plans to reduce it. And other impact measuring tools, partly made possible by new technologies, are in the pipeline, says Adam Grant.

“All the technology for monitoring impacts, such as satellites, smartphones, and decentralized data controls, is already there. We just need to find a common methodology and an efficient way of applying it to our standards,” said Adam Grant.
 

“Huge amount of movement”

According to Adam Romo, Senior Coordinator of Innovations at ISEAL, the global membership organization for transparent sustainability systems, there is “a huge amount of movement in this space” with members improving their ability to collect, analyze, and make sense of new data.

“We are seeing a shift amongst our members towards more performance-based M&E and assurance systems, which requires certification bodies to measure things such as greenhouse gas emissions, how much fertilizer is being used, and forest conserved within a certified area etcetera,” said Adam Romo.

“Instead of just assessing whether sustainability practices are being implemented, performance-based monitoring and assurance makes it possible to report changes over time. This means certification bodies can make much more detailed and interesting claims to their clients,” said Adam Romo.

All sources point to the fact that impact data is only useful if there is also baseline data and data from non-certified areas to give perspective.

So, who is driving this trend? Rens Rutten points to the private sector:

“Companies. Today, for instance, they need to report what they are doing on CO2 emissions. The label is not sufficient any longer. They want to show what improvements they are contributing to.”

Similar views are held by both Adam Grant and his namesake Adam Romo at ISEAL. Romo, however, points out that the drive towards data and impact enlightenment is of course also spurred by an internal will to improve performance.

“Schemes are realizing that in order to improve continuously, they need to be better at measuring and understand what is happening on the ground - whether they are making a difference,” said Adam Romo.