A structured check and impact story of a climate impact forecast by a trained third party LCA expert, who provides feedback and time for a revision before determining if the impact forecast is positive, significant and valid.
Validation offers a third party review and assures a quality level that makes the results suitable for comparison between projects. Impact claims with third party validation are valuable for teams competing for impact funding or awards, and in general to communicate impact effectively. The validation process brings value to teams as it includes expert feedback, and an opportunity for revision, to increase the quality of their work where possible.
Validations are performed by Life Cycle Assessment experts with an MSc in environmental science or sustainability engineering, actively working as CIF trainers and coaching impact startups, who followed the Validation training to be able to provide the validation process. They are commissioned for validations by Impact Forecast and carry out validations independently.
A six step process
- The team hands in their Climate Impact Forecast, which includes all relevant assumptions about their product and process, impact data and calculations, with sources and supporting documents. (#note: e.g. via profile; deck, links, business figures, baseline data and additional sources/appendices)
- The validator performs a structured check where all inputs (e.g. impact indicators, functional unit, all assumptions) have to meet specific requirements.
- The validator drafts the impact story, describing how the company creates impact, mainly to check with the team that the innovation and reduction mechanism is understood. The impact story also serves as a summary and conclusion of the forecast and validation.
- Team and validator meet for an intake call to discuss the draft impact story and structured check to clear up any ambiguity in assumptions, discuss how to find missing data if any, and to be sure the forecast is an accurate model of the innovation.
- Time for revision and further research where needed, finalizing the impact story.
- Validator hands over the results by email, followed up with a call to discuss the result and help the company interpret it, for external communication and in developing the solution.
There are three properties tested in validation; validity, impact and significance. Respectively, a CIF file can be deemed Valid or invalid, positive or negative, and significant or marginal. Validity tests if the assigned differences and used data, correctly represent a comparison between the innovation and baseline at hand. Impact tells us something we already know from the CIF itself, if the total impact is positive (emission reduction) or negative (more emissions). And significance tests the size of the impact against a threshold, that threshold is 5t/year by default.
Table 1 – Possible outcomes of the structured check
Table 2 – Possible outcomes of a Cif (independent of its validity)
Table 3 – Significance (independent of validity, applicable to positive impact only)
The structured checkFor step 2 in the validation process, the validator performs the following checks using our validation interface;
Check all inputs from the start-up one-by-one. For each field, there are requirements that should be fulfilled. Per input, you can mark the information:
- Approve (so you approve that the input meets all requirements)
- Edited and approved (you fixed a small error, eg a typo or math error)
- Question (the input might fulfill the requirements, but you do not understand smth.)
- Concern (the input could be ok, but you want to suggest an important change)
- Rejected (the input does not fulfill the requirements, even if changed a bit)
You can put in comments to each field, even if it is approved – so you can share your thoughts and some feedback. Use this also to indicate what changes you made, in case you approved the field with edits. In these checks, also apply these transparency requirements:
- The start-up should provide explanations in the text boxes, to make all assumptions and inputs understandable. Sometimes, a list of materials can be understood as a part list with little explanation, but generally, every row in the table needs some clarification; what real-world difference is indicated here, and how is the quantity determined? If there is unexplained information in the Cif it is grounds for rejection, even if you know what is meant there from attachments, context or your background knowledge. The Cif should be self-standing.
- To be used in a valid Cif, any external sources need to be fully referenced, or linked to. Check if any external data comes from a reputable, relevant institute or peer reviewed journal, and if the information was read correctly (eg Mt/kgCO2 unit errors are common).
- Please provide as much written feedback in the validator as possible, to show the start-up the strong/weak points and help them to further improve the impact forecast.
- After going through all fields, click “Feedback and final review”
- Strong points: what things are very good/clear? (mandatory if valid) Try to summarize in one sentence what is made clear in this Cif. Usually that means, how you can describe how the innovation creates (most of its) impact? (e.g by saving/ replacing/ avoiding something with high climate impact). You can also comment on the quality of any custom data if that was used, on the clarity of assumptions and calculations, and on the elegance of the model.
- Weak points (not mandatory if valid, mandatory if unclear or invalid) This is a good place to repeat or reframe the most important rejection, comment or question, if there are multiple. Any errors and data gaps are also to be reported here. This way, it is clear on the one page overview what needs to be done to make the next Cif better, for teams and their coaches to follow up on. Notes written on individual questions do not show up in the overview.
- Sensitivity: Is the result robust? Or sensitive to small/ expected changes? This only needs to be done if the Cif has passed all checks so far. Is there still a positive impact, even if the inputs for some major impact levers are changed, in a range of +/- 10 %? If there is a highly unpredictable yet important element, for instance behavior change, or theoretical efficiency gains, find out when the total impact becomes insignificant. If e.g. 90% efficiency is promised and you find anything above 10% efficiency will create positive impact, the result is robust even when the efficiency is unpredictable.
Determine the validation result
- Outcome: Valid <> Invalid
- Impact: Positive <> Negative (some options may be inactive)
- Significance: Significant or Marginal (inactive, this is automatic)
- a Cif with a rejection is invalid, even if it is only 1 rejection.
- a Cif can be valid with one or more outstanding questions or concerns, if better (e.g. more realistic) information would not change the conclusion (positive and/or significant impact).
- if a question or concern is about a large enough difference that realistic values do change the conclusion (positive and/or significant), change it to ‘Rejection’ to avoid confusion While questions and concerns are no grounds for rejection per se, a large enough number of them can be grounds for invalidating a Cif, that does not have any rejection.
When validation is started, Positive, Negative, Significant and Marginal are determined automatically.
- Submit the validation. This will save two files – one that is still editable (for the validator) and one that is not editable anymore (for the start-up) with “Validated’ in the title (also if the result is not valid). This concludes step 2 of the validation process. Next, the structured check and impact story (described in the next section) are sent ahead, as the basis for the the first call between the validator and team. Preview of the structured check in the CIF tool
As step 3 of the validation process, the validator writes an impact story. This is a couple of paragraphs describing how the company creates impact, and what the reduction potential is, compared to which baseline. We take key information, assumptions and results from the CIF file and write that into a narrative that anyone can understand.
It can be useful for the startup, to have an impact claim in the words of an impartial impact expert. But the reason an impact story is required in validation, is to make sure the validator understands the company and their innovation. A draft of the impact story is written after receiving the validation request and before the first call. This draft is shared with the team along with the validated CIF file upon its first review. Often, information from the project website or team is needed to complete the story. Or to correct it, along with the validator’s understanding of the innovation.
The impact story in its final version is delivered with the validation results and optionally, collated in reports. This version has been updated with the final numbers on the validated CIF file and information from the first call.
Usually set up as a 30 minute online call, the aim of the intake is to discuss the current validation result and the draft impact story. The validation result and impact story are sent ahead so the startup can review the written feedback before the call. The conversation that covers the team’s key technical and business assumptions, and the validator’s questions, concerns and perhaps rejections in the CIF file. The call concludes with the startup and validator’s to-do’s.
If the CIF is found valid, positive and significant, and the startups does not want to make any changes based on the feedback, the validator completes the impact story, and finalizes the validation. The startup has no to-do’s other than receiving the result, and sharing it. This rarely is the case, there is usually something that can be improved or clarified, that is no grounds for rejection but are a good use of the revision opportunity.
If the CIF is not valid, or negative, or marginal, it is because of concerns or rejections in the information provided by the startups. The validator describes what the problem is in the validated CIF file, and explains this in the intake call if needed. It is up to the startup to make revisions if they want to, and set a deadline with the validator.
Revision and delivery
If the startup uses the revision opportunity they share a new and improved CIF file to the validator. The validator can quickly see if their concerns and rejections have been addressed effectively.
The structured check is performed again to produce the final validated file and validation result. The impact story is finalized and delivered along with the validated CIF file. Optionally, a report is compiled for a group of validations, to provide an overview of the validation process and results.
Validation result and id
A validated CIF file is marked with a seal, showing the validation result and a validation ID. The ID is unique to the validation, and can be used to check a validation result. We keep a copy of every validation that is never shared, but can be used for verification of impact claims.
It is possible that despite the startup’s and validator’s efforts, the revised result is less than valid, positive or significant. There is no second opportunity for revision in the validation process, and the undesirable outcome becomes the official result. The startup or organizer could consider requesting a new validation to restart the process, but this would be a separate validation process with its own official result.Example of a valid, positive and significant result.