4.3. Frameworks for assessment of alternatives

4.3. Frameworks for assessment of alternatives

So, what does it take to change the conventional practice with possibly hazardous or harmful chemicals or processes to a more sustainable solution?

The major step is assessment of alternative solutions, which takes into account a wide range of criteria. When thinking about replacing the existing process with an innovative alternative, chemists and engineers try to avoid so-called "regrettable substitutions". In other words, avoid switching to an alternative process or chemical that either transfers risk to another point in the production chain or lifecycle or contains unknown future risks.

Ideally, in the concluded assessment, chosen alternatives must:

• be technologically feasible;
• provide the same or better value in performance and cost;
• have an improved profile for human health and environment;
• account for economic and social considerations;
• have potential to be sustainable over long period of time (look out for restrictions that may arise in the future; for example, shortage of rare elements, etc.).

We see that choosing the best alternative requires careful investigation! Such investigation must be comprehensive, i.e., would require cross-disciplinary expertise, and based on high quality data.

Let us look at some typical criteria that may be used in chemical industry and research for evaluation of various processes and reactions. In the "green chemistry" context, the main emphasis is put on the environmental profile of a chemical alternative, while economic feasibility is included in the picture at the stage of technology transfer.

Evaluation criteria

Table 4.1 below represents the set of criteria that can be effectively used for assessing alternatives in chemical and material manufacturing. On the left, the top-level criteria are listed, which are key points of concern when introducing new chemicals to the manufacturing process. The middle column lists some sub-criteria, which show how the impacts can be distributed. The right column specifies specific measures for each type of impact, which basically become guides for data search and analysis.

Table 4.1. Criteria and measures used to assess green chemistry alternatives to existing processes (adapted from UCLA Sustainable Technology & Policy Program, 2011)

Criteria	Sub-Criteria	Measures
Physical Chemical Hazards	-	Flammability
		Flashpoint
		Explosivity limits
		Auto-ignitability temperature
		Oxidizing properties
Human Health Impact	Toxicity	Acute toxicity
		Carcinogenicity
		Developmental toxicity
		Endocrine toxicity
		Endocrine disruption
		Epigenetic toxicity
		Genotoxicity
		Organ, tissue, cell toxicity
	Human Exposure	Volume in manufacturing
		Volume in consumer use
		Extent in dispersive use
		Sensitive sub-populations
		Persistence
		Bioaccumulation
Ecological Impacts	Adverse Impacts	Aquatic, animal or plant species
		Aquatic and terrestrial ecosystems
		Endangered or threatened species
		Environmentally sensitive habits
	Exposure	Volume in manufacturing
		Volume in consumer use
		Extent in dispersive use
		Persistence
		Bioaccumulation
Environmental Impacts	Adverse Air Quality Impacts	Nitrogen oxide
		Sulfur oxides
		Greenhouse gases
		Ozone-depleting compounds
		Photochemically reactive compounds
		Particulate matter
		Fine particle matter
	Adverse Water Quality Impacts	Biological oxygen demand
		Total dissolved solids
		Thermal pollution
	Adverse Soil Quality Impacts	Chemical contamination
		Biological contamination
		Loss of organic matter
		Erosion
	Natural Resource Use Impacts	Non-renewable material use
		Renewable material use
		Water Use
		Energy Use
		Waste generation and end-of-life disposal
		Reusability and recyclability
Technical Feasibility	-	Functionality
		Reliability
		Usability
		Maintainability
		Efficiency
Economic Feasibility	-	Manufacturer impact
		Purchaser impact

The list of criteria given in Table 4.1 has been proved effective for some case studies. While it puts the main emphasis on the hazard assessment and environmental impact, the technical and economic criteria are also included and can play a significant role even at the stage of selection of particular chemical reagents for the process. Note that the above list of criteria and sub-criteria is not something written in stone. It is presented here as an illustration. For each specific assessment project, choice of criteria needs to be justified through expert and stakeholder involvement and will depend on the goals of assessment. Depending on the assessment team decision, some criteria can be added, some – removed, and weights of all factors can be tuned. Clear identification and justification of the selected criteria is critical.

Data collection

In any assessment project, clear and consistent requirements should be set for the sources of data to be used. Information should meet specific data quality criteria for inclusion into the assessment. Quality of data will determine their utility. Data selection should follow the internationally recognized definition for reliable information: "Reliable information is from studies or data generated according to valid accepted testing protocols in which the test parameters documented are based on specific testing guidelines or in which all parameters described are comparable to a guideline method. Where such studies or data are not available, the results from accepted models and quantitative structure activity relationship (QSAR) approaches may be considered. The methodology by Organization for Economic Cooperation and Development (OECD) can be used for the determination of reliable studies." (Principles of Alternative Assessment, 2012)

Preferably, data should be obtained from authoritative bodies, those referenced by US government agencies (e.g. EPA). The following are links to some of such resources:

• U.S. EPA PBT Profiler software can be used to gain information on persistence, bioaccumulation potential and toxicity of organic substances.
• Government Toxicology Data Network

Technical data sources

Information should be obtained from published studies or directly from technical experts or users of the alternatives. In other cases, information can be requested from product manufacturers. The specific performance information (reactions, energy effects, thermodynamic analysis) available from experimental labs may be needed to draw conclusions about technical feasibility for each individual application. Clear referencing of the data sources is important.

Economic data sources

Data sources for financial information may include manufacturers, stakeholders, the Chemical Economics Handbook, and other standard reference sources. For many emerging alternatives, hard cost information may be unavailable. Cost comparisons today may not be directly extrapolated to emerging technologies because learning curves, scaling, and other factors can affect costs over time. Assumptions and use of surrogate data should be clearly explained in the assessment.

Ranking the alternatives

Quantification (scoring) of the impacts based on the criteria listed above is typically done via a multi-criteria analysis (MCA) model, appropriately build for the project. MCA provides techniques for comparing and ranking different outcomes of existing and alternative processes. When setting up an assessment project, it is important that the scoring system is transparent and is consistently applied to all scenarios under consideration.

MCA is a great tool for comparison of different options, but it is hardly objective because choice of criteria and metrics to quantify impacts varies from case to case. In contrast, cost analysis is aimed at providing objective measure of economic feasibility based on predicted cash flow. Cost analysis requires impacts to be expressed in monetary terms. MCA can use both monetary and non-monetary measures, as well as both quantitative and qualitative measures.

In MCA, ranking of chemicals or processes with respect to the listed criteria can be done in a variety of ways. One way is to assign each criterion a score that spans from 0 to 1, with the value of 1 corresponding to the best (most preferable) choice and the value of 0 corresponding to the worst (least preferable) choice among the available. The rest of the choices would score in between.

Another possible approach for assigning scores is outranking. There is no relative scoring, but instead, alternatives are compared by each criteria in pairs (two at a time). This way, we try to identify the extent to which one alternative out-performs the other. In the end, the dual performance scores (1 - "win"; 0 - "lose") are aggregated, and the preference index is calculated for each alternative.

Evaluation of the economic impacts associated with the implementation of a new product or practice generally focuses on the changes in capital and operational costs and revenues. (These terms of cost analysis were overviewed in Lesson 3.). The main areas where impact is expected are:

• cost of new equipment or production process;
• operation and maintenance costs (labor costs, energy costs, etc.);
• cost differences for different substances;
• cost of transportation;
• cost of design, monitoring, and training;
• regulatory costs.

The data on economic impacts is collected in consultation with relevant supply chain actors and possibly trade associations. Evaluation can be an iterative process, starting from qualitative comparison of the old and new scenarios and ending at quantification of impacts with monetary values.

The European Chemicals Agency (ECHA) website provides a more detailed guide to economic assessment of alternatives and can be used as a resource for this task. There are some documents linked that you are not required to read unless you're specifically interested in the socio-economic assessment.

Weighing factors

In most situations, decision-makers are not equally concerned about all highlighted criteria. For instance, a particular decision-maker may place more importance on whether a household cleaner causes cancer than on whether it contributes to smog formation. Thus, the decision-making method should account for respective “weight" of each criterion in the evaluation process. Since different stakeholders may place different weights upon criteria, the weighting raises significant questions in the context of a regulatory program. For example, can we consistently compare the alternatives without regulating the weight of factors? This is something to watch out for.

The criteria weights can be established by three methods:

1. using generic or recommended weights;
2. calculating the weights based from objective measures; and
3. eliciting weights from stakeholders or experts.

Method (1) is exemplified by Table 4.2 which lists several sets of generic weights recommended by National Institute of Standards and Technology (NIST) based on the data of Environmental Protection Agency (EPA) and Harvard Study for a set of criteria usually used in life cycle assessment (LCA).

Table 4.2. Suggested weights on assessment criteria from various regulatory studies normalized to 100% (source: UCLA Sustainable Technology & Policy Program, 2011)

Criteria	NIST	EPA	Harvard	Equal Weights
Global warming	29.3	16	11	7.7
Fossil Fuel Depletion	9.7	5	7	7.7
Air pollutants	8.9	6	10	7.7
Water intake	7.8	3	9	7.7
Human health cancerous	7.6	11	6	7.7
Human health non-cancerous	5.3	11	6	7.7
Ecological toxicity	7.5	11	6	7.7
Eutrophication	6.2	5	9	7.7
Habitat alteration	6.1	16	6	7.7
Smog	3.5	6	9	7.7
Indoor air quality	3.3	11	7	7.7
Acidification	3.0	5	9	7.7
Ozone Depletion	2.1	5	11	7.7

In the above table, the NIST panel generated weights from stakeholder consulting that involved 7 building product manufacturers, 7 product users, and 5 LCA experts. EPA weights and Harvard weights were derived by NIST from sets of qualitative rankings of impacts developed respectively by EPA’s Science Advisory Board in 1990 and Harvard researchers in 1992.

Method (2) of calculating corresponding weights can be based on distance-to-target approach, when each criterion is weighted by the variance between the existing and desired conditions. For example, if the global community is further away from achieving the goal for global warming than it is for ozone depletion, then greater weight is given to the global warming potential. Another way to such calculation is monetary evaluation, when weighing is done based on the cost of environmental consequences.

Method (3), which assumes obtaining weights from stakeholders directly, may be based on public opinion surveys, community working group decisions, and different multi-criteria analysis models. The main types of stakeholders to consider: (1) Environmental Non-Government Organizations, Industry, Policymakers, and Consumers (Public). Weight assignments collected through surveys are then averaged across the board of stakeholders and then normalized to 100%.

Use of any of the methods depends on the goals of the assessment project, its scope, resources, and timeline. When building an assessment project, the weighing process should be transparent and well justified. When comparing different cases within one study, keep the weighing scale the same across the evaluation criteria.

Within the MCA approach, the final score (S_i) of a particular option (alternative) with respect to any major top-level criterion i is estimated as an average of all sub-criteria scores under that criterion:

$$S_i=\frac{{\displaystyle {\sum }_{j=1}^n{s}_j}}{n}$$

where n is number of sub-criteria or metrics used to assess the option under top-level criterion i. The final total score (S_tot) is the weighted sum of all top-level criteria scores:

$$S_{tot}={\displaystyle \sum _{i=1}^N{S}_i{w}_i}$$

where N is the number of top-level criteria considered in assessment; w_i is the weight factor of a particular criterion. The example study presented in the next section of this lesson demonstrates how the MCA scores are calculated and compared.

Consider the following supplemental reading materials on this topic: