Data Validation versus Data Usability Assessments

Two Important Methods of Review and Evaluation for Environmental Data

Collecting environmental data is a process that involves various steps and phases including submittal and approval of sampling and/or work plans, sample collection, laboratory analysis, and ultimately decision-making based on the results. To be successful, it is critical to also include the important step of reviewing the data to ensure data quality is as is required by regulatory guidance. To demonstrate the importance of data review, it’s helpful to distinguish between data validation and data usability.

Data Validation and Data Usability Assessments: What do these terms mean?

Data validation (DV) and Data Usability Assessments (DUAs) are two different methods used to evaluate analytical data. The use of one over the other is typically determined based on project-specific needs and regulatory or client requirements. The phrases “DV” and “DUA” are often used interchangeably, but they are two different types of review.

Data Validation (DV)

• DV is a formal, systematic process, but the DV report can be provided in a variety of different formats.
• The reviewer follows specific guidelines created by EPA or other regulatory agencies, as applicable.
• During DV, the effects of lab and field performance and matrix interferences on sample results are evaluated.
• If nonconformances are noted, specific validation qualifiers may be applied to the data to indicate estimated, non-detect, and/or rejected results (e.g., J, UJ, R, J-, J+).
• Various levels of DV can be performed depending on project needs.
  •  Limited DV typically includes basic verification checks and review of sample-related batch quality control (QC) (e.g., method blanks and blank spike samples). This level of DV requires a “Level II” type lab deliverable at a minimum.
  • Full DV builds upon limited DV and adds review of instrument-related QC (e.g., calibrations, tunes) as well as recalculations and verification of reported results using the raw data. Full DV requires a “Level IV” type lab deliverable.

Data Usability Assessments (DUAs)

• DUAs are performed using a less formalized, less prescribed approach, compared to DV.
• DUAs can be performed under a variety of state or project-specific guidance documents and can include several deliverable formats, similar to DV.
• Similar to DV, DUAs also look at the effects of lab and field performance and matrix interferences on sample results, but DUAs focus more on the effect of the lab and field performance and matrix interference issues on the achievement of the project objectives.
• In DUAs, validation qualifiers are not typically applied to the data, but a DUA is sometimes performed with DV (generally after the DV is completed).
• DUAs require a “Level II” type lab deliverable at a minimum.
• DUAs answer the questions “Can we use the data for decision-making purposes?” and “Do the data allow the achievement of project objectives?”
• If nonconformances are noted,
  1. Data may be “flagged” with descriptive statements:
     • High or Low Bias
     • Uncertainty
     • Sensitivity (Are reporting limits above/below project screening criteria?)
  2. The DUA will discuss how the nonconformances may or may not impact usability of the data in relation to project objectives. The reviewer will evaluate the identified nonconformances and consider how significant the biases or uncertainties are. Possible considerations include:
     • How close are the affected results to the project screening criteria?
     • Is the affected analyte a contaminant of concern at the site?
     • Do we have more data points for that specific analyte without any biases, uncertainties, or sensitivity issues, etc.?

How much will it cost to perform these reviews?

Typically, DV and DUA costs are determined per data package and per analytical parameter; costs may vary if the reviewer is looking at 4 VOCs vs 40 VOCs or 2 metals vs 23 metals.

• In general, full DV will cost the most and take the longest to complete.
• DUAs and limited DV are similar in cost and usually take less time to complete than full DV.

It is also important to keep in mind that laboratories may add a surcharge for a Level IV data package (required for full DV) but some laboratories may provide these Level IV data packages at no additional cost. Project-specific cost estimates can be provided by a member of the TRC quality assurance (QA) team.

Data review is important! TRC Can Help

The data we generate are often used to make costly decisions for TRC’s clients; some data may even identify potential impacts on human health or the environment. It is imperative to confirm that quality data are available and are appropriate to support project decisions. By performing DV or DUAs, TRC can determine potential low or high biases, potential uncertainties, and even potential false positive or false negative results. Even if the lab follows all method-required procedures, there can still be data quality/usability issues. We have a responsibility to our clients and other data users to understand the data quality and effectively communicate our conclusions for future use of the data.

Project staff should plan early with the laboratory and project chemists/data reviewers for their data deliverable and data review needs. Not sure what level of data review you need or who to ask for DV and DUA needs? TRC’s QA/Chemistry CORE Team can help! We can work with you to understand what is needed for successful achievement of project objectives and help find a qualified staff member to perform the required level of data review.