Setting Quality Goals with Biological Variation

The hardware and bandwidth for this mirror is donated by dogado GmbH, the Webhosting and Full Service-Cloud Provider. Check out our Wordpress Tutorial.
If you wish to report a bug, or if you are interested in having us mirror your free-software or open-source project, please feel free to contact us at mirror[@]dogado.de.

Marcello Grassi

Introduction

A fundamental question in laboratory medicine is: “How good does my analytical method need to be?” The answer depends on the intended clinical use. A method that is acceptable for population screening may be inadequate for monitoring individual patients. This vignette introduces the biological variation model for setting analytical performance specifications, implemented in the valytics package through three functions: - ate_from_bv(): Calculate specifications from biological variation data - sigma_metric(): Quantify performance using the Six Sigma metric - ate_assessment(): Evaluate observed performance against specifications

library(valytics)

The Biological Variation Model

Concept

Every measurand (analyte) exhibits natural variation even in healthy individuals. This variation has two components: - Within-subject variation (CV_I): Day-to-day fluctuation within an individual - Between-subject variation (CV_G): Differences between individuals in a population The biological variation model, developed by Fraser, Petersen, and colleagues, uses these inherent variations to derive meaningful analytical performance goals. The logic is straightforward: analytical error should be small enough that it does not significantly increase the total variation observed in test results.

The Formulas

At the desirable performance level, the formulas are:

Allowable Imprecision: \[CV_A \leq 0.50 \times CV_I\]
Allowable Bias:
\[Bias \leq 0.25 \times \sqrt{CV_I^2 + CV_G^2}\]
Total Allowable Error:
\[TEa \leq k \times CV_A + Bias\] Where k is a coverage factor (typically 1.65 for ~95% of results).

Performance Hierarchy

Three performance tiers are defined, each with different multipliers:

Level	Imprecision	Bias	Stringency
Optimal	0.25 × CV_I	0.125 × √(CV_I² + CV_G²)	Most stringent
Desirable	0.50 × CV_I	0.25 × √(CV_I² + CV_G²)	Standard target
Minimum	0.75 × CV_I	0.375 × √(CV_I² + CV_G²)	Least stringent

Calculating Specifications with ate_from_bv()

Basic Usage

The ate_from_bv() function calculates all three specifications from biological variation data:

# Example: Glucose
# CV_I = 5.6%, CV_G = 7.5% (illustrative values)
ate_glucose <- ate_from_bv(cvi = 5.6, cvg = 7.5)
ate_glucose
#> 
#> Analytical Performance Specifications from Biological Variation
#> ------------------------------------------------------------ 
#> 
#> Input:
#>   Within-subject CV (CV_I): 5.60%
#>   Between-subject CV (CV_G): 7.50%
#>   Performance level: desirable
#>   Coverage factor (k): 1.65
#> 
#> Specifications:
#>   Allowable imprecision (CV_A): 2.80%
#>   Allowable bias: 2.34%
#>   Total allowable error (TEa): 6.96%

Comparing Performance Levels

The summary() method shows all three performance tiers:

summary(ate_glucose)
#> 
#> Analytical Performance Specifications - Detailed Summary
#> ============================================================ 
#> 
#> Biological Variation Data:
#> ------------------------------------------------------------ 
#>   Within-subject CV (CV_I):  5.60%
#>   Between-subject CV (CV_G): 7.50%
#>   Total BV [sqrt(CV_I^2 + CV_G^2)]: 9.36%
#> 
#> Settings:
#> ------------------------------------------------------------ 
#>   Selected performance level: desirable
#>   Coverage factor (k): 1.65
#> 
#> Formulas (Fraser & Petersen 1993):
#> ------------------------------------------------------------ 
#>   CV_A  = 0.50 x CV_I
#>   Bias  = 0.250 x sqrt(CV_I^2 + CV_G^2)
#>   TEa   = k x CV_A + Bias
#> 
#> Specifications (desirable level):
#> ------------------------------------------------------------ 
#>   Allowable imprecision (CV_A): 2.80%
#>   Allowable bias:               2.34%
#>   Total allowable error (TEa):  6.96%
#> 
#> Comparison Across Performance Levels:
#> ------------------------------------------------------------ 
#>   Level            CV_A     Bias      TEa
#>   -----            ----     ----      ---
#>   optimal         1.40%    1.17%    3.48%
#>   desirable *     2.80%    2.34%    6.96%
#>   minimum         4.20%    3.51%   10.44%
#> 
#>   * Selected level
#> 
#> Data Source:
#> ------------------------------------------------------------ 
#>   Biological variation values should be obtained from the EFLM
#>   Biological Variation Database: https://biologicalvariation.eu/

Different Performance Levels

You can calculate specifications for any tier:

# Optimal (most stringent)
ate_optimal <- ate_from_bv(cvi = 5.6, cvg = 7.5, level = "optimal")
ate_optimal$specifications$tea
#> [1] 3.480003

# Minimum (least stringent)
ate_minimum <- ate_from_bv(cvi = 5.6, cvg = 7.5, level = "minimum")
ate_minimum$specifications$tea
#> [1] 10.44001

When CV_G is Unknown

If only within-subject variation is available, imprecision goals can still be calculated:

ate_cv_only <- ate_from_bv(cvi = 5.6)
ate_cv_only
#> 
#> Analytical Performance Specifications from Biological Variation
#> ------------------------------------------------------------ 
#> 
#> Input:
#>   Within-subject CV (CV_I): 5.60%
#>   Between-subject CV (CV_G): not provided
#>   Performance level: desirable
#>   Coverage factor (k): 1.65
#> 
#> Specifications:
#>   Allowable imprecision (CV_A): 2.80%
#>   Allowable bias: requires CV_G
#>   Total allowable error (TEa): requires CV_G

The Six Sigma Metric

Concept

The sigma metric provides a standardized way to express method quality. It answers: “How many standard deviations of analytical error fit between my observed performance and the allowable limit?” \[\sigma = \frac{TEa - |Bias|}{CV}\]

Higher sigma values indicate better performance:

Sigma	Category	Defects per Million
≥ 6	World Class	~3.4
≥ 5	Excellent	~230
≥ 4	Good	~6,200
≥ 3	Marginal	~66,800
≥ 2	Poor	~308,500
< 2	Unacceptable	> 690,000

Calculating Sigma

# Assume observed: bias = 1.5%, CV = 2.5%
# Using TEa from biological variation
sm <- sigma_metric(
 bias = 1.5,
  cv = 2.5,
  tea = ate_glucose$specifications$tea
)
sm
#> 
#> Six Sigma Metric
#> ---------------------------------------- 
#> 
#> Input:
#>   Observed bias: 1.50%
#>   Observed CV: 2.50%
#>   Total allowable error (TEa): 6.96%
#> 
#> Result:
#>   Sigma: 2.18
#>   Performance: Poor
#>   Defect rate: ~308,500 per million

Detailed Sigma Summary

summary(sm)
#> 
#> Six Sigma Metric - Detailed Summary
#> ================================================== 
#> 
#> Formula:
#> -------------------------------------------------- 
#>   Sigma = (TEa - |Bias|) / CV
#>   Sigma = (6.96 - |1.50|) / 2.50
#>   Sigma = (6.96 - 1.50) / 2.50
#>   Sigma = 5.46 / 2.50
#>   Sigma = 2.18
#> 
#> Result:
#> -------------------------------------------------- 
#>   Sigma metric: 2.18
#>   Performance category: Poor
#>   Expected defect rate: ~308,500 per million
#> 
#> Sigma Scale Reference:
#> -------------------------------------------------- 
#>   Sigma    Category        Defects/Million
#>   ------   -------------   ---------------
#>   >= 6     World Class                 3.4  
#>   >= 5     Excellent                   230  
#>   >= 4     Good                      6,210  
#>   >= 3     Marginal                 66,800  
#>   >= 2     Poor                    308,500 *
#>   < 2      Unacceptable          > 690,000  
#> 
#>   * Current performance level
#> 
#> Note: Defect rates assume 1.5 sigma long-term shift.

Interpreting Sigma in Clinical Context

In clinical laboratories:

Sigma ≥ 6: Minimal QC needed; method is highly reliable
Sigma 4-6: Standard QC procedures adequate
Sigma 3-4: Enhanced QC recommended; monitor closely
Sigma < 3: High risk of errors; consider method improvement

Comprehensive Assessment with ate_assessment()

Basic Assessment

The ate_assessment() function evaluates observed performance against specifications:

assess <- ate_assessment(
  bias = 1.5,
  cv = 2.5,
  tea = ate_glucose$specifications$tea
)
assess
#> 
#> Analytical Performance Assessment
#> -------------------------------------------------- 
#> 
#>   >>> METHOD ACCEPTABLE <<<
#> 
#> Performance Summary:
#>   Parameter              Observed  Allowable     Status
#>   -------------------- ---------- ---------- ----------
#>   Bias                      1.50%        ---        ---
#>   CV (Imprecision)          2.50%        ---        ---
#>   Total Error               5.62%      6.96%       PASS
#> 
#> Sigma Metric: 2.18 (Poor)

Full Component Assessment

When you have specifications for all components:

assess_full <- ate_assessment(
  bias = 1.5,
  cv = 2.5,
  tea = ate_glucose$specifications$tea,
  allowable_bias = ate_glucose$specifications$allowable_bias,
  allowable_cv = ate_glucose$specifications$allowable_cv
)
summary(assess_full)
#> 
#> Analytical Performance Assessment - Detailed Summary
#> ============================================================ 
#> 
#> Overall Result: METHOD ACCEPTABLE
#> 
#> Observed Performance:
#> ------------------------------------------------------------ 
#>   Bias: 1.50%
#>   CV (Imprecision): 2.50%
#>   Total Error (k=1.65): 5.62%
#>     [TE = 1.65 x 2.50 + |1.50| = 5.62]
#> 
#> Allowable Specifications:
#> ------------------------------------------------------------ 
#>   Allowable Bias: 2.34%
#>   Allowable CV: 2.80%
#>   Total Allowable Error (TEa): 6.96%
#> 
#> Component Assessment:
#> ------------------------------------------------------------ 
#>   Bias: PASS (margin: +0.84%)
#>     |1.50| <= 2.34
#>   CV: PASS (margin: +0.30%)
#>     2.50 <= 2.80
#>   Total Error: PASS (margin: +1.34%)
#>     5.62 <= 6.96
#> 
#> Sigma Metric:
#> ------------------------------------------------------------ 
#>   Sigma = (TEa - |Bias|) / CV
#>   Sigma = (6.96 - 1.50) / 2.50 = 2.18
#>   Category: Poor
#> 
#>   Sigma Scale:
#>     >= 6: World Class | >= 5: Excellent | >= 4: Good
#>     >= 3: Marginal    | >= 2: Poor      | < 2: Unacceptable
#> 
#> Interpretation:
#> ------------------------------------------------------------ 
#>   Method technically passes but sigma < 3 indicates high risk.
#>   Strongly recommend method improvement or enhanced QC.

Handling a Failing Method

# A method with poor performance
assess_poor <- ate_assessment(
  bias = 4.0,
  cv = 5.0,
  tea = ate_glucose$specifications$tea,
  allowable_bias = ate_glucose$specifications$allowable_bias,
  allowable_cv = ate_glucose$specifications$allowable_cv
)
summary(assess_poor)
#> 
#> Analytical Performance Assessment - Detailed Summary
#> ============================================================ 
#> 
#> Overall Result: METHOD NOT ACCEPTABLE
#> 
#> Observed Performance:
#> ------------------------------------------------------------ 
#>   Bias: 4.00%
#>   CV (Imprecision): 5.00%
#>   Total Error (k=1.65): 12.25%
#>     [TE = 1.65 x 5.00 + |4.00| = 12.25]
#> 
#> Allowable Specifications:
#> ------------------------------------------------------------ 
#>   Allowable Bias: 2.34%
#>   Allowable CV: 2.80%
#>   Total Allowable Error (TEa): 6.96%
#> 
#> Component Assessment:
#> ------------------------------------------------------------ 
#>   Bias: FAIL (margin: -1.66%)
#>     |4.00| > 2.34
#>   CV: FAIL (margin: -2.20%)
#>     5.00 > 2.80
#>   Total Error: FAIL (margin: -5.29%)
#>     12.25 > 6.96
#> 
#> Sigma Metric:
#> ------------------------------------------------------------ 
#>   Sigma = (TEa - |Bias|) / CV
#>   Sigma = (6.96 - 4.00) / 5.00 = 0.59
#>   Category: Unacceptable
#> 
#>   Sigma Scale:
#>     >= 6: World Class | >= 5: Excellent | >= 4: Good
#>     >= 3: Marginal    | >= 2: Poor      | < 2: Unacceptable
#> 
#> Interpretation:
#> ------------------------------------------------------------ 
#>   Method does not meet specifications.
#>   - Bias exceeds allowable limit: consider recalibration.
#>   - Imprecision exceeds allowable limit: investigate sources.
#>   - Total error exceeds TEa: method requires improvement.

Complete Workflow Example

Here is a typical workflow for evaluating a new glucose method:

# Step 1: Define quality goals from biological variation
specs <- ate_from_bv(cvi = 5.6, cvg = 7.5, level = "desirable")
cat("Quality Specifications:\n")
#> Quality Specifications:
cat(sprintf("  Allowable CV: %.2f%%\n", specs$specifications$allowable_cv))
#>   Allowable CV: 2.80%
cat(sprintf("  Allowable Bias: %.2f%%\n", specs$specifications$allowable_bias))
#>   Allowable Bias: 2.34%
cat(sprintf("  TEa: %.2f%%\n\n", specs$specifications$tea))
#>   TEa: 6.96%

# Step 2: Assume we measured method performance
# (In practice, from validation studies)
observed_bias <- 1.8
observed_cv <- 2.2

# Step 3: Calculate sigma metric
sm <- sigma_metric(observed_bias, observed_cv, specs$specifications$tea)
cat(sprintf("Sigma Metric: %.2f (%s)\n\n", sm$sigma, sm$interpretation$category))
#> Sigma Metric: 2.35 (Poor)

# Step 4: Full assessment
assessment <- ate_assessment(
  bias = observed_bias,
  cv = observed_cv,
  tea = specs$specifications$tea,
  allowable_bias = specs$specifications$allowable_bias,
  allowable_cv = specs$specifications$allowable_cv
)

# Step 5: Decision
if (assessment$assessment$overall) {
  cat("DECISION: Method acceptable for clinical use\n")
} else {
  cat("DECISION: Method requires improvement\n")
}
#> DECISION: Method acceptable for clinical use

Obtaining Biological Variation Data

The quality of your specifications depends on reliable biological variation estimates.

Recommended Source

The EFLM Biological Variation Database is the current authoritative source:

Website: https://biologicalvariation.eu/
Provides rigorously reviewed BV estimates
Includes quality grades for each estimate
Updated regularly with new studies

Using the Database

Navigate to https://biologicalvariation.eu/
Search for your analyte
Review the CV_I and CV_G values
Note the quality grade (A, B, C, D) and number of studies
Use values appropriate for your population and context

Important Considerations

BV data may vary by population, age, sex, and health status
Ensure the BV data match your intended use population
Higher quality grades (A, B) indicate more reliable estimates
When multiple estimates exist, consider the range

Beyond Biological Variation

While the biological variation model is widely used, it is not the only approach to setting quality specifications. Other models include:

Clinical outcome-based: Specifications derived from clinical decision limits
State-of-the-art: Based on achievable performance (e.g., proficiency testing data)
Regulatory requirements: Fixed limits from agencies (e.g., CLIA)

The ate_assessment() and sigma_metric() functions work with TEa values from any source—simply provide your specification directly rather than calculating from biological variation.

# Using a CLIA-based TEa for glucose (example: ±6 mg/dL or ±10%)
# For a sample at 100 mg/dL, 10% = 10 mg/dL
sm_clia <- sigma_metric(bias = 2, cv = 3, tea = 10)
sm_clia
#> 
#> Six Sigma Metric
#> ---------------------------------------- 
#> 
#> Input:
#>   Observed bias: 2.00%
#>   Observed CV: 3.00%
#>   Total allowable error (TEa): 10.00%
#> 
#> Result:
#>   Sigma: 2.67
#>   Performance: Poor
#>   Defect rate: ~308,500 per million

Summary

The biological variation model provides a scientifically grounded approach to setting analytical quality specifications:

ate_from_bv() translates biological variation into actionable specifications
sigma_metric() provides a universal quality scale for comparing methods
ate_assessment() gives a clear pass/fail evaluation

These tools help laboratories make informed decisions about method acceptability while recognizing that the final decision depends on clinical context and regulatory requirements.

References

Fraser CG, Petersen PH (1993). Desirable standards for laboratory tests if they are to fulfill medical needs. Clinical Chemistry, 39(7):1447-1453.

Ricos C, Alvarez V, Cava F, et al. (1999). Current databases on biological variation: pros, cons and progress. Scandinavian Journal of Clinical and Laboratory Investigation, 59(7):491-500.

Aarsand AK, Fernandez-Calle P, Webster C, et al. (2020). The EFLM Biological Variation Database. https://biologicalvariation.eu/

Westgard JO, Westgard SA (2006). The quality of laboratory testing today: an assessment of sigma metrics for analytic quality using performance data from proficiency testing surveys and the CLIA criteria for acceptable performance. American Journal of Clinical Pathology, 125(3):343-354.

These binaries (installable software) and packages are in development.
They may not be fully stable and should be used with caution. We make no claims about them.
Health stats visible at Monitor.

Setting Quality Goals with Biological Variation

Marcello Grassi

Introduction

The Biological Variation Model

Concept

The Formulas

Performance Hierarchy

Calculating Specifications with ate_from_bv()

Basic Usage

Comparing Performance Levels

Different Performance Levels

When CVG is Unknown

The Six Sigma Metric

Concept

Calculating Sigma

Detailed Sigma Summary

Interpreting Sigma in Clinical Context

Comprehensive Assessment with ate_assessment()

Basic Assessment

Full Component Assessment

Handling a Failing Method

Complete Workflow Example

Obtaining Biological Variation Data

Recommended Source

Using the Database

Important Considerations

Beyond Biological Variation

Summary

References

When CV_G is Unknown