Rare-Event Backtest and Confidence Bounds

Install and import#

bash

npm install fintech-algorithms

import { rareEventBacktestAndConfidenceBounds } from "fintech-algorithms/model-validation-and-backtesting/classification-and-score-validation/rare-event-backtest-and-confidence-bounds";

Signature#

rareEventBacktestAndConfidenceBounds(inputs)

Worked example#

verified This is the worked example published in the article, replayed by the test suite on every run. The output cannot drift.

Input#

inputs

{
  "observed_events": 3,
  "trials": 1000,
  "expected_probability": 0.002,
  "confidence_level": 0.95,
  "horizon_start": "2025-01-01T00:00:00Z",
  "horizon_end": "2026-01-01T00:00:00Z",
  "knowledge_cutoff": "2026-06-30T00:00:00Z",
  "sampling_assumption": "independent-bernoulli-approximation"
}

Call#

rareEventBacktestAndConfidenceBounds(inputs)

Returns#

object with 14 fields: observed_events, trials, observed_rate, expected_probability, expected_count, confidence_level, z_value, wilson_center, …

{
  "observed_events": 3,
  "trials": 1000,
  "observed_rate": 0.003,
  "expected_probability": 0.002,
  "expected_count": 2,
  "confidence_level": 0.95,
  "z_value": 1.959963984540054,
  "wilson_center": 0.004901898967320896,
  "wilson_half_width": 0.0038811150861822775,
  "wilson_lower": 0.0010207838811386186,
  "wilson_upper": 0.008783014053503173,
  "consistency": "inside-interval",
  "sampling_assumption": "independent-bernoulli-approximation",
  "state": "rare-event-evaluated"
}

Other exports#

This module also exports rocCurveAndRocAuc, precisionRecallCurveAndPrAuc, brierScore, logLoss, reliabilityDiagramAndExpectedCalibrationError, gainsLiftAndDecileCapture, costSensitiveThresholdOptimization, scoreStabilityAndMigrationMatrix, sliceBasedValidationBySectorCountryAndRegime, calculate. Every module additionally exports run as an alias of its primary function, and a meta object carrying its catalog id, domain, family, shape and article URL.