Cost-Sensitive Threshold Optimization

Install and import#

bash

npm install fintech-algorithms

import { costSensitiveThresholdOptimization } from "fintech-algorithms/model-validation-and-backtesting/classification-and-score-validation/cost-sensitive-threshold-optimization";

Signature#

costSensitiveThresholdOptimization(inputs)

Evaluates the classify-nothing option and every distinct score as a cutoff, prices each one with the caller's confusion-cell costs, and returns the full candidate ledger alongside the cheapest cutoff.

Parameters#

Name	Type	Notes
`inputs`	`{ records: Array<{ id: string; label: 0 \| 1; score: number; weight?: number; score_available_at?: string; label_available_at?: string }>; evaluation_cutoff?: string; costs: { false_positive: number; false_negative: number; true_positive: number; true_negative: number } }`	The scored population plus the cost of each confusion cell. Every record needs a unique nonempty `id`, a `label` that is exactly the number 0 or 1, and a finite `score`; `weight` defaults to 1 and must be positive. All four `costs` entries are required, must be finite and nonnegative, and at least one of `false_positive` and `false_negative` must be nonzero. If `evaluation_cutoff` is supplied, any `score_available_at` or `label_available_at` on a record is compared against it as a string and must not sort after it. costs: all four cells required, finite and nonnegative · weight: positive, default 1

Returns#

{ candidates: Array<{ threshold: number | null; true_positive: number; false_positive: number; true_negative: number; false_negative: number; selected_weight: number; selected_rate: number; expected_cost: number }>; optimal_threshold: number | null; optimal_expected_cost: number; optimal_selected_rate: number; optimal_confusion: { true_positive: number; false_positive: number; true_negative: number; false_negative: number }; costs: { false_positive: number; false_negative: number; true_positive: number; true_negative: number }; tie_break: string; state: string }

candidates runs from the classify-nothing option (threshold: null) through every distinct score in descending order, each with its weighted confusion counts and per-unit-weight expected_cost. The optimal_* keys and optimal_confusion repeat the winning candidate, costs echoes the validated cost map, tie_break names the rule that settles ties, and state is threshold-selected.

Errors#

When records is absent, not an array, or empty — throws RangeError
When a record id is missing, empty, or repeats an earlier one — throws RangeError
When a label is anything other than the number 0 or 1 — throws RangeError
When a weight is zero or negative — throws RangeError
When costs is missing or is not a plain object — throws TypeError
When any of the four cost entries is missing or not a finite number — throws TypeError
When any cost is negative, or both false_positive and false_negative are 0 — throws RangeError
When score_available_at or label_available_at sorts after evaluation_cutoff — throws RangeError

Complexity: time O(n * k), space O(n + k).

Worked example#

verified This is the worked example published in the article, replayed by the test suite on every run. The output cannot drift.

Input#

inputs

{
  "records": [
    {
      "id": "R01",
      "label": 1,
      "score": 0.95,
      "probability": 0.92,
      "weight": 1,
      "sector": "Banking",
      "country": "Egypt",
      "regime": "Expansion",
      "score_available_at": "2025-01-01T00:00:00Z",
      "label_available_at": "2026-01-01T00:00:00Z"
    },
    {
      "id": "R02",
      "label": 0,
      "score": 0.9,
      "probability": 0.88,
      "weight": 1,
      "sector": "Insurance",
      "country": "Egypt",
      "regime": "Expansion",
      "score_available_at": "2025-01-01T00:00:00Z",
      "label_available_at": "2026-01-01T00:00:00Z"
    },
    {
      "id": "R03",
      "label": 1,
      "score": 0.9,
      "probability": 0.84,
      "weight": 1,
      "sector": "Markets",
      "country": "Saudi Arabia",
      "regime": "Expansion",
      "score_available_at": "2025-01-01T00:00:00Z",
      "label_available_at": "2026-01-01T00:00:00Z"
    }
  ],
  "evaluation_cutoff": "2026-06-30T00:00:00Z",
  "costs": {
    "false_positive": 1,
    "false_negative": 6,
    "true_positive": 0,
    "true_negative": 0
  }
}

Call#

costSensitiveThresholdOptimization(inputs)

Returns#

object with 8 fields: candidates, optimal_threshold, optimal_expected_cost, optimal_selected_rate, optimal_confusion, costs, tie_break, state

{
  "candidates": [
    {
      "threshold": null,
      "true_positive": 0,
      "false_positive": 0,
      "true_negative": 14,
      "false_negative": 10,
      "selected_weight": 0,
      "selected_rate": 0,
      "expected_cost": 2.5
    },
    {
      "threshold": 0.95,
      "true_positive": 1,
      "false_positive": 0,
      "true_negative": 14,
      "false_negative": 9,
      "selected_weight": 1,
      "selected_rate": 0.041666666666666664,
      "expected_cost": 2.25
    },
    {
      "threshold": 0.9,
      "true_positive": 2,
      "false_positive": 1,
      "true_negative": 13,
      "false_negative": 8,
      "selected_weight": 3,
      "selected_rate": 0.125,
      "expected_cost": 2.0416666666666665
    }
  ],
  "optimal_threshold": 0.1,
  "optimal_expected_cost": 0.5,
  "optimal_selected_rate": 0.9166666666666666,
  "optimal_confusion": {
    "true_positive": 10,
    "false_positive": 12,
    "true_negative": 2,
    "false_negative": 0
  },
  "costs": {
    "false_positive": 1,
    "false_negative": 6,
    "true_positive": 0,
    "true_negative": 0
  },
  "tie_break": "minimum-cost-then-lower-selected-weight-then-higher-threshold",
  "state": "threshold-selected"
}

Other exports#

This module also exports rocCurveAndRocAuc, precisionRecallCurveAndPrAuc, brierScore, logLoss, reliabilityDiagramAndExpectedCalibrationError, gainsLiftAndDecileCapture, scoreStabilityAndMigrationMatrix, sliceBasedValidationBySectorCountryAndRegime, rareEventBacktestAndConfidenceBounds, calculate. Every module additionally exports run as an alias of its primary function, and a meta object carrying its catalog id, domain, family, shape and article URL.

Diagrams#

Calculation flow#

Cost-Sensitive Threshold Optimization calculation flow

flowchart LR
    S1["Validate score direction labels weights costs and cuto"]
    S2["Create selectnone and distinctscore candidates"]
    S3["Build confusion weights at each threshold"]
    S4["Multiply cells by declared costs"]
    S5["Normalize by total weight"]
    S1 --> S2
    S2 --> S3
    S3 --> S4
    S4 --> S5
    S5 --> D{"cost assumptions and tiebreak must remain visible"}
    D --> O["optimal_expected_cost + diagnostics"]
    O --> A["Audit: For every candidate TPFPTNFN equals total weight"]

How it works#

This page states the contract — how to call it correctly. The article explains the concept: why it works, and where it breaks.

Read the article →

References#

Revised Guidance on Model Risk Management — Board of Governors of the Federal Reserve System, OCC, and FDIC
The Foundations of Cost-Sensitive Learning — Charles Elkan
Metrics and scoring: quantifying the quality of predictions — scikit-learn maintainers
Evidence boundary

Cost-Sensitive Threshold Optimization

Install and import#

Signature#

Parameters#

Returns#

Errors#

Worked example#

Input#

Call#

Returns#

Other exports#

Diagrams#

Calculation flow#

How it works#

References#

The rest of the Classification and Score Validation family#