Refresh-Time Sampling

Install and import

bash

npm install fintech-algorithms

import { refreshTimeSample } from "fintech-algorithms/market-data-engineering/time-synchronization/refresh-time-sampling";

Signature

refreshTimeSample(observations, requiredInstruments, maxStalenessMs)

Builds a common clock for several instruments by advancing only when every one of them has refreshed. The standard remedy for the bias that non-synchronous trading introduces into correlations.

Parameters

Name	Type	Notes
`observations`	`Observation[]`	Observations across all instruments, each with `partition`, `instrument`, `event_time` and `available_at`.
`requiredInstruments`	`string[]`	The instruments that must all have refreshed before a refresh time is emitted.
`maxStalenessMs`	`number`	How old any instrument's value may be at a refresh time. min: 0

Returns

{ rows, partitions }

The synchronised rows, plus per-partition diagnostics showing which instrument was the binding constraint at each step.

Errors

When requiredInstruments is empty — throws

Complexity: time O(n log n), space O(n).

Worked example

executed Captured by running this function on the input its own test provides. Real output of real code — but not asserted against a published figure.

Input

observations

[
  {
    "partition": "P1",
    "instrument": "A",
    "record_id": "a1",
    "revision": 0,
    "event_time": "2026-01-01T00:00:01.000Z",
    "available_at": "2026-01-01T00:00:01.000Z",
    "value": 1
  },
  {
    "partition": "P1",
    "instrument": "A",
    "record_id": "a3",
    "revision": 0,
    "event_time": "2026-01-01T00:00:03.000Z",
    "available_at": "2026-01-01T00:00:03.000Z",
    "value": 3
  },
  {
    "partition": "P1",
    "instrument": "A",
    "record_id": "a4",
    "revision": 0,
    "event_time": "2026-01-01T00:00:04.000Z",
    "available_at": "2026-01-01T00:00:04.000Z",
    "value": 4
  }
]

Showing 3 of 5 elements.

requiredInstruments

["A", "B"]

maxStalenessMs

Call

refreshTimeSample(observations, requiredInstruments, maxStalenessMs)

Returns

object with 2 fields: rows, partitions

{
  "rows": [
    {
      "partition": "P1",
      "sequence": 1,
      "previous_refresh_available_at": null,
      "refresh_available_at": "2026-01-01T00:00:02.000Z",
      "controller_instruments": ["B"],
      "status": "accepted",
      "values": {
        "A": 1,
        "B": 20
      },
      "sources": {
        "A": {
          "record_id": "a1",
          "revision": 0,
          "event_time": "2026-01-01T00:00:01.000Z",
          "available_at": "2026-01-01T00:00:01.000Z"
        },
        "B": {
          "record_id": "b2",
          "revision": 0,
          "event_time": "2026-01-01T00:00:02.000Z",
          "available_at": "2026-01-01T00:00:02.000Z"
        }
      },
      "event_age_ms": {
        "A": 1000,
        "B": 0
      },
      "arrivals_by_instrument": {
        "A": 1,
        "B": 1
      },
      "discarded_updates": 0
    },
    {
      "partition": "P1",
      "sequence": 2,
      "previous_refresh_available_at": "2026-01-01T00:00:02.000Z",
      "refresh_available_at": "2026-01-01T00:00:04.000Z",
      "controller_instruments": ["B"],
      "status": "accepted",
      "values": {
        "A": 4,
        "B": 40
      },
      "sources": {
        "A": {
          "record_id": "a4",
          "revision": 0,
          "event_time": "2026-01-01T00:00:04.000Z",
          "available_at": "2026-01-01T00:00:04.000Z"
        },
        "B": {
          "record_id": "b4",
          "revision": 0,
          "event_time": "2026-01-01T00:00:04.000Z",
          "available_at": "2026-01-01T00:00:04.000Z"
        }
      },
      "event_age_ms": {
        "A": 0,
        "B": 0
      },
      "arrivals_by_instrument": {
        "A": 2,
        "B": 1
      },
      "discarded_updates": 1
    }
  ],
  "partitions": [
    {
      "partition": "P1",
      "input_updates": 5,
      "refresh_candidates": 2,
      "accepted_rows": 2,
      "stale_rows": 0,
      "discarded_updates": 1,
      "unmatched_tail_updates": 0,
      "loss_fraction": {
        "numerator": 1,
        "denominator": 5
      }
    }
  ]
}

Diagrams

Calculation flow

Availability-time refresh sequence

sequenceDiagram
    participant A as Fast stream A
    participant B as Medium stream B
    participant C as Sparse stream C
    participant S as Refresh sampler
    A->>S: first new availability A1
    A->>S: later availability A2
    B->>S: first new availability B1
    Note over S: wait, C is missing
    C->>S: first new availability C1
    S->>S: barrier = max(A1, B1, C1)
    S-->>S: select latest known effective event per stream

How it works

This page states the contract — how to call it correctly. The article explains the concept: why it works, and where it breaks.

Read the article →

References

Multivariate realised kernels — Ole E. Barndorff-Nielsen, Peter R. Hansen, Asger Lunde, and Neil Shephard
On covariance estimation of non-synchronously observed diffusion processes — Takaki Hayashi and Nakahiro Yoshida
RFC 3339 — Internet Engineering Task Force; Graham Klyne and Chris Newman
Evidence classification