Snapshot/Incremental-Feed Reconciliation

Install and import

bash

npm install fintech-algorithms

import { reconcileSnapshotIncrementals } from "fintech-algorithms/market-data-engineering/order-book-feed-engineering/snapshot-incremental-feed-reconciliation";

Signature

reconcileSnapshotIncrementals(snapshot)

Reconciles a periodic snapshot against the book built from incrementals. Any divergence means the incremental path has been silently wrong — possibly for hours — and this is the only routine that catches it.

Parameters

Name	Type	Notes
`snapshot`	`Snapshot`	The authoritative snapshot, together with the incrementally-maintained book to compare it against.

Returns

{ status, differences, matched_levels, … }

Level-by-level differences, so a divergence can be localised rather than triggering a blanket rebuild.

Errors

When the snapshot and book are for different instruments — throws

Complexity: time O(levels), space O(differences).

Worked example

executed Captured by running this function on the input its own test provides. Real output of real code — but not asserted against a published figure.

Input

snapshot

{
  "bids": [
    {
      "price_ticks": 10000,
      "quantity": 20
    },
    {
      "price_ticks": 9999,
      "quantity": 25
    },
    {
      "price_ticks": 9998,
      "quantity": 30
    }
  ],
  "asks": [
    {
      "price_ticks": 10002,
      "quantity": 18
    },
    {
      "price_ticks": 10003,
      "quantity": 22
    },
    {
      "price_ticks": 10004,
      "quantity": 26
    }
  ]
}

argument 2

{
  "snapshot_sequence": 200,
  "incrementals": [
    {
      "sequence": 201,
      "side": "bid",
      "price_ticks": 10000,
      "quantity": 10
    },
    {
      "sequence": 202,
      "side": "ask",
      "price_ticks": 10002,
      "quantity": 13
    },
    {
      "sequence": 203,
      "side": "bid",
      "price_ticks": 9999,
      "quantity": 16
    }
  ],
  "checkpoint": {
    "bids": [
      {
        "price_ticks": 10000,
        "quantity": 23
      },
      {
        "price_ticks": 9999,
        "quantity": 29
      },
      {
        "price_ticks": 9998,
        "quantity": 35
      }
    ],
    "asks": [
      {
        "price_ticks": 10002,
        "quantity": 26
      },
      {
        "price_ticks": 10004,
        "quantity": 38
      },
      {
        "price_ticks": 10005,
        "quantity": 15
      }
    ]
  },
  "checkpoint_sequence": 224
}

Call

reconcileSnapshotIncrementals(snapshot)

Returns

object with 11 fields: snapshot_sequence, checkpoint_sequence, compared_level_count, difference_count, mismatch_counts, replayed_incremental_count, pending_after_checkpoint_count, differences, …

{
  "snapshot_sequence": 200,
  "checkpoint_sequence": 224,
  "compared_level_count": 12,
  "difference_count": 0,
  "mismatch_counts": {
    "quantity-mismatch": 0,
    "missing-reconstructed-level": 0,
    "missing-checkpoint-level": 0
  },
  "replayed_incremental_count": 24,
  "pending_after_checkpoint_count": 0,
  "differences": [],
  "reconstructed": {
    "bids": [
      {
        "price_ticks": 10000,
        "quantity": 23
      },
      {
        "price_ticks": 9999,
        "quantity": 29
      },
      {
        "price_ticks": 9998,
        "quantity": 35
      }
    ],
    "asks": [
      {
        "price_ticks": 10002,
        "quantity": 26
      },
      {
        "price_ticks": 10004,
        "quantity": 38
      },
      {
        "price_ticks": 10005,
        "quantity": 15
      }
    ]
  },
  "checkpoint": {
    "bids": [
      {
        "price_ticks": 10000,
        "quantity": 23
      },
      {
        "price_ticks": 9999,
        "quantity": 29
      },
      {
        "price_ticks": 9998,
        "quantity": 35
      }
    ],
    "asks": [
      {
        "price_ticks": 10002,
        "quantity": 26
      },
      {
        "price_ticks": 10004,
        "quantity": 38
      },
      {
        "price_ticks": 10005,
        "quantity": 15
      }
    ]
  },
  "state": "reconciled"
}

Other exports

This module also exports normalizeEvents, reconstructL2, aggregatePriceLevels, reconstructL3, recoverSequenceStream, consolidateVenues, calculate. Every module additionally exports run as an alias of its primary function, and a meta object carrying its catalog id, domain, family, shape and article URL.