Tessera

What it is

Testing a market idea shouldn't require building an engine first. Three obstacles stand in the way, and the middle one is where most attempts die: getting the data, making it usable, and expressing the strategy. Tessera addresses all three as one system.

The four principles everything follows from

These are not aspirations. Each one has cost a design decision somewhere in this document, and each rejects an approach that would otherwise have been easier.

• Wrong answers are worse than no answers. A backtest that produces plausible-but-wrong numbers is the worst thing this software can do — and with live execution, that number has money attached. Errors are a closed catalogue; there is no "log it and carry on".
• Local first. The default install runs entirely on your machine: no server, no network. Cloud is an optional topology of the same software, never a requirement. This principle alone has vetoed several otherwise-attractive designs.
• The format is the product. The specification is public, versioned and precise. If someone writes a better engine against it, the format still won.
• Written for agents as well as people. A stable vocabulary, a published schema, and errors that name the offending field — so a machine can author a strategy and correct itself from the rejection.

Layers, and which way they point

The kernel sits at the bottom of the dependency graph and imports nothing above it. That is not a convention — an architecture test enforces it, failing the build on any internal import or any standard-library import outside an explicit allow-list.

The kernel holds only the types every part must agree on — money, time, instrument identity, units, the error catalogue. It deliberately contains no record layout, no adapter, no evaluator and no strategy grammar, so a change in any of those cannot ripple through the type everyone shares.

How data moves

Why there are two formats

Backtesting has two access patterns that pull in opposite directions, and this is the single fact the storage design turns on. The system being replaced kept two stores for exactly this reason — that was not accidental duplication, it was the patterns asserting themselves.

That split is measured, not assumed. On one instrument-day of real data (800 contracts, 2,018,361 ticks), an mmap'd fixed-width binary read a single contract in 7 µs against Parquet's 2.36 ms — but Parquet compressed to 2.37 bytes/row against the raw record's 16, and beat a purpose-built per-contract ZSTD binary by 2.4×.

The fixed grid

The bar cache stores no timestamps. Position is the timestamp: record i is minute i from the session open, so finding any bar is arithmetic rather than a search.

The two record shapes

Confusing these is easy, so the distinction is worth stating plainly: one is truth, the other is a convenience that can always be rebuilt.

Money is an integer count of minor units throughout, never a float; the scale is venue data and is enforced to be a power of ten. Timestamps are parsed against an explicit venue location and stored as absolute UTC, so a run cannot depend on the host's clock settings.

message tessera_tick {                          struct CacheBar {   // 24 bytes, LE
  required int32  contract_id;   // dict+RLE      int32 open, high, low, close;
  required int64  ts (TIMESTAMP(MICROS,utc));     int32 volume;
  required int32  seq;           // RLE           int32 oi;
  required int64  ltp_paise;                    }
  required int64  ltq;
  required int64  oi;                           // empty minute = INT32_MIN
}                                               // offset = start + idx*24

One contract, all the way through

Real rows from the sample — a liquid at-the-money weekly call on 2022-10-03 — at each stage. Note the first four ticks share a second, which is exactly the case seq exists for.

filename   NIFTY22100617100CE.csv
key        OPT:NIFTY:20221006:W:CE:1710000

raw        20221003,09:15:00,132.65,1,2967350
           20221003,09:15:00,132.65,1,2967350
           20221003,09:15:00,93.85,1,2967350
           20221003,09:15:01,101.70,11700,2967350

tick       contract_id  ts (UTC)              seq   ltp_paise     ltq        oi
           4711         2022-10-03T03:45:00Z    0       13265       1   2967350
           4711         2022-10-03T03:45:00Z    1       13265       1   2967350
           4711         2022-10-03T03:45:00Z    2        9385       1   2967350
           4711         2022-10-03T03:45:01Z    0       10170   11700   2967350

bar        09:15  open 13265  high 13275  low 9000  close 13160  vol 1461803
           15:29  open  4640  high  4650  low 4560  close  4560  vol  442700

bytes      d1 33 00 00  db 33 00 00  28 23 00 00  68 33 00 00  2b 4e 16 00  36 47 2d 00
           open=13265   high=13275   low=9000     close=13160  vol=1461803  oi=2967350

result     21,843 ticks → 375 one-minute bars → 9,000 bytes

The index spot series is not a special case. A vendor's BANKNIFTY.csv is the underlying series, so it becomes EQ:BANKNIFTY and lives in the same file and the same tables as that day's contracts, with volume and open interest at zero because an index has neither. There is deliberately no separate index asset class.

Four classes, and no fifth

Every failure is classified, and a condition that fits none of the first three fails loudly. There is no "log it and continue" — that habit is what let the system being replaced return a result whether all, some or none of its work succeeded.

Partial success is unrepresentable as success. A function that can partly fail returns a result carrying its holes, each naming which unit failed — never a bare slice with a nil error. The check is called Complete() rather than OK() precisely so it is never read as a nil-error test.

Coverage is a first-class output of every run, and counts correct non-entry separately from failure. A selective strategy that enters 38 times across five years produces thousands of no-trigger candidates; folding those into holes would report a 97% failure rate for a system working exactly as intended.

Reproducibility is a property, not an aspiration

Same document, same data, same engine version means identical numbers on any machine. Several rules exist only to hold that line:

• Money is integer minor units; rates and percentages are exact rationals. One function rounds, half-to-even, in 128-bit intermediate precision. No float64 appears anywhere on the money path.
• Timestamps are parsed against an explicit venue location — never the host zone — and an architecture test fails the build if any source file calls the timezone-defaulting parse function.
• Anything rendered from a map sorts its keys first, so iteration order cannot leak into a report and make two runs differ.
• Ticks are written once and never edited. Bars are re-derived from them, never patched.

Build status

built & tested   in progress   specified, no code   decision open

• P1 foundations Money, time, instrument taxonomy, units and the error catalogue are built and tested — 9 of 14 acceptance criteria. Identity and config remain; the provider seam is deferred to P4 by decision.
• P2 venue & catalog A minimal calendar slice is scheduled before P4: expiry-bearing instruments cannot be constructed without it.
• P3 storage Records, file store and catalog schema specified. The tick section still describes the pre-D30 binary format and needs reconciling to Parquet.
• P4 conversion Adapter specified. Test data extracted and verified: 48,226 files across four trading days.
• P5–P10 Strategy format, resolution, evaluation, reporting and CLI all have written specs with acceptance criteria; no code yet.
• Licence Open-core split agreed — Apache for the format and SDK, AGPL plus CLA for the engine. Pending qualified legal sign-off.
• Source repository Not public yet. The design, the decision log and the specs are published here; the repository opens once the licence above is settled, so that what is opened is opened under the right terms rather than retroactively.
• Regulatory posture Blocking for live execution. Needs counsel, not inference.

Decision log

Every decision is recorded with its reasoning and, where one was overturned, what replaced it. Reversals are kept rather than edited away.

Roadmap

The risk is not difficulty, it is never finishing anything end to end. So v0 is one of everything, proven all the way through before any axis widens.

What is still open

Recorded rather than resolved, because a written unknown is safer than an assumed answer.

• Are raw ticks actually cold? Keeping Parquet for ticks rests on the evaluator riding the bar cache. If real strategies replay raw ticks heavily, the latency gap matters and a custom binary becomes competitive again. Needs a profile of a real run.
• Licence. The open-core split is agreed in shape; the exact terms need qualified legal advice before release.
• Regulatory posture for algorithmic trading. Blocking for live execution. It governs who may run it and what compliance is required — not whether the code is open.
• The catalog locator against Parquet. Byte offsets are a contiguous-extent idiom that maps awkwardly onto row groups; the addressing needs pinning down before the first tick file is written.
• Trigger timing. Evaluate the partial candle, or trigger on close and enter at the next open. Different strategies, different results.