Compare parameters and walk forward

Use run_grid to compare parameter combinations and run_walk_forward to choose parameters on an earlier slice, then evaluate them on a later slice. These helpers call a function you supply. They do not turn an on_tick function, strategy declaration or strategy module into a backtest automatically. They do not calculate trading returns, Sharpe ratios, costs or drawdown for you.

Choose a screening or execution model

Fast research modes answer narrower questions than the event-driven on_tick engine. They can screen candidate signals, but switching engines does not preserve execution prices, risk decisions, cash balances or returns.

Vectorized next-close simulation

run_vectorized(bars, on_signals, params=None) accepts a Polars DataFrame for one instrument. Required columns are timestamp (Date, Datetime or integer ordinal, strictly increasing, unique and non-null), symbol (one nonempty string, no nulls), and numeric close. Prices and any supplied open, high, low or volume must be finite and non-null. It does not sort or split mixed-symbol data for you. A typed empty frame is valid and returns empty results.

Existing daily Date inputs and integer observation indices remain supported; they are not converted to Datetime or interpreted as Unix time. Next-bar means the next input row for all three types, regardless of calendar gaps. Returned timestamps retain the input axis type. Floating-point and string timestamps are rejected; convert real date/time strings explicitly before running.

The callback is called once for each observed prefix, from one row through the full history. Return two Boolean Polars Series of exactly that prefix's length: entries and exits. Only their final values decide that bar. Fill missing warmup signals explicitly with False. Entry and exit cannot both be true on a row, including the final row. Enter only when flat and exit only while long; repeated entries or exits from flat raise NonVectorizableError, rather than silently resizing a position.

This complete example deliberately gives the next bar different open and close prices:

from datetime import datetime, timedelta
import polars as pl
from atom.engine.vectorized import run_vectorized

bars = pl.DataFrame({
    "timestamp": [datetime(2026, 1, 1) + timedelta(days=i) for i in range(3)],
    "symbol": ["A"] * 3,
    "open": [100.0, 110.0, 120.0],
    "close": [100.0, 150.0, 125.0],
})

def on_signals(bars, params):
    return pl.Series([True, False, False]).head(bars.height), pl.Series([False, True, False]).head(bars.height)

result = run_vectorized(bars, on_signals)
assert result.orders["side"].to_list() == ["BUY", "SELL"]
assert result.orders["fill_price"].to_list() == [150.0, 125.0]
assert result.equity["position_value"].to_list() == [0.0, 150.0, 0.0]

The first signal buys one unit at the next row's close, 150; the second sells at the following close, 125. A final-row signal has no following fill. With the same signals and no costs, the actual event engine's default next-open convention instead fills at 110 and 120. The difference is intentional: existing vectorized next-close behavior is retained, not silently migrated to next-open.

The historical field name equity is retained for compatibility, but contains position value only, not account equity or accumulated profit. There is no cash ledger, capital admission, commission, protective-stop, short, multi-position, settlement or recovery model here. slippage_bps defaults to zero and must be finite, nonnegative and below 10,000; it increases buy prices and decreases sell prices. A supplied fill_model.apply(order, bar) takes precedence over that shortcut and must return finite prices. This model still does not acquire the event engine's execution capabilities. Callbacks now receive only observations through the decision bar. A full-window mean is therefore an expanding historical mean; a negative shift has no next observation at the decision row and must handle its null explicitly. Future suffix changes cannot alter earlier inputs. This costs one callback per bar, so a calculation that scans every prefix can take quadratic work. Keep callbacks pure and use only the supplied history/parameters: external future-data closures or mutable global data are outside this guarantee. Previously callbacks saw the whole frame once; fixed full-length masks must now be sliced to the supplied prefix, and callbacks with side effects must be rewritten.

Start with at most 5,000 bars per research run and benchmark your own callback before increasing that budget. This is an operational guard, not an enforced limit: at 20,000 bars a full-prefix scan visits 200,010,000 rows versus 12,502,500 at 5,000 (16 times the work). Wide panels and expensive indicators multiply this cost. Use event-driven incremental on_tick calculations for larger histories, or reduce the research window explicitly; do not feed future rows to recover speed. Both vectorized signals and triggered predicates retain prefix isolation.

Triggered intent screening

run_triggered(bars, strategy, params=None, state=None) evaluates trigger_predicate(bars, params) on each observed prefix, then calls on_trigger(bar, params, state) only on matching rows. The predicate returns a Boolean Polars expression or a Python boolean. A scalar broadcasts; null, non-Boolean or misaligned results are rejected. Omitting the predicate selects every row. Original row indices appear as bar_index.

from types import SimpleNamespace
import polars as pl
from atom.engine.triggered import run_triggered

screen = SimpleNamespace(
    on_start=lambda: ({"threshold": 120.0}, {"hits": 0}),
    trigger_predicate=lambda bars, params: pl.col("close") > params["threshold"],
)

def collect(bar, params, state):
    state["hits"] += 1
    return [{"symbol": bar["symbol"], "side": "BUY", "qty": 1}]

screen.on_trigger = collect
result = run_triggered(pl.DataFrame({"symbol": ["A", "A"],
                                    "close": [100.0, 150.0]}), screen)
assert result.triggers_fired == 1
assert result.final_state["hits"] == 1
assert result.metadata["result_kind"] == "order_intents"

The 150-price row produces one intent; it is not a fill or an admitted position. No mark-to-market, risk supervision or lifecycle work occurs on skipped rows. Callbacks share in-memory state during this screen, not durable portfolio state. With multiple symbols, maintain any symbol-specific callback state yourself. Returned orders retain their existing field name but are explicitly labeled order_intents in result metadata.

Custom thresholds and periods remain accepted. on_start parameters override supplied parameters. Known execution settings in either source are rejected, even if set to None, false or zero: capital, risk limits, cash/margin, commissions/fills, warmup, checkpoint/persistence, trade logs, schedules, settlement (settlement_days and settlement_calendar) and lifecycle settings belong to the event engine. Vectorized mode permits only its documented fill model and slippage settings from this execution family. No market settlement calendar is inferred.

Engine.start() routes hook-exclusive on_signals and on_trigger strategies in backtest mode only and returns the native result in routed_result. It rejects execution clients, feeds, persistence, resume, schedules and execution overrides instead of ignoring them. Remove previously ignored execution settings only when intent screening is what you want. Otherwise retain those settings and use an on_tick strategy with the event engine. Recheck selected candidates there before interpreting trading performance. Repository contributors can read docs/benchmarks.md for measured costs and their limits; these are not trading performance or production throughput guarantees.

Supply a runner

A runner has the contract runner(params, data_window) -> metrics: two positional arguments and a flat dictionary with string keys and scalar float/int/string/bool/null values. Keep the metric names and types consistent between combinations. Metric names must not overwrite parameter names. Convert NumPy scalar results to Python scalars before returning them.

For an actual trading experiment, your runner must construct a fresh backtest, apply the parameters, run the supplied data, check execution health and derive metrics from its results. Reset positions, cash, history, indicators and strategy state for each call. The helpers neither create nor reset that state. A cached hit skips the call entirely, so runners must not depend on side effects.

This complete example verifies the plumbing using arithmetic, not a strategy or a performance claim:

from atom.research import run_grid


def score_window(params, values):
    return {"score": float(sum(values)) * params["scale"] - params["penalty"]}


def main():
    results = run_grid(
        score_window,
        {"scale": [1, 2], "penalty": [0, 3]},
        [1, 2, 3],
        workers=1,
    )
    assert results["score"].to_list() == [6.0, 3.0, 12.0, 9.0]
    print(results)


if __name__ == "__main__":
    main()

The grid visits combinations in parameter insertion order, with the last parameter varying fastest. Each result row contains the chosen parameters and returned metrics. An empty parameter mapping {} runs once with no overrides; an empty candidate list, such as {"scale": []}, raises ValueError. Candidates must be non-empty lists or tuples. An empty data sequence is passed to the runner as supplied; decide explicitly whether your runner accepts it.

workers=1 executes in the current process. workers=None, the default, uses the machine's CPU count; a positive integer chooses the limit. Zero, negative values, booleans and non-integers are errors. A single outstanding combination runs in the parent even when the limit is larger. Put runners at the top level of an importable Python module, and keep the if __name__ == "__main__" guard when running a script with multiple workers. Lambdas and local functions fail the picklability check, including with workers=1. Worker inputs also need to be picklable. A source file is required for cached runners; interactive definitions without retrievable source are not a supported cache identity.

Parallel grids serialize the runner/window once into an immutable pool input, then send parameter dictionaries through a queue bounded to eight outstanding tasks per requested worker. Mutable windows are deserialized fresh for every experiment, preserving task isolation; plain immutable scalar/bytes windows can be reused within a worker. The final result still needs one row per combination, so total result memory grows with grid size. Smaller grids may spend more time starting workers than calculating; choose workers=1 for cheap experiments.

Reuse identical experiments

Caching is off by default. Supply cache_backend=LocalDiskBackend(root=...) to run_grid, or explicitly set ATOM_CACHE=1 to use ~/.atom/cache. The backend's has, get and set calls all run in the parent process; custom backends need not be picklable. Only cache misses are submitted for execution. Cached and uncached results keep the same combination order.

The following complete example uses a temporary private cache, so it leaves no persistent research entries. Its walk-forward part uses the same arithmetic runner but does not enable caching:

import json
from tempfile import TemporaryDirectory

from atom.research import run_grid, run_walk_forward
from atom.research.cache import LocalDiskBackend


def score_window(params, values):
    return {"score": float(sum(values)) * params["scale"]}


def main():
    values = [1, 2, 3, 4, 5, 6, 7, 8]
    with TemporaryDirectory() as directory:
        backend = LocalDiskBackend(root=directory)
        first = run_grid(score_window, {"scale": [1, 2]}, values,
                         workers=1, cache_backend=backend, cache_ttl_days=7)
        again = run_grid(score_window, {"scale": [1, 2]}, values,
                         workers=1, cache_backend=backend, cache_ttl_days=7)
        assert first.equals(again)
        assert first["score"].to_list() == [36.0, 72.0]

    result = run_walk_forward(
        score_window, {"scale": [1, 2]}, values,
        train_window=3, test_window=2, step=2,
        workers=1, selection_metric="score",
    )
    assert result["train_start"].to_list() == [0, 2]
    assert result["test_end"].to_list() == [5, 7]
    assert [json.loads(p) for p in result["best_params"]] == [{"scale": 2}] * 2
    assert result["oos_score"].to_list() == [18.0, 26.0]
    print(result)


if __name__ == "__main__":
    main()

Change workers=1 to workers=2 in a source-file script to use the process pool when at least two combinations miss the cache. Cache hits do not start workers. cache_bust=True forces recomputation and replaces stored results; it does not disable caching. cache_ttl_days=None writes without expiry, including clearing an older expiry on a replaced entry. Zero expires immediately, including at the exact write timestamp. Finite negative TTLs retain their existing meaning of already expired; fractional durations are supported. NaN and positive/negative infinity are rejected. LocalDiskBackend.set validates the duration before changing either file; run_cached validates it before reading a hit or calling the runner. A custom grid backend owns its own TTL validation. A hit does not refresh its expiry; set a new TTL with a forced recomputation. Changing only TTL does not change an experiment's key.

What identifies the data?

Cache-key version 3 hashes complete content, not printed representations, and distinguishes supported timezone identities. Version 2 and older cache entries are ignored automatically; they are not deleted. Changing a hidden middle row now invalidates a result, while changing Polars display settings does not. Identity includes the following supported inputs:

• Polars DataFrames and Series: schema, column names/order, row order and values, including nulls and timestamp precision/timezone. Physical chunk boundaries are ignored. Object columns are unsupported, including nested ones.
• NumPy arrays/scalars: dtype, shape and every value in logical C order. Object arrays, structured dtypes and dtype metadata are unsupported.
• Atom Bar, lists of bars and BarSeries: frame contents and declared metadata, including timestamp where present, symbols, mode, timeframe and adjustment flag. Treat bars/series as immutable after construction; replacing their frames behind already-materialized properties is not supported.
• Nested plain lists, tuples and dictionaries of supported values, preserving their order, plus strings, bytes, bool/int/float/null, Decimal, enum values, dates/times, timedeltas, ranges, slices and pathlib paths. A path identifies its spelling, not the file contents. A string dataset identifier is safe only when you guarantee it names an immutable dataset revision.

Python datetimes and times may be naive, use the exact standard-library datetime.timezone type (offset and name both participate in identity), or use zoneinfo.ZoneInfo loaded by a named key, including ZoneInfo.no_cache(key). ZoneInfo.from_file, dateutil/pytz timezones, subclasses and other custom timezone implementations raise TypeError before a cached runner executes. For example, 2026-01-01 12:00 in ZoneInfo("America/New_York") observes a different offset 180 days later than a fixed UTC-05:00 time, so those inputs must not share a result. Fixed-offset zones with different names also get different keys because a runner can inspect tzname().

To migrate a dateutil regional datetime, explicitly convert it with stamp.astimezone(ZoneInfo("America/New_York")), using the intended regional key, before building the window. Do not strip timezone information or replace it with a fixed current offset to silence the error: that loses future DST behavior. Keyed ZoneInfo identity assumes a compatible timezone database; include your timezone-data revision in explicit metadata when comparing environments with different timezone rules.

Unsupported objects, lazy frames, sets, generators and cyclic containers raise TypeError before the cached runner executes. Convert them to supported materialized data or turn caching off by omitting the backend and unsetting ATOM_CACHE. cache_bust=True still requires a valid identity. Uncached grids can pass other objects through, subject to the worker serialization contract.

Keys also include parameters, runner source and Atom version. Function defaults and supported closure values are included, but imported helper implementations, dependency versions, external files, random seeds and mutable globals are not automatically tracked. Put every result-affecting revision/seed/configuration in explicit parameters or window metadata, and have your runner use it. Keep inputs unchanged during a call. Identical content is reusable within a compatible serialization environment; dependency upgrades can produce safe cache misses. Full-content hashing takes time and memory proportional to data size, so caching very cheap runners may cost more than recomputing them.

Cached metrics are JSON, never pickle. Invalid cache content raises an error; use cache_bust=True to replace a damaged entry. The lower-level run_cached returns stored bytes on a hit and stores only byte results on a miss; its caller must encode/decode a safe data format. It does not serialize arbitrary Python objects for you.

Use a private directory you control. Each cache entry stores its payload and expiry together in one versioned record, committed through an exclusive random temporary file and atomic replacement. Readers see one complete payload/expiry pair, including with concurrent writers. A failed replacement leaves the previous entry intact; both get and has enforce expiry. Opt-in JSON state checkpoints retain their separate atomic writer. An existing predictable temporary symlink is not followed. The cache rejects existing entry symlinks. Direct backend keys must be nonempty ASCII filename tokens: start with a letter, digit, underscore or hyphen, followed by those characters or dots. For example, run_1-v3.result is valid; ../run, absolute paths and backslashes are rejected for reads and writes. Normal run_grid/run_cached keys already use safe generated hashes. Nonfinite expiry metadata is treated as a cache miss. These checks do not protect a directory that an adversary can modify between filesystem operations, authenticate cached results or make a writable shared directory trusted. Atomic replacement is not a complete power-loss durability guarantee.

For example, an entry written at time 100 with a one-second TTL becomes a miss at time 101 even if an earlier has call succeeded. Replacing it with a no-expiry entry commits that payload and its no-expiry state together. A failed replacement cannot remove the old expiration or publish only the new bytes. Local files now use .entry; old .bin/.ttl pairs are ignored and recomputed on demand because an interrupted old write cannot be migrated safely. They are not automatically deleted. The public backend still accepts and returns opaque payload bytes.

Choose walk-forward windows

Window sizes and step count observations, not days. Pass a chronological, sliceable sequence, usually a list of bars. With train 3, test 2 and step 2, the example trains on indices [0, 3), tests on [3, 5), then trains on [2, 5) and tests on [5, 7). The trailing eighth observation cannot fit a third complete pair and is omitted. Each training slice precedes its own test slice. Different folds can overlap; step < test_window overlaps test slices, so summing their results would double-count observations.

Train/test/step must all be positive integers; booleans and fractional values are rejected before window generation. Empty or insufficient data produces an empty DataFrame with no columns. Invalid worker counts and empty candidate lists still raise even when no complete window fits. The helper does not sort data, validate timestamps or detect information leakage inside your runner.

selection_metric defaults to "sharpe", but your runner must actually supply that metric. Higher is better; negate a loss if you want to minimize it. Every training score must be a finite number. Missing metrics now raise instead of silently selecting the first combination. Ties choose the first combination. The chosen original parameter values are passed to the test call, which runs in the parent process. No training state or positions are carried into it. Include any warmup/history handling deliberately in your runner without using future observations to select parameters.

Rows contain exclusive-end index boundaries, best_params as JSON text and test metrics prefixed oos_. Use JSON-compatible parameter values when you need to reconstruct them exactly; non-JSON values in best_params use string representations. There is no automatic aggregation or trading-return claim. run_walk_forward has no backend/TTL arguments; ATOM_CACHE=1 enables only its training grids' default cache, not test calls. For isolated experiments leave that environment flag unset.