Why Opteryx Is Fast

Opteryx doesn't get its speed from one big trick. It comes from a single habit applied at every layer of the engine: don't do work the query doesn't need. Don't read columns nobody asked for. Don't decode row groups that can't match. Don't compare a string 100 million times when it has 200 distinct values. Don't make the same decision once per row when it can be made once per query.

This page walks through where that habit shows up, then gives the published numbers, with the machine, version and date for each. It ends with where Opteryx isn't the fastest option, because that's worth knowing too.

Read Less

The cheapest byte is the one you never read. Before anything is decoded, Opteryx narrows a scan down in stages. Each stage is described in Rugo, the engine's native file reader:

• Projection. Only the columns a query references are decoded. The others are never read from disk.
• Row-group pruning. Parquet keeps min/max statistics for every row group. If a row group's range proves it holds no matching row, it's skipped without decoding a single value. For = and IN, which min/max handles poorly, Rugo also checks bloom filters where the file has them. Pruning fails open: if a statistic is missing, the row group is read rather than wrongly skipped.
• Dictionary skipping. A dictionary-encoded column carries a small table of its distinct values. Rugo tests the predicate against that table first. If no entry matches, the column's data pages are never decoded.
• Materialise last. Projection and filtering happen before bytes are turned into typed values. The expensive step runs on the smallest amount of data the query allows.

The optimizer feeds all of this. Predicate and projection pushdown move filters and column lists as close to the scan as they'll go, so the reader knows exactly what it may skip. See How Opteryx Plans and Runs a Query.

Keep Data Compact

Once data is in memory, Opteryx keeps the structure the file already had instead of flattening it. Columns live in Draken vectors, which come in three shapes: dense, dictionary and constant.

A dictionary column stores each distinct value once, plus a small integer code per row. That saves memory, but it saves even more work:

A constant shape goes further: a literal in WHERE region = 'APAC', or a page where every value is the same, is stored once with a row count. Reading it costs the same for any number of rows.

Strings get their own layout. Every value takes a fixed 16-byte slot. Strings of 12 bytes or fewer, which covers most identifiers, codes and categories, sit entirely inside the slot. Longer strings keep their length and first four bytes there. Equality is usually settled by a single 64-bit compare, and ordering by a 4-byte prefix compare, without touching the string bytes at all. See How Draken Stores Strings.

Decide Once, Then Run

Expressions such as price * quantity, a > 10 AND b LIKE 'x%' or a CASE are compiled by the bytecode engine once, while the query is planned, not per row and not per batch:

• Each literal becomes a constant vector once, rather than a value created per row.
• Each operation is bound to a native function pointer up front, so running it is a direct call rather than a name lookup.
• Temporary results come from a per-batch memory arena that's freed in one go, so there are no per-row objects to clean up.

By the time data flows, every decision has already been made. The data path just runs.

Stay Native While It Runs

Planning is Python; execution is native. The handover happens once, when the physical plan is built, and the executor never calls back into Python to decide anything. Data moves through the operators in morsels, batches of columnar vectors, so each operator handles a batch in one pass instead of once per row.

The native scheduler runs operators across worker threads with Python's interpreter lock released, so execution isn't held up by Python. Reading is part of the same pipeline: Rugo fetches byte ranges on worker threads while other threads decode, so waiting for one row group overlaps with work on another. See Execution Model.

Start Instantly

Speed also means not having to wait before the first query.

• No load step. Opteryx queries Parquet, JSONL and CSV where they already are. Its ClickBench entry records a load time of zero, and its JSONBench entry reads the published files directly with no schema and no index.
• No cluster. It's one process. Embedded, that process is yours. Hosted at opteryx.app, there's nothing to provision or size.
• A small footprint. Rugo, the file reader underneath, installs at 11.6 MB with no dependencies and imports in 30 ms. PyArrow is 153.7 MB and 88 ms, Polars 213.8 MB and 186 ms (rugo.dev, Debian 12 x86_64, CPython 3.12, measured 14 August 2026).

Reading Less Costs Less

On the hosted service, the data queried part of the bill is the data a query actually reads (Cost Model). Each technique under Read Less that skips a column, row group or page therefore lowers what that query costs as well as how long it takes. Studio shows the bytes scanned under each result, so you can see the effect query by query.

The Numbers

Benchmarks deserve scepticism, including ours. Benchmarking explains what we run, how it's measured, and what the numbers don't tell you. Every figure below links to its source, so you can check it and rerun it.

ClickBench

ClickBench runs 43 analytical queries over a 100-million-row web-analytics table. The Opteryx entry is in the ClickBench repository and reads the partitioned Parquet files directly. On ClickBench's reference machine:

"Cold" here is strict. Before the first run of every query, the harness restarts the process and drops the operating system's page cache.

Opteryx isn't the fastest entry. On the same machine, ClickHouse, DuckDB and DataFusion reading the same Parquet files are faster on warm runs, and DuckDB is faster cold. Use the live comparison to see where Opteryx lands for the machine and engines you care about.

Release Over Release

Wrenchy Bench reruns the suites release over release, on AWS Graviton4 with 16 CPUs, to catch regressions and track progress. As of engine 0.9.155 on 4 October 2026, across 48 recorded runs:

The same harness also runs ClickBench against Skene, the engine's own file format, in 10.89 s. Skene is still a draft format, so treat that figure as a preview.

JSONBench

JSONBench queries Bluesky event data stored as JSON. Opteryx reads the NDJSON files directly, with no load step, schema or index. The entry is awaiting review; these figures are from that submission (opteryx-core 0.9.153, m6i.8xlarge):

The cold runs are limited by the benchmark's disk (a default gp3 volume, 125 MB/s), not by the engine. Nothing is loaded or cached ahead of time, so a cold query reads every byte from storage.

Where It Isn't Fast

Knowing the limits matters as much as the numbers:

• One machine. Opteryx scales up, not out. There's no distributed execution, so a working set bigger than one machine can handle needs a different engine. See Known Limits.
• Cold reads are bound by storage. Because there's no load step, the first query over files on slow storage runs at the speed of that storage. Pruning helps exactly as much as the files' statistics allow.
• Files without statistics or dictionaries prune less. Row-group and dictionary skipping depend on what the writer put in the file. A Parquet file with no min/max statistics, or a CSV, has to be read in full.

When to Use Opteryx covers where it fits, and where something else will serve you better.

Related

• How Opteryx Plans and Runs a Query
• Rugo: the file engine
• Draken: the vector library
• The bytecode expression engine
• Benchmarking