When to Use Opteryx
What is Opteryx covers what the engine is; this page is about where it fits and where it doesn't.
Good fits
- Ad-hoc analysis over files — SQL directly against Parquet, JSONL, or Skene datasets on local disk, GCS, or plain HTTP(S), without an ETL step to load them somewhere first. See Querying Local Data.
- Embedded analytics inside a Python process —
pip install opteryx-coregets you a full SQL engine in-process, with no server to run and no cluster to operate. See Embedding in a Service. - Read-heavy, analytical workloads — wide scans, filters, joins, and aggregations over columnar data, where the win comes from reading less data rather than from a distributed shuffle. The engine is built around pushing predicates and projections into the scan; see Troubleshooting Queries for what that looks like in practice.
- A dataset that fits on one machine — the design target is up to hundreds of millions of rows on a single node, not a cluster-scale table.
- Hosted, multi-tenant querying without embedding anything — opteryx.app runs the same engine as a service, reached through the Jobs API, OData, or Arrow Flight SQL, if you'd rather not run the engine yourself.
Not a good fit
- Data too large for one machine. Opteryx is single-node by design — it scales up, not out. If a working set genuinely exceeds what one machine can hold, no amount of query tuning fixes that; the answer is a smaller working set or a different, distributed engine. See Known Limits.
- Transactional or write-heavy workloads. There is no
COMMIT,ROLLBACK, or isolation level — a query reads the snapshot it resolves at plan time, and each statement commits on its own.INSERTis experimental and limited to some storage backends;UPDATE,DELETEandMERGEare experimental, need a catalog-backed table, and are refused rather than queued if another writer commits first. They are built for periodic corrections and feed merges, not for a stream of small concurrent writes. See SQL Conformance for the full statement-by-statement breakdown. - A system of record with enforced integrity constraints.
PRIMARY KEY,FOREIGN KEY,CHECK, andUNIQUEare not enforced by the engine. - Storage backends outside Parquet, JSONL, Skene, local disk, GCS, S3, and HTTP(S). Azure Blob Storage, MinIO, ORC, and Avro are not implemented — see Compatibility.
- A strict ANSI SQL, PostgreSQL, or MySQL dialect. Opteryx parses its own dialect, close to but not identical to any of those — check syntax that leans on another engine's specifics before assuming it carries over.
Still not sure?
- Known Limits lists the specific architectural and feature gaps.
- Troubleshooting Queries covers the point at which a slow query stops being a tuning problem and becomes a "wrong tool" problem.