Compatibility matrix (tested against Spark)

Each row is a test that runs the same query with VecRuntime on and off and compares the results with Spark's — identical rows (doubles at tolerance 1e-9, since SIMD reductions reorder floating-point additions), and the expected VecRuntime operator asserted in the final post-AQE plan. A row marked ✓ means the query ran on a VecRuntime operator and matched Spark; a row marked falls back means the planner declined it, the query stayed on Spark, and the test asserts the recorded fallback reason. A known gap is a case the suite marks ignore.

The rows below come only from the five ported test suites named in each section (and, for the range() leaf, from its own suite) — nothing is inferred for a case that is not tested. For the full per-operator and per-expression reference (every condition and every fallback reason), see Supported operators and Supported expressions; this page is the tested-against-Spark subset.

Every matrix here is ported from Apache DataFusion Comet (github.com/apache/datafusion-comet); each section names the upstream Comet suite it follows, from the port's own header comment.

# Expressions

Ported from Apache DataFusion Comet's CometBitwiseExpressionSuite, CometMathExpressionSuite and CometExpressionSuite. Every case runs a SELECT over a mixed Parquet table and asserts the projection ran on VectorProjectExec, in non-ANSI (legacy) arithmetic so overflow and divide-by-zero rows null rather than raise. Source: VectorPortedCometExprSuite.scala (#497).

ANSI overflow / divide-by-zero raises are a separate suite's concern (the golden files under nonansi/); these cases pin the arithmetic, not the raise.

# Casts

Ported from Apache DataFusion Comet's CometCastSuite — one test per (from, to) pair over generated columns with edge values (min/max, zero, NaN, ±Infinity, nulls, valid/invalid string spellings). A CAST is a projection, so ✓ = the result matched Spark and a VectorProjectExec is in the plan; a target with no lane in this project falls back. Session zone fixed to UTC. Source: VectorPortedCometCastSuite.scala (#500).

The grid, from-type (rows) × to-type (columns). ✓ = accelerated and equal to Spark; fb = falls back with a reason; n/a = not exercised as a cast pair by the suite.

Notes.

• Narrowing (bigint→int, double→int/bigint): legacy wraps/truncates toward zero (NaN→0, ±Inf saturates) and matches Spark's null path; ANSI is run over an in-range column so no raise is expected (the ANSI raise itself is out of scope).
• Boolean: v != 0, NaN is true; boolean→number is 1/0.
• To string: Java toString, including the NaN/Infinity/-Infinity spellings.
• String→number/date/timestamp: Spark's own parser per row; invalid spellings null in legacy.
• Decimal(≤18): rescale-and-check; a value out of range nulls in legacy (ci→decimal(18,2), cl→decimal(18,0), cd→decimal(10,2)).
• Fallbacks (contract, not a wrong answer): → float — unsupported cast target float; string → decimal(10,2) — unsupported cast string -> decimal(10,2) (Spark's Decimal.fromString not reproduced yet); → binary — unsupported cast target binary.
• tinyint/smallint are deliberately not a clean cast-target fallback here: #327 compiles int → byte/short onto the int lane via NarrowIntExpr, so they are not asserted in this grid.

# Aggregates

Ported from Apache DataFusion Comet's CometAggregateSuite. Every case runs the aggregate twice and asserts a VectorHashAggregateExec in the accelerated plan — all cases in this suite are expected to stay on our operator. Source: VectorPortedCometAggregateSuite.scala (#501).

A grouped aggregate over zero rows is folded to EmptyRelation by AQE and has no operator to assert — deliberately not tested. An ungrouped aggregate over zero rows still emits one row and is tested.

# Joins

Ported from Apache DataFusion Comet's CometJoinSuite, over Comet's two-column (k, v) tables built so keys repeat on both sides. Each case asserts the specific vector join operator, or a fallback with its reason. Source: VectorPortedCometJoinSuite.scala (#507).

# Window functions

Ported from Apache DataFusion Comet's CometWindowExecSuite, over Comet's (a, b, c) fixture plus a decimal(10,2) measure d (partitions, an order key with ties, a null partition, a null value). Each case asserts a VectorWindowExec, or a fallback with its reason. Source: VectorPortedCometWindowSuite.scala (#510).

Analysis errors, not fallbacks (both engines raise the same): a non-literal lag/lead offset (lag(b, c)) is a Spark analysis error; nth_value without an ORDER BY is a Spark analysis error. The suite asserts both engines raise, so these are neither accelerated nor a recorded fallback.

# Range (spark.range, the range() table-valued function)

Not a Comet port: VectorRangeExec replaces Spark's RangeExec leaf, so every shape below runs the same range() with the plugin on and off and compares the rows -- and, since the leaf decides the partitions, the partition-dependent results too. Each case asserts VectorRangeExec in the final plan, that Spark's RangeExec is gone, and that no RowToColumnarExec sits above ours. Source: VectorRangeSuite.scala (lines 67-291) and, for the batch lifecycle without a Spark job, VectorRangeIteratorSuite.scala.