VecRuntime vs Gluten + Velox vs DataFusion Comet on TPC-DS 3 TB, AWS Graviton4

VecRuntime runs Spark SQL's filters, projections, aggregates, sorts and joins on Arrow-layout batches with the Java Vector API -- on the JVM, no native code -- and moves batches between executors over its own Arrow Flight shuffle. awslabs data-on-eks publishes TPC-DS 3 TB results for two native engines, Gluten + Velox and DataFusion Comet, on twelve Graviton4 nodes. This page runs VecRuntime on the same generated data, the same instance type and count and the same executor layout, and sets its times against theirs.

Summary

Engine	Completion time (s)	vs Gluten	vs Comet	Per-query geomean vs Gluten / Comet	Faster than Gluten / Comet on
VecRuntime (300 partitions)	1,069.1	2.10x	2.31x	1.74x / 1.71x	92 / 94
Gluten + Velox 1.6.0	2,239.9
DataFusion Comet 0.16.0	2,467.5

Where each engine is ahead

Largest leads

Query	VecRuntime (s)	Gluten (s)	Comet (s)	vs Gluten	vs Comet
q72	11.8	193.5	32.5	16.33x	2.74x
q95	11.8	106.7	69.2	9.04x	5.87x
q88	10.5	76.5	58.2	7.29x	5.55x
q22	1.9	10.1	8.4	5.43x	4.49x
q2	7.2	27.8	29.9	3.84x	4.13x
q9	21.7	82.0	55.3	3.77x	2.54x
q30	4.2	14.2	8.8	3.39x	2.11x
q60	2.3	6.9	7.2	3.05x	3.20x
q45	2.2	6.4	6.3	2.96x	2.88x
q23b	38.6	113.7	242.6	2.95x	6.29x

Where VecRuntime is behind

Queries where Gluten or Comet is faster than VecRuntime at 300 partitions, by the gap to the faster of the two (seconds).

Query	VecRuntime 300	Gluten	Comet	Gap
q87	14.6	11.2	12.5	+3.4
q38	14.3	11.1	12.5	+3.1
q99	11.0	8.0	13.5	+3.0
q1	6.1	4.7	3.7	+2.5
q58	3.0	2.3	1.8	+1.2
q78	47.5	46.4	67.6	+1.1
q83	1.7	1.5	1.2	+0.6
q97	13.4	13.1	17.4	+0.3
q39b	3.1	2.8	6.7	+0.3
q61	2.5	3.3	2.2	+0.3
q41	0.7	0.6	0.5	+0.2
q21	1.4	1.4	1.2	+0.2
q73	2.1	2.6	2.1	+0.0

Benchmark infrastructure

Methodology. The dataset was generated with data-on-eks's own generator image and arguments, and VecRuntime ran on the node type, node count and executor layout of their benchmark (23 executors x 5 cores, 58 GB each), at 300 shuffle partitions with AQE coalescing against their 200, and on S3A's classic input stream. The run had the cluster to itself; each query ran once after the plan was compiled, and its time is the wall-clock of its execution as the runner measures it.

Test environment

Component	Configuration
Dataset	TPC-DS scale factor 3000 (3 TB), Parquet on Amazon S3, generated with the data-on-eks generator image and arguments unchanged (103 query variants)
Cluster	Amazon EKS; 12 x r8gd.12xlarge (48 vCPU AWS Graviton4, 384 GB, local NVMe as RAID0 for shuffle and spill), the data-on-eks benchmark's instance type and count
Executors	23 executors x 5 cores x 58 GB each, as data-on-eks (Gluten / Comet: 20 GB heap + 6 GB overhead + 32 GB off-heap); VecRuntime: 35 GB heap + 23 GB overhead; driver 5 cores x 24 GB
Storage	Amazon S3 through Hadoop 3.4.3 S3A with the classic input stream (fs.s3a.input.stream.type=classic, the Analytics Accelerator off, #835)

Versions

Component	Version
VecRuntime	main at af9d367 (2026-10-18): the native Parquet reader on S3A's classic input stream (#835); arm64 image
Spark (VecRuntime)	4.1.3, Scala 2.13, Amazon Corretto 25.0.4.1
Gluten + Velox (data-on-eks)	1.6.0 on Spark 3.5.8, Scala 2.12, JDK 17
DataFusion Comet (data-on-eks)	0.16.0 on Spark 3.5.8, Scala 2.12, JDK 17

VecRuntime configuration

spark.plugins=io.vecruntime.spark.VectorPlugin
spark.shuffle.manager=org.apache.spark.sql.vecruntime.shuffle.VectorShuffleManager
spark.sql.shuffle.partitions=300   (data-on-eks: 200, Spark's default)
spark.sql.adaptive.enabled=true
spark.sql.adaptive.maxShuffledHashJoinLocalMapThreshold=64MB
spark.vecruntime.exec.strictFloatingPoint=false
spark.hadoop.fs.s3a.connection.maximum=1000, threads.max=256, max.total.tasks=128
spark.hadoop.fs.s3a.input.stream.type=classic   (the Analytics Accelerator off, #835)
Executors: -XX:+UseParallelGC -XX:ParallelGCThreads=5 (data-on-eks runs ParallelGC too); ACCP as the JCA provider
--add-modules=jdk.incubator.vector --enable-native-access=ALL-UNNAMED (driver and executors)

Notes

• The Gluten and Comet numbers are data-on-eks's published results (the mean of their iterations); they were not re-run here. VecRuntime's are one measured iteration per query.
• VecRuntime runs at 300 shuffle partitions with AQE coalescing, the setting of its own benchmarks; data-on-eks's Gluten and Comet runs use 200. At 200 partitions on 2026-10-16 (main fac37478, the Analytics Accelerator stream) VecRuntime took 1,118.6 s.
• VecRuntime reads S3 through S3A's classic input stream, the default of the benchmark launcher since #835.
• VecRuntime needs the larger overhead (35 GB + 23 GB): at 200 partitions with 44 GB + 14 GB an executor exceeded its container in q93.
• The engines run on different Spark lines: VecRuntime needs Spark 4.1+ and JDK 25, the data-on-eks Gluten and Comet runs are on Spark 3.5.8 and JDK 17.
• There is no Apache Spark baseline at 3 TB, so result checksums are not compared against Spark here; every row count and checksum equals the 2026-10-16 run's, and VecRuntime's results match Spark's on the 1 TB runs and in the test suites.
• The previous versions of this page are kept in benchmarks/results/tpcds-sf3000-doeks-2026-10-16/ (200 partitions) and benchmarks/results/tpcds-sf3000-doeks-2026-10-10/.

Per query

Seconds per query (hover for values; click a legend entry to hide an engine).

All queries

Seconds per query, the fastest engine in bold

Query	VecRuntime 300	Gluten	Comet	vs Gluten	vs Comet
q1	6.1	4.7	3.7	0.76x	0.59x
q2	7.2	27.8	29.9	3.84x	4.13x
q3	3.2	5.0	4.1	1.57x	1.29x
q4	32.9	60.8	71.8	1.85x	2.18x
q5	9.9	11.5	19.7	1.15x	1.98x
q6	2.6	3.9	4.9	1.54x	1.92x
q7	5.6	9.5	7.3	1.70x	1.31x
q8	2.2	3.5	4.4	1.63x	2.02x
q9	21.7	82.0	55.3	3.77x	2.54x
q10	3.2	5.5	4.1	1.71x	1.27x
q11	13.9	33.3	33.7	2.40x	2.43x
q12	1.5	2.1	2.4	1.42x	1.58x
q13	6.4	11.9	7.3	1.88x	1.16x
q14a	55.5	76.6	90.1	1.38x	1.62x
q14b	49.5	71.8	86.9	1.45x	1.76x
q15	2.2	5.1	6.6	2.29x	2.96x
q16	9.0	25.2	23.7	2.82x	2.64x
q17	4.8	4.9	6.9	1.01x	1.42x
q18	6.3	15.3	10.1	2.43x	1.61x
q19	2.7	4.0	3.3	1.50x	1.23x
q20	1.3	3.0	3.3	2.40x	2.60x
q21	1.4	1.4	1.2	0.96x	0.86x
q22	1.9	10.1	8.4	5.43x	4.49x
q23a	36.3	99.2	203.2	2.73x	5.59x
q23b	38.6	113.7	242.6	2.95x	6.29x
q24a	42.8	83.1	82.8	1.94x	1.93x
q24b	41.2	81.7	79.6	1.98x	1.93x
q25	4.9	5.1	6.0	1.05x	1.24x
q26	3.7	5.7	5.1	1.57x	1.38x
q27	4.5	9.9	5.6	2.20x	1.24x
q28	42.4	113.1	95.2	2.67x	2.25x
q29	11.1	14.2	26.5	1.28x	2.39x
q30	4.2	14.2	8.8	3.39x	2.11x
q31	3.7	10.1	10.1	2.69x	2.70x
q32	1.1	2.5	1.8	2.17x	1.55x
q33	2.8	5.4	4.1	1.94x	1.46x
q34	3.1	4.9	3.6	1.58x	1.16x
q35	5.9	8.0	10.3	1.36x	1.74x
q36	3.7	8.6	6.0	2.36x	1.66x
q37	9.6	12.5	11.9	1.30x	1.23x
q38	14.3	11.1	12.5	0.78x	0.88x
q39a	3.6	4.6	7.2	1.29x	2.02x
q39b	3.1	2.8	6.7	0.90x	2.16x
q40	3.5	6.8	9.9	1.95x	2.82x
q41	0.7	0.6	0.5	0.85x	0.69x
q42	1.0	1.5	1.3	1.50x	1.28x
q43	2.7	4.0	3.8	1.45x	1.37x
q44	24.6	32.3	27.8	1.31x	1.13x
q45	2.2	6.4	6.3	2.96x	2.88x
q46	4.2	7.7	6.9	1.82x	1.63x
q47	10.1	18.3	21.6	1.81x	2.13x
q48	5.0	7.8	5.3	1.56x	1.05x
q49	8.9	16.3	17.7	1.84x	2.00x
q50	27.7	33.2	107.5	1.20x	3.88x
q51	6.4	8.2	9.0	1.27x	1.41x
q52	1.4	1.9	1.4	1.36x	1.05x
q53	3.1	4.5	3.6	1.44x	1.15x
q54	2.9	3.6	3.4	1.23x	1.19x
q55	1.1	1.6	1.3	1.39x	1.17x
q56	1.8	2.5	2.0	1.35x	1.07x
q57	7.6	15.5	15.8	2.05x	2.08x
q58	3.0	2.3	1.8	0.78x	0.61x
q59	5.9	9.2	22.7	1.56x	3.87x
q60	2.3	6.9	7.2	3.05x	3.20x
q61	2.5	3.3	2.2	1.33x	0.90x
q62	4.6	5.2	9.4	1.14x	2.06x
q63	3.1	4.8	3.6	1.54x	1.16x
q64	35.8	61.7	95.2	1.72x	2.66x
q65	14.9	15.5	17.3	1.03x	1.16x
q66	5.2	9.2	7.5	1.75x	1.44x
q67	34.1	64.8	50.7	1.90x	1.49x
q68	2.5	6.4	4.3	2.56x	1.71x
q69	2.3	4.8	3.9	2.10x	1.71x
q70	5.2	9.0	11.0	1.74x	2.11x
q71	2.6	3.5	3.0	1.35x	1.15x
q72	11.8	193.5	32.5	16.33x	2.74x
q73	2.1	2.6	2.1	1.22x	0.99x
q74	11.3	23.2	24.7	2.05x	2.18x
q75	26.3	44.7	45.8	1.70x	1.74x
q76	25.2	61.2	35.9	2.43x	1.43x
q77	2.1	3.4	2.1	1.64x	1.02x
q78	47.5	46.4	67.6	0.98x	1.42x
q79	3.8	6.0	4.3	1.56x	1.12x
q80	9.2	19.3	20.7	2.11x	2.26x
q81	4.0	10.8	8.4	2.74x	2.13x
q82	10.6	13.7	14.4	1.30x	1.37x
q83	1.7	1.5	1.2	0.86x	0.68x
q84	3.0	8.0	8.7	2.66x	2.89x
q85	6.9	11.1	8.0	1.60x	1.16x
q86	1.8	4.7	3.7	2.61x	2.07x
q87	14.6	11.2	12.5	0.77x	0.86x
q88	10.5	76.5	58.2	7.29x	5.55x
q89	3.2	4.6	4.0	1.46x	1.27x
q90	4.8	9.0	8.9	1.88x	1.86x
q91	1.6	2.1	1.8	1.29x	1.10x
q92	1.1	2.2	1.6	1.94x	1.40x
q93	35.7	47.7	157.1	1.34x	4.40x
q94	12.1	22.0	22.8	1.83x	1.89x
q95	11.8	106.7	69.2	9.04x	5.87x
q96	10.0	13.9	11.9	1.39x	1.19x
q97	13.4	13.1	17.4	0.98x	1.29x
q98	1.7	4.3	5.1	2.50x	2.95x
q99	11.0	8.0	13.5	0.73x	1.23x

Running the benchmark

The cluster runner, the Spark-on-Kubernetes manifests and the image are in the benchmark runner's README; the image builds for arm64 on an arm64 node. run-matrix.sh writes one JSON-lines result file per run; this page is rendered from one of them and data-on-eks's two result CSVs by benchmarks/scripts/render-doeks-page.py. The 1 TB runs against Apache Spark are on the Graviton4 1 TB page.

TPC-DS is a benchmark of the Transaction Processing Performance Council; these results are not audited TPC results and are not comparable to published TPC-DS results. VecRuntime's times are one measured iteration per query; run-to-run variation on the heavy queries is a few percent.