warefeats
Source

Benchmarks Caching proxies

Candidates

HTTP caching proxies for HLS delivery

Tick the candidates you care about and the charts, winners and table redraw for just them: same run, same rig, same samples. Winners come from the numbers alone; the story is in the post.

From the run of on MacBook Pro, Apple M2 Max. Read the post

Hit-path TTFB at 5000 rps

Test 1 p50 TTFB

50th-percentile time to first byte at 5000 rps (LOWER ms = FASTER)

p50 TTFB (ms)
Varnish 9.0.3 (plaintext)0.5 ms
Varnish 9.0.3 (TLS)0.4 ms
Varnish 9.0.3 (PROXYv2)0.5 ms
Vinyl c67fd5f57 (plaintext)0.4 ms
Vinyl c67fd5f57 (PROXYv2)0.5 ms
NGINX stable (plaintext)0.7 ms
NGINX stable (TLS)0.4 ms
NGINX stable (PROXYv2)0.4 ms
p50 TTFB in ms
Varnish 9.0.3 (plaintext)0.5 ms
Varnish 9.0.3 (TLS)0.4 ms
Varnish 9.0.3 (PROXYv2)0.5 ms
Vinyl c67fd5f57 (plaintext)0.4 ms
Vinyl c67fd5f57 (PROXYv2)0.5 ms
NGINX stable (plaintext)0.7 ms
NGINX stable (TLS)0.4 ms
NGINX stable (PROXYv2)0.4 ms

Test 2 p99 TTFB

99th-percentile time to first byte at 5000 rps (LOWER ms = FASTER)

p99 TTFB (ms)
Varnish 9.0.3 (plaintext)2.6 ms
Varnish 9.0.3 (TLS)3.1 ms
Varnish 9.0.3 (PROXYv2)6.0 ms
Vinyl c67fd5f57 (plaintext)0.8 ms
Vinyl c67fd5f57 (PROXYv2)5.3 ms
NGINX stable (plaintext)2.4 ms
NGINX stable (TLS)3.4 ms
NGINX stable (PROXYv2)2.3 ms
p99 TTFB in ms
Varnish 9.0.3 (plaintext)2.6 ms
Varnish 9.0.3 (TLS)3.1 ms
Varnish 9.0.3 (PROXYv2)6.0 ms
Vinyl c67fd5f57 (plaintext)0.8 ms
Vinyl c67fd5f57 (PROXYv2)5.3 ms
NGINX stable (plaintext)2.4 ms
NGINX stable (TLS)3.4 ms
NGINX stable (PROXYv2)2.3 ms

Winner Vinyl c67fd5f57 (plaintext) (2.2 ms)

4 MB segment serve

Test 1 p50 full GET

50th-percentile TTFB for 4 MB full GET at 500 rps (LOWER ms = FASTER)

p50 full GET (ms)
Varnish 9.0.3 (plaintext)2.0 ms
Varnish 9.0.3 (TLS)3.6 ms
Varnish 9.0.3 (PROXYv2)5.0 ms
Vinyl c67fd5f57 (plaintext)2.2 ms
Vinyl c67fd5f57 (PROXYv2)5.3 ms
NGINX stable (plaintext)2.2 ms
NGINX stable (TLS)9.3 ms
NGINX stable (PROXYv2)5.2 ms
p50 full GET in ms
Varnish 9.0.3 (plaintext)2.0 ms
Varnish 9.0.3 (TLS)3.6 ms
Varnish 9.0.3 (PROXYv2)5.0 ms
Vinyl c67fd5f57 (plaintext)2.2 ms
Vinyl c67fd5f57 (PROXYv2)5.3 ms
NGINX stable (plaintext)2.2 ms
NGINX stable (TLS)9.3 ms
NGINX stable (PROXYv2)5.2 ms

Test 2 p50 range GET

50th-percentile TTFB for 64 KB range GET at 500 rps (LOWER ms = FASTER)

p50 range GET (ms)
Varnish 9.0.3 (plaintext)0.4 ms
Varnish 9.0.3 (TLS)0.5 ms
Varnish 9.0.3 (PROXYv2)0.7 ms
Vinyl c67fd5f57 (plaintext)0.5 ms
Vinyl c67fd5f57 (PROXYv2)0.9 ms
NGINX stable (plaintext)0.4 ms
NGINX stable (TLS)0.7 ms
NGINX stable (PROXYv2)0.9 ms
p50 range GET in ms
Varnish 9.0.3 (plaintext)0.4 ms
Varnish 9.0.3 (TLS)0.5 ms
Varnish 9.0.3 (PROXYv2)0.7 ms
Vinyl c67fd5f57 (plaintext)0.5 ms
Vinyl c67fd5f57 (PROXYv2)0.9 ms
NGINX stable (plaintext)0.4 ms
NGINX stable (TLS)0.7 ms
NGINX stable (PROXYv2)0.9 ms

Test 3 p99 full GET

99th-percentile TTFB for 4 MB full GET at 500 rps (LOWER ms = FASTER)

p99 full GET (ms)
Varnish 9.0.3 (plaintext)2.8 ms
Varnish 9.0.3 (TLS)10.4 ms
Varnish 9.0.3 (PROXYv2)17.2 ms
Vinyl c67fd5f57 (plaintext)3.0 ms
Vinyl c67fd5f57 (PROXYv2)17.9 ms
NGINX stable (plaintext)5.7 ms
NGINX stable (TLS)64.9 ms
NGINX stable (PROXYv2)15.9 ms
p99 full GET in ms
Varnish 9.0.3 (plaintext)2.8 ms
Varnish 9.0.3 (TLS)10.4 ms
Varnish 9.0.3 (PROXYv2)17.2 ms
Vinyl c67fd5f57 (plaintext)3.0 ms
Vinyl c67fd5f57 (PROXYv2)17.9 ms
NGINX stable (plaintext)5.7 ms
NGINX stable (TLS)64.9 ms
NGINX stable (PROXYv2)15.9 ms

Test 4 p99 range GET

99th-percentile TTFB for 64 KB range GET at 500 rps (LOWER ms = FASTER)

p99 range GET (ms)
Varnish 9.0.3 (plaintext)1.0 ms
Varnish 9.0.3 (TLS)1.0 ms
Varnish 9.0.3 (PROXYv2)2.8 ms
Vinyl c67fd5f57 (plaintext)1.8 ms
Vinyl c67fd5f57 (PROXYv2)3.3 ms
NGINX stable (plaintext)1.2 ms
NGINX stable (TLS)3.6 ms
NGINX stable (PROXYv2)3.8 ms
p99 range GET in ms
Varnish 9.0.3 (plaintext)1.0 ms
Varnish 9.0.3 (TLS)1.0 ms
Varnish 9.0.3 (PROXYv2)2.8 ms
Vinyl c67fd5f57 (plaintext)1.8 ms
Vinyl c67fd5f57 (PROXYv2)3.3 ms
NGINX stable (plaintext)1.2 ms
NGINX stable (TLS)3.6 ms
NGINX stable (PROXYv2)3.8 ms

Winner Varnish 9.0.3 (plaintext) (3.1 ms)

Miss-storm coalescing

Test 1 p50 TTFB

50th-percentile TTFB per 200-client burst on a cold segment (LOWER ms = FASTER)

p50 TTFB (ms)
Vinyl c67fd5f57743.6 ms
NGINX stable854.1 ms
Varnish 9.0.31.051 s
p50 TTFB in ms
Vinyl c67fd5f57743.6 ms
NGINX stable854.1 ms
Varnish 9.0.31.051 s

Test 2 p99 TTFB

99th-percentile TTFB per 200-client burst on a cold segment (LOWER ms = FASTER)

p99 TTFB (ms)
Vinyl c67fd5f571.245 s
NGINX stable2.719 s
Varnish 9.0.32.713 s
p99 TTFB in ms
Vinyl c67fd5f571.245 s
NGINX stable2.719 s
Varnish 9.0.32.713 s

Test 3 Coalescing efficiency

Ratio of concurrent clients to origin requests (200x = perfect coalescing) (HIGHER x = BETTER)

Coalescing efficiency (x)
Vinyl c67fd5f57200 x
NGINX stable200 x
Varnish 9.0.3200 x
Coalescing efficiency in x
Vinyl c67fd5f57200 x
NGINX stable200 x
Varnish 9.0.3200 x

Winner Vinyl c67fd5f57 (plaintext) (703.7 ms)

Origin-flap grace

Test 1 p50 TTFB

50th-percentile TTFB during origin failure cycles (LOWER ms = FASTER)

p50 TTFB (ms)
NGINX stable5.9 ms
Vinyl c67fd5f575.9 ms
Varnish 9.0.36.1 ms
p50 TTFB in ms
NGINX stable5.9 ms
Vinyl c67fd5f575.9 ms
Varnish 9.0.36.1 ms

Test 2 Grace hit ratio

Percentage of requests served from stale cache during origin failure (HIGHER % = BETTER)

Grace hit ratio (%)
NGINX stable100 %
Vinyl c67fd5f57100 %
Varnish 9.0.3100 %
Grace hit ratio in %
NGINX stable100 %
Vinyl c67fd5f57100 %
Varnish 9.0.3100 %

Winner NGINX stable (plaintext) (5.9 ms)

Test conditions: rig, corpus, protocol
Rig
Machine
MacBook Pro
Chip
Apple M2 Max
Cores
12 CPU cores
Memory
96 GB
OS
macOS 26.0 (arm64)
Runtime
Docker (OrbStack)
Protocol
Warmups
3 unmeasured passes
Measured
20 passes per tool
Process
Container per engine, shared origin
Cache
Warm cache after three curl requests (W1/W2) or cold by construction (W3)
Output
TTFB via oha 1.16.0 with --latency-correction

Scorecard

Mean of each section's samples for the candidates you picked. The best value in each column is marked in red.
CandidateHit-path TTFB at 5000 rps (ms, lower is better)4 MB segment serve (ms, lower is better)Miss-storm coalescing (ms, lower is better)Origin-flap grace (ms, lower is better)
Varnish 9.0.3 (plaintext)3.4 ms3.1 ms830.5 ms6.2 ms
Varnish 9.0.3 (TLS)3.7 ms10.9 ms––
Varnish 9.0.3 (PROXYv2)5.8 ms20.3 ms––
Vinyl c67fd5f57 (plaintext)2.2 ms3.4 ms703.7 ms6.0 ms
Vinyl c67fd5f57 (PROXYv2)5.7 ms39.5 ms––
NGINX stable (plaintext)2.6 ms6.5 ms986.6 ms5.9 ms
NGINX stable (TLS)3.5 ms67.2 ms––
NGINX stable (PROXYv2)3.2 ms15.9 ms––

Logos identify the products under test and imply no endorsement. Varnish is a registered trademark of Varnish Software AB. NGINX is a trademark of F5, Inc. Vinyl Cache logo: CC BY 4.0 Rhubarbe.design.