Add SIEVE eviction policy

[MichaReiser]

Summary

This adds a new SIEVE eviction policy (eviction(policy = sieve, capacity = 4000)). It is optimized for throughput and should, for most use cases, be more efficient (picking the right items to evict) than LRU. I picked this new generic eviction keyword so that we can add other eviction policies in the future (generational, within same revision, etc)

I decided to remove support for lru(1000), I'd be okay restoring the syntax but, given the other breaking changes that we made recently, this feels minimal and it also hints users towards using sieve over LRU

Closes #1032

Benchmarks

Codspeed benchmarks side to side

Benchmark	NoEviction	LRU	SIEVE
fast_path	185.07 µs	242.22 µs (+30.9%)	196.99 µs (+6.4%)
disabled_eviction	185.09 µs	185.92 µs (+0.4%)	189.27 µs (+2.3%)
fast_path_and_sweep	186.34 µs	243.42 µs (+30.6%)	198.30 µs (+6.4%)
fill_and_evict	331.34 µs	390.57 µs (+17.9%)	361.86 µs (+9.2%)
one_hit_wonders	216.1 µs	331.5 µs (+53.4%)	246.5 µs (+14.1%)
parallel_fast_path	148.0 µs	2.241 ms (+1414.2%)	154.9 µs (+4.7%)
phase_change	75.78 µs	90.68 µs (+19.7%)	82.93 µs (+9.4%)
project_check_then_incremental	2.426 ms	3.192 ms (+31.6%)	2.575 ms (+6.1%)
scan_resistance	198.2 µs	275.2 µs (+38.8%)	281.1 µs (+41.8%)

The only case where SIEVE is slower is scan_resistance. This is not unexpected, because SIEVE is not scan resident. This is where an algorithm like W-Tiny-LFU would show better results (at the cost of higher computations in the fast path etc)

Running ty on homeassistant:

Threads	NoEviction	LRU	SIEVE
1	40.178 s	40.149 s (−0.1%)	40.049 s (−0.3%)
4	10.932 s	11.072 s (+1.3%)	10.956 s (+0.2%)
16	3.143 s	3.234 s (+2.9%)	3.134 s (−0.3%)
32	1.977 s	2.377 s (+20.2%)	1.958 s (−1.0%)

Now, we only use eviction for parsed_module, which hasn't the most hits and is itself a fairly heavy query. I did the same with infer_expression_types, which we have more instances of and is also a much lighter query (does very limited work). Here, the overhead becomes noticeable, but is still close to noise:

Threads	NoEviction	LRU	SIEVE
4	11.027 s	11.102 s (+0.7%)	11.088 s (+0.6%)
8	5.769 s	5.847 s (+1.3%)	5.805 s (+0.6%)
16	3.169 s	3.235 s (+2.1%)	3.208 s (+1.2%)
32	2.040 s	2.344 s (+14.9%)	2.081 s (+2.0%)

My read is that this is mainly due to the exclusive lock on the insertion path and a limit of 10k is probably also way too low, meaning we push a lot of instances into the eviction queue. infer_expression_types is also a much better fit for within-revision GC, because recomputation is so cheap, that it's rarely worth caching. Except for the few instances that are very expensive (expensive control flows, cycle, ...). But that's another side project, which hopefully will build on top of the SIEVE work in this PR

An even more extreme example from ty (semantic_index)

Threads	NoEviction	LRU	SIEVE
4	10.957 s	11.854 s (+8.2%)	11.060 s (+0.9%)
8	5.778 s	7.556 s (+30.8%)	5.792 s (+0.2%)
16	3.193 s	12.674 s (+297.0%)	3.177 s (−0.5%)
32	2.065 s	20.927 s (+913.2%)	2.038 s (−1.3%)

[netlify]

✅ Deploy Preview for salsa-rs canceled.

Name	Link
🔨 Latest commit	90b8802
🔍 Latest deploy log	https://app.netlify.com/projects/salsa-rs/deploys/6a4ff90b15d9de00087d6c22

[codspeed-hq]

Merging this PR will degrade performance by 6.84%

⚠️ Different runtime environments detected

Some benchmarks with significant performance changes were compared across different runtime environments,
which may affect the accuracy of the results.

Open the report in CodSpeed to investigate

❌ 4 regressed benchmarks
✅ 70 untouched benchmarks
🆕 13 new benchmarks
⏩ 1 skipped benchmark¹

Warning

Please fix the performance issues or acknowledge them on CodSpeed.

Performance Changes

	Mode	Benchmark	BASE	HEAD	Efficiency
❌	Memory	fill_and_evict[NoEviction]	23.4 KB	25.9 KB	-9.64%
❌	Memory	fill_and_evict[Lru]	35.5 KB	38 KB	-6.58%
❌	WallTime	project_check_then_incremental[Lru]	10 ms	10.6 ms	-6.09%
❌	WallTime	phase_change[Lru]	373.3 µs	392.8 µs	-4.98%
🆕	WallTime	one_hit_wonders[Sieve]	N/A	996.4 µs	N/A
🆕	WallTime	parallel_fast_path[Sieve]	N/A	413.2 µs	N/A
🆕	WallTime	phase_change[Sieve]	N/A	355.3 µs	N/A
🆕	WallTime	project_check_then_incremental[Sieve]	N/A	7 ms	N/A
🆕	WallTime	scan_resistance[Sieve]	N/A	1.2 ms	N/A
🆕	Memory	disabled_eviction[Sieve]	N/A	0 B	N/A
🆕	Memory	fast_path_and_sweep[Sieve]	N/A	0 B	N/A
🆕	Memory	fast_path[Sieve]	N/A	0 B	N/A
🆕	Memory	fill_and_evict[Sieve]	N/A	27 KB	N/A
🆕	Simulation	disabled_eviction[Sieve]	N/A	182.7 µs	N/A
🆕	Simulation	fast_path_and_sweep[Sieve]	N/A	196 µs	N/A
🆕	Simulation	fast_path[Sieve]	N/A	194.7 µs	N/A
🆕	Simulation	fill_and_evict[Sieve]	N/A	421.7 µs	N/A

Tip

Investigate this regression by commenting @codspeedbot fix this regression on this PR, or directly use the CodSpeed MCP with your agent.

---

_{Comparing MichaReiser:sieve-eviction-policy (90b8802) with master (25ed9d3)}

Footnotes

1. 1 benchmark was skipped, so the baseline result was used instead. If it was deleted from the codebase, click here and archive it to remove it from the performance reports. ↩

[MichaReiser]

The concurrent benchmarks look promising. But I need to do a pass over the benchmarks. I don't think they really measure what they should

[MichaReiser]

Okay, I think this is ready. The implementation is a bit more involved because I wanted to minimize overhead and maximize throughput in concurrent use cases.

[lnicola]

FWIW, I came across an S3-FIFO implementation, which should handle scans a bit better. While I can't quite vouch for its correctness, it might be worth looking into.

[MichaReiser]

@lnicola can you share a branch with the implementation?

The results look promising and, from a quick chat with codex, S3-FIFO also seems to perform slightly better in selection. However, a key focus of my work has been to optimize for high-throughput under heavy concurrent load. And it seems, SIEVE still performs better there. But maybe that' something that can be optimized.

[lnicola]

@MichaReiser https://github.com/lnicola/salsa/tree/s3-fifo-0.28.2, but I won't vouch for its correctness.

Nb. it also includes the SIEVE PR to make testing easier.