dnsdist: add generic object cache

[esensar]

Short description

This adds a new object usable from Lua - GenericCache - which can be used to store any values under string keys and retrieve them later on.

Checklist

I have:

• read the CONTRIBUTING.md document
• read and accepted the Developer Certificate of Origin document, including the AI Policy, and added a "Signed-off-by" to my commits
• compiled this code
• tested this code
• included documentation (including possible behaviour changes)
• documented the code
• added or modified regression test(s)
• added or modified unit test(s)

---

Sponsored by Quad9

[esensar]

This was mostly implemented to work with: #17387, but it could be useful otherwise too I think.
The idea is to also integrate it into redis clients, since delays introduced by redis lookups would be too slow for most users.
That is already implemented in our testing fork, with support for result caching, copy caching (copying all redis values that could be found by the specified lookup action), filtering (using the object cache as a set of keys, but also bloom and cuckoo filters) and negative caching.
This is just a small initial PR, to check if this is acceptable in the context of dnsdist.

[rgacogne]

Thanks for opening this PR! I'll do a high-level review soon to see if this is something that are interested in merging, please don't spend more time on this until we have reached a decision!

[esensar]

Thanks for opening this PR! I'll do a high-level review soon to see if this is something that are interested in merging, please don't spend more time on this until we have reached a decision!

Thanks. That makes sense.

[coveralls]

Coverage Report for CI Build 28880705225

Warning

Build has drifted: This PR's base is out of sync with its target branch, so coverage data may include unrelated changes.
Quick fix: rebase this PR. Learn more →

Coverage decreased (-1.7%) to 69.36%

Details

• Coverage decreased (-1.7%) from the base build.
• Patch coverage: No coverable lines changed in this PR.
• 3420 coverage regressions across 67 files.

Uncovered Changes

No uncovered changes found.

Coverage Regressions

3420 previously-covered lines in 67 files lost coverage.

Top 10 Files by Coverage Loss	Lines Losing Coverage	Coverage
pdns/tcpiohandler.cc	521	27.29%
pdns/libssl.cc	500	6.54%
pdns/ws-auth.cc	314	80.63%
pdns/packethandler.cc	279	69.01%
pdns/lua-record.cc	233	0.16%
pdns/dnsparser.cc	227	73.05%
pdns/tcpiohandler.hh	179	31.19%
pdns/misc.cc	142	50.45%
pdns/iputils.hh	118	71.0%
pdns/iputils.cc	109	40.87%

---

Coverage Stats

Relevant Lines:	114931
Covered Lines:	86440
Line Coverage:	75.21%
Relevant Branches:	57362
Covered Branches:	33063
Branch Coverage:	57.64%
Branches in Coverage %:	Yes
Coverage Strength:	7327930.66 hits per line

---

💛 - Coveralls

[rgacogne]

I'm wondering if we could somehow merge this work with the one happening in #16692, by implementing a generic enough cache class and then specializing it for DNS packets. That way we would (hopefully) avoid duplicating a lot of Lua bindings / YAML settings, if we manage to make them generic enough to apply to the packet cache and this generic object cache. Or am I too optimistic? :-)

[esensar]

Well it seems like it could be possible, considering this cache is supposed to be generic. I need to take a deeper look into that PR, but at first glance it seems possible.

[esensar]

I'm wondering if we could somehow merge this work with the one happening in #16692, by implementing a generic enough cache class and then specializing it for DNS packets. That way we would (hopefully) avoid duplicating a lot of Lua bindings / YAML settings, if we manage to make them generic enough to apply to the packet cache and this generic object cache. Or am I too optimistic? :-)

What is the status on that PR? I believe I could integrate this work into that (with some minor modifications maybe, not sure), but I just wanted to know if that is a solid foundation that I could build on or if it is something that will change a lot.

[rgacogne]

I'm wondering if we could somehow merge this work with the one happening in #16692, by implementing a generic enough cache class and then specializing it for DNS packets. That way we would (hopefully) avoid duplicating a lot of Lua bindings / YAML settings, if we manage to make them generic enough to apply to the packet cache and this generic object cache. Or am I too optimistic? :-)

What is the status on that PR? I believe I could integrate this work into that (with some minor modifications maybe, not sure), but I just wanted to know if that is a solid foundation that I could build on or if it is something that will change a lot.

That's a very good question! Perhaps @karelbilek can chime in? One possibility might be to define a generic enough interface that you could based your work on, and even if it turns out that we need to change a few things later that shouldn't be too hard. Note that we could also approach this the other way around: use the existing interface as the basis for your generic object cache, merge it before #16692, and adapt it later if needed for the changes that #16692 wants to introduce.

[esensar]

I'm wondering if we could somehow merge this work with the one happening in #16692, by implementing a generic enough cache class and then specializing it for DNS packets. That way we would (hopefully) avoid duplicating a lot of Lua bindings / YAML settings, if we manage to make them generic enough to apply to the packet cache and this generic object cache. Or am I too optimistic? :-)

What is the status on that PR? I believe I could integrate this work into that (with some minor modifications maybe, not sure), but I just wanted to know if that is a solid foundation that I could build on or if it is something that will change a lot.

That's a very good question! Perhaps @karelbilek can chime in? One possibility might be to define a generic enough interface that you could based your work on, and even if it turns out that we need to change a few things later that shouldn't be too hard. Note that we could also approach this the other way around: use the existing interface as the basis for your generic object cache, merge it before #16692, and adapt it later if needed for the changes that #16692 wants to introduce.

That second option sounds good too. Maybe better to not base my work off of another draft PR. We can easily adjust later I think.

[esensar]

Just to confirm - the existing interface refers to DNSDistPacketCache?

[rgacogne]

Just to confirm - the existing interface refers to DNSDistPacketCache?

Yes, it doesn't have to be fully compatible, of course!

[esensar]

@rgacogne did you mean something like this? I haven't been able to use the same iinterface exactly (because of some packet cache specific things), but the generic cache is largely based on the packet cache and I have just changed the packet cache to use the generic cache internally.

[rgacogne]

@rgacogne did you mean something like this? I haven't been able to use the same iinterface exactly (because of some packet cache specific things), but the generic cache is largely based on the packet cache and I have just changed the packet cache to use the generic cache internally.

That's pretty close to what I had in mind, yes, thank you! I did not review all the changes yet but at a quick glance this looks great, code duplication is very low and the new interfaces will likely make it easier to implement S3-FIFO/LRU/etc versions of the packet cache in the future.

[esensar]

If this approach makes sense, should I start adding tests and documentation for this, to get this PR ready for review?

[rgacogne]

That sounds great, please do :)