Better segment pools

[fzhinkin]

Currently, segment pools exist only on JVM (on other platforms, implementations are effectively no-op) and behave more like caches than pools.

There are a few directions in which we may/should develop segment pools:

• support pools configuration (both in terms of the pool capacity and the size of allocated segments)
• make sure every segment is returned to a pool once there are no users remaining
• support unlinking a segment from a pool (for scenarios when we have a byte-buffer backed segment and now sending it via Netty -> segment is no longer in use once we "consumed" it by wrapping into a ByteBuf, but it could not be released as a Netty owns it now)
• support adding already allocated data into a buffer (related to Missing API: zero-copy wrapping of ByteArray with Source. #166)
• support pools on other platforms
• support pool-level isolation (for instance, if there are multiple threads make sure that each of them uses a separate pool, so that data used in one thread would never leak into another thread)
• support leak tracing (Leak tracing mechanism #144)

This is an epic describing what could be done and tracking progress rather than an instruction to what should be implemented.

[fzhinkin]

The first two points (not returning segments back to the pool and small pool size) are blockers for integration with Ktor (https://youtrack.jetbrains.com/issue/KTOR-6030, ktorio/ktor#4032), so they need to be fixed.

[e5l]

It would be great to have a system property to adjust the pool size as well

[fzhinkin]

As @bjhham pointed out, Buffer::close should release segments (currently, it's no-op).

[fzhinkin]

However, it could be problematic, as a typical Buffer use scenario is "allocate, use, and forget".

[e5l]

Maybe we can try to fix this by manual allocation buffer for writing using a different constructor method, so the buffers for channels and primitives in Ktor will be tracked