PoolKit

PoolKit is a managed framework for creating pools of resources. A pool is simply a set of resources. You can borrow resources from it, you can return them back (or ask the pool to forget them), have them brushed up for reuse and then borrow them again. Again and again. Over and over.

Installation and usage

PoolKit is a framework that ships as one (or several) DLLs along with XML documentation and some PDB-s. Therefore, it, on its own, does not require any installation other than copying the libraries along with your product. Hooking the library up in your project is as simple as referencing those DLLs (make sure you put the XML file beside DLLs, so Visual Studio will be able to pull the comments out, to use with Intellisense). With that done, you should be able to start constructing pools with PoolKit.

Supported frameworks

Currently only .Net Framework 4 is supported. This is subject to change in the future.

When do you need pools

Pools might come in handy in several cases. Imagine you have a resource that takes a long time to initialize -- a database connection is a good example of such a resource. Then imagine your service is hitting the database creating a new connection in every service request. This would work faster if you remove that "long time" the connection takes to initialize and reuse the connections you've created by pooling them and then taking them from pool. In fact, such a mechanism already exists for database connections. With PoolKit you can create similar mechanism for any kind of resource.

There are other performance-related scenarios in which you might need to use pools. Say, you have a performance critical part of application where you cannot afford to spend even a hundred of milliseconds allocating those next fifty instances of some resource. You can preallocate them beforehand (when it's not a problem to allocate -- for example, when application starts), pool them and query the pool for them afterwards. Another twist of the story might be like this: you systematically create a lot of objects of some type and they go out of scope really fast -- it might be a good idea to have those objects reused instead of creating them from scratch every time.

Following, is a piece of code that shows how you can construct and use a pool via borrow and return pattern. Note that I'm leaving the discussion of what strategies (and their functional counterparts) are, for later sections:
using (var byteArrayPool =
                Pools.CreateStaticPoolToleratingStarvation(
                    arraysToPool,
                    new ReusePreparationFunction<byte[]>(x=>x.Initialize(), ThreadingContract.Multithreaded),
                    new RepairStrategyNotSupportingRepairs<string>(),
                    new ResourceCreationFunction<byte[]>(()=>
                                                             {
                                                                 return new byte[TestByteArraySize];
                                                             }, ThreadingContract.Multithreaded)))
{
    for (int i = 0; i < 50; i++)
    {
        byte[] array = byteArrayPool.Lend();

        // working with the array

        byteArrayPool.AcceptBack(array);
    }
}

Note It's always a good strategy to test if resource allocation is a bottleneck. Often it is not, and PoolKit does not help your application in those cases -- make sure you measure before putting it to use.

Note PoolKit is no replacement for Garbage Collector. GC is a cleverly crafted thing that does a great job in .Net.

Kinds of pools

In current version there are two kinds of pools -- Static Pool Tolerating Starvation and Dynamic Pool.

Static Pool Tolerating Starvation is dead-simple: you create it with a set of resources, and this set cannot grow or shrink in size, no matter what you do. When drained, the pool cannot lend you any more resources, it starts to construct resources on the expense of threads calling into it. Note, however, that it does not put newly created resources to pool. When you don't need eviction of old resources or keeping up with the load, and the amount of resources you will ever need is known beforehand, it is a good idea to stick to the Static Pool.

Note By making a Static Pool give up on the resources (by calling GiveUp() instead of Lend()), you render it useless, as it eventually degrades to creating resources on the calling threads. It's recommended to use Lend() -- AcceptBack() pair when using the Static Pool.

Dynamic Pool is a more complicated thing than the Static one. It's different in two ways. Dynamic Pool can, too, generate resources on the fly, but it will strive to keep at least the minimum count of them ready at all times for you. That is, when you construct a dynamic pool telling it to have at least 10 resources available at all times, and try to starve it, it will keep producing and pooling resources until it has at least 10 of them available. The second way, in which Dynamic Pool is different from the Static one, is this: Dynamic pool has eviction strategies built-in, and it evicts resources that haven't been touched for a while (honoring the minimum resource count, of course).

Customizing pools

Strategy, a well known GoF pattern (strategy is an object that encapsulates an algorithm) is used to customize how the pools work. Currently there are three strategies you should be aware of: ReusePreparationStrategy<TResource>, RepairStrategy<TResource>, ResourceCreationStrategy<TResource>. All of them have one thing in common: virtual property named ThreadingContract.

Note By default ThreadingContract property tells the strategy isn't thread-safe (that is, it cannot be called from several threads at once), override the property if you believe your strategy is, -- this will gain you some speed. If you are not sure about thread-safety, leave the default.

Out of those three strategies, RepairStrategy is the only one demanding a real explanation. Well, here goes. When you return a broken resource to the pool (that is, a resource with inconsistent state), the pool calls into RepairStartegy, passing the broken resource as parameter. If the strategy tries to repair the resource and fails, it provides a replacement as Repair's return value, the pool will accept that replacement and forget about the resource that has been replaced. If the strategy successfully repairs the resource, it returns a reference to the same object that was passed into the strategy and the resource is returned to the pool.

Note Throwing from any strategy, or producing nonsense as strategy work result (if any) tears down the process -- PoolKit leaves unhandled exceptions unhandled and the default CLR policy for unhandled exceptions since .Net FW 2 is to terminate the process.

Note Introducing nonsense strategies that hang forever (e.g. by Thread-Sleeping for a day) will most certainly cripple the pool and, with very high probability, -- your application.

ReusePreparationStrategy is something the pool calls every time it gets a resource back, usually to wipe clean the resource' state, so resource can be reused (How is it different from RepairStrategy then? -- you might ask. It can be different and it can be not, depending on the resource type).

Here is an example implementation of a pool strategy (again, you can use functional shortcuts instead of creating classes by hand):

namespace PoolKit.Sample
{
    internal class ArrayZeroingStrategy<T> : ReusePreparationStrategy<T[]>
    {
        public override void PrepareForReuse(T[] resourceToBeReused)
        {
            for (int i = 0; i < resourceToBeReused.Length; i++)
            {
                resourceToBeReused[i] = default(T);
            }
        }

        public override ThreadingContract ThreadingContract
        {
            get
            {
                return ThreadingContract.Multithreaded;
            }
        }
    }
}

ResourceCreationStrategy is pretty straightforward: it is used to create new resources when pool's underlying resource collection is drained.

Note There is no other customization in version 2.1, and for better or worse you cannot influence the threading model used to outsource work from the main thread: reuse preparation and repairs are all happening in CLR's thread-pool threads.

Putting it all together

In summary, to make this work you must create the strategies (use the functional shortcuts if you don't want any new classes), construct the pool you like, then repeat the borrow-return pattern, or you can gain total control over resources by making the pool forget about them. Finally, dispose of the pool when you're done.

Following is a complete sample showing how to construct and use a pool via the GiveUp method:
using (var dynamicPool = Pools.CreateDynamicPool(
    new[] {"a", "b", "c"},
    new ReusePreparationFunction<string>(x => { }, ThreadingContract.Multithreaded),
    new RepairStrategyNotSupportingRepairs<string>(),
    new ResourceCreationFunction<string>(() => Guid.NewGuid().ToString("N"), ThreadingContract.Multithreaded),
    39,
    TimeSpan.FromSeconds(10)))
{
    for (int i = 0; i < 1500; i++)
    {
        // Get the resource and do something useful with it
        Console.WriteLine(dynamicPool.GiveUp());
    }
}

Note It is a good idea to dispose of the pool when you're done with it -- to explicitly release the system resources its implementation might use internally and all the resources pooled.

Last edited Mar 16, 2012 at 9:28 PM by MikhailDutikov, version 14

Comments

No comments yet.