Scalability in .NET

Scalability in .NET

If you have a multiprocessor system, then you'll see threads really show their worth. The Windows OS manages the allocation of threads to processors and, as you have seen throughout this book, firing any process automatically starts a thread. The .NET Framework does not provide fine-grained control of the processor allocation, preferring to allow the operating system to control the scheduling, as it will have more information on the loading of the processors than the CLR would. It does, however, provide some control over which processor an entire process runs on. However, this applies to all of its threads, so its use is not applicable to this book.

If you have only one thread, the main thread, then every task within that thread will operate on the same processor. However, if a new thread is fired, then the operating system schedules which processor it should be executed on. This decision as to which processor will run the thread does itself consume some processor resources and so, for small tasks, it isn't generally worth it as the time to execute may be only as long as the time it takes for the OS to allocate the task to a processor. However, this allocation has been taking less and less time in successive versions of Windows, and for anything other than the most trivial of tasks, when using threads, you should find a performance improvement by creating a new thread to execute your task. It is in symmetric multi-processor (SMP) systems that the benefits of threading are really shown, as the processors can be used to their full effect to distribute the load of the application.

In the next section, we describe how to create a thread pool manager with which you can create and manage threads, and which will ensure that a maximum and minimum number of threads exist in a pool and that idle threads get reused.

A Thread Pool Manager

Throughout this book, you have seen different ways of creating threads and in this chapter we have described the ThreadPool class to make use of the operating system's own thread pool for short-lived threads. We can implement a half-way house between the two, however. We can create a class that will keep a pool of a specified number of threads to be supplied to any requesting application. This will enable the threads to be managed more easily by your code, and also allow for faster thread execution as you may be able to use a previously instantiated thread object. This class will draw together much of the knowledge acquired so far, and you will be able to use it in your own multithreaded applications. We will explain this class as we go along, and at the end provide an application to test that this assembly is working as expected.

So, let's get started and explain the code of our thread pool manager contained in a file named MyThreadPool.cs:

using System;
using System.Text;
using System.Threading

namespace GenThreadPool

This defines two methods for the thread pool, AddJob() and GetStats(), which will be detailed in the following definitions of the GenThreadPoolImpl class, which generates the thread pool that we will be using:

public class GenThreadPoolImpl : IThreadPool

set



public ArrayList PendingJobs

set



public ArrayList AvailableThreads


public int MaxIdleTime

set



public int MaxThreads

set



public int MinThreads

set

This class implements the IThreadPool interface, which we defined earlier, and then goes on to define a few Private fields. The properties are just wrappers around the relevant Private members to prevent users from altering the values directly, in case further rules need to be added later. The fields m_maxThreads, m_minThreads, and m_maxIdleTime specify the maximum and minimum number of threads in the pool, and how long in milliseconds to allow a thread to remain idle before removing it from the pool. There are three constructors for this class:

public GenThreadPoolImpl()

The default constructor only permits one thread to be present in the pool, and will destroy it after only 0.3 seconds. It also performs some lazy initialization, creating an array list to contain the jobs awaiting a thread, and the threads not yet allocated to a method call. The m_debug flag, when set to true, would allow further debugging information while testing:

public GenThreadPoolImpl(int maxThreads, int minThreads,
int maxIdleTime)

When a GenThreadPoolImpl class is instantiated with three integers, we specify how the minimum and maximum number of threads, and the idle time of the threads. It also fires off the InitAvailableThreads() method, detailed below:

private void InitAvailableThreads()


Console.WriteLine('Initialized the ThreadPool. '
+ ' Number of Available threads: '
+ this. availableThreads.Count);

This creates the threads needed for the pool on instantiation. The default constructor only specified one thread, so it wasn't necessary before. This cycles through, creating the maximum number of threads allowed by the pool, specified in m_maxThreads. Below is the constructor for four arguments:

public GenThreadPoolImpl(int maxThreads, int minThreads,
int maxIdleTime, bool debug )

This constructor does the same as the above, only allowing us to set the debugging flag. We now go on to describe the business end of this class, the AddJob() method:

public void AddJob(Thread job)

catch(OutOfMemoryException)


return;


if(_debug)
Console.WriteLine('No Threads Available ..'
+ this.GetStats().ToString());

The ThreadElement class is another helper class that will be defined later. It adds some additional properties to a standard thread so that the pool can manage it effectively. The thread's Start() method is fired before it is added to the m_availableThreads collection.

else

Here we lock the thread so that it cannot be affected by any other process. We then alert all waiting threads that it is now available for use, so we issue a Monitor.Pulse() instruction, and then release the lock:

catch(Exception ex)


}//end of else

}//lock
}//end of method

Finally, we catch any exceptions and output the results to the command line, providing more useful debugging information if the this.Debug flag has been set. That completes the AddJob() method so now let's look at the implementation of the GetStats() method:

public Stats GetStats()

The GetStats() method returns a Stats() structure, which we will define later. As we will see, it contains the minimum and maximum number of threads, as well as other values set in the constructor. Now let's look at the FindIdleThreadCount() method:

public int FindIdleThreadCount()


return idleThreads;

This method is one called earlier in the class and it simply goes through the array list of threads and returns the how many of them are idle. We also used the FindFirstIdleThread() method so let's see it:

public int FindFirstIdleThread()


return -1;

As we can see, the method returns the index of the first idle thread in the array list. We will also need the following method:

public int FindThread()


return -1;

This method is used to determine in which index position in the array list the current thread is located. We'll also need the following method:

public void RemoveThread()

This removes the current thread from the array list of threads. This is, of course, used to remove a thread from the pool when it is finished with and has been idle for longer than the time specified in this.MaxIdleTime. Now we start to define the rest of the classes for this assembly:

public class GenPool

The GenPool class executes all of the pending threads, and once complete, after the period specified in MaxIdleTime, will remove them from the pool. It checks to see if there are any threads available on the GenThreadPoolImpl passed as a reference to the constructor, and it locks the values of the object passed as the first parameter. In general, this will be the same GenThreadPoolImpl object passed as the second argument:

public void Run()


job = (Thread) gn.PendingJobs[0];
gn.PendingJobs.RemoveAt (0);
}//end of lock

This Run() method starts a loop to attempt to find a thread in the pool that matches the current thread, and begin its execution. You can see above that it locks the object passed in as a parameter to the constructor, and if there are no pending jobs, then it just finds the thread in the pool that matches the current one, returning if there isn't one. If there is a pending job, then it retrieves the first one, and then removes it from the queue:

//run the job
job.Start ();

It then begins execution of the method on the pending thread, and returns to the start of the loop:

try

In the next part of the loop (once it has no more pending jobs), it locks the current object and waits for the thread to be free for execution for the period specified in MaxIdleTime.

lock(_lock)

Finally, it locks the object again, and if there are no pending jobs and there are more than the minimum required number of threads, then it removes the thread from the pool. We now move on to the ThreadElement class:

public class ThreadElement

A ThreadElement is what is stored in the thread pool, and takes a thread as the parameter for its constructor. It sets the thread as idle on construction of this object:

public bool Idle

set


public Thread GetMyThread()

The above code is straightforward. The Idle property essentially defines when the thread's execution is complete, and the GetMyThread() method just returns the Thread object. Now look at the following structure:

public struct Stats

This ToString() method returns the structure in a string format, using StringBuilder to build up the string. The argument initializes the StringBuilder's size to 107 characters, as it is fair to assume that there are not likely to be more than 99,999 threads. If so, then StringBuilder will resize itself anyway. This capacity specification allows a small performance boost.

If you have an application that is firing methods repeatedly on different threads, this class can manage the process and help ensure that too many threads aren't spawned. Apart from containing a maximum and minimum number of threads, it will reuse an existing thread if possible. You can now compile this project into a DLL, and use this class from within other projects. Below is code that will allow you to test this thread pool class, TestGenThreadPool.cs:

using System;
using System.Threading;
using GenThreadPool;

namespace TestGenThreadPool


Console.WriteLine('End Time for Job is ' +
System.DateTime .Now);

Console.WriteLine('Performance using Pool[in ms]: ');
Console.WriteLine(''
+ (System.DateTime.Now - start).ToString());

count = 0;
start = System.DateTime.Now;

Console.WriteLine('Start Time for JobThread is ' +
System.DateTime.Now.ToString());

for (int i = 0; i < times; i++)


while (true)

try

catch



Console.WriteLine('End Time for JobThread is '
+ System.DateTime.Now.ToString());
Console.WriteLine('Performance using no Pool[in ms]: ');
Console.WriteLine(''
+ (System.DateTime.Now - start).ToString());

sealed class JobThread


public void Run()




sealed class Job


public void Run()





class TestPool

The above application just mechanically attempts to add new threads to an instance of the thread pool, with the debug flag set to true. It is quite straightforward, but the best way to see this thread pool in action is to try it out in your own applications. You can use this class, once it is compiled.