dqd_threadpool

One of Rob Mayoff's AOLserver modules.

This is dqd_threadpool release 1.1.

The dqd_threadpool module manages a pool of worker threads and a job queue. Client threads submit Tcl scripts to the job queue and wait for the results. Worker threads pull jobs off the queue, execute the scripts, and post the results for the clients to retrieve.

Compiling and Installating

This module should work with AOLserver 3.0 or later, but I have only tested it with AOLserver 3.2.

Set environment variable INST to the location where you installed AOLserver. Your AOLserver binary should be in $INST/bin.

In the dqd_threadpool directory, run make install. This will compile the module and copy it to the AOLserver installation. You must have write permission on $INST/bin to install the module.

Basic Configuration

To configure a thread pool, you must load the dqd_threadpool module: 

ns_section ns/server/server1/modules
ns_param mypool dqd_threadpool.so

This creates a thread pool named mypool. You may create several thread pools by loading the module repeatedly: 

ns_section ns/server/server1/modules
ns_param bookpool dqd_threadpool.so
ns_param moviepool dqd_threadpool.so

This creates two thread pools, named bookpool and moviepool.

Using Thread Pools

Let's say you want to write a simple book price comparison site. When a user types in an ISBN, you will use ns_httpget to look up the price of the book at several online bookstores.

The simplest approach is to call ns_httpget for each bookstore one at a time However, it takes time for the HTTP requests and responses to travel over the network, and each bookstore will take some time to look up the book and create the page to send back. It would be better to query all the bookstores simultaneously. A thread pool makes that easy.

Suppose you have configured bookpool as a thread pool. Then you can submit one job to the pool for each bookstore: 

set jobs {}
lappend jobs [dqd_threadpool create bookpool [list amazon_lookup $isbn]]
lappend jobs [dqd_threadpool create bookpool [list bn_lookup $isbn]]
lappend jobs [dqd_threadpool create bookpool [list 1bookstreet_lookup $isbn]]
lappend jobs [dqd_threadpool create bookpool [list wordsworth_lookup $isbn]]
Now $jobs is the list of job ids. Next, you need to wait for each job to complete. Each time a job completes, you need to get its results and process them. 
while {[llength $jobs] > 0} {
    # The wait command will return a list of two sublists.
    set pair [dqd_threadpool wait bookpool $jobs]

    # The first sublist is the list of job ids that have completed.
    set finished [lindex $pair 0]

    # The second sublist is the list of job ids that have not completed.
    set remaining [lindex $pair 1]

    foreach job $finished {
	set pair [dqd_threadpool get bookpool $job]
	set code [lindex $pair 0]
	set result [lindex $pair 1]

	if {$code == 1} {
	    # Error.
	    # Process error message here....
	} else {
	    # Success.
	    # Process result here...
	}
    }

    # Continue looping until there are no remaining jobs.
    set jobs $remaining
}

The complete source for this example is in the examples/book subdirectory. After installing the dqd_threadpool module, you can run the example using this command: 

/usr/local/aolserver/bin/nsd -ft examples/book/nsd.tcl

Then connect to http://localhost:10080/.

Using nscache With dqd_threadpool

You can use the nscache module to improve performance further.

In the book price comparison example, you would keep a cache of the responses from each bookstore for each ISBN. For each bookstore, you see if the cache already has the price for the book at that store. If not, then you queue a job to get and cache that store's price. 

set jobs {}

foreach bookstore {amazon bn 1bookstreet wordsworth} {
    set key "$bookstore.$isbn"

    if {[ns_cache get bookprices $key result]} {
	# Process result here...
    } else {
	lappend jobs [dqd_threadpool create bookpool \
	    [list ns_cache eval bookprices $key \
		[list ${bookstore}_lookup $isbn]]]
    }
}

Then you wait for any queued jobs to complete and process the results, as in the non-caching case.

The complete source for this example is in the examples/book-cache subdirectory. After installing the dqd_threadpool and nscache modules, you can run the example using this command: 

/usr/local/aolserver/bin/nsd -ft examples/book-cache/nsd.tcl

Then connect to http://localhost:10081/.

Reference

Configuration

To create a threadpool named X, load the module by putting these lines in your configuration file: 

ns_section ns/server/yourserver/modules
ns_param X dqd_threadpool.so

If the configuration contains a section named ns/server/yourserver/module/X, then the module will read parameters from that section.

The module understands these parameters:

MaxThreads

The maximum number of threads that can exist simultaneously in this pool.

Default: 4

MinThreads

The minimum number of threads that can exist simultaneously in this pool. These threads will be created when the server starts.

Default: 0

MaxIdle

The maximum number of seconds a thread in the pool can be idle before exiting. If a thread goes MaxIdle seconds without running a job, and there are more than MinThreads threads in the pool, then the thread will exit.

Setting MaxIdle to zero disables the timer, so threads in the pool never exit.

Default: 30 (seconds)

MaxSize

The maximum total size, in bytes, of all queued and uncompleted jobs. The size of a job is the length of the queued script in bytes. When a program calls create or detach, the command checks the total size of the jobs. If the new job would make the total exceed MaxSize, then the command blocks until enough jobs complete to make room for the new job.

If a single job's size exceeds MaxSize, then create or detach will block until the queue is completely empty.

Setting MaxSize to zero disables the limit.

Default: 0 (disabled)

Trace

Enables debug messages in the server log. Setting this to on will make the module emit various messages about thread lifecycles and queue operations.

Default: off

dqd_threadpool Command

dqd_threadpool create pool script

The create subcommand creates a new job. Script is added to the job queue for pool. The command returns a numeric job id which can later be used to retrieve the result of the script's execution.

You must eventually call get on the returned job id. The return value of the script will be stored in memory until you do.

dqd_threadpool wait pool jobid-list

The wait subcommand waits for zero or more jobs on pool's job queue to complete. Jobid-list is a list of job ids previously returned by the create subcommand for the same pool. As soon as any of the jobs in the list completes, wait returns a list with two sublists. The first sublist contains the ids of any completed jobs in the original list. The second sublist contains the ids of the remaining incomplete jobs.

Wait silently removes invalid job ids. The invalid ids will not be included in either of the returned sublists.

dqd_threadpool get pool jobid

The get subcommand retrieves the result of a completed job. If job jobid is invalid or has not completed, get raises an error. Otherwise, get returns a list of two elements. The first element is the return code from the job. The second element is the return value from the job. Typically a job script ends with a return statement like this: 
return some-value
In this case, the return code is 2 and the return value is some-value. If the script raised an error, the return code will be 1 and the return value will be the error message.

Get can only be called once (successfully) for each job id. Once get has returned the result of a job, the job id becomes invalid.

dqd_threadpool detach pool script

The detach subcommand creates a new job, like the create subcommand. However, detach does not return a job id. It is not possible to retrieve the result of script using the normal wait/get mechanism. Use detach if you wish to run a job but you do not need the result and do not wish to leak memory.

Version History

mayoff@dqd.com
 
$Header: /u/cvsroot/nsd-modules/dqd_threadpool/index.html,v 1.6 2004/10/16 02:51:54 mayoff Exp $