ALib Module ThreadModel - Programmer's Manual

Table of Contents

• 1. Introduction
  • 1.1 Thread Types
    • 1.1.1 Single Threads Dedicated to a Group of Jobs
    • 1.1.2 Thread Pools with Random Assignment
  • 1.2 Pros and Cons
• 2. Class ThreadPool
• 3. Class DedicatedWorker
• 4. Class Trigger and Interface class Triggered

Attention

This ALib Module was introduced only with the latest release version 24TODO and is considered unfinished and highly experimental.
Both attributes likewise apply to this "manual"!
Please do not read it and please do not use this module, yet - unless you are like experiments! :-)

1. Introduction

This is a rather high-level ALib Module. It is by design separated from the low-level module ALib Threads, which provides just fundamental functionality like a simple thread class, thread registration and identification and several locking and asynchronous notification mechanics.

In contrast to that, this module introduces a thread model. So what is that? The answer is that programmers have various different options to implement and manage parallelism. The choice is dependent from the type of application and quite a lot also a matter of "taste". Once more, ALib here chooses a rather traditional design.

This is the main reason why this module is separated from the low-level functionality: It is purely a matter of the library's user to select this module into an ALib Distribution or not. The second important reason for separating this module from module ALib Threads is simply to avoid massive cyclic dependencies with other modules.

1.1 Thread Types

Massive multi-threading applications like server-software do not use simple thread pools, but rather more sophisticated concepts like partitioned thread pools or task-affinity scheduling. It is not the ambition and scope of ALib to support such software.

Instead, ALib aims to provide reasonable efficient concepts, which are easily manageable and understandable.

This module differentiates between two sorts of worker threads, which are here discussed in theory.

1.1.1 Single Threads Dedicated to a Group of Jobs

Dedicated threads can be established to process tasks that are related by data, context, or type. Each thread works with a specific dataset or set of tasks, which minimizes the need for locking and helps avoid conflicts.

The advantages are:

• Reduced Synchronization Overhead:
  Because each thread deals with a specific set of data, there’s less contention for shared resources, which can reduce the need for locks.
• Cache Efficiency:
  Keeping similar tasks or data with the same thread can help improve cache locality, as data accessed by one thread is likely to remain in its cache.
• Predictable Task Execution:
  A programmer gains more control over the execution flow, as tasks of a particular type or priority are handled by dedicated threads.

When It’s Ideal: This approach is especially effective when you have distinct task categories with minimal cross-dependency. It’s common in real-time systems, certain types of game loops, or applications where tasks have strict priorities or dependencies on specific resources.

1.1.2 Thread Pools with Random Assignment

Very common in modern libraries are thread pools. Those are collections of threads that are assigned tasks as they become available. Each task is handled by the next available thread, regardless of its type or data.

Advantages:

• High Throughput and Scalability:
  Pools can dynamically balance load by assigning tasks to any available thread, which makes them very flexible for handling a large number of small, independent tasks.
• Optimal for Short-Lived, Independent Tasks:
  This approach works well when tasks are quick and mostly independent, as it maximizes CPU utilization with minimal idle time.
• Reduced Overhead in Thread Management:
  Rather than constantly creating and destroying threads, pools reuse threads, which reduces overhead.

When It’s Ideal: Thread pools are ideal for workloads where tasks are mostly independent, like web servers handling requests, background processing jobs, or any system where tasks are numerous and lightweight.

1.2 Pros and Cons

The following table recaps the pros and cons associated with each type:

Aspect	Dedicated Threads	Thread Pools (Library Approach)
Task Locality	High, due to grouped tasks by type/data	Low to moderate, as tasks go to any thread
Synchronization Need	Lower, fewer locks if data sets are disjoint	Higher, requires locks or atomic operations for shared data
CPU Utilization	May have idle threads if tasks are unbalanced	High, as all threads are actively used when needed
Cache Efficiency	Good, better locality due to fixed data per thread	Variable, depends on task scheduling
Adaptability to Load	Less adaptable, might require load-balancing strategies	Very adaptable, dynamic balancing by pool
Implementation Complexity	Higher, requires more explicit management	Lower, handled by the library
Application Control	Higher, and probably better structured code entities	Lower, handled by the library

2. Class ThreadPool

The class ThreadPool implements the concept of pooled threads, as discussed in the introductory sections above.

Let us start with a simple sample. The first thing to do is defining a job-type, derived from class Job:

// Define a custom job type
struct MyJob : alib::JPromise  {
 
    int input = 0; // the input given with construction
    int result= 0; // the result calculated in Do()
 
    // Constructor.
    // Passes the typeid of this class to parent constructor, which in turn
    // passes it to base type Job.
    MyJob(int i) : JPromise(typeid(MyJob))
                 , input{i}
    {}
 
    // Mandatory to overwrite. Has to return this type's size.
    virtual size_t  SizeOf()  override { return sizeof(MyJob); }
 
    // Job logic goes here:
    bool Do() override  {
 
        // The work
        result= 2 * input;
 
        // Notify the sender
        Fulfill(ALIB_CALLER_PRUNED);
 
        // Pool jobs always have to return true
        return true;
    }
};

With this in place, we can pass jobs of this type to a thread pool:

// Create a thread pool
alib::ThreadPool pool;
 
// Schedule a job with input '21'
auto& myJob= pool.Schedule<MyJob>(21);
 
// wait for execution and print result
myJob.Wait(ALIB_CALLER_PRUNED);
Log_Info( "MyJob(21) result: {}", myJob.result )
 
// delete the job instance with the pool (otherwise, its a memory leak)
pool.DeleteJob(myJob);
 
// Schedule a job without caring for the job execution and its result.
// With this version of scheduling a job we can't see the result and don't know when it
// is executed.
// The benefit is that job-deletion is performed automatically and thus we do not need to wait
// for execution to do it ourselves.
pool.ScheduleVoid<MyJob>(123);
 
// Wait a max of 1h for all threads to finish (with dbg, warn after 1ms)
pool.WaitForAllIdle( 1h  ALIB_DBG(, 1ms) );
 
// Shutdown the pool. Because this also waits until all jobs are processed and workers are
// idle, whe wouldn't have needed the line above.
pool.Shutdown();

This already showed the basic idea how the type can be used.

Please Read Now:

To avoid redundancy, we would ask you to read the following pieces of the reference documentation:

• The reference documentation of class ThreadPool, and
• the reference documentation of class Job.

This might be all that is needed to explain in this chapter.

3. Class DedicatedWorker

As discussed in the introductory sections above, one principle type of threads are ones that are "dedicated to a group of jobs".
While the foundational module ALib Threads already provides the simple type Thread, which implements this concept along the design provided with class Thread of the Java programming language, this module introduces a more sophisticated implementation with the class DedicatedWorker.

Here is a quick sample code that demonstrates the use of this class. As a prerequisite we rely on the same class MyJob that had been introduced in the previous section:

// Define a custom job type
struct MyJob : alib::JPromise  {
 
    int input = 0; // the input given with construction
    int result= 0; // the result calculated in Do()
 
    // Constructor.
    // Passes the typeid of this class to parent constructor, which in turn
    // passes it to base type Job.
    MyJob(int i) : JPromise(typeid(MyJob))
                 , input{i}
    {}
 
    // Mandatory to overwrite. Has to return this type's size.
    virtual size_t  SizeOf()  override { return sizeof(MyJob); }
 
    // Job logic goes here:
    bool Do() override  {
 
        // The work
        result= 2 * input;
 
        // Notify the sender
        Fulfill(ALIB_CALLER_PRUNED);
 
        // Pool jobs always have to return true
        return true;
    }
};

Having this in place, a simple dedicated worker is quickly created and fed with such a job:

// Derive my own dedicated worker type
struct MyDedicatedWorker : threadmodel::DedicatedWorker {
 
    // Constructor. Passes a name to the parent type. The name is passed to grand-parent
    // class "alib::threads::Thread".
    MyDedicatedWorker()  : DedicatedWorker(A_CHAR("My-DW"))  {}
 
    // Dedicated interface exposed to users of this type.
    MyJob& DoMySuperJob( int input ) {
 
        return Schedule<MyJob>( threadmodel::Priority::Standard,
                                input ); // <- constructor parameters of JobWait
    }
 
    // Same as previous method but does not provide sender with result value
    // (Caring for the result value might be a "burdon" in some cases)
    void  DoMySuperJobVoid( int input ) {
 
        ScheduleVoid<MyJob>( threadmodel::Priority::Standard,
                               input ); // <- constructor parameters of JobWait
    }
 
};
 
// Create the worker and start it by adding it to the manager singleton
MyDedicatedWorker dw;
DWManager::GetSingleton().Add(dw);
 
// Push a job by using that interface method that returns the job object
// on which we can wait.
Log_Info( "Pushing a job" )
MyJob&    req= dw.DoMySuperJob( 21 );
 
Log_Info( "Waiting for job execution" )
req.Wait(ALIB_CALLER_PRUNED);
 
// When wait returned, we can access the resul
Log_Info( "Job executed. Calculated result is: ", req.result )
 
// Now we have to dispose the job object. If not done, this is a memory leak.
dw.DeleteJob( req );
 
// Now we use the second interface method that does not return the job.
// Hence, we can't wait but we also are not burdend with deletion of the object
dw.DoMySuperJobVoid(123);
 
// Remove our dedicated worker from the manager. This waits for execution of all open job and
// terminates (joins) the thread.
DWManager::GetSingleton().Remove( dw );
Log_Info( "Max queue length (gives 1): ",  dw.DbgMaxQueuelength  )
Log_Info( "Jobs open (gives 0):        ",  dw.Load()  )

As the using sample shows, two interface versions are offered in this sample: One that returns the Job instance, a second that does not return anything. Both versions have advantages and disadvantages (explained in the comments above and in the reference documentation).

In more complicated cases it may be necessary to receive the job to be able to periodically check for processing, but then the sender may "lose interest" in it. To enable a due deletion of an unprocessed job, method DeleteJobDeferred is offered. This pushes a new job (of a special internal type) into the execution queue of the worker, which cares for deletion.

The sample furthermore showed that the very same job-type which had been used in the previous section with class ThreadPool, can be used with the dedicated worker. If done so, the advantage lies exactly here, namely that a job can be used with both concepts.
However, this usually is neither needed nor wanted, just because the decision which thread-concept to use, is dependent from exactly the nature of the job-types!

Therefore, the more common option of processing jobs with class DedicatedWorker is to override its virtual method process and perform execution there.

Here is a sample code:

// Derive my own dedicated worker type
struct MyDedicatedWorkerV2 : threadmodel::DedicatedWorker {
 
    // Constructor. Passes a name to the parent type. The name is passed to grand-parent
    // class "alib::threads::Thread".
    MyDedicatedWorkerV2()  : DedicatedWorker(A_CHAR("My-DW-V2"))  {}
 
    // Dedicated interface exposed to users of this type.
    MyJob& DoMySuperJob( int input ) {
 
        return Schedule<MyJob>( threadmodel::Priority::Standard,
                                input ); // <- constructor parameters of JobWait
    }
 
 
    // Override the process-method. If this returns true, then the method Do() of the job
    // is not executed.
    bool            process(Job& job) override {
 
        // check job type
        if ( job.Is<MyJob>() ) {
 
            // cast job using special templated method Cast()
            MyJob& myJob= job.Cast<MyJob>();
 
            // calculate the result (we tripple instead of double to be able to check which
            // method is in fact called)
            myJob.result= 3 * myJob.input;
 
            // set job processed
            myJob.Fulfill(ALIB_CALLER_PRUNED);
            return true;
        }
 
        // job not processed
        return false;
    }
};

If the overridden method returns true for a job passed, then the virtual method Job::Do is not even called. In the sample above, both implementation even do different things. The first doubles the input value, the second triples it.

Let us summarize this:

• When used exclusively with class DedicatedWorker, class Job does not need to override method Job::Dos().
  
• Instead, method DedicatedWorker::process is implemented which checks for the different types of jobs, which are scheduled by its dedicated public interface methods.
• The advantage here is that the derived worker type may hold data which is needed to execute the jobs.
• From a programmer's perspective, an additional advantage might be that all execution code is aggregated in one place, namely method process().
  

Please Read Now:

To avoid redundancy, for further information, we would ask you to read the following pieces of the reference documentation:

• The reference documentation of class DedicatedWorker,
• the reference documentation of class Job, and
• the reference documentation of class DWManager.

4. Class Trigger and Interface class Triggered

This pair of classes offers a next method to execute tasks asynchronously.
Here is a quick sample:

// My triggered type
struct MyTriggered : threadmodel::Triggered
{
    // Uses a fixed sleep time in this sample.
    // Note that in general, the method triggerPeriod() may return a different value
    // with every invocation.
    Ticks::Duration     sleepTime;
 
    // We count the calls, that' it
    int                 ctdTriggerCalls = 0;
 
    // Constructor. Parameter name needed for parent interface class Triggered
    MyTriggered(const String& name, Ticks::Duration::TDuration pSleepTime)
    : Triggered(name)
    , sleepTime(pSleepTime)   {}
 
    // Mandatory to overwrite. Has to return the next sleep duration.
    virtual Ticks::Duration triggerPeriod()   override { return sleepTime; }
 
    // The virtual method called to trigger.
    virtual void            trigger()         override {
        Log_Verbose( "I got triggered. I am: {} Sleep-period: ", this->Name, this->triggerPeriod() )
        ctdTriggerCalls++;
    }
};
 
// Create a trigger instance and attach two triggered 'clients'
// Note that it is allowed to attach triggered objects also while the trigger-thread is
// already running.
Trigger trigger;
 
MyTriggered t1( A_CHAR("MyTriggered 1"),  10us );
MyTriggered t2( A_CHAR("MyTriggered 2"),  30us );
trigger.Add( t1 );
trigger.Add( t2 );
 
// First, we start the trigger as an own thread. We wait a 10ms and then stop the thread
Log_Info( "Starting Trigger" )
trigger.Start();
Thread::Sleep(10ms);
trigger.Stop();
 
// We will see that t1 was roughly three times more often called than t2:
Log_Info( "Trigger calls t1: ", t1.ctdTriggerCalls )
Log_Info( "Trigger calls t2: ", t2.ctdTriggerCalls )
 
// Second, we run the trigger manually for 10 ms
Log_Info( "Running trigger 'manually'" )
trigger.Do(10ms);
 
// We will see that both triggered objects where called roughly double as often then before
Log_Info( "Trigger calls t1: ", t1.ctdTriggerCalls )
Log_Info( "Trigger calls t2: ", t2.ctdTriggerCalls )

Please Read Now:

To avoid redundancy, for further information, we would ask you to read the following pieces of the reference documentation:

• The reference documentation of class Trigger, and
• the reference documentation of class Triggered.

As a final not, class DedicatedWorker implements the interface Triggered in order that it can be attached to a trigger. If done, a trigger-job will be pushed in its command queue, and with that, the execution of interface method Triggered::Trigger is performed asynchronously.