HP recommends that you inspect your multithreaded application's code to determine whether a word-tearing race condition is possible for any two or more of the application's threads. That is, determine whether any two or more threads can concurrently write contiguously defined members of the same composite data object where those members occupy the same memory granule whose size is greater than or equal to the application's actual granularity.

If you find that you must change your application to avoid a word-tearing scenario, there are several approaches available. The simplest techniques require only that you change the definition of the target composite data object before recompiling the application. The following sections offers some suggestions.

3.7.5.1 Changing the Composite Data Object's Layout

If you can change the organization or layout of the composite data object's definition, you should do both of the following:

• Widen the structures or union members to the granule. If that is unacceptable, define padding storage after each structure or union member (except the last) or add padding storage to the array's element definition. This forces all members/elements to be placed in separate granules by the compiler.
• If your system's compiler offers a choice, compile the application's modules to produce the preferred actual granularity for the application's target system.

If you cannot change the organization or layout of the composite data object's definition, you should do one of the following:

• (On OpenVMS Alpha or OpenVMS VAX) Compile all application modules for byte actual granularity. Doing so automatically prevents word-tearing race conditions for structure or union members and array elements of size byte or larger that are accessed concurrently by different threads. No other program modification is required. This may have a performance penalty on Alpha EV4 and EV5 processors.
• (On Tru64 UNIX systems) For arrays, add the C language volatile storage qualifier to the definition of the entire array; for structures, add volatile to the declaration of only those members that share the pertinent memory granule. You must also compile the application's modules using the Compaq C or Compaq C++ compiler's -strong-volatile switch. Doing so causes the compiler to produce code that forces all accesses to those members to occur as atomic operations. See the description of the -strong-volatile switch in the Compaq C or Compaq C++ documentation and on the cc reference page. This may also have a severe performance penalty.

If you must maintain the composite data object's layout and cannot change the storage qualifiers for the application's composite objects, you can instead use the technique described in the next section.

3.7.5.3 Using One Mutex Per Composite Data Object

Your source code inspection may identify an array or a set of contiguously defined structure or union members that is subject to a word-tearing race condition. In this case, your program can use a mutex that is dedicated to protect all write accesses by all threads to those data objects, rather than change the definition of the composite data objects.

To use this technique, create a separate mutex for each composite data object where any members share a memory granule that is greater than or equal to the program's actual granularity. For example, given an application with quadword actual granularity, if structure members M1 and M2 occupy the same longword in structure S and those members can be written concurrently by more than one thread, then the application must create and reserve a mutex for use only to protect all write accesses by all threads to those two members.

In general, this is a less desirable technique due to performance considerations. However, if the absolute number of thread accesses to the target data objects over the application's run-time will be small, this technique provides explicit, portable correctness for all thread accesses to the target members.

3.7.6 Identifying Possible Word-Tearing Situations Using Visual Threads

For Tru64 UNIX systems, the Visual Threads tool can warn the developer at application run-time that a possible word-tearing situation has been detected. Enable the UnguardedData rule before running the application. This rule causes Visual Threads to track whether any memory location (that is, granule) in the application has been accessed from two threads without proper synchronization. This includes detection of word tearing as well as more straightforward synchronization errors. See the Visual Threads product's online help for more information.

Visual Threads is available as part of the Developer's Toolkit for Tru64 UNIX.

3.8 One-Time Initialization

Your program might have one or more routines that must be executed before any thread executes code in your facility, but that must be executed only once, regardless of the sequence in which threads start executing. For example, your program can initialize mutexes, condition variables, or thread-specific data keys---each of which must be created only once---in a one-time initialization routine.

You can use the pthread_once() routine to ensure that your program's initialization routine executes only once---that is, by the first thread that attempts to initialize your program's resources. Multiple threads can attempt to call the program initialization routine via the pthread_once() routine, and the Threads Library ensures that the specified initialization routine is called only once.

On the other hand, rather than use the pthread_once() routine, your program could statically initialize a mutex and a flag, then simply lock the mutex and test the flag. In many cases, this technique might be more straightforward to implement.

Finally, you can use implicit (and nonportable) initialization mechanisms, such as OpenVMS LIB$INITIALIZE, Tru64 UNIX dynamic loader __init_ code.

3.9 Managing Dependencies Upon Other Libraries

Because multithreaded programming has become common only recently, many existing code libraries are incompatible with multithreaded uses. For example, many traditional run-time library routines maintain state across multiple calls using static storage. This storage can become corrupted if routines are called from multiple threads at the same time. Even if the calls from multiple threads are serialized, code that depends upon a sequence of return values might not work.

For example, the UNIX getpwent(2) routine returns the entries in the password file in sequence. If multiple threads call getpwent(2) repeatedly, even if the calls are serialized, no thread will obtain all entries in the password file. (This is not a problem on Tru64 UNIX, because the state is maintained using thread-specific data.)

Different library routines are compatible with multithreaded programming to different extents. The important distinctions are thread reentrancy and thread safety.

3.9.1 Thread Reentrancy

A routine is reentrant if it can be used simultaneously when called by different threads. For example, the standard C run-time library routine strtok() can be made reentrant most efficiently by adding an argument that specifies a context for the sequence of tokens. Thus, multiple threads can simultaneously parse different strings without interfering with each other.

A reentrant routine should have no dependency on static data. Because access to static data must be synchronized, there is always a performance penalty due to the cost of synchronizing. There is also a loss of potential parallelism throughout the program. A routine that does not use any data that is shared between threads can proceed without locking.

If you are developing new interfaces, make sure that any persistent context information (like the last-token-returned pointer in strtok() ) is passed explicitly so that multiple threads can process independent streams of information independently. Return information to the caller through routine values or output parameters (where the caller passes the address and length of a buffer). You could also return information to the caller by allocating dynamic memory and requiring the caller to free that memory when finished. Avoid using errno or other global variables for returning error or diagnostic information; use routine return values instead.

3.9.2 Thread Safety

A routine is thread-safe if it can be called simultaneously from multiple threads without risk of corruption. If the routine is not actually reentrant, generally this means that it does some level of locking to prevent simultaneously active calls in different threads.

Thread-safe routines tend to be less efficient than reentrant routines. For example, a package that is thread-safe might still block all threads in the process while one thread executes the code.

Routines such as localtime() or strtok() , which traditionally rely on static storage, can be made thread-safe by using thread-specific data instead of static variables as is done on Tru64 UNIX. This prevents corruption and avoids the overhead of synchronization. However, using thread-specific data is not without its own cost, and it is not always the best solution. Using an alternate, reentrant version of the routine, such as the POSIX strtok_r() interface, is often preferable.

3.9.3 Lacking Thread Safety

When your program must call a routine that is not thread-safe, your program must ensure serialization and exclusivity of the unsafe routine across all threads in the program.

If a routine is not specifically documented as reentrant or thread-safe, you can assume that it is not safe to use. Never assume that a routine is fully thread-safe unless it is expressly documented as such; a routine can use static data in ways that are not obvious from its interface. A routine carefully written to be thread-safe but that calls some other routine that is not thread-safe without proper protection, is itself not thread safe.

3.9.3.1 Using Mutex Around Call to Unsafe Code

Holding a mutex while calling any unsafe code accomplishes this. All threads and libraries using the routine should use the same mutex. Note that even if two libraries carefully lock a mutex around every call to a given routine, if each library uses a different mutex, the routine is not protected against multiple simultaneous calls from different libraries.

Note that your program might be required to protect a series of calls, rather than a single call, to routines that are not thread safe.

3.9.3.2 Using the Global Lock

To ensure serialization and exclusivity of the unsafe code, the Threads Library provides one global lock that can be used by all threads in a program when calling either routines or code that are not thread-safe while already holding the lock. Because there is only one global lock, you do not need to fully analyze all of the dependencies in unsafe code that your program calls.

Acquire the global lock by calling pthread_lock_global_np() ; release the global lock by calling pthread_unlock_global_np() .

The global lock allows a thread to acquire the lock recursively, so you do not need to be concerned if you call a routine that also may acquire the global lock.

Use the global lock whenever calling unsafe routines. All Threads Library routines are thread-safe.

3.9.3.3 Using or Copying Static Data Before Releasing the Mutex

In many cases your program must protect more than just the call itself to a routine that is not thread-safe. Your program must either use or copy any static return values before releasing the mutex that is being held.

3.9.4 Use of Multiple Threads Libraries Not Supported

The Threads Library performs user-mode execution context-switching within a process by exchanging register sets, including the program counter and stack pointer. If any other code within the process also performs this sort of context switch, neither the Threads Library nor that other code can ever know the proper identity of the context which is active at any time. This can result in, at best, unpredictable behavior---and, at worst, severe errors.

For example, under OpenVMS VAX, the VAX Ada run-time library provides its own tasking package that does not use Threads Library scheduling. Therefore, VAX Ada tasking cannot be used within a process that also uses the Threads Library. (This restriction does not exist for Compaq Ada for Tru64 UNIX, or Compaq Ada for OpenVMS Alpha, because they use the Threads Library.)

3.10 Detecting Error Conditions

The Threads Library can detect some of the following types of errors:

• Application programming interface (API) errors can occur when the program either specifies an invalid parameter or attempts an inappropriate operation on some Threads Library object.
• Internal errors can occur when the Threads Library determines that internal information has become corrupted to the point where it cannot continue operation.

The pthread interface reports API errors by returning an integer value indicating the type of error.

The Threads Library internal errors result in a bugcheck. The Threads Library writes a message that summarizes the problem to the process' current error device, and (on OpenVMS) writes a file that contains more detailed information. (On Tru64 UNIX systems, the core file is sufficient for analysis of the process using the Ladebug debugger.)

By default, the file is named pthread_dump.log and is created in the process' current (or default) directory. To cause the Threads Library to write the bugcheck information into a different file, define PTHREAD_CONFIG and set its dump= major keyword. (See Section C.1 for more information about using PTHREAD_CONFIG .)

If the Threads Library cannot create the specified file when it performs the bugcheck, it will try to create the default file. If it cannot create the default file, it will write the detailed information to the error device.

3.10.1 Bugcheck Information

The header message written to the error device starts with a line reporting that the Threads Library has detected an internal problem and that it is terminating execution. It also includes the version of the Threads Library. The message resembles this:

% Threads Library bugcheck (version V3.13-180), terminating execution.

The next line states the reason for the failure. On Tru64 UNIX, this is followed by process termination with SIGABRT (SIGIOT), which causes writing of a core dump file. On other platforms, a final line on the error device specifies the location of the file that contains detailed state information produced by the Threads Library, as in the following example:

% Dumping to pthread_dump.log

The detailed information file contains information that is usually necessary to track down the problem.

3.10.2 Interpreting a Bugcheck

The fact that the Threads Library terminated the process with a bugcheck can mean that some subtle problem in the Threads Library has been uncovered. However, the Threads Library does not report all possible API errors, and there are a number of ways in which incorrect code in your program can lead to a bugcheck.

A common example is the use of any mutex operation or of certain condition variable operations from within an interrupt routine (that is, a Tru64 UNIX signal handler or OpenVMS AST routine). This type of programming error most commonly results in a bugcheck that reports a "krnSpinLockPrm: deadlock detected" message or a "Can't find null thread" message. To prevent this type of error, do not use Threads Library routines other than those with the _int suffix in their names, such as pthread_cond_signal_int_np() from an interrupt routine.

In addition, the Threads Library maintains a variety of state information in memory which can be overwritten by your own code. Therefore, it is possible for an application to accidentally modify the Threads Library state by writing through invalid pointers, which can result in a bugcheck or other undesirable behavior.

If you encounter a bugcheck, first check your application for memory corruptions, calls from AST routines, and so on, and then contact your HP support representative and include this information file (or the Tru64 UNIX core file) along with sample code and output. Always include the full name and version of the operating system, and any patches that have been installed. If complete version information is lacking, useful core file analysis might not be possible.