Addison wesley UNIX network programming volume 1 the sockets networking API 3rd edition nov 2003 ISBN 0131411551

Forexample, if pthread_create cannot create a new thread because of exceeding some system limit on the number of threads, the function return value is EAGAIN.. Since this function must b

Section 26.8 Condition Variables

Section 26.9 Web Client and Simultaneous Connections(Continued)

Section 26.10 Summary

Exercises

In the traditional Unix model, when a process needs somethingperformed by another entity, it forks a child process and letsthe child perform the processing Most network servers underUnix are written this way, as we have seen in our concurrentserver examples: The parent accepts the connection, forks achild, and the child handles the client

While this paradigm has served well for many years, there areproblems with fork:

fork is expensive Memory is copied from the parent to thechild, all descriptors are duplicated in the child, and so on

Current implementations use a technique called copy-on-write, which avoids a copy of the parent's data space to the

child until the child needs its own copy But, regardless ofthis optimization, fork is expensive

IPC is required to pass information between the parent and

child after the fork Passing information from the parent to

the child before the fork is easy, since the child starts with

a copy of the parent's data space and with a copy of all theparent's descriptors But, returning information from thechild to the parent takes more work

Threads help with both problems Threads are sometimes called

lightweight processes since a thread is "lighter weight" than a

process That is, thread creation can be 10100 times faster thanprocess creation

All threads within a process share the same global memory.This makes the sharing of information easy between the

threads, but along with this simplicity comes the problem of

More than just the global variables are shared All threadswithin a process share the following:

execution (one thread) and a signal handler (anotherthread) If the main flow is in the middle of updating alinked list when a signal occurs, and the signal handleralso tries to update the linked list, havoc normally

pthread_ This chapter is an introduction to threads, so that wecan use threads in our network programs For additional detailssee [Butenhof 1997]

Termination

In this section, we will cover five basic thread functions andthen use these in the next two sections to recode our TCP

terminates either explicitly (by calling pthread_exit) or

implicitly (by letting the function return) The address of the

function is specified as the func argument, and this function is called with a single pointer argument, arg If we need multiple

arguments to the function, we must package them into a

structure and then pass the address of this structure as thesingle argument to the start function

Notice the declarations of func and arg The function takes one

argument, a generic pointer (void *), and returns a genericpointer (void *) This lets us pass one pointer (to anything wewant) to the thread, and lets the thread return one pointer

(again, to anything we want)

The return value from the Pthread functions is normally 0 ifsuccessful or nonzero on an error But unlike the socket

functions, and most system calls, which return 1 on an errorand set errno to a positive value, the Pthread functions returnthe positive error indication as the function's return value Forexample, if pthread_create cannot create a new thread

because of exceeding some system limit on the number of

threads, the function return value is EAGAIN The Pthread

functions do not set errno The convention of 0 for success ornonzero for an error is fine since all the Exxx values in

<sys/errno.h> are positive A value of 0 is never assigned toone of the Exxx names

A thread is either joinable (the default) or detached When a

joinable thread terminates, its thread ID and exit status areretained until another thread calls pthread_join But a

detached thread is like a daemon process: When it terminates,all its resources are released and we cannot wait for it to

terminate If one thread needs to know when another threadterminates, it is best to leave the thread as joinable

The pthread_detach function changes the specified thread sothat it is detached

#include <pthread.h>

int pthread_detach (pthread_t tid);

Returns: 0 if OK, positive Exxx value on error

This function is commonly called by the thread that wants todetach itself, as in

If the thread is not detached, its thread ID and exit status areretained for a later pthread_join by some other thread in thecalling process

The pointer status must not point to an object that is local to

the calling thread since that object disappears when the threadterminates

There are two other ways for a thread to terminate:

The function that started the thread (the third argument to

pthread_create) can return Since this function must bedeclared as returning a void pointer, that return value isthe exit status of the thread

If the main function of the process returns or if any threadcalls exit, the process terminates, including any threads

Our first example using threads is to recode the str_cli

function from Figure 16.10, which uses fork, to use threads.Recall that we have provided numerous other versions of thisfunction: The original in Figure 5.5 used a stop-and-wait

protocol, which we showed was far from optimal for batch

input; Figure 6.13 used blocking I/O and the select function;and the version starting with Figure 16.3 used nonblocking I/O.Figure 26.1 shows the design of our threads version

Figure 26.2 shows the str_cli function using threads

of the two arguments to str_cli: fp, the standard I/O FILE

pointer for the input file, and sockfd, the TCP socket connected

terminates by calling exit (Section 5.4) When this happens, all

thread, which is what we want

1625 This thread just copies from standard input to the socket.When it reads an EOF on standard input, a FIN is sent acrossthe socket by shutdown and the thread returns The return

from this function (which started the thread) terminates thethread

At the end of Section 16.2, we provided measurements for thefive different implementation techniques that we have used withour str_cli function The threads version we just presentedtook 8.5 seconds, which is slightly faster than the version using

fork (which we expect), but slower than the nonblocking I/Oversion Nevertheless, comparing the complexity of the

nonblocking I/O version (Section 16.2) versus the simplicity ofthe threads version, we still recommend using threads instead

of nonblocking I/O

We now redo our TCP echo server from Figure 5.2 using onethread per client instead of one child process per client We alsomake it protocol-independent, using our tcp_listen function.Figure 26.3 shows the server

Create thread

1721 When accept returns, we call pthread_create instead of

fork The single argument that we pass to the doit function isthe connected socket descriptor, connfd

We cast the integer descriptor connfd to be a void

close the connected socket since the thread shares all

descriptors with the main thread With fork, the child did notneed to close the connected socket because when the childterminated, all open descriptors were closed on process

termination (see Exercise 26.2)

each call to accept overwrites this variable with a new value

26.3, we solved this problem by passing the value of connfd to

Historically, the malloc and free

functions have been nonre-entrant That is, calling either function from a signal handler

while the main thread is in the middle of one of these two

functions has been a recipe for disaster, because of static data

Figure 26.5 Thread-safe functions.

Unfortunately, POSIX says nothing about thread safety with

property of gethostbyname and gethostbyaddr in Section

caller allocates space for the result and passes that pointer asthe argument to the function

/* next three are used internally by the function */

char *rl_bufptr; /* initialize to rl_buf */ char rl_buf[MAXLINE];

} Rline;

void readline_rinit(int, void *, size_t, Rline *); ssize_t readline_r(Rline *);

of structures per process, which we call key structures, as weshow in Figure 26.7

which we call the pkey array We show this in Figure 26.8

Figure 26.8 Information maintained by the

system about each thread.

These 128 pointers are the "values" associated with each of thepossible 128 "keys" in the process

When we create a key with pthread_key_create, the systemtells us its key (index) Each thread can then store a value

Figure 26.9 Associating malloced region with

thread-specific data pointer.

In this figure, we note that the Pthread structure is maintained

by the system (probably the thread library), but the actual

thread-specific data that we malloc is maintained by our

function (readline, in this case) All that

pthread_setspecific does is set the pointer for this key in the

Pthread structure to point to our allocated memory Similarly,all that pthread_getspecific does is return that pointer to us

readline calls pthread_once to initialize the key for this

Figure 26.10 Data structures after thread n

initializes its thread-specific data.

Thread n continues executing in readline, using and

modifying its own thread-specific data

thread terminates? If the thread has called our readline

function, that function has allocated a region of memory thatneeds to be freed This is where the "destructor pointer" fromFigure 26.7 is used When the thread that creates the thread-specific data item calls pthread_key_create, one argument to

this function is a pointer to a destructor function When a

thread terminates, the system goes through that thread's pkey

array, calling the corresponding destructor function for eachnon-null pkey pointer What we mean by "corresponding

destructor" is the function pointer stored in the Key array inFigure 26.7 This is how the thread-specific data is freed when athread terminates

The first two functions that are normally called when dealingwith thread-specific data are pthread_once and

#include <pthread.h>

int pthread_once(pthread_once_t *onceptr, void (*init) (void));

int pthread_key_create(pthread_key_t *keyptr, void (*destructor) (void

*value));

Both return: 0 if OK, positive Exxx value on error

pthread_once is normally called every time a function that usesthread-specific data is called, but pthread_once uses the value

in the variable pointed to by onceptr to guarantee that the init

function is called only one time per process

pthread_key_create must be called only one time for a given

key within a process The key is returned through the keyptr pointer, and the destructor function, if the argument is a non-

null pointer, will be called by each thread on termination if that

function uses the value pointed to by its onceptr argument (the

contents of the variable rl_once) to make certain that its init

function is called only one time This initialization function,

readline_once, creates the thread-specific data key that is

stored in rl_key, and which readline then uses in calls to

The pthread_getspecific and pthread_setspecific functionsare used to fetch and store the value associated with a key Thisvalue is what we called the "pointer" in Figure 26.8 What thispointer points to is up to the application, but normally, it points

earlier)

Example: readline

Function Using Thread-Specific Data

readline_destructor function, the readline_once function,

and our Rline structure that contains all the information we

caused the problem by being declared static in Figure 3.18.One of these structures will be dynamically allocated per threadand then released by our destructor function

Figure 26.12 shows the actual readline function, plus the

function my_read it calls This figure is a modification of Figure3.18

my_read function

1935 The first argument to the function is now a pointer to the

Rline structure that was allocated for this thread (the actualthread-specific data)

42 We first call pthread_once so that the first thread that calls

readline in this process calls readline_once to create the

thread-specific data key

Fetch thread-specific data pointer

4346 pthread_getspecific returns the pointer to the Rline

structure for this thread But if this is the first time this thread

has called readline, the return value is a null pointer In this

case, we allocate space for an Rline structure and the rl_cnt

member is initialized to 0 by calloc We then store the pointer

for this thread by calling pthread_setspecific The next time

this thread calls readline, pthread_getspecific will return

26.6 Web Client and Simultaneous Connections (Continued)

We now revisit the Web client example from Section 16.5 andrecode it using threads instead of nonblocking connects Withthreads, we can leave the sockets in their default blocking modeand create one thread per connection Each thread can block inits call to connect, as the kernel will just run some other threadthat is ready

Figure 26.13 shows the first part of the program, the globals,and the start of the main function

36 The home_page function that is called is unchanged from

Figure 16.16

Figure 26.13 Globals and start of main function.

threads/web01.c

1 #include "unpthread.h"

than maxnconn), we do so The function that each new thread

executes is do_get_read and the argument is the pointer to the

socket, so the thread will block in the call to connect until the

Notice in Figure 26.14 that when a thread terminates, the mainloop decrements both nconn and nlefttoread We could haveplaced these two decrements in the function do_get_read,

letting each thread decrement these two counters immediatelybefore the thread terminates But this would be a subtle, yetsignificant, concurrent programming error

The problem with placing the code in the function that each

thread executes is that these two variables are global, not

thread-specific If one thread is in the middle of decrementing avariable, that thread is suspended, and if another thread

executes and decrements the same variable, an error can

result For example, assume that the C compiler turns the

decrement operator into three instructions: load from memoryinto a register, decrement the register, and store from the

register into memory Consider the following possible scenario:

1 Thread A is running and it loads the value of nconn (3) into a register.

The system switches threads from A to B A's registers aresaved, and B's registers are restored

Thread B executes the three instructions corresponding tothe C expression nconn , storing the new value of 2

Sometime later, the system switches threads from B to A A'sregisters are restored and A continues where it left off, at thesecond machine instruction in the three-instruction sequence.The value of the register is decremented from 3 to 2, and thevalue of 2 is stored in nconn

The end result is that nconn is 2 when it should be 1 This is

These types of concurrent programming errors are hard to findfor numerous reasons First, they occur rarely Nevertheless, it

is an error and it will fail (Murphy's Law) Second, the error ishard to duplicate since it depends on the nondeterministic

timing of many events Lastly, on some systems, the hardwareinstructions might be atomic; that is, there might be a

hardware instruction to decrement an integer in memory

(instead of the three-instruction sequence we assumed above)and the hardware cannot be interrupted during this instruction.But, this is not guaranteed by all systems, so the code works onone system but not on another

We call threads programming concurrent programming, or

parallel programming, since multiple threads can be running

concurrently (in parallel), accessing the same variables Whilethe error scenario we just discussed assumes a single-CPU

system, the potential for error also exists if threads A and B arerunning at the same time on different CPUs on a multiprocessorsystem With normal Unix programming, we do not encounterthese concurrent programming problems because with fork,nothing besides descriptors is shared between the parent andchild We will, however, encounter this same type of problemwhen we discuss shared memory between processes

We can easily demonstrate this problem with threads Figure26.17 is a simple program that creates two threads and thenhas each thread increment a global variable 5,000 times