[Bernstein09] Section 5.5. Making Process State Durable

5.5. Making Process State Durable
Wehave seen several reasons why business process state needs to bemaintained durably. This section discusses several techniques for doingso:
A special-purpose runtime system for business processes
Persistent queues
Pseudo-conversations
Message logging
Using a Special-Purpose Runtime System
Considera system that uses a special-purpose runtime system to support businessprocess management. Suppose the runtime supports a function SaveStatethat stores the current state of a business process in persistentstorage. Suppose the business process calls SaveState and then executesanother step as a transaction, T. In general, executing a transactionis not an idempotent operation. So if the system fails and, afterrecovering, resumes executing the business process, it is importantthat the business process doesn’t execute T a second time. This problemis reminiscent of the problem of handling non-undoable andnon-idempotent operations inSection 4.4,where we wanted to process the output of a transaction (e.g., print acheck) exactly once. In this case, we want to run T in the businessprocess exactly once. It is essentially the same problem, except inthis case the non-undoable operation is a transaction.
The way we solved this problem inSection 4.4 was the equivalent of invoking SaveState immediately before running T, as illustrated inFigure 5.3.Then, before invoking T, check that it didn’t run once before. Thisassumes that T produces some testable state; that is, some persistentstate that proves whether it did or did not run. This can be arrangedif the invocation of the step that corresponds to T is done via aspecial function in the business process’s runtime. That function canperform the required update of persistent storage that, if needed, canbe checked later to determine if T ran.
Figure 5.3. The SaveState Operation Runs Before Transaction T. Saving the state of the business process in step S1 allows the state to be tested before running step S2 to prevent re-execution of T in the event of a recovery from failure.

SinceT is a transaction, another possible solution is to have the businessprocess’s state be a transactional resource and make SaveState the lastoperation of the transaction. That way, the business process’s state issaved in persistent storage if and only if T commits. If the systemfails before T commits, then at recovery time the business process willcontinue executing with the last state that was saved before theexecution of T. So T will execute again, which is what we want sinceT’s first execution aborted as a result of the failure. If the systemfails after T commits, then at recovery time the business process willcontinue executing with the statement that follows the SaveStateoperation. We would like this to be the statement that follows theexecution of the commit operation for T. So we would like the SaveStateoperation to save the state that the transaction committed.
Noticethat it is not satisfactory to invoke the SaveState operationimmediately after T commits, instead of invoking it as part of T. Thereason is that the system might fail after T commits and beforeexecuting the SaveState operation that follows the commit. That wouldresult in having T execute again at recovery time, which is one of theoutcomes we want to avoid.
Using Queued Requests
Anotherway to manage the state of a business process is to use persistentqueues. The business process can be initiated by a request that isstored in a queue. Each step of the business process executes as atransaction and produces requests for the succeeding steps of thebusiness process if and only if the step commits. Thus, the state ofthe process is stored in the queue elements holding the active requestsof the business process.
Sincethe steps are separated in time, some requests in a business processmay stay in a queue for a long time. However, since the queue ispersistent, the request cannot be lost. Moreover, no special recoveryprocedure is required. If the system fails and then recovers, therecovery procedure will restart the configuration of the serverprocesses, which includes the dispatchers associated with its queues.These dispatchers will then start dequeuing requests and invoking theappropriate programs to run them.
Inthis approach, there may be no one program that encapsulates thecontrol flow of the business process. Instead, that logic could bedistributed among the steps of the process. Each step of the processhas an associated program that performs the work of that step, whichincludes defining what happens next by virtue of the requests that itproduces. Since no one program defines the business process, theprocess has no local variables that need to be stored persistently. Ifstep S produces any information that is needed by subsequent steps,then S needs either to pass that information along in the requests itproduces or to store it in a shared database that is accessible tosubsequent steps.
Insteadof distributing the logic of business process steps, the businessprocess could be encapsulated in a single program. The program executeseach step as a transaction that dequeues the expected request, executesit, and enqueues output requests for the next steps. Since queueelements are persistent, the process can save its state there insteadof saving it periodically in a separate transactional resource.However, it still needs to save its control state periodically in awell-known location, as we described in the previous subsection, sothat at recovery time the business process can be resurrected andresume execution at the point where it left off.
Thetop-level request that initiates a business process often requires areply that tells the user when the business process is completed. Forexample, the reply might include the itinerary for a trip, thereimbursement of an insurance claim, or the acknowledgment of a moneytransfer. However, intermediate steps of the workflow often do not needto reply to the originator of the step’s request. Rather than sending areply to the previous step of the business process, each intermediatestep feeds a request to perform the next step of the business process.Each intermediate step might also send an e-mail or other form ofnotification to the end-user of the latest step that was performed(e.g., the order was received or the order was shipped), but it doesnot expect a reply to such notifications.
Forexample, consider the problem of moving orders from one office toanother in a global enterprise. The Tokyo office runs a transaction toenqueue an order request (seeFigure 5.4).The server recognizes the request as one that requires remoteprocessing, so it runs a transaction that dequeues the order from thequeue in Tokyo and enqueues it to another queue in New York. Now aserver in New York dequeues the order, processes the order, andenqueues a shipping request. When the order ships, a transactionrecords that fact, enqueues a message containing an invoice and anacknowledgment that the order was filled, and perhaps sends a shippingnotification to the end user. A transaction forwards the reply from thequeue in New York back to the queue in Tokyo. The Tokyo server printsthe invoice and acknowledgment and mails it to the customer. That finalstep in Tokyo is effectively a reply to the original order request. Theintermediate steps are a chain of steps where each step sends a requestto perform the next step.
Figure 5.4. A Multitransaction Business Process. Eachboxed action runs as a transaction. An order is entered in Tokyo,forwarded to New York for processing, processed, and shipped, and areply is forwarded to Tokyo, which prints an invoice for the order.

Pseudo-Conversations
Anothertype of business process arises from an interactive request; that is,one that interacts with a display device. As before, due to resourcecontention and availability, it is wise to break up the execution intoseveral transactions, one for each point of interaction.
Forexample, consider an airline reservation transaction that gets a flightnumber as input from a display, reads the number of seats available onthat flight from the database, and then displays that number and asksthe ticket agent how many seats the customer wants to reserve (seeFigure 5.5).After the customer says how many seats to reserve, this number isentered as input, the number of available seats is decremented, and thetransaction commits. To make such transactions serializable, the systemordinarily holds a lock on that flight record for the duration of thetransaction. This blocks other customers from making reservations onthat flight. The blocking delay could be significant, while thecustomer is deciding how many seats to reserve. For this reason,determining the number of available seats usually runs as a separatetransaction from reserving the seats. That way, the flight record isn’tlocked while the customer is deciding how many seats to reserve. Ofcourse, this means that the number of available seats can change whilethe customer is deciding. That’s why the ticket agent often reservesseats on the flight you inquire about, to make sure the seats don’t goaway while you’re deciding; if you decide not to reserve them, theagent cancels the reservation.
Figure 5.5. An Interactive Transaction. Duringthe delay while the user is deciding how many seats to reserve, theflight information is locked, preventing other users from accessing it.

Inmost ways, making this airline reservation is an ordinary multisteprequest, consisting of two transactions. The first displays the numberof available seats and the second makes a reservation. Like othermultistep requests, one could implement it as a business process; forexample, by moving requests between client and server queues. However,these sorts of interactive situations arise often enough that somesystems have a special mechanism, called pseudo-conversations, where the request is shuttled back and forth between client and server.
Withpseudo-conversations, each time the server processes a request message,it saves some information that was gathered from that transaction stepin the message that it returns to the client device (essentially aqueued reply). Some of this information may be displayed to the user(e.g., number of available seats). Other information may be there justas context that can be sent back from the client to the nexttransaction step (e.g., an identifier for a partially-completedreservation record). The message is saved in persistent storage on theclient and the server. But since there’s only one message ever intransit between them, the system doesn’t need a mechanism as elaborateas queues. It just needs a block of persistent storage reserved foreach client. In a sense, the pseudo-conversation is a session with themessage containing the state being shared by the client and server.Thus, any technique for managing persistent session state could beused. For example, the client state could be stored in a cookie in theweb browser.
Thisway of exchanging messages is called a pseudo-conversation because itlooks as if it’s a conversational transaction; that is, it looksinteractive. In fact, it’s just a sequence of noninteractive requests,each of which has one input and one output.
Using Logging
Loggingis another way to make interactive requests reliable, without apseudo-conversation or queuing. In this approach, the system runs theinteractive request as one transaction (not a sequence of transactions)and logs all the transaction’s input/output operations to the displayor the communications system.
Ifthe transaction aborts and restarts, then the system executes therestarted transaction in “restart mode.” In this mode the system usesthe log to service the transaction’s input requests. That is, insteadof reading from the display, the restarted transaction reads the valuesproduced during the previous execution, which are in the log (seeFigure 5.6).If there are no more logged input values to read, then the systemresumes processing the restarted transaction’s input operations in“normal mode,” by reading from the display or communication system.
Figure 5.6. Message Log for Transaction Recovery. Duringrecovery, the transaction replays by (1) getting its input messagesfrom the log, (2) executing until it produces output, and (3) comparingits output messages to the result of the previous execution. If theoutput is the same as its previous execution it continues the replay.

Whilein restart mode the system processes each of the restartedtransaction’s output operations by comparing it to the output that wasrecorded in the log during the original execution. These two outputsmight differ because the restarted transaction read a different valuefrom the database than the original transaction read. If so, then thesystem aborts the transaction and restarts it, but this time executingit in normal mode. If not, then the restarted transaction continuesexecuting in restart mode. If there are no more logged output values tocompare it to, then the system resumes processing the restartedtransaction’s output operations in “normal mode,” by writing to thedisplay or communication system.
Theimplementation of this approach can get quite complicated. There aremany ways a transaction can perform input and output. Each type ofinput and output must be logged during the original execution. And foreach type of operation the system must be able to reproduce the loggedbehavior and to detect when the restarted transaction has exhibiteddifferent behavior.
Theexecution of a business process can use this message logging techniqueto attain a similar level of fault tolerance as it would have using anyof the techniques described earlier in this section, namely, aspecial-purpose runtime, queuing, or pseudo-conversations. However,unlike those earlier approaches, in this case the business process mustexecute as one transaction and not be decomposed into multipletransactions. Therefore, this technique by itself does not avoid theproblems of resource contention and availability, which are two otherreasons why it may be undesirable to execute a business process as onetransaction. Thus, it’s suitable only when they are not criticalproblems.