QUEUED TRANSACTION PROCESSING WITH WEB SERVICE RELIABLE MESSAGING A.C. SURIARACHCHI This Dissertation was submitted to the Department of Computer Science and Engineering of the University of Moratuwa in partial fulfillment of the requirements for the Degree of MSc in Computer Science specializing in Software Architecture Department of Computer Science and Engineering University of Moratuwa February 201 0 96424 ABSTRACT With the popularity of the distributed business applications, the application data is distributed in various physical storages. However most of the business transactions require to update data stored in more than one storage. hence updating two data storages reliably is a common problem for most of the distributed business applications. Queued transaction processing is a concept widely used to achieve such a processing model using intermediate queues to transfer messages reliably. In such a system at the client side, both updating the client storage and writing the message to be sent to the client side message queue happens in the same distributed transaction. Similarly at the server side reading the message from the server side queue and updating the sever storage happens in the same distributed transaction. Bur such a system may have interoperability problems if client and server use different types of technologies. Web services are used to communicate among the heterogeneous systems by passing SOAP messages using standard transport mechanisms like http. Web services can reliably communicate by using WS-Reliable messaging specification(WS-RM). WS-RM uses concepts of Reliable messaging source (RMS) and Reliable messaging destination ( RMD) between which it guarantees reliable massage delivery. By combining these two concepts, we introduce an approach to solve the above mentioned problem in an interoperable manner using WS-RM ..,to communicate between nodes while keeping RMS and RMD as intermediate storages. In our model reliable message delivery happens in three phases. First both updating application client storage and writing message to the RMS happens in the same distributed transaction. Then WS- RM protocol reliably transfers the message to RMD at the server side . Finally- at the server reading the message from the RMD and updating the server storage happens in the same distributed transaction. The middleware software entity that we developed to encapsulate this approach is called Mercury which implements WS-RM protocol. DECLARATION "The work included in thi.s report was done by me, and only by me, and the work has not been submitted for any other academic qualification at any institution" ____ __ QJ ~{J>~J~L ___ ___________________ __ Name: A.C. Suriarachchi (08H254P) Date: 2010.02.26 .... "I certify that the declaration above by Lite candidate is true to the best of my knowledge and that this dissertation is acceptable for evaluation for the Degree of M.Sc in Computer Science specializing in Software Architecture" ....... - i - ACKNOWLEDGMENTS I wish to sincerely thank my supervisors Mr. Paul Fremantle and Dr. Sanjiva Weerawarana for providing me the research idea and supervision of my work continuously. They provided me necessary guidance, various levels of requirements and encouragement to fulfill my objective. I would also like to thank Dr Sri nath Perera who reviewed my work and provided me valuable feedback. I am also grateful to Prof. Gihan Oias and Dr Sanath Jayasena who worked as the course coordinators, and provided valuable feed hack at various levels of the pr~jecl. I would like to extend my thank to all the academic staff of the University of Moratuwa for the great work they did for us during the course of study. No student can survive in a university without the help of their fellow students to discuss ideas, share opinions, and to make time spent in the lab and all round enjoyable experience. T would be grateful for all M.Sc 08 colleagues for the corporation given to the successful completion of my project involvements. I must also be grateful to my parents and brothers for the encouragement they provided to follow the Msc. I wish to express my gratitude to all my colleagues at WS02 who have enormously helped to learn a lot about web services and distributed systems . .., ~ - iii - TABLE OF CONTENTS DECLARJ\ TION ..................................................................................................... ....... .................. i ABSTRACT ...................................... ............ ............. ... .................................................. ................. ii ACKNOWLEDGMENTS .............................................................................................................. iii TABLE OF CONTENTS ................................................................................................................ iv LIST OF FIGURES ........................................................................................................................ vi LIST OF ABBREYIATIONS ....................................................................................................... viii Chapter I Introduction ..................................................................................................................... I 1.1 Background ........................................................................................................................... I 1.2 Abstract Problen1 .................................................................................................................. 2 1.3 Method of study .................................................................................................................... 3 1.4 Previous work ............................................ .. ......................................................................... 3 1.5 Expected rcsu It. .......................... , .......................................................................................... 3 Chapter 2 Literature Review ............................................................................................................ 4 2.1 Web service standards .......................................................................................................... 5 2.1.1 SOAP ............................................................................................................................ 5 2.1.2 WS-Addressing ............................................................................................................. 5 2.1.3 WS-Reliable messaging ................................................................................................ 6 2.2 Transactions and messaging standards ................................................................................. 9 2.2.1 2PC ............................................................................................................................... 9 ... 2.2. 2 X/ Open distributed transaction standards ................................................................... I 0 2.2.3 JTA ............................................................................................................................. 11 2.3 WS-Transactions ................................................................................................................. 13 2.3.1 WS-Coordination ........................................................................................................ 13 < 2.3.2 WS-Atorrlic transactions ............................................................................................. 15 2.3.3 WS-BussinessActivity .............................................................. ~ ............................... 15 2.4 Queued Transaction processing .......................................................................................... 16 2.5 Different types of reliable web services .............................................................................. 16 2.5.1 Using Message Oriented Middlt.!warc for Reliable Web Services Messaging ........... 16 2.5.2 Three facets of Reliability .......................................................................................... 17 2.5.3 Reliable messaging for web services .......................................................................... I? 2.5.4 Assessn1ent ................................................................................................................. 18 2.6 Transactions and messaging ............................................................................................... 20 2.6.1 MQ Integrating Transactions ...................................................................................... 20 2.6.2 Message delivery transactions .................................................................................... 20 2.6.3 Message processing transactions ................................................................................ 21 2.6.4 Full messaging transactions ........................................................................................ 22 -IV- 2.7 Middleware mediated transactions ..................................................................................... 22 2. 7.1 D-sphere ..................................................................................................................... 23 Chapter 3 Methodology ................................................................................................................. 25 3.1 Previous Solution ................................................................................................................ 25 3.2 Proposed Solution ............................................................................................................... 26 3.3 Alternative Solutions .......................................................................................................... 27 3.3.1 Integrate WS-RM protocol with the client storage and server storage hy taking then1 as RMS and RMD ....................................................................................................... 27 3.3.2 Use the same transaction to update both application level storages and WS-RM storages ................................................................................................................................ 28 3.4 Solution Architecture .......................................................................................................... 29 3.4.1 State n1achine model ................................................................................................... 29 3.4.2 Run time Architecture ................................................................................................ 36 3.4.3 Storage API ................................ ................................................................................ 40 3.4.4 Other issues and solutions .......................................................................................... 45 Chapter 4 Use case scenarios ......................................................................................................... 46 4.llnMernory ........................................................................................................................... 47 4.l.l ln Only invocation ...................................................................................................... 47 4.1.2 In Out lnvocation ........................................................................................................ 48 4.1.3 Fault Ilandling ............................................................................................................ 49 4.2 Persistence .......................................................................................................................... 49 4.2.lln Only Invocation ..................................... "! ....... : ........................................................ 50 4.2.2 In Out Invocation and Fault Handling ........................................................................ 51 4.3 JTA .............................................. , ...................................................................................... 52 4.3.1 In Only Invocation ...................................................................................................... 55 Chapter 5 Observations & Results ........................ • ......................................................................... 57 Chapter 6 Conclusion & Future Work ........................................................................................... 59 -. REFERENCES .............................................................................................................................. 61 - v - LIST OF FIGURES Figure 1-1 Abstract Proble1n ...................................................................................................... 2 'figure 2-1 Reliable Messaging Model ............................................................ .......... ................. 7 Figure 2-2 Reliable Messaging Protocol .................................................................................... 7 Figure 2-3 Coordinator States .................................................................................................... 9 figure 2-4 Participant States .... .. .............................................................................................. 10 Figure 2-5 X/Open Distributed Transaction Standards ............................................................ II figure 2-6 JTA Overview ........................................................................................................ 12 Figure 2-7 WS Coordination Framework ................................................................................ 14 Figure 2-8 Queued Transaction Processing ............................................................................. 16 Figure 2-9 MQ Integrating Transactions .................................................................................. 20 Figure 2-10 Message Delivery Transactions ............................................................................ 21 Figure 2-11 Message Processing Transactions ........................... .......................... ................... 22 Figure 2- 12 Application Without 0 -sphcrc ........................................... ................ ..... ............. 23. Figure 2-13 Application With D-sphcrc ............................. ..... .................................... ............ 23 Figure 2-14 0 -sphere Architecture .................................... ... ................................................... 24 Figure 3-1 Previous Solution ................................................................................................... 25 Figure 3-2 Proposed Solution .................................................................................................. 26 Figure 3-3 Integrate Persistence Storage with RM Storages ................................................... 27 Figure 3-4 Using the same Transaction ................................................................................... 28 .., figure 3-5 RMSSequence Events ............................................................................................ 29 Figure 3-6 RMSSequence State~ ................................................................... .......... ................. 30 Figure 3-7 RMSSequence State Machine ................................................................................ 31 Figure 3-8 RMDSequence Events ............................................................................................ 31 . Figure 3-9 RMDSequence States ............................................................................................. 32 Figure 3-10 RMDSequence State Machine .................................................... -:-:~ ....................... 33 Figure 3- 11 InvokerBuffer Events ........................................................................................... 34 Figure 3-12 Invoker Buffer States ............................................... ................................ ............. 35 Figure 3-13 Invoker Buffer State Machine ........................................ : ...................................... 35 rigure 3-14 InOnly Messages Runtime ................................................................. .... ...... ........ 36 Figure 3-15 InOut Messages Runtime ..................................................................................... 38 Figure 3-16 Fault Handling Runtime ....................................................................................... 39 Figure 3-17 Storage API .......................................................................................................... 40 Hgure 3-18 lnMemory Implementation .................................................................................. 41 Figure 3-19 Persistence Implementation .................................................................................. 42 Figure 3-20 Database Design ................................................................................................... 44 Figure 4-1 InMen1ory Invocation ............................................................................................. 47 -vi - Figure 4-2 In Only Client ......................................................................................................... 47 Figure 4-3 TnMen1ory In Out Client. ........................................................................................ 48 Figure 4-4 SendAsynchornousMessagc Method ..................................................................... 48 Figure 4-5 Fault Message Receiver ......................................................................................... 49 Figure 4-6 Using Persistence Storage .......................................... ............................................ 49 Figure 4-7 Persistence Storage Configuration ......................................................................... 50 Figure 4-8 Persistence In Only Client ...................................................................................... 50 Figure 4-9 Persistence In Only McssagcRcceivcr ................................................................... 51 Figure 4-10 Persistence Callback llandlcr Methods ................................................................ 52 Figure 4-11 JT A Invocation ..................................................................................................... 52 Figure 4-12 JT A Storage Configuration .................................................................................. 53 Figure 4-13 Atomikos JT A Connection Manager .................................................................... 54 Figure 4-14 JT A In On I y Client ............................................................................................... 55 Figure 4-15 InvokerWorker ..................................................................................................... 56 ... '"10o; - vii - ws WS-RM RMS RMD 2PC JTA JTS SOAP RPC MOM LIST OF ABBREVIATIONS Web Service Web Service Reliable Messaging Reliable Messaging Source Reliable Messaging Destination Two Phase Commit Java Transaction API Java Transaction Service Simple Object Access Protocol Remote Procedure Calls Message Oriented Middlcwan.: - viii - ... """' Chapter 1 Introduction 1.1 Background Updating two data storages reliably is a widely researched area in distributed computing. Most of the existing solutions follows a queued transacti~n processing model. In such a model first client writes the message stored in its persistence storage to the request queue within a distributed transaction and server reads the request from the request queue within another distributed transaction. If there i~ a re~ponse to be sent, server writes the response to response queue within the same transaction it read the message and finally client reads the response from the response queue within another distributed transaction. TQLs processing model can operate even with the presence of the node failures due to recovery nature of the distributed transactions. However these systems may not properly inter-operate due to use of many proprietary messaging protocols. Web services are used to communicate among the heterogeneous systems by passing SOAP messages using standard transport mechanisms like http. Web services can reliably communicate by using WS-Reliable messaging specification(WS-RM). WS-RM introduces concepts of Reliable messaging source (RMS) and Reliable messaging destination (RMD) between which it guarantees reliable me~sage delivery. As a result of this both RMS and RMD can be considered as intermediate queue~ by using a persistence storage to implement them. - 1 - An inter operable reliable message transferring system can be made by combining above two concepts where client writes messages from the persistence storage to RMS within a distributed transaction and server writes the messages from RMD to persistence storage within a distributed transaction. Reliable conununication between RMS and RMD is guaranteed by WS-RM. 1.2 Abstract Problem r==~--- ~~~ Persistence storage -----·- ____ . ./ Client Node Figure 1-1 Abstract Problem ,..--- '-·----- -~ _ ... - ·· - Persistence Storage ...... ____ _ SeNer Node This project focus on updating client persistence storage and server persistence storage reliably by sending a message in a systt!m shown in Pig 1-l. Assume there are two nodes called client node and server nodt! with persistence storages, connected through a network. How to guarantee both client side and server side Mo'rage updates by sending a message from client node to server node with the presence of failures in an inter operable manner? Message provides the necessary information to update the server persistence storage. The term reliably refers to the exactly one delivery. This means there can be no message losses or . duplicate messages. Failures can either be network or node failures. For this work web services and standards arc being ust:d as the means of achieving inter~rability. Further it is assumed that although it is possible to have network and node failures they recover in finite time and there arc no persistence storage failures. The main goal of this project is to implement a web service reliable messaging middlcware which can generally be used in such a situation. Writing a WS-RM implementation from the scratch means a lot of work. Therefore this project aims to re-engineer the existing WS02 Mercury to solve the above mentioned problem. WS02 Mercury is a WS-RM implementation written on top of Axis2 by using a state machine model. However WS02 Mercury keeps the state of the WS-RM communication in an in memory object model. It achievt:s the persistence by saving this in memory object model to a persistence storage. Although WS02 Mercury has successfully implemented the state -2- machine model. its' in memory model described above does not allow it to support user initiated transactions. Therefore the main objective of this project can be narrowed down tore engineer the existing WS02 Mercury code to come up with a storage API which support user level transactions. However some of the WS-RM usage scenarios do not require user level transactions and hence it is enough to have an in memory storage model. Therefore above storage API should support simple in memory implementations as well. 1.3 Method of study Implementing a new storage API directly with the WS02 Mercury can be complex. Further in such an attempt main focus may not be in the storage API design. Therefore this project first designs the storage API within a simulator. Again the simulator which is used at the time of designing Mercury state machine model can be used for that. Then the new storage API can be implemented in an in memory model with the simulator and can be transferred to the actual Mercury implementation with the necessary refactoring or the Mercury. Finally the storage API can be implemented with a persistence storage and can be tested for distributed transaction scenarios. 1.4 Previous work As given in the background section this problem has been solved by using intenncdiate queues. But this project aims to do that using web serviC'es and related standards to achieve interoperability. Apache Sandesha2 which is another WS-RM specification implementation uses such a transactional data store model. However Apache Sandesha2 does not use a state machine c model and further a transactional storage to support even an in memory model as well . .,.. 1.5 Expected result In summary this project aims to come up with a storage API ·with the necessary WS02 Mercury runtime architecture which supports both user level transactions and simple in memory implementations. To prove this point it expects to have at least two storage API implementations one for simple memory implementation and other for a transactional permanent storage implementation. r:urther it aims to provide necessary usage scenarios which uses the distributed transactions to achieve end to end reliability. - 3 - Chapter 2 Literature Review Literature review of this projects spans across many areas. One of the obvious areas is the web ... services standards and related specifications. Web services primarily use SOAP[3] as the messaging format. WS-Addressing[ I 01 provides a way to address end point references in a transport independent way. WS Reliable messaging specification[8] uses WS-Addrcssingf 10] to correlate the request and response messages. There are some set of standard protocols and standards to generally support transactions and -... messaging. 2PC[7] is the widely used protocol to achieve distributed transactions. X/Opcn promotes standards for many protocols to improve the interoperability. X/Open distributed transactio n specificatio n[l l ] s tandardi;o; the use of 2PC protocol. JTN JTS[ l 2Jl l 3] provide java specific APls for distributed transactions. WS-Transaction specifications which includes WS-Coordination[4 ]. WS- AtomicTransactions[5] and WS-BussinessActivity[6] provides means to achieve distributed transactions using 2PC' protocol in an inter-operable way. Queued transaction processing is used for processing a transaction between a client and an application server asynchronously in a distributed transaction processing environment having at least one transaction queue manager. - 4- IBM has done some work[ 14] related to this area. This includes their classification of varios ways to integrate the web services and transactions. llttpr is an effort to build a reliable protocol on top of Http. Finally there have been many researches for message oriented and object oriented transactions. Further these researches have been extended to middleware mediated transactions[ IS] which combines the above two concepts to achieve better transaction support. 2.1 Web service standards 2.1.1 SOAP Simple Object Access Protocol (SOAP)[3] is a protocol to exchange information in a decentralized, distributed environments developt!d by Microsoft and IBM. SOAP can support to enable remote procedure calls (RPC) over llTfP using XML. SOAP protocol specification mainly consists of three parts. 1. SOAP Envelope SOAP envelope describes what is in the m<.:ssagc and how to process it. A SOAP envelope has a required body part which is used to send the actual message, and header parts which can be used to provide the soap envelope processing instructions. 2. Set of encoding rules There are a set of encoding rules which specify how to encode application-defined data types in to XML fom1at. This is important since SOAP provides an inter operable XML based messaging format. 3. Convention to represent remote procedure calls and responses SOAP defines a way to encode a RPC invocation request and the response into a SOAP envelope. This is used in RPC type service invocations. ~ 2.1.2 WS-Addressing Web services can be accessed by sending SOAP messages to· their respective endpoints. However the endpoint details may depend on the transport protocol. And also there are some infonnation required by the messaging systems in order to dispatch messages to corresponding processes and correlate them. Web Services Addressing (WS-Addressing)[IO] defines two inter operable constructs that convey information that is typically provided by transport protocols and messaging systems. These constructs normalite this underlying information into a unifonn format that can be processed independently of transport or application. - 5 - 1. Endpoint references A Web service endpoint is an entity wh<.!re Web service messages can be targeted. Endpoint references convey the information needed to identify/reference a Web service endpoint. Endpoint references are suitable for conveyi ng the information needed to access a Web service endpoint, but are also used to provide addresses for individual messages sent to and from Web services. 2. message information headers This defines a famjly of message information headers that allows uniform addressing of messages independent of underlying transport. These message information headers convey end-to-end message characteristics including addressing for source and destination endpoints as well as message identity. WS-Reliable messaging uses WS-addressing headers to specify endpoint addresses and convey message related information. 2.1.3 WS-Reliable messaging Reliable message delivery is a common concept in message oriented communication. WS-ReliableMessaging spccification[8] (WS-RM) describes a protocol that allows messages to be delivered reliably betwet:n distributed applicat~ns in the presence of network failures. The protocol is described in this specification in a transport-independent manner allowing it to be implemented using different network technologies. To support inter operable Web services, a SOAP binding is defined within this specification. The protocol defined in this specification depends upon other Web services specifications for the identification of service endpoint addresses and policies. This protocol does not talk about ~ the delivery guarantees and persistence. However WS-RM implementations can provide persistence and delivery guarantees using the available protocol constructs. WS-Reliable messaging is based on a rcliabl<.! message model which is given below. - 6 - ====--- --- -- Acknowledge Figure 2-1 Reliable Messaging Model Following diagram shows the entities and events in a simple reliable message exchange. First, the Application Source sends a message for reliable delivery. The Reliable Messaging (RM) Source accepts the message and Transmits it one or more times. After receiving the message, the RM Destination acknowledges it. Finally, the RM Destination delivers the message to the Application Destination. Reliable Messaging Protocol Establish Protocol Precon'tlitions CreateSequence() Endpoint B ~ ___ _E!:_e~t!~9.U.!~c_:~e_:P2'2.s~(1d_:!.!t!fi_:r_ =_h~tf:~~a~r~k~~ 1]~-=.o!"IaE:._) ________ _ j Sequence( Identifier= http://fabrikam123.coni'/abc, MessageNumber = 1 ) .._ Sequence( Identifier= http://fabrikam123.com/a.bc, MessageNumber = 2~ X Sequence( Identifier= http://fabrikam123.com/abc, MessageNumber = 3, LastMessage) l. ___ _:;~q~~n_:e,l'=.k~~w1e_?~e~_:~ J. ~~_!i~e.!.: ~t_!p..:_lif~b_!l~a!" .2 P.:.c~~~!~!... ______ _ ! AcknowledgementRange : 1, 3 ) : Sequence( Identifier = http://fabrikam123.co' n/abc, MessageNumber = 2, AckRequested ,. : l. ___ _:;~q~~n~e.i'=.~~w_!c_9~e~!~ J. ~~_!l~e! _: ~t_!P_:I_~f~b_:l~a!"'_2~3.:.c~~/~~!... ______ _ ~ AcknowledgementRange : 1 ... 3 ) : TerminateSequence ( Identifier : http://fabrikam1 23.com/abc ) .._ Figure 2-2 Reliable Messaging Protocol - 7 - Following steps illustrates a typical set of messages passed in one RM sequence and how it provides fault tolerance. It uses a acknowledgment based retransmission similar to TCP. 1. The protocol preconditions arc established. These include policy exchange, endpoint resolution, establishing trust. 2. The RM Source requests creation or a new Sequence. 3. The RM Destination creates a Sequence by returning a globally unique identifier. 4. The RM Source begins sending me~sagcs beginning with McssageNumbcr 1. In the figure the RM Source sends 3 messages. 5. Since the 3rd message is the last in this exchange, the RM Source includes a token. 6. The 2nd message is lost in transit. 7. The RM Destination acknowledges receipt of message numbers 1 and 3 in response to the RM Source's token. 8. The RM Source retransmits the 2nd message. This is a new message on the underlying transport, but since it has the same sequence identifier and message number so the RM Destination can recognize it as equivalent to the earlier message, in case both are received. 9. The RM Source includes an clement so the RM Destination will expedite an acknowledgment. 10. The RM Destination receives the second transmission of the message with .., MessageNumber 2 and acknowledges receipt of message numbers 1, 2, and 3 which carried the token. 11. The RM Source receives this acknowledgment and sends a TerminatcScquence message to the RM Destination indicating that the sequence is completed and reclaims < any resources associated with the Sequence. 1 12. The RM Destination receives the TerminateSequence message indiceting that the RM Source will not be sending any more messages, and reclaims any resources associated with the Sequence. - 8 - 2.2 Transactions and messaging standards 2.2.1 2PC Two phase commit protocol[7] is a protocol to support transactions in a distributed environment. In a distributed environment there arc multiple participants. These multiple participants update multiple data sources. Two phase conunit protocol ensure either these participants corrunit or abort atomically. Two phase commit protocol is executed by a process called the coordinator process and other participant processes. As the name suggests two phase commit protocol has two phases called prepare phase and conunit phase. Both of these participants' life cycles has been defined by the state transfer diagrams. Figure 2-3 Coordinator States -9 - Figure 2-4 Pat1icipant States Before the commit process starts, both coordinator and participants processes are at the initial state. Commit process starts when the initiator sends the commit message to the coordinator. Getting the commit message coordinator sends the prepare message to all the participants and moves to the prepared state and waits until all the responses come. When a participant .., receives a prepare message from the coordinator it sends the response as 'yes' and moves to prepared state if it is prepared to commit or sends the response as 'no' and moves to aborted state if it is not prepared to commit. I Iere if a participant sends a 'yes' response it can't later say it is not prepared to commit. Once all the participants sends their responses coordinator c can decide either to commit the transaction or abort it. If there is at least one 'no' response coordinator have to decide to abort the transactions. After that coordinatOl"tells its participants either to abort or commit and then moves to either commit or abort state. Once the participants gets the global commit or abort message from the coordinator it moves to the corresponding state and sends the acknowledgment back to the coordinator. 2.2.2 X/Open distributed transaction standards X/Open is a independent, worldwide, open systems organization which supports implementation of open systems. In the context of the distributed transactions, X/Opcn has standardize the interface between the Transaction Manager and the Resource Manager in order to make them as open systems[ II]. X/Open distributed transaction processing (DTP) model assumes three software components. - 10- Resource Managers (RMs) Figure 2-5 X/Open Distributed Transaction Standards Application program specify the transaction boundaries and ~pecifies the actions that constitute the transaction. Resource managers provides the resources which application program updates during a transaction. Transaction manager is the main component which assigns identifiers to transactions, monitor their progress and do the transaction completion or failure recovery. Out of these interactions X/Open specification introduces a standard interface to communicate between the Transaction manager and the Resource managers. These interfaces arc specified in C programming language. "> 2.2.3 JTA Java transaction API specification[ 12] provides a set of java interfaces to support distributed transactions. It specifics the local Java interfaces between a transaction manager and the parties involved in a distributed transaction system. Following diagram shows the interfaces it defines and the relevant areas of those specifications. ....... - 11 - JTA Tt un~actt ontvl atmg~:r - - - - ·--._ EJB __.. -,,,\ .-· / .t·~ /(1 A.ppl icatit)ll Servt>r , ...... -/-:. JDBC, JIVlS ~·~'\ I I I I Transaction \ Resource ' t\ Ia nn~er .... i Application ( -----·Y 1\ lana~cr , ..... , 1/ r - - ~)\~~~-~~ - - -Tr----'~\_\ _ __ _ JT.-\ JTA U<..crT rans:lctton Inbound 1.\ Prmocol c;pec tfi( Trnnsncttun ~en 1 Ct' l mplementmton ( tlx example. JT')) J Commun~e-.HJon R<.':-ourcc l\lnt'k1gc 1 (( Rl\ 1 1 Figure 2-6 JT A Overview ... .XARe:.our•e Outbound tx Protocol ~pcctlic UserTransaction interface provides the application the ability to control the transaction boundaries programmatically. The application can obtain user transaction and use begin and commit method to demarcate the transactions. Transaction manager interface allows applicatiQn server to control transaction boundaries. Transaction Manager allows users to begin and conmlit transactions associated with a thread. ~ Transaction interface allows operations to be performed on the transaction associated with target object. This interface can be used to 1. Enlist the transactional resources in use by the application 2. Register for transaction synchronization callbacks 3. Commit or rollback the transaction XAResource Interface provides a java mapping of the industry standard XA interface based on the X/Open Specification. This interface defines the contracts between the Resource Manager and the Transaction Manager in a distributed transaction processing (DTP) environment. - 12 - JTA specification defines five players which are involved in a distributed transaction services. Each of these players contribute to the distributed transaction processing system by implementing different sets of transaction API and functionalities. 1. A transaction Manger provides the services and management functions required to support transaction demarcation, transactional resource management, synchroni7ation, and transaction context propagation. 2. A application server provides the infrastructure required to support the application run time environment which includes transaction state management. 3. A resource manager provides the application access to resources. 4. User application which uses the transaction provided by the application server. 5. A conununication resource managl!r supports transaction context propagati<3 / ...... access to the transaction service for incoming and outgoing requests. 2.3 WS-Transactions WS-Transactions defined in three specifications. WS-Coordination defines a coninmn framework to coordinate web services activities among different web services using different types of coordinating protocols. 2.3.1 WS-Coordination .., WS-Coordination[4] describes an extensible framework for providing protocols that coordinate the actions of distributed applications. Such coordination protocols are used to support a number of applications, including thosl! tJ'lat need to reach consistent agreement on the outcome of distributed activities. ~ The framework defined in this spl!cification enables an application service to create a context needed to propagate an activity to othl!r sl!rvici.!s and to register for coordination protocols. The framework enables usage of existing proprietary transaction processing systems while providing an inter operable mechanism to conununicate. The following diagram shows typical usage scenario of the WS-Coordination specification to coordinate the activities among different web services. - 13 - Figure 2-7 WS Coordination Framework 1. Appl sends a CreateCoordinationC'ontext for coordi nation type Q, getting back a Context Ca that contains the activity identifier A I, the coordination type Q and an Endpoint Reference to Coordinator A's Registration service Rsa. 2. Appl then sends an application message to App2 containing the Context Ca. 3. App2 prefers CoordinatorS, so it uses CreateCoordinationContext with Ca as an input to interpose CoordinatorS. CoordinatorS creatil,S its own CoordinationContext Cb that contains the same activity identifier and coordination type as Ca but with its own Registration service RSb. 4. App2 determines the coordination protocols supported by the coordination type Q and then Registers for a coordination protocol Y at CoordinatorS, exchanging Endpoint References for App2 and the protocol service Yb. This forms a logical connection -between these Endpoint Refcrcnct:s that the protocol Y can use. 5. This registration causes CoordinatorB to forward the registration onto CoordinatorA's Registration service RSa, exchanging Endpoint References for Yb and the protocol service Ya. This forms a logical connection between these Endpoint References that the protocol Y can usc. - 14 - 2.3.2 WS-Atomic transactions WS-Atomic transactions specification[5) defines an atomic transaction coordination type that can be used with the WS-Coordination speci11cation. This specification describes such coordination type protocols which can be used with the short lived atomic transactions. Completion This protocol is used to communicate between the initiator and the coordinator. Initiator starts the com£TUtmcnt processing by sending a corrunit message. After that coordinator starts the volatile 2PC and proceed to durable 2PC. Then the final result is send to the initiator. Two phase commit protocol Two phase commit protocol is used to perform the atomic transaction among the participators. This protocol ensures all the participators comes to a final decision. There are two variations of this protocol. 1. Volatile two phase commit Used with the participators who usc the volatile resources such as memory cache. 2. Durable two phase commit Use with the participators use the durable resources such as databases. 2.3.3 WS-BussinessActivity ... SirrUlar to WS-Atomic transactions specification this specification also defines coordination types and protocols to be used with WS~Coordination specification. These coordination types typically has to be used with the long running transactions. There are two coordination types and protocols has defined in this specification. Coordination types ~ There are two coordination types have defined with this specification called atomic outcome and mixed outcome. In the atomic outcome coordination type all the participators either end up with end state or compensated state while in the mixed outcome mode participators and be end up within any state . Coordination protocols There are two types of coordination protocols deftncd with this specification called BusinessAgreementWithParticipantCompletion and BusincssAgreementWithCoordinatorCompletion. The former protocol initiation starb b} the participant while for the latter it is started by the coordinator. - 15 - 2.4 Queued Transaction processing Figure 2-8 Queued T ransaction Processing Queued transaction processing is used to process transactions in a distributed environment asynchronously. This happens within three transaction boundaries. Firstly user application creates the request message and enqueues the request message to request queue within a transaction. After that s~.:rver dequeues the message, process it and enqueues the response to response queue within another transaction. Pinally user .qpplication dequeues the message from the response queue. 2.5 Different types of reliable web s~rvices 2.5.1 Using Message Oriented Middleware for Reliable W,_eb Services Messaging. Web services are applications that are described, published and accessed over the web using open XML standards. Different Message Oriented Middleware can be used with web services. Reliable communication is one of the most important aspects of any application. 'I here arc five ways that an web s~o:rvice can use MOM. 1. Messaging Middlcware Reliability Messaging middlcware is specialized software that accepts messages from sending processes and delivers them to receiving processes. The two principle styles for MOM is centraliLed and distributed. - 16 - 2. Aspects of reliability The main aspect of the reliability is to tolerate the network fai lures. MOM can tolerate the network failures by repeatedly sending the message until it is acknowledged by the receivers component. In addition to acknowledged delivery, ordered delivery is another aspect of reliable messaging. Further important aspect of reliability is the integration of a message delivery in a larger processing context. Therefore a MOM should be able to group a message with other messages and other process activities. 2.5.2 Three facets of Reliability I. Middleware endpoint to endpoint reliability A message once delivered from an application to the messaging middleware, is guaranteed to be available for consumption by the receiving process. 2. Application to m.iddleware reliability The middleware's messaging API, supports reliability properties such as message delivery guarantees, message persistence and transactional messaging. 3. Application to application reliability Sending and receiving applications engage in transactional business processes that rely on .., 2.5.3 Reliable messaging for web services This describes five different ways in which a web service can use the MOM for a reliable communication. 1. SOAP (with or without a reliability protocol like WS-ReliableMessagjng) is used with an unreliable transport (like ll ttp); reliability mechanisms are implemented on the application/SOAP messaging layer. 2. A Reliable transport like HTTPR is used for SOAP messaging 3. A Reliable, proprietary middleware system like IBM Websphere MQ is used for SOAP messaging. 4. A Reliable messaging standard Like JMS is used for SOAP messaging. A JMS implementation is required 5. A Reliable proprietary middlewarc ~y~tem like IBM Webpshere MQ is directly used independent of SOAP - 17- 2.5.4 Assessment Middleware endpoint to endpoint reliability The middleware endpoint mediation essentially refers to the idea that messages arc stored locally on the sender and receiver sides before and after they are being sent. 1. Option 1 does not provide thb reliability since HTTP is not reliable. HTJ'P docs not provide the status of the message on a connection failure. Therefore either SOAP messaging layer or application layer should provide the reliability. 2. SOAP over HTTPR provides the middleware endpoint to endpoint reliability. H"ITPR persists the messages at the sender and receiver sides. 3. SOAP over MQ also provides the middleware endpoint to endpoint reliability. The middleware endpoints are message queue managers provided by the messaging middleware product. Unlike in the liiTPR case here the message delivery pattern is asynchronous. 4. SOAP over JMS requires a JMS implementation. Depending on the JMS implementation it provides the reliability. 5. This option also supports the reliability since underline MOM is reliable. Adapters must be used at the each side to send and receive XML messages at each side. Application to Middleware rel iability .., Application to middleware reliability refers to the reliability features provided by the middleware's application to endpoint interface. This includes message delivery guarantees, fault tolerant invocation, the ability to atomically group messaging operations with other application actions. 1. When using SOAP over HTTP the reliability mechanisms ifnly be implemented as part of the application. Application can transactionally coordinate with the message store to guarantee the reliability. 2. For options 2 - 4 applications can't conununicate transactionally with MOM message store without using the MOM specific APls. 3. For last option application to middlcware reliability relates to the direct use of the underlying middleware's API and its reliability features. - 18 - Application to application reliability LIBRARY OfaR\lUWA, SRI lANKA MORATUWA Application to application reliability can be achieved in two ways. 1. In direct transaction processing, an agreement protocol is used to directly include one application's transaction processing as part of another application's transaction process. Here both applications interact with the same global transaction. 2. In Queued transaction processing two intermediate data stores can be used for sending and receiving messages. There arc three transactions involve in communication between two applications. Hrst transaction conunits the message to sending data store. Then the receiving application reads the message from there and commit back to the second storage. Finall> original sending application reads the response message from the second storage. .., -.... g r~24 - 19- ~ 2.6 Transactions and messaging Messaging can be integrated with the object transactions in different ways. This paperl l6 j pointed out such for patterns possibly used. 2.6.1 MQ Integrating Transactions MQ Integrating transactions do the reading messages from the queue, updating the distributed object and writing the response message back to the queue in the same transaction. But this transaction corresponds only a part of the global transaction. 01 local OutputQueue 03 Figure 2-9 MQ Integrating Transactions 2.6.2 Message delivery transactions ... Message delivery transactions integrates the message delivery model into distributed transactions. It allows clients to send the messages asynchronously while doing the other distributed object transactions. Message delivery failures can be observed and abort the transaction accordingly. If there arc messages already sent then compensation messages can be send. --- -20- ~ set of message recopoents tx cloent 11 tx server objects 0 0000 01 02 03 04 OS I I I I I I I I 1~1 I-- ~ - -...-l I I m1 I I I I I t I I t4- --r----;r4 ~ ,: m2 :-----t-_J I I I I I I I . I I Message Delivery TransactiOn tx client r-1 0 01 I m1 r::---- m1 m sot of message recopoerts tx server objects 0 A c,·" o·.·" v · _.t 02 03 04 0~ Standard OTS Transaction Figure 2-10 Message Delivery Transactions 2.6.3 Message processing transactions Message processing transactions integrates the message processing model to the distributed object transactions. This enables the asynchronous request processing between transactional distributed objects. The transaction is not committed until tQ,e response is received. -.... - 21 - ' . tx client tx server objects II 01 02 03 04 I I I I I I I I I m1 I I I I I I I I I I .. I I~ I I I I I I I I I ... I I I~ I I I I I I I I I I I I I I I I m1 result I I I I t I I I I Message Processing Transaction Figure 2-11 Message Processing Transactions 2.6.4 Full messaging transactions Full messaging transactions refers to the system which has both the message delivery transactions as well as the message processing transactions. 2. 7 Middleware mediated transactions There are two widely used transaction processing systems called. Object oriented transactions and message oriented transactions. Object oricntl.!d Lransactions happe.Q.s in a synchronous blocking way . Further object oriented transactions uses 2PC protocol to achieve the atomicity of the transactions. In message oriented transactions only enqueuing and dequeuing messages are done transactionally. Therefore message oriented Lransac.tions does not preserve the atomicity. Middleware mediation transactions[ 15] suggest a way to provide the end to end transactions while keeping the advantages of the message mediation transactions. It provides some end to end checking at the middleware layer. -22- 1\ .. 2. 7.1 D-sphere D-sphere[ 17] is one of the implementations of the mlddlcware mediation transactions. D- sphere provides the end to end reliability by providing an rnlddleware to the user which manages the end to end transactions. rollowing figures show how it works with and without D-sphere. Figure 2-12 Applicati on Without D-sphere Figure 2-13 Application With D-sphere -23- ' .. D-sphere architecture supports above requirements by providing a middleware layer to users which manage transactions internally. Message Sender, Transactional Client Transactional Resource Message Receiver Figure 2-14 D-sphere Architecture -~ -24- '\ ~ Chapter 3 Methodology 3.1 Previous Solution --_-_-~:.> Persistence Storage T1 commit ' ' Client Program AM Client 12 COiPitill .., ~ RMS RMD Figure 3-1 Previous Solution -25- T41commlt I Se!Ver Pr:ram [____:F :- ~fl""ll ' ... Fig 3-1 shows the probable way of solving the above problem with the original WS02 Mercury implementation. Pirst the client program has to read its' persistence storage within a transaction, build the message in memory and commit the transaction. Then it gives the message to RMClient which again has to start a transaction and commit the message to RMS. Once the message stored at RMS it reliably transferred to the RMD by the WS-RM protocol. At the server side RMReceiver receives the message within a transaction from the RMD, build the message in memory and commit the transaction. Hnally server program commits the message to server persistence storage within a trano;action. This model reliably operates with the pre~en<.:e of network failures since WS-RM protocol can handle it using retransmissions and acknowledgement~. But if the client node fails after first transaction being committed to the client storage and before RMClient commit it to the RMS then the message can be lost. Same failu re can occur at the server side as well. On the other hand if the first transaction commits after the second transaction there can be duplicated messages. 3.2 Proposed Solution ( :-_- - ·~-~.~ Persistence Storage _, Client Program Commit :_ __ --r-----' ,, /,c;~~:;;1 T4 1 J I --- - -- , I ,....-- ---"--, ... SeNer PrograM AM ReceNer ..._... Figure 3-2 Proposed Solution . rig 3-2 shows the proposed solution with the distributed transaction support to address the node failure scenarios. Unlike in the previous case now client uses a distributed transaction to update both client storage and RMS storage. Client only corrunits to the transaction manger and if client node fails when this conunit happens, the recovery process of the 2PC ensures the atomicity of the transaction. Similar process happens at the RMReceiver as well. At the server side server program transaction has to participate the distributed transaction started by the RMReceiver and again node failure handle by the recovery process of the 2PC protocol. . 26. ' ... 3.3 Alternative Solutions 3.3.1 Integrate WS-RM protocol with the client storage and server storage by taking them as RMS and RMD c-------~ ------ -- ------ I Persistence Storage RMS , _____ - --· -- - --___ / .-1 WSRM ---.. _ '-...J --. _ .... ;"" ~--~-~-··---- \ •'-. _______ ___ , __ .. / Persistence Storage RMD '-.. / ... ___ __ . __ - --· ... ----· Figure 3-3 Integrate Persistence Storage with RM Storages Fig 3-3 shows a possible solution for this problem in a spccil1c way to a given problem. For this solution distributed transactions arc not required since RM protocol tightly integrated to the client and server storages. But the advantages of this type of approach is less since it does not try to solve the problem in a gcncrali7cd way. Tl'le focus of this project is to come up with a middleware which provides the WS-RM functionality to any application. ~ -27- ' .. 3.3.2 Use the same transaction to update both application level storages and WS-RM storages , -- - Persistence Storage T1 T2 1 Client Program Server Program RM Client RM Recetver ... 2 Figure 3-4 Usi ng the same Transaction :--;~;stste~~e:] Storage "' lot ·- Fig 3-4 shows a special case of using same transaction to update both application level storages and the WS-RM storages. In order to use this scenario WS-RM storages should be there with the same application storages. Therefore this may not be useful when integrating message receive functionality with different storages and different application servers. Although this functionality can be provided with the proposed storage API based Mercury implementation, this project only focus on the distributed transaction based solution. -..... -28- ' . 3.4 Solution Architecture 3.4.1 State machine model WS02 Mercury is based on a state machine model where the state is kept in objects called RMSScquence, RMDSequence and InvokerBuffer. This state machine model is based on the fact that various external events change the state of each object. Further a set of workers namely RMSSequenceWorker, RMDSequenceWorker and InvokerBufferWorker performs set of actions based on the state of the object. It docs not assume any order of the events. If a WS- RM message get lost while transmining through the network, only the event which would have occurred get lost while system <;tate remains same. Therefore system operates in the previous state which causes the retransmission of lost message hence achieving reliability. RMSSequence -~ _,, /' ---- ................. I I CHH ~; RMSSequence :.;~ AMR \.__ _,/ " ~- ~ ~ ....J ~ Figure 3-5 RMSSeq, Fig 3-5 shows the possible events that would change , tate. These events namely create sequence response rcceivt! (CRR), last 111 1.::> ~R), application mt!ssage receive (AMR) and receive acknowledgement for at. · V '?~ her cause by o"~ the application client or network message reception. RMSSet.. 't ).if' '<; on four factors called sequence started (SS), message in the buffer (MIB), 1. r{f. 1.c-/'~ 'LMR) and terminate message send (TMS). l hcsc four factors create possib1 ~ 1nly seven states are valid as shown in the Fig 3-6. -'G (J . 29- MIB (' 0 Figure 3-6 RNa-. f-ig 3-7 shows the complete stale tran~ition diagram .. which change those states. -30- .., 1:> '--{ 1io ()' . and events . 'l~J# £ .'1~ /hi) /t- ... "' . ------··-1 LMR 11,.- .... \.. 01 10 "·· ) 1100 ") - ~~. '\ ...... _ - /--- """ [ 1011 .I "......_ .. ________ / A t J l ~ -·10 ,_, r - .. '-- 10 1 G~~~;~ ) - j Figure 3-7 RMSSequence State Machine IYJ 'o o'"' -~ ~../ ;h· (\ i,e f)VJ h F::> .. (] )...j .... i . ,..-~ .... -- --- ------ ................ /'" / ' tr -.... AMR£SA'J ~ : RMDSequence : 41111 LMR(SG) \ I '·~----- ---< I • i f/!. ~ ~~ 1 "-' I \, Figure 3-8 R:viOSequence Events - 31 - ~ r rig 3-8 shows the possible events that would change the RMDSequence state. These events namely application message receive completing the sequence (/\MR(SC)), application message receive without completing the sequl.!nce (AMR(SNC)), last message receive completing the sequence (LMR(SC')), last message rl.!ceive without completing the sequence (LMR(SNC')) and term.inate message recci ve (TMR) would cause by the message rl.!cl.!ivc from the network. RMDSequl.!nce state depends on four factors called ftrst message receive (I MR), last message receive (LMR). every message has received (EMR) and terminate message receive (TMR). These four factors forms possible sixteen states but only five states are valid as shown in the Fig 3-9. " ...... Figure 3-9 RMDSequence States Fig 3-10 shows the complete state transition diagram with the set of valid states and events which change those states. - 32- ~ . /-~--- ------ LMR(SNC) f .---~ . 0000 "'\ / -- AMR{$NC) "· / ~· 1100 \ -H -- - -~ AMR(SNC) / __ I).R(S1~) LMR~SNC) "- j .... 1000 ' " / AMRffiB LMB!SC) /---- ---- - . 1111 ""' TN1R "-._,_ I - --I - --- - ·- ' /' ' / 1110 / ,_ '"·-. ,-- -- . Figure 3-10 RMDSequence State Machine .... ~, - 33 - I nvokerB uffer :' I ./ )./ , ~ ~------ -- - - - ----............. ,/ "'-. I \ LM n IIIJJ : 1 nvokerBuffer :, ·"·- / ··-·------ - - -\ \ \ ((l ~ \ \ Figure 3-11 [ nvokerBuffer li:vents \ ·, ' ·, Fig 3-11 shows the possible events that would change the lnvokerBuffer state. These events namely Last message receive (LMR), application message receive (AMR) and send all available messages to application (SAM) can cause by the messages receive through the . network or the invoker which sends the message to application layer. InvokerBuffer state depends on three factors namely messages in the buffer (MID), last message received (LMR) and every message send (EMS). These three factors forms possible eight states but only four slates are valid as shown in the Pig 3- 12. - 34- :> Figur·e 3-12 lnvokerBuffer States Fig 3-13 shows the complete state transition diagram with the set of possible states and events which change tho:.c states. o _____ _ ------" w 100 ) ' .--""' -... __ _ SAM ANtlR // i _ ___..-- _, - -- ----. \.. .... ______ _ 000 ,.) __ ....... ... --- -~ /'/' ,_, .::.:.:.:,~ I Mn llllr { 110 \ '""-- _/ .... __ ... _- __ .,.. , ... . /- -----:: .. -- - --.. '\ I. 011 : . J "'-.._ _/ ..... .. __ . -· - ....... -- Figure 3-13 lnvokerBuffer State Machine - 35- 3.4.2 Run time Architecture WS02 Mercury is a WS-RM implementation written on top of Apache Axis2. Apache Axis2 provides a set of extension points calkd handlers. These handlers fonns the Axis2 Engine execution chain and can be deployed as modules. Therefore WS02 Mercury in other words is an Axis2 module. A typical Axis2 message send starts with the application client which calls the service client. Then the message is passed through the Axis2 Engine handlers and finally is sent to the network using the transport sender. At the server side message is received by the transport receiver. After that it invokes the Axis2 Engine where message is passed through a set of handlers and finall) receives at the message receiver which invokes the application at the server side. WS02 Mercury consists mainly of two handlers called MercuryOutllandler and MercurylnHandler which are used at out and in t1ows respectively, and a set of workers called RMSSequenceWorker, RMDSequenceWorker and lnvokerBufferWorker which read the respective storages and perform the appropriatl..! action based on the state. Next set of scenarios shows the runtime architecture necessarily with the persistence storage which uses transactions to read/update storage. Although in memory storage does not support transactions it is also has the same runtime arc hi lecture. In Only Messages ...-----.._ /___/1 ~ . ' ]ij1 __ MeiCUJYOUtHandl~. l(ij !.2 ~ ~ I < ~> Mercurylnhandler '};j ~ r_l ._, ·I RMD I ltnvoker Bufferl l I :.,........,......,.~-:-:-:::---~""' .._ __ lr- - I v MercuryOutHandler ~ ~ / { MercurylnHandler '\.._., ~< < Figure 3-14 In Only Messages Runtime :s ~ (/) c Q (ij 0 a < Fig 3-14 shows the runtime execution for an in only message scenario. Mercury receives the message from the Application client at the MercuryOutllandler, creates a sequence if it already not there and stored the message in the RMS. RMSSequenceWorker picks this message and invokes the MessagcWorkcr. MessagcWorker sends this message through the rest of the handlers and finally the message being sent to the network through transport - 36- sender. At the server side transport receiver gets this message and invokes the Axis2 Engine. Mercury receives the message at the Mercurylnllandler which updates the RMD and stores the message in the In vokcrBuffer. Then InvokerHul'fcrWorker picks this message from the lnvokerBuffer and invokes the rest of the handlers so that ultimately message receives at the application at the server side. Reliability of the WS-RM protocol is achieved by retransmissions and acknowledgments as in any other reliable protocol. A separate worker called RMDSequenceWorker is used to send acknowledgments back to the client side. Upon receiving an acknowledgment client side updates its' state as message has succes5full} send. As shown in the figure, RMDSequenceWorker reads the RMD state and sends an acknowledgment message using MessageWorker which generally is used to send any message. At the client side Mt:rcury picks this message using the Mcrcurylnllandlt!r and it updates the Rl\1S. Although it is not shown in tht! diagram RMSSequenceWorker sends the create sequence message when establishing the sequl!ncc and sends the terminate sequence message to terminate the sequence. Similarly RMOScquenccWorker sends the create sequence responsl! message according to the state of the RMD. llow this architecture supports usl!r transactions? 1\.s it is shown in the Fig 3-14 it does not keep anything in memory. Any ewnt read<; and updates the storages using a transaction which is at the serializable isolation level. Therefore any update is not visible to other threads or workers until the transaction commits successfully. .., -.... - 37 - In Out Messages -~ 0 c: 0 -~ .s-1 0. 0. et..luen::,E, L---------~----~ I nMemocy RMSSequenceManager RMSSequenceManagerMap I k ] 1... t ~-~ f )~>.aqJHr •r p I nMemory AM DSequence tiAanager RMDSequenceManagerMap [ t( II llv--~.~~suf:eJ l lnMemorylnvokerBufferManager lnv ok. erB u ff e rM ap c 0 t5 C'Cl ~ .... .... c 0 E Q) ~ c I nMemory StorageManager .., Figure 3-18 lnMemory Implementation Fig 3-18 shows the in memory storage design for Mercury. In memory storage keeps a separate sequence manager object for each sequence and it keeps these objects in three hash . maps called iSKRMSequenceManagerMap. sequenceiDRMDSequenceMangerMap, sequenceiDinvokerBufferMap. A sequence manager object has a lock amJ another object to keep the details for the sequence manager object. Any sequence manager object can be retrieved from hash tables giving the key as the parameter. But before accessing the sequence manager object the corresponding transaction has to acquire the lock for that object. Synchronization For proper state machine cxecution only om.: thread can update the sequence at a given time. Hencc it is required to synchroni;.c the state machine or sequence manager objects. Two phase locking is used to synchronize the sequence manager objects when; a transaction acquires the locks when accessing objccts and releases them upon a conunit or a rollback. A transaction always acquires sequence managers in the order of RMDSequenccManager, lnvokerBufferManager and RMSSequenceManagcr to avoid deadlocks. - 41 - Persistence Implementation Com1ectionManager .... .._ ~ ~ CD ~ ~ c c c: ~ cc ('\j ~ ~ ~ oQ) ~ e g d> c ~ Q> 92 .:J ::::; ...J co g @" \.. , Q) (/) (/) ~ U) 0 ~ ~ E: a: a: Q} (J) Q,) 0 u 0 c:: as c: Q) <1.1 - - .... (/) ~ I/' ·- t:! cr. ~ 1.-. Q) ~ ~ Cl. a_ a_ Maooe Pers i stenceStorageM anager Figure 3-19 Persistence Implementation Fig 3-19 shows the important components of the persistence storage. It has a connection manager which is used to create either normal database connections or xaConnections to ... database. There are two types of transactions called JDBCTransactions and JTAThransaction. A JDBCTransaction contains a normal database connection where as a JTATransaction contains an xaConnections. Once a thread requests a transaction persistence storage access the connection manager and creates the requested type transaction. Unlike in the in memory . model, persistence storage manager keeps one set of sequence manager objects for all sequences. All sequence managers usc a set of helper classes called table-n1appers to create sql queries for dto objects and to create dto objects from result set objects. Sequence manager objects gets the connection object to usc from the thread local. Synchronization Again for proper state machine execution only one thread hence a transaction can update the sequence state. This can be achieved by setting the isolation level of the transactions to TRANSACTION_SERIALIZABLI: . This i!-.olation level can leads to deadlocks. First there can be deadlocks due to different order of table access. This has been solved by always accessing the RMDSequenceManager related tables fust, then InvokerBufferManagcr related tables and finally RMSSequenceManager related tables. One transaction may not acquire all sequence manager objects but if it requires it has to access in the given order. -42- At TRANSACTION_SERIALIZAHLE isolation kvd a transaction has to acquire a writer lock (an exclusive lock) to update a tabh.:. i\ writer lock can only be acquired after all the other transactions release the reader lock at a conunit or a rollback. This gives another type of dead lock if two transactions try to read and update a table concurrently. Since both can not release the reader lock until write. This problem can only be solved by acquiring an exclusive lock at a read. An exclusive lock can be acquired at the read time by using 'select for update' statement. Mercury persistence storage usel> above two techniques to achieve synchroni.lation avoiding deadlocks. It has been tested with an embedded Derby database with row level locking . .., -.... -43- Data base design ~--- . RlwS_aQUEMT 1 ID_C long SEQUENCE .. ..ID_C stnng SEQUENCE_OFFER._C strmg ·ST.ATLC ont RMS.AX IS UNFO. T ·ID_C long · SERVICE_NAME....C · strong ·CURRENT.JiANDLER..JNDE;<._C. lnt -CURRENT...PrlASE..JNDEX.....C ont - IS_SERVER._SIDE.....C ont ·ACKS_TO_C stn ng -lA~T..J.IESS..<>.CE...NLT•IBER.....C: long STAPT_TII~E_C long -END_TIME_C long f-------......,..-f·SOAP ...NAMESPACE...URI_C stnng -ADORESSINC.J'IAMESPACE_URI .. C stnng • TRAI~SPOIIT _IN_NAME....C strong -TRANSPORT _OUT NAM E....C · lnt ·lAST .. ACCESSED_llME.....C long ·RETRANSMIT_TIME....C long TIMEOUT_TIME_C long ·IUNOI'J)MOUS_C 1r11 ·lAST .. CREATE_SEQUEl'ICf....RESPO'HE... ~lfSSA CE...Sfll. 1_ TIME .. C • loog ·lAST...AC'"Y.NCNv'UDCI~ENT .SENT_TIME...C long -C~oE_SEQIJENCUIESSACE....JD_C str:ng · "'ESSACE_NUM8ER.....C !ong ·fND..J'CWIT...ADDRESS_C: StMg .. N er _c string ·N'IXIMU~LRETRANSMIO_TIME_C lon; ·EXP01ENTI>L2.AO:_O~F ....J. ont ·CRE.OTE_SEQUENC!::...JlETIIANSM IT _(OU" T _C ont Y.ELC stnng J • TO...ADDRESS_C srrong R~I S_~IESSAGL T ·ID_C · long · M ESSACE...NUM BER.C. long - I S_lAST _MESSACE.....C int ·SOAP...fNVELOPE_C st ring · IS....SUJD. C ont -.0.-JS_fl!ESSACE.....ID_C: string ·RElA TES_TO J•l ESSACEJ D_C strong ·REri...Y_TO_C stnng ·CALL.8Acr~_CLASS..)MME.....C · Stnng - .ACTION_C stnng ·OPEPATIOI'J...ACTION_c · string ·SERVI CE_NA.:.I E.....C strong -PMS3EQUENCE..JO_C long -lAST..J.IESSACE...SENT_7 ,ME...C : long ·IIETRAIJSMIT_COUNT_C: tong ·FLOW_( • lnt •OI'tAA f'ON_•IAME...C stnng ·OPEPATo OI'UIAI~E...SP.ACE_C st i ng INVOKEfCBUff£1\... T -IO_C long $TATE_C ont lAST.J.!ESSACE....C long ·lAST.J.!ESSACE.....TO...APPUCATION_C ·LAST...ACCESS_TIMf.....C long SEQUEN CE...IO_C strong -ACKLTO_C stn ng TIME_OUT_TIM E....C long · IS...ANONYMOUS_C int -RMD_SEQU ENCE....ID_C · lOng tong r IWOk£R_8U .. r r .r1 M£SSA~f- h\1.' 8 -scAr ...f~••aor ·15_SENC•_C ont S£RI/1 Cf....NA~IL ·ACfiON .. C son 1 ,-MESSACE_IO_C ; nnng t--:- ·TO_C SHtrog l -REPLY_TO ... C· Strong -~10'¥\'_C. lnt · I NVOt: £1\...BUH ERjO _C long ·OPfPA 1'1 O'II_N AI.IE....C strong ·OP~PATI O"' N-.ME...SPACE....C · R~lATES IO_C str1 nq ·IS_loi,ST_ME'SSACE....C lnt L_ strtng Figure 3-20 Database Design -IS_USE....SEPERATE...LSHJER.....C l ru -Tlt•IE.....O!JT J II_MILISECOIJDS _C 10'10 -RMS3EQUENCE..JD_C long 1 BUFFER.REO:MD _NUMOCR_T -ro_c long -t~UMBEP_C long -I NTEPNAL_BU~FER._ID_C : long hg 3-20 shows the underline database design f'or persistence storage. It contains a separate set of tables for each sequence manager in order to avoid deadlocks by accessing them in a clear order always. - 44 - 3.4.4 Other issues and solutions Starting the terminated sequences at the client side Client node can fail while sending a :-.e<.juence of messages. Therefore for application client there is no way to know whether it properly tenninated the sequence or not if the client node fails just after sending the all the mes!>ages (this case happens only when there is no explicit last message but application client semh a terminate message to Mercury). As a solution to this problem Mercury sends an explicit terminate signal for all the sequences that has not been terminated. If the application client has not send all the messages then it can start a new sequence and send the remaining messages. For in out client scenarios once the client node fails, addressing dispatch information stored at configuration context also get lost. And also there is no axis2 service to receive the messages as well. In order to solve the above two issues Mercury uses a deployment life cycle listener to terminate the non terminated sequences, to add the axis2 service and to register dispatch information in order to dispatch sequences. Distributed transaction recovery Two phase conunit (2PC) protocol guarantee!> the atomicity of a global transaction even when node fails. 2PC protocol has a recovery phase to recover from the node failures if the coordinator or any other node fails within the commit prtase. Therefore in order to guarantees the atomicity of the global transaction the XA implementations should properly support the recovery phase. However it seems some database X/\ drivers have problems with the recovery phase. ..,.., -45- Chapter 4 Use case scenarios Mercury can be used to invoke services using both in only and in out message exchange ... patterns (MEP). Although this research work focus on user level transaction support it is designed in a way that it can be used with simple inmemory implementations as well. Following use case scenarios arc used to demonstrate how to use Mercury with different storage implementation types. WS-RM 1.0 ck~cribes an addressing based dual channel mode . to send and receive messages. Therefore for all use case scenarios given here uses addressing based dual channel mode. There i~ another specification describes an piggyback message based system which uses http back channel to rcct.:i ve messages. Mercury supports the latter kind of invocations only with the in memory implementations. - 46- 4.1 lnMemory Client Program Server Program Figure 4-1 lnMemory Invocation InMemory invocation is the most simple way of using Mercury. It does not requires to do anything other than the engaging the Mercury module as in any other module engagement. Mercury uses in memory implenH.:ntalion as the default storage. Messages can be send through a tcp monitor and start and stop channels in order to prove the reliability with the presence of network failures. 4.1.11n Only invocation Conf1gurat1 onContext con t igurat ionContett - Con f1gurat1onConteJtFact ory. createConflguratlonContextFroMF 11eSyste•( AXI52..}l'EPOSlTORY_tOCA110\, .lX1S2_Cllf.\T_COo\Fl f(_JHE) ; Serv1ceCl1 ent serv1ceCl1ent = ne~ Service< ,en-(ront1 ~~rat l onConteJt , null ) ; serv1 ceCl 1 ent . secTargetEPR(new Enapo1 ntRefer ence( "ht tp: ; 1 local host :8\l88i a'O s2; senrices/ ln'1Prory TnSPr .. H'e")) ; serv1 ceC 1 ent. getOpn ors() . setAct l on("um: ln~cr-u!yln0pPrat1on") ; serv1 ceCl1 en): . engageMoaul e("::l!r·cun "); servlceC11ent. get0ptiors() .setUseSeparatellstene trt~) ; servl ceCl 1 em: . getOpn ons() . set Property(Herw ry(ll en': Constants.I~TERY~LJD, "1-e)•l"); for (int 1 c 1; 1 < 20; 1++) { } serv1 ceCl 1 ent. fl reA.ndForgPt(getTestONE1e•ert(1 )) ; try { Th read.sleep(1000) ; } catch ( InterruptedExcept1on e) ( } • ~ercuryC11 ent ~ercuryC l1 ent- new Me rcu ryC11ent(s~rv1ceC 1l ent) : •ercuryC11 ent .terll1nateSequencP( "I\P)1") ; Figure 4-2 In Only Client -..... It uses a service client object to invoke the service. First it creates a configuration context pointing to an repository location. Then it sets the endpoint reference and soap action associated with the operation as in any J\xis2 client invocation. After that it engage Mercury module in order to make this connt:ction reliable. Here InMemorylnService should also have engaged the Mercury module. lr set!> the u~eSepcrateListner parameter to make this a dual channel invocation. Mercury uses the internal key parameter to distinguish messages belonging to different sequences. After setting all the necessary parameters it ~ends 20 messages and finally terminate the sequence by invoking the temunate sequence method. -47- 4.1.2 In Out Invocation Conflgurati onContext conflguratlonContext • Cont i guran onContextFanory. crear:eConf1 guratl onCom:extFro.wF1 JeSyste•( AXlS2_REPOSl70RY_LOCAT10N, 4XJS2 CLllNT CO\~JC_flLE); ServiceCll ent: serv1ceCl1ent • new Serv1ceC'ien1:(conf1guratlonContext, null ); serv1 ceCll ent: . setTargetEPR(new EM POl ntR~ference("http:ftloc-a I hOst :ll038/ol•ls2; <;Pr'vlces; lnMer.or) I nOut'\Prv1 ce")); serv1 ceCl i ent: . getOptl ons (). se-tAct 1 on("UI n: l11'1~nur )' IIIOUtllrwr·dtwn"); se r v1ceCl ient.get0Ptlons().setUseSeparatPl1~t~nP r ( true); serv1 ceCl1 ent. get0Pt1 on;() . setPrope rtY(Mercury, 11 entCn~ta.nts. SEQUEI!tC[ OfFER. Constants. VAWE_TRl!£ ); serv1ceCl1ent.engageNodule("'lerc.ury"); for ( int 1 = 1; 1 < 20; 1++) { } sendAsyncl10rnousNessage(senl1ceC11Pn't, , , "~evl" ) ; try { Threa.il. sleeo(lOOO); } catch ( Inter ruptedExcept1on e) { } Nercury(l1 ent •ercury(lient a neo.. Mereu r(l i ent(ser~l ceCl 1 ent): •ercury(11 ent. t:er•1 nateSequence("lie) 1'') ; Figure 4-3 InMemory In Out Client WS-RM supports in out invocations hy establishing two RM sequences for in and out message sequences. For incoming sequence an sequence identifier can be offered when sending the createSequence message for out sequence. In this sample client it sets the sequence offer to ask Mercury to send a sequence offer with the createSeqence message. Unlike in the in only scenario it docs an asynchronous in out invocation usi ng the sendAsynchronousMessage method. pr111ate \Oid sendAsynchornous~eslsCallback() { }; public vo1d onMessage(MessageContert •sgContert) { .., Systea . out. pnr.t ln("1;ot the lt!'<:<;agp -> " + ISgContert. getEn~elope(). getBody(). getFHstEl e1ent( } I?" LIBRARY public vo1d onFau l t (MessageCor,tert ••g<,onte>.1:) { \~ -, Systea. out. pnntl n('"Co t the fault , " + •sgCo~tert . getEnvelope(). getBody(). getFault().getOetall()); * } . . . ~~~LA~ b. public vo1d onErro r (Except1on e) { e.pn ntStackTrace(); publi c void onCo•plete() -.... servi ceCt 1 ent. sendReceweNonBl otk1 ng(getTestOI!E 1 e~ent (key + " " + ; + " "), aw1 sCallback); Figure 4-4 SendAsynchornousMessage Method It sets an Axis Call back object to receive the messages and do an asynchronous invocation so that it can send the out messages without waiting for the incoming sequence. - 48 - I 4.1.3 Fault Handling public class InMe11oryFaultMessageRece1ver P\ll'nds Abst ractin0utMessageRece1ver { publ ic void 1nvokeBus1nessLog1c(Messagecontext 1nMessage, Messagecontext outMessage) thro"s AxlsFault ( Systel'll. out. pn ntln(""Sencllng lh•• f,1u ll Ml':'.~~o2. org/tecpt" , "nsl "); Q!jEJ e11ent oaEl eaent c o11Factory. c r~ati'JNEl ene' ,t , "lr•stErrurEle•ent•·, oaiJa•espace); 011Ele3ent.setText("Ut>pl) • + te•t); retum oaEle11ert; Figure 4-5 Fault Message Receiver Fault scenarios has been implem~ntcd by ustng a message receiver which always sends an AxisFault. Mercury sends application ll!vel exceptions reliably by using the response message sequence. When an application exception receives at the InvokerWorker it rollbacks the transaction used to invoke the business logic and starts a new transaction to send the fault message. 4.2 Persistence APPLICATiON CLIENT r -~- --:-.::> ·· - -· - - - ---~1 comm1t " RM Client ... Figure 4-6 Using Persistence Storage c:.-_-- - :-:.:-> APPLIC'A TION SERVER ___ , RM Rece1ver -..... Fig 4-6 shows a sample persistence storage u~age to transfer set of messages stored at the APPLICATTON_CL IENT storage to APPLICA'fJON_SERVER storage. This configuration works even with the presence of node failures (i.e. start and stop client or server) but there can be message losses if the node fails at the stages where message only resides in memor). - -+9 - In order to use a persistence storage it has to be configured at the axis2.xml file both at the server and client side. We usc Apache Derby as the underlying database. <:ll.Y.li'Pter MJJll'•'"~tor~M,lf'\.-.qf"I-ClM.~">OtU ,W)o> . • f"f """ \IUJ i iUe hrl1,fll't 'l\lt"fK'1" PM \1'\lenU!~ hlrd(JC"tPrlflager<(pdl"aJCtL-'f > aoe1u1econno n.are ... ~C'rr.ur)'"> <.p¥cneter nane-'·(21:1. COI"'IW'c t1o:•o;tr1oa .. ,. Jdbc: cttrby: thOf'e/ aru l &'•st;rw ·oJtt Or'Q.dP.Xht•.dt....-tr) Jcllc f"•bl•!ll:lrcJo, '"t•t ·<,TJ.r•~c..,.~ -c;/p-lf'"MPttr"> Figure 4-7 Persistence Storage Configuration 4.2.1 In Only Invocation :.t Conflouran cnCcntPr~ac r• ''Y· -,,..rP(~~'Qur•! 11)n(lJr.t~•rF ,,. ,, IP";Ystt.w( AJ1152....RCP011CIIf) .LOCATIO>., 4tiS1_U ILV7.( G.V"JILC): Serv·ceClJent servtceClie·1: • new Ser .. Htr\tCe"):; serv· cent ent. getOonors~). Sel:Acr1 on("urn; rerststf'•~rr4tlon" ) ; serv• ceC11 Pnt . e,ga)eNodul e("'leretw-) • j; strv·cec I 1 ent. get0)t1 ons:) . set:UseSeparauL IHent'( lru• ) ; serv1ce(l1 ent .oetO)ttons:) . setProoerwCNercury(lt e·ot :or stan~ ./MEH\H..AEl', "~eYl") ; t() ) { oessaQeNJabers . add(resul tSet. get long (" JO C")); } resu1 t SH. close() ; for ( long aessageio : oessageNuobtrs) { Str ing QJerVStnno- "sel~ct • fro" TE~T \1).0 DATA 1 •her~ 10 C•" • oessageiC: r~sultS~t • snte•ent.~xccuteQucry(queryStnrg); 1 r (resultSet. 1ex: ()' { Stnng aessoge = resu1tSet .gotStrtng(''IC~~Hl.( "); Stnng JpdateStnng • "urd>t., ILST <;r!>().04U_1 vt 1:.. ~L,I•.t• l ~~·~r~ IC>_t •" + ressageiD; state1e1t. erec~ceUpdate(uodateStrt no); Sy~e•.out. pnrtln("'S('n•hnq l"t'S~t ·• • ·~•~agOP); tr)• { Thread. sJ2op:lOOO); "'> } catdl (InterruptedExceptton e) { } servlCeClt•rt. t1 ·eAncForgtt{getT.,;t0~£1et~nt (re~•aot)); statuent. close(); comecnon.cl ose(); ~trcury(llent •ercury(hent • ,_ Nercury(htnt{Str.tct(11tnt) ; aercuryC11ent. tero'"aaSequencec·~~} 1"); Figure 4-8 Persistence In Only Client -..... As shown in the Fig 4-8 first it reads available message numbers to be send within a separate database connection. Then for each and every message it gets .the message from the database record and updates message as send. Since we have not set the auto conunit to false, executeUpdate statement conunit the transaction automatically. After that as in the in memory case it sends the message. If the client node fails while it sleeps then this message get lost at the client side. -50- protected vo1d invokeBus1nessLog1c(MessageCont~~t ~essageContext) throws Ax 1sFaul t { Str1 ng ~essage = ~essageContext.gatEnve l ope().getBody().getF irstElement() .getText(); Syst em.oot. pri ntln("Go t the 'ioap l t"•<>.JCJI' ••> "+ ~essage) ; .:! connect1on connect1on = oet DatabaseConnect1on() ; try { Statement state~ent = connectlon.c reateState~ent(); Stn ng 1 nsertQuery • "ln'>l't't 111to 11·.\l....RHI: IV[J!,l,TA_T C':ES'>,l,GE_C) 11alues C'" + ~essage + .. , ) " ; state~ent.execute(lnsertQuery, State~enr. RFTLRN_CfhERAT[D~EYS); state~ent .close() ; connect1on . close() ; } catch (SQLExcept1on e) e.prlntStackTrace(); } Figure 4-9 Persistence In Only MessageReceiver At the message receiver it saves the message to APPLICATION_SERVER database within a transaction. Before sending this message lnvokcrWorker reads the message from the RMD and starts a transaction but commit it after invoking the business logic. Therefore duplicate message can result if server node fail before invoker Worker conunits the transaction. 4.2.2 In Out Invocation and Fault Handling Both In Out and Fault handling clients areal mo~t equal to the in memory cases but they read the messages from the database and update ocfore sending them as given in the in only case. The only difference is that persistence invocations uses concrete AxisCallback class to receive the messages. This is useful when a client node restart while transmitting a sequence of .., messages. Then Mercury can create an instance of callback and register it at the Axisoperation callback receiver. -.... - 51 - publi c void onMessage(MessageContext •sgContP); } Connection connect1on - getOatabaseConnectlon() ; try { Stateaent statement • connect1on.createStateeent(); Stn ng 1nsertQuery • ''inser t Hlto lf '>r H~Cf IV[.J>ATA T ('IESSACL() values ('" + •essage + "' )"; statement.execute(lnsertQuery, State~ent. HCTUH~ CE~RATED ~~YS); stateeent.close(); connectlon.close() ; ) catch (SQLException e) e. pnnt Stacf.Trace(); pu!Jllc void onFault(MessageCcnten •sgCor.text) { Stnng Fessage : esgCor.text. getEnvel o~~O. g.,tBody(). Q!?tFiiult() . get0eta11 () . getFl rstEl ~•ent(). geHert(); Systu.out. pnntln("G1 E ll' .. l'nt ••> • • •essage): Connectlon connectlon • getDatabaseCon~ectlon(); try { Stateaent statenent • connectlon.createSta~eeent(); Stnng insertQuery ~ "insert 1nto H'iT FlfCri~E_OAT~ T (~SAC.(_() ~alues ('" + •essage + "')": state•ent. execute (lnsertQuery, StatP•ent. HHIIR'r_~.~R1TED /IEY:5); stateeent . close(); connectl on.close(); } catch (SQLExceptl on e) e.prlntStackTrace(); Figure 4-10 Per·sistence Call back Handler Methods 4.3 JTA ,--- - ---~ ~P-PLICA. TIO~l CliENT J --·-- - - - -- "" T1 I 01ent Program --·-. Comm - ...---.......J TM AM Ghent .., c:::.--_-_----=-> APPLICATION SERVER / ..... _., Tj- ---- - - -j- Sen~er Program ._. RM Receiver Figure 4- L1 JTA Invocation Fig 4-1 1 shows a sample JT A usage with Mercury to reliably transfer a set of messages stored at the APPLICATION_CUI:.NT to APLICATION_SERVER database. This configuration does not loose or send duplicate messages due to recovery nature of the distributed transactions. Here we use the Apache Derby as the underlying database and the Atomikos -52- opensource library to provide the JT 1\ functionality. The persistence storage with the jta connection manager have to be configured both at the client and server axis2.xml files. trtJP cltJ.,vam> trr> !db<:dorby / hO .. / Mil.Vn> <.Par.111Ctl5" nane-~db.u~r"'~aneter> - ;"-f U 1 t"- I'Pef'ty Figure 4-12 J TA Storage Configuration For jta connections application clil.!nt program supposed to provide a JT AConnectionManager which is used to get the transaction manager and connection objects. InvokerWorker use MercuryUseJTA Transaction property to decide whether to start a JT A transaction or not before invoking the business logic. ... --. - 53- } publfc class At OIIlkosJTAConnectlonManager 1rp l er~nl ~ JToConnectlonNanager { private static Log l og • LogFactory.{letLoll(l\io~HosJTA Iter • jtaProoertleSE1e•ent.getCh1ldEle~ents (); l te r.has~ext();) { o•Element = lter.next(); } propertyHau • 01Ele1ent. QetAtt r1 buteVal ~e(ne11 Q~a~e('"o , "nure")); 1 r (propert)l)la•e.equals(Constants.I!8..JlRH'fR)) { this.dbDrher • o•El~•~nt.getTe•t ' ); } else { } properties.put(propertyHuP 0 o•El ~ne•:t. getTe•tO); th1s . dataSourcetlean • new AtOili); th1s. dataSourceBean.setMa•PoolS1Zeo'S ; publ1c Connect1on oetNewConnect lon() lhro\\5 StorageExceptlon { try { Connecti on connect! on • th1 s. !lataSourcei!P.Ul. getConnect1 on(); r e turn connection; } catch (SQLExcept1 on e) { log.er ro r ("Cdn not create the- AIOI"I~O\ connecl1on", e); throw ne\11 StoraoeExceptl on("CcUI r10t creitll' the Alu11111w., et ccnf1 our an on~ontext Cnnfl Juratl on((lntPY"tFactnry. c rPATP(flnfJ r)Ju·:nlon ·nr'ltPxrFrnMf J J~~Y~tP.w( AXIS2...RFPOS1TORY WCATJO>;, Af!S7 CIIF.\11 ((.l\~/C 1-/U); Scrvicc;t :8~Sij_- oLXl!>l/-.·nl(~S/ JT ~lr.Servl ce")); serv1eeCl1 en~. getOpt:lors() . setACtl Jn("ur u : JUluOpt·r "'""'"); serv1 ceC 11 ent. engageMoOul e('·,ercur")'") ; servi ceC11 ent. oetOpnors(). setUseSeparateL 1 stener( tn•P) : serv1 ceCl1 ent . getOonons(). setProperty(Herc~ry(l ant Constant . IWER.YAl..AH, "ke~ !"); serV1ceC11ent. getOOtlors().setProperty(Mercur)/(11fntC• n1tants. LSC. JTA 1Hi<\ISACTIO.\, Constarts. VAllE TRUE); Atooi kosOataSourceBean dataSourceBe.,, • ,_ A too• ko$0•taSJurceBean(); oausourceBea.1. setlkllq~eResource~a.e("Appll(olll c-uc 1 H!nt~o~r (P"); dataSourceBea1. set)(al)ataSourceCl as;Haee("ot·Q. dll~e". " /hO!»'./oll"l la.'r:<< 1pr UJI'< t '~•:-cuq' nKIUII'> lllt•ro 'j t.l·cltent!datah4Se/.tP!'LICUID'I ( llf'IT"); d•uSourceBea• . setXaPrcpertl es(p,opertl es); cen~gura::1 ont:o•lten. setProperty{ .. .\J'Ittl lc.1ttt1n( l trht'S():wcc•, d.ltaSo~.~ceGe3.n): •••s•geNu•bers • new Arrallt stO: ..tnle (resultSet .ne~()) { oessa9eNuabers. ado(resullSet . g-tlong("IO < ";), } resul tSet. close() ; State• ent. cl ose() ; connect1 on. close() ; UserTra.nsactl onNanage - userTransocti onManager ror ( long •essageiD : messageNu•Uers) { try { userTransani on~anager • now UserTr•nsaction~anagtr(); userTransactlonManager. i r1 t (~; userTransac:1 onKanager. beg1n(); ccnrect1 on • dataSourceBean. getConnect1 on(); stateoent = connectt on. createStatete•tO; String QueryS:nng = "sele 0) { -...... serv1 ceC11ent. getOptl ons() . setProperty(Her:uryCil entconstants. USE..JTA_TIIA.\ISACTION, Constants . \'AWEJAlSE) ; Me rcu ry( llent me rcury(l ient • new "ercury(l1ent(serv1CeC11ent ); •Prcur)/(11 ent . ter•• natPSPqu• ncP(' ~Pyl "); Figure 4-14 JTA In Only Client first it sets the USE_JTA_TRANSACTION property to true in order to indicate application client going to perform a jta transaction. When the Mercury sees this property it invokes the Atom.ikosJT AConnectionManager class to obtain the JT A connection resource. Then it creates Atomi kosDa1aSourccBean resource. an the to JTA get connection Atom.ikosDataSourceBean automatically enlist the connection to distributed transaction. After that it reads the available message nurnhcrs to send using a normal database connection. Next -55- it starts sending messages one by one. llnlikc in a normal persistence scenario now it starts a distributed transaction using UserTransactionManager. After updating the client database and sending message to Mercury it closes the connection and commit the distributed transaction. Hence it reliably transfer messages stored at APPr J(' A TTON_CLIENT database to RMS. 1 r (tsUseJTATransactl on) { transact! onNanager = storageNanager. getTra~sactl onManag~r(); transacti onNanager . begin(); transacn on • storageManaoer. get TransactlM(lsUseJTATr&nsactlon); InvokerBuffer~anager •nvo~erBufferHaoagtr • storagt~an<~tr. gttinvol erBuffer~arager(seq"enceiO); InvokerBuffer)to 1 nvokerBufferOto • lnvO~~r6uffer~ tnager. get!nvo~erButter(.to(sequenceiD); try { tr ((lnYO<.e rBufferOto.getState() !• ln•o•~rBuffer.SUT£.011 M (lnvoi<.erBufferDto.Ge'Stat e() !• Invokereutrer.STHLTE!I'lto.I7ED)) { H ((5Yste•. currEntTI•P¥>111 s() - • nvo~ erBuffe•Oto. ;l•tlastAccessTI•e()) < r 'OkerBuf"fer(}(o. getTlleout" e[:) { !1voJlR . BtiH~r 't4te ·•~ t~r.,1nat~d Since lh<' s"~'t1!nce ts til"@d out"); t 1YOItorages . ... The reliability and fault tolerance achieved varies according to the type of storage being used. Rest of the chapter describes some of the observations made with the different scenarios mentioned in the earlier chapter. Any reliable messaging framework downgrades the performance of sending messages. In other words reliability is invcr!>ely proportional to the performance. In WS-RM this is mainly -~ because initial sequence creation and acknowkdgement messages. Further it takes time to store the message to the persistence storage in tht! case of persistence and jta scenarios. In memory model provides the weakest form of reliability. It provides the reliability for network failures but can't survive with the node failures. If the node fails it loses all the messages and sequence state and hence fail to recover. Persistence model provides better reliability than in memory model. It provides the reliability for network failures. Since it persists sequence state and messages received it can restart RM sequences after a node fail. For this pn~ject work we tested this model by sending 20 messages while stopping and starting the client node and server node. Although it can recover sequences we observe some messages has lost. The number of messages at the -57- APPLICATION_SERVER databast! was less than 20 for in only case. Further number of reply messages were also less than 20 in 1\PPLIC/\TION_SERVER for in out case. Persistence storage with JT 1\ support provides the best reliability. first it provides reliability for network failures. Furtht!r JT/\ support provides the reliability for node failures without losing any message. For this project work we teMed the JTA support by sending 20 messages while stopping and starting the client node and server node. But there were no message loses either at the APPLICATION_SERVI ~R database or /\PPLICATION_CLIENT database . ... ........ -58 - Chapter 6 Conclusion & Future Work This thesis describes a queued tran!.action processing based solution using web service " reliable messaging in order to guarantee the client side and server side persistence storage updates. It achieves this goal by re engineering the WS02 mercury with a storage based API. Hence this project presents a storage API based WS-RM implementation which can support distributed transactions. It provide!. a set of use case scenarios to describe the way to use the new Mercury Implementation and prove its point in reliability. Sample scenarios uses Apache Derby as the database for its persistence storage and Atomikos as the Trbrary to provide the JT A support. The reliability of Mercury is handled by using a state machine model. Although there is a state machine for WS-RM 1.1 specification there is no such a model for WS-RM 1.0 specification. Therefore the state machine model described here which is independent of the implementation can be used for any WS-RM 1.0 specification implementation. The storage API developed provides explicit support for both in memory and persistence storage implementations. This storagl.! API which is independent of implementation logic can also be used with any WS-RM implcmemation. - 59 - There are some problems with the 2PC recovery with the Apache Derby XA driver and other commonly used opcnsource database XA drivers. However investigating deeply into these problems and finding out XA drivers that properly support 2PC recovery, goes beyond the scope of this work and we kept it as a possible future work. Further research can be done to integratt.: the WS-RM transactions with the application servers. This allows application dcvclopt.:r!> to integrate Enterprise Java Bean Objects transactions with the WS-RM transactions. This thesis concentrates only on supporting distributed transactions on a WS-RM implementation. But a WS-RM implementation should address a lot of features with different aspects. Hence we kept adding new features such as implementing WS-RM 1.1 support, usc single threaded invocations for synchronous communication, WS-RM level error handling and Secure Reliable Messaging as another possible future work. .., --. -60- REFERENCES [1] ] Andrew S. Tanenbaum and Maarten van Steen, Distributed Systems: Principles and Paradigms. New jersey U.S.A : Prenticl.! llall, inc, 2002 [2] Philip A. Bernstein and Eric Newcomer, Principles of Transaction Processing. San Francisco, CA: Morgan Kaufmann Publishes Inc, 1997. pp 101-116 [3] Don Box et al. ''Simple Object Access Protocol (SOAP) 1.1" W3C Note 08 May 2000. [4] Luis Felipe Cabrera et al. "Web Services Coordination (WS-Coordination)" August 2005 [5] Luis Felipe Cabrera et al. "Web Services Atomic Transaction (WS- AtornicTransaction)" August 2005 [6] Luis Felipe Cabrera et al. "Web Services Business Activity Framework (WS- BusinessActivity)" August 2(X)5 [7] Jens Lechte nb orger "2-PIIASE COMMri PROTOCOL" University of M"unster, Germany. [8] Ruslan Bilorusets et al. "W!.!b S!.!rvices Reliable Messaging Protocol (WS- Re liableMessaging)" February. [9] Paul Fremantle et al. "Web Servict.:s Rdiable Messaging (WS-1 ReliableMessaging) Version 1.1" 14 June 2007. [10] Don Box et al. "Web Servict.:s Addrt.:ssing (WSAddressing)" Augest 2004. [11] "Distributed Transaction Proct.:ssi ng: The XA Specification" X/Open Company Ltd. [12] "Java Transaction API (JTA)" Sun Microsystcms lnc. [13] "Java Transaction Services (JTS)" Sun Microsystems Inc. [14] Stefan Tai, Thomas A. Mikalscn, Isabelle-. Rouvellou "Using Message-oriented Middleware for Reliable Web Services Messaging" IBM T.J. Watson Research Center, Hawthorne, New York, USA [15] Christoph Liebig and Stefan Tai "Middlcware Mediated Transactions" Dannstadt University of Technology, Darmstadt, Gennany IBM T.J. Watson Research Center, New York, U.S.A., 2001 [16] Stefan Tai and Isabelle Rouvellou "Strategies for Integrating Messaging and Distributed Object Transactions" IBM 'I .J . Watson Research Center, New Y-Qfk, USA, 2000 [ 17] Stefan Tai, Thomas A. Mikalsen, Isabelle Rouvellou, Stanley M. Sutton Jr. [18] "Dependency-Spheres: A Global Transaction Context for Distributed Objects and Messages" IBM T.J. Watson Research Center, New York, U.S.A., 2001 - 61 - '-"''' . .,