User Tools

Site Tools


opensrf_over_http

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

opensrf_over_http [2007/10/18 10:38] (current)
miker created
Line 1: Line 1:
 +As posted on the open-ils-dev mailing list:
 +----
  
 +Bill and I have been puzzling over how to give HTTP clients support
 +for the more advanced features of OpenSRF, such as stateful,
 +transactional sessions and streaming responses. ​ The main problem is
 +that HTTP is entirely stateless on its own, and OpenSRF is not.  The
 +second biggest problem is that most current HTTP clients do not lend
 +themselves to the kind of connection control we'd like to have.
 +
 +Well, with the addition of (what really should be) a fairly trivial
 +translator service written as an Apache module, and a new server-side
 +setting that allows OpenSRF Applications to loosen their restrictions
 +on who can talk to them and when, we think we have a way to provide
 +advanced, robust, degradable OpenSRF services to all modern HTTP
 +clients.
 +
 +Let us know what you think.
 +
 +And so, herein follows ...
 +
 +
 +**A modest proposal for streaming, stateful OpenSRF-over-HTTP**
 +----
 +
 +
 +
 +====Client Request====
 +
 +The method is POST, URL is that of the HTTP/XMPP translator. ​ The body of the
 +POST data should be a JSON encoded array of one or more osrfMessage objects.
 +See the bottom of this document for both XMPP and HTTP examples of OpenSRF
 +communication.
 +
 +OpenSRF HTTP headers from client to server:
 +
 +  * X-OpenSRF-service=<​service-name>​ //for example, open-ils.search//​
 +  * X-OpenSRF-to=<​XMPP-address> ​     //value of the last **X-OpenSRF-from** delivered by the server//
 +  * X-OpenSRF-xid=<​timestamp-ms> ​    //for tracing messages to the client//
 +  * X-OpenSRF-thread=<​guid-string> ​  //​uniquely identifies this conversation//​
 +
 +The **X-OpenSRF-service** and **X-OpenSRF-to** headers are mutually exclusive, and
 +cannot be used together. ​ There are exactly two instances where the
 +**X-OpenSRF-service** header should be used:
 +
 + - At the beginning of a stateful session, when sending the CONNECT message
 + - For stateless requests
 +
 +In all other instances, which is to say within any stateful session that is
 +past the **CONNECT** phase, the client should use the **X-OpenSRF-to** header. ​ The
 +value of this header is set to the XMPP address returned in the by **STATUS_OK**
 +message from the current session **CONNECT** request.
 +
 +
 +If **multipart/​x-mixed-replace** mode is supported by the client, then it should
 +send the following header with every HTTP request:
 +
 + ​X-OpenSRF-multipart=true
 +
 +Otherwise, the translator must assume multipart is not supported by the client.
 +
 +Setting aside the underlying transport and supposing the complete equivalence
 +of the HTTP headers to XMPP XML attributes as described above, all other data
 +within the OpenSRF message remains entirely the same.  In other words, The body
 +of the HTTP request is exactly equivalent to the ''<​body/>''​ element with an XMPP
 +''<​message/>''​ element.
 +
 +
 +===Treament of these connection headers within the HTTP/XMPP translator===
 +
 +  * **X-OpenSRF-service** -- Should be mapped to the XMPP endpoint within the
 +configured router, as described in the core config file for OpenSRF clients.
 +Typically this will look something like ''​router@localhost/​open-ils.search'',​
 +where the service name provided in the header is used as the last component,
 +the XMPP resource. ​ This is used in the ''​@to''​ attr of the ''<​message/>''​ element on
 +the XMPP network. ​ This header is only used for the first message in a stateful
 +connection or for stateless requests.
 +
 +  * **X-OpenSRF-to** -- Within the client, he value for this header is taken by from
 +value provided in the first **X-OpenSRF-from** header from the **STATUS_OK** message
 +provided by the server in response to the initial **CONNECT** request. ​ This is
 +used in the ''​@to''​ attr of the ''<​message/>''​ element on the XMPP network.
 +
 +  * **X-OpenSRF-xid** -- Typically, the time since the Unix epoc in milliseconds.
 +Used to trace the path of messages through the OpenSRF network. ​ If not
 +supplied by the client, this should be created and supplied by the translator.
 +This datum is used in the ''​@osrf_xid''​ attr of the ''<​message/>''​ element on the XMPP
 +network.
 +
 +  * **X-OpenSRF-thread** -- A unique identifier for this request or session. ​ Can be
 +any string, though typically, a combination of the epoc time and originating
 +process id are used.  This is used by OpenSRF endpoints for identifying and
 +continuing stateful sessions. ​ If not supplied by the client, the translator
 +should create and supply a value for this element. ​ This datum is used as the
 +text content of the ''<​thread/>''​ element on the XMPP network.
 +
 +
 +====Response from the server====
 +
 +Assuming no error was encountered,​ the OpenSRF service will respond with among
 +other things several bits of data that are critical to the ongoing
 +communication between the client and server. ​ The most important of these is
 +the XMPP ''​@from''​ attr of the ''<​message/>''​ element. This must be used by the client
 +for all future requests within a stateful session, assuming a stateful session
 +was requested and created. ​ This datum should be mapped by the translator to an
 +HTTP header called **X-OpenSRF-from** in the response to the client.
 +The client will then use this value in the **X-OpenSRF-to** header of all
 +future in-session communication.
 +
 +The body of the response, or responses if a multipart request is used and a
 +streaming method called, will each be an array of one or more ''​osrfMessage''​
 +objects.
 +
 +If the client does not request **multipart/​x-mixed-replace** mode using the
 +**X-OpenSRF-multipart** header, as described above, then the translator will
 +collect all response ''​osrfMessage''​ objects, up to and including the first
 +**COMPLETE** message, and packaged them into a single array object as a single
 +request response to the client.
 +
 +
 +====Changes to OpenSRF====
 +
 +Stateful communication with backend OpenSRF service instances will be made
 +possible by giving each service the ability to accept "​migratable"​ client
 +remote IDs.  If the configuration section for a given OpenSRF application
 +has the setting
 +
 +<​code>​
 +  <​migratable-clients>​true</​migratable-clients>​
 +</​code>​((This name will probably change ... it has met with resistance based on the fact that "​migratable"​ is not a word. ;) ))
 +
 +then that service will ignore the requirement that the remote ID of a stateful
 +client must match across all communication within a session, and instead rely
 +only on the thread of the session for session tracking. ​ It is normally, and
 +currently always, an error condition for multiple client XMPP identifiers to
 +be used with one thread and one session. ​ This is to prevent session hijacking
 +among other security concerns. ​ However, loosening this restriction is required
 +in order to support OpenSRF-over-HTTP,​ and concerns are largely mitigated by
 +requiring the service to allow the feature explicitly in its configuration.
 +
 +There is no restriction on the type of ''​osrfMessage''​ objects that can be sent,
 +and as long as their order (and associated succesful responses) would
 +constitute a normal stateful OpenSRF session on an XMPP network, the use of
 +the ''<​migratable-clients/>''​ setting will allow this same functionality over
 +HTTP.
 +
 +
 +====Overview of the HTTP/XMPP translator====
 +
 +The translator will be written as an Apache module. ​ Its entire purpose is to
 +smooth out the differences between the HTTP and XMPP transports with respect
 +to OpenSRF communication. ​ It will do as little interpretation and manipulation
 +of data as possible, modifying only that data which must absolutely be changed
 +in order to support certain sub-optimal client software, and to map concepts
 +between HTTP and XMPP.
 +
 +
 +===Upstream HTTP-side handling===
 +
 +For upstream (client to server) communication,​ no data modification need occur
 +other than the mapping request headers to XMPP attributes, and the adding of
 +XMPP-specific router information to the service initially requested by the
 +client. ​ No modification or parsing of the message body will be performed on
 +upstream communication.
 +
 +===Downstream HTTP-side handling===
 +
 +For downstream (server to client) communication,​ there are to possible logic
 +paths. ​ The first, being both simpler to implement and more generally useful
 +on the client end, is the case where the client supports
 +**multipart/​x-mixed-replace** mode, as is the case with the Mozilla browser from
 +version 1.7 on and all version of the Firefox browser. ​ We will call this
 +**multipart mode**. ​ The second case contains all other browsers and clients, as
 +no other browsers are known to support this mode at this time.  We will call
 +this **collected mode**.
 +
 +  - For **multipart mode** capable clients, no modification of any data will occur
 +other than the mapping of the XMPP attributes to their respective HTTP
 +counterparts. ​ All data contained in the body of the OpenSRF message will be
 +passed unaltered as a multipart chunk followed by a boundary marker. ​ The
 +translator will use this response mode when it receives a request that is
 +accompanied by a **X-OpenSRF-multipart** header set to **true**.
 +
 +  - For **collected mode** clients, all osrfMessage objects sent by the server will
 +be collected into a single JSON array, in the order they arrive, and delivered
 +as a single HTTP response to the client. ​ This will require parsing of the JSON
 +in order to pull all ''​osrfMessage''​ objects out of each response packet and build
 +the response array.
 +
 +
 +===XMPP-side communication===
 +
 +The translator will use standard OpenSRF libraries to communicate with the XMPP
 +network-based services, just as the existing gateway and srfsh applications do.
 +
 +
 +====Example Data====
 +
 +===XMPP message example===
 +
 +<​code>​
 +<message
 +       ​to='​osrf@localhost/​open-ils.search_listener_at_grendel.local_31319'​
 +       ​from='​gateway@localhost/​1192540419__1192540420.166380_31367'​
 +       ​router_command=''​
 +       ​router_class=''​
 +       ​osrf_xid='​1192540419313673'>​
 + <​thread>​1192540427.567678.119254042731367</​thread>​
 +   <​body>​
 +[
 +   {
 +       "​__c":"​osrfMessage",​
 +       "​__p":​{
 +           "​threadTrace":"​1",​
 +           "​locale":"​en-us",​
 +           "​type":"​REQUEST",​
 +           "​payload":​{
 +               "​__c":"​osrfMethod",​
 +               "​__p":​{
 +                   "​method":"​open-ils.search.biblio.record.mods_slim.retrieve",​
 +                   "​params":​[1487101]
 +               }
 +           }
 +       }
 +   }
 +]
 +   </​body>​
 +</​message>​
 +</​code>​
 +
 +===HTTP message example===
 +
 +<​code>​
 +
 +X-OpenSRF-to=open-ils.search
 +X-OpenSRF-xid=1192540419313673
 +X-OpenSRF-thread=1192540427.567678.119254042731367
 +X-OpenSRF-multipart=true
 +
 +[
 +   {
 +       "​__c":"​osrfMessage",​
 +       "​__p":​{
 +           "​threadTrace":"​1",​
 +           "​locale":"​en-us",​
 +           "​type":"​REQUEST",​
 +           "​payload":​{
 +               "​__c":"​osrfMethod",​
 +               "​__p":​{
 +                   "​method":"​open-ils.search.biblio.record.mods_slim.retrieve",​
 +                   "​params":​[1487101]
 +               }
 +           }
 +       }
 +   }
 +]
 +
 +</​code>​
opensrf_over_http.txt · Last modified: 2007/10/18 10:38 by miker

© 2008-2017 GPLS and others. Evergreen is open source software, freely licensed under GNU GPLv2 or later.
The Evergreen Project is a member of Software Freedom Conservancy.