Differences

This shows you the differences between two versions of the page.

--- opensrf_over_http [2007/10/18 10:38] – created miker
+++ opensrf_over_http [2022/02/10 13:34] (current) – external edit 127.0.0.1
@@ Line 1: / Line 1: @@
+As posted on the open-ils-dev mailing list:
+----
+Bill and I have been puzzling over how to give HTTP clients support
+for the more advanced features of OpenSRF, such as stateful,
+transactional sessions and streaming responses.  The main problem is
+that HTTP is entirely stateless on its own, and OpenSRF is not.  The
+second biggest problem is that most current HTTP clients do not lend
+themselves to the kind of connection control we'd like to have.
+Well, with the addition of (what really should be) a fairly trivial
+translator service written as an Apache module, and a new server-side
+setting that allows OpenSRF Applications to loosen their restrictions
+on who can talk to them and when, we think we have a way to provide
+advanced, robust, degradable OpenSRF services to all modern HTTP
+clients.
+Let us know what you think.
+And so, herein follows ...
+**A modest proposal for streaming, stateful OpenSRF-over-HTTP**
+----
+====Client Request====
+The method is POST, URL is that of the HTTP/XMPP translator.  The body of the
+POST data should be a JSON encoded array of one or more osrfMessage objects.
+See the bottom of this document for both XMPP and HTTP examples of OpenSRF
+communication.
+OpenSRF HTTP headers from client to server:
+  * X-OpenSRF-service=<service-name> //for example, open-ils.search//
+  * X-OpenSRF-to=<XMPP-address>      //value of the last **X-OpenSRF-from** delivered by the server//
+  * X-OpenSRF-xid=<timestamp-ms>     //for tracing messages to the client//
+  * X-OpenSRF-thread=<guid-string>   //uniquely identifies this conversation//
+The **X-OpenSRF-service** and **X-OpenSRF-to** headers are mutually exclusive, and
+cannot be used together.  There are exactly two instances where the
+**X-OpenSRF-service** header should be used:
+ - At the beginning of a stateful session, when sending the CONNECT message
+ - For stateless requests
+In all other instances, which is to say within any stateful session that is
+past the **CONNECT** phase, the client should use the **X-OpenSRF-to** header.  The
+value of this header is set to the XMPP address returned in the by **STATUS_OK**
+message from the current session **CONNECT** request.
+If **multipart/x-mixed-replace** mode is supported by the client, then it should
+send the following header with every HTTP request:
+ X-OpenSRF-multipart=true
+Otherwise, the translator must assume multipart is not supported by the client.
+Setting aside the underlying transport and supposing the complete equivalence
+of the HTTP headers to XMPP XML attributes as described above, all other data
+within the OpenSRF message remains entirely the same.  In other words, The body
+of the HTTP request is exactly equivalent to the ''<body/>'' element with an XMPP
+''<message/>'' element.
+===Treament of these connection headers within the HTTP/XMPP translator===
+  * **X-OpenSRF-service** -- Should be mapped to the XMPP endpoint within the
+configured router, as described in the core config file for OpenSRF clients.
+Typically this will look something like ''router@localhost/open-ils.search'',
+where the service name provided in the header is used as the last component,
+the XMPP resource.  This is used in the ''@to'' attr of the ''<message/>'' element on
+the XMPP network.  This header is only used for the first message in a stateful
+connection or for stateless requests.
+  * **X-OpenSRF-to** -- Within the client, he value for this header is taken by from
+value provided in the first **X-OpenSRF-from** header from the **STATUS_OK** message
+provided by the server in response to the initial **CONNECT** request.  This is
+used in the ''@to'' attr of the ''<message/>'' element on the XMPP network.
+  * **X-OpenSRF-xid** -- Typically, the time since the Unix epoc in milliseconds.
+Used to trace the path of messages through the OpenSRF network.  If not
+supplied by the client, this should be created and supplied by the translator.
+This datum is used in the ''@osrf_xid'' attr of the ''<message/>'' element on the XMPP
+network.
+  * **X-OpenSRF-thread** -- A unique identifier for this request or session.  Can be
+any string, though typically, a combination of the epoc time and originating
+process id are used.  This is used by OpenSRF endpoints for identifying and
+continuing stateful sessions.  If not supplied by the client, the translator
+should create and supply a value for this element.  This datum is used as the
+text content of the ''<thread/>'' element on the XMPP network.
+====Response from the server====
+Assuming no error was encountered, the OpenSRF service will respond with among
+other things several bits of data that are critical to the ongoing
+communication between the client and server.  The most important of these is
+the XMPP ''@from'' attr of the ''<message/>'' element. This must be used by the client
+for all future requests within a stateful session, assuming a stateful session
+was requested and created.  This datum should be mapped by the translator to an
+HTTP header called **X-OpenSRF-from** in the response to the client.
+The client will then use this value in the **X-OpenSRF-to** header of all
+future in-session communication.
+The body of the response, or responses if a multipart request is used and a
+streaming method called, will each be an array of one or more ''osrfMessage''
+objects.
+If the client does not request **multipart/x-mixed-replace** mode using the
+**X-OpenSRF-multipart** header, as described above, then the translator will
+collect all response ''osrfMessage'' objects, up to and including the first
+**COMPLETE** message, and packaged them into a single array object as a single
+request response to the client.
+====Changes to OpenSRF====
+Stateful communication with backend OpenSRF service instances will be made
+possible by giving each service the ability to accept "migratable" client
+remote IDs.  If the configuration section for a given OpenSRF application
+has the setting
+<code>
+  <migratable-clients>true</migratable-clients>
+</code>((This name will probably change ... it has met with resistance based on the fact that "migratable" is not a word. ;) ))
+then that service will ignore the requirement that the remote ID of a stateful
+client must match across all communication within a session, and instead rely
+only on the thread of the session for session tracking.  It is normally, and
+currently always, an error condition for multiple client XMPP identifiers to
+be used with one thread and one session.  This is to prevent session hijacking
+among other security concerns.  However, loosening this restriction is required
+in order to support OpenSRF-over-HTTP, and concerns are largely mitigated by
+requiring the service to allow the feature explicitly in its configuration.
+There is no restriction on the type of ''osrfMessage'' objects that can be sent,
+and as long as their order (and associated succesful responses) would
+constitute a normal stateful OpenSRF session on an XMPP network, the use of
+the ''<migratable-clients/>'' setting will allow this same functionality over
+HTTP.
+====Overview of the HTTP/XMPP translator====
+The translator will be written as an Apache module.  Its entire purpose is to
+smooth out the differences between the HTTP and XMPP transports with respect
+to OpenSRF communication.  It will do as little interpretation and manipulation
+of data as possible, modifying only that data which must absolutely be changed
+in order to support certain sub-optimal client software, and to map concepts
+between HTTP and XMPP.
+===Upstream HTTP-side handling===
+For upstream (client to server) communication, no data modification need occur
+other than the mapping request headers to XMPP attributes, and the adding of
+XMPP-specific router information to the service initially requested by the
+client.  No modification or parsing of the message body will be performed on
+upstream communication.
+===Downstream HTTP-side handling===
+For downstream (server to client) communication, there are to possible logic
+paths.  The first, being both simpler to implement and more generally useful
+on the client end, is the case where the client supports
+**multipart/x-mixed-replace** mode, as is the case with the Mozilla browser from
+version 1.7 on and all version of the Firefox browser.  We will call this
+**multipart mode**.  The second case contains all other browsers and clients, as
+no other browsers are known to support this mode at this time.  We will call
+this **collected mode**.
+  - For **multipart mode** capable clients, no modification of any data will occur
+other than the mapping of the XMPP attributes to their respective HTTP
+counterparts.  All data contained in the body of the OpenSRF message will be
+passed unaltered as a multipart chunk followed by a boundary marker.  The
+translator will use this response mode when it receives a request that is
+accompanied by a **X-OpenSRF-multipart** header set to **true**.
+  - For **collected mode** clients, all osrfMessage objects sent by the server will
+be collected into a single JSON array, in the order they arrive, and delivered
+as a single HTTP response to the client.  This will require parsing of the JSON
+in order to pull all ''osrfMessage'' objects out of each response packet and build
+the response array.
+===XMPP-side communication===
+The translator will use standard OpenSRF libraries to communicate with the XMPP
+network-based services, just as the existing gateway and srfsh applications do.
+====Example Data====
+===XMPP message example===
+<code>
+<message
+       to='osrf@localhost/open-ils.search_listener_at_grendel.local_31319'
+       from='gateway@localhost/1192540419__1192540420.166380_31367'
+       router_command=''
+       router_class=''
+       osrf_xid='1192540419313673'>
+ <thread>1192540427.567678.119254042731367</thread>
+   <body>
+[
+   {
+       "__c":"osrfMessage",
+       "__p":{
+           "threadTrace":"1",
+           "locale":"en-us",
+           "type":"REQUEST",
+           "payload":{
+               "__c":"osrfMethod",
+               "__p":{
+                   "method":"open-ils.search.biblio.record.mods_slim.retrieve",
+                   "params":[1487101]
+               }
+           }
+       }
+   }
+]
+   </body>
+</message>
+</code>
+===HTTP message example===
+<code>
+X-OpenSRF-to=open-ils.search
+X-OpenSRF-xid=1192540419313673
+X-OpenSRF-thread=1192540427.567678.119254042731367
+X-OpenSRF-multipart=true
+[
+   {
+       "__c":"osrfMessage",
+       "__p":{
+           "threadTrace":"1",
+           "locale":"en-us",
+           "type":"REQUEST",
+           "payload":{
+               "__c":"osrfMethod",
+               "__p":{
+                   "method":"open-ils.search.biblio.record.mods_slim.retrieve",
+                   "params":[1487101]
+               }
+           }
+       }
+   }
+]
+</code>