====== Proposal: Use Websocketd for OpenSRF Websocket Gateway ======

This started as a research project resulting primarily from trying to address [[https://bugs.launchpad.net/opensrf/+bug/1774703|LP bug #1774703]].

I propose we replace our Apache-driven OpenSRF Websocket handler with a websocketd implementation.

  * Code: https://github.com/joewalnes/websocketd
  * Binary builds: http://websocketd.com/

Websocketd is a purpose-built Websockets relay.  It handles the Websocket protocol, communicating with back-end handlers via STDIO.

===== Why? =====

  * The code is considerably less complicated
  * No threads -- we use a select() event loop.
  * Should be easier to test and maintain
  * Using the load test script (below) I am consistently able to cause failures in the apache-websocket handler, but not the websocketd handler.
  * See also LP bug noted above.
  * Uses about 15% less RAM (on my test VM)
  * Speed is consistent with Apache (within fractions of milliseconds)
  * Websocketd project has a lot more activity and contributors than the apache-websocket code we're currently using.
  * fewer steps to set up

===== Concerns? =====

Only concern so far is lack of support for pre-forking backends.  Each new WS connection requires an OpenSRF connect on startup.  On my test VM this adds roughly 5ms of overhead to the first request.
 

===== Quick Setup Guide for NGINX Users =====

**Note that a proxy is not strictly required, because websocketd supports SSL connections, but I prefer to run it behind a proxy.**

==== Install OpenSRF Branch ====

Check out and install OpenSRF branch user/berick/websocketd-backend @ working normally.

http://git.evergreen-ils.org/?p=working/OpenSRF.git;a=shortlog;h=refs/heads/user/berick/websocketd-backend

==== Install websocketd binary ====

<code>
cd /tmp
wget 'https://github.com/joewalnes/websocketd/releases/download/v0.3.0/websocketd-0.3.0-linux_amd64.zip'
unzip websocketd-0.3.0-linux_amd64.zip
sudo cp websocketd /usr/local/bin/
</code>

==== Run Websocketd on port 7682 ====

<code>
# Stop apache-websockets first since we're reusing the port.
sudo systemctl stop apache2-websockets # or similar

# AS USER opensrf
/usr/local/bin/websocketd --loglevel info --maxforks 250 --port 7682 /openils/bin/osrf-websocket-stdio

# Or optionally override the config file path
/usr/local/bin/websocketd --loglevel info --maxforks 250 --port 7682 /openils/bin/osrf-websocket-stdio /path/to/opensrf_core.xml

# Optionally background the process with a trailing '&'
</code>

Full set of command line args at https://github.com/joewalnes/websocketd/blob/master/help.go
==== Modify NGINX to proxy websocketd ====

Modify /etc/nginx/sites-enabled/osrf-ws-http-proxy. 

<code>

# Websocketd supports SSL, but I run it in non-SSL mode since it's on the same machine as NGINX.
# Replace https:// with http://
proxy_pass http://localhost:7682;

# Update timeout values within the /osrf-websocket-translator section 
# to disconnect idle clients.
# The osrf-websocket-stdio handler has no idle timeout threads.
# Change to suit
proxy_send_timeout 5m;
proxy_read_timeout 5m;

</code>

Restart NGINX:
<code>
sudo systemctl restart nginx
</code>


Reminder:

Confirm WEBSOCKET_PORT_SSL has the expected value (usually 443) near the top of
/openils/var/web/js/dojo/opensrf/opensrf_ws.js

==== OPTIONAL Running the perl test code ====

The Perl test script runs a few calls / response loops resulting in several hundred websocket calls.

<code>
# beware this installs quite a few modules
sudo cpan Net::Async::WebSocket::Client;
sudo cpan IO::Async::SSL; # for testing full path

cd OpenSRF/src/websocket-stdio
perl tester.pl wss://localhost:443/osrf-websocket-translator
</code>


==== Testing Evergreen ====


Log in to the browser client as usual.

==== Optional Systemd Setup ====

Websocketd is a standalone program with no daemon mode ([[https://github.com/joewalnes/websocketd/issues/300|not yet anyway]]).  Systemd to the rescue.

Put this content into file /lib/systemd/system/websocketd-osrf.service

<code>
[Unit]
Description=Websocketd OpenSRF Gateway

[Service]
Type=simple
User=opensrf
Group=opensrf
Environment=PATH=/openils/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
Environment=LD_LIBRARY_PATH=/openils/lib:/usr/local/lib:/usr/local/lib/dbd:$LD_LIBRARY_PATH
ExecStart=/usr/local/bin/websocketd --loglevel error --maxforks 250 --port 7682 /openils/bin/osrf-websocket-stdio

# modify websocketd command line options to taste
# --sameorigin and --origin=domain1,domain2 flags are also supported for security.

# On Ubuntu 18.04, you may also need to include something like this:
[Install]
WantedBy=multi-user.target
</code>

Then add & start the service.

<code>
# disable apache2-websockets to avoid unintended starts & port conflicts
sudo systemctl disable apache2-websockets

# enable and start websocketd-osrf
sudo systemctl daemon-reload
sudo systemctl enable websocketd-osrf
sudo systemctl start websocketd-osrf
</code>