====== Proposal: Use Websocketd for OpenSRF Websocket Gateway ======
This started as a research project resulting primarily from trying to address [[https://bugs.launchpad.net/opensrf/+bug/1774703|LP bug #1774703]].
I propose we replace our Apache-driven OpenSRF Websocket handler with a websocketd implementation.
* Code: https://github.com/joewalnes/websocketd
* Binary builds: http://websocketd.com/
Websocketd is a purpose-built Websockets relay. It handles the Websocket protocol, communicating with back-end handlers via STDIO.
===== Why? =====
* The code is considerably less complicated
* No threads -- we use a select() event loop.
* Should be easier to test and maintain
* Using the load test script (below) I am consistently able to cause failures in the apache-websocket handler, but not the websocketd handler.
* See also LP bug noted above.
* Uses about 15% less RAM (on my test VM)
* Speed is consistent with Apache (within fractions of milliseconds)
* Websocketd project has a lot more activity and contributors than the apache-websocket code we're currently using.
* fewer steps to set up
===== Concerns? =====
Only concern so far is lack of support for pre-forking backends. Each new WS connection requires an OpenSRF connect on startup. On my test VM this adds roughly 5ms of overhead to the first request.
===== Quick Setup Guide for NGINX Users =====
**Note that a proxy is not strictly required, because websocketd supports SSL connections, but I prefer to run it behind a proxy.**
==== Install OpenSRF Branch ====
Check out and install OpenSRF branch user/berick/websocketd-backend @ working normally.
http://git.evergreen-ils.org/?p=working/OpenSRF.git;a=shortlog;h=refs/heads/user/berick/websocketd-backend
==== Install websocketd binary ====
cd /tmp
wget 'https://github.com/joewalnes/websocketd/releases/download/v0.3.0/websocketd-0.3.0-linux_amd64.zip'
unzip websocketd-0.3.0-linux_amd64.zip
sudo cp websocketd /usr/local/bin/
==== Run Websocketd on port 7682 ====
# Stop apache-websockets first since we're reusing the port.
sudo systemctl stop apache2-websockets # or similar
# AS USER opensrf
/usr/local/bin/websocketd --loglevel info --maxforks 250 --port 7682 /openils/bin/osrf-websocket-stdio
# Or optionally override the config file path
/usr/local/bin/websocketd --loglevel info --maxforks 250 --port 7682 /openils/bin/osrf-websocket-stdio /path/to/opensrf_core.xml
# Optionally background the process with a trailing '&'
Full set of command line args at https://github.com/joewalnes/websocketd/blob/master/help.go
==== Modify NGINX to proxy websocketd ====
Modify /etc/nginx/sites-enabled/osrf-ws-http-proxy.
# Websocketd supports SSL, but I run it in non-SSL mode since it's on the same machine as NGINX.
# Replace https:// with http://
proxy_pass http://localhost:7682;
# Update timeout values within the /osrf-websocket-translator section
# to disconnect idle clients.
# The osrf-websocket-stdio handler has no idle timeout threads.
# Change to suit
proxy_send_timeout 5m;
proxy_read_timeout 5m;
Restart NGINX:
sudo systemctl restart nginx
Reminder:
Confirm WEBSOCKET_PORT_SSL has the expected value (usually 443) near the top of
/openils/var/web/js/dojo/opensrf/opensrf_ws.js
==== OPTIONAL Running the perl test code ====
The Perl test script runs a few calls / response loops resulting in several hundred websocket calls.
# beware this installs quite a few modules
sudo cpan Net::Async::WebSocket::Client;
sudo cpan IO::Async::SSL; # for testing full path
cd OpenSRF/src/websocket-stdio
perl tester.pl wss://localhost:443/osrf-websocket-translator
==== Testing Evergreen ====
Log in to the browser client as usual.
==== Optional Systemd Setup ====
Websocketd is a standalone program with no daemon mode ([[https://github.com/joewalnes/websocketd/issues/300|not yet anyway]]). Systemd to the rescue.
Put this content into file /lib/systemd/system/websocketd-osrf.service
[Unit]
Description=Websocketd OpenSRF Gateway
[Service]
Type=simple
User=opensrf
Group=opensrf
Environment=PATH=/openils/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
Environment=LD_LIBRARY_PATH=/openils/lib:/usr/local/lib:/usr/local/lib/dbd:$LD_LIBRARY_PATH
ExecStart=/usr/local/bin/websocketd --loglevel error --maxforks 250 --port 7682 /openils/bin/osrf-websocket-stdio
# modify websocketd command line options to taste
# --sameorigin and --origin=domain1,domain2 flags are also supported for security.
# On Ubuntu 18.04, you may also need to include something like this:
[Install]
WantedBy=multi-user.target
Then add & start the service.
# disable apache2-websockets to avoid unintended starts & port conflicts
sudo systemctl disable apache2-websockets
# enable and start websocketd-osrf
sudo systemctl daemon-reload
sudo systemctl enable websocketd-osrf
sudo systemctl start websocketd-osrf