Table of Contents
Proposal: Use Websocketd for OpenSRF Websocket Gateway
This started as a research project resulting primarily from trying to address LP bug #1774703.
I propose we replace our Apache-driven OpenSRF Websocket handler with a websocketd implementation.
- Binary builds: http://websocketd.com/
Websocketd is a purpose-built Websockets relay. It handles the Websocket protocol, communicating with back-end handlers via STDIO.
Why?
- The code is considerably less complicated
- No threads – we use a select() event loop.
- Should be easier to test and maintain
- Using the load test script (below) I am consistently able to cause failures in the apache-websocket handler, but not the websocketd handler.
- See also LP bug noted above.
- Uses about 15% less RAM (on my test VM)
- Speed is consistent with Apache (within fractions of milliseconds)
- Websocketd project has a lot more activity and contributors than the apache-websocket code we're currently using.
- fewer steps to set up
Concerns?
Only concern so far is lack of support for pre-forking backends. Each new WS connection requires an OpenSRF connect on startup. On my test VM this adds roughly 5ms of overhead to the first request.
Quick Setup Guide for NGINX Users
Note that a proxy is not strictly required, because websocketd supports SSL connections, but I prefer to run it behind a proxy.
Install OpenSRF Branch
Check out and install OpenSRF branch user/berick/websocketd-backend @ working normally.
Install websocketd binary
cd /tmp wget 'https://github.com/joewalnes/websocketd/releases/download/v0.3.0/websocketd-0.3.0-linux_amd64.zip' unzip websocketd-0.3.0-linux_amd64.zip sudo cp websocketd /usr/local/bin/
Run Websocketd on port 7682
# Stop apache-websockets first since we're reusing the port. sudo systemctl stop apache2-websockets # or similar # AS USER opensrf /usr/local/bin/websocketd --loglevel info --maxforks 250 --port 7682 /openils/bin/osrf-websocket-stdio # Or optionally override the config file path /usr/local/bin/websocketd --loglevel info --maxforks 250 --port 7682 /openils/bin/osrf-websocket-stdio /path/to/opensrf_core.xml # Optionally background the process with a trailing '&'
Full set of command line args at https://github.com/joewalnes/websocketd/blob/master/help.go
Modify NGINX to proxy websocketd
Modify /etc/nginx/sites-enabled/osrf-ws-http-proxy.
# Websocketd supports SSL, but I run it in non-SSL mode since it's on the same machine as NGINX. # Replace https:// with http:// proxy_pass http://localhost:7682; # Update timeout values within the /osrf-websocket-translator section # to disconnect idle clients. # The osrf-websocket-stdio handler has no idle timeout threads. # Change to suit proxy_send_timeout 5m; proxy_read_timeout 5m;
Restart NGINX:
sudo systemctl restart nginx
Reminder:
Confirm WEBSOCKET_PORT_SSL has the expected value (usually 443) near the top of /openils/var/web/js/dojo/opensrf/opensrf_ws.js
OPTIONAL Running the perl test code
The Perl test script runs a few calls / response loops resulting in several hundred websocket calls.
# beware this installs quite a few modules sudo cpan Net::Async::WebSocket::Client; sudo cpan IO::Async::SSL; # for testing full path cd OpenSRF/src/websocket-stdio perl tester.pl wss://localhost:443/osrf-websocket-translator
Testing Evergreen
Log in to the browser client as usual.
Optional Systemd Setup
Websocketd is a standalone program with no daemon mode (not yet anyway). Systemd to the rescue.
Put this content into file /lib/systemd/system/websocketd-osrf.service
[Unit] Description=Websocketd OpenSRF Gateway [Service] Type=simple User=opensrf Group=opensrf Environment=PATH=/openils/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin Environment=LD_LIBRARY_PATH=/openils/lib:/usr/local/lib:/usr/local/lib/dbd:$LD_LIBRARY_PATH ExecStart=/usr/local/bin/websocketd --loglevel error --maxforks 250 --port 7682 /openils/bin/osrf-websocket-stdio # modify websocketd command line options to taste # --sameorigin and --origin=domain1,domain2 flags are also supported for security. # On Ubuntu 18.04, you may also need to include something like this: [Install] WantedBy=multi-user.target
Then add & start the service.
# disable apache2-websockets to avoid unintended starts & port conflicts sudo systemctl disable apache2-websockets # enable and start websocketd-osrf sudo systemctl daemon-reload sudo systemctl enable websocketd-osrf sudo systemctl start websocketd-osrf