If you have Evergreen installed and are encountering systematic errors, here is how you might go about troubleshooting and gathering logs for others to troubleshoot. These instructions assume standard locations and filenames for Evergreen installations, and may also include commands that assume a Debian or Ubuntu Linux operating system.
**NOTE:** This content is also available in the Evergreen documentation at http://docs.evergreen-ils.org/1.6/draft/html/. In github: http://github.com/rsoulliere/Evergreen-DocBook/blob/master/1.6/admin/troubleshooting.xml.
===== Step 1: Stop all the OpenSRF services, as well as Apache =====
==== Stopping Apache ====
Typically, you would use a command like this as user root:
/etc/init.d/apache2 stop
Another command might look like this:
apache2ctl stop
To double-check and see if apache has actually been terminated, run a command like so:
ps -aef | grep apache2
Apache has not been stopped if you see any entries like this:
root 4609 1 2 13:05 ? 00:00:00 /usr/sbin/apache2 -k start
opensrf 4612 4609 3 13:05 ? 00:00:00 /usr/sbin/apache2 -k start
==== Stopping OpenSRF ====
Typically, you would use a command like this as user opensrf:
/openils/bin/osrf_ctl.sh -c /openils/conf/opensrf_core.xml -l -a stop_all
**Note:** You may need to invoke the osrf_ctl.sh command without the -l option if the installation instructions you used did not have you use localhost in leu of your fully qualified domain name (FQDN) in your configuration file. If this is the case, omit -l from all invocations of osrf_ctl.sh asked for in this document.
You should get either output resembling this:
Stopping OpenSRF C process 12515...
Stopping OpenSRF C process 12520...
Stopping OpenSRF C process 12526...
Stopping OpenSRF Perl process 12471...
Stopping OpenSRF Router process 12466...
Or, if services have already been stopped, like this:
OpenSRF C not running
OpenSRF Perl not running
OpenSRF Router not running
If you run into any troubles with stopping services, stop here and ask for help on either [[http://open-ils.org/listserv.php|OPEN-ILS-DEV]] or [[http://open-ils.org/irc.php|#evergreen]]. Other tips may be available on this [[troubleshooting:checking_for_errors_step1|page]].
Occasionally, in the trunk/ version of Evergreen, I have seen osrf_ctl.sh fail to kill OpenSRF processes, so we should check to make sure that none are still running with a command like this:
ps -aef | grep OpenSRF
OpenSRF has not completely shutdown if you see any entries like this:
opensrf 4400 1 0 12:50 ? 00:00:00 OpenSRF System
opensrf 4425 1 0 12:50 ? 00:00:00 OpenSRF System-C
opensrf 4401 4400 0 12:50 ? 00:00:00 OpenSRF master [opensrf.settings]
opensrf 4408 4400 0 12:50 ? 00:00:00 OpenSRF master [open-ils.cat]
opensrf 4409 4400 0 12:50 ? 00:00:00 OpenSRF master [open-ils.supercat]
opensrf 4410 4400 0 12:50 ? 00:00:01 OpenSRF master [open-ils.search]
opensrf 4411 4400 0 12:50 ? 00:00:01 OpenSRF master [open-ils.circ]
opensrf 4412 4400 0 12:50 ? 00:00:00 OpenSRF master [open-ils.actor]
opensrf 4413 4400 1 12:50 ? 00:00:02 OpenSRF master [open-ils.storage]
opensrf 4414 4400 0 12:50 ? 00:00:01 OpenSRF master [open-ils.penalty]
opensrf 4415 4400 0 12:50 ? 00:00:00 OpenSRF master [open-ils.collections]
opensrf 4416 4400 0 12:50 ? 00:00:00 OpenSRF master [open-ils.ingest]
opensrf 4417 4400 0 12:50 ? 00:00:00 OpenSRF master [open-ils.reporter]
opensrf 4407 4400 0 12:50 ? 00:00:00 OpenSRF listener [opensrf.settings]
opensrf 4426 4425 0 12:50 ? 00:00:00 OpenSRF Listener [opensrf.math]
opensrf 4428 4425 0 12:50 ? 00:00:00 OpenSRF Listener [opensrf.dbmath]
opensrf 4430 4425 0 12:50 ? 00:00:00 OpenSRF Listener [open-ils.auth]
opensrf 4431 4425 0 12:50 ? 00:00:00 OpenSRF Listener [open-ils.cstore]
opensrf 4437 4425 0 12:50 ? 00:00:00 OpenSRF Listener [open-ils.reporter-store]
opensrf 4441 4400 0 12:50 ? 00:00:00 OpenSRF listener [open-ils.cat]
opensrf 4442 4400 0 12:50 ? 00:00:00 OpenSRF listener [open-ils.supercat]
opensrf 4443 4400 0 12:50 ? 00:00:00 OpenSRF listener [open-ils.search]
opensrf 4444 4400 0 12:50 ? 00:00:00 OpenSRF listener [open-ils.circ]
opensrf 4445 4400 0 12:50 ? 00:00:00 OpenSRF listener [open-ils.actor]
opensrf 4446 4400 0 12:50 ? 00:00:00 OpenSRF listener [open-ils.storage]
opensrf 4448 4400 0 12:50 ? 00:00:00 OpenSRF listener [open-ils.penalty]
opensrf 4449 4400 0 12:50 ? 00:00:00 OpenSRF listener [open-ils.collections]
opensrf 4450 4400 0 12:50 ? 00:00:00 OpenSRF listener [open-ils.ingest]
opensrf 4451 4400 0 12:50 ? 00:00:00 OpenSRF listener [open-ils.reporter]
opensrf 4402 4401 0 12:50 ? 00:00:00 OpenSRF drone [opensrf.settings]
opensrf 4403 4401 0 12:50 ? 00:00:00 OpenSRF drone [opensrf.settings]
opensrf 4404 4401 0 12:50 ? 00:00:00 OpenSRF drone [opensrf.settings]
opensrf 4405 4401 0 12:50 ? 00:00:00 OpenSRF drone [opensrf.settings]
opensrf 4406 4401 0 12:50 ? 00:00:00 OpenSRF drone [opensrf.settings]
opensrf 4427 4426 0 12:50 ? 00:00:00 OpenSRF Drone [opensrf.math]
opensrf 4429 4428 0 12:50 ? 00:00:00 OpenSRF Drone [opensrf.dbmath]
opensrf 4438 4414 0 12:50 ? 00:00:00 OpenSRF drone [open-ils.penalty]
opensrf 4439 4437 0 12:50 ? 00:00:00 OpenSRF Drone [open-ils.reporter-store]
opensrf 4447 4430 0 12:50 ? 00:00:00 OpenSRF Drone [open-ils.auth]
opensrf 4452 4431 0 12:50 ? 00:00:00 OpenSRF Drone [open-ils.cstore]
opensrf 4454 4409 0 12:50 ? 00:00:00 OpenSRF drone [open-ils.supercat]
opensrf 4455 4416 0 12:50 ? 00:00:00 OpenSRF drone [open-ils.ingest]
opensrf 4456 4416 0 12:50 ? 00:00:00 OpenSRF drone [open-ils.ingest]
opensrf 4457 4416 0 12:50 ? 00:00:00 OpenSRF drone [open-ils.ingest]
opensrf 4458 4416 0 12:50 ? 00:00:00 OpenSRF drone [open-ils.ingest]
opensrf 4459 4416 0 12:50 ? 00:00:00 OpenSRF drone [open-ils.ingest]
opensrf 4460 4412 0 12:50 ? 00:00:00 OpenSRF drone [open-ils.actor]
opensrf 4461 4417 0 12:50 ? 00:00:00 OpenSRF drone [open-ils.reporter]
opensrf 4462 4408 0 12:50 ? 00:00:00 OpenSRF drone [open-ils.cat]
opensrf 4463 4415 0 12:50 ? 00:00:00 OpenSRF drone [open-ils.collections]
opensrf 4464 4410 0 12:50 ? 00:00:00 OpenSRF drone [open-ils.search]
opensrf 4465 4411 0 12:50 ? 00:00:00 OpenSRF drone [open-ils.circ]
opensrf 4466 4413 0 12:50 ? 00:00:00 OpenSRF drone [open-ils.storage]
You should manually kill these processes at this point.
If you were uable to stop OpenSRF with the above methods, you can also try this command:
rm –R /openils/var/run/*.pid
This will remove the temporary OpenSRF process files from the run directory which may have been left over from a previous shutdown of the system.
===== Step 2: Delete or archive the existing log files =====
By default, Evergreen log files are stored in
/openils/var/log/
To make it easier to scan the logs for errors, we should either erase or move the old logs out of the way while OpenSRF is not running. As user opensrf, you might use a sequence of commands like so:
cd /openils/var/log/
mkdir old
mv *.* old/
===== Step 3: Check that jabber and memcached are running =====
Usually, you shouldn't have to restart your jabber and memcache servers when restarting OpenSRF. But we do want to make sure they're at least running.
For ejabberd, you might try a command like so:
ps -aef | grep ejabberd
and expect output like this:
ejabberd 5508 1 0 Feb26 ? 00:00:00 /usr/lib/erlang/erts-5.5.5/bin/epmd -daemon
ejabberd 5525 1 0 Feb26 ? 00:01:47 /usr/lib/erlang/erts-5.5.5/bin/beam -- -root /usr/lib/erlang -progname erl -- -home /var/lib/ejabberd -sname ejabberd -s ejabberd -kernel inetrc "/etc/ejabberd/inetrc" -ejabberd config "/etc/ejabberd/ejabberd.cfg" log_path "/var/log/ejabberd/ejabberd.log" -sasl sasl_error_logger {file,"/var/log/ejabberd/sasl.log"} -mnesia dir "/var/lib/ejabberd" -noshell -noshell -noinput
ejabberd 13640 5525 0 Mar14 ? 00:00:00 /usr/lib/erlang/lib/ssl-3.1.1.1/priv/bin/ssl_esock
For memcached, you might try a command like so:
ps -aef | grep memcached
and expect output like this:
nobody 13646 1 0 Mar14 pts/6 00:00:00 /usr/bin/memcached -m 64 -p 11211 -u nobody -l 127.0.0.1
If you have trouble getting memcached and apache running, stop here and ask for help on either [[http://open-ils.org/listserv.php|OPEN-ILS-DEV]] or [[http://open-ils.org/irc.php|#evergreen]].
===== Step 4: Change the Log Levels in your config files =====
We're going to change the level of detail we log to help us narrow down any errors. This is something you might not want to do in a production environment, because it will produce vastly larger log files that will take up more space.
To do this, we edit the OpenSRF bootstrap config file with a command like so as user opensrf:
editor /openils/conf/opensrf_core.xml
The editor involved will probably default to a Vi-clone like [[http://vim.org/|Vim]], which won't be immediately intuitive to a new user to Unix-type environments. You may want to use an editor such as [[http://www.nano-editor.org/|nano]].
In any case, regardless of your text editor, you will want to search for every line containing .
By default, you will see entries like this:
3
You will want to change every instance of loglevel 3 to loglevel 4, like so:
4
If we get really desperate with troubleshooting, we may want to use a loglevel of 5, instead. And if we narrow things down to the database, we might want to change the logging options for Postgres as well, but I'll defer such tinkering to this [[troubleshooting:checking_for_errors_step4|page]]. If you run into trouble changing these settings, please stop here and ask for help on either [[http://open-ils.org/listserv.php|OPEN-ILS-DEV]] or [[http://open-ils.org/irc.php|#evergreen]].
===== Step 5: Start the OpenSRF router and check for errors =====
Typically, you would use a command like this as user opensrf:
/openils/bin/osrf_ctl.sh -c /openils/conf/opensrf_core.xml -l -a start_router
You should get output resembling this:
Starting OpenSRF Router
If you do not, then please stop here and ask for help on either [[http://open-ils.org/listserv.php|OPEN-ILS-DEV]] or [[http://open-ils.org/irc.php|#evergreen]]. We will want to see a copy of your router log and configuration files here:
/openils/var/log/router.log
/openils/conf/opensrf.xml
/openils/conf/opensrf_core.xml
And of course, any on-screen errors. You should not send us files if they contain passwords that you may use in production environments.
**Note:** Evergreen 1.4 uses two routers by default, a public one and a private one, with two different logfiles:
/openils/var/log/private.router.log
/openils/var/log/public.router.log
If the router does appear to have started, then let's double check by grepping the router log file (or files) for errors like so:
grep ERR /openils/var/log/*router.log
If you encounter any output from that command, please stop here and contact the development community as stated above.
As a final sanity check, look for the router process with a command like so:
ps -aef | grep Router
You should see output like this if you're running EG1.2:
opensrf 5108 1 0 17:38 ? 00:00:00 OpenSRF Router
Or like this with EG1.4:
opensrf 22530 1 0 Mar09 ? 00:00:00 OpenSRF Router
opensrf 22531 1 0 Mar09 ? 00:00:00 OpenSRF Router
If not, please contact us.
===== Step 6: Start the OpenSRF perl services and check for errors =====
Typically, you would use a command like this as user opensrf:
/openils/bin/osrf_ctl.sh -c /openils/conf/opensrf_core.xml -l -a start_perl
With Evergreen 1.4 and later, you should get output resembling this:
Starting OpenSRF Perl
* starting all services for norcrossfx.norcross.esi
* starting servivce pid=7484 opensrf.settings
* starting servivce pid=7493 open-ils.cat
* starting servivce pid=7495 open-ils.supercat
* starting servivce pid=7497 open-ils.search
* starting servivce pid=7499 open-ils.circ
* starting servivce pid=7501 open-ils.actor
* starting servivce pid=7502 open-ils.storage
* starting servivce pid=7509 open-ils.penalty
* starting servivce pid=7512 open-ils.collections
* starting servivce pid=7514 open-ils.ingest
* starting servivce pid=7517 open-ils.permacrud
* starting servivce pid=7522 open-ils.fielder
* starting servivce pid=7527 open-ils.vandelay
* starting servivce pid=7516 open-ils.reporter
With earlier versions of Evergreen, you may simply get:
Starting OpenSRF Perl
If you do not, then please stop here and ask for help on either [[http://open-ils.org/listserv.php|OPEN-ILS-DEV]] or [[http://open-ils.org/irc.php|#evergreen]]. At a minimum, we will want to see a copy of your router log, osrfsys log and configuration files here:
/openils/var/log/router.log
/openils/var/log/osrfsys.log
/openils/conf/opensrf.xml
/openils/conf/opensrf_core.xml
And of course, any on-screen errors. Though at this point, a copy of all files in /openils/var/log may be useful. You should not send us files if they contain passwords that you may use in production environments.
If the Perl services do appear to have started, then let's double check by grepping the log files for errors like so:
grep ERR /openils/var/log/*.log
If you encounter any output from that command, please stop here and contact the development community as stated above.
As a final sanity check, look for OpenSRF processes with a command like so:
ps -aef | grep -i opensrf
You should see output like this:
opensrf 7453 1 0 16:04 ? 00:00:00 OpenSRF Router
opensrf 7484 1 0 16:09 ? 00:00:00 OpenSRF controller [opensrf.settings]
opensrf 7486 7484 0 16:09 ? 00:00:00 OpenSRF master [opensrf.settings]
opensrf 7487 7484 0 16:09 ? 00:00:00 OpenSRF listener [opensrf.settings]
opensrf 7488 7486 0 16:09 ? 00:00:00 OpenSRF drone [opensrf.settings]
opensrf 7489 7486 0 16:09 ? 00:00:00 OpenSRF drone [opensrf.settings]
opensrf 7490 7486 0 16:09 ? 00:00:00 OpenSRF drone [opensrf.settings]
opensrf 7491 7486 0 16:09 ? 00:00:00 OpenSRF drone [opensrf.settings]
opensrf 7492 7486 0 16:09 ? 00:00:00 OpenSRF drone [opensrf.settings]
opensrf 7493 1 0 16:09 ? 00:00:00 OpenSRF controller [open-ils.cat]
opensrf 7495 1 0 16:09 ? 00:00:00 OpenSRF controller [open-ils.supercat]
opensrf 7497 1 0 16:09 ? 00:00:00 OpenSRF controller [open-ils.search]
opensrf 7499 1 0 16:09 ? 00:00:00 OpenSRF controller [open-ils.circ]
opensrf 7501 1 0 16:09 ? 00:00:00 OpenSRF controller [open-ils.actor]
opensrf 7502 1 0 16:09 ? 00:00:00 OpenSRF controller [open-ils.storage]
opensrf 7503 7495 0 16:09 ? 00:00:01 OpenSRF master [open-ils.supercat]
opensrf 7504 7495 0 16:09 ? 00:00:00 OpenSRF listener [open-ils.supercat]
opensrf 7505 7493 0 16:09 ? 00:00:01 OpenSRF master [open-ils.cat]
opensrf 7506 7497 0 16:09 ? 00:00:01 OpenSRF master [open-ils.search]
opensrf 7507 7497 0 16:09 ? 00:00:00 OpenSRF listener [open-ils.search]
opensrf 7509 1 0 16:09 ? 00:00:00 OpenSRF controller [open-ils.penalty]
opensrf 7510 7493 0 16:09 ? 00:00:00 OpenSRF listener [open-ils.cat]
opensrf 7512 1 0 16:09 ? 00:00:00 OpenSRF controller [open-ils.collections]
opensrf 7514 1 0 16:09 ? 00:00:00 OpenSRF controller [open-ils.ingest]
opensrf 7516 1 0 16:09 ? 00:00:00 OpenSRF controller [open-ils.reporter]
opensrf 7517 1 0 16:09 ? 00:00:00 OpenSRF controller [open-ils.permacrud]
opensrf 7520 7517 0 16:09 ? 00:00:01 OpenSRF master [open-ils.permacrud]
opensrf 7521 7517 0 16:09 ? 00:00:00 OpenSRF listener [open-ils.permacrud]
opensrf 7522 1 0 16:09 ? 00:00:00 OpenSRF controller [open-ils.fielder]
opensrf 7524 7499 0 16:09 ? 00:00:01 OpenSRF master [open-ils.circ]
opensrf 7525 7522 0 16:09 ? 00:00:01 OpenSRF master [open-ils.fielder]
opensrf 7526 7522 0 16:09 ? 00:00:00 OpenSRF listener [open-ils.fielder]
opensrf 7527 1 0 16:09 ? 00:00:00 OpenSRF controller [open-ils.vandelay]
opensrf 7529 7509 0 16:09 ? 00:00:01 OpenSRF master [open-ils.penalty]
opensrf 7531 7516 0 16:09 ? 00:00:01 OpenSRF master [open-ils.reporter]
opensrf 7532 7509 0 16:09 ? 00:00:00 OpenSRF listener [open-ils.penalty]
opensrf 7533 7516 0 16:09 ? 00:00:00 OpenSRF listener [open-ils.reporter]
opensrf 7534 7501 0 16:09 ? 00:00:01 OpenSRF master [open-ils.actor]
opensrf 7535 7501 0 16:09 ? 00:00:00 OpenSRF listener [open-ils.actor]
opensrf 7536 7502 1 16:09 ? 00:00:02 OpenSRF master [open-ils.storage]
opensrf 7537 7499 0 16:09 ? 00:00:00 OpenSRF listener [open-ils.circ]
opensrf 7538 7502 0 16:09 ? 00:00:00 OpenSRF listener [open-ils.storage]
opensrf 7539 7512 0 16:09 ? 00:00:01 OpenSRF master [open-ils.collections]
opensrf 7540 7512 0 16:09 ? 00:00:00 OpenSRF listener [open-ils.collections]
opensrf 7541 7514 0 16:09 ? 00:00:01 OpenSRF master [open-ils.ingest]
opensrf 7542 7527 0 16:09 ? 00:00:01 OpenSRF master [open-ils.vandelay]
opensrf 7543 7527 0 16:09 ? 00:00:00 OpenSRF listener [open-ils.vandelay]
opensrf 7544 7514 0 16:09 ? 00:00:00 OpenSRF listener [open-ils.ingest]
opensrf 7545 7503 0 16:09 ? 00:00:00 OpenSRF drone [open-ils.supercat]
opensrf 7546 7505 0 16:09 ? 00:00:00 OpenSRF drone [open-ils.cat]
opensrf 7547 7524 0 16:09 ? 00:00:00 OpenSRF drone [open-ils.circ]
opensrf 7548 7520 0 16:09 ? 00:00:00 OpenSRF drone [open-ils.permacrud]
opensrf 7549 7520 0 16:09 ? 00:00:00 OpenSRF drone [open-ils.permacrud]
opensrf 7550 7520 0 16:09 ? 00:00:00 OpenSRF drone [open-ils.permacrud]
opensrf 7551 7520 0 16:09 ? 00:00:00 OpenSRF drone [open-ils.permacrud]
opensrf 7552 7520 0 16:09 ? 00:00:00 OpenSRF drone [open-ils.permacrud]
opensrf 7553 7525 0 16:09 ? 00:00:00 OpenSRF drone [open-ils.fielder]
opensrf 7554 7525 0 16:09 ? 00:00:00 OpenSRF drone [open-ils.fielder]
opensrf 7555 7525 0 16:09 ? 00:00:00 OpenSRF drone [open-ils.fielder]
opensrf 7556 7525 0 16:09 ? 00:00:00 OpenSRF drone [open-ils.fielder]
opensrf 7557 7529 0 16:09 ? 00:00:00 OpenSRF drone [open-ils.penalty]
opensrf 7558 7525 0 16:09 ? 00:00:00 OpenSRF drone [open-ils.fielder]
opensrf 7559 7531 0 16:09 ? 00:00:00 OpenSRF drone [open-ils.reporter]
opensrf 7560 7534 0 16:09 ? 00:00:00 OpenSRF drone [open-ils.actor]
opensrf 7561 7539 0 16:09 ? 00:00:00 OpenSRF drone [open-ils.collections]
opensrf 7562 7541 0 16:09 ? 00:00:00 OpenSRF drone [open-ils.ingest]
opensrf 7563 7541 0 16:09 ? 00:00:00 OpenSRF drone [open-ils.ingest]
opensrf 7564 7541 0 16:09 ? 00:00:00 OpenSRF drone [open-ils.ingest]
opensrf 7565 7542 0 16:09 ? 00:00:00 OpenSRF drone [open-ils.vandelay]
opensrf 7566 7541 0 16:09 ? 00:00:00 OpenSRF drone [open-ils.ingest]
opensrf 7567 7541 0 16:09 ? 00:00:00 OpenSRF drone [open-ils.ingest]
opensrf 7568 7536 0 16:09 ? 00:00:00 OpenSRF drone [open-ils.storage]
opensrf 7569 7506 0 16:09 ? 00:00:00 OpenSRF drone [open-ils.search]
If not, please contact us.
===== Step 7: Start the OpenSRF C services and check for errors =====
Typically, you would use a command like this as user opensrf:
/openils/bin/osrf_ctl.sh -c /openils/conf/opensrf_core.xml -l -a start_c
You should get output resembling this:
Starting OpenSRF C (host=localhost)
If you do not, then please stop here and ask for help on either [[http://open-ils.org/listserv.php|OPEN-ILS-DEV]] or [[http://open-ils.org/irc.php|#evergreen]]. At a minimum, we will want to see a copy of your router log, osrfsys log and configuration files here:
/openils/var/log/*router.log
/openils/var/log/osrfsys.log
/openils/conf/opensrf.xml
/openils/conf/opensrf_core.xml
And of course, any on-screen errors. Though at this point, a copy of all files in /openils/var/log may be useful. You should not send us files if they contain passwords that you may use in production environments.
If the C services do appear to have started, then let's double check by grepping the log files for errors like so:
grep ERR /openils/var/log/*.log
If you encounter any output from that command, please stop here and contact the development community as stated above.
As a final sanity check, look for OpenSRF processes with a command like so:
ps -aef | grep -i opensrf
In addition to what you saw before from the Perl services, you should see output like this:
opensrf 7780 1 0 16:39 ? 00:00:00 OpenSRF System-C
opensrf 7781 7780 0 16:39 ? 00:00:00 OpenSRF Listener [opensrf.math]
opensrf 7784 7780 0 16:39 ? 00:00:00 OpenSRF Listener [opensrf.dbmath]
opensrf 7785 7780 0 16:39 ? 00:00:00 OpenSRF Listener [open-ils.auth]
opensrf 7786 7780 0 16:39 ? 00:00:00 OpenSRF Listener [open-ils.cstore]
opensrf 7787 7780 0 16:39 ? 00:00:00 OpenSRF Listener [open-ils.reporter-store]
opensrf 7789 7781 0 16:39 ? 00:00:00 OpenSRF Drone [opensrf.math]
opensrf 7792 7784 0 16:39 ? 00:00:00 OpenSRF Drone [opensrf.dbmath]
opensrf 7793 7785 0 16:40 ? 00:00:00 OpenSRF Drone [open-ils.auth]
opensrf 7794 7786 0 16:40 ? 00:00:00 OpenSRF Drone [open-ils.cstore]
opensrf 7796 7787 0 16:40 ? 00:00:00 OpenSRF Drone [open-ils.reporter-store]
If not, please contact us.
===== Step 8: Smoke test with autogen.sh =====
The Autogen tool will take some dynamic information from the database and generate static Javascript files for use by the OPAC and staff client. It is also able to refresh the proximity map between libraries for the purpose of efficiently routing hold requests.
As user opensrf, you invoke Autogen with a command like this:
/openils/bin/autogen.sh -c /openils/conf/opensrf_core.xml -u
You can expect output like this:
Updating fieldmapper
Updating web_fieldmapper
Updating OrgTree
removing OrgTree from the cache...
Updating OrgTree HTML
Updating locales selection HTML
Updating Search Groups
Refreshing proximity of org units
Successfully updated the organization proximity
Done
If you encounter any issues with Autogen, then please stop here and ask for help on either [[http://open-ils.org/listserv.php|OPEN-ILS-DEV]] or [[http://open-ils.org/irc.php|#evergreen]]. We will be interested in the usual log and configuration files:
/openils/var/log/*.log
/openils/conf/opensrf.xml
/openils/conf/opensrf_core.xml
And of course, please include any on-screen errors.
===== Step 9: Try to authenticate with the srfsh command-line OpenSRF client =====
As user opensrf, you should be able to invoke srfsh very simply like so:
srfsh
That command should give you a prompt which looks like this:
srfsh#
If it does not, then please stop here and ask for help on either [[http://open-ils.org/listserv.php|OPEN-ILS-DEV]] or [[http://open-ils.org/irc.php|#evergreen]]. At a minimum, we will want to see a copy of your router log, osrfsys log and configuration files here:
/openils/var/log/*router.log
/openils/var/log/osrfsys.log
/openils/conf/opensrf.xml
/openils/conf/opensrf_core.xml
/home/opensrf/.srfsh.xml
And of course, any on-screen errors. Though at this point, a copy of all files in /openils/var/log may be useful. You should not send us files if they contain passwords that you may use in production environments. The .srfsh.xml file may be located elsewhere depending on your operating system, but it should be in the home directory for your opensrf user.
~/.srfsh.xml
Assuming you have a srfsh prompt, try logging in, **substituting the admin account credentials you created during the install process** into the following command:
login myAdminUsername myAdminPassword
You should see output similar to this:
Received Data: "6f63ff5542da1fead4431c6c280efc75"
------------------------------------
Request Completed Successfully
Request Time in seconds: 0.018414
------------------------------------
Received Data: {
"ilsevent":0,
"textcode":"SUCCESS",
"desc":" ",
"pid":7793,
"stacktrace":"oils_auth.c:312",
"payload":{
"authtoken":"28804ebf99508496e2a4d2593aaa930e",
"authtime":420.000000
}
}
------------------------------------
Request Completed Successfully
Request Time in seconds: 0.552430
------------------------------------
Login Session: 28804ebf99508496e2a4d2593aaa930e. Session timeout: 420.000000
srfsh#
If not, please collect the log and configuration files previously mentioned and contact us. Also useful is the srfsh log, the location of which is defined in your ~/.srfsh.xml configuration file. By default, it is found here:
/openils/var/log/srfsh.log
Pressing Control+D or entering "exit" will terminate srfsh.
exit
===== Step 10: Start Apache and check for errors =====
Evergreen has Apache modules that expect to find a running OpenSRF system. Typically you could start Apache as user root with a command like
/etc/init.d/apache2 start
or
apache2ctl start
You may see output like this:
* Starting web server apache2
...done.
You can check for Apache processes with a command like this:
ps -aef | grep -i apache
and can expect output like this:
root 8662 1 3 22:25 ? 00:00:01 /usr/sbin/apache2 -k start
opensrf 8667 8662 3 22:25 ? 00:00:01 /usr/sbin/apache2 -k start
opensrf 8668 8662 3 22:25 ? 00:00:01 /usr/sbin/apache2 -k start
opensrf 8669 8662 3 22:25 ? 00:00:01 /usr/sbin/apache2 -k start
opensrf 8670 8662 3 22:25 ? 00:00:01 /usr/sbin/apache2 -k start
opensrf 8671 8662 3 22:25 ? 00:00:01 /usr/sbin/apache2 -k start
Upon a successful start of the Apache OpenSRF modules, there will be a new log file:
/openils/var/log/gateway.log
However, you should check all of the log files for errors:
grep ERR /openils/var/log/*.log
If you encounter any issues with Apache, then please stop here and ask for help on either [[http://open-ils.org/listserv.php|OPEN-ILS-DEV]] or [[http://open-ils.org/irc.php|#evergreen]]. In addition to your normal log and configuration files, we'll also want to see some pertinent Apache configuration files:
/etc/apache2/eg_vhost.conf
/etc/apache2/sites-available/eg.conf
/openils/var/log/*.log
/openils/conf/opensrf.xml
/openils/conf/opensrf_core.xml
And of course, please include any on-screen errors. The location of your Apache files may vary depending on your operating system. Again, you should not send us files if they contain passwords that you may use in production environments. It may also be useful to share your Apache logs:
/var/log/apache2/access.log
/var/log/apache2/error.log
===== Step 11: Try to authenticate with the staff client =====
===== Step 12: Try to authenticate with the OPAC =====
===== Bonus Step: Checking Ejabberd Account Credentials =====
You may have an ejabberd account problem, if you see an error like:
Checking Jabber connection for user router, domain public.localhost
Use of uninitialized value $@ in concatenation (.) or string at /usr/local/share/perl/5.10.1/OpenSRF/Transport/SlimJabber/Client.pm line 147, line 35.
* Error connecting to jabber:
SCALAR(0x1244e20)
Verify that the username/password combinations are set correctly in opensrf_core.xml, but you can also check with ejabberd to make sure that they were setup correctly in the first place.
As root:
ejabberdctl check_password ; echo $?
If the result is 0, then things are fine with that username/password on the given host. If the result is 1, then the credentials are not working.
You can change your ejabberd password with:
ejabberdctl change_password