Deploying multiple Rails apps with Passenger Standalone and systemd

Last week, we moved one of our test servers to Nginx + Phusion Passenger Standalone 5 aka Raptor, and this is where systemd came in handy.

Updated on 2015-03-12: Provide “current” app directory as argument to fix restarts

2014 has seen systemd making its way into even more Linux distributions, including CentOS 7, which we’re using at trademate for running our Rails application servers.

While I’m seriously opposed to systemd’s “let’s do it all” architecture – I don’t want to be forced to use an init system which includes an HTTP server by default, and that’s just one of the many problematic design decisions of systemd (let’s skip the journal) – there’s also some good parts from an administrator’s point of view.

Systemd unit files support a thing called instantiation, which allows you to run multiple instances of the same service with different configurations. In our deployment, we want to run multiple Rails apps, each within its own Passenger Standalone instance. To get instantiation, the service unit file name must end with an “@” sign. The service can then be started multiple times by providing an extra parameter, the instance name, when enabling the service.

TL;DR

I know how to work unit files – just get me the code

Installing Passenger Standalone

Passenger can be installed in different ways. We’re using the classic gem method, but other methods should work as well.

# gem install passenger

This will provide you with the passenger-config commandline tool, which can be used to get the passenger installation root:

# passenger-config --root
/opt/rubies/2.1.5/lib/ruby/gems/2.1.0/gems/passenger-5.0.2

Determining Rubygems environment

Use gem env to get the GEM_HOME environment variable, which may vary depending on your ruby installation:

# gem env
RubyGems Environment:
  - RUBYGEMS VERSION: 2.2.2
  - RUBY VERSION: 2.1.5 (2014-11-13 patchlevel 273) [x86_64-linux]
  - INSTALLATION DIRECTORY: /opt/rubies/2.1.5/lib/ruby/gems/2.1.0
[...]

In our case, GEM_HOME is /opt/rubies/2.1.5/lib/ruby/gems/2.1.0.

Deploying your Rails apps

All of your Rails apps should be deployed to some common location, each app occupying its own subfolder. For this guide, we’re going to use /apps to place our apps, so the application named demo would be deployed to /apps/demo, with its current release symlink at /apps/demo/current respectively.

Building the passenger systemd unit file

Using the following template, you can build your own passenger service unit file and place it in /etc/systemd/system/passenger@.service. Note the @ sign, which is important.

[Unit]
Description=Passenger Standalone Application Server
After=network.target
 
[Service]
Type=forking
PrivateTmp=yes
User=USERNAME
Group=GROUPNAME
WorkingDirectory=/apps/%i
#RuntimeDirectory=passenger
#RuntimeDirectoryMode=0755
PIDFile=/run/passenger/app.%i.pid
Environment="PATH=/opt/rubies/2.1.5/bin:/usr/local/bin:/usr/bin"
Environment="GEM_HOME=/opt/rubies/2.1.5/lib/ruby/gems/2.1.0"
Environment="GEM_PATH=/opt/rubies/2.1.5/lib/ruby/gems/2.1.0"
Environment="PASSENGER_INSTANCE_REGISTRY_DIR=/run/passenger"
ExecStart=/opt/rubies/2.1.5/lib/ruby/gems/2.1.0/gems/passenger-5.0.2/bin/passenger start current --daemonize --instance-registry-dir /run/passenger --socket /run/passenger/app.%i.sock --pid-file /run/passenger/app.%i.pid --log-file /apps/%i/shared/log/passenger.log --environment production --max-pool-size=16
ExecReload=/opt/rubies/2.1.5/lib/ruby/gems/2.1.0/gems/passenger-5.0.2/bin/passenger-config restart-app /apps/%i
ExecStop=/opt/rubies/2.1.5/lib/ruby/gems/2.1.0/gems/passenger-5.0.2/bin/passenger stop --pid-file /run/passenger/app.%i.pid
 
[Install]
WantedBy=multi-user.target

Depending on the Ruby deployment method you’re using, it may be necessary to change some of the paths and/or script names to use your version manager’s wrapper scripts.

  • First, you should set the User and Group options to the server user which you want to run the apps as.
  • The WorkingDirectory should be set to the Rails app root. Systemd will replace %i with the application name upon instantiation of the service.
  • The Environment options should be used to define PATH, GEM_HOME and GEM_PATH. Insert the values from your local installation and gem env.
  • Replace the passenger and passenger-config binaries’ path in ExecStart, ExecReload and ExecStop with the path provided by passenger-config --root.

We’ll be using Nginx as SSL frontend and have it connect to the Passenger Standalone instances via UNIX sockets. This way, we can have a single UNIX socket per Rails app, which can be named like the Rails application, so we’re able to run multiple passenger instances without configuring a separate TCP listening port per instance.

Providing the service runtime directory

Passenger requires a filesystem location where it can place its PID files and the UNIX socket. By default, Passenger uses the Rails app’s tmp dir for its PIDs. This is problematic in conjunction with Capistrano (or any other deployment system which uses release symlinks), as after a deploy the tmp dir points to another location.

Therefore, we use the runtime directory functionality provided by systemd. Depending on your systemd version, two different approaches are required.

systemd >= 211

You’re lucky! Simply uncomment the RuntimeDirectory and RuntimeDirectoryMode options in your unit file. Systemd will automatically provide the runtime directory.

systemd < 211

CentOS 7 is currently using systemd 208, where the RuntimeDirectory options are not yet available. However, there’s systemd-tmpfiles which provides the same functionality. To enable it, create /etc/tmpfiles.d/passenger.conf with the following content:

#Type Path Mode UID GID Age Argument
d /run/passenger 0755 username groupname

Of course, username and groupname must be replaced by the user Passenger is running as. After a restart, the /run/passenger dir will have been created. If you don’t want to reboot your machine, you can create the directory manually for now:

# mkdir /run/passenger
# chown username:groupname /run/passenger
# chmod 755 /run/passenger

The tmpfiles.d will ensure that it is recreated on reboot, as /run is a tmpfs filesystem.

Running multiple applications

This is where the magic happens. To start the demo application, run the following command:

# systemctl start passenger@demo

If everything went well, passenger should be up and running, which can be checked using

# systemctl status passenger@demo

If something goes wrong, check the journal:

# journalctl -xn

To stop the app:

# systemctl stop passenger@demo

To have the Rails application started on boot:

# systemctl enable passenger@demo

All of your other apps can be started and queried the same way, replacing the instance name demo with the app’s actual name. If you have a second app called foobar, you would enable it using

# systemctl enable passenger@foobar

Reloading

Passenger Standalone doesn’t support the classic touch restart.txt reloading method. To reload an application:

# systemctl reload passenger@demo

Alternatively, you can use the passenger-config tool for restarts while not root:

$ PASSENGER_INSTANCE_REGISTRY_DIR=/run/passenger passenger-config restart-app /apps/demo

Instance registry, passenger-config and passenger-status

Passenger uses the so-called instance registry directory to keep track of the application instances currently running. By default, /tmp is used. As the PrivateTmp option is enabled in the service file to improve security, Passenger management tools like passenger-status cannot access the instance registry any more. To avoid this, Passenger is configured to store its instance registry in the runtime directory provided by systemd.

Instance registry entries are named passenger.*. In order to prevent name clashes, per-app UNIX sockets and PID filenames are prefixed with app., so you shouldn’t run into problems when naming one of your apps passenger by coincidence.

To use passenger-status or passenger-config, you have to provide the PASSENGER_INSTANCE_REGISTRY_DIR env variable:

# PASSENGER_INSTANCE_REGISTRY_DIR=/run/passenger passenger-status
Version : 5.0.2
Date    : 2015-01-23 20:38:19 +0100
Instance: HExxsQpf (Phusion_Passenger/5.0.2)

----------- General information -----------
Max pool size : 2
Processes     : 1
Requests in top-level queue : 0

----------- Application groups -----------
/apps/demo/releases/20150121135417#default:
  App root: /apps/demo/releases/20150121135417
  Requests in queue: 0
  * PID: 15101   Sessions: 0       Processed: 2       Uptime: 14m 5s
    CPU: 0%      Memory  : 158M    Last used: 14m 1s ago

Integrating with Nginx

Configuration is as usual, except for the backend server, which is accessed via UNIX socket instead of TCP:

location / {
    proxy_pass http://unix:/run/passenger/app.demo.sock:;
}

The various applications can then be mounted on separate virtual servers.

Tuning

As you might have noticed, Passenger runtime options like MaxPoolSize are currently hard-coded in the unit file. While this is okay for our current requirements, you might want to customize some of the options on an per-application basis.

Adding a systemd environment file

Services can load environment variables from a file defined by the EnvironmentFile option.

[Service]
EnvironmentFile=-/apps/%i/current/config/passenger.env

By prepending the path with a hyphen, you can tell systemd not to complain should the environment file not exist.

In the environment file, you can define well, … environment variables:

PASSENGER_OPTS="--max-pool-size 2 --min-instances 1"

These variables can then be supplied to the ExecStart et al. directives:

ExecStart=/opt/rubies/2.1.5/lib/ruby/gems/2.1.0/gems/passenger-5.0.2/bin/passenger start current --daemonize --instance-registry-dir /run/passenger --socket /run/passenger/app.%i.sock --pid-file /run/passenger/app.%i.pid --log-file /apps/%i/shared/log/passenger.log --environment production $PASSENGER_OPTS

This way, each app can have its individual configuration options, while all apps share one common unit file.

Running different Ruby versions… and tricking systemd

It’s even possible to customize the Ruby version on an per-app basis by moving PATH, GEM_HOME, GEM_PATH and PASSENGER_ROOT into the EnvironmentFile:

PATH="/opt/rubies/1.9.3-p194/bin:/usr/local/bin:/usr/bin"
PASSENGER_ROOT="/opt/rubies/1.9.3-p194/lib/ruby/gems/1.9.1/gems/passenger-4.0.24"
GEM_HOME="/opt/rubies/1.9.3-p194/lib/ruby/gems/1.9.1"
GEM_PATH="/opt/rubies/1.9.3-p194/lib/ruby/gems/1.9.1"

Variables defined by EnvironmentFile will override variables defined by Environment.

However, systemd doesn’t allow environment variable interpolation in the actual ExecStart command, which is required to be an absolute path. Environment variables may only be used in the arguments following the command. By wrapping the commandline into a bash exec call, we get env variable interpolation back and can set the executable paths dynamically through the EnvironmentFile.

[Service]
Environment="PATH=/opt/rubies/2.1.5/bin:/usr/local/bin:/usr/bin"
Environment="GEM_HOME=/opt/rubies/2.1.5/lib/ruby/gems/2.1.0"
Environment="GEM_PATH=/opt/rubies/2.1.5/lib/ruby/gems/2.1.0"
Environment="PASSENGER_ROOT=/opt/rubies/2.1.5/lib/ruby/gems/2.1.0/gems/passenger-5.0.2"
Environment="PASSENGER_INSTANCE_REGISTRY_DIR=/run/passenger"
EnvironmentFile=-/apps/%i/current/config/passenger.env
ExecStart=/usr/bin/bash -c 'exec ${PASSENGER_ROOT}/bin/passenger start current --daemonize --instance-registry-dir /run/passenger --socket /run/passenger/app.%i.sock --pid-file /run/passenger/app.%i.pid --log-file /apps/%i/shared/log/passenger.log --environment production $PASSENGER_OPTS'
ExecReload=/usr/bin/bash -c 'exec ${PASSENGER_ROOT}/bin/passenger-config restart-app /apps/%i'
ExecStop=/usr/bin/bash -c 'exec ${PASSENGER_ROOT}/bin/passenger stop --pid-file /run/passenger/app.%i.pid'

Code

A unit file template which integrates all of the above features can be downloaded at https://github.com/mtgrosser/passenger-systemd.

Leave a Reply

Your email address will not be published. Required fields are marked *