Node.js + PM2 + Nginx + Redis on CentOS 7

This is a quick setup guide for the application stack I have been using recently. Most of my latest work has been a throwback to the first server-side programming I did in the 90’s – javascript – except this time it’s Node.js instead of Netscape Communication Server. In this setup PM2 is used to manage the Node process running as an arbitrary user, running on an unprivileged port. This means that the application can be restarted without root credentials. The front-end is served by Nginx and it does need to be started as root because it runs on the privileged ports 80 and 443 in this use case. It also gives us a lot of the built in features that Nginx gives you on the front end, like serving all your content over SSL – for free using Let’s Encrypt event. My caching needs are provided by Redis.

Here is the setup – run as root or use sudo.

Firewalld

Enable the firewalld service and only allow http/s traffic to the server – in addition to the default of just ssh.

#!/bin/bash

# enable on book
systemctl enable firewalld

# (re)start the service
(service firewalld status > /dev/null && service firewalld restart) || service firewalld start

# add the http and http services and reload
firewall-cmd --permanent --zone=public --add-service=http
firewall-cmd --permanent --zone=public --add-service=https
firewall-cmd --reload

Letsencrypt

Use Letsencrypt for free SSL certificates.

yum -y install letsencrypt
openssl dhparam -out /etc/ssl/certs/dhparam.pem 2048

Yum Cron

#!/bin/bash

# make sure the YUM_CRON_EMAIL is set
if [[ -z $YUM_CRON_EMAIL ]]; then
  echo "You must specify an email using \$YUM_CRON_EMAIL";
else
  # install and enable, plus patch for bug fixing
  yum -y install yum-cron patch
  chkconfig yum-cron on

  # configure via sed replacements
  sed -i "s|^email_to = root|email_to = ${YUM_CRON_EMAIL}|" /etc/yum/yum-cron.conf
  sed -i 's|^update_messages = no|update_messages = yes|' /etc/yum/yum-cron.conf
  sed -i 's|^download_updates = no|download_updates = yes|' /etc/yum/yum-cron.conf
  sed -i 's|^apply_updates = no|apply_updates = yes|' /etc/yum/yum-cron.conf
  sed -i 's|^emit_via = stdio|emit_via = email|' /etc/yum/yum-cron.conf

  sed -i "s|^email_to = root|email_to = ${YUM_CRON_EMAIL}|" /etc/yum/yum-cron-hourly.conf
  sed -i 's|^update_cmd = default|update_cmd = security|' /etc/yum/yum-cron-hourly.conf
  sed -i 's|^update_messages = no|update_messages = yes|' /etc/yum/yum-cron-hourly.conf
  sed -i 's|^download_updates = no|download_updates = yes|' /etc/yum/yum-cron-hourly.conf
  sed -i 's|^apply_updates = no|apply_updates = yes|' /etc/yum/yum-cron-hourly.conf
  sed -i 's|^emit_via = stdio|emit_via = email|' /etc/yum/yum-cron-hourly.conf   

  egrep '^email_to|^update_messages|^download_updates|^apply_updates|^emit_via' /etc/yum/yum-cron.conf
  egrep '^email_to|^update_cmd|^update_messages|^download_updates|^apply_updates|^emit_via' /etc/yum/yum-cron-hourly.conf
  
  # fix bug in yum-cron nightly updates
  if [[ $(grep -q "# success, dependencies resolved" /usr/sbin/yum-cron) -ne 0 ]]; then
    patch /usr/sbin/yum-cron <<PATCHFILE
--- yum-cron.orig 2016-10-23 19:24:57.099859931 +0000
+++ yum-cron  2016-10-23 19:27:58.048784006 +0000
@@ -504,7 +504,13 @@
     except yum.Errors.RepoError, e:
       self.emitCheckFailed("%s" %(e,))
       sys.exit()
-        if res != 2:
+        if res == 0:
+            # success, empty transaction
+            sys.exit(0)
+        elif res == 2:
+            # success, dependencies resolved
+            pass
+        else:
       self.emitCheckFailed("Failed to build transaction: %s" %(str.join("\n", resmsg),))
       sys.exit(1)
PATCHFILE
  fi  
  # (re)start the yum-cron service
  (service yum-cron status > /dev/null && service yum-cron restart) || service yum-cron start
fi

Install Nginx

We are going to use the “mainline” repo to get HTTP2 support. I like to create a conf file in “sites-available” that is linked to “sites-enabled” so I can disable things easily – this is enabled by adding a file under /etc/nginx/conf.d.

#!/bin/bash

# import src utility
if [[ -z $(type -t src) ]]; then
  source <(curl -sL https://www.doublesharp.com/src)
fi

src osname
src osversion

cat <<REPO > /etc/yum.repos.d/nginx.repo
[nginx]
name=nginx repo
# default repo
#baseurl=http://nginx.org/packages/$(osname)/$(osversion)/\$basearch/
# mainline "dev" repo for http2 support
baseurl=http://nginx.org/packages/mainline/$(osname)/$(osversion)/\$basearch/
gpgcheck=0
enabled=1
REPO

#install nginx
yum install -y nginx

# turn on for reboots
systemctl enable nginx

mkdir -p /etc/nginx/includes
mkdir -p /etc/nginx/sites-enabled
mkdir -p /etc/nginx/sites-available
mkdir -p /etc/nginx/streams-enabled
mkdir -p /etc/nginx/streams-available

# use a conf file to include our sites-enabled conf files
cat <<SITESENABLED > /etc/nginx/includes/sites-enabled.conf
include                 /etc/nginx/sites-enabled/*.conf;
SITESENABLED

[[ -f "/etc/nginx/conf.d/_.sites-enabled.conf" ]] || ln -s /etc/nginx/includes/sites-enabled.conf /etc/nginx/conf.d/_.sites-enabled.conf

# enable httpd in selinux
semanage permissive -a httpd_t

cat <<NGINX_CONF > /etc/nginx/nginx.conf
user                    nginx;
worker_processes        auto;

error_log               /var/log/nginx/error.log warn;
pid                     /var/run/nginx.pid;

worker_rlimit_nofile    100000; 

events {
  # determines how much clients will be served per worker
  # max clients = worker_connections * worker_processes
  # max clients is also limited by the number of socket connections available on the system (~64k)
  worker_connections      100000;

  # optmized to serve many clients with each thread, essential for linux
  use                     epoll;

  # accept as many connections as possible, may flood worker connections if set too low
  multi_accept on;
}

# web servers / virtual hosts
http {
  include                 /etc/nginx/mime.types;
  default_type            application/octet-stream;

  log_format              main    '\$remote_addr - \$remote_user [\$time_local] "\$request" '
                                  '\$status \$body_bytes_sent "\$http_referer" '
                                  '"\$http_user_agent" "\$http_x_forwarded_for"';

  access_log              /var/log/nginx/access.log combined flush=1m buffer=128k;

  # cache informations about FDs, frequently accessed files
  # can boost performance, but you need to test those values
  open_file_cache         max=200000 inactive=20s;
  open_file_cache_valid   30s;
  open_file_cache_min_uses 2;
  open_file_cache_errors  on;

  # send headers in one peace, its better then sending them one by one
  tcp_nopush              on;
  
  # don't buffer data sent, good for small data bursts in real time
  tcp_nodelay             on;
  
  # server will close connection after this time
  keepalive_timeout       30;
  
  # allow the server to close connection on non responding client, this will free up memory
  reset_timedout_connection on;
  
  # request timed out -- default 60
  client_body_timeout     10;
  
  # if client stop responding, free up memory -- default 60
  send_timeout            2;
  
  # reduce the data that needs to be sent over network
  gzip                    on;
  gzip_min_length         10240;
  gzip_proxied            expired no-cache no-store private auth;
  gzip_types              text/plain text/css text/xml text/javascript application/x-javascript application/xml;
  gzip_disable            "MSIE [1-6]\.";

  proxy_buffer_size       128k;
  proxy_buffers           64 256k;
  proxy_busy_buffers_size 256k;
  proxy_ignore_client_abort on;

  include                 /etc/nginx/conf.d/*.conf;
}

# load balancer streams
stream {
  include                 /etc/nginx/streams-enabled/*.conf;
}
NGINX_CONF


# create a virtual server conf file that is in sites-available
cat <<NGINX_HOST > /etc/nginx/sites-available/myapp.conf
upstream myapp {
        # our app will be on localhost port 3000, but you can change this here
        server                  127.0.0.1:3000 fail_timeout=0;
}
 
server {
        listen                  80;
        server_name             myapp.example.com;
 
        location / {
                proxy_set_header        Host \$host:\$server_port;
                proxy_set_header        X-Real-IP \$remote_addr;
                proxy_set_header        X-Forwarded-For \$proxy_add_x_forwarded_for;
                proxy_set_header        X-Forwarded-Proto \$scheme;
 
                proxy_pass              http://myapp;
        }
}
NGINX_HOST
 
# link this conf to sites-enabled. it's important to use the full path
#ln -s /etc/nginx/sites-available/myapp.conf /etc/nginx/sites-enabled/myapp.conf

nginx -t && (service nginx status > /dev/null && service nginx restart)

Install Redis

To install Redis with yum, first you need to install EPEL. Once the installed, you will have access to the repository containing the Redis install.

#!/bin/bash

# install the EPEL repo to access Redis
yum install -y epel-release
yum install -y redis

# fix redis background saves on low memory
sysctl vm.overcommit_memory=1 && cat <<SYSCTL_MEM > /etc/sysctl.d/88-vm.overcommit_memory.conf
vm.overcommit_memory = 1
SYSCTL_MEM

# increase max connections
sysctl -w net.core.somaxconn=65535 && cat <<SYSCTL_CONN > /etc/sysctl.d/88-net.core.somaxconn.conf
net.core.somaxconn = 65535
SYSCTL_CONN

sysctl -w fs.file-max=100000 && cat <<SYSCTL_FILEMAX > /etc/sysctl.d/88-fs.file-max.conf
fs.file-max = 100000
SYSCTL_FILEMAX

sed -i "s|^tcp-backlog [[:digit:]]\+|tcp-backlog 65535|" /etc/redis.conf

# enable redis service on reboot
systemctl enable redis

# start service
(service redis status > /dev/null && service redis restart) || service redis start

Install Node.js & PM2

We want to install Node.js and then the PM2 package globally so that it can be accessed by other users.

#!/bin/bash

# make sure the MALDET_EMAIL is set
if [[ -z $SRC_NODE_VERSION ]]; then
  echo "You must specify a node version using \$SRC_NODE_VERSION";
else
  # Select node version to install
  curl --silent --location https://rpm.nodesource.com/setup_$SRC_NODE_VERSION.x | bash -
  
  # install via yum
  yum install -y git gcc-c++ make nodejs
fi

# PM2 - install as global
npm install [email protected] -g

Create “appuser”

As root, create a new user named “appuser”, or whatever you want your app user to be named. This could even be the default centos@/ec2-user@/etc that many hosts provide.

adduser appuser
passwd appuser

Create PM2 service as “appuser”

Log in as the “appuser” user and create the Node app in your home directory. This directory should be owned by the “appuser”. In this case we assume the server is going to be listening on localhost port 3000, which means we can manage it with pm2 without having root permissions.

mkdir ~/apps
cd /apps

# create your app here, git clone, whatever
# we assume the app is in ~/apps/myapp/server.js

pm2 start ~/apps/myapp/server.js --name=myapp
pm2 status myapp
pm2 restart myapp

You may also like...

Leave a Reply

Your email address will not be published. Required fields are marked *