The post Docker-CE on CentOS 7 appeared first on Justin Silver.
]]>Install Docker-CE (not just “docker) to get the latest version on CentOS.
yum install -y yum-utils device-mapper-persistent-data lvm2 yum-config-manager --add-repo https://download.docker.com/linux/centos/docker-ce.repo yum makecache fast yum install -y docker-ce chkconfig docker on service docker start
Update firewalld to allow host/container traffic.
Restart firewalld to pick up the changes.
# trust the docker interface firewall-cmd --permanent --zone=trusted --change-interface=docker0 # accept IPv4 traffic firewall-cmd --permanent --direct --add-rule ipv4 filter INPUT 4 -i docker0 -j ACCEPT # any ports on the host you want to access from the containers (strapi port 1337 here) firewall-cmd --permanent --zone=trusted --add-port=1337/tcp firewall-cmd --reload service docker restart
Create a group named “docker” and add any users that are allowed to create containers.
groupadd docker # allow users to access docker by adding them to the docker group # usermod -aG docker $USER
Use an entrypoint file to create an entry in /etc/hosts to point to the host. Requires ping
and ip
on the machine, on Debian these are found in the apt packages inetutils-ping
and iproute
.
#!/bin/bash # fix for linux hosts HOST_DOMAIN="host.docker.internal" ping -q -c1 $HOST_DOMAIN > /dev/null 2>&1 if [ $? -ne 0 ]; then HOST_IP=$(ip route | awk 'NR==1 {print $3}') echo -e "$HOST_IP\t$HOST_DOMAIN" >> /etc/hosts fi exec "$@"
The post Docker-CE on CentOS 7 appeared first on Justin Silver.
]]>The post MongoDB Clustering for CentOS 7 appeared first on Justin Silver.
]]>A quick guide to installing a MongoDB cluster on CentOS 7 hosts.
10.0.10.1 config01 10.0.20.1 shard01 10.0.20.2 shard02 10.0.30.1 app01
First make sure the system is up to date and install the MongoDB binaries via the Mongo yum repository:
# update via yum yum update -y # install the mongodb repository cat <<REPO > /etc/yum.repos.d/mongodb.repo [mongodb] name=MongoDB Repository baseurl=http://downloads-distro.mongodb.org/repo/redhat/os/x86_64/ gpgcheck=0 enabled=1 REPO # install mongodb yum install -y mongodb-org # change the block side - NOTE: use the /dev that hosts the mongo data blockdev --setra 256 /dev/xvda1 # disable access times sed -rn 's/( \/ \s*(ext[[:digit:]]|xfs)\s*defaults) /\1,noatime /p' -i /etc/fstab # remount mount -o remount /
# mongod.conf: config server(s) # use as config server, changes port to 27019 configsvr=true # data directory dbpath=/var/lib/mongo ####### STANDARD CONFIG ####### # fork and run in background fork=true # location of pidfile pidfilepath=/var/run/mongodb/mongod.pid # where to log logpath=/var/log/mongodb/mongod.log # append to log, don't create a new one on restart logappend=true
# mongod.conf: shard server(s) # shard server, changes the port number to 27018 shardsvr=true # data directory dbpath=/var/lib/mongo ####### STANDARD CONFIG ####### # fork and run in background fork=true # location of pidfile pidfilepath=/var/run/mongodb/mongod.pid # where to log logpath=/var/log/mongodb/mongod.log # append to log, don't create a new one on restart logappend=true
# mongos.conf: query server(s) # config server IP and port, comma delimited if there is more than one #configdb=config01:27019,config02:27019,... configdb=config01:27019 ####### STANDARD CONFIG ####### # fork and run in background fork=true # location of pidfile pidfilepath=/var/run/mongodb/mongos.pid # where to log logpath=/var/log/mongodb/mongos.log # append to log, don't create a new one on restart logappend=true
We need to make a few more changes to make sure that the Query Router uses mongos
– not mongod
– when the MongoDB service starts. We also never want this server to start as a MongoDB node by accident, so we will rename the initialization script and make a few edits.
mv /etc/init.d/mongod /etc/init.d/mongos vi /etc/init.d/mongos
In the mongos script you want to look for and replace the following lines
# Old - uses mongod.conf CONFIGFILE="/etc/mongod.conf" # New - uses mongos.conf CONFIGFILE="/etc/mongos.conf"# Old - uses mongod SYSCONFIG="/etc/sysconfig/mongod" # New - uses mongos SYSCONFIG="/etc/sysconfig/mongos" # replace in vi/vim with: :%s/sysconfig\/mongod/sysconfig\/mongos/gBelow this I placed a new variable for the lockfile and updated all the instances of the hard coded values.
# replace in vi/vim with: :%s/\var\/lock\/subsys\/mongod/"$LOCKFILE"/g # now add this, so the variable value isn't overwritten by the regex above LOCKFILE="/var/lock/subsys/mongos"Next we want to change the $mongod to $mongos.
# Old - uses mongod mongod=${MONGOD-/usr/bin/mongod} # New - uses mongos mongos=${MONGOD-/usr/bin/mongos} # replace in vi/vim using :%s/\$mongod/$mongos/g # replace the status text in vi/vim with: :%s/mongod: "/mongos: "/gLeave the MONGO_USER and MONGO_GROUP as mongod to prevent permissions issues.
There is also a bug in the script where the path to the binary is used directly instead of the variable for
stop)
, called byservice mongod stop
, so we need to fix it as well.# Old - uses path killproc -p "$PIDFILE" -d 300 /usr/bin/mongod # New - uses variable killproc -p "$PIDFILE" -d 300 $mongos # replace the status text in vi/vim with: :%s/\/usr/bin\/mongod/$mongos/gThe full customized mongos startup script follows.
#!/bin/bash # mongos - Startup script for mongos # chkconfig: 35 85 15 # description: Mongo is a scalable, document-oriented database. # processname: mongos # config: /etc/mongos.conf # pidfile: /var/run/mongodb/mongos.pid . /etc/rc.d/init.d/functions # things from mongos.conf get there by mongos reading it # NOTE: if you change any OPTIONS here, you get what you pay for: # this script assumes all options are in the config file. CONFIGFILE="/etc/mongos.conf" OPTIONS=" -f $CONFIGFILE" SYSCONFIG="/etc/sysconfig/mongos" LOCKFILE="/var/lock/subsys/mongos" # FIXME: 1.9.x has a --shutdown flag that parses the config file and # shuts down the correct running pid, but that's unavailable in 1.8 # for now. This can go away when this script stops supporting 1.8. DBPATH=`awk -F'[:=]' -v IGNORECASE=1 '/^[[:blank:]]*dbpath[[:blank:]]*[:=][[:blank:]]*/{print $2}' "$CONFIGFILE" | tr -d '[:blank:]'` PIDFILE=`awk -F'[:=]' -v IGNORECASE=1 '/^[[:blank:]]*pidfilepath[[:blank:]]*[:=][[:blank:]]*/{print $2}' "$CONFIGFILE" | tr -d '[:blank:]'` PIDDIR=`dirname $PIDFILE` mongos=${MONGOD-/usr/bin/mongos} MONGO_USER=mongod MONGO_GROUP=mongod if [ -f "$SYSCONFIG" ]; then . "$SYSCONFIG" fi # Handle NUMA access to CPUs (SERVER-3574) # This verifies the existence of numactl as well as testing that the command works NUMACTL_ARGS="--interleave=all" if which numactl >/dev/null 2>/dev/null && numactl $NUMACTL_ARGS ls / >/dev/null 2>/dev/null then NUMACTL="numactl $NUMACTL_ARGS" else NUMACTL="" fi start() { # Make sure the default pidfile directory exists if [ ! -d $PIDDIR ]; then install -d -m 0755 -o $MONGO_USER -g $MONGO_GROUP $PIDDIR fi # Recommended ulimit values for mongod or mongos # See http://docs.mongodb.org/manual/reference/ulimit/#recommended-settings # ulimit -f unlimited ulimit -t unlimited ulimit -v unlimited ulimit -n 64000 ulimit -m unlimited ulimit -u 64000 echo -n $"Starting mongos: " daemon --user "$MONGO_USER" --check $mongos "$NUMACTL $mongos $OPTIONS >/dev/null 2>&1" RETVAL=$? echo [ $RETVAL -eq 0 ] && touch "$LOCKFILE" } stop() { echo -n $"Stopping mongos: " killproc -p "$PIDFILE" -d 300 $mongos RETVAL=$? echo [ $RETVAL -eq 0 ] && rm -f "$LOCKFILE" } restart () { stop start } RETVAL=0 case "$1" in start) start ;; stop) stop ;; restart|reload|force-reload) restart ;; condrestart) [ -f "$LOCKFILE" ] && restart || : ;; status) status $mongos RETVAL=$? ;; *) echo "Usage: $0 {start|stop|status|restart|reload|force-reload|condrestart}" RETVAL=1 esac exit $RETVAL
Now we’re ready to add the shards, which we do from the Query Router. So either log on to MongoDB using mongo
from the command line, or from a different server use mongo --host app01
.
use admin; db.runCommand( { addShard: "shard01:27018", name: "shard01" } ); { "shardAdded" : "shard01", "ok" : 1 } db.runCommand( { addShard: "shard02:27018", name: "shard02" } ); { "shardAdded" : "shard02", "ok" : 1 }db.runCommand( { listShards: 1 } ); { "shards" : [ { "_id" : "shard01", "host" : "shard01:27018" }, { "_id" : "shard02", "host" : "shard02:27018" } ], "ok" : 1 }
use sharded_db; sh.enableSharding( "sharded_db" ); { "ok" : 1 } db.sharded_db.ensureIndex( { _id : "hashed" } ); { "raw" : { "shard01:27018" : { "createdCollectionAutomatically" : true, "numIndexesBefore" : 1, "numIndexesAfter" : 2, "ok" : 1 } }, "ok" : 1 } sh.shardCollection( "sharded_db.sharded_collection", { "_id": "hashed" } ); { "collectionsharded" : "sharded_db.sharded_collection", "ok" : 1 }for ( var i=0; i<=100; i++ ) db.sharded_collection.insert( { x : i } ); WriteResult({ "nInserted" : 1 }) db.sharded_collection.find() { "_id" : ObjectId("571ac1ded785cd0adda63b59"), "x" : 1 } { "_id" : ObjectId("571ac1ded785cd0adda63b5b"), "x" : 3 } { "_id" : ObjectId("571ac1ded785cd0adda63b5a"), "x" : 2 } { "_id" : ObjectId("571ac1ded785cd0adda63b5d"), "x" : 5 } { "_id" : ObjectId("571ac1ded785cd0adda63b5c"), "x" : 4 } { "_id" : ObjectId("571ac1ded785cd0adda63b5e"), "x" : 6 } { "_id" : ObjectId("571ac1ded785cd0adda63b60"), "x" : 8 } { "_id" : ObjectId("571ac1ded785cd0adda63b5f"), "x" : 7 } { "_id" : ObjectId("571ac1ded785cd0adda63b62"), "x" : 10 } { "_id" : ObjectId("571ac1ded785cd0adda63b61"), "x" : 9 } { "_id" : ObjectId("571ac1ded785cd0adda63b69"), "x" : 17 } { "_id" : ObjectId("571ac1ded785cd0adda63b63"), "x" : 11 } { "_id" : ObjectId("571ac1ded785cd0adda63b6a"), "x" : 18 } { "_id" : ObjectId("571ac1ded785cd0adda63b64"), "x" : 12 } { "_id" : ObjectId("571ac1ded785cd0adda63b6d"), "x" : 21 } { "_id" : ObjectId("571ac1ded785cd0adda63b65"), "x" : 13 } { "_id" : ObjectId("571ac1ded785cd0adda63b6e"), "x" : 22 } { "_id" : ObjectId("571ac1ded785cd0adda63b66"), "x" : 14 } { "_id" : ObjectId("571ac1ded785cd0adda63b70"), "x" : 24 } { "_id" : ObjectId("571ac1ded785cd0adda63b67"), "x" : 15 } Type "it" for more
sh.status(); --- Sharding Status --- sharding version: { "_id" : 1, "version" : 4, "minCompatibleVersion" : 4, "currentVersion" : 5, "clusterId" : ObjectId("571a019ce3c89d90cb6e1715") } shards: { "_id" : "shard01", "host" : "shard01:27018" } { "_id" : "shard02", "host" : "shard02:27018" } databases: { "_id" : "admin", "partitioned" : false, "primary" : "config" } { "_id" : "sharded_db", "partitioned" : true, "primary" : "shard01" } sharded_db.sharded_collection shard key: { "_id" : "hashed" } chunks: shard01 2 shard02 2 { "_id" : { "$minKey" : 1 } } -->> { "_id" : NumberLong("-4611686018427387902") } on : shard01 Timestamp(2, 2) { "_id" : NumberLong("-4611686018427387902") } -->> { "_id" : NumberLong(0) } on : shard01 Timestamp(2, 3) { "_id" : NumberLong(0) } -->> { "_id" : NumberLong("4611686018427387902") } on : shard02 Timestamp(2, 4) { "_id" : NumberLong("4611686018427387902") } -->> { "_id" : { "$maxKey" : 1 } } on : shard02 Timestamp(2, 5)
db.dropDatabase(); { "dropped" : "sharded_db", "ok" : 1 }
The post MongoDB Clustering for CentOS 7 appeared first on Justin Silver.
]]>The post NGINX Configuration Monitor appeared first on Justin Silver.
]]>I wanted a way to quickly distribute configuration files to my servers and have NGINX automatically reload. I found a solution for Debian servers and adapted it for CentOS 7 here. You will first create a bash script, make it executable, then call it from a systemd service. The script uses inotifywait to monitor the /etc/nginx/sites-enabled directory for changes and reloads NGINX if the configuration is valid.
#!/bin/bash # Check inotify-tools is installed or not rpm -qa | grep -q inotify-tools &> /dev/null if [ $? -ne 0 ] then echo "Installing inotify-tools, please wait..." yum -y install inotify-tools fi while true do inotifywait --exclude .swp -e create -e modify -e delete -e move /etc/nginx/sites-enabled # Check NGINX Configuration Test # Only Reload NGINX If NGINX Configuration Test Pass nginx -t if [ $? -eq 0 ] then echo "Reloading Nginx Configuration" service nginx reload fi done
chmod +x /usr/local/bin/nginx-monitor
[Unit] Description=Nginx Config Monitor Service After=nginx.service [Service] Type=simple ExecStart=/usr/local/bin/nginx-monitor Restart=on-abort [Install] WantedBy=multi-user.target
chmod 755 /etc/systemd/system/nginx-monitor.service # reload systemd services systemctl daemon-reload # start the service service nginx-monitor start # load after reboot chkconfig nginx-monitor on
Adapted for CentOS from Auto Reload NGINX.
The post NGINX Configuration Monitor appeared first on Justin Silver.
]]>The post LEMP: CentOS 7, NGINX, PHP7, and Redis for WordPress appeared first on Justin Silver.
]]>Scripts to setup a WordPress server on CentOS 7 with NGINX, PHP/PHP-FPM 7, Redis and more.
Enable the firewalld service and only allow http/s traffic to the server – in addition to the default of just ssh.
systemctl enable firewalld service firewalld start firewall-cmd --permanent --zone=public --add-service=http firewall-cmd --permanent --zone=public --add-service=https firewall-cmd --reload
Use Letsencrypt for free SSL certificates.
yum -y install letsencrypt openssl dhparam -out /etc/ssl/certs/dhparam.pem 2048
#!/bin/bash # make sure the YUM_CRON_EMAIL is set if [[ -z $YUM_CRON_EMAIL ]]; then echo "You must specify an email using \$YUM_CRON_EMAIL"; else # install and enable, plus patch for bug fixing yum -y install yum-cron patch chkconfig yum-cron on # configure via sed replacements sed -i "s|^email_to = root|email_to = ${YUM_CRON_EMAIL}|" /etc/yum/yum-cron.conf sed -i 's|^update_messages = no|update_messages = yes|' /etc/yum/yum-cron.conf sed -i 's|^download_updates = no|download_updates = yes|' /etc/yum/yum-cron.conf sed -i 's|^apply_updates = no|apply_updates = yes|' /etc/yum/yum-cron.conf sed -i 's|^emit_via = stdio|emit_via = email|' /etc/yum/yum-cron.conf sed -i "s|^email_to = root|email_to = ${YUM_CRON_EMAIL}|" /etc/yum/yum-cron-hourly.conf sed -i 's|^update_cmd = default|update_cmd = security|' /etc/yum/yum-cron-hourly.conf sed -i 's|^update_messages = no|update_messages = yes|' /etc/yum/yum-cron-hourly.conf sed -i 's|^download_updates = no|download_updates = yes|' /etc/yum/yum-cron-hourly.conf sed -i 's|^apply_updates = no|apply_updates = yes|' /etc/yum/yum-cron-hourly.conf sed -i 's|^emit_via = stdio|emit_via = email|' /etc/yum/yum-cron-hourly.conf egrep '^email_to|^update_messages|^download_updates|^apply_updates|^emit_via' /etc/yum/yum-cron.conf egrep '^email_to|^update_cmd|^update_messages|^download_updates|^apply_updates|^emit_via' /etc/yum/yum-cron-hourly.conf # fix bug in yum-cron nightly updates if [[ $(grep -q "# success, dependencies resolved" /usr/sbin/yum-cron) -ne 0 ]]; then patch /usr/sbin/yum-cron <<PATCHFILE --- yum-cron.orig 2016-10-23 19:24:57.099859931 +0000 +++ yum-cron 2016-10-23 19:27:58.048784006 +0000 @@ -504,7 +504,13 @@ except yum.Errors.RepoError, e: self.emitCheckFailed("%s" %(e,)) sys.exit() - if res != 2: + if res == 0: + # success, empty transaction + sys.exit(0) + elif res == 2: + # success, dependencies resolved + pass + else: self.emitCheckFailed("Failed to build transaction: %s" %(str.join("\n", resmsg),)) sys.exit(1) PATCHFILE fi # (re)start the yum-cron service (service yum-cron status > /dev/null && service yum-cron restart) || service yum-cron start fi
Redis is available via EPEL and provides a great in memory cache.
#!/bin/bash # install the EPEL repo to access Redis yum install -y epel-release yum install -y redis # fix redis background saves on low memory sysctl vm.overcommit_memory=1 && cat <<SYSCTL_MEM > /etc/sysctl.d/88-vm.overcommit_memory.conf vm.overcommit_memory = 1 SYSCTL_MEM # increase max connections sysctl -w net.core.somaxconn=65535 && cat <<SYSCTL_CONN > /etc/sysctl.d/88-net.core.somaxconn.conf net.core.somaxconn = 65535 SYSCTL_CONN sysctl -w fs.file-max=100000 && cat <<SYSCTL_FILEMAX > /etc/sysctl.d/88-fs.file-max.conf fs.file-max = 100000 SYSCTL_FILEMAX sed -i "s|^tcp-backlog [[:digit:]]\+|tcp-backlog 65535|" /etc/redis.conf # enable redis service on reboot systemctl enable redis # start service (service redis status > /dev/null && service redis restart) || service redis start #!/bin/bash # Create Service to disable THP cat <<DISABLE_THP > /etc/systemd/system/disable-thp.service [Unit] Description=Disable Transparent Huge Pages (THP) [Service] Type=simple ExecStart=/bin/sh -c "echo 'never' > /sys/kernel/mm/transparent_hugepage/enabled && echo 'never' > /sys/kernel/mm/transparent_hugepage/defrag" [Install] WantedBy=multi-user.target DISABLE_THP sudo systemctl daemon-reload sudo systemctl start disable-thp sudo systemctl enable disable-thp
Install PHP and PHP-FPM from the Remi Safe Repo. Some config files and binaries will need to be symlinked for compatibility.
# install the remi-safe.repo cat <<REMISAFE > /etc/yum.repos.d/remi-safe.repo # This repository is safe to use with RHEL/CentOS base repository # it only provides additional packages for the PHP stack # all dependencies are in base repository or in EPEL [remi-safe] name=Safe Remi's RPM repository for Enterprise Linux \$releasever - \$basearch #baseurl=http://rpms.remirepo.net/enterprise/\$releasever/safe/\$basearch/ mirrorlist=http://rpms.remirepo.net/enterprise/\$releasever/safe/mirror enabled=1 gpgcheck=1 gpgkey=http://rpms.remirepo.net/RPM-GPG-KEY-remi REMISAFE # install php7 and modules yum install -y \ php70 \ php70-php-fpm \ php70-php-gd \ php70-php-json \ php70-php-mbstring \ php70-php-mysqlnd \ php70-php-pdo \ php70-php-pecl-apcu \ php70-php-pecl-apcu-bc \ php70-php-pecl-igbinary \ php70-php-pecl-imagick \ php70-php-pecl-redis \ php70-php-xml # start php-fpm at boot systemctl enable php70-php-fpm # link the systemd service to "php-fpm" [[ -f /usr/lib/systemd/system/php-fpm.service ]] || ln -s /usr/lib/systemd/system/php70-php-fpm.service /usr/lib/systemd/system/php-fpm.service # link the binaries [[ -f /usr/bin/php ]] || ln -s `which php70` /usr/bin/php [[ -f /usr/bin/php-cgi ]] || ln -s `which php70-cgi` /usr/bin/php-cgi [[ -f /usr/bin/php-phar ]] || ln -s `which php70-phar` /usr/bin/php-phar # link the php-fpm configs [[ -f /etc/php-fpm.conf ]] || ln -s /etc/opt/remi/php70/php-fpm.conf /etc/php-fpm.conf [[ -d /etc/php-fpm.d ]] || ln -s /etc/opt/remi/php70/php-fpm.d /etc/php-fpm.d mkdir -p /var/log/php-fpm && chown -R nginx.nginx /var/log/php-fpm mkdir -p /var/lib/php/session && mkdir -p /var/lib/php/wsdlcache && mkdir -p /var/lib/php/opcache chown -R nginx.nginx /var/lib/php/* sed -i -e 's/user = apache/user = nginx/' /etc/php-fpm.d/www.conf sed -i -e 's/group = apache/group = nginx/' /etc/php-fpm.d/www.conf sed -i -e 's|/var/opt/remi/php70/log/php-fpm/www-error.log|/var/log/php-fpm/www-error.log|' /etc/php-fpm.d/www.conf sed -i -e 's|/var/opt/remi/php70/lib/php/session|/var/lib/php/session|' /etc/php-fpm.d/www.conf sed -i -e 's|/var/opt/remi/php70/lib/php/wsdlcache|/var/lib/php/wsdlcache|' /etc/php-fpm.d/www.conf sed -i -e 's|/var/opt/remi/php70/lib/php/opcache|/var/lib/php/opcache|' /etc/php-fpm.d/www.conf
Install NGINX with HTTP2 support.
# install nginx repo cat <<REPO > /etc/yum.repos.d/nginx.repo [nginx] name=nginx repo # default repo #baseurl=http://nginx.org/packages/centos/\$releasever/\$basearch/ # mainline "dev" repo for http2 support baseurl=http://nginx.org/packages/mainline/centos/\$releasever/\$basearch/ gpgcheck=0 enabled=1 REPO # install and enable nginx yum install -y nginx systemctl enable nginx # enable httpd in selinux semanage permissive -a httpd_t # test your configuration and reload nginx -t && service nginx start
Add a bunch of config files for Nginx.
# includes for nginx configurations mkdir -p /etc/nginx/includes # ssl settings for virtual hosts cat <<BLACKLIST > /etc/nginx/includes/blacklist.conf # ██████╗ ██╗ █████╗ ██████╗██╗ ██╗██╗ ██╗███████╗████████╗ # ██╔══██╗██║ ██╔══██╗██╔════╝██║ ██╔╝██║ ██║██╔════╝╚══██╔══╝ # ██████╔╝██║ ███████║██║ █████╔╝ ██║ ██║███████╗ ██║ # ██╔══██╗██║ ██╔══██║██║ ██╔═██╗ ██║ ██║╚════██║ ██║ # ██████╔╝███████╗██║ ██║╚██████╗██║ ██╗███████╗██║███████║ ██║ # ╚═════╝ ╚══════╝╚═╝ ╚═╝ ╚═════╝╚═╝ ╚═╝╚══════╝╚═╝╚══════╝ ╚═╝ #-*- mode: nginx; mode: flyspell-prog; ispell-local-dictionary: "american" -*- ### This file implements a blacklist for certain user agents and ### referrers. It's a first line of defense. It must be included ### inside a http block. ## Add here all user agents that are to be blocked. map \$http_user_agent \$bad_bot { default 0; ~*^Lynx 0; # Let Lynx go through libwww-perl 1; ~*(?i)(httrack|htmlparser|libwww|JikeSpider|proximic|Sosospider|Baiduspider|msnbot|BBBike|WWWOFFLE|Widow|SuperHTTP|BlackWidow|HTTrack|Java|Pixray|CPython|Spinn3r|Abonti|MSIECrawler|Baiduspider|Yandex|Siteimprove|Aboundex|80legs|360Spider|^Java|Cogentbot|^Alexibot|^asterias|^attach|^BackDoorBot|^BackWeb|Bandit|^BatchFTP|^Bigfoot|^Black.Hole|^BlackWidow|^BlowFish|^BotALot|Buddy|^BuiltBotTough|^Bullseye|^BunnySlippers|^Cegbfeieh|^CheeseBot|^CherryPicker|^ChinaClaw|Collector|Copier|^CopyRightCheck|^cosmos|^Crescent|^Custo|^AIBOT) 1; } ## Add here all referrers that are to blocked. map \$http_referer \$bad_referer { default 0; ~*(?i)(adult|babes|click|diamond|forsale|girl|jewelry|love|nudit|organic|poker|porn|poweroversoftware|sex|teen|webcam|zippo|casino|replica|en.savefrom.net|7makemoneyonline.com|acunetix-referrer.com|adcash.com|bithack.ru|buttons-for-website.com|cenokos.ru|cenoval.ru|cityadspix.com|darodar.com|econom.co|edakgfvwql.ru|gobongo.info|iedit.ilovevitaly.com|ilovevitaly.com|ilovevitaly.co|ilovevitaly.info|ilovevitaly.org|ilovevitaly.ru|iskalko.ru|luxup.ru|make-money-online.7makemoneyonline.com|maps.ilovevitaly.com|myftpupload.com|savefrom.net|savetubevideo.com|screentoolkit.com|semalt.com|seoexperimenty.ru|shopping.ilovevitaly.ru|slftsdybbg.ru|socialseet.ru|srecorder.com|st3.cwl.yahoo.com|superiends.org|vodkoved.ru|websocial.me|ykecwqlixx.ru|yougetsignal.com|priceg.com|responsinator.com|o-o-6-o-o.ru|o-o-8-o-o.ru|12masterov.com|4webmasters.org|acads.net|adviceforum.info|affordablewebsitesandmobileapps.com|anal-acrobats.hol.es|anticrawler.org|bard-real.com.ua|best-seo-offer.com|best-seo-solution.com|bestwebsitesawards.com|billiard-classic.com.ua|blackhatworth.com|brakehawk.com|buttons-for-your-website.com|buy-cheap-online.info|buy-forum.ru|cardiosport.com.ua|ci.ua|customsua.com.ua|delfin-aqua.com.ua|dipstar.org|domination.ml|drupa.com|dvr.biz.ua|e-kwiaciarz.pl|este-line.com.ua|europages.com.ru|event-tracking.com|forum20.smailik.org|forum69.info|free-share-buttons.com|free-social-buttons.com|generalporn.org|get-free-traffic-now.com|ghazel.ru|googlsucks.com|guardlink.org|hulfingtonpost.com|humanorightswatch.org|ico.re|iloveitaly.ro|iloveitaly.ru|iminent.com|it-max.com.ua|kabbalah-red-bracelets.com|kambasoft.com|makemoneyonline.com|maridan.com.ua|masterseek.com|mebeldekor.com.ua|med-zdorovie.com.ua|mirobuvi.com.ua|ok.ru|onlywoman.org|o-o-6-o-o.com|palvira.com.ua|pornhub-forum.ga|pornhub-forum.uni.me|prodvigator.ua|ranksonic.info|ranksonic.org|rapidgator-porn.ga|resellerclub.com|sanjosestartups.com|search-error.com|sexyteens.hol.es|shop.xz618.com|simple-share-buttons.com|social-buttons.com|theguardlan.com|trion.od.ua|webmaster-traffic.com|websites-reviews.com|youporn-forum.ga|youporn-forum.uni.me|наркомания.лечениенаркомании.com|непереводимая.рф|floating-share-buttons.com|traffic2money.com|site7.free-floating-buttons.com|sexyali.com|get-free-social-traffic.com|site2.free-floating-buttons.com|success-seo.com|trafficmonetizer.org|chinese-amezon.com|free-social-buttons.com) 1; } ## Add here all hosts that should be spared any referrer checking. geo \$bad_referer { 127.0.0.1 0; 192.168.1.0/24 0; 217.23.7.130 0; 78.110.60.230 0; 193.227.240.37 0; 193.227.240.38 0; } map \$http_user_agent \$limit_bots { default 0; ~*(google|bing|yandex|msnbot) 1; ~*(AltaVista|Googlebot|Slurp|BlackWidow|Bot|ChinaClaw|Custo|DISCo|Download|Demon|eCatch|EirGrabber|EmailSiphon|EmailWolf|SuperHTTP|Surfbot|WebWhacker) 2; ~*(Express|WebPictures|ExtractorPro|EyeNetIE|FlashGet|GetRight|GetWeb!|Go!Zilla|Go-Ahead-Got-It|GrabNet|Grafula|HMView|Go!Zilla|Go-Ahead-Got-It) 2; ~*(rafula|HMView|HTTrack|Stripper|Sucker|Indy|InterGET|Ninja|JetCar|Spider|larbin|LeechFTP|Downloader|tool|Navroad|NearSite|NetAnts|tAkeOut|WWWOFFLE) 2; ~*(GrabNet|NetSpider|Vampire|NetZIP|Octopus|Offline|PageGrabber|Foto|pavuk|pcBrowser|RealDownload|ReGet|SiteSnagger|SmartDownload|SuperBot|WebSpider) 2; ~*(Teleport|VoidEYE|Collector|WebAuto|WebCopier|WebFetch|WebGo|WebLeacher|WebReaper|WebSauger|eXtractor|Quester|WebStripper|WebZIP|Wget|Widow|Zeus) 2; ~*(Twengabot|htmlparser|libwww|Python|perl|urllib|scan|Curl|email|PycURL|Pyth|PyQ|WebCollector|WebCopy|webcraw) 2; } BLACKLIST cat <<CLOUDFLARECONF > /etc/nginx/includes/cloudflare.conf # ██████╗██╗ ██████╗ ██╗ ██╗██████╗ ███████╗██╗ █████╗ ██████╗ ███████╗ # ██╔════╝██║ ██╔═══██╗██║ ██║██╔══██╗██╔════╝██║ ██╔══██╗██╔══██╗██╔════╝ # ██║ ██║ ██║ ██║██║ ██║██║ ██║█████╗ ██║ ███████║██████╔╝█████╗ # ██║ ██║ ██║ ██║██║ ██║██║ ██║██╔══╝ ██║ ██╔══██║██╔══██╗██╔══╝ # ╚██████╗███████╗╚██████╔╝╚██████╔╝██████╔╝██║ ███████╗██║ ██║██║ ██║███████╗ # ╚═════╝╚══════╝ ╚═════╝ ╚═════╝ ╚═════╝ ╚═╝ ╚══════╝╚═╝ ╚═╝╚═╝ ╚═╝╚══════╝ set_real_ip_from 199.27.128.0/21; set_real_ip_from 173.245.48.0/20; set_real_ip_from 103.21.244.0/22; set_real_ip_from 103.22.200.0/22; set_real_ip_from 103.31.4.0/22; set_real_ip_from 141.101.64.0/18; set_real_ip_from 108.162.192.0/18; set_real_ip_from 190.93.240.0/20; set_real_ip_from 188.114.96.0/20; set_real_ip_from 197.234.240.0/22; set_real_ip_from 198.41.128.0/17; set_real_ip_from 162.158.0.0/15; set_real_ip_from 104.16.0.0/12; set_real_ip_from 172.64.0.0/13; set_real_ip_from 2400:cb00::/32; set_real_ip_from 2606:4700::/32; set_real_ip_from 2803:f800::/32; set_real_ip_from 2405:b500::/32; set_real_ip_from 2405:8100::/32; set_real_ip_from 127.0.0.1; real_ip_header X-Forwarded-For; CLOUDFLARECONF cat <<MIMETYPESCONF > /etc/nginx/includes/mime.types.conf # ███╗ ███╗██╗███╗ ███╗███████╗ ████████╗██╗ ██╗██████╗ ███████╗███████╗ # ████╗ ████║██║████╗ ████║██╔════╝ ╚══██╔══╝╚██╗ ██╔╝██╔══██╗██╔════╝██╔════╝ # ██╔████╔██║██║██╔████╔██║█████╗ ██║ ╚████╔╝ ██████╔╝█████╗ ███████╗ # ██║╚██╔╝██║██║██║╚██╔╝██║██╔══╝ ██║ ╚██╔╝ ██╔═══╝ ██╔══╝ ╚════██║ # ██║ ╚═╝ ██║██║██║ ╚═╝ ██║███████╗ ██║ ██║ ██║ ███████╗███████║ # ╚═╝ ╚═╝╚═╝╚═╝ ╚═╝╚══════╝ ╚═╝ ╚═╝ ╚═╝ ╚══════╝╚══════╝ types { application/x-font-ttf ttf; font/opentype otf; } MIMETYPESCONF # use a conf file to include our sites-enabled conf files mkdir -p /etc/nginx/sites-available mkdir -p /etc/nginx/sites-enabled cat <<SITESENABLED > /etc/nginx/includes/sites-enabled.conf # ██╗ ██╗███████╗██████╗ ███████╗██╗████████╗███████╗███████╗ # ██║ ██║██╔════╝██╔══██╗██╔════╝██║╚══██╔══╝██╔════╝██╔════╝ # ██║ █╗ ██║█████╗ ██████╔╝███████╗██║ ██║ █████╗ ███████╗ # ██║███╗██║██╔══╝ ██╔══██╗╚════██║██║ ██║ ██╔══╝ ╚════██║ # ╚███╔███╔╝███████╗██████╔╝███████║██║ ██║ ███████╗███████║ # ╚══╝╚══╝ ╚══════╝╚═════╝ ╚══════╝╚═╝ ╚═╝ ╚══════╝╚══════╝ include /etc/nginx/sites-enabled/*.conf; SITESENABLED ln -s /etc/nginx/includes/blacklist.conf /etc/nginx/conf.d/_.blacklist.conf ln -s /etc/nginx/includes/cloudflare.conf /etc/nginx/conf.d/_.cloudflare.conf ln -s /etc/nginx/includes/mime.types.conf /etc/nginx/conf.d/_.mime.types.conf ln -s /etc/nginx/includes/sites-enabled.conf /etc/nginx/conf.d/_.sites-enabled.conf
These Nginx include files are meant to be using inside virtual server blocks.
# ssl settings for virtual hosts cat <<SSLCONF > /etc/nginx/includes/ssl.conf # ███████╗███████╗██╗ # ██╔════╝██╔════╝██║ # ███████╗███████╗██║ # ╚════██║╚════██║██║ # ███████║███████║███████╗ # ╚══════╝╚══════╝╚══════╝ # Use TLS (so don't use old version of SSL) ssl_protocols TLSv3 TLSv1.1 TLSv1.2; ssl_prefer_server_ciphers on; ssl_ciphers 'ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-DSS-AES128-GCM-SHA256:kEDH+AESGCM:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA:ECDHE-ECDSA-AES256-SHA:DHE-RSA-AES128-SHA256:DHE-RSA-AES128-SHA:DHE-DSS-AES128-SHA256:DHE-RSA-AES256-SHA256:DHE-DSS-AES256-SHA:DHE-RSA-AES256-SHA:AES128-GCM-SHA256:AES256-GCM-SHA384:AES128-SHA256:AES256-SHA256:AES128-SHA:AES256-SHA:AES:CAMELLIA:DES-CBC3-SHA:!aNULL:!eNULL:!EXPORT:!DES:!RC4:!MD5:!PSK:!aECDH:!EDH-DSS-DES-CBC3-SHA:!EDH-RSA-DES-CBC3-SHA:!KRB5-DES-CBC3-SHA'; ssl_dhparam /etc/ssl/certs/dhparam.pem; ssl_session_timeout 1d; ssl_session_cache shared:SSL:50m; ssl_stapling on; ssl_stapling_verify on; SSLCONF # use a conf file to include our sites-enabled conf files cat <<SECURITY > /etc/nginx/includes/security.conf # ███████╗███████╗ ██████╗██╗ ██╗██████╗ ██╗████████╗██╗ ██╗ # ██╔════╝██╔════╝██╔════╝██║ ██║██╔══██╗██║╚══██╔══╝╚██╗ ██╔╝ # ███████╗█████╗ ██║ ██║ ██║██████╔╝██║ ██║ ╚████╔╝ # ╚════██║██╔══╝ ██║ ██║ ██║██╔══██╗██║ ██║ ╚██╔╝ # ███████║███████╗╚██████╗╚██████╔╝██║ ██║██║ ██║ ██║ # ╚══════╝╚══════╝ ╚═════╝ ╚═════╝ ╚═╝ ╚═╝╚═╝ ╚═╝ ╚═╝ if (\$bad_referer){ return 444; } location ~* (readme|changelog)\\.txt\$ { return 444; } # don't show this as it can leak info location ~* /(\\.|(wp-config|xmlrpc)\\.php|(readme|license|changelog)\\.(html|txt)) { return 444; } location ~ /mu-plugins/ { return 444; } # no PHP execution in uploads/files location ~* /(?:uploads|files)/.*\\.php\$ { deny all; } # hide contents of sensitive files location ~* \\.(conf|engine|inc|info|install|make|module|profile|test|po|sh|.*sql|theme|tpl(\\.php)?|xtmpl)\$|^(\\..*|Entries.*|Repository|Root|Tag|Template)\$|\\.php_ { return 444; } # don't allow other executable file types location ~* \\.(pl|cgi|py|sh|lua)\$ { return 444; } location = /robots.txt { if ( \$limit_bots != 1 ) { return 444; } expires 30d; add_header Cache-Control public; try_files /robots.txt @shared; } location @shared { root /var/www/shared; } SECURITY # use a conf file to include our sites-enabled conf files cat <<WORDPRESSCONF > /etc/nginx/includes/wordpress.conf # ██╗ ██╗ ██████╗ ██████╗ ██████╗ ██████╗ ██████╗ ███████╗███████╗███████╗ # ██║ ██║██╔═══██╗██╔══██╗██╔══██╗██╔══██╗██╔══██╗██╔════╝██╔════╝██╔════╝ # ██║ █╗ ██║██║ ██║██████╔╝██║ ██║██████╔╝██████╔╝█████╗ ███████╗███████╗ # ██║███╗██║██║ ██║██╔══██╗██║ ██║██╔═══╝ ██╔══██╗██╔══╝ ╚════██║╚════██║ # ╚███╔███╔╝╚██████╔╝██║ ██║██████╔╝██║ ██║ ██║███████╗███████║███████║ # ╚══╝╚══╝ ╚═════╝ ╚═╝ ╚═╝╚═════╝ ╚═╝ ╚═╝ ╚═╝╚══════╝╚══════╝╚══════╝ # include standard security file include /etc/nginx/includes/security.conf; # allow CORS for fonts location ~* \\.(ttf|ttc|otf|eot|woff2?|font.css|css|svg)\$ { add_header Access-Control-Allow-Origin *; } #Yoast sitemap location ~ ([^/]*)sitemap(.*)\\.x(m|s)l\$ { ## this redirects sitemap.xml to /sitemap_index.xml rewrite ^/sitemap\\.xml\$ /sitemap_index.xml permanent; ## this makes the XML sitemaps work rewrite ^/([a-z]+)?-?sitemap\\.xsl\$ /index.php?xsl=\$1 last; rewrite ^/sitemap_index\\.xml\$ /index.php?sitemap=1 last; rewrite ^/([^/]+?)-sitemap([0-9]+)?\\.xml\$ /index.php?sitemap=\$1&sitemap_n=\$2 last; ## The following lines are optional for the premium extensions ## News SEO rewrite ^/news-sitemap\\.xml\$ /index.php?sitemap=wpseo_news last; ## Local SEO rewrite ^/locations\\.kml\$ /index.php?sitemap=wpseo_local_kml last; rewrite ^/geo-sitemap\\.xml\$ /index.php?sitemap=wpseo_local last; ## Video SEO rewrite ^/video-sitemap\\.xsl\$ /index.php?xsl=video last; } index index.php; location / { try_files \$uri \$uri/ /index.php\$is_args\$args; } location ~ ^/(fpm-status|ping)\$ { include fastcgi_params; fastcgi_pass 127.0.0.1:9000; fastcgi_param SCRIPT_FILENAME \$document_root\$fastcgi_script_name; break; } location ~ \\.php\$ { # zero-day exploit defense. try_files \$uri =404; # logging vi PHP-FPM fastcgi_intercept_errors on; # pass request to fastcgi/php-cgi via spawn-fcgi fastcgi_pass unix:/var/run/php-fpm.sock; # default fastcgi_params include fastcgi_params; # max timeouts (should match php.ini) fastcgi_connect_timeout 600s; fastcgi_send_timeout 600s; fastcgi_read_timeout 600s; # override fastcgi_params fastcgi_param SERVER_NAME \$host; fastcgi_param SCRIPT_FILENAME \$document_root\$fastcgi_script_name; break; } location ~ /purge(/.*) { #fastcgi_cache_purge WORDPRESS "\$scheme\$request_method\$host\$1"; } WORDPRESSCONF
This is an example of a virtual host running WordPress.
# use a conf file to include our sites-enabled conf files cat <<VIRTUALHOST > /etc/nginx/sites-available/virtualhost.conf server { # Domain validation is on port 80 listen 80; # Hostnames to listen on, you will pass each of these to letsencrypt with "-w www.example.com" server_name www.example.com; # Your document root, you will pass this path to letsencrypt with "-w /var/www/www.example.com/html/" root /var/www/www.example.com/html/; # handle letsencrypt domain validation location ~ /.well-known { allow all; } # permanently redirect everything else location / { return 301 https://$server_name$request_uri; } } server { # All SSL is served on 443. If available include "http2", otherwise remove it. listen 443 ssl http2; # Hostnames to listen on server_name www.example.com; # Add SSL Keys here once they are generated #ssl_certificate /etc/letsencrypt/live/example.com/fullchain.pem; #ssl_certificate_key /etc/letsencrypt/live/example.com/privkey.pem; include /etc/nginx/includes/ssl.conf; # WordPress Sites # include /etc/nginx/includes/wordpress.conf; # include /var/www/www.example.com/html/nginx.conf; # handle all requests... # location / { # } } VIRTUALHOST # link the virtual host using full pathnames for source and target! # ln -s /etc/nginx/sites-available/virtualhost.conf /etc/nginx/sites-enabled/virtualhost.conf nginx -t && service nginx reload
yum install -y barcode barcode-devel php-tcpdf.noarch cd /usr/local/src wget https://ashberg.de/php-barcode/download/files/genbarcode-0.4.tar.gz tar -xzvf genbarcode-0.4.tar.gz cd genbarcode-0.4 make && make install # use to generate via php wget https://ashberg.de/php-barcode/download/files/php-barcode-0.4.tar.gz
The post LEMP: CentOS 7, NGINX, PHP7, and Redis for WordPress appeared first on Justin Silver.
]]>The post Install ClamAV + Maldet on CentOS 7 appeared first on Justin Silver.
]]>A quick guide for installing ClamAV and Maldet on CentOS 7 for malware scanning and quarantining.
Using the ClamAV malware library will speed up the scans performed by Maldet.
You will need to install Maldet from source, which can be downloaded from rfxn.net.
Use the sed commands below to edit values in the /usr/local/maldetect/conf.maldet
configuration file.
email_alert="1"
: Send notifications via email when cron job runsemail_addr="YOUR_EMAIL@HOSTNAME"
: The email address notifications should be send toemail_subj="Malware alerts for $HOSTNAME - $(date +%Y-%m-%d)"
: The email subject for notificationsquarantine_hits="1"
: Move the malware to quarantine.quarantine_clean="1"
: Delete any malware detected.#!/bin/bash # make sure the MALDET_EMAIL is set if [[ -z $MALDET_EMAIL ]]; then read -e -p "Please enter a notification email: " MALDET_EMAIL fi if [[ -z $MALDET_EMAIL ]]; then exit 0 fi yum install -y epel-release wget perl yum install -y clamav cd /usr/local/src curl -s http://www.rfxn.com/downloads/maldetect-current.tar.gz -o maldetect-current.tar.gz tar -xzf maldetect-current.tar.gz cd maldetect-* ./install.sh maldet --update-sigs --update-ver sed -i "s|[email protected]|${MALDET_EMAIL}|" /usr/local/maldetect/conf.maldet sed -i 's|^scan_clamscan="0"|scan_clamscan="1"|' /usr/local/maldetect/conf.maldet sed -i 's|^email_alert="0"|email_alert="1"|' /usr/local/maldetect/conf.maldet sed -i 's|^quarantine_hits="0"|quarantine_hits="1"|' /usr/local/maldetect/conf.maldet sed -i 's|^quarantine_clean="0"|quarantine_clean="1"|' /usr/local/maldetect/conf.maldet # add email subject if it doesn't exist grep -q email_subj /usr/local/maldetect/conf.maldet || echo 'email_subj="Malware alerts for $HOSTNAME - $(date +%Y-%m-%d)"' >> /usr/local/maldetect/conf.maldet
Your system should now perform a daily scan via cron job. You can also scan manually from the command line and specifying the directory to check for malware – or you can use root to scan the entire filesystem.
maldet --scan-all /path/to/scan
The post Install ClamAV + Maldet on CentOS 7 appeared first on Justin Silver.
]]>The post Node.js + PM2 + NGINX + Redis on CentOS 7 appeared first on Justin Silver.
]]>This is a quick setup guide for the application stack I have been using recently. Most of my latest work has been a throwback to the first server-side programming I did in the 90’s – javascript – except this time it’s Node.js instead of Netscape Communication Server. In this setup PM2 is used to manage the Node process running as an arbitrary user, running on an unprivileged port. This means that the application can be restarted without root credentials. The front-end is served by NGINX and it does need to be started as root because it runs on the privileged ports 80 and 443 in this use case. It also gives us a lot of the built in features that Nginx gives you on the front end, like serving all your content over SSL – for free using Let’s Encrypt event. My caching needs are provided by Redis.
Here is the setup – run as root or use sudo.
Enable the firewalld service and only allow http/s traffic to the server – in addition to the default of just ssh.
#!/bin/bash # enable on book systemctl enable firewalld # (re)start the service (service firewalld status > /dev/null && service firewalld restart) || service firewalld start # add the http and http services and reload firewall-cmd --permanent --zone=public --add-service=http firewall-cmd --permanent --zone=public --add-service=https firewall-cmd --reload
Use Letsencrypt for free SSL certificates.
yum -y install letsencrypt openssl dhparam -out /etc/ssl/certs/dhparam.pem 2048
#!/bin/bash # make sure the YUM_CRON_EMAIL is set if [[ -z $YUM_CRON_EMAIL ]]; then echo "You must specify an email using \$YUM_CRON_EMAIL"; else # install and enable, plus patch for bug fixing yum -y install yum-cron patch chkconfig yum-cron on # configure via sed replacements sed -i "s|^email_to = root|email_to = ${YUM_CRON_EMAIL}|" /etc/yum/yum-cron.conf sed -i 's|^update_messages = no|update_messages = yes|' /etc/yum/yum-cron.conf sed -i 's|^download_updates = no|download_updates = yes|' /etc/yum/yum-cron.conf sed -i 's|^apply_updates = no|apply_updates = yes|' /etc/yum/yum-cron.conf sed -i 's|^emit_via = stdio|emit_via = email|' /etc/yum/yum-cron.conf sed -i "s|^email_to = root|email_to = ${YUM_CRON_EMAIL}|" /etc/yum/yum-cron-hourly.conf sed -i 's|^update_cmd = default|update_cmd = security|' /etc/yum/yum-cron-hourly.conf sed -i 's|^update_messages = no|update_messages = yes|' /etc/yum/yum-cron-hourly.conf sed -i 's|^download_updates = no|download_updates = yes|' /etc/yum/yum-cron-hourly.conf sed -i 's|^apply_updates = no|apply_updates = yes|' /etc/yum/yum-cron-hourly.conf sed -i 's|^emit_via = stdio|emit_via = email|' /etc/yum/yum-cron-hourly.conf egrep '^email_to|^update_messages|^download_updates|^apply_updates|^emit_via' /etc/yum/yum-cron.conf egrep '^email_to|^update_cmd|^update_messages|^download_updates|^apply_updates|^emit_via' /etc/yum/yum-cron-hourly.conf # fix bug in yum-cron nightly updates if [[ $(grep -q "# success, dependencies resolved" /usr/sbin/yum-cron) -ne 0 ]]; then patch /usr/sbin/yum-cron <<PATCHFILE --- yum-cron.orig 2016-10-23 19:24:57.099859931 +0000 +++ yum-cron 2016-10-23 19:27:58.048784006 +0000 @@ -504,7 +504,13 @@ except yum.Errors.RepoError, e: self.emitCheckFailed("%s" %(e,)) sys.exit() - if res != 2: + if res == 0: + # success, empty transaction + sys.exit(0) + elif res == 2: + # success, dependencies resolved + pass + else: self.emitCheckFailed("Failed to build transaction: %s" %(str.join("\n", resmsg),)) sys.exit(1) PATCHFILE fi # (re)start the yum-cron service (service yum-cron status > /dev/null && service yum-cron restart) || service yum-cron start fi
We are going to use the “mainline” repo to get HTTP2 support. I like to create a conf file in “sites-available” that is linked to “sites-enabled” so I can disable things easily – this is enabled by adding a file under /etc/nginx/conf.d
.
#!/bin/bash # import src utility if [[ -z $(type -t src) ]]; then source <(curl -sL https://www.doublesharp.com/src) fi src osname src osversion cat <<REPO > /etc/yum.repos.d/nginx.repo [nginx] name=nginx repo # default repo #baseurl=http://nginx.org/packages/$(osname)/$(osversion)/\$basearch/ # mainline "dev" repo for http2 support baseurl=http://nginx.org/packages/mainline/$(osname)/$(osversion)/\$basearch/ gpgcheck=0 enabled=1 REPO #install nginx yum install -y nginx # turn on for reboots systemctl enable nginx mkdir -p /etc/nginx/includes mkdir -p /etc/nginx/sites-enabled mkdir -p /etc/nginx/sites-available mkdir -p /etc/nginx/streams-enabled mkdir -p /etc/nginx/streams-available # use a conf file to include our sites-enabled conf files cat <<SITESENABLED > /etc/nginx/includes/sites-enabled.conf include /etc/nginx/sites-enabled/*.conf; SITESENABLED [[ -f "/etc/nginx/conf.d/_.sites-enabled.conf" ]] || ln -s /etc/nginx/includes/sites-enabled.conf /etc/nginx/conf.d/_.sites-enabled.conf # enable httpd in selinux semanage permissive -a httpd_t cat <<NGINX_CONF > /etc/nginx/nginx.conf user nginx; worker_processes auto; error_log /var/log/nginx/error.log warn; pid /var/run/nginx.pid; worker_rlimit_nofile 100000; events { # determines how much clients will be served per worker # max clients = worker_connections * worker_processes # max clients is also limited by the number of socket connections available on the system (~64k) worker_connections 100000; # optmized to serve many clients with each thread, essential for linux use epoll; # accept as many connections as possible, may flood worker connections if set too low multi_accept on; } # web servers / virtual hosts http { include /etc/nginx/mime.types; default_type application/octet-stream; log_format main '\$remote_addr - \$remote_user [\$time_local] "\$request" ' '\$status \$body_bytes_sent "\$http_referer" ' '"\$http_user_agent" "\$http_x_forwarded_for"'; access_log /var/log/nginx/access.log combined flush=1m buffer=128k; # cache informations about FDs, frequently accessed files # can boost performance, but you need to test those values open_file_cache max=200000 inactive=20s; open_file_cache_valid 30s; open_file_cache_min_uses 2; open_file_cache_errors on; # send headers in one peace, its better then sending them one by one tcp_nopush on; # don't buffer data sent, good for small data bursts in real time tcp_nodelay on; # server will close connection after this time keepalive_timeout 30; # allow the server to close connection on non responding client, this will free up memory reset_timedout_connection on; # request timed out -- default 60 client_body_timeout 10; # if client stop responding, free up memory -- default 60 send_timeout 2; # reduce the data that needs to be sent over network gzip on; gzip_min_length 10240; gzip_proxied expired no-cache no-store private auth; gzip_types text/plain text/css text/xml text/javascript application/x-javascript application/xml; gzip_disable "MSIE [1-6]\."; proxy_buffer_size 128k; proxy_buffers 64 256k; proxy_busy_buffers_size 256k; proxy_ignore_client_abort on; include /etc/nginx/conf.d/*.conf; } # load balancer streams stream { include /etc/nginx/streams-enabled/*.conf; } NGINX_CONF # create a virtual server conf file that is in sites-available cat <<NGINX_HOST > /etc/nginx/sites-available/myapp.conf upstream myapp { # our app will be on localhost port 3000, but you can change this here server 127.0.0.1:3000 fail_timeout=0; } server { listen 80; server_name myapp.example.com; location / { proxy_set_header Host \$host:\$server_port; proxy_set_header X-Real-IP \$remote_addr; proxy_set_header X-Forwarded-For \$proxy_add_x_forwarded_for; proxy_set_header X-Forwarded-Proto \$scheme; proxy_pass http://myapp; } } NGINX_HOST # link this conf to sites-enabled. it's important to use the full path #ln -s /etc/nginx/sites-available/myapp.conf /etc/nginx/sites-enabled/myapp.conf nginx -t && (service nginx status > /dev/null && service nginx restart)
To install Redis with yum, first you need to install EPEL. Once the installed, you will have access to the repository containing the Redis install.
#!/bin/bash # install the EPEL repo to access Redis yum install -y epel-release yum install -y redis # fix redis background saves on low memory sysctl vm.overcommit_memory=1 && cat <<SYSCTL_MEM > /etc/sysctl.d/88-vm.overcommit_memory.conf vm.overcommit_memory = 1 SYSCTL_MEM # increase max connections sysctl -w net.core.somaxconn=65535 && cat <<SYSCTL_CONN > /etc/sysctl.d/88-net.core.somaxconn.conf net.core.somaxconn = 65535 SYSCTL_CONN sysctl -w fs.file-max=100000 && cat <<SYSCTL_FILEMAX > /etc/sysctl.d/88-fs.file-max.conf fs.file-max = 100000 SYSCTL_FILEMAX sed -i "s|^tcp-backlog [[:digit:]]\+|tcp-backlog 65535|" /etc/redis.conf # enable redis service on reboot systemctl enable redis # start service (service redis status > /dev/null && service redis restart) || service redis start
We want to install Node.js and then the PM2 package globally so that it can be accessed by other users.
#!/bin/bash # make sure the SRC_NODE_VERSION is set if [[ -z $SRC_NODE_VERSION ]]; then echo "You must specify a node version using \$SRC_NODE_VERSION"; else # Select node version to install curl --silent --location https://rpm.nodesource.com/setup_$SRC_NODE_VERSION.x | bash - # install via yum yum install -y git gcc-c++ make nodejs fi # PM2 - install as global npm install pm2@latest -g
As root, create a new user named “appuser”, or whatever you want your app user to be named. This could even be the default centos@/ec2-user@/etc that many hosts provide.
adduser appuser passwd appuser
Log in as the “appuser” user and create the Node app in your home directory. This directory should be owned by the “appuser”. In this case we assume the server is going to be listening on localhost port 3000, which means we can manage it with pm2 without having root permissions.
mkdir ~/apps cd /apps # create your app here, git clone, whatever # we assume the app is in ~/apps/myapp/server.js pm2 start ~/apps/myapp/server.js --name=myapp pm2 status myapp pm2 restart myapp
The post Node.js + PM2 + NGINX + Redis on CentOS 7 appeared first on Justin Silver.
]]>The post Install Jenkins as a Service on CentOS 7 appeared first on Justin Silver.
]]>I have previously written about how to Install Jenkins on CentOS as a Service where it was necessary to write your own startup, shutdown, configuration, and init.d scripts. Luckily this is all much easier now as you can install the software directly from a yum
repository – you’ll just need to fetch the repo from http://pkg.jenkins-ci.org/redhat/jenkins.repo.
Make sure you have Java on your system, then fetch the yum repository and install Jenkins.
yum -y install java curl http://pkg.jenkins-ci.org/redhat/jenkins.repo -o /etc/yum.repos.d/jenkins.repo rpm --import https://jenkins-ci.org/redhat/jenkins-ci.org.key yum -y install jenkins
Since CentOS 7 uses Systemd, use it to start the service on reboot.
systemctl enable jenkins service jenkins start
This will start jenkins on port 8080 by default (you can change these settings in /etc/sysconfig/jenkins). Leaving it as is and setting up a reverse Nginx proxy is my preference. Once you load the Jenkins home page you will be prompted to enter a password located in a file on your system to continue the setup. Here is a sample of my Nginx configuration.
# jenkins is upstream listening on port 8080 upstream jenkins { server 127.0.0.1:8080 fail_timeout=0; } # nginx is listening on port 80 server { listen 80; server_name jenkins.example.com; location / { proxy_set_header Host $host:$server_port; proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_set_header X-Forwarded-Proto $scheme; proxy_pass http://jenkins; } }
Keep in mind that you may have issues initially proxying to Jenkins if SELinux is configured to block access to port 8080. If you try to load the site via Ngnix and get a “502 Bad Gateway” error, check out the /var/log/audit/audit.log
– you will probably see errors regarding Nginx connecting to your port. You can either add the port by hand, or do it automatically with audit2allow
.
mkdir ~/.semanage && cd ~/.semanage cat /var/log/audit/audit.log | grep nginx | grep denied | audit2allow -M semanage semodule -i semanage.pp
If you need to generate an SSH key for the Jenkins user, use sudo to run as the proper user.
sudo -u jenkins ssh-keygen
Enjoy!
The post Install Jenkins as a Service on CentOS 7 appeared first on Justin Silver.
]]>The post Letsencrypt: Free SSL Certificates for NGINX appeared first on Justin Silver.
]]>I always wanted all of my sites to run over SSL, but it also didn’t seem worth the expense of buying certificates for all the domains I own. Enter Let’s Encrypt which offers free 90 day SSL certificates. This guide shows how to install and use letsencrypt to generate SSL certificates for NGINX running on CentOS 7, however it should be similar on other supported systems. A bit about Let’s Encrypt from their site:
Let’s Encrypt is a free, automated, and open certificate authority (CA), run for the public’s benefit. Let’s Encrypt is a service provided by the Internet Security Research Group (ISRG).
The key principles behind Let’s Encrypt are:
- Free: Anyone who owns a domain name can use Let’s Encrypt to obtain a trusted certificate at zero cost.
- Automatic: Software running on a web server can interact with Let’s Encrypt to painlessly obtain a certificate, securely configure it for use, and automatically take care of renewal.
- Secure: Let’s Encrypt will serve as a platform for advancing TLS security best practices, both on the CA side and by helping site operators properly secure their servers.
- Transparent: All certificates issued or revoked will be publicly recorded and available for anyone to inspect.
- Open: The automatic issuance and renewal protocol will be published as an open standard that others can adopt.
- Cooperative: Much like the underlying Internet protocols themselves, Let’s Encrypt is a joint effort to benefit the community, beyond the control of any one organization.
Install letsencrypt
with yum
. Next generate a strong Diffie-Hellman key – you can specify a different path but you need to change it in the Nginx server
block.
yum -y install letsencrypt openssl dhparam -out /etc/ssl/certs/dhparam.pem 2048
Edit your Nginx configuration to listen on HTTP and HTTPS, and respond to Let’s Encrypt domain validation requests to /.well-known
. Go ahead and add the SSL configuration, but no keys (since they don’t exist yet).
server { # Domain validation is on port 80, SSL is served on 443. If available include "http2", otherwise remove it. listen 80 443 ssl http2; # Hostnames to listen on, you will pass each of these to letsencrypt with "-w www.example.com" server_name www.example.com; # Your document root, you will pass this path to letsencrypt with "-w /var/www/www.example.com/html/" root /var/www/www.example.com/html/; # Add SSL Keys here once they are generated # Use TLS (so don't use old version of SSL) ssl_protocols TLSv1 TLSv1.1 TLSv1.2; ssl_prefer_server_ciphers on; ssl_ciphers 'ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-DSS-AES128-GCM-SHA256:kEDH+AESGCM:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA:ECDHE-ECDSA-AES256-SHA:DHE-RSA-AES128-SHA256:DHE-RSA-AES128-SHA:DHE-DSS-AES128-SHA256:DHE-RSA-AES256-SHA256:DHE-DSS-AES256-SHA:DHE-RSA-AES256-SHA:AES128-GCM-SHA256:AES256-GCM-SHA384:AES128-SHA256:AES256-SHA256:AES128-SHA:AES256-SHA:AES:CAMELLIA:DES-CBC3-SHA:!aNULL:!eNULL:!EXPORT:!DES:!RC4:!MD5:!PSK:!aECDH:!EDH-DSS-DES-CBC3-SHA:!EDH-RSA-DES-CBC3-SHA:!KRB5-DES-CBC3-SHA'; ssl_dhparam /etc/ssl/certs/dhparam.pem; ssl_session_timeout 1d; ssl_session_cache shared:SSL:50m; ssl_stapling on; ssl_stapling_verify on; # handle letsencrypt domain validation location ~ /.well-known { allow all; } # handle all requests... location / { } }
Validate the configuration, and when it passes reload Nginx. You can then generate an SSL key with letencrypt
using the --webroot
method. With this method you need to pass your web root with “-w /path/to/your/webroot
” and each domain you want an SSL for with “-d www.example.com -d example.com -d images.example.com
“, and so on. The first time you run Let’s Encrypt you will need to accept some terms, enter your email, etc, but subsequent runs won’t ask for this.
# validate nginx configuration nginx -t # reload nginx configuration service nginx reload # generate SSL keys letsencrypt certonly --webroot -w /var/www/www.example.com/html/ -d www.example.com
Once the keys have generated, you will need to add the certificate and key to your Nginx configuration. Edit the server
block and add the following – you may need to change the path for the letsencrypt
location on your system. Don’t move them since you will need to be able to renew them every 90 days.
# ssl certs from letsencrypt ssl_certificate /etc/letsencrypt/live/www.example.com/fullchain.pem; ssl_certificate_key /etc/letsencrypt/live/www.example.com/privkey.pem;
Validate the Nginx configuration again, then reload the service. Once it is up, you can use curl
to validate that it is serving requests over SSL.
# validate nginx configuration nginx -t # reload nginx configuration service nginx reload # see if you can load your site over SSL curl -s https://www.example.com
If you have trouble validating your domain and get 403 errors and use SELinux, it’s possible that you will need to run the following command to give nginx permission to read the .well-known directory.
chcon -Rt httpd_sys_content_t /var/www/yoursite/.well-known
Your certificate will expire every 90 days so it’s easiest to set up a cron job to automatically check for soon to expire certificates once per day so they can be renewed – this is why we don’t want to move the certs out of the /etc/letsencrypt/live/...
directory. You may need to reload nginx as well if the certificate is updated but this should generally be transparent to clients. Edit your crontab by running crontab -e
and adding the following to check for updates at 1AM.
# LetsEncrypt Renewals 0 1 * * * letsencrypt renew >/dev/null 2>&1 && service nginx reload
Note that your certificates will only be renewed if they are close to expiration, otherwise the system will skip it and continue using the currently installed cert. You want to update at least weekly although daily is prefered to make sure you everything is up to date.
[root@www ~]# letsencrypt renew ------------------------------------------------------------------------------- Processing /etc/letsencrypt/renewal/justinsilver.com.conf ------------------------------------------------------------------------------- Cert not yet due for renewal The following certs are not due for renewal yet: /etc/letsencrypt/live/justinsilver.com/fullchain.pem (skipped)
The post Letsencrypt: Free SSL Certificates for NGINX appeared first on Justin Silver.
]]>