I am using MariaDB – the open source version of MySQL – and wanted an easy way to backup my databases on a regular schedule without having to do anything when I created or dropped a database. MariaDB is a drop in replacement for MySQL, so this script should work as is. I put together this base script that will use the show databases command to iterate over each database so that it can be passed to mysqldump. The nice command is used to lower the impact to your regular server work.

Once exported the resulting SQL files are compressed into an archive using tar with bzip2 compression, which saves quite a bit of space over gzip from my tests with this kind of data. After the archive is created the source SQL files are deleted (again using nice) and finally anything in the directory older than the specified archive days value will be deleted.

You will need to create a file called /etc/sysconfig/dbbackup that stores the USERNAME and PASSWORD parameters for the script.

#/bin/bash
#
# MySQL/MariaDB backup script
# Justin Silver
# http://www.justinsilver.com
#
# Use cron to schedule this script to run as frequently as you want.
###################################################################################

# Set properties in this file
SYSCONFIG="/etc/sysconfig/dbbackup"

# User with SELECT, SHOW VIEW, EVENT, and TRIGGER, or... root
#USERNAME="USERNAME"
#PASSWORD="PASSWORD"

# Archive path
ARCHIVE_PATH="/var/backups"

# Archive filename
ARCHIVE_FILE="databases_`date +%F_%H-%M-%S`.tbz2"

# Archives older than this will be deleted
ARCHIVE_DAYS="15"

# Set or override config variables here
if [ -f $SYSCONFIG ]; then
    source $SYSCONFIG
fi

if [ -z "$USERNAME" ] || [ -z "$PASSWORD" ]; then
    echo "You must set USERNAME and PASSWORD in $SYSCONFIG";
    exit
fi

# Change working directory
cd $ARCHIVE_PATH

# Get all of the databases
for database in `mysql -u $USERNAME -p"$PASSWORD" -Bse 'show databases'`; do

        # Skip ones we don't want to back up
        if [ "performance_schema" == "$database" ]; then continue; fi
        if [ "information_schema" == "$database" ]; then continue; fi

        # Use Nice to dump the database
        nice mysqldump -u $USERNAME -p"$PASSWORD" --events $database > $database.sql

done

# Use Nice to create a tar compressed with bzip2
nice tar -cjf $ARCHIVE_FILE *.sql

# Remove the SQL files
nice rm -rf *.sql

# Remove old archive files
nice find . -mtime +$ARCHIVE_DAYS -exec rm {} \;

The post MySQL / MariaDB Backup & Archive Script appeared first on Justin Silver.

Here is how you upgrade from MySQL 5.5+ to MariaDB 5.5, 5.6 or 10.0.

After running MySQL 5.5 for a while and getting tired of not taking advantage of the features and performance enhancements, I took things to the next level and decided to just install MariaDB 10. It’s a drop in replacement for MySQL, which means that MariaDB will be able to use the same client binaries, data files, and configurations but will also support the new features found in the latest version of MySQL, as well as some things only found in the MariaDB fork.

Backup First!

The first things you should do is make a back-up of your existing configuration. MySQL has it’s data in /var/lib/mysql on my server, so I just ran cp -R /var/lib/mysql /var/lib/mysql-bak before getting wild.

Install MariaDB Yum Repository

You will then want to install the yum repository for the version of MariaDB you want to install. There is a pretty handy tool provided by the developers of MariaDB to choose the repository best suited for your system located at https://downloads.mariadb.org/mariadb/repositories/. On my setup – 64 bit CentOS 5.10 – I created a file at /etc/yum.repos.d/MariaDB.repo with the following contents.

# MariaDB 10.0 CentOS repository list - created 2014-04-21 22:57 UTC
# http://mariadb.org/mariadb/repositories/
[mariadb]
name = MariaDB
baseurl = http://yum.mariadb.org/10.0/centos5-amd64
gpgkey=https://yum.mariadb.org/RPM-GPG-KEY-MariaDB
gpgcheck=1

Remove MySQL

Since I already had a database running on this server, the next task is to remove the existing packages on the server. You can see the list of what’s installed with yum list installed | grep mysql. Just uninstall the MySQL packages with yum.

yum -y remove mysql*

Remove InnoDB Log Files

This isn’t a step that I took when I did this the first time, and it resulted in MariaDB not being able to start the mysqld daemon. Trying service start mysqld did nothing at all and running sudo -u mysql mysqld only got me as far as seeing the following error.

140421 23:10:40 mysqld_safe Starting mysqld daemon with databases from /var/lib/mysql
140421 23:10:40 [Note] InnoDB: The InnoDB memory heap is disabled
140421 23:10:40 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins
140421 23:10:40 [Note] InnoDB: Compressed tables use zlib 1.2.3
140421 23:10:40 [Note] InnoDB: Using Linux native AIO
140421 23:10:40 [Note] InnoDB: Using CPU crc32 instructions
140421 23:10:40 [Note] InnoDB: Initializing buffer pool, size = 128.0M
140421 23:10:40 [Note] InnoDB: Completed initialization of buffer pool
140421 23:10:40 [ERROR] InnoDB: Log file ./ib_logfile2 is of different size 0 bytes than other log files 5242880 bytes!
140421 23:10:40 [ERROR] Plugin 'InnoDB' init function returned error.
140421 23:10:40 [ERROR] Plugin 'InnoDB' registration as a STORAGE ENGINE failed.
140421 23:10:40 [Note] Plugin 'FEEDBACK' is disabled.
140421 23:10:40 [ERROR] Unknown/unsupported storage engine: InnoDB
140421 23:10:40 [ERROR] Aborting
140421 23:10:40 [Note] /usr/sbin/mysqld: Shutdown complete

After much searching, I found out that I just needed to remove the ib_logfiles before the call to service mysql start.

rm -rf /var/lib/mysql/ib_logfile*

Run mysql_upgrade

Once MariaDB is up and running, you should now be able to connect to the database server. That said, you may not actually be able to run any queries. Many of mine were returning with the error Error in query (1548): Cannot load from mysql.proc. The table is probably corrupted. Running mysql_upgrade to complete the switch to MariaDB did the trick and all was now working as expected.

mysql_upgrade -u root -p

Start MariaDB

I am used to the init.d script being known as mysqld so I accounted for this by running ln -s /etc/init.d/mysql /etc/init.d/mysqld before starting the daemon.

service mysqld start

The post Upgrade From MySQL to MariaDB on CentOS appeared first on Justin Silver.

If you ever have the need to convert a MySQL (or MariaDB) table from the MyISAM engine to the InnoDB engine, you can use the following script to get it done for an entire database. There are some caveats of course – InnoDB requires a primary key, does not support fulltext indexes, etc. – but if you’re feeling lazy….

DBUSER=root
DBPWD="my password";
DBNAME="database_name";
mysql -u "$DBUSER" -p"$DBPWD" "$DBNAME" -e \
"SHOW TABLE STATUS WHERE Engine='MyISAM';" | \
awk 'NR>1 {print "ALTER TABLE "$1" ENGINE = InnoDB;"}' | \
mysql -u "$DBUSER" -p"$DBPWD" "$DBNAME"

The post Convert From MyISAM to InnoDB Engine appeared first on Justin Silver.

After installing phpMyAdmin I kept getting the above error which reads:

The mysqli extension is missing. Please check your PHP configuration. <a href="Documentation.html#faqmysql" target="documentation"><img src="themes/dot.gif" title="Documentation" alt="Documentation" /></a>

I had run “yum install php-mysqli” with success, and “php -i | grep mysqli” was showing the mysqli was enabled but looking at phpinfo() through the Apache it didn’t look like mysqli was enabled after all. It then occurred to me that I am using the Apache Worker MPM, which requires the use of php-zts rather than regular php. It looked like php-mysqli doesn’t include the extension for php-zts, which was verified by checking out /usr/lib64/php-zts/modules – no php_mysqli.ini file here.

What to do.

The answer for me was to remove php-mysqli and replace it with php-mysqlnd.

>yum remove php-mysqli
yum install php-mysqlnd
service httpd restart

Reload the page, and the error should be gone. Yay!

The post phpMyAdmin + CentOS + Apache Worker MPM appeared first on Justin Silver.

The documentation for Trac is decent, but it isn’t super clear what to do, especially if you aren’t familiar with python. Some quick random info:

• For this example we are going to be hosting a Trac server using a vhost in Apache via mod_python
• Download Trac from http://download.edgewall.org/trac/Trac-0.12.3.tar.gz
• PYTHON_EGG_CACHE on my system is set to /tmp. It has temp files and whatnot.
• Subversion repository is install in /var/svn, but we’ll assume there is already a repository there called repository.
• Trac projects are going to be in /var/trac, go ahead and create it.
• Everywhere you see PROJECT replace it with your project name

First install the prerequisites using yum, then make sure you have everything with easy_setup

yum install python MySQL-python mod_fastcgi mod_python subversion
wget http://peak.telecommunity.com/dist/ez_setup.py
python ez_setup.py
easy_install Genshi

Now to install Trac, we must first get the source and then run the installer. Note that you will need root or sudo access to install.

cd ~
wget http://download.edgewall.org/trac/Trac-0.12.3.tar.gz
tar -xzvf Trac-0.12.3.tar.gz
cd Trac-0.12.3
sudo python ./setup.py install

Now let’s create the Trac project. It will ask you for a project name and a database connection – I’m using Mysql and already created a project schema called trac_PROJECT and user that has full access to it. My connection string looks like mysql://user:password@host/trac_PROJECT,

mkdir /var/trac
trac-admin /var/trac/PROJECT initenv

Now edit the trac.ini file for your project, located at /var/trac/PROJECT/conf/trac.ini. We want to set the properties “repository_dir = /var/svn/repository” and “repository_sync_per_request = ” – remove (default).

Next we need to create an htpasswd file, as this is what Trac uses for authentication. Enter a password for the admin user. If you want to create more users, leave out the -c since this creates a new file every time and will overwrite your first entries if it’s present. The next line gives this user TRAC_ADMIN rights to your project

sudo htpasswd -c /var/trac/.htpasswd admin 
trac-admin /var/trac/PROJECT permission add admin TRAC_ADMIN

Create the commit hook in your Subversion repository.

/var/svn/repository/hooks/post-commit

#!/bin/sh
export PYTHON_EGG_CACHE="/tmp"
/usr/bin/trac-admin /var/trac/PROJECT changeset added "$1" "$2"

Make it executable

chmod 755 /var/svn/repository/hooks/post-commit

Configure Apache to use mod_fastcgi and not mod_php to serve PHP files – this is the first step to serving up Trac pages.

First we need to create the FastCGI wrappers that we will use to execute requests. We are going to need two, one for PHP and one for Trac. Both will go in /var/www/cgi-bin and it’s important to make sure that they are executable.

/var/www/cgi-bin/php.fcgi

#!/bin/sh
export PHP_FCGI_CHILDREN=4
exec /usr/bin/php-cgi -c /etc/php.ini

The second file is named trac.fcgi and will contain the following – note that this is one place where PYTHON_EGG_CACHE is set, and that we have to include the parent path to our Trac environment. You can also serve a single project by just setting TRAC_ENV instead of TRAC_ENV_PARENT_DIR.

/var/www/cgi-bin/trac.fcgi

#!/usr/bin/env python
import os

from trac.web.main import dispatch_request
try:
    from flup.server.fcgi import WSGIServer
except ImportError:
    from trac.web._fcgi import WSGIServer

if __name__ == '__main__':
    os.environ['TRAC_ENV_PARENT_DIR'] = '/var/trac'
    os.environ['PYTHON_EGG_CACHE'] = '/tmp'
    WSGIServer(dispatch_request).run()

Make both executable:

chmod 755 /var/www/cgi-bin/*.fcgi

Now we need to tell Apache to use these FastCGI wrappers. In /etc/httpd/conf.d/ you will need rename php.conf to php.conf.off and edit fastcgi.conf.

mv /etc/httpd/conf.d/php.conf /etc/httpd/conf.d/php.conf.off

Now create the FastCGI wrapper for PHP.

/etc/httpd/conf.d/fastcgi.conf

LoadModule fastcgi_module modules/mod_fastcgi.so

FastCgiServer /var/www/cgi-bin/php.fcgi
AddHandler php-fastcgi .php

SetHandler fastcgi-script
Action php-fastcgi /cgi-bin/php.fcgi
DirectoryIndex index.html index.shtml index.cgi index.php
AddType application/x-httpd-php .php

Create a VirtualHost in Apache that will serve Trac.

<VirtualHost *:80>
        ServerName trac.DOMAIN.com
	DocumentRoot /var/trac
	<Directory /var/trac>
	        Order allow,deny
	        Allow from all
	</Directory>
	ScriptAlias / /var/www/cgi-bin/trac.fcgi/
	<LocationMatch "/[^/]+/login">
		AuthType Basic
		AuthName "Trac"
		AuthUserFile /var/trac/.htpasswd
		Require valid-user
	</LocationMatch>
</VirtualHost>

Restart Apache by running

service httpd restart

You should be able to visit http://trac.DOMAIN.com/PROJECT/login in your browser and log on using admin and the password you selected. All you have left to do is enable the plugins to tie Trac and Subversion commits together.

Choose Admin on the top right, then Plugins on the left. You will need to click the arrow to expand the list of plugins and go to the bottom. You should see CommitTicketReferenceMacro and CommitTicketUpdater – check the box next to them to enable both.

Should be good to go!

The post Trac 0.12.3 + MySQL + Apache + Subversion on CentOS 5.5 appeared first on Justin Silver.