Amason S3 cloud backup

I’m looking at the S3 service from amazon as a solution for backing up all data from our Plesk servers. We already have backups saved to each server using totalbackup from 4psa, and to another server. However, they are all in the same datacentre, and it wouldbe nice to have another layer of redundancy.

1. sign up to S3
2. get the accesskey and secretkey
3. use a service like s3browse.com to look at your space and create a “bucket”
4. use something like duplicity to backup the entire system
   

S3 uses the concept of buckets to refer to a collection of all the data that is backed up. Each bucket has a name: however, each name is unique across all users. So, the ‘testing’ bucket won’t be available for anybody else.

Installing duplicity on ubuntu and setting up the backup

sudo apt-get install duplicity python-boto

Spamassassin upgrade (Debain etch)

I needed to install the latest version of spamassassin on one of the servers because of the bug that flags false positive results for MS Outlook (http://wiki.apache.org/spamassassin/FORGED_MUA_OUTLOOK):

apt-get install debian-backports-keyring

Edit /etc/apt/sources.list

deb http://www.backports.org/debian etch-backports main contrib non-free

Do an apt-get update

apt-get -t etch-backports install spamassassin

Confirm whether you want to keep the old config files and the latest backport will be installed. You can check that latest version of all packages using:

dpkg -l

Plesk and qmail

I spent several hours this evening tying down a problem with the qmail server on one of our machines. Eventually, I found that 77000 messages had been sent from an auto-emailer function triggered by a malformed piece of MySQL on another server.

However, it has allowed me to find a use a couple of command line functions that I’ve not really used all that much before

/etc/init.d/qmail stop | start | restart

/var/qmail/bin/qmail=qstat

qmHandle -l

qmHandle -h’Subject: blah’

qmHandle is a helpful command line tool that allows you to manipulate the qmail queue (remembering that you need to edit the perl script once downloaded to set the user config)

# Set this to your qmail queue directory (be sure to include the final slash!)
my ($queue) = ‘/var/qmail/queue/’;
my ($bigtodo) = (-d “${queue}todo/0″) ? 0 : 1; # 1 means no big-todo

# While this is if you have a Debian GNU/Linux with its qmail package
my ($stopqmail) = ‘/etc/init.d/qmail stop’;
my ($startqmail) = ‘/etc/init.d/qmail start’;

# Enter here the system command which returns qmail PID.
my ($pidcmd) = ‘pidof qmail-send’;

NautilusSVN

This is an absolute dream... like many who use SVN and have used Windows in the past, TortoiseSVN was it. There wasn't anything like it for Ubuntu... or at least that's what I thought until I took a look around and finally found: NautilusSVN on GoogleCode.

It's not as feature-rich as TortoiseSVN, but it does what it needs to do and has icon overlays which prettily tell you when files are out of sync with the repro.

Thanks guys!

Continued PCI Plesk issues

I thought I'd fixed the problems that related to PCI compliance, but it would appear that I was wrong.

The most recent PCI check flagged SSL2 errors on the imaps and pops ports. Thanks to a helpful site, the solution wasn't that far away though:

sudo -s
joe /etc/courier-imap/imapd-ssl
TLS_CIPHER_LIST="HIGH:MEDIUM:!SSLv2:!LOW:!EXP:!aNULL:@STRENGTH"

sudo -s
joe /etc/courier-imap/pop3d-ssl
TLS_CIPHER_LIST="HIGH:MEDIUM:!SSLv2:!LOW:!EXP:!aNULL:@STRENGTH"

Then restart courier:

/etc/init.d/courier restart

You can test the connection with the following:

openssl s_client -connect localhost:995 -ssl2
openssl s_client -connect localhost:995 -cipher EXP:LOW

Loading SSL certificates in Plesk

Recently had an issue with the SSL certificates on our Plesk servers. After some playing around, it turned out to be an issue with the way IE and Firefox handle the certificates differently and the way in which they are loaded into the Plesk GUI.

Our certificates are purchased through UK2 and uploaded to Plesk > 8.1

• Alter the domain to run on an exclusive IP address
• update dns settings for domain appropriately.
• plesk -> select domain -> settings -> select IP from 'IP address' drop down
• Get a new SSL certificate for the domain.
• Plesk -> select domain -> certificates -> add new certificate
• complete form.
• copy resulting CSR to clip board
• request free SSL cert from uk2 control panel for the related server
• Once it's issued, manipulate supplied SSL certificate files
• From the supplied zip file, open UTNAddTrustServerCA.crt and AddTrustExternalCARoot.crt PositiveSSLCA.crt within a text editor.
• Create a NEW file with the content of each of the above files (in same order) with no spaces between them and save it as chain.crt (concatenate the files)
• Upload the www_domain_name_com.crt file and chain.crt file to the server.
• Plesk -> select domain -> certificates -> the new certificate
• Under 'upload certificate files', browse for www_domain_name_com.crt under 'certicate' and chain.crt under 'CA certificate' and then click send file.
• Alter the domain to use the new certificate
• Plesk -> select domain -> settings -> select new certificate from the certificate drop down

SSL and Plesk (IE and Firefox)

Recently had an issue with the SSL certificates on our Plesk servers. After some playing around, it turned out to be an issue with the way IE and Firefox handle the certificates differently and the way in which they are loaded into the Plesk GUI.

Our certificates are purchased through UK2 and uploaded to Plesk > 8.1

• Alter the domain to run on an exclusive IP address
• update dns settings for domain appropriately.
• plesk -> select domain -> settings -> select IP from 'IP address' drop down
• Get a new SSL certificate for the domain.
• Plesk -> select domain -> certificates -> add new certificate
• complete form.
• copy resulting CSR to clip board
• request free SSL cert from uk2 control panel for the related server
• Once it's issued, manipulate supplied SSL certificate files
• From the supplied zip file, open UTNAddTrustServerCA.crt and AddTrustExternalCARoot.crt PositiveSSLCA.crt within a text editor.
• Create a NEW file with the content of each of the above files (in same order) with no spaces between them and save it as chain.crt (concatenate the files)
• Upload the www_domain_name_com.crt file and chain.crt file to the server.
• Plesk -> select domain -> certificates -> the new certificate
• Under 'upload certificate files', browse for www_domain_name_com.crt under 'certicate' and chain.crt under 'CA certificate' and then click send file.
• Alter the domain to use the new certificate
• Plesk -> select domain -> settings -> select new certificate from the certificate drop down

Mounting home on a different partition

I recently did a reinstall of Ubuntu and rather than loosing all my personal data and having to copy it all back from the backup server, decided to put /home on a separate partition, leaving the rest of the drive for Ubuntu.

Thanks to the help on the Ubuntu blog, I got it set up and running with no problems, leaving a quick mount after the reinstall to get think back up and running. No more faffing around trying to get the machine setup as it was before... all the app specific information is already where it needs to be.

sudo -s
move /home /home1
mkdir /home
mount /dev/sda1 /home
joe /etc/fstab

/dev/sda1 /home ext3 nodev,nosuid 0 2

Find and remove

Was needing to find all the .jpg files in my music folder and remove them - good old terminal...:

find -name "*.jpg" -exec rm {} \;

Hiding apache and PHP version numbers

Finally figured out how to hide the fact that apache and PHP weren't the latest version. Needed to do this becuase of PCI checking on the server which doesn't taken into account that the non-latest versions of the software have backported security.

Apache:

edit /etc/apache2/apache.conf and set:

ServerSignature Off
ServerTokens Prod

PHP:

edit php.ini and set:

expose_php = Off

For plesk, the php.ini file is in /usr/local/psa/admin/conf on a Debian related system. The httpd.conf file is in this directory too.

Terminal based IP traffic monitor

I'd been looking for a terminal based IP traffic monitor for a while but had never found one fit for purpose.

Not sure where I was looking though, because I found this fine little package: iptraf. It's not the prettiest thing in the world, but it works and works well.

automating mysqltuner

I found the mysqltuner a while back and had been using it on one of my servers sporadically. I suddenly thought... why don't I run it automatically via a cron every week and test the server optimisation. Easy...

sudo -s
wget http://mysqltuner.com/mysqltuner.pl
chmod +x mysqltuner.pl
cp ./mysqltuner.pl to /usr/local/bin/mysqltuner

Ensure that it's in the right place by running it and then optimise the current mysql server and restart it:

mysqltuner
joe /etc/mysql/my.cnf
/etc/init.d/mysql restart

Now we need to add a cron entry to make it run every week.

crontab -e

43 4 * * * thur mysqltuner

And that's it, saving the crontab file will install it and on the next thursday at 4:43 in the morning, the optimiser will run as root and email the server admin the output.

Multiple apache sites on localhost

I do all my application development locally, but for a while haven't needed to have two instances of locally running applications. In the past, I had setup apache to serve multiple local instances, but I hadn't written down the process.

First of all, alter /etc/hosts to add a line for your new site.

127.0.0.1 localhost.localdomain localhost<br />127.0.0.2 localccc.localdomain localccc

Then, alter the apache config: create a file per site in /etc/apache2/sites-available:

<VirtualHost localhost>

ServerName localhost
ServerAdmin webmaster@localhost
ServerSignature On

DocumentRoot /home/elements/htdocs/

<Directory>
Options FollowSymLinks
AllowOverride None
</Directory>

<Directory home/elements/htdocs/>
Options Indexes FollowSymLinks MultiViews
AllowOverride All
Order allow,deny
allow from all
</Directory>

ErrorLog /var/log/apache2/error.log
CustomLog /var/log/apache2/access.log combined
LogLevel warn

</VirtualHost>

Also remove the contents of default: just empty it.
Then enable the sites you've created (run it once for each filename you've created):

a2ensite localhost

Finally, restart the server

/etc/init.d/apache

And, as Gordon Ramsay would say... DONE.

PCI compliance

Some of my clients are being hassled with these new PCI checks... not that they're really all that new, but PDQ companies are certainly using them a lot more.

A really helpful article can be found here: Making Plesk more PCI compliant

On the last check, the servers were failing on several fronts, and I thought I'd note down some pointers for the future.

1.
SSH host keys. Being Debian, the servers fell fowl of a packaging mistake with SSH in which the keys became less than secure... need to re-generate them.

sudo -s
rm /etc/ssh/ssh_host_*
dpkg-reconfigure openssh-server

Then all you need to do is go around the update all the known_host files... yawn

2.
PHP version. Problem with Plesk 8.4 is that is uses one version of PHP for Apache and one for the control panel. And, the version used for the control panel was horribly out of date.

The version used for the forward facing pages is not completely up to date either, so I've used the dotdeb repros to get the latest versions.

sudo -s
joe /etc/apt/sources.list

deb http://dotdeb.netmirror.org/ stable all
deb-src http://dotdeb.netmirror.org/ stable all

apt-get update
apt-get upgrade

Make sure that you keep your old config files, or at least do a diff to work out which bits have been changed.

The version that is used for the Plesk control panel is independant on that installed on the system and is shipped separately. According to the Forum, it will be upgraded to 5.2.6 from the next release... 8.5... whenever that will be... have posted to the thread to find out.

As well as this, the scores for psa were being counted twice because port 8880 and port 8443 were both open and creating the same errors. As far as I can tell, port 8880 isn't used in a standard setup. So, until the next release, I thought I'd just try and block its use.

sudo -s
joe /opt/psa/admin/conf/httpds.conf

#Listen 8880

/etc/init.d/psa restart

You can then test the connection to port 8880 from a remote location.

telnet servername 8880

3.
Somebody had left a phpinfo() file in a forward facing position. (5 points)

4.
Apache needs to be over 2.2.8 and if it's not, that's another 6 points.

5.
Port 53 was open
Close it using the Firewall module in Plesk (DNS server)

6.
SSL 2.0 was being used... that added another 4 points.

sudo -s
joe /etc/apache2/httpd.conf

SSLCipherSuite ALL:!ADH:!LOW:!SSLv2:!EXP:+HIGH:+MEDIUM
SSLProtocol all -SSLv2

/etc/init.d/apache2 restart

You also need to alter the psa (Plesk Server Admin):

sudo -s
/opt/psa/admin/conf/httpsd.custom.include

SSLCipherSuite ALL:!ADH:!LOW:!SSLv2:!EXP:+HIGH:+MEDIUM
SSLProtocol all -SSLv2

/etc/init.d/psa restart

(If the .custom.include file doesn't exist, create it: it will ensure that the config changes persist on package upgrade)

You can double check that the server doesn't respond to a SSL2.0

openssl s_client -connect hostname:443 -ssl2
openssl s_client -connect hostname:8443 -ssl2
openssl s_client -connect hostname:25 -starttls smtp -ssl2

If SSL2.0 is disabled, you should get the following returned.

419:error:1407F0E5:SSL routines:SSL2_WRITE:ssl handshake failure:s2_pkt.c:428:

420:error:1406D0B8:SSL routines:GET_SERVER_HELLO:no cipher list:s2_clnt.c:450:

.

Plesk 8.4 upgrade

When I bought Plesk initially for my growing server farm (small collection of servers that sits in a proper server farm), I didn't purchase the continuing version support... I didn't think it was worth the cash.

Well, I finally decided to crack out the credit card and but upgrades for them all. Especially since an amnesty on upgrades was being offered by the new owners, Parallels, and it was cheaper...

So, armed with my new keys, I went ahead and upgraded... it was one of those fingers crossed moments, but I'd backed everything up. And... everything worked... there were some niggles with the switch from PHP4 to PHP5, but everything went surprisingly well.

Now I just need to get round to upgrading Debian to etch on two of the older machines... that's not going to be as pretty, I'm sure.

Upgrading Ubuntu server

I'm not quite sure why I'd not got round to upgrading my Ubuntu server locally, but... I hadn't.

Took me a wee while to work out exactly how to do it, so I thought I'd post the results of my searches here... so that I have it for next time.

sudo -s
apt-get update
apt-get upgrade
apt-get dist-upgrade

apt-get install upgrade-manager-core
do-release upgrade

Simple...

MySQL optimisation

I found a nice MySQL tuner at mysqltuner.com
After using it to find out where there were issues, all I needed to do was alter the my.cnf file in /etc/mysql/

I added the following definitions and saw a significant improvement in efficiency.

#
# * Personal additions JBJ
#
set-variable = thread_cache_size=4M
set-variable = join_buffer_size=128K
set-variable = table_cache=512
set-variable = key_buffer_size=64M

log-slow-queries = /var/log/mysql/mysql-slow.log
long-query_time=1

innodb_buffer_pool_size = 16M
innodb_additional_mem_pool_size = 2M

Adding mysql admin user

Needed to add an admin to the mysql db and couldn't remember how:
Log in with admin credentials from a root login

sudo -s
mysql -uadmin -p`cat /etc/psa/.psa.shadow`

Then:

mysql> GRANT ALL PRIVILEGES ON *.* TO 'monty'@'localhost'
-> IDENTIFIED BY 'some_pass' WITH GRANT OPTION;
mysql> FLUSH PRIVILEGES;

Updating SA

A while ago I blogged about backporting SA for Debian.
Once the SA is updated, run sa-update to get the most up to date rules.
You can also add other "channels" to add more updates for SA - currently using the ones from saupdates.openprotect.com and loving them! They have a nice todo on their site.

Make sure that gnupg is installed
sudo apt-get install gnupg

gpg --keyserver pgp.mit.edu --recv-keys BDE9DC10
gpg --armor -o pub.gpg --export BDE9DC10
sudo -s
sa-update --import pub.gpg
sa-update --gpgkey D1C035168C1EBC08464946DA258CDB3ABDE9DC10 --channel saupdates.openprotect.com

All the updates now live at /var/lib/spamassassin/~version_number/
Happy spam hunting.