Server Infrastructure Updates - Part 1: The Servers

Hosting Provider:

For a few years now, I've hosted all my severs with Linode. They are great.

In the past, I've had shared hosting plans, Virtual Private Servers (VPS), and full blown dedicated servers with many different providers. So why Linode? Well, for starters, they are priced right. I can spin up a 1024 virtual host (1gb of memory) for $10/mo https://www.linode.com/pricing. That's hard to beat. They also make it easy to scale the servers vertically, I can in a matter if minutes add resources to an individual Linode with ease. Making it easy to grow with traffic increases.

Linode provides a great API allowing me to automate things like Linode resizes, DNS changes and server reboots programmatically. Since I've been with them, they've more than doubled the resources provided to a single Linode, and have preformed HUGE network and infrastructure updates. Including generous chunks of space on SSD arrays. I can't say enough good things about Linode. If you are in the market for hosting, I'd seriously encourage you to check them out.

Note: I've not been compensated in any way for this article by Linode, however if you'd like to thank me for pointing you their way feel free to signup using my referral code: c93ddeb29c3827142fe885c5b8d66b2334301bdc

Operating System:

Now that I've selected the location for the servers the first step was to provision them, a few clicks in the Linode account manager, and I had 3 simple installs of Ubuntu 14.04 (the current long term release from Ubuntu). I know there are those of you who would fault me for going with Ubuntu over CentOS or Redhat or any of the many other perfectly fine Linux Distro's out there (and believe me there are many and I've used them and they are all fine). I choose to stick with Ubuntu because that's what I've used for years, and it has worked for me.

Networking:

Iscreen capture from Linode manager configurations started with completely blank servers, and began the setup process. I configured all 3 to have 1 public and 1 private (internal to Linodes network) IP address. I gave each machine a descriptive name; LoadBalancer, webserver1, and webserver2. I also setup a DNS entry for each server in Linode's DNS Manager to make it easier to connect to and test each server independently.

I gave each server a through test and ensured connectivity from each to each, making sure each server could talk to both the public and private IPs of each of the other servers in the cluster.

Once I was satisfied that the networking was stable, I configured SSH. Since by default Linode assigns a root password for each server, I decided to lock that down. I uploaded my SSH Keys (both public and private) to /root/.ssh and configured ssh to only accept logins for root with a Key. I could go through the setup process for all of this here, but the basics are covered for it over in the Ubuntu Community Pages: https://help.ubuntu.com/community/SSH/OpenSSH/Configuring. It's always wise to disable root access completely or at least have it protected by a strong password and SSH key based encryption.

Server Software:

The next step was configuring the software. The first is to ensure the operating system and installation sources are up to date.

Two simple commands to do this:

apt-get install update
apt-get install upgrade

This ensures that you have the latest versions of the core Ubuntu packages ready and at your disposal.

Step 2 Install Apache:

Another single command to install Apache and any dependent packages.

apt-get install apache2

We should always check each step to be sure it's working, going back to figure out what we missed or what didn't work after several steps makes troubleshooting a nightmare. To check that Apache is alive and serving pages, simply navigate to the public IP of the server in your browser, something along the lines of http://12.34.56.78, if you did what I did and assigned a DNS entry for each server you could also navigate to that address (eg. http://webserver1.example.com). You should be greeted by a simple "It Works!" page from Apache.

Step 3 PHP Install:

Next step install PHP:

apt-get install php5 libapache2-mod-php5

That command installs PHP, the Apache PHP module. Again, lets test that to make sure it worked. To do this I delete the basic "It Works!" page provided by Apache usually stored at /var/www/html . I replace all the contents there with a single php page named phpinfo.php.

The contents of this page should be

<?php
phpinfo();
?>

Then we test that out by again navigating to the server in the browser (eg. http://webserver1.example.com/phpinfo.php) which should give you a chunk of data about your PHP installation. Feel free to take a read through the stuff there. Lots of interesting tidbits.

So now that we have Apache and php working well together, I like to add in a few common PHP Modules. Your needs may vary, but I find that I want at least the PHP GD module (for image manipulation), the MCrypt module, which handles cryptography functions for PHP, and the CURL library, which is used for communication with other servers. You may want to install others, or may not even need these. The nice thing is it's always easy to add them later if you find you need one.

apt-get install php5-mcrypt php5-gd php5-curl

Just to be sure we are still ok, we will restart Apache

service apache2 restart

And just for warm fuzzies, lets pull up that phpinfo page again (eg. http://webserver1.example.com/phpinfo.php) to be sure installing those modules didn't break things.

Step 4 Install MySQL:

Lets install MySQL, and the Apache and PHP modules.

apt-get install mysql-server libapache2-mod-auth-mysql php5-mysql

At some point during the install you should be prompted for a root password, this should again be a secure password. Make sure to store this password somewhere, as you will need it again.

MySQL comes with a few configuration scripts. Lets run them first

mysql_install_db

and

mysql_secure_installation

You will be prompted for that root password that we set a minute ago. It will prompt you with a few Yes or No questions for a secure installation, you should select yes for all of those, except for maybe changing the root password.

Step 5 Fail2Ban:

Fail2Ban is a great utility, it monitors the logs for multiple servers. I particularly like it for SSH (It can also protect Apache, ProFTP, Postfix, Dovecot, and many more). There are multiple configuration options for it, but I find the the defaults usually work for me.  Some people like to make modifications to the config file to setup email notifications, and IP Addresses to ignore. I don't do either of these, I see enough bans in the log that I would get a never ending stream of emails. While I like the idea of reserving a hole for myself by ignoring my IP or the IPs of the other servers, in the event that one of my machines is compromised, it would allow an easy path to compromising the next machine in line, and since the bans automatically expire we can just wait for that in the event we accidentally ban ourselves. More info is available for this in the Ubuntu Community https://help.ubuntu.com/community/Fail2ban

Install Fail2Ban:

apt-get install fail2ban

We can test this pretty easily, ideally from another location, by attempting to SSH into your machine with incorrect credentials. After several incorrect tries, Fail2Ban should add that IP to the Jail (I tested this from my Android phone over 4G so I didn't ban my home IP).

On the server you can check that the IP address has been added to the Jail by running:

iptables -L

That should be enough to get you running with an extra level of protection.

Step 6 Webmin:

I'm sure this one will catch me some flack from the "hardcore" Linux administrators. Generally for administration of my servers, I'll use SSH and issue command line commands to perform nearly all major work, however for some common tasks I find Webmin to be very useful. I like it for reviewing logs, the log viewer has a simple search function ("Only show lines with text"), as well as the ability to limit it to a selectable number of recent lines. Handy for managing CRON jobs, users and groups, MySQL and Apache. I find it much easier to pull up Webmin to make quick changes when I'm away from a computer, than to make a SSH connection from my cell phone. A quick note about Apache, I noticed as of the current versions Webmin 1.700 and Apache 2.4.7, that Webmin still seems to be using the "Allow from  All" format for access permissions instead of the newer "Require All Granted" that came with Apache 2.4.

Installation of Webmin is pretty easy, I prefer to install everything using APT if possible, as it provides a simple method of keeping them updated.

First, edit the /etc/apt/sources.list file to add the lines:

deb http://download.webmin.com/download/repository sarge contrib
deb http://webmin.mirror.somersettechsolutions.co.uk/repository sarge contrib

The easiest way to add these lines to the sources list via the command line is to execute:

add-apt-repository 'deb http://download.webmin.com/download/repository sarge contrib'
add-apt-repository 'deb http://webmin.mirror.somersettechsolutions.co.uk/repository sarge contrib'

 

You'll also need to add the authors GPG key to APT, using the following commands:

cd /root
wget http://www.webmin.com/jcameron-key.asc
apt-key add jcameron-key.asc

Once that is done, have  APT update the sources and install webmin.

apt-get update
apt-get install webmin

If everything went smoothly, APT should have installed Webmin and all of it's dependencies, and you can now log in to Webmin at https://webserver1.example.com:10000 . I must point out at this point that Webmin does allow login as root, with out the SSH key restrictions, so it is of vital importance that you have a strong password for the root user, and look into using one of the 2 factor authentication methods built into Webmin.

Finishing Touches:

Now would be a good time to switch over to the Linode Manager and "Imagize" (currently in beta) your new Linode. Linode allows you to save 10gb worth of disk images to be used to image new Linodes, making starting additional Linodes much easier and faster as your new working config is instantly in place. If you are going to continue on with the rest of this guide, setting up multiple load balanced machines, this will be very helpful. I'll get more into that in the next few articles along with setting up file system sync, MySQL replication, and other essentials.