10 steps to migrate Web site servers with the least of problems
* 1. Prepare your DNS
* 2. Setup the new server
* 3. Tune the server file system for performance and integrity
* 4. Install the base applications
* 5. Initial data migration
* 6. Put the old server in migration mode
* 7. Bring your new server to life
* 8. Double check your server security
* 9. Epilogue
* 10. The forgotten step that you remembered
So the PHPClasses site server had to be migrated. The main reason was bandwidth. The new served had premium bandwidth. This means the data center guaranteed the necessary bandwidth to my server.
Some data centers offer un-metered bandwidth. This means you can transfer as many data as you want, but the available bandwidth is shared with all other servers in the same data center.
Additionally, since a new machine was used in the new data server, it uses a faster CPU, larger hard disk and more RAM.
The choice of the new data center was handled by Greg of Net Virtual. So he knows better the details of the bandwidth offers. For me, what matters is that I am paying the same price as before for a better server in a better hosting facility.
Despite the benefits of migrating to a better server, the actual migration is a complicated process that can fail in so many ways that I thought it would be a good idea to share what you need to be concerned if you need to migrate your servers too.
You need to go through several steps that need to be done in the right order to prevent major headaches.
- 1. Prepare your DNS
When you move a site to a new server in a different network, it will get new IP addresses. This means that during the switch between the old and the new server, your site domain records in the DNS need to be updated.
The problem is that DNS servers cache domain name queries for as long as possible to provide the fastest response.
Usually the queries are cached for days. So if you change the DNS records for your domain, many sites users will only see the change of server after several days.
The solution for this problem is to to lower the TTL (Time To Live) values of your DNS records. This is a value that indicates for how long DNS queries can be cached. When the time that has passed exceeds the TTL, DNS queries need to be done again.
So the first thing you need to do before migrating is to lower the TTL to reasonable levels. Initially you can set it to 30 minutes. It depends on how long you expect to be your downtime during the actual server switch.
This must be done days before the actual migration day. Usually you migrate the site on a weekend day or whatever is the day of the week with less traffic in your site. So, if you are going to migrate on Saturday, lower the TTL on a Tuesday or Wednesday.
Also if you are using SPF (Sender Framework Policy), it is advisable that you disable it now.
SPF are special DNS TXT records that you can use to tell the world that e-mail sent from your domain can only come from servers with certain IP addresses. This is used to avoid that spammers forge messages with addresses from your domain.
http://en.wikipedia.org/wiki/Sender_Policy_Framework
Since your site will be changing IP addresses, either you remove the SPF records or add both the old and new server IP addresses.
You also need to setup a temporary migration sub-domain pointing to the IP of the new server. This is explained below.
For instance, if your site domain is www.somedomain.com, create another sub-domain named temporary.somedomain.com set to the new server IP.
- 2. Setup the new server
Once the new server is ready in the data center, you need to install the operating system and setup the new IP addresses.
Good data centers provide you the services of a capable systems administrator that can install the OS for you. If for some reason you need or you prefer to install the OS yourself or it has to be done by a person that you trust, all the decent data centers can provide you a KVM switch.
KVM stands for Keyboard, Video and Mouse. It is an hardware and software solution that has provides means for you to control the actual machine remotely, so you do not have to go to the data center in person.
http://en.wikipedia.org/wiki/KVM_switch
It consists of a box with cables that send keyboard and mouse signals and captures video output. The operator uses a client program that communicates with the KVM switch box using a regular Internet connection.
Since before you install the OS the server network settings are not configured, using a KVM is essential for anybody to configure the server remotely. KVM is also great for accessing a server that may have crashed or for some reason it cannot be accessed remotely.
- 3. Tune the server file system for performance and integrity
While you setup your server, you also need to partition the hard disks and setup RAID configuration.
RAID stands for Redundant Array of Independent Disks. This means that your server must use several redundant disks.
http://en.wikipedia.org/wiki/Redundant_array_of_independent_ ...
Having a RAID setup is essential, not only to achieve greater performance, but also assure that if one disk fails, at least another disk will take over and your server does not stop until the bad drive is replaced.
Therefore, you should use at least RAID 1. This means that your system only sees one disk, but there are two disks with exactly the same data being replicated.
There other levels of RAID that provide greater performance. Obviously you need to spend more money in hardware and controllers.
A cheap alternative is to use software RAID. This means that the operating system will take care of reading and writing the data to the disks. Nowadays software RAID can be almost as fast as hardware RAID. This way you save the money of the cost of buy and hardware RAID controller.
Additionally, you can setup several partitions achieve greater performance and integrity of the data. For instance you can have a partition for the database server, a partition for the e-mail server, a partition for the Web server and a partition for all the rest.
This way the accesses to the database server partition will not be hold by the accesses to the e-mail server partition and so on.
Also, if you have a file system problem in one partition, the data in other partitions is not affected. You can fix partition problems independently.
Since you can have different types of file systems on each partition, you can tune each partition according to the purpose of its data.
For instance you can use a transactional file system for the database partition like for instance ReiserFS, but use a non-transactional (faster) file system like ext2fs for the e-mail server, that usually handles less critical data. It depends on how critical each kind of data is.
Once the file system partitions are set, you may need to create users and groups in your system. When you do that try to copy the same user and group identifiers used in the old server, so you do not get problems with denied permissions.
- 4. Install the base applications
Once you have installed the OS and setup the file system, you are ready to install the base applications, like the Web server, e-mail server, database server, PHP, etc..
I recommend that you use the exact same versions of these applications on the new server that you were using on the old server.
If you want to upgrade the version of some application, do it before or after the migration. Never do it as part of the migration procedure.
This way it reduces the chances of the migration to break because something in the new application versions work differently.
If you have installed a newer version of the OS in the new server, it is very likely that it will install newer versions of the applications.
In this case I strongly recommend that you do not install the versions that come in the OS installation media. Instead try to find the old versions and install them manually. This will help you migrate the database data much faster, as you may read below.
If you use MySQL, you may find older version packages in their site. Same goes for PHP, Apache, etc.. If you do not find old version packages with compiled binaries, you may need to build the binaries from source archives.
Actually recompiling applications from source may get you better performance, as the binaries will be optimized for your platform.
- 5. Initial data migration
Once the base applications are installed, you are ready to do the initial migration of your application data. This includes the database data, your application code files, mail server account configuration, logs, file caches, etc..
For simple file migration, the best tool to use is rsync. This is a tool that replicates files and directories from one machine to the other.
http://samba.anu.edu.au/rsync/
For migrating databases this is a bit more delicate matter. If the old and the new servers use different versions of OS, CPU or database server version, you may need to use database backup and restore methods to migrate the databases.
Since this is something that may take a long time, you need leave the database migration for a later step to minimize site down time.
However, if the OS, CPU, and database server are exactly the same, you may also use rsync to synchronize database server data files. This may reduce the migration down time from hours to minutes.
You may still migrate database data with rsync when the OS versions are not very different. For instance I used rsync to migrate MySQL database data that was in a server with OpenSuse 9 to a new server with OpenSuSE 11 both using different 32 bits Linux 2.6 kernel revisions.
When in doubt if it will work, use rsync to migrate the data initially and the start the database server. Use database table check queries to verify that all worked well.
The initial data migration done with rsync may take a while. But when you do the final data migration step using rsync again, it is all much faster and rsync only copies the parts of the files that changed.
- 6. Put the old server in migration mode
You need to configure your old server so it does nothing that changes your site data. This means it must not change your site database in any form. Make your old server Web server only exhibit a message that says the site is being migrated.
To make it safer, shut down your database server and maybe even your e-mail server, so no database data is changed nor any e-mail is processed.
If you have maintenance tasks being run from a task scheduler program like cron, disable all tasks now.
When you are sure all data is frozen, you are ready for the final data migration step.
Since this is the last step before switching to the new server, you also should lower further the TTL of the DNS records before migrating the database data.
That will help making all site users start accessing the new server much faster. Set it to 5 minutes to make them all see the new server very quickly.
Now just use rsync to synchronize changed files, like logs and cache files. If you have used rsync to migrate your database data, use rsync again to do the final database data migration. Make sure the database server in the new server is shutdown before you do this.
If you could not use rsync to migrate database data, you need to use a database backup tool instead, like for instance mysqldump in the case of MySQL. Then transfer the backup files to the new server and restore the database.
- 7. Bring your new server to life
Once all the data is moved to the new server, you are ready to bring it to life.
Your Web server in the new machine must be already configured to use the new server IP addresses by now. Once that is done, you should change your site domain DNS records to point to the new server address.
Even if you do all this, some DNS servers in world will still be caching queries pointing to the IP of the old server. To make all users come to the new server, you should configure the Web server to redirect all users to a special temporary domain that was previously configured in the first step.
In Apache you just need to add a line like this to your virtual host configuration of the old server:
Redirect / http://temporary.somedomain.com/
It will look like this:
DocumentRoot ...
ServerName www.somedomain.com
Redirect / http://temporary.somedomain.com/
In the new server you need to add a line like this to your domain virtual host configuration:
ServerAlias temporary.somedomain.com
It will look like this:
DocumentRoot ...
ServerName www.somedomain.com
ServerName temporary.somedomain.com
Just keep in mind that cookies from one domain may not be shared with other domain. So if you use cookies based sessions for logged users, users that are redirected this way will no longer be seen as logged.
- 8. Double check your server security
In your old server you may have taken some security precautions, like closing unused ports in the firewall, auditing the server security, etc... You should take the same precautions on the new server, especially if it uses a different version of the OS and applications.
For instance, if you use security auditing tools on the old server, use them again on the new server.
You should for instance security tools like nessus. This is a free tool that performs thousands of tests to verify if your server has known security vulnerabilities.
http://www.nessus.org/nessus/
If you do not do this, script kiddies will do it and abuse from your server soon or later.
Nessus will tell you if you are using insecure software that needs to be updated or you have unnecessary ports opened.
Some Linux distributions already come with nessus. However, it is better to download and install the application from the Nessus site. Usually it is a newer version and it does not require you to install all the GUI packages on the server.
Nessus is a client-server application. You can use a client program from your desktop machine that communicates with the server and performs the tests. You can run both the client and the server from your desktop machine, but then you may not be able to test local server vulnerabilities.
There is a commercial version of Nessus, which is basically the same, but it is updated more quickly in terms of tests of new vulnerabilities that were found. You need to pay a subscription to have access to the most recent test scripts.
You need to understand a bit about security to understand the reports and fix the issues that may be reported. Alternatively, you may also use online professional auditing services from companies like SecuritySpace. Once you pay for the audits, they perform automated tests from their servers to check your server.
http://www.securityspace.com/sspace/index.html?refid=1057382 ...
Some of these security checks can and should be done before the actual migration to the new server. The only reason why I listed this step after the migration is because your server will not yet be responding under your site domain before you make the DNS switch.
- 9. Epilogue
Finally, you may re-enable the your site maintenance cron tasks.
You should also restore your DNS records TTL value, usually set to several days.
If you have disable the SPF record in the first step, you may re-enable it after a few days when all the world DNS servers are seeing the new server IP address for your domain.
- 10. The forgotten step that you remembered
As you may have read, migrating servers is complicated, but it can get worse if you do not anticipate all the problems that may arise.
Often it is better to leave it to a systems administration professional that has experience and competence to do it.
If you are an experience systems administrator, I am sure you have been through circumstances that I forgot to mention and you had to find a clever solution on demand.
In that case it would be great if you could post a comment to this article and let us know how you did it.
No comments:
Post a Comment