I’m a fan of Tenets of IT

Number 15 of which is “Everyone has a test environment, not everyone is lucky enough to have a separate production environment.”

Heh.

This post will be how I copied a production web site to a test environment.

Prerequisites are:

• A virtual machine server
• A domain name
• A wildcard certificate for that domain name

In my case, for the virtual machine server, I bought a used Lenovo Tiny PC from Amazon, loaded it up with RAM and installed Proxmox on it.

I had bought a domain name, really for my Nextcloud instance, but I can also use it for my home lab.

I have a firewall, which can get SSL certificates from the EFF project Let’s Encrypt, via the certbot / acme protocol. I went through the trouble to get a wildcard certificate, so that any box in my domain name can be SSL protected.

The basic steps

1. Prepare the new machine
2. Install WordPress
3. Export WordPress “production” and import to “test”
4. .
5. .
6. .
7. Profit!

Prepare new machine

1. Install Debian
2. Add vim and other configurations
3. Change host name and domain
4. Add ssh key login
5. Install Apache and MariaDB
6. Install WordPress
7. Update Apache enabled sites to include SSL
8. Update Apache default Debian setting
9. Install one WordPress plugin to import the export
10. A note about ASE (Admin Site Enhancements)

Install Debian

This was a Proxmox step, and I think I did it from a .ISO file

Add vim and other configurations

apt-get install vim

update-alternatives --config editor

vim ~/.bash_profile

export EDITOR=vim
[ -r $HOME/.bashrc ] && source $HOME/.bashrc
export PS1='\[\e[32;40m\]\u\[\e[37m\]@\[\e[32m\]\H:\w\[\e[30m\] \[\e[32m\]\$\[\e[30m\] \[\e[0m\]'

vim ~/.bashrc

Find the aliases I want and add them, uncomment them, etc. I always add:

alias ..='cd ..'

vim /etc/inputrc

Find # "\e[5~": history-search-backward and uncomment it

Change host name and domain

I should mention that in my home firewall, it is also my local DNS resolver. So inside there, I have server1.example.com mapped to the IP address Proxmox gave to my new virtual machine (Proxmox got it from my DHCP server).

hostnamectl set-hostname server1.example.com

vim /etc/hosts

In here, I added an entry for 127.0.1.1 which maps the fully qualified host and domain name to the host. So for example, 127.0.1.1 server1.example.com server1

The address 127.0.1.1 is specified because Apache will try to identify the site by name (later). Everything in the 127.x.x.x maps to the local machine, so they all go to the same place. But having it as 127.0.1.1 stops a duplication conflict with 127.0.0.1 for localhost

Add ssh key login

ssh-copy-id root@server1.example.com

Never, in production, would I be commonly logging in as root. But this is a test / play environment, and I find the process cumbersome to set up an alternative user, and then have to be constantly doing a su - (switch user) to root. Since this is in my home lab and not visible on the public Internet, this is not so much a risk. And … before I really try anything screwy, I can take a Proxmox snapshot.

Another thing I run into, is that because I can rip and replace virtual machines easily, I tend to have to delete old entries from the ./ssh/known_hosts file.

ssh-keygen -R "server1"

Install Apache and MariaDB

Essentially, I am following the instructions here at Rose Hosting

One change I make is that immediately after installing MySQL, I run the process to secure the MySQL installation:

mysql_secure_installation

Yes, I set the switch to unix_socket authentication. I have zero need for MySQL to authenticate across a network. This is overkill for a home lab, but since this is how production is going to be set, the machine in test should match it.

Install WordPress

I still follow the instructions at Rose Hosting

Update Apache enabled sites to include SSL

The Rose Hosting instructions don’t disable the 000-default.conf web site

a2dissite 000-default.conf

Now, how to make https:// work? Well, there is already a default-ssl.conf file, so all it really needs is a certificate and key, and for Apache to use SSL. The SSL certificate files mentioned there are /etc/ssl/certs/ssl-cert-snakeoil.pem and /etc/ssl/private/ssl-cert-snakeoil.key

I have exported my wild card certificate and key to my local machine, so now I upload them to those directories, and change the names in the default-ssl.conf file.

Update the ServerName setting in default-ssl.conf to server1.example.com

Update the DocumentRoot setting in default-ssl.conf to /var/www/html/wordpress

Add rewrite rules to the non-SSL site to redirect to the SSL site. In wordpress.conf add:

RewriteEngine On
RewriteCond %{REQUEST_URI} !.well-known/acme-challenge
RewriteRule ^(.*)$ https://%{SERVER_NAME}$1 [R=301,L]
RewriteRule ^ https://%{SERVER_NAME}%{REQUEST_URI} [END,NE,R=permanent]

apachectl -t

If this checks out well, that’s nice, but there is still one more thing to add:

a2enmod ssl

Update Apache default Debian setting

This one threw me for a loop – all my redirects were going to 404 error pages.

It turns out that the default setting on Debian has an Apache configuration file with rewrites not allowed.

vim /etc/apache2/apache2.conf

Find my way to the <Directory /var/www/> section. Change the AllowOveride setting from None to All.

The /var/www directory is of course higher up in the directory structure of what Apache is going to serve up. Because it is higher, the AllowOveride None directive overrides the lower level allow all. Whoops – for WordPress this is no bueno.

Finally, restart (not reload) Apache:

systemctl restart apache2

Install one WordPress plugin to import the export

For this, I am following the instructions by Ferdy Korpershoek on this YouTube video

(The YouTube front page has become politicized trash, but for technical videos, it still has good stuff one can find).

Essentially, I install the All-in-One WP Migration plugin on the production web site, do an export, and then install a fixed version of the All-in-One WP Migration plugin on the test web site.

Ferdy does clean up the new / fresh web site first, by deleting and emptying the default pages and posts. I also deleted the default plugins.

After the import is done, I need to log in again, because the database was replaced, which is where my login credentials are stored. After getting logged in, I need to save Permalinks twice.

A note about ASE (Admin and Site Enhancements)

I was almost at hooray! But, I have a small security enhancement via Admin and Site Enhancements (ASE) which threw a tiny wrench in the monkeyworks. Yes, in production, I’ve hidden the login URL to somewhere other than normal. So after the import, All-in-One WP Migration (100 GB version) provides a link to update the Permalinks, but because of ASE, that URL was not found. No biggie, I simply had to use the URL that was appropriate for the production web site.

Profit!

And now I get to bask in the glory of messing the heck out of the test (not production!) WordPress web site. Fun times. 🙂

I think I’ll take a Proxmox snapshot first.

One problem is that I need to install WP-CLI on the new server, and dealing with it is not easy.

The installation instructions don’t say one way or another, but WP-CLI should not be installed as root. Later, if you go to run it as root, it will bark at you that you’re doing a bad thing. Okay, nice to know.

But we do now have the problem that the user who runs WordPress (well, Apache, which runs the PHP code that is WordPress) is the www-data user. I cannot log in as the www-data user, by design (it is a good design). So, how to run this WP-CLI stuff?

sudo -u www-data wp <command>

Okay, this says to switch to user www-data (sudo = switch user and do) (the -u option specifies which user, in this case www-data), and run the wp binary and any command line options you want it (wp) to do.

Cool, but the user I’m logged in as has no idea of where the WordPress installation is. So now, every freaking command I have to type, sudo -u www-data wp <command>, also needs --path=/var/www/html/wordpress in there too.

This sucks.

There is supposed to be a file, wp-cli.local.yml, that I can put the path into. But that file is in my directory, and the sudo command switches away from that.

This sucks, still.

The www-data user does have a home directory; but, it won’t ever be used because the account runs /usr/sbin/nologin on every access. That is secure, but it doesn’t help me from having to type sudo -u www-data wp --path=/var/www/html/wordpress<command> every freaking time I need to do something.

Also, I am a fan of using the page-up key to search my bash history. That works great when I type a few letters, say gre and hit PgUp to search through my last few grep commands. Do I need to reassign ownership of files I’ve added to /var/www/html/wordpress/ ? chow and PgUp, and in a keystroke or two, I’ve got chown -R www-data:www-data /var/www/html/wordpress/ ready to run. Ditto the Apache2 enable and disable site commands. There are a ton of examples where just a few keystrokes and the PgUp key are great.

But having to type su and hitting PgUp presents me with a wall of noise, to finally find the command at the end that I actually want to repeat.

This sucks.

So, there is a solution. It seems kludgy, but it works, as long as you are willing to put up with its kludginess.

1. cd to /var/www/html/wordpress/
2. create a file, wp-cli.local.yml, in the location where WordPress is installed (where you just did the cd to), and inside it, put:
   • path: /var/www/html/wordpress

So, as long as you are already in the “right” place, and you have this file which points to your “right place”, you don’t have to specify the “right place” on the command-line of WP-CLI.

The other option is to be in my home directory, and do everything via bash scripts. I wanted to use the command-line, but I may need to put one more level of indirection in the process to get things to work easily. Like I said: kludge.

However, since the bash script has WPPATH="/var/www/html/wordpress" in it, all that sudo -u www-data nonsense goes away. Sure, I’m running it as some random user from some random location, but (I assume) that the WP-CLI people are just fine with that because if a random hacker gets into an ssh session on my box, I’m done for, anyway. Why not just assume whoever is running these commands is authorized?

This sucks quite a bit less, although it doesn’t make me warm and fuzzy about security.

I have a production WordPress site on Amazon Lightsail that I need to migrate away from. These are notes on how to migrate over only the stuff I want to keep.

Backstory: Amazon Lightsail was very inexpensive, at under $5 per month for hosting on their smallest machine, and it did fine. Two things became problems, however:

• Bitnami WordPress is super easy to spin up, and everything just works. But upgrading to a newer version of something (say PHP or MySQL or something) is a non-starter. The only way to upgrade is to spin up a new machine and do a migration to a new machine.
• Amazon recently did a price increase. Now, I can get a Linode machine with double the RAM for only $2 more, and that will include backups.

Okay, so I need to migrate, but over the years, I’ve tried different plugins, and even though many of them were uninstalled, the installation routine left crap in the database. How to migrate to a new server, but leave behind the crap? This will be the topic of this post.

First, I installed WP-CLI, instructions can be found here.

Then, on the new machine, I installed only those Plugins which I know I need.

I took a snapshot backup at this point, simply because it seems prudent.

On the new machine, I logged in with ssh and ran this:

wp --path='/var/www/html/wordpress' db query "SHOW TABLES" --skip-column-names --allow-root

This gives me a list of the tables in the new machine that I want from the old machine.

+-----------------------+
| wp_commentmeta        |
| wp_comments           |
| wp_links              |
| wp_options            |
| wp_postmeta           |
| wp_posts              |
| wp_term_relationships |
| wp_term_taxonomy      |
| wp_termmeta           |
| wp_terms              |
| wp_usermeta           |
| wp_users              |
+-----------------------+

This is a pretty minimal list; the old machine has a list 362 tables long! Matomo was a particularly egregious offender here.

With this information, I can use a script written by Mike Andreasen over on the WP Bullet website to dump the databases on the old machine:

# set WP-CLI flags
WPFLAGS="--allow-root"

# define path to the database dumps without trailing slash
DBSTORE="/tmp"
# get the name of the database
DBNAME=$(wp config get DB_NAME ${WPFLAGS})

# list all of the tables regardless of database prefix
TABLELIST=(wp_posts wp_postmeta)

# create the temporary directory for storing the dumps
mkdir -p ${DBSTORE}/${DBNAME}

# loop through tables and export, log details to /tmp/mysqlexport-<database>.txt
for TABLE in ${TABLELIST[@]}
do
    # export the table
    wp db export ${DBSTORE}/${DBNAME}/${TABLE}.sql --tables=${TABLE} ${WPFLAGS} | tee /dev/stderr
done > /tmp/mysqlexport-${DBNAME}.txt

With this done, I scp the files from the old machine to my local machine. Then I scp them up to the new machine. The next script assumes they are in the sql directory in the wordpress folder.

I tried it, but I should have taken a snapshot, first. 😉

I need to search-and-replace all instances of the old domain name in the MySQL dump files, and put in the new domain name. Technically, once the actual switch happens, the new machine will be found at the old name, so this shouldn’t be necessary. But, the whole reason for migrating to a development machine is to test out this migration process. And the new machine does have a different domain name.

The script to upload the MySQL dumps looks like this:

# define WordPress path
WPPATH="/var/www/html/wordpress"

# loop through all of the 
for DUMP in /var/www/html/wordpress/sql/*.sql;
do
    wp db import ${DUMP} --allow-root --path=${WPPATH}
done

But, until the data is cleaned up, the new WordPress website gets the dreaded white-screen-of-death.