WordPress migration notes, part 2

One problem is that I need to install WP-CLI on the new server, and dealing with it is not easy.

The installation instructions don’t say one way or another, but WP-CLI should not be installed as root. Later, if you go to run it as root, it will bark at you that you’re doing a bad thing. Okay, nice to know.

But we do now have the problem that the user who runs WordPress (well, Apache, which runs the PHP code that is WordPress) is the www-data user. I cannot log in as the www-data user, by design (it is a good design). So, how to run this WP-CLI stuff?

sudo -u www-data wp <command>

Okay, this says to switch to user www-data (sudo = switch user and do) (the -u option specifies which user, in this case www-data), and run the wp binary and any command line options you want it (wp) to do.

Cool, but the user I’m logged in as has no idea of where the WordPress installation is. So now, every freaking command I have to type, sudo -u www-data wp <command>, also needs --path=/var/www/html/wordpress in there too.

This sucks.

There is supposed to be a file, wp-cli.local.yml, that I can put the path into. But that file is in my directory, and the sudo command switches away from that.

This sucks, still.

The www-data user does have a home directory; but, it won’t ever be used because the account runs /usr/sbin/nologin on every access. That is secure, but it doesn’t help me from having to type sudo -u www-data wp --path=/var/www/html/wordpress<command> every freaking time I need to do something.

Also, I am a fan of using the page-up key to search my bash history. That works great when I type a few letters, say gre and hit PgUp to search through my last few grep commands. Do I need to reassign ownership of files I’ve added to /var/www/html/wordpress/ ? chow and PgUp, and in a keystroke or two, I’ve got chown -R www-data:www-data /var/www/html/wordpress/ ready to run. Ditto the Apache2 enable and disable site commands. There are a ton of examples where just a few keystrokes and the PgUp key are great.

But having to type su and hitting PgUp presents me with a wall of noise, to finally find the command at the end that I actually want to repeat.

This sucks.

So, there is a solution. It seems kludgy, but it works, as long as you are willing to put up with its kludginess.

  1. cd to /var/www/html/wordpress/
  2. create a file, wp-cli.local.yml, in the location where WordPress is installed (where you just did the cd to), and inside it, put:
    • path: /var/www/html/wordpress

So, as long as you are already in the “right” place, and you have this file which points to your “right place”, you don’t have to specify the “right place” on the command-line of WP-CLI.

The other option is to be in my home directory, and do everything via bash scripts. I wanted to use the command-line, but I may need to put one more level of indirection in the process to get things to work easily. Like I said: kludge.

However, since the bash script has WPPATH="/var/www/html/wordpress" in it, all that sudo -u www-data nonsense goes away. Sure, I’m running it as some random user from some random location, but (I assume) that the WP-CLI people are just fine with that because if a random hacker gets into an ssh session on my box, I’m done for, anyway. Why not just assume whoever is running these commands is authorized?

This sucks quite a bit less, although it doesn’t make me warm and fuzzy about security.

WordPress migration notes

I have a production WordPress site on Amazon Lightsail that I need to migrate away from. These are notes on how to migrate over only the stuff I want to keep.

Backstory: Amazon Lightsail was very inexpensive, at under $5 per month for hosting on their smallest machine, and it did fine. Two things became problems, however:

  • Bitnami WordPress is super easy to spin up, and everything just works. But upgrading to a newer version of something (say PHP or MySQL or something) is a non-starter. The only way to upgrade is to spin up a new machine and do a migration to a new machine.
  • Amazon recently did a price increase. Now, I can get a Linode machine with double the RAM for only 27% more, and that will include backups.

Okay, so I need to migrate, but over the years, I’ve tried different plugins, and even though many of them were uninstalled, the installation routine left crap in the database. How to migrate to a new server, but leave behind the crap? This will be the topic of this post.

First, I installed WP-CLI, instructions can be found here.

Then, on the new machine, I installed only those Plugins which I know I need.

I took a snapshot backup at this point, simply because it seems prudent.

On the new machine, I logged in with ssh and ran this:

wp --path='/var/www/html/wordpress' db query "SHOW TABLES" --skip-column-names --allow-root

This gives me a list of the tables in the new machine that I want from the old machine.

+-----------------------+
| wp_commentmeta        |
| wp_comments           |
| wp_links              |
| wp_options            |
| wp_postmeta           |
| wp_posts              |
| wp_term_relationships |
| wp_term_taxonomy      |
| wp_termmeta           |
| wp_terms              |
| wp_usermeta           |
| wp_users              |
+-----------------------+

This is a pretty minimal list; the old machine has a list 362 tables long! Matomo was a particularly egregious offender here.

With this information, I can use a script written by Mike Andreasen over on the WP Bullet website to dump the databases on the old machine:

# set WP-CLI flags
WPFLAGS="--allow-root"

# define path to the database dumps without trailing slash
DBSTORE="/tmp"
# get the name of the database
DBNAME=$(wp config get DB_NAME ${WPFLAGS})

# list all of the tables regardless of database prefix
TABLELIST=(wp_posts wp_postmeta)

# create the temporary directory for storing the dumps
mkdir -p ${DBSTORE}/${DBNAME}

# loop through tables and export, log details to /tmp/mysqlexport-<database>.txt
for TABLE in ${TABLELIST[@]}
do
    # export the table
    wp db export ${DBSTORE}/${DBNAME}/${TABLE}.sql --tables=${TABLE} ${WPFLAGS} | tee /dev/stderr
done > /tmp/mysqlexport-${DBNAME}.txt

With this done, I scp the files from the old machine to my local machine. Then I scp them up to the new machine. The next script assumes they are in the sql directory in the wordpress folder.

I tried it, but I should have taken a snapshot, first. 😉

I need to search-and-replace all instances of the old domain name in the MySQL dump files, and put in the new domain name. Technically, once the actual switch happens, the new machine will be found at the old name, so this shouldn’t be necessary. But, the whole reason for migrating to a development machine is to test out this migration process. And the new machine does have a different domain name.

The script to upload the MySQL dumps looks like this:

# define WordPress path
WPPATH="/var/www/html/wordpress"

# loop through all of the 
for DUMP in /var/www/html/wordpress/sql/*.sql;
do
    wp db import ${DUMP} --allow-root --path=${WPPATH}
done

But, until the data is cleaned up, the new WordPress website gets the dreaded white-screen-of-death.

Abandoned OpenSuSE Tumbleweed for Leap 15.6 beta: much better

In a previous post, I said how I made a huge mistake by “upgrading” to a fresh installation of OpenSuSE Tumbleweed, which came with KDE 6 and Wayland. This broke the KDE window tiling, and every interaction I had with KDE reminded me of what a huge mistake I had made. I’ve re-installed a fresh OS install from Leap 15.6 beta, and everything is good, back to the way it was before.

Firefox did bark at me that my profile was newer than previous; I had to start it with firefox --allow-downgrade

Also, I lost all my Firefox multi-account containers I had set. Thankfully, I had a previous containers.json file lying around.

But yes, now, everything is working excellently. Well, I haven’t tried Factorio or YouTube videos yet: but the important stuff is working.

Previously, I’d moved off Leap to Tumbleweed because tesseract-ocr was too old. It looks like in Leap 15.6 beta that it is a pretty new version.

Reddit + Google partnership seems like a bad idea to me

Exclusive: Reddit in AI content licensing deal with Google

The problem is that (if you live in the USA) your and my tax dollars are spent by national security agencies polluting Reddit with content from sock-puppet accounts to promote certain agendas.

This means, that by design, Google will be training its AI on untrustworthy sources.

Nothing about this plan is wise.

I know that Google does plenty of stupid things accidentally, but this seems willfully stupid.

New OpenSuSE install – whoops, that was a mistake (no KDE tiling window manager) – HUGE mistake

OpenSuSE Tumbleweed was acting squirrelly, so I downloaded an ISO and installed the latest OS from scratch. That was a huge mistake. Now, sometimes my machine spontaneously reboots, and other times windows get blocked for keyboard input.

On the good side, getting back to a working production system was never easier: delete the HDMI sound card and sound works again, add the external repos and codecs, and YouTube works again, add tesseract-ocr and The GIMP, and I can do my web work again. Install my Epson printer, and I can print a document for an upcoming event I’m a volunteer for.

On the bad side, that brand-spanking-new install came with Wayland and KDE 6, which is so new that it doesn’t have automatic window tiling. I hate it.

Whining about a problem isn’t the same as proposing a fix, so here’s what I wish I could fix:

When a new window opens (and it is not a dialog box), re-tile everything on that screen so that everything that showed before, still shows, but the new thing too, takes half the screen. I use “focus follows mouse”, so it is infuriating that as I move my mouse toward the newly opened window, the window underneath activates focus and hides the new screen behind it because the window underneath is full screen. I wouldn’t mind so bad if my old keystrokes worked, and I could shove the full-screen to half-screen: but that doesn’t work either. The previous behavior, which is what I want, is that the previous full-screen window would automatically resize to the other half of the screen when a new window opens.

This weekend I went to the Southern California Linux Expo, and had thought someone might be able to guide me to a solution. Nope, the KDE guy was anti-helpful, pointing me to a non- KDE solution. Checking it out, it is not what I want. I just want the old KWin tiling script to work.

Self-will got me a brand-new OS installation that frustrates me. Yay. I should have just lived with the squirrelly behavior until I heard the “all-clear” signal from the OpenSuSE forums.

New Debian install; ssh and sudo changes

Similar to what I wrote in New OpenSuSE Tumbleweed cannot ssh in but this time with Debian. This has to be done from a physical console login on the machine (or if it was a VM, from the hosting company’s console login desktop service). I’m logged in as root.

apt-get install vim

Debian is pretty bare-metal, man. This is probably good from a security and stability point-of-view.

cd /etc/ssh/
cd /etc/ssh/

Find PermitRootLogin and uncomment it, and change it to yes

Find #PubkeyAuthentication yes and uncomment it.

Find #AuthorizedKeysFile     .ssh/authorized_keys .ssh/authorized_keys2 and uncomment it and remove the second file authorized_keys2

Find PasswordAuthentication no and uncomment it and change it to yesnote that this is temporary!

Save and exit the sshd_config file. I’m not sure which service(s) would need to be restarted here, so I issue the reboot now command and watch the machine reboot. Today’s hardware is amazingly fast, compared to what we lived with a decade ago.

Now, from my remote machine, I ssh in as root. I get asked about accepting the private key, and get prompted for the password. Once I get in, I know I’m good to proceed to the next step.

ssh-copy-id root@host.domain

I get asked to put in my password again, and now public key logins are enabled, instead of password-based logins.

I log in as root again, but this time without a password. At this point, I do some customizations per How to make Ubuntu have a nice bash shell like OpenSuSE (although this is Debian). One nice thing is that ~/.bashrc already had aliases ready for ll being an alias for ls -l

Something I don’t understand is why I cannot copy / paste from the Debian ssh session. My guess is that is has something to do with LS_OPTIONS in the bashrc file. Anyway….

I still needed to add alias ..='cd ..' though.

I log out.

I log in as a non-root user, with a password.

ssh-copy-id user@host.domain

I log in as the non-root user, without a password. Same thing: I add the customizations I like, where I can edit with vim, from doing a less on a file, the .. alias for changing directory up one, and using PageUp to search history. I log out.

I log in as root again. Now, I need to give my non-root user sudo rights.

adduser whatever-the-non-root-user-is sudo

Back to editing /etc/ssh/sshd_config

Find PermitRootLogin and uncomment it, and change it to no

Find PasswordAuthentication yes and uncomment it and change it to no

And then I save and exit the file and reboot the box.

Now I can ssh as the non-root user, and I cannot log in via ssh as root. Also, no-one can attempt to log in with just a password. This is good.