Introduction
This page describes how Alexis Huxley configured WordPress services for multiple WordPress sites.
Two big procedures are described on this page:
- Installing WordPress and plugins and optionally migrating content and attachments
- Installing WordPress and migrating plugins, content and attachments (i.e. OS upgrade only)
The second procedure is described near the end of this page.
It was last completed using WordPress 5.2.
The main points regarding the configuration are:
- it supports multiple WordPress sites
- each WordPress site runs in a dedicated vhost on one (of possibly multiple) backend webserver(s)
- the backend webserver(s) is only connected to a local private network
- each WordPress vhost on its backend webserver has a thin reverse-proxy vhost on a single (shared) frontend webserver
- the frontend webserver is connected to the local private network and to the internet
Hopefully, the following diagram makes these points clear:
In addition:
- each WordPress site has its own WordPress installation on its backend webserver (in order to make upgrades granular)
- the frontend vhosts’ names are of the form <public-fqhn>, which you must somehow acquire (e.g. by purchase or using a free dynamic DNS service such as freemyip.com)
- the backend’s vhost name is of the form <private-uqhn>-<dedotted-public-fqhn>.<private-domain-name> (e.g. the backend webserver hosting this website is called rotelle.pasta.net and the backend vhost running on it is called rotelle-wwwpastafreemyipcom.pasta.net):
Collect information
- What is the fully-qualified hostname of the frontend vhost?
- What is the fully-qualified hostname of the backend host (not vhost)?
- What will be the password for the WordPress database (stored in MariaDB)?
- If the WordPress site is being migrated then:
- Are attachments organised in a hierarchy?
Environment variables
- This procedure references several site-specific values. To allow the procedure to be as generic as possible we read site-specific values from environment variables. Therefore run:
FRONTENDVHOST=www.pasta.freemyip.com # or whatever BACKENDHOST=vermicelli.pasta.net # or whatever DATABASEPASSWD=XXXXXXXXXX # or whatever # Derive some other stuff BACKENDVHOST=${BACKENDHOST%%.*}-${FRONTENDVHOST//./}.${BACKENDHOST#*.} DATABASENAME=${FRONTENDVHOST//./} DATABASEUSER=${FRONTENDVHOST//./}
- This procedure expects that these variables are available on any machine at any time, so be sure to set them in each shell you start.
Exporting a site
This section applies only if migrating a WordPress site between identical WordPress versions on different servers.
To be run on the old backend webserver as root unless stated otherwise.
- Backup the WordPress root directory by running:
cd /var/www/$FRONTENDVHOST tar czf ~/$FRONTENDVHOST.tar.gz .
- Back up the database by running:
mysqldump $DATABASENAME > ~/$DATABASENAME.sql
DNS changes
- Ensure that the frontend webserver can resolve $BACKENDHOST and ${BACKENDHOST%%.*} (in my case, that means I need to add A and PTR records to my local DNS).
- Ensure that the frontend webserver can resolve $BACKENDVHOST (in my case, that means I need to add an additional A record to my local DNS, but perhaps a CNAME record would have been neater).
- In order to allow hosts on the local private network to access the frontend vhost using the same URL that hosts on the internet would use, without encountering router-redirects-internal-traffic-to-external-interface-and-then-gets-confused-like issues, extra measures may need to be taken (in my case, I add suitable entries to my local DNS server’s RPZ domain).
- Ensure that $FRONTENDVHOST is resolvable from the internet (in my case, I go to freemyip.com, register a new name and set up inadyn).
Configure empty https-only backend vhost
To be run on the backend webserver as root unless stated otherwise.
- If not already done, run:
apt-get -y install apache2
- If not already done, run:
a2enmod ssl a2dissite 000-default
- The backend webserver will only be accessed by the frontend webserver and the frontend webserver will know to use only https to reach the backend webserver. So, if not already done, edit /etc/apache2/ports.conf and comment out the line:
Listen 80
- Create the vhost config file <private-uqhn>-<dedotted-public-wordpress-fqhn>.<private-domain-name> by running something like:
cp /etc/apache2/sites-available/default-ssl.conf \ /etc/apache2/sites-available/$BACKENDVHOST-ssl.conf
- Edit the new file and:
- remove comments and blank lines
- remove the ‘if mod_ssl loaded’ wrapper
- normalise indentation
- change the following lines:
<VirtualHost *:443> ServerName BACKENDVHOST DocumentRoot /var/www/FRONTENDVHOST ErrorLog ${APACHE_LOG_DIR}/BACKENDVHOST-ssl-error.log CustomLog ${APACHE_LOG_DIR}/BACKENDVHOST-ssl-access.log combined
and then substitute these placeholders for their values by running:
sed -i -e "s/BACKENDVHOST/$BACKENDVHOST/g" \ -e "s/FRONTENDVHOST/$FRONTENDVHOST/g" \ /etc/apache2/sites-available/$BACKENDVHOST-ssl.conf
- Check the configuration file looks similar to this:
<VirtualHost *:443> ServerName rotelle-wwwpastafreemyipcom.pasta.net DocumentRoot /var/www/www.pasta.freemyip.com ErrorLog ${APACHE_LOG_DIR}/rotelle-wwwpastafreemyipcom.pasta.net-ssl-error.log CustomLog ${APACHE_LOG_DIR}/rotelle-wwwpastafreemyipcom.pasta.net-ssl-access.log combined LogLevel warn SSLEngine on SSLCertificateFile /etc/ssl/certs/ssl-cert-snakeoil.pem SSLCertificateKeyFile /etc/ssl/private/ssl-cert-snakeoil.key <FilesMatch "\.(cgi|shtml|phtml|php)$"> SSLOptions +StdEnvVars </FilesMatch> <Directory /usr/lib/cgi-bin> SSLOptions +StdEnvVars </Directory> </VirtualHost>
- Create the document root, regenerate the snake-oil certificate (only needed if OS installation is a clone but does no harm otherwise), enable the site and restart Apache by running:
mkdir -p /var/www/$FRONTENDVHOST chown -R www-data:www-data /var/www/$FRONTENDVHOST make-ssl-cert generate-default-snakeoil --force-overwrite a2ensite $BACKENDVHOST-ssl systemctl restart apache2
- Make some basic checks:
echo hello > /var/www/$FRONTENDVHOST/index.html yes y | w3m -dump https://$BACKENDVHOST/ nmap localhost rm -f /var/www/$FRONTENDVHOST/index.html
- Note that we never edit stuff using the /etc/apache2/sites-enabled/ path, as doing so can convert symlinks into files and cause divergence from the thing the symlink had pointed to.
Configure thin frontend vhost
To be run on the frontend webserver as root unless stated otherwise.
- Create a simple vhost to redirect http to https by creating file /etc/apache2/sites-available/$FRONTENDVHOST.conf containing:
<VirtualHost *:80> ServerName lsa.freemyip.com CustomLog /var/log/apache2/FRONTENDVHOST/FRONTENDVHOST-access.log combined2 ErrorLog /var/log/apache2/FRONTENDVHOST/FRONTENDVHOST-error.log RedirectMatch permanent /(.*) https://FRONTENDVHOST/$1 LogLevel warn ServerSignature Off </VirtualHost>
and then make the necessary substitutions and enable the site:
sed -i -e "s/FRONTENDVHOST/$FRONTENDVHOST/g" \ /etc/apache2/sites-available/$FRONTENDVHOST.conf a2ensite $FRONTENDVHOST
- For a totally new https vhost:
- Create /etc/apache2/sites-available/$FRONTENDVHOST-ssl.conf containing:
<VirtualHost *:443> ServerName FRONTENDVHOST CustomLog /var/log/apache2/FRONTENDVHOST/FRONTENDVHOST-access.log combined2 ErrorLog /var/log/apache2/FRONTENDVHOST/FRONTENDVHOST-error.log ServerSignature Off SSLCertificateFile /etc/ssl/certs/ssl-cert-snakeoil.pem SSLCertificateKeyFile /etc/ssl/private/ssl-cert-snakeoil.key # Keep this commented out until we have a LetsEncrypt certificate. #Include /etc/letsencrypt/options-ssl-apache.conf #SSLCertificateFile /etc/letsencrypt/live/FRONTENDVHOST/fullchain.pem #SSLCertificateKeyFile /etc/letsencrypt/live/FRONTENDVHOST/privkey.pem SSLProxyEngine on SSLProxyVerify none SSLProxyCheckPeerCN off SSLProxyCheckPeerName off SSLProxyCheckPeerExpire off ProxyPass / https://BACKENDVHOST/ ProxyPassReverse / https://BACKENDVHOST/ </VirtualHost>
and then make the necessary substitutions and enable the site:
sed -i -e "s/FRONTENDVHOST/$FRONTENDVHOST/g" \ -e "s/BACKENDVHOST/$BACKENDVHOST/g" \ /etc/apache2/sites-available/$FRONTENDVHOST-ssl.conf a2ensite $FRONTENDVHOST-ssl mkdir -p /var/log/apache2/$FRONTENDVHOST systemctl reload apache2
- From the internet, go to the https://$FRONTENDVHOST; the browser should warn of an invalid SSL certificate (and then give a ‘Forbidden’ error as there is no content, etc).
- Get a valid certificate by running:
certbot
and following the prompts.
- Depending how much you let certbot do, yo may need to edit /etc/apache2/sites-available/$FRONTENDVHOST-ssl.conf and:
- comment out the snakeoil stuff
- uncomment the letsencrypt stuff in
- run:
systemctl reload apache2
- From the internet, restart the browser (Chromium does not re-get the certificate, I think, so it won’t detect if the certificate is now valid) and go to https://$FRONTENDVHOST; the browser should report a valid SSL certificate.
- Create /etc/apache2/sites-available/$FRONTENDVHOST-ssl.conf containing:
- For a https vhost that is being migrated, when the public should continue to be proxied to the old backend vhost but developers should be proxied to the new backend vhost:
- Verify that /etc/apache2/sites-available/$FRONTENDVHOST-ssl.conf contains something like this:
ProxyPass / https://rotelle-wwwpastafreemyipcom.pasta.net/ ProxyPassReverse / https://rotelle-wwwpastafreemyipcom.pasta.net/ RewriteEngine On RewriteRule ^/$ https://rotelle-wwwpastafreemyipcom.pasta.net/index.php [P]
- Change it to something like this:
RewriteEngine On RewriteCond %{REMOTE_ADDR} !^130\.183\.2\.35 RewriteRule ^/(.+) https://rotelle-wwwpastafreemyipcom.pasta.net/$1 [P] RewriteCond %{REMOTE_ADDR} !^130\.183\.2\.35 RewriteRule ^/$ https://rotelle-wwwpastafreemyipcom.pasta.net/index.php [P] ProxyPassReverse / https://rotelle-wwwpastafreemyipcom.pasta.net/ RewriteCond %{REMOTE_ADDR} ^130\.183\.2\.35 RewriteRule ^/(.+) https://vermicelli-wwwpastafreemyipcom.pasta.net/$1 [P] RewriteCond %{REMOTE_ADDR} ^130\.183\.2\.35 RewriteRule ^/$ https://vermicelli-wwwpastafreemyipcom.pasta.net/index.php [P] ProxyPassReverse / https://vermicelli-wwwpastafreemyipcom.pasta.net/
(This directs source IP 130.183.2.35 to the new server rotelle, while everybody else goes to the old one. Note that this means that the new wordpress site can download attachments from the old wordpress site.)
- Run:
systemctl reload apache2
- Verify that the old site is accessible except from the designated IP address, which sees the new site.
- Verify that /etc/apache2/sites-available/$FRONTENDVHOST-ssl.conf contains something like this:
- To support client IP logging on the backend webserver do the following on the backend webserver:
- Edit /etc/apache2/apache2.conf and, in all LogFormat lines, change ‘%h’ for ‘%a’.
- To ensure that even hosts on the local network have their IPs logged instead of the frontend webserver’s edit /etc/apache2/conf-available/remoteip.conf and add:
RemoteIPHeader X-Forwarded-For RemoteIPInternalProxy <ip-address-of-frontend-webserver>
- Run:
a2enmod remoteip a2enconf remoteip systemctl reload apache2
“Famous 5-Minute Installation”
To be run on the backend webserver as root unless stated otherwise.
- If not already done, run:
apt-get -y install mariadb-server libapache2-mod-php7.3 php7.3-mysql php-apcu \ php-imagick php7.3-curl php7.3-gd php7.3-intl php7.3-mbstring php7.3-xml \ php7.3-xmlrpc php7.3-zip
- If not already done, allow sites to be imported by editing /etc/php/7.3/apache2/php.ini and setting:
upload_max_filesize = 101M post_max_size = 102M
(The sizes are deliberately slightly different in order to determine which, if any, limit is hit.)
- On the backend webserver, download and unpack the WordPress software by running:
cd rm -f latest.tar.gz wget https://wordpress.org/latest.tar.gz tar xzfC ~/latest.tar.gz /var/www/$FRONTENDVHOST/ --strip-components=1 chown -R www-data:www-data /var/www/$FRONTENDVHOST/ find /var/www/$FRONTENDVHOST/ -type f -exec chmod 640 {} \; find /var/www/$FRONTENDVHOST/ -type d -exec chmod 750 {} \;
- Use the sample config file as the config file (we’ll edit it shortly) by running:
mv /var/www/$FRONTENDVHOST/wp-config-sample.php \ /var/www/$FRONTENDVHOST/wp-config.php
- Create a database for WordPress to store its content and settings:
{ echo "CREATE DATABASE $DATABASENAME;" echo "CREATE USER \"$DATABASEUSER\"@\"localhost\" IDENTIFIED BY \"$DATABASEPASSWD\";" echo "GRANT ALL PRIVILEGES ON $DATABASENAME.* TO \"$DATABASEUSER\"@\"localhost\";" echo "FLUSH PRIVILEGES;" } | mariadb
- Let the WordPress software know how to connect to the database by editing /var/www/$FRONTENDVHOST/wp-config.php and setting:
define( 'DB_NAME', 'DATABASENAME' ); define( 'DB_USER', 'DATABASEUSER' ); define( 'DB_PASSWORD', 'DATABASEPASSWD' ); define( 'DB_HOST', 'localhost' );
and then making the substitutions:
sed -i -e "s/DATABASENAME/$DATABASENAME/g" \ -e "s/DATABASEUSER/$DATABASEUSER/g" \ -e "s/DATABASEPASSWD/$DATABASEPASSWD/g" \ /var/www/$FRONTENDVHOST/wp-config.php
- Various other passwords are used internally within WordPress. Go to https://api.wordpress.org/secret-key/1.1/salt/ and copy the content into wp-config.php:
define('AUTH_KEY', 'XXXXXXXXXXXXXXXXXXXXXXX'); define('SECURE_AUTH_KEY', 'XXXXXXXXXXXXXXXXXXXXXXX'); define('LOGGED_IN_KEY', 'XXXXXXXXXXXXXXXXXXXXXXX'); define('NONCE_KEY', 'XXXXXXXXXXXXXXXXXXXXXXX); define('AUTH_SALT', 'XXXXXXXXXXXXXXXXXXXXXXX'); define('SECURE_AUTH_SALT', 'XXXXXXXXXXXXXXXXXXXXXXX'); define('LOGGED_IN_SALT', 'XXXXXXXXXXXXXXXXXXXXXXX'); define('NONCE_SALT', 'XXXXXXXXXXXXXXXXXXXXXXX');
- At this point we should visit https://$FRONTENDVHOST/ to complete the installation. But there is a problem: the WordPress software – running on the backend, even if proxied – thinks that the hostname is $BACKENDHOST, which means it will generate HTML containing links to the backend, which will not be accessible directly. We can verify this by going to https://$FRONTENDVHOST/, noting the absence of CSS-styling, pressing CTRL-U to view the page source, noting lines like this:
link rel='stylesheet' ... href='https://vesuvio-lsafreemyipcom.pasta.net/...
The cleanest solution is to edit wp-config.php and to temporarily set:
define( 'WP_HOME', 'https://FRONTENDVHOST' ); define( 'WP_SITEURL', 'https://FRONTENDVHOST' );
and then run:
sed -i -e "s/FRONTENDVHOST/$FRONTENDVHOST/g" \ /var/www/$FRONTENDVHOST/wp-config.php
- Log in to the frontend webserver as root.
- Note that I did not do this on lsa, but did on all others! I should try this in coordination with Louise! As a workaround for a bug (see here, here and here) edit /etc/apache2/sites-available/$FRONTENDVHOST-ssl.conf and add something like the following line before the ProxyPass directive:
ProxyPassMatch ^/$ https://rotelle-wwwpastafreemyipcom.pasta.net/index.php ProxyPass / https://rotelle-wwwpastafreemyipcom.pasta.net/ ProxyPassReverse / https://rotelle-wwwpastafreemyipcom.pasta.net/
or, if using the mod_rewrite approach, then add something like the following stanza before any other rewrite stanzas (which probably I can put it right at the end of the file):
ProxyPass / https://rotelle-wwwpastafreemyipcom.pasta.net/ ProxyPassReverse / https://rotelle-wwwpastafreemyipcom.pasta.net/ RewriteEngine On # Next line commented out but left here to remind myself how to apply restriction. #RewriteCond %{REMOTE_ADDR} ^130\.183\.2\.35\ RewriteRule ^/$ https://rotelle-wwwpastafreemyipcom.pasta.net/index.php [P]
and reload the apache configuration.
- Log in to the backend webserver as root.
- To allow symlinks to work edit /etc/apache2/sites-available/$BACKENDVHOST-ssl.conf and add:
<Directory /var/www/FRONTENDVHOST/> Options FollowSymLinks AllowOverride All </Directory>
and then run:
sed -i "s/FRONTENDVHOST/$FRONTENDVHOST/g" /etc/apache2/sites-available/$BACKENDVHOST-ssl.conf service apache2 restart
Hardening WordPress
This section has many small steps to be run in different places; see the text for more details.
- Login to the backend webserver as root.
- To restrict access to some sensitive bits of the WordPress installation itself:
- Add the following to /etc/apache2/sites-available/$BACKENDVHOST-ssl.conf:
<Files wp-config.php> order allow,deny deny from all </Files> <DirectoryMatch ".*/uploads/"> <Files ~ "\.ph(?:p[345]?|t|tml)$"> deny from all </Files> </DirectoryMatch> <Location /wp-login.php> Require ip 130.183.2.35 192.168.1.0/24 </Location> <Location /wp-admin/> # Use this block for digest authentication #Require valid-user #AuthType Digest #AuthName "BACKENDVHOST-wp-admin" #AuthUserFile /etc/apache2/BACKENDVHOST-wp-admin.htdigest # Use this block for LDAP authentication - edit as required! <RequireAll> Require ip 130.183.2.35 192.168.1.0/24 Require ldap-user alexis suzie </RequireAll> AuthType Basic AuthName "BACKENDVHOST-wp-admin" AuthBasicProvider ldap AuthLDAPUrl ldap://ziti.pasta.net/ou=Users,dc=pasta,dc=net?uid </Location> RewriteEngine On <Directory /var/www/FRONTENDVHOST/> RewriteRule ^wp-admin/includes/ - [F,L] RewriteRule !^wp-includes/ - [S=3] RewriteRule ^wp-includes/[^/]+\.php$ - [F,L] RewriteRule ^wp-includes/js/tinymce/langs/.+\.php - [F,L] RewriteRule ^wp-includes/theme-compat/ - [F,L] </Directory>
and run:
sed -i -e "s/FRONTENDVHOST/$FRONTENDVHOST/g" \ -e "s/BACKENDVHOST/$BACKENDVHOST/g" \ /etc/apache2/sites-available/$BACKENDVHOST-ssl.conf
- Edit the authentication section as required. You might need to enable additional modules or create users.
a2enmod auth_digest rewrite htdigest -c /etc/apache2/$BACKENDVHOST-wp-admin.htdigest $BACKENDVHOST-wp-admin alexis ...
- Restart apache with:
service apache2 restart
- Add the following to /etc/apache2/sites-available/$BACKENDVHOST-ssl.conf:
- Optionally, set up filesystem monitoring as follows:
- Install fad.
- Run:
find /var/www/ | fadscan -i var_www -
- Add the following cronjob:
0 2 * * * find /var/www/ | fadscan -c -m root var_www -
- Strip some information from the generated HTML and from HTTP responses by editing /var/www/$FRONTENDVHOST/wp-config.php and adding at the bottom:
function crunchify_remove_version() { return ''; } add_filter('the_generator', 'crunchify_remove_version'); remove_action('template_redirect', 'rest_output_link_header', 11, 0); remove_action('wp_head', 'rest_output_link_wp_head', 10); remove_action('wp_head', 'rsd_link'); remove_action('wp_head', 'wlwmanifest_link'); remove_action('wp_head', 'wp_oembed_add_discovery_links', 10);
(This was taken from here.)
- The ‘Link: ‘ HTTP header, which adds a header like to this to every HTTP response:
Link: <https://www.pasta.freemyip.com/?p=68>; rel=shortlink
can be removed as follows:
- Run:
cd /var/www/$FRONTENDVHOST/wp-content/plugins mkdir remove-shortlink touch remove-shortlink/remove-shortlink.php chown -R www-data:www-data remove-shortlink/remove-shortlink.php
- Edit remove-shortlink/remove-shortlink.php and add:
<?php /* Plugin Name: Remove Shortlink Description: Removes the shortlink from the HTTP response. Version: 1.0 Author: Alexis Huxley (based on work by ECAddOns) */ function remove_shortlink() { remove_action('wp_head', 'wp_shortlink_wp_head', 10); remove_action('template_redirect', 'wp_shortlink_header', 11); } add_filter('after_setup_theme', 'remove_shortlink');
(This was based upon work by ECAddOns.)
- Navigate to Dashboard –> Plugins.
- Activate the Remove Shortlink plugin.
- Test with something like this:
wget -S https://www.pasta.freemyip.com/recipes/
There should be no header like this:
Link: <https://www.pasta.freemyip.com/?p=332>; rel=shortlink
- Note that other solutions on the web suggest editing functions.php but that file gets overwritten at upgrade time.
- Run:
- Disable feed pages by editing /var/www/$FRONTENDVHOST/wp-config.php and adding at the bottom:
function disable_feeds() { wp_redirect( home_url() ); die; } // Disable global RSS, RDF & Atom feeds. add_action( 'do_feed', 'disable_feeds', -1 ); add_action( 'do_feed_rdf', 'disable_feeds', -1 ); add_action( 'do_feed_rss', 'disable_feeds', -1 ); add_action( 'do_feed_rss2', 'disable_feeds', -1 ); add_action( 'do_feed_atom', 'disable_feeds', -1 ); // Disable comment feeds. add_action( 'do_feed_rss2_comments', 'disable_feeds', -1 ); add_action( 'do_feed_atom_comments', 'disable_feeds', -1 ); // Prevent feed links from being inserted in the <head> of the page. add_action( 'feed_links_show_posts_feed', '__return_false', -1 ); add_action( 'feed_links_show_comments_feed', '__return_false', -1 ); remove_action( 'wp_head', 'feed_links', 2 ); remove_action( 'wp_head', 'feed_links_extra', 3 );
(This was taken from here.)
- Strip XML-RPC responses by editing wp-config.php:
// Disable all xml-rpc endpoints add_filter('xmlrpc_methods', function () { return []; }, PHP_INT_MAX);
To test that this works try running:
cat > x <<'EOF' <?xml version="1.0" encoding="utf-8"?> <methodCall> <methodName>system.listMethods</methodName> <params></params> </methodCall> EOF wget --no-check-certificate -S -qO - --post-file=x https://<your-website>/xmlrpc.php
before adding the snippet above and after adding it. (This was taken from here.) The
--no-check-certificate
option is only needed if the host does not yet have a proper SSL certificate (one host didn’t so I needed it) but it causes no harm in other cases.It’s difficult to tell if that last step worked because the logs remain full of entries like this:
... "POST //xmlrpc.php HTTP/1.1" 200 691 "-" ...
so add the following /etc/sites-available/<site>-ssl.conf: Redirect 410 /xmlrpc.php
and reload. (Note that it will not work if it is in the front-end site file or the front-end apache2.conf file.)
Very very simple migration
This section only applies when moving a site from one server to another where both the old and new server have the same WordPress version.
- Transfer the database backup and the wordpress directory backup from the old WordPress server to the new WordPress server.
- Log in to the backend webserver.
- Restore the database by running:
mariadb $DATABASENAME < $DATABASENAME.sql
- Restore the WordPress directory backup by running:
cd /var/www mv $FRONTENDVHOST $FRONTENDVHOST.delete-soon mkdir $FRONTENDVHOST cd $FRONTENDVHOST tar xzf ~/$FRONTENDVHOST.tar.gz
- Reboot.
- Log in to the frontend webserver.
- Adjust /etc/apache2/sites-available/$FRONTENDVHOST-ssl.conf so that all requests are sent to the new backend webserver.
- Run:
systemctl restart apache2
- Log in to the old backend webserver.
- Run:
a2dissite
and disable the old site.
- Run:
systemctl reload apache2
- Test access to the website.
- Skip all remaining sections of this page!
Very very basic wordpress configuration
This section has many small steps to be run in different places; see the text for more details.
- Visit https://$FRONTENDVHOST/
- If you see ‘hello’ then it is because your ‘rm -f’ command earlier failed (presumably because some other command in the same stanza is slurping stdin).
- Check that you are prompted for a language.
- Choose a language.
- Fill in the basic information requested (your login, email, password, site name, etc).
- Log in.
- Note that the WordPress URLs under Settings->General will be greyed out as a result of adding the last two definitions to wp-config.php.
- There may already be updates available; if so apply them.
- Remove all sample posts but retain the sample pages, which are needed shortly.
- Go to Settings–>Permalinks and choose the ‘postname’ option.
- Go to Settings–>Reading and choose the static front page.
- Optionally edit the front page now and add some text explaining that this it the front page.
- Go to Settings–>Reading and choose the static front page.
- Go to Settings–>Media and untick “Organise my uploads into month- and year-based folders” and set all thumbnail sizes to 0x0.
- Disable use of pingomatic by going to Settings–>Writing and removing all (there is only one) entries from the ‘Update Services’ box.
WordPress plugins
To be run logged in to WordPress unless stated otherwise.
- Remove the following plugins:
- Akismet
- Hello Dolly
- Install the following plugins:
- Disable Comments (by WPDeveloper)
- Title Remover (by WPGurus) [was not available when configuring suzanneramsay.freemyip.com so I copied it from www.pasta.freemyip.com]
- Justified Gallery (by Matheusz Czardybon)
- File Upload Types (by WPForms)
- Media Library Folders for WordPress (by Max Foundry)
- OSM (by MiKa)
- PhotoSwipe Light (by Louy Alakkad)
- Smart Attachment Page Remove (by Peter Raschendorfer)
- TablePress (by Tobias Bäthge)
- Velvet Blues Update URLs (by VelvetBlues.com)
- WordPress Importer (by wordpressdotorg)
- DNUI (by Nicearma)
- Broken Links Checker (WPMP DEV)
- Classic Editor (by WordPress Contributors)
- Enable Media Replace (by ShortPixel)
- Login LockDown (by Michael VanDeMar)
- Add Anchor Links (by Karolína Vyskočilová)
- Highlighting Code Block (by LOOS, Inc.)
- JSM’s file_get_contents() Shortcode (by JM Morisset)
- Advanced Editor Tools (previously TinyMCE Advanced) (by Automattic)
- Snitch (by pluginkollectiv)
- WP Control (by John Blackbourn and contributors)
- Disable WP Sitemaps (by Jeff Starr)
- Activate all plugins.
- Configure the following plugins immediately:
- block comments everywhere
- allow the following uploads (Settings–>Media):
- xls application/vnd.ms-excel
- gpx application/gpx+xml
- tar.gz application/tar+gzip
- zip application/zip
- diff text/x-diff
- iso application/iso-image
- smart Attachment Page Remove –> 410
- Justified Gallery –> Skip
- Log in to the backend webserver.
- Edit the file /var/www/$FRONTENDVHOST/wp-content/plugins/broken-link-checker/modules/checkers/http.php and replace this:
$ua = '... Safari/537.36';
with this:
$ua = '... Safari/537.36 (broken links checker plugin)';
(This will allow scripts to exclude Apache log entries caused by the Broken Links checker plugin.) Bear in mind that this patch will need reapplying in the event that the plugin is upgraded.
WordPress themes
To be run logged in to WordPress unless stated otherwise.
- Install the following themes:
- Reddle (from my archive)
- BlogFeedly
- GeneratePress
- Maxwell
- WP Less Is More
- Zakra
- pho by Thematosoup (on hsrs2019.freemyip.com only)
- Activate one of the installed themes in order to be able to remove the currently active theme:
- on www.pasta.freemyip.com and suzanneramsay.freemyip.com activate GeneratePress
- on judithhabgood.freemyip.com activate Reddle (will be changed later)
- on hsrs2019.freemyip.com activate Pho
- Remove the following themes:
- Twenty Nineteen
- Twenty Seventeen
- Twenty Sixteen
- Check for updates and apply any that are available. (I had a translation update for the Reddle theme.)
WordPress Cron
- Ensure that
DISABLE_WP_CRON
is not set in wp-config.php. - From the control panel, locate the ‘Site Health’ widget and if there is a link there then follow it to resolve any issues (typically superfluous themes).
Removing temporary settings from config.php
- Earlier we made temporary settings of WP_HOME and WP_SITEURL in wp-config.php. These can now be removed.
Preparing to migrate WordPress content
This section and some of those that follow and are also related to migrating WordPress content are only needed if you are migrating content from an old WordPress installation to a new one; it should be clear from the section’s title whether or not you need to do it.
To be run on the backend webserver as root unless stated otherwise.
- In order to better debug potention content migration issues edit /var/www/$FRONTENDVHOST/wp-config.php and add:
define ( 'IMPORT_DEBUG', true );
- In order to avoid error messages of the format
Failed to import Media “.......”
edit /var/www/$FRONTENDVHOST/wp-config.php and at the very bottom add:
add_filter( 'http_request_host_is_external', '__return_true' ); add_filter( 'https_local_ssl_verify', '__return_false' ); add_filter( 'https_ssl_verify', '__return_false' );
- In order to avoid error messages of the format
Failed to import Media .... Remote file is incorrect size
tedit /var/www/$FRONTENDVHOST/wp-config.php and at the very bottom add:
add_filter( 'http_request_args', function( $r, $url ) { $r['headers']['Accept-Encoding'] = 'identity'; return $r; }, 10, 2 );
(This is taken from here.)
- If there is a lot of media content then the migration can timeout. To avoid this edit /etc/php/7.0/apache2/php.ini and set:
max_execution_time = 3600
(This is taken from here) and on the front-end webserver edit /etc/apache2/sites-available/$FRONTENDVHOST-ssl.conf and set:
ProxyTimeout 3600
and restart Apache on both webservers.
- Log in to the new backend webserver.
- Make a test download (using wget or curl) of some media in the old WordPress installation using the URL that the HTML uses to reference the media; if this fails then media migration will fail, so it is essential to get this working.
- Make a backup or VM snapshot of the new backend webserver; it is highly likely that the migration or one of the post-migration cleanup steps will fail (and need tweaking) and that you will want to rollback to this point.
Migrating WordPress content
This section has many small steps to be run in different places; see the text for more details.
- On the old WordPress site, export the site to an XML file. Note that this does not export media, but this is okay.
- On the new WordPress site, import the XML file, making sure to tell the importer to download and import file attachments.
- Watch the front-end webserver’s web logs to track progress and errors.
- Note that when I did this for www.pasta.freemyip.com I had four obvious errors:
- a few attachments were referenced in the old WordPress site’s HTML but were not available on the old WordPress site to download, resulting in errors:
Failed to import Media “jochberg”: Remote server returned error response 404 Not Found Failed to import Media “kranzberg”: Remote server returned error response 404 Not Found Failed to import Media “nntpcache_3.0.1-4.diff”: Remote server returned error response 404 Not Found Failed to import Media “nntpcache_3.0.1-4”: Sorry, this file type is not permitted for security reasons. Failed to import Media “bodensee-waypoints-camps”: Remote server returned error response 404 Not Found Failed to import Media “donau-track-20110521TO20110621”: Remote server returned error response 404 Not Found Failed to import Media “donau-waypoints-camps”: Remote server returned error response 404 Not Found Failed to import Media “fuenffluesse-track-20120912TO20120916”: Remote server returned error response 404 Not Found Failed to import Media “fuenffluesse-waypoints-camps”: Remote server returned error response 404 Not Found Failed to import Media “mecklenburgischerseen-track-20090815TO20090904”: Remote server returned error response 404 Not Found Failed to import Media “ochsenweg-waypoints-camps”: Remote server returned error response 404 Not Found Failed to import Media “ochsenweg-track-20100824TO20100904”: Remote server returned error response 404 Not Found Failed to import Media “romantischestrasse2010-route-deviations”: Remote server returned error response 404 Not Found Failed to import Media “romantischestrasse2010-waypoints-camps”: Remote server returned error response 404 Not Found Failed to import Media “romantischestrasse2010-track-20100430TO20100508”: Remote server returned error response 404 Not Found Failed to import Media “result”: Remote server returned error response 404 Not Found
- a few attachments were successfully downloaded from the old WordPress site but could not be uploaded into the new WordPress site due to being of types that were not allowed to be uploaded, resulting in errors:
Failed to import Media “my-network-overview”: Sorry, this file type is not permitted for security reasons. Failed to import Media “virtio-win-0.1.100”: Sorry, this file type is not permitted for security reasons. Failed to import Media “virtio-win-0.1.100_x86”: Sorry, this file type is not permitted for security reasons. Failed to import Media “OSM-Europe-2016-04.zip”: Sorry, this file type is not permitted for security reasons.
- a few attachments were successfully downloaded from the old WordPress site but could not be uploaded into the new WordPress site because an upload with the same name already existed, resulting in errors:
Media “MINOLTA DIGITAL CAMERA” already exists. Media “MINOLTA DIGITAL CAMERA” already exists. Media “MINOLTA DIGITAL CAMERA” already exists. Media “MINOLTA DIGITAL CAMERA” already exists. Media “MINOLTA DIGITAL CAMERA” already exists. Media “MINOLTA DIGITAL CAMERA” already exists. Media “MINOLTA DIGITAL CAMERA” already exists.
- more generally, attachments in the old WordPress site were organised into directories using a (now unsupported) WordPress file manager plugin whereas the files were all uploaded into a single directory (hence the error described in the previous point).
These issues will be addressed in the next section.
- a few attachments were referenced in the old WordPress site’s HTML but were not available on the old WordPress site to download, resulting in errors:
- Note that when I did this for suzanneramsay.freemyip.com I had four obvious errors:
Failed to import Media “MINOLTA DIGITAL CAMERA” Failed to import Media “cropped-tels-banner-2.jpg” Failed to import Media “spectrum4” Failed to import Media “suzie2” Failed to import Media “suzie1” Failed to import Media “compare-results” Failed to import Media “suzuki-alto” Notice: Undefined offset: 0 in /var/www/suzanneramsay.freemyip.com/wp-content/plugins/photo-swipe/photo-swipe.php on line 176
These issues will be addressed in the next section.
- Note that when I did this for judithhabgood.freemyip.com I had no errors.
- Note that whenI did hsrs2019.freemyip.com I had no errors.
- Revert the changes (to wp-config.php, etc) made in the previous section.
- If necessary:
- change the default static page
- set (or erase) the site title
- set (or erase) the site tagline
- If the URL is changing and that includes from http to https then use Tools->Update URLs to change all occurences of the old site URL to the new one (select each possible context except GUIDs).
- Delete the Sample page.
- Delete or change the Privacy Policy page.
Fixing media import problems – part 0: installing wp-cli
To be run on the backend webserver as root unless stated otherwise.
- If wp-cli is not already install, then run:
curl -O https://raw.githubusercontent.com/wp-cli/builds/gh-pages/phar/wp-cli.phar chmod +x wp-cli.phar mv wp-cli.phar /usr/local/bin/wp
Fixing media import problems – part 1: restoring the attachment hierarchy
If your old WordPress site did not have attachments organised into a directory hierarchy then skip this section. I did this for www.pasta.freemyip.com and judithhabgood.freemyip.com but not for suzanneramsay.freemyip.com or hsrs2019.freemyip.com (because there were duplicately named images in different directories found in the first steps below, at which point, given that there is only one image used in the site, to scrap the library and re-upload that one image).
This section has many small steps to be run in different places; see the text for more details.
-
- Log in to the old backend webserver as root.
- Check if there are duplicate attachment names (that were in different directories) on the old WordPress site by running:
cd /var/lib/wordpress/$FRONTENDVHOST/wp-content/uploads diff <(find . -type f |sed 's@.*/@@' | sort -u) \ <(find . -type f |sed 's@.*/@@' | sort)
and:
- if there are no duplicates then there is nothing to do.
- if there are duplicates but they are attachments you don’t care about (i.e. in obsolete pages) then there is nothing to do.
- if there are duplicates and they are attachments you do care about then this sub-procedure needs improving!
- Check if there are spaces in filenames and directories (which might screw up later commands in this section) on the old WordPress site by running:
cd /var/lib/wordpress/$FRONTENDVHOST/wp-content/uploads find . -type f | grep ' '
and:
- if there are no spaces then there is nothing to do.
- if there are spaces then this sub-procedure needs improving!
- Create a map mapping each attachment’s filename to its containing directory on the old WordPress site by running:
cd /var/lib/wordpress/$FRONTENDVHOST/wp-content/uploads find . -type f | while read X; do echo "${X##*/} ${X#./}"; done | sort > /tmp/map.txt
- Transfer the map to /tmp on the new backend webserver.
- Log in to the new backend webserver.
- Convert the map to a series of calls to
- wp (to adjust paths in the target server’s wordpress database)
- mkdir and mv (to move the attachment from the single directory where they all arrived to the particular subdirectory where it belongs)
by running:
{ echo '#!/bin/bash' R=/var/www/$FRONTENDVHOST T=$(wc -l < /tmp/map.txt) I=0 while read OLD NEW; do ((++I)) echo "echo -ne \"$I/$T\\r\" >&2" echo "wp --path=$R --quiet search-replace '$OLD' '$NEW' --skip-columns=guid" echo "mkdir -p '$R/wp-content/uploads/${NEW%/*}'" echo "mv '$R/wp-content/uploads/$OLD' '$R/wp-content/uploads/${NEW%/*}'" done < /tmp/map.txt } > /tmp/map.sh
- After reviewing the output script, run it with:
su www-data -s /bin/bash -c 'bash /tmp/map.sh'
Note that:
- if the script goes wrong then it cannot be re-run; you must restore from backup first!
- it can take a long time to run but let it complete because the next steps rely on it having done so
- Note that many attachments on the source side are not referenced in the HTML content. Therefore the importer (which downloads referenced attachments) will not have transferred then. Therefore the associated mv command in the script just run will fail. These errors can be ignored.
- Note that, conversely, the importer will have created thumbnails in new sizes that the source site did not have. Since the script was generated based on a list of files on the source site, there will be no associated mv command in the script just run and so many thumbnails will remain in the top of the uploads directory. There may also be non-thumbnail files. We will deal with these two cases in the next steps.
- Regarding the thumbnails, find them and verify that there is precisely one image in the map file that the above-generate script has already moved by running:
cd /var/www/$FRONTENDVHOST/wp-content/uploads for X in $(ls *.jpg *.png 2>/dev/null); do echo $X | \ sed -r -e 's/-((150|300|768|1024)x[1-9][0-9]*|[1-9][0-9]*x(150|300|768|1024))\.(jpg|png)$/.\4/' | \ xargs find -name | \ wc -l | \ xargs test 1 = || echo $X done
and then remove them:
rm *.jpg *.png
- Regarding the non-thumbnails, find them by running:
cd /var/www/$FRONTENDVHOST/wp-content/uploads ls -p | grep -v / | egrep -v -- '-((150|300|768|1024)x[1-9][0-9]*|[1-9][0-9]*x(150|300|768|1024))\.(jpg|png)$'
and fix them.
Quite what I meant by this I now don’t remember but my notes on this subject were:I searched for the two filenames in a SQL dump of the source database and in the migration XML file. This showed me that the files were not in directories in my normal layout, which made me think they were no longer referenced by content. I also deleted the 2019 hierarchy (empty) and the imagecleanup directory.
- Note to self: For www.pasta.freemyip.com there were thumbnails for images I’d already removed. I removed these too. Which images? For suzanneramsay.freemyip.com there was nothing. For judithhabgood.freemyip.com there was nothing.
- [Don’t do this step; we now remove the thumbnails above and regenerate them below!] Generate a script to move the thumbnails physically and to tell the database that you have moved them by running:
cd /var/www/$FRONTENDVHOST/wp-content/uploads { echo '#!/bin/bash' echo "cd /var/www/$FRONTENDVHOST/wp-content/uploads" T=$(ls *.jpg *.png 2>/dev/null | wc -l) I=0 for X in *.jpg *.png; do ((++I)) echo "echo -e -n \"$I/$T\\r\" >&2" D="$(find . -name $(echo $X | sed -r -e 's/-((150|300|768|1024)x[1-9][0-9]*|[1-9][0-9]*x(150|300|768|1024))\.(jpg|png)$/.\4/') | sed -e 's/^.\///' -e 's/\/[^\/]*$//')" echo "wp --path=$(cd ../.. && pwd) --quiet search-replace '$X' '$D/$X' --skip-columns=guid" echo "mkdir -p '$D'" echo "mv '$X' '$D/'" done } > /tmp/map2.sh
and after reviewing the output script, run it with:
su www-data -s /bin/bash -c 'bash /tmp/map2.sh'
- Check the media gallery; blanks are probably being shown instead of thumbnails.
- Regenerate the thumbnails by running:
su www-data -s /bin/bash -c "wp --path=/var/www/$FRONTENDVHOST media regenerate --yes"
- You may see some errors regarding wp’s attempts to thumbnail non-image attachments, like this:
Warning: not authorized `/var/www/judithhabgood.freemyip.com/wp-content/uploads/publications/historic-orpington/mayfieldmanor.pdf' @ error/constitute.c/ReadImage/412 (ID 657) Warning: No metadata. (ID 657)
- Reload the media gallery; thumbnails of images that appear in galleries should now be present.
- Verify that most pages are okay.
Fixing media import problems – part 2a: superfluous images
To be run logged in to WordPress unless stated otherwise.
- Go to Settings–>Media and set all thumbnail sizes to 0x0; this will prevent WordPress from generating thumbnails.
- Go to Tools–>DNUI–>Options and set:
- number of images shown on 1 page: 350
- ignore size list: original (this is the only way I could get DNUI to ignore banners, which are not attached to pages)
- Go to Tools->DNUI->Images and
- do not delete banners (which DNUI false identifies as not in use)
- delete images whose have names are of the format <prefix>-<width>x<height>.<suffix>
- carefully consider whether other images may be deleted or not (e.g. if a page may be modified in the future to include already-uploaded-but-not-currently-used images then don’t delete those images).
Fixing media import problems – part 2b: unregistered images
To be run logged in to WordPress unless stated otherwise.
By ‘unregistered’, I mean not registered in the media library.
- Ensure banners are in the Media Library as follows:
- Verify that pages display banners and galleries correctly.
- Go to the Media Library and check. If all banners are present then skip the rest of this sub-procedure.
- Log in to the backend webserver as root and run something like:
cd /var/www/$FRONTENDVHOST/wp-content/uploads su www-data -s /bin/bash ls common/banners/* | \ egrep -v -- '-(150|300|768|1024)x[1-9][0-9]*\.(png|jpg)$' | \ xargs -n 1 wp --path=/var/www/$FRONTENDVHOST media import --skip-copy
The exact command you will needed will depend on whether you have stored banners in a dedicated subdirectory and what that directory is. For suzanneramsay.freemyip.com I ran:
cd /var/www/$FRONTENDVHOST/wp-content/uploads eval ls "*{$(file * | grep 990 | sed 's/:.*//' | sed 's/\..*//' | paste -s -d,)}*" | \ egrep -v -- '-(150|300|768|1024)x[1-9][0-9]*\.(png|jpg)$' | \ xargs -n 1 wp --path=/var/www/$FRONTENDVHOST media import --skip-copy
- Repeat this section until it reports no errors.
- Check all pages for errors and either fix them or record the error in order to fix it later. Typically there will be relatively few different kinds of errors.
Fixing media import problems – part 2c: missing images
Missing images will be dealt with when we crawl the website.
Fixing media import problems – part 2d: 768x images
- Go to DNUI and see if it lists any 768x thumbnails. If not then skip the rest of this section.
- I saw that DNUI was listing the banners and many 768x thumbnails.
- I found this size defined here:
www-data$ wp --path=/var/www/www.pasta.freemyip.com media image-size +--------------+-------+--------+------+-------+ | name | width | height | crop | ratio | +--------------+-------+--------+------+-------+ | full | | | N/A | N/A | | medium_large | 768 | 0 | soft | N/A | | large | 0 | 0 | soft | N/A | | medium | 0 | 0 | soft | N/A | | thumbnail | 0 | 0 | hard | 0:0 | +--------------+-------+--------+------+-------+ www-data$
- But the ‘medium_large’ size is not listed in the Settings->Media page.
- So in DNUI I hid the ‘original’ sized images (banners) and deleted all the rest.
- It may be that the 768×0 size is required; it seems to be something to do with responsive images.
Reverse proxy pagination issue
- Log in to WordPress.
- Go to Pages and move the mouse over the ‘next page’ button. If it displays the correct link then skip the rest of this section.
- The bug is described here and a workaround here, which we will now apply.
- Log in to the backend webserver as root.
- Edit /var/www/$FRONTENDVHOST/wp-admin/includes/class-wp-list-table.php ans change the two occurrences of:
$current_url = set_url_scheme( 'http://' . $_SERVER['HTTP_HOST'] . $_SERVER['REQUEST_URI'] );
to this:
/* $current_url = set_url_scheme( 'http://' . $_SERVER['HTTP_HOST'] . $_SERVER['REQUEST_URI'] ); */ if(!empty($_SERVER['HTTP_X_FORWARDED_HOST'])){ $hostname = $_SERVER['HTTP_X_FORWARDED_HOST']; } else { $hostname = $_SERVER['HTTP_HOST']; } $current_url = ( is_ssl() ? 'https://' : 'http://' ) . $hostname . $_SERVER['REQUEST_URI'];
- Log in to WordPress.
- Check that the problem is now fixed and that browsing your site when not logged in still works.
- Note that, since this workaround is in core WordPress file, upgrades may remove the workaround and as such this section needs to be repeated until the bug is properly fixed. (I am unwilling to make the file unwritable in case that results in partly broken upgrades.)
Broken links
- Go to Settings–>Link Checker–>Advanced and click ‘Recheck all pages’.
- Go to Tools–>Broken Links.
- Fix the errors reported there.
Users
- For other sites, review the list of users and their roles and adjust accordingly.
- Each user should adjust to suit their own taste:
- the widgets that appear on the dashboard
- the widgets show in the editor
- page list length (under Pages)
- personal settings (under Users)
Non-user-specific settings
- Justified Gallery–>Tiles Style–>Select tiles style: without styling
Selected theme
- For www.pasta.freemyip.com:
- Select theme GeneratePress.
- Make the following theme customisations:
- Customize->Site Identity:
- Site title: www.pasta.freemyip.com
- Hide site title: yes
- Tagline: <empty>
- Hide site tagline: yes
- Customize->Layout->Container:
- Container width: 990px
- Content layout: One Container
- Customize->Layout->Primary Navigation:
- Navigation Location: No Navigation
- Customize->Layout->Sidebars:
- Sidebar Layout: Content (no sidebars)
- Blog Sidebar Layout: Content (no sidebars)
- Single Post Sidebar Layout: Content (no sidebars)
- Customize->Footer:
- Footer Widgets: 0
- Footer Width: contained
- Customize->Colors:
- Link colour: #b12930
- Link colour Hover: #b12930
- Customize->Typography:
- Body: Roboto
- H1, H2, H3: Roboto Slab
- #Customize->Menues:
- # create a menu, add no items, set it as primary menu, don’t automatically add top level items to it
- Customize->Home Page Settings
- Your homepage displays: a static page
- Homepage: Welcome!
- Customize->Additional CSS:
/* * hide stuff above banner */ .inside-header { display: none; } /* * suppress native copyright */ footer.site-info { display: none; } /* * for man pages */ pre { font-family: monospace; font-size: 108%; } h1, h2, h3 { color: #777777; } /* * less space after banner */ div.entry-content { margin-top: 0px; } /* * make automatically inserted * heading look like manually * inserted heading. */ header.entry-header h1 { margin-bottom: 20px; } /* * preformatted code blocks look * fine, but inline code looks too * light and very slightly the * wrong font. * * font-weight: besides being able * to use words, we can use numbers. * 400 is normal and 700 is the same * as bold. */ code { font-family: monospace; font-weight: 550; font-size: 17px; } /* * block quote font size is * significantly bigger than the * rest of the text in the body. * Fix that. */ blockquote { font-size: 17px; }
- For suzanneramsay.freemyip.com:
- Suzie did the theme customisation.
- For judithhabgood.freemyip.com:
- Select theme Reddle (this will be changed later but I’m trying to make the transition from one WordPress installation to another as painless as possible for the main author).
- Make the following theme customisations:
- Customize->Site Identity:
- Site title: www.pasta.freemyip.com
- Tagline: <empty>
- Display Site Title and Tagline: no
- Customize->Menus:
- If main-menu not present:
- Create New Menu
- Menu Name: main-menu
- If main-menu present:
- main-menu: add items including Contact
- Menu Location: Primary Menu
- Menu Options–>Automatically add new top-level pages to this menu: yes
- If main-menu not present:
- Customize->Widgets: remove all
- Customize->Site Identity:
- For hsrs2019.freemyip.com:
- Alexis did this second time, cloning the original site’s style.
- Then click Publish.
Other themes I considered
I did a very quick review of several themes before deciding to use GeneratePress. This is mainly here to prevent myself from wondering about switching.
Agama
-
-
- site title and tag line can be blanked to clear box at the top, although a box still appears.
- create empty menu and use it *primary* menu
- my current featured images are not wide enough (even in boxed layout)
-
GeneratePress
-
-
- looks good; banners do not need adjusting
- need additional CSS to remove the top box:
.inside-header { padding: 0px; } footer.site-info { display: none; } pre { font-family: monospace; }
-
BlogFeedly
-
-
- need to set smaller font for pre-formatted sections (e.g. man pages) else they wrap wrong
- looks good; banners do not need adjusting; better edge-of-page padding than GeneratePress
-
Bloggist
-
-
- didn’t experiment as initial view looked bad
-
Davis
-
-
- didn’t experiment as initial view looked bad
-
Dazzing
-
-
- featured images not displayed
-
Flat
-
-
- featured images not displayed
- nice fonts
-
Flat-Sky
-
-
- featured images not displayed
-
Generic
-
-
- didn’t experiment as initial view looked bad
-
Isola
-
-
- featured images not displayed
-
Less Reloaded
-
-
- featured images not displayed
-
Lyretail
-
-
- didn’t experiment as initial view looked bad
-
Maxwell
-
-
- looks almost as good as the first couple, but the CSS editing to hide blocks near the top didn’tw work
-
Minnow
-
-
- featured images not displayed
-
Modernize
-
-
- my featured images got stretched vertically to consume about 50% of space
-
Neve
-
-
- featured images not displayed
-
Newspaperist
-
-
- didn’t experiment as initial view looked bad
-
OceanWP
-
-
- didn’t experiment as initial view looked bad
-
Pitch
-
-
- featured images not displayed
-
Reddle
-
-
- I know what this is like, it’s just not supported any more
-
Responsive Mobile
-
-
- couldn’t hide ‘In Archive’ panel on right
-
Reyl Lite
-
-
- featured images not displayed
-
Roohani
-
-
- featured images not displayed
-
Seasonal
-
-
- didn’t experiment as initial view looked bad
-
Shift
-
-
- large featured image size
-
SS Infinity
-
-
- odd space/title/image ordering
-
SuevaFree
-
-
- didn’t experiment as initial view looked bad
-
TecBlogger
-
-
- featured images not displayed
-
Tempo
-
-
- didn’t experiment as initial view suggested a lot of work required
-
Tiny
-
-
- odd space/title/image ordering
- seem unable to get rid of space all the way down left of page
-
Tonal
-
-
- featured images not displayed
-
TwentyNineteen
-
-
- featured images enormous
-
TwentySeventeen
-
-
- parallax
-
TwentySixteen
-
-
- looks dated
-
Wilson
-
-
- didn’t experiment as initial view looked bad
-
WP Less is more
-
-
- experiment!
-
Writings
-
-
- didn’t experiment as initial view looked bad
-
Zakra
-
-
- can’t put featured image above title (though I think the designer intended that it’s possible, but it seems to only be possible on posts, not pages
-
Fixing media import problems – part 3: wget –spider
This section can only be done once the theme has been selected and customised; this is because the theme may present a ‘link bar’ that can include links that do not appear in the text body, which would make a spider not see the link and so not explore it and so not discover possible broken links or missing media.
- Loop #1:
- Scan for broken links using:
wget --spider -o wget.log -e robots=off -r -p https://$FRONTENDVHOST/
- Get a summary of the errors that found using:
grep -i 'awaiting response' wget.log | sort | uniq -c
- For www.pasta.freemyip.com the errors broken down into a few large groups and plus just four singletons:
lasagne$ grep -i 'awaiting response' wget.log | sort | uniq -c ??? HTTP request sent, awaiting response... 200 OK 338 HTTP request sent, awaiting response... 404 Not Found lasagne$
of which the 404 errors broke down further as follows:
lasagne$ grep -B 2 '404 Not Found' wget.log | grep https | fgrep -c '/?p=' 266 lasagne$ grep -B 2 '404 Not Found' wget.log | grep https | grep -c '/cycling/.*/cycling/' 56 lasagne$ grep -B 2 '404 Not Found' wget.log | grep https | grep -c '/hiking/.*/hiking/' 11 lasagne$ grep -B 2 '404 Not Found' wget.log | grep https | grep -c '/installing-xp/.*/installing-xp/' 1 lasagne$ grep -B 2 '404 Not Found' wget.log | grep https | egrep -v '(/cycling/.*/cycling/|/\?p=|/hiking/.*/hiking/|/installing-xp/.*/installing-xp/)' | sed 's/.*uploads\///' xp-eventlog-alert.png --2019-04-10 14:49:49-- https://www.pasta.freemyip.com/computing/articles/using-e-trex-with-linux/ --2019-04-10 14:49:57-- https://www.pasta.freemyip.com/private/computing/ kompass-wanderkarte-3-Allg%C3%A4uer-Alpen-Kleinwalsertal-1-150x150.jpg lasagne$
and for suzanneramsay.freemyip.com I got:
lasagne$ grep -i 'awaiting response' wget.log | sort | uniq -c 149 HTTP request sent, awaiting response... 200 OK 5 HTTP request sent, awaiting response... 401 Unauthorized 1 HTTP request sent, awaiting response... 403 Forbidden 28 HTTP request sent, awaiting response... 404 Not Found lasagne$
and for judithhabgood.freemyip.com I got:
lasagne$ grep -i 'awaiting response' wget.log | sort | uniq -c 210 HTTP request sent, awaiting response... 200 OK 3 HTTP request sent, awaiting response... 401 Unauthorized 1 HTTP request sent, awaiting response... 403 Forbidden 44 HTTP request sent, awaiting response... 404 Not Found 1 HTTP request sent, awaiting response... 405 Method Not Allowed 1 HTTP request sent, awaiting response... 414 Request-URI Too Long lasagne$
of which the 404 errors broke down further as follows:
lasagne$ grep -B 2 '404 Not Found' wget.log | sed -n 's/^--2.\{22\}//p' | sort https://judithhabgood.freemyip.com/?p=16 https://judithhabgood.freemyip.com/?p=203 https://judithhabgood.freemyip.com/?p=213 https://judithhabgood.freemyip.com/?p=221 https://judithhabgood.freemyip.com/?p=231 https://judithhabgood.freemyip.com/?p=237 https://judithhabgood.freemyip.com/?p=243 https://judithhabgood.freemyip.com/?p=250 https://judithhabgood.freemyip.com/?p=256 https://judithhabgood.freemyip.com/?p=267 https://judithhabgood.freemyip.com/?p=274 https://judithhabgood.freemyip.com/?p=278 https://judithhabgood.freemyip.com/?p=280 https://judithhabgood.freemyip.com/?p=368 https://judithhabgood.freemyip.com/?p=393 https://judithhabgood.freemyip.com/?p=395 https://judithhabgood.freemyip.com/?p=426 https://judithhabgood.freemyip.com/?p=451 https://judithhabgood.freemyip.com/?p=458 https://judithhabgood.freemyip.com/?p=467 https://judithhabgood.freemyip.com/?p=476 https://judithhabgood.freemyip.com/?p=480 https://judithhabgood.freemyip.com/?p=487 https://judithhabgood.freemyip.com/?p=550 https://judithhabgood.freemyip.com/?p=558 https://judithhabgood.freemyip.com/?p=561 https://judithhabgood.freemyip.com/?p=572 https://judithhabgood.freemyip.com/?p=593 https://judithhabgood.freemyip.com/?p=599 https://judithhabgood.freemyip.com/?p=604 https://judithhabgood.freemyip.com/?p=638 https://judithhabgood.freemyip.com/?p=647 https://judithhabgood.freemyip.com/?p=667 https://judithhabgood.freemyip.com/?p=673 https://judithhabgood.freemyip.com/?p=692 https://judithhabgood.freemyip.com/?p=7 https://judithhabgood.freemyip.com/?p=729 https://judithhabgood.freemyip.com/?p=732 https://judithhabgood.freemyip.com/?p=735 https://judithhabgood.freemyip.com/?p=738 https://judithhabgood.freemyip.com/?p=771 https://judithhabgood.freemyip.com/chapter-2-william-habgood/ https://judithhabgood.freemyip.com/orpchurchinv/ https://judithhabgood.freemyip.com/willmarygee/ lasagne$
and for hsrs2019.freemyip.com I got:
lasagne$ grep -i 'awaiting response' wget.log | sort | uniq -c 53 HTTP request sent, awaiting response... 200 OK 5 HTTP request sent, awaiting response... 404 Not Found 1 HTTP request sent, awaiting response... 405 Method Not Allowed lasagne$
of which the 404s were all …/?p=…
- For the ‘/?p=’ links, I looked at the source code of some pages and saw this in the headers:
<link rel='shortlink' href='https://www.pasta.freemyip.com/?p=308' />
and I fixed this by going to WP Head Optimizer and setting: Remove Shortlink: yes.
- For the few links for the form ‘…/<some-word>/…/<same-word-again>/…’. I can only attribute this to an error in the exporter or importer. It was easy enough to correct with wp:
grep -B 2 '404 Not Found' wget.log | grep https | \ egrep '(/cycling/.*/cycling/|/hiking/.*/hiking/|/installing-xp/.*/installing-xp/)' | \ sed 's/.*uploads\///' | while read X; do \ echo wp --path=/var/www/www.pasta.freemyip.com search-replace $X "${X#*/*/}" --skip-columns=guid done
and, after verifying the output, piping it in to ‘sh’ (or removing the ‘echo’).
- For the ‘401 Unauthorized’, these are due to the Apache-level protection of the wp-admin directory and, as such, are acceptable.
- For the ‘403 Fobidden’, this is due to the …/feed link in the headers and I fixed this by going to WP Head Optimizer and setting: Remove RSS Feed URL: yes.
- For the ‘405 Method Not Allowed’, this was due to the Disable XML RPC plugin and, as such, is acceptable.
- For the ‘414 Request-URI Too Long’, this is to the oEmbed in the headers and I fixed this by going to WP Header Optimizer and setting: Remove oEmbed Discovery Links: yes.
- For the four other 404s for www.pasta.freemyip.com, I checked the Apache logs for the referrers. The errors were then seemingly all easily solved by correcting the HTML or download then image and re-uploading it (more on this again later).
- For the three other 404s for judithhabgood.freemyip.com, I checked the Apache logs for the referrers:
rotelle# grep 'HEAD /chapter-2-william-habgood/ .* 404 ' /var/log/apache2/$BACKENDVHOST-ssl-access.log | tail -1 192.168.1.9 - - [14/May/2019:12:58:06 +0200] "HEAD /chapter-2-william-habgood/ HTTP/1.1" 404 1814 "https://judithhabgood.freemyip.com/publications/habgood-vs-habgood-in-chancery/chapter-1-james-habgood/" "Wget/1.17.1 (linux-gnu)" rotelle#
verified the wrong link in the referring page and corrected it. The other two were similar.
- Scan for broken links using:
- Loop #2:
- For www.pasta.freemyip.com I got:
lasagne$ grep -B 2 '404 Not Found' wget.log | grep https | egrep -v '(/\?p=[0-9]+)$' | sed 's/.*\.//' | sort | uniq -c 1 com/computing/articles/using-e-trex-with-linux/ 1 com/private/computing/ 52 gpx 1 iso 14 xls lasagne$
and for judithhabgood.freemyip.com I got:
lasagne$ grep -i 'awaiting response' wget.log | sort | uniq -c 130 HTTP request sent, awaiting response... 200 OK 3 HTTP request sent, awaiting response... 401 Unauthorized 1 HTTP request sent, awaiting response... 403 Forbidden 1 HTTP request sent, awaiting response... 405 Method Not Allowed 1 HTTP request sent, awaiting response... 414 Request-URI Too Long lasagne$
- For www.pasta.freemyip.com I got:
-
- So I’ll solve the gpx, xls and iso problems first. I think they were not download and uploaded by the importer because these file types were not on the whitelist. I tarred up the source files and copied them over manually.
- Note that “unattached” does not imply not-known to media library or file non-existent. It just means there’s no “associated” page. Is that a bad thing?
- For the other two, they were really due to errors in the HTML; I therefore de-linked them and re-linked them to the right thing.
- For judithhabgood.freemyip.com I see a lot of the same things I saw on loop #1.
- For hsrs2019.freemyip.com, all issues were now solved.
- Loop #3:
- For www.pasta.freemyip.com I got:
lasagne$ grep -B 2 '404 Not Found' wget.log | grep https --2019-04-11 12:58:44-- https://www.pasta.freemyip.com/private/computing/ --2019-04-11 12:59:31-- https://www.pasta.freemyip.com/computing/articles/using-e-trex-with-linux/ --2019-04-11 12:59:39-- https://www.pasta.freemyip.com/wp-content/uploads/hiking/maps-covers/kompass-wanderkarte-8-Tegernsee-Schliersee-Wendelstein-e1554908297154.jpg --2019-04-11 12:59:45-- https://www.pasta.freemyip.com/wp-content/uploads/hiking/maps-covers/kompass-wanderkarte-3-Allg%C3%A4uer-Alpen-Kleinwalsertal-e1554907440657.jpg lasagne$
and for judithhabgood.freemyip.com I got:
lasagne$ grep -i 'awaiting response' wget.log | sort | uniq -c 126 HTTP request sent, awaiting response... 200 OK 1 HTTP request sent, awaiting response... 405 Method Not Allowed 1 HTTP request sent, awaiting response... 414 Request-URI Too Long lasagne$
- The first ones are due to more occurences of the same wrong-link-in-HTML as documented just above (remember that wget doesn’t show duplicate errors so I don’t see all occurrences, only the first). I fixed those.
- For these bloody map covers, I stripped the e<timestamp> suffixes the references in the HTML, but then the images went missing from the pages even though the non-suffixed file existed. I went to the uploads dir and search for them and there were two of each image: a suffixed one and a non-suffixed one. In the library there was only one: the suffixed one. I deleted it from the library and both files were removed. At this point I renamed the images on my desktop and uploaded them under new names and then went to each page and made it it use the new name.
- For the ‘414 Request-URI Too Long’, I mirrored the entire site and then just grepped for image/png which had appeared in the much-too-long URL, which revealed that an image was encoded and directly placed in the HTML. The image did not display in the old site so I had no reluctance to remove the long link from the new site.
- For www.pasta.freemyip.com I got:
- Loop #4:
- For www.pasta.freemyip.com I got:
lasagne$ grep -B 2 '404 Not Found' wget.log | grep https --2019-04-11 14:43:08-- https://www.pasta.freemyip.com/private/computing/ lasagne$
- and for judithhabgood.freemyip.com I got no errors.
- More occurences of the same wrong-link-in-HTML as documented just above.
- For www.pasta.freemyip.com I got:
- Loop #5:
- For www.pasta.freemyip.com I got:
lasagne$ grep -B 2 '404 Not Found' wget.log | grep https --2019-04-12 15:41:11-- https://www.pasta.freemyip.com/wp-content/uploads/hiking/maps-covers/kompass-wanderkarte-3-Allgäuer-Alpen-Kleinwalsertal-2.jpg lasagne$
- I renamed the file in the library and updated the html of the affected pages and then this was okay. So no umlauts!
- For www.pasta.freemyip.com I got:
- Loop #6:
- Spider now happy:
lasagne$ grep -B 2 '404 Not Found' wget.log | grep https lasagne$
- Do not delete the wget.log; we’ll need it in a moment.
- Spider now happy:
Fixing media import problems – part 4: cross-checking DNUI, uploads and the database
- Put the DNUI results in file dnui.txt.
- Run:
cat dnui.txt | awk '{ print $5 }' | sort -u > dnu-deduped.txt
It should look something like this:
common/banners/beckt-tour.png common/banners/beirut.jpg ... private/filmnight/filmight-20130306.png private/filmnight/filmight-20130417.png
- Extract the attachments list out of the spider logs by running:
cat wget.log | sed -n 's/^--.*\/uploads\///p' | sort -u > referenced-uploads-unique.txt
The output looked like this:
common/banners/alexis.jpg common/banners/beirut.jpg ... recipes/tiramisu/tiramisu13.jpg recipes/tiramisu/tiramisu14.jpg
- Then check if DNUI is seeing banners as unused even when they are:
lasagne$ comm -1 -2 dnui-unique.out referenced-uploads-unique.txt common/banners/beirut.jpg common/banners/cards.jpg common/banners/centos-logo.png common/banners/cloud.png common/banners/computing.jpg common/banners/cooking.jpg common/banners/cycling2.jpg common/banners/dummy-index-page.png common/banners/fritzbox.png common/banners/gardening.png common/banners/hiking.jpg common/banners/it-support.png common/banners/japonica.jpg common/banners/liverpool.jpg common/banners/manpage.png common/banners/poetry-own.jpg common/banners/poetry-rilke.jpg common/banners/poetry.png common/banners/private.png common/banners/recipes.png common/banners/software.png common/banners/storage.png common/banners/ubuntu-logo.png common/banners/virtualisation.png common/banners/vmware.png common/banners/welcome.png common/banners/windows-xp-logo.png common/banners/wordpress.png lasagne$
which confirms that banners are in use and cannot be deleted.
- So if we exclude these from DNUI output and look at what other banners it thinks can be deleted by running:
lasagne$ comm -2 -3 dnui-unique.out <(comm -1 -2 dnui-unique.out referenced-uploads-unique.txt) | grep banner common/banners/beckt-tour.png common/banners/calendar.jpg common/banners/debian-logo.png common/banners/filmnight.png common/banners/flat-banner.jpg common/banners/personal-manifesto.png common/banners/sandbox.png common/banners/welcome2.png lasagne$
- But, by the names, I can tell most of these are in use, but on private pages. The ones that aren’t in use and won’t be required in the future (welcome2 and personal-manifesto) I deleted using Media Library.
- So at this point I think, excluding banners, the things DNUI thinks can be deleted really can be deleted.
- But I’ll do an image backup of the VM first.
- So then I deleted the images one-by-one, rather than using the ‘Delete All’ buttom, because I wanted to exclude banners.
- Now check what is in the library that is not on the filesystem:
rotelle# comm -1 -3 <(find * -type f | sort) \ <(mysql wwwpastafreemyipcom -sN -e "SELECT distinct meta_value FROM wp_postmeta WHERE meta_key = '_wp_attached_file' order by meta_value;" | sort) rotelle#
I.e. there was no output, which is good.
- And how about the other way round? I.e. not in the library but is on the filesystem:
rotelle# comm -2 -3 <(find * -type f | sort) \ <( mysql wwwpastafreemyipcom -sN -e "SELECT distinct meta_value FROM wp_postmeta WHERE meta_key = '_wp_attached_file' order by meta_value;" | sort) | egrep -v '(imagecleanup|ithemes-security)' | wc -l 67 rotelle#
- To add these 67 files to the library, let’s try one manually with wpcli. First choose a file:
rotelle# comm -2 -3 <(find * -type f | sort) <( mysql wwwpastafreemyipcom -sN -e "SELECT distinct meta_value FROM wp_postmeta WHERE meta_key = '_wp_attached_file' order by meta_value;" | sort) | egrep -v '(imagecleanup|ithemes-security)' | head -1 computing/installing-xp/virtio-win-0.1.100.iso rotelle#
- Manually verify that it is not in the media library.
- Verify that it it is in the filesystem:
rotelle# cd /var/www/www.pasta.freemyip.com/wp-content/uploads/ rotelle# ls -ld computing/installing-xp/virtio-win-0.1.100.iso -rw-r--r-- 1 www-data www-data 160704512 Apr 11 11:33 computing/installing-xp/virtio-win-0.1.100.iso rotelle#
- Then register it in the library:
rotelle# su www-data -s /bin/bash -c 'wp media import computing/installing-xp/virtio-win-0.1.100.iso --skip-copy' Imported file 'computing/installing-xp/virtio-win-0.1.100.iso' as attachment ID 9950. Success: Imported 1 of 1 items. rotelle#
- So now we do it for all the files:
rotelle# cd /var/www/www.pasta.freemyip.com/wp-content/uploads rotelle# comm -2 -3 <(find * -type f | sort) <( mysql wwwpastafreemyipcom -sN -e "SELECT distinct meta_value FROM wp_postmeta WHERE meta_key = '_wp_attached_file' order by meta_value;" | sort) | egrep -v '(imagecleanup|ithemes-security)' | while read F; do > su www-data -s /bin/bash -c "wp media import $F --skip-copy" > done rotelle#
- Now go back to the first section dealing with import errors.
Miscellaneous
- Try uploading an Excel spreadsheet. If it fails then complete the following sub-procedure:
- Verify that the plugin
- File Upload Types (by WPForms) is installed (actually this did not help, but is documented above so stick with it).
- Go to Settings –> Media and verify that XLS is an allowed extension.
- Edit wp-config.php and add:
define('ALLOW_UNFILTERED_UPLOADS', true);
Installing WordPress and migrating plugins, content and attachments
- On the backend webserver run:
apt -y install apache2 php-cli php-common php-curl php-gd php-intl php-json \ php-mbstring php-mysql php7.4-opcache php-readline php-xml php-xmlrpc \ php-imagick libapache2-mod-php php-zip mariadb-server
(Note that there is no php-opcache package that depends on php7.4-opcache, as there is for the other PHP packages.)
- Run:
a2enmod ssl authnz_ldap remoteip rewrite a2dissite 000-default
- Edit /etc/apache2/ports.conf and comment out the line:
Listen 80
- To allow sites to be imported and if not already done, edit /etc/php/7.4/apache2/php.ini and set:
upload_max_filesize = 101M post_max_size = 102M
- Create an SSH key pair on the old backend webserver and add its public key to the new backend webserver’s authorized_keys file.
- For each website to be migrated:
- Go the WordPress dashboard and verify that it is complaining about the PHP version being old.
- Clone the old backend webserver’s vhost configuration to the new backend webserver (i.e. copy the file over, rename it, change references in the file from old vhost to new vhost).
- Make an empty document root directory
- use
a2ensite
to enable the site - restart apache
- Set the following environment variables:
DATABASENAME=<whatever> DATABASEUSER=<whatever> DATABASEPASSWD=<whatever>
(You can look in the old backend webserver’s wp-config.php if you don’t know the values.)
- Create the database by running:
{ echo "CREATE DATABASE $DATABASENAME;" echo "CREATE USER \"$DATABASEUSER\"@\"localhost\" IDENTIFIED BY \"$DATABASEPASSWD\";" echo "GRANT ALL PRIVILEGES ON $DATABASENAME.* TO \"$DATABASEUSER\"@\"localhost\";" echo "FLUSH PRIVILEGES;" } | mariadb
- Close all WordPress editing sessions.
- Rsync over the entire directory /var/www/<wordpress-site>.
- On the old backend webserver run:
DATABASENAME=<whatever> mysqldump $DATABASENAME > ~/$DATABASENAME.sql
- Transfer the dump from the old backend webserver to the new backend webserver.
- On the new backend webserver run:
mysql $DATABASENAME < ~/$DATABASENAME.sql
- On the frontend webserver backup the vhost configuration file and then update it to point to the new backend vhost. Don’t restart apache just yet.
- Make sure the hostname it points to (e.g. vesuvio-wwwpastafreemyipcom.pasta.net) is resolvable.
- Restart front end apache.
- Test access to the new site, in particular go the WordPress dashboard and verify that it is no longer complaining about the PHP version being old.