A new venture, WP Lookout

A few months ago I had an idea for a new tool and service that I thought could be useful to WordPress developers and agencies who manage lots of WordPress sites, but who can't or don't want to go all in on automatic updates. After spending some time thinking through the details and researching what was already out there, I decided to go ahead and build it.

I started writing code and building features on July 1. A month later in early August I launched the first version to a production environment, and now I'm starting to tell people about it.

Please, allow me to introduce WP Lookout.

I've already written about why I think this service fills in some important gaps in the WordPress ecosystem so I won't say a lot more about that here. If you're someone who has worked with keeping multiple WordPress sites up to date and secure, hopefully you see the potential benefit. And there are a lot more features I'm excited to be working on.

Even if you don't travel in those circles, perhaps you'll identify with this sentiment: we have all of these things in our lives running software that someone else has written (routers, computers, apps on our phones, the navigation systems in our cars, the firmware on our TVs), and we're supposed to keep them up to date with the latest versions, usually in the name of speed and security. But often that turns into a part time job of logging in to those devices, finding the settings screen where updates are displayed, and trying to understand what's changed and how much it matters. Automatic background updates are helping in some cases. But wouldn't it be nice if these devices and tools could come to us and tell us when they're ready for an update, and what exactly has changed? So, WP Lookout is set to do that for at least one small corner of the Internet.

It's been a while since I've set out to launch and grow a new business from nothing. It's thrilling to again be thinking through architecture, strategy, business model, marketing, finances, legal paperwork, scaling, innovation and all the other pieces that go along with trying to make something thrive for the long term. It's also vulnerable to be putting something out into the world and wondering how it will be received, or if anyone will care.

I'm hopeful that good things are ahead. Regardless, I'm proud of what I've done so far. I've learned a lot along the way — about building a SaaS ("software as a service") business, about how awesome developing with Laravel is (I'll write more about that soon on my tech blog) and about what solutions people who manage multiple WordPress sites might or might not be looking for to make their lives easier.

After I'm a bit further along I'll share more details and updates. In the meantime, please check out WP Lookout.

Multimedia journalism and the WordPress block editor

I recently finished a really interesting course in my graduate journalism studies program, focused on cross-platform, multimedia storytelling. It was fun in part because it wove together a lot creative disciplines I had already been experimenting with, including writing, photography, audio production, video production, interviewing people, and assembling the results of those efforts into a coherent, interesting final product that would engage the attention of readers and visitors. I got academic credit for doing things I love to do — nice!

It was also an experience that opened my eyes further to the power of the WordPress block editor for multi-media journalism and storytelling, and I want to share a bit more about that.

(If you're not already familiar with it, the WordPress block editor, also known as Gutenberg, is the updated content editing experience introduced into the WordPress publishing platform back in 2018. It transitions content authoring and editing in WordPress from a fairly linear "title plus paragraphs of text plus maybe some images" workflow into a much more flexible and powerful workflow that can include a wide variety of multi-media, interactive elements positioned throughout the body of a page or article. If you were talking to someone using a few tin cans and some string before, now you're enjoying the wonders of a smartphone.)

Like others I've had some skepticism in the past about the block editor and its place in the world of everyday WordPress users who may not want the additional power it offers. And at a personal level, old habits can die hard; I found myself writing this very post in the "classic" editor because it's how I've mostly always done things. So I haven't always been sure when and were I could definitively say to others that the block editor would be an essential part of their publishing toolkit.

Continue reading "Multimedia journalism and the WordPress block editor"

Flickr to WordPress Part 2: replacing old image references

Almost exactly a year ago, I shared my tool and process for moving a Flickr-hosted photo collection to a WordPress-powered photo website. That tool has been used by a few folks now and I'm so glad it was helpful.

The thing that I hadn't done yet, but now have and detail in this follow-up post, is to create an automated way to clean up references to Flickr-hosted photos that exist in an existing WordPress website or blog. Without this critical final step, one could have a lot of historical content that still references Flickr-hosted photos instead of the version being hosted on your shiny new WordPress-powered photo site.

So here's how I did it. As with my previous post, this information is geared toward a technical audience that is comfortable with the command line and possibly modifying PHP code to suit their own purposes.

Method

I thought about a couple of different ways I could handle this "find and replace" operation.

With 13 years of blog posts, many of which contain references to Flickr photos in some form, making the changes manually was not an option.

I thought about continuing to iterate on my existing PHP command line tool that takes the exported Flickr dataset and generates a WordPress import, which would mean creating some kind of lookup data file where a find-replace command would use the Flickr photo URL to find the new WordPress-hosted photo URL. But when I saw all of the different ways I had embedded Flickr photos in my blog post content:

  • on a line by itself for oembed rendering
  • as simple links to a Flickr photo page
  • as <a> plus <img> tag groups that displayed the images full size inline with a link
  • as <img> tag groups that displayed the images at various smaller sizes, aligned left or right

I realized that I would need to be able to lookup the proper image URL for each display scenario. And given that my WordPress-powered photo site generated different image sizes (and that some of these had changed since the original data migration), that was not going to be simple. No one-size-fits-all substitution would work.

The good news is that WordPress easily supports building a custom REST API endpoint that would support a dynamic lookup of the information I needed on the photo site, for use on any site where I was finding-replacing content. Once I realized I could decouple those operations, it was clear how to proceed.

Creating a "Find by Flickr URL" API Endpoint

The first step, then, was to create a REST API endpoint on my WordPress-powered photo site that would allow me to specify the original Flickr photo URL and find the related WordPress post that had been generated during the migration process.

If you look at the code of the original migration tool, you'll note that for each WordPress post it creates, it adds a post meta field _flickr_photopage where it stores the URL of the Flickr-hosted photo. That usually looks something like https://www.flickr.com/photos/myflickrusername/123456789/. We can use that post meta field to do a simple lookup of the equivalent WordPress post object.

Since I want to be able to retrieve an image URL at a specific size so that I'm not embedding full size, large image files in posts that only need, say, the 300 pixel wide version, I also needed to accept width and height parameters, and then do a lookup of the related attachment file URL in WordPress.

Here's the class that I created to do all of this. If you use it, you'll need to customize a few things, including the Flickr username.

With that class loaded into my WordPress photo site's theme or in a plugin, now I have access to this kind of API call:

https://my-wp-photo-website.com/wp-json/myphotos/v1/find-by-flickr-url/?flickr-url=https://www.flickr.com/photos/myflickrusername/123456789/

And the returned JSON response of that kind of API call would look something like this:

{
     result: "found",
     post_id: 2425,
     permalink: "https://photos.chrishardie.com/2014/08/updesk-setup/",
     thumbnail_id: 5424,
     thumbnail_url: "https://photos.chrishardie.com/wp-content/uploads/2014/08/15130177147_75d885dc2a_o.jpg",
     thumbnail_width: 2448,
     thumbnail_height: 3264
}

Nice! Now I have the permalink of the photo post that replaces the original Flickr photo page, the URL of the image media file I can use in an <img> tag, and some other meta info if I need it.

The Find-and-Replace WP CLI Plugin

Now, it's time to use that API endpoint in a big find-replace operation.

The clear choice was to make a WP CLI command that could be used to run this on the command line, where I could log and review warnings and errors and have better memory management.

In creating this command, I used this general approach:

  1. Get all the posts in the WordPress database that mentioned a Flickr URL with my username in it
  2. For each post found, look for specific kinds of Flickr link and image references that need to be replaced
  3. Extract the original Flickr photo page URL from those references
  4. Use the API endpoint created above to look up the corresponding information on the photo site
  5. Update the post content with the information retrieved from the API

It sounds fairly simple, but I ran into several challenges and opportunities for optimization:

  • Not only were there references to the Flickr URL structure above, there were variations to consider such as the internal version of my Flickr user account ID that was in wider use years ago, or the flic.kr shortened version of their domain.
  • The API lookups could generate a lot of activity on my photo site, so I added some caching since those responses should rarely be changing.
  • I found some photos that I had apparently set to "private" or "contacts only" on Flickr but had left referenced in my blog posts, so I had to manually address those.
  • My Flickr-to-WordPress migration tool didn't handle Flickr "sets" (although it preserved and stored the data needed to handle the), so I had to redirect those references.
  • I had to make sure not to replace Flickr references to other people's photos.
  • Flickr varied its use of http versus https in different embed code it generated over the years.

In the end, I had a working plugin that could do a dry-run operation to see what it was going to change, and then do a "for real" run to actually update the posts as stored in the database.

$ wp flickr-fixer fix-refs --dry_run=false
Getting all posts containing Flickr references...
Found 203 posts to process.
Success: 621 replacement(s) made across all posts

With the API lookup cache primed, on my site it only took a minute or two to run. YAY!

You can see the final plugin code here.

(If you use it, you'll need to find/replace my Flickr username and a few other hardcoded references accordingly.)

Lessons Learned

When I think about the time I put in to first creating the original Flickr-to-WordPress migration tool, and then the time put into this content cleanup tool, it turns out it was a non-trivial project. But it always felt like the right thing to do, since once I was moved fully into WordPress I would have absolute control over my photo collection without depending on the changing services or business model of Flickr or anyone else.

It also highlights a few important lessons for migrations and owning your data online:

  • Try to be consistent in the ways you reference third-party tools and services in your content or workflows. If you have a bunch of variations and inconsistencies in place, any future move to another tool is going to be that much more painful.
  • Hold on to as much metadata as you can. You never know when it's going to come in handy.
  • When tackling big migrations, break hard problems up into smaller, slightly easier problems.
  • Document your thinking and your work along the way. It's too easy to get stuck going in circles on longer projects if you forget where you've already been.
  • APIs are magical and user-facing services that don't have them should be avoided at all costs.

I think this concludes my 14.5 years of being a Flickr user. I've canceled my Pro subscription and plan to delete my account in the weeks ahead.

If you find any of these tools useful, or if you have a different approach, I'd love to hear about it.

Better WordPress multisite image URLs

I've been running a personal WordPress multisite instance for several years now, and I use it to host a variety of personal and organizational sites, including this one. I really like the ways it allows me to standardize and consolidate my management of WordPress as a tool, while still allowing a lot of flexibility for customizing my sites just as though they were individual self-hosted sites.

For the most part, my use of WordPress in multisite/network mode doesn't have any user-facing implications, especially since I use the WordPress MU Domain Mapping plugin to map custom domain names to every site I launch. As far as anyone visiting my sites knows, it's a standalone WordPress site that looks and works like any other.

The one exception to this has been the URL structure for images and other attachments that I upload to any site hosted on this multisite instance. Whereas the typical WordPress image URL might look like this:

https://example.com/wp-content/uploads/2019/03/my_image.jpg

on a multisite instance, there is an directory structure added in to separate each site's uploads into its own subdirectory:

https://example.com/wp-content/uploads/sites/25/2019/03/my_image.jpg

where 25 might be the site's unique site ID within that multisite setup.

There's nothing wrong with this approach and it certainly makes technical sense if you have lots of sites on your multisite instance that are either subdirectories or subdomains of the main multisite domain.

Continue reading "Better WordPress multisite image URLs"

Moving photos from Flickr to WordPress

If you're ready to move your own Flickr photo collection to WordPress and feel comfortable on the command line, you can go straight to the Flickr to WordPress tool I built and get started.

Update on Feb 18, 2020: You can now also learn about my tool for finding/replacing old Flickr image references in your WordPress post content.

Here's some backstory:

I used to love Flickr as a place to store photos, and as a community for sharing and discussing photography. But as its ownership changed hands and its future became at times uncertain, I grew reluctant to trust that it could continue to be a permanent home for my own photos. My discomfort increased as I have become more engaged with the need to have full ownership over the things I create online.

So, I set out to migrate my 3.6GB collection of 2,481 Flickr photos, along with their tags, comments and other metadata, into a new home while I still could.

Continue reading "Moving photos from Flickr to WordPress"

Hiding authors and users in WordPress

As an advanced publishing tool, WordPress typically defaults to displaying information about the author behind a given post or page on a WordPress site. But sometimes you want to build a website that has a more "singular" editorial identity, and that doesn't appear to be authored and managed by multiple people, even if it is. I see this regularly with corporate brands, political organizations, larger not-for-profits, and advocacy groups where the identity of a post or page's author could distract from the content being shared.

So how do you keep WordPress from revealing the author information - names, usernames and more - for the administrative users of your site? Here are a few tips, aimed at WordPress developers comfortable customizing their sites through changing the theme and plugin code.

Continue reading "Hiding authors and users in WordPress"

My new team at Automattic

Starting next week, I'll be shifting my focus at Automattic and joining our "WordPress Concierge" team.

It's a small group of talented folks who build custom WordPress sites for influential people and organizations across a bunch of different industries. I'll be creating and supporting the sites we work on while also helping to help the team scale up its internal tools and processes. I'm excited to flex a different mix of skills and ship some new kinds of things.

It's hard to believe that it's already been more than three years since I joined Automattic and began working with the VIP team, where I'm wrapping up my time this week. That role has been full of its own interesting and rewarding adventures that have taken me deep into the heart of the systems and tools that power the modern open web.

Continue reading "My new team at Automattic"

WordCamp for Publishers Denver

I'm excited to be a part of the team that is organizing the first WordCamp for Publishers happening in Denver, Colorado this coming August 17th to 19th.

In my work on Automattic's VIP service offering I've been able to see some of the incredible things that journalists and publications of all types are doing with WordPress - really interesting stuff that pushes the software in new directions. So when I saw some of my colleagues discussing an event that would bring engineering, product and editorial teams together around all the things WordPress can do in the publishing world (for publications big and small), I wanted to be a part of making it happen.

We're looking for speakers and sponsors now. If you know someone who might want to be a part of this first-of-a-kind event, please point them in our direction. And if you're involved in publishing with WordPress at any scale, or just want to learn about how media organizations are using it to modernize their publishing  workflow, I hope you'll consider attending. Tickets ($40) will go on sale soon!

Creating a private website with WordPress

When we became parents in 2015, Kelly and I talked about where and how we wanted to share the initial photos and stories of that experience with a small group of our family and friends. In case you haven't noticed, I feel pretty strongly about the principle of owning our digital homes. So I felt resistance to throwing everything up on Facebook in hopes that we'd always be able to make their evolving privacy and sharing settings and policies work for us, while also trusting that every single Facebook friend would honor our wishes about re-sharing that information.

I took some time to explore tools available for creating a private website that would be relatively easy for our users to access, relatively easy to maintain, and still limited in how accessible the content would be to the wider world. (I tend to assume that all information connected to the Internet will eventually become public, so I try to avoid ever thinking in terms of absolute privacy when it comes to websites of any kind.)

I thought about using WordPress.com, which offers the ability to quickly create a site that is private and viewable only by invited users while maintaining full ownership and control of the content. I passed on this idea in part because it didn't allow quite the level of feature customization that I wanted, and partly because it's a service of my employer, Automattic. While I fully trust my colleagues to be careful and sensitive to semi-private info stored there, it felt a little strange to think of creating something a bit vulnerable and intended for a small group of people within that context. I would still highly recommend the WordPress.com option for anyone looking for a simple, free/low-cost solution to get started.

Here are the WordPress tools I ended up using, with a few notes on my customizations:

Basic WordPress Configuration

For the basic WordPress installation and configuration, I made the following setup choices:

  • I put the site on a private, dedicated server so that I had control over the management and maintenance of the site software (as opposed to a shared server where my content, files or database may be accessible to others).
  • I used a Let's Encrypt SSL certificate and forced all traffic to the SSL version of the site, to ensure all communication and access would be encrypted.
  • I set up a child theme of a default WordPress theme so I could add a few customizations that would survive future parent theme updates.
  • I set "Membership" so that "Anyone can register" in the role of Subscriber (see more below on why this is okay).
  • For Search Engine Visibility I set "Discourage search engines from indexing this site".
  • For discussion I set "

Continue reading "Creating a private website with WordPress"