This article describes some work in progress. I’m sharing to get feedback form the community and there are questions in bold. You can answer in the comments, by replying in social media or emailing me.
This is an oft misunderstood topic. I used to put a low value on training. During my time in tech I am mostly self-taught, figuring things out with the help of Google, reading documentation and generally looking to see how others have solved similar problems. This has worked for me on the whole, but on some occasions it’s definitely extended my learning period. Stepping out to take a course could have helped me understand some fundamentals sooner, which would have allowed me to build up my knowledge more quickly.
I have also come to learn through life that what works for me doesn’t necessarily work for other people (obvious, I know, but I was stuck in that rut for a long time). For some training is the best way of learning, for others like me, figuring stuff out in a more ad-hoc manner is our normal approach. I’m no expert in learning styles, but I’m sure there are many nuances to this. Our job as leaders is to allow our people to tailor their learning experience to their needs.
Certifications & Training Paths
Having said this, I also am aware that we need to level-up the skills of our team. I find most engineers, even senior ones, have gaps in their foundational knowledge. This is only natural in such a complex and dynamic environment as ours, but what can we do about it?
This is where structured certification or training paths come in. These gather together the knowledge you need to be an expert in a particular technology. They ensure you fill all the gaps, and in the case of a certification, they provide a nice certificate to prove that to your clients or future employers of your team members.
At Nolte we are working to put together a list of certifications or training paths to cover off on the requirements for different engineers. It’s still in its early stages, but I’d like to share some important ones:
Work in progress! This will include elements from the back- and front-end paths, plus some WordPress-specific training. Do you know any certifications or training paths for WordPress engineers? Please comment below.
In addition to the above we also expect our engineers to train on core web and engineering principles, such as performance, SEO, accessibility, security, systems design and engineering fundamentals like object-oriented programming.
And we expect training on the “soft” skills, which are so important to being a successful engineer. These include problem solving, technical writing, presentation of ideas, risk management, mentorship and business analysis. Do you know any certifications focussed on the general soft skills required by all engineers?
It’s not one-size-fits-all
Our training paths provide guidance, but it’s up to individuals to figure out what’s required with their managers. Certifications or the completion of a path are not necessary for promotion at Nolte, but they will help accelerate the process in most cases.
We’re still in the process of defining our training paths, any feedback would be wonderful. How do you organise training for your engineering team?
Most engineers aspire to be considered “senior”, the title being recognition of their career journey and the value they can add to a business. However, I find there are many misconceptions amongst engineers about what it really means to be senior. In this article I’m going to lay out my perspective on what it means to be a Senior Engineer.
Let’s start with their technical skills. A senior engineer is an “expert” in one or more technologies. I put the word expert in quotes, because it in itself is a vague word. By expert, I don’t mean that they know every last thing about a language, framework, etc; and I definitely don’t mean that they need to memorise syntax or the like. An expert understands the fundamental principles behind the tool and can apply them to use the tool in the optimal way. The expert may need to Google solutions and they will likely copy and paste code from StackOverflow like the rest of us, but they will be able to quickly understand the explanations found by mapping them to their knowledge of the technology ecosystem they work in.
The expert’s knowledge also goes deeper than a given tool or language, they understand the fundamental principles of the programming paradigms they work with, eg object oriented programming. They also understand the related protocols and systems they need to interact with, eg a senior web engineer understands how information is transmitted over the Internet.
The Software Development Lifecycle
Top engineers don’t just know how to write well structured code, they know how to solve problems throughout the entire SDLC:
Analysis: they have the people skills required to proactively obtain business/technical knowledge and distill it into actionable points. They are not afraid to appear “stupid” by asking the seemingly obvious questions, they know that by doing this they are advancing everybody’s knowledge of the problem sphere. They make sure they understand as much as they can around the problem as they tackle it.
Design: senior engineers can architect a viable solution to the problem taking all the relevant factors into consideration, such as: budget (upfront and ongoing), maintenance, timeline, technical requirements, uptime, scalability, SEO, performance, accessibility and security. They use their creativity to think of and research non-conventional solutions to the problem at hand (they know writing code isn’t always the answer).
Implement: they build the solution whilst staying creative in the face of inevitable change. They maintain transparent communication with stakeholders as things change, they don’t leave people in the lurch by unexpectedly not delivering without prior communication.
Test: they ensure they release code of high quality, using a combination of automated and manual testing strategies depending on what makes sense for the project. They understand the different types of testing (unit, functional, UI, integration, etc) and when to apply them. Even if they work with dedicated testers, they still understand testing and still strive to deliver high-quality code for the testers.
Deploy: senior engineers know the feature is not complete until it is in production, up to that point things can change which add additional work or even make the feature obsolete. They do what they can to ensure their work makes it to production as quickly as possible, and they make sure things are setup to monitor that it’s working once it does.
Underlying the above are great problem solving skills, you can’t be a senior engineer without these. Solving problems in a great way means they (credit to Marc Backes for this list):
make sure not to introduce unnecessary sources of errors
create as little friction with the existing system as possible
think of the bigger picture
have expandability/reusability in mind
make decisions about potential trade-offs
In addition, great engineers understand the value of estimations as a tool for determining what is and is not possible for the given set of constraints. They use this tool to drive conversations with stakeholders. And of course, they have good communication skills (written and spoken) which they use to explain technical concepts and sell ideas, to both technical and non-technical people.
Finally, senior engineers can identify risks on their projects, they communicate them and drive them to mitigation. They are also team players, thinking about how to help their team mates grow into great engineers.
Senior engineers understand that their job is not to write code. I’ll say that again – an engineer’s job is not to write code. Their job is to solve problems. Writing code is often great way of doing this, but it’s not the only way. Sometimes there are more optimal solutions, such as purchase a plugin or SaaS solution, leverage open source or even just tweak an offline process.
Many engineers don’t realise this. They focus on code as the only solution, and they are doing themselves and their businesses an injustice. How many countless hours have been wasted building solutions that didn’t need to be built?
As we continue to strive towards excellence in QA, we are looking to automate more of our QA process. WordPress presents some very specific challenges, and this article discussed some of the strategies we’ve used to automate testing for bespoke WordPress websites.
We have recently started using Happo for screenshot testing. The main reason I like Happo over the alternatives is the browser selection it offers – many UI bugs occur in the browsers we use least ourselves, like IE11 (unfortunately still required by a lot of our clients) and Edge. For us, it’s invaluable having a tool that forces us to review and accept the UI on these browsers, and that catches any subsequent regressions.
Happo solves the issues of setting up a snapshot comparison system and ironing out the inconsistencies you can get with these. However, this still leaves us with the tricky problem of how to ensure consistent snapshots when the content can be edited in the CMS. We solve it by adding a styleguide / component library to our WordPress themes – currently we use this implementation we built ourselves, but we are experimenting with Fractal, an open-source solution, for future projects (check out the impressive Cali Theme by our very own Francisco Giraldo).
We use the Happo wrapper for Cypress to run our tests, as we are using Cypress anyway for our functional testing. Cypress also gives us a way to interact with the UI before taking screenshots, eg if we want to test an element into a certain state.
For a typical project we want to cover a broad spectrum of browsers, devices and screen sizes. This includes Chrome and iOS Safari for mobile; and Chrome, IE11 (if required), Edge, Firefox and Safari on desktop. Only Chrome and Firefox allow for screen widths wider than 1,200px, so we use these to test the UI on wider screens. Our .happo.js file looks like the following for a typical project (with IE11 support required):
We then create a Cypress test (in the cypress/integrations folder) to visit the relevant pages of the styleguide and take a screenshot:
In our case, we had to take a shot of the whole page (by targeting the body tag) because of the way Tailwind CSS is setup to make its CSS more specific (and therefore higher priority for the browser) by targeting an id on the body tag before the actual class.
Once you’ve done the above you can test locally if you replace the process.env variables with the ones from your Happo account (but please be careful not to commit these to your repo, that’s not secure) or you could setup dotenv to inject them. In our case we haven’t done this, because we really only want Happo to run in our CI pipeline to keep costs down.
For us, UI testing is the most important type of testing when it comes to WordPress. WordPress is a tool we mostly use for content-driven sites, like marketing or portfolio sites. For these there is not a lot of functionality in the UI, but we do want them to look as close to perfect as possible.
However, we also build some sites which have very customised back-ends or more advanced functionality, such as eCommerce built on the WooCommerce plugin. In these cases we do want to ensure that critical functionality does not break during a release and this is where Cypress comes in.
Of course we have the same problem with changing content, and this time it can’t be solved with a static styleguide. We have some up with 2 broad solutions:
Under this strategy we aim to write tests that work no matter what the content is. This works when we can target elements based on attributes which are fixed and/or the workflow cannot be changed via the CMS. A WooCommerce flow or browsing for a product and purchasing it is a classic example. Let’s look at another one too in more detail:
Nav Functional Test Example
We can work off the available data to build our tests, making some assumptions: there will always be a menu, there will always be a top-level menu item with no children and there will always be one with children.
First, we can target a menu item with no children using #main-menu li:not(.menu-item-has-children) and test that clicking it correctly redirects us to a new page. We can then assert that we are no longer on the homepage: cy.location('pathname').should('not.eq', '/').
Next, we can look for one with children, #main-menu li.menu-item-has-children and hover over it to make the sub-menu appear. We can then click the first item and run a similar test to the above.
The nice thing with the Cypress/Happo integration is that we could also choose to use intersperse our tests with a call to happoScreenshot(), allowing us to take a screenshot of the component / page in the state we choose.
The other approach is a bit more involved, but has the benefits of greater flexibility and it also tests the admin experience is working as expected. This last point is often overlooked when it comes to testing, but in my experience it’s just as common to cause admin issues as it is to cause front-end issues when making a change or performing plugin updates.
The idea here is for Cypress to login to the WordPress admin, build a page using the page editor and then test it looks and functions as expected on the front-end. This way we have no dependencies on content, we just need a WordPress instance with the required plugins installed and configured.
Let’s take a look at an example of this:
The first thing we do is login to the WordPress dashboard using a custom command we created, cy.adminLogin(). We created a suite of such commands in the Cypress commands.js file, you can see them all here. The commands are sets of reusable steps we can use to perform repeated tasks, like adding a block to a page.
Next we delete any old test data from a previous run using cy.deletePost(), and then create and publish a new page to work on in this test run using cy.createPost() and cy.publishPost(). Note the Cypress.env('hash') suffix on the page slug: we use this to ensure we don’t create a page which clashes with an existing one on the site (as we’re generally working off a copy of the live, staging or dev database).
Now we get to the meat of the test: adding the blocks to the page. In this site we are using ACF to generate custom blocks, and we have of course created some useful commands to facilitate adding them to the page. The sequence is as follows:
Use cy.addBlock() to add the block at the end of the page (ie the new block is the last one).
Use cy.switchToEdit(last) to make the newly added block active.
Use cy.fillACFField() repeatedly to populate all the fields required for the block.
When we’re done editing the page, we use cy.updatePost() to save our changes. We can then run a standard set of Cypress tests on the front-end, such as my simple example of checking the button text is as expected and that clicking the button actually works.
I hope this article gave you some ideas on how you can automate your WordPress testing. A good next step from here is to include these tests in your continuous integration workflow – we use Bitbucket Pipelines combined with Pantheon hosting to automatically spin up new environments per feature.
Do let me know if you want to hear about this, or if you have any other ideas on automating WordPress tests.
This is part 2 of a two-part series (part 1 is here)
In part 1 I looked at where to start when it comes to improving LCP for WordPress sites, and at some steps you can take to improve back-end performance. Part 2 looks at the front-end.
If you’ve covered the server-side and are happy with the performance there (or at least happy that the biggest issues are not there!) then it’s time to look at the front-end.
Remember LCP (Largest Contentful Paint) is defined as the time taken to render the largest block in the viewport. We can use Chrome Dev Tools to determine what that is in different scenarios. I suggest starting with the first fold of your homepage (or other significant landing page as determined by your analytics) as most people will land at the top of the page the first time they load it. But bear in mind that the largest element in the viewport could be part way down the page if the user lands there (eg by clicking an anchor link or if they have visited the page before and left at that scroll depth).
What to do
There are a lot of options on what to do next and I won’t be able to cover everything. But here are the most common issues I have come across:
If your LCP element contains an image then there are several things you can do to optimise the image’s load time (which I also recommend in general, not just for LCP!). The most important are:
Render images with the srcset attribute (wp_get_attachment_image will automate this for you) to ensure the browser does not have to load larger images than are necessary. It’s important you add the relevant image sizes to WordPress to generate the alternate sizes.
Never load the fullsize image, always a thumbnail.
Compress images as much as possible, and serve them in next-gen formats too where this is supported (most browsers do these days). We use the ShortPixel plugin to cover off on both of these.
Use vector formats for icons and logos where possible. These tend to be smaller files and scale to any size with no distortion.
If loading images as background images (eg style=”background-image: url(hero.jpg);”) this will delay the image load until all the stylesheets have been fully processed, thus delaying the paint of the element. Take a look at this thread for an explanation and some tips on fixing the problem, such as pre-loading the image in the <head> or repeating the image in an <img> tag. This is a particularly common problem for LCP, as we often use background images on hero elements.
If your LCP element contains text then it may be delayed by the time taken to load any custom fonts used by your website. Here are some things to look out for:
Make sure you are using font-display: swap; which will display text using the system default font and then swap it to the custom font once it becomes available.
Also make sure your fonts are not render blocking! This will impact the load time of everything on the page. Here are some techniques you can use to load fonts asynchronously.
Render Blocking Scripts
With WordPress it can be tricky to completely eliminate render blocking scripts, especially on an existing site that has plugins/themes which rely on these. In these cases I like to look into which plugins are generating scripts and determine if they are really necessary and if they are then try to re-enqueue the scripts in the footer checking that nothing breaks. You can also look for alternative plugins which do the same job in a more performant way.
If you are building the site from scratch I strongly suggest running regular Lighthouse scans and checking which scripts are loaded, especially after installing new plugins (we run Lighthouse in our Bitbucket Pipeline and fail the build if performance drops below standard). I also recommend dequeuing jQuery as this is not necessary in many cases and ends up slowing the site down.
I wanted to briefly mention design too. I encourage engineers to loop back with the site designer where they see issues that could be fixed with a UI change. For example, it may be more efficient to redesign a high-definition image grid rather than optimise it for performance. The cost / benefit trade-off is always important to keep in mind.
Let’s all keep performance centre of mind, it’s so important. What tips do you have for optimising LCP on WordPress?
In part 1 I look at what LCP is, why it’s important and then I explains how to measure LCP and steps to take on the back-end to improve it.
We all know that performance is a key success factor for most online businesses. Bounce rates increase exponentially with every extra second of perceived load time, and this can be tied back to lost revenue. Check out this article by Jeffrey Nolte if you need more convincing.
Google’s Core Web Vitals put most weight on a metric called LCP – Largest Contentful Paint. This is the time taken to render the largest block in the viewport. Google say a good score is 2.5s or less, and that at least 75% of your website’s visitors should experience an LCP within this range. So let’s talk about LCP…
Where to Start
Google PageSpeed Insights is a great place to start. It shows field data (gathered from real Chrome users) and lab data of the Core Web Vitals – LCP, FID and CLS.
Start by looking at the field data, as this gives you aggregated data showing what’s happening with your actual users (as opposed to lab data which is calculated from a one-off test and subject to fluctuations and bias). It’s a good idea to check the Origin Summary too, which gives you an average over all pages on the domain.
Take a look at LCP. If the green area to the left is less than 75% then we are below par and have some work to do!
Note that Google may not show field data if the site is new or just hasn’t had enough traffic for it to produce reliable numbers. In this case you will need to jump straight down to the lab results. Here we are looking for an LCP of 2.5s or lower on both mobile and desktop, if it’s higher then it’s likely there is problem. I advise running PageSpeed 2 or 3 times to make sure, as the results can fluctuate a bit from run to run.
In addition, you can use Google Analytics to identify particular pages which are slow and have significant traffic, and make decisions about which pages are important to your business (/shop probably is!). This is a nice way to decide where to start, as the issues may be different on each page. In Analytics, go to Behaviour > Site Speed > Speed Suggestions:
So your LCP needs some work? Let’s take a look at what we can do:
Before the browser can render anything it needs to receive the main page HTML document from the server. Often TTFB (Time To First Byte) is used to measure how long the wait is from when the browser requests the page to when the first byte of the response is returned. We want a TTFB below 500ms to be considered reasonable, and ideally below 100ms to be excellent.
The Network tab in Chrome Dev Tools will show you the TTFB if you hover over the main page’s bar in the Waterfall. Again, I advise running this 2-3 times to make sure you didn’t get an outlying result. In this example you can see we have some work to do:
We want to serve cached pages as much as possible, which removes the back-end processing time from the equation, getting the page to the user’s browser more quickly (and reducing load on the server, saving resources). Using the above example, we can click on the main page response (www.ekahau.com) to view the response headers. In this case the page is not cached and reloading it to eliminate the possibility that the cache wasn’t primed the first time didn’t make a difference:
The x-cache header is added by the CDN (we are hosting on Pantheon which includes Fastly CDN). We want to see at least one HIT to show that the page was served from the cache. I then tried using curl to see if something in the request header is causing this and sure enough:
$ curl -I https://www.ekahau.com/ | grep x-cache:
x-cache: HIT, MISS
There’s probably a cookie being sent by the browser which is causing the server to serve an uncached page. Sure enough, if I run the test in incognito mode I see that the page is served from cache. In these cases it is important to determine who will be affected by this problem, if it is only admin users logged into the WordPress dashboard then it’s probably not a big deal, but if it’s a large chunk of your audience you need to figure out what’s going on and fix it.
If none of your pages are ever being served form the cache, even in incognito mode or via curl, then there’s likely an issue on your server. If using a managed host like Pantheon you should contact them to see what’s going on, if you manage the server yourself then it’s on you.
Uncached Page Views
Whilst we want to serve cached pages as much as possible, some users will inevitably be served uncached pages. Caches expire and are primed on the next visit. Logged-in users or those POSTing data will also skip the cache in most cases. So the speed at which the server can build the page always matters. Remember we want a TTFB below 500ms to be considered acceptable, if you aren’t getting this:
The architecture provided by your host is the most important thing. However well optimised your site is, it won’t run well if your host isn’t up to the job. The WP Benchmark plugin can show you where you stand here.
One thing we’ve found with Pantheon is that their high-availability architecture has the trade-off of slow disk I/O speeds. This means that any site which has a large number of files in the codebase or includes plugins that write to the filesystem can run slow, eg WooCommerce sites often run into this problem. Whilst you may be able to optimise the code, the easiest option is to simply switch to a host that better meets your needs (is performance more important to you than availability?). A Digital Ocean server managed through Cloudways is one good option.
The Query Monitor plugin also provides some great insights into overall performance. You need to remember the total page load time and therefore the TTFB will be greater than the total query time. The plugin includes some tools to dig into queries and is a good place to go if you’ve ruled out your hosting as an issue.
If the page needs to handle potential spikes in traffic then it’s a good idea to stress test it by simulating large number of requests concentrated over a short period. You can use Loader for this.
This steps is optional in my opinion, it really depends on what you expect to happen during the lifetime of the site. If you will be running campaigns that may result in large spikes then I strongly suggest you run a stress test.
A new project, a new tech, a chance to build something better than I’ve ever built before. I’m sure all engineers have felt something like this when starting a new project. But then the constraints come along, and our enthusiasm is dashed. Unless we view these constraints as an opportunity for creativity! Read on…
One of my main responsibilities as a technology leader is to ensure we meet out clients’ goals without breaking their budget. This is tough, clients often have high expectations of what can be built in a given timeframe and budgets are usually tightly managed.
I don’t feel I have a perfect solution to this problem, but this article explains what I’ve learnt in almost 20 years in technology of which the past 8 have been spent in client services.
By the time we start thinking about technology, the client should have some idea of their brand. They generally know what problem they are trying to solve and have done some studies into the viability of their idea (if not we guide them through this process first). So let’s assume we know who the audience is and what problem they need to solve.
At this stage we want to bring in senior members of our team to help guide the conversation. Usually the client has big ideas about what their product will look like and we need to guide with some questions, like: what is actually a necessity for launch? what would a successful launch look like? We’re thinking along the lines of a Minimal Viable Product (ref The Lean Startup by Eric Reis). We emphasise that we won’t compromise on the quality of the user experience, in most cases we believe the risk to the brand of a subpar experience is too high and could damage it permanently (plus it doesn’t reflect well on us!). At this stage we’re talking about high-level features that are essential versus those that could be dropped, simplified or pushed down the line.
A simple technique we often use for this is the MoSCoW Method which groups features into buckets of Must Have, Should Have, Could Have and Won’t Have. We’ll work through this with the client, at the same time building up a better understanding of the constraints, assumptions and risks around each feature.
Once we know what an MVP could look like in terms of features we can start to map technologies onto them. I like to start with the core of the user experience, usually the web or mobile app. Throughout this process I’m trying to get the biggest bang for their buck – how can we build as quickly as possible a solution that meets the requirements, meets the budget and won’t cause a nightmare for our team in the near future? What are the short- to mid-term implications of these decisions?
We have some go-to technologies in our toolbox, such as WordPress, Laravel and the Google Cloud Platform, but I also like to look outside of these. We all know that technology changes faster than anyone can keep up with, and it’s important to consider new solutions, even for old problems.
I also like to keep this process as collaborative as possible, both internally within our team and with the client stakeholders. After all, our job as engineers is to solve business problems, and how can you solve a business problem without collaborating with experts in this business? Inevitably there are trade-offs to be made and by iterating in small steps we can gather feedback and switch course as we go, wasting less time going deep on solutions that won’t work.
This in itself is worthy of a deeper mention. Our clients have different levels of experience with technology, some come from similar agency or technical backgrounds or are used to working with agencies like us, and just get it (though this can have its own perils). Others do not, and there’s the whole spectrum in between. There is some judgement required based on our experience as to how much information a client needs to make a decision, but in all cases they need to understand the business impact of the options in concrete terms.
Once we have defined the high-level architecture, we start to go deeper into each of the must have features. The goal is to agree a budget (or “appetite”) for each. Recently I’ve been using a process adapted from the Principles of Shaping and Setting Boundaries chapters of Shape Up (by Ryan Singer).
Based on the knowledge I have already gained, I will define the following for each feature. I’ll go to Nolte team and client for questions as I go, but this is largely a solo exercise:
The Problem: What is the problem we are trying to solve? By clearly defining this we focus the rest of our work and set boundaries for the team as they work on the feature. Without this, it becomes hard to make decisions on whether certain elements of the feature are critical or not. But with this we can always ask the question: is this requirement necessary to solve the problem?
The Solution: Here I’m defining at a high-level what the proposed solution is. Generally this will link back to the high-level architecture (which components of the architecture come into play?) and will define the key parts of the solution without going into detail with wireframes or the like. The idea is to set boundaries without directing the team too firmly as this will stifle creativity further on.
Rabbit Holes: Identify any parts of the solution I see as risky, particularly where there is a risk of team going down a hole that over-complicates the work. A recent example is the implementation of recurring payments with Stripe, in this case I clarified that we won’t allow users to store multiple credit cards on file.
No-gos: Anything identified as definitely not included in the solution. For example, in the same product I stated in the order management feature that we will not handle payment failures in the system, there would be a simple notification to customer support who will handle them offline.
Appetite Range: Taking all this information into a client meeting and expecting them to make a decision on how much they want to spend on each item is unreasonable. Most of our clients don’t know how much things cost and can’t imagine what they may get by spending less or more on a given feature. So I set a range of high/low appetite (we use team days which translate to dollars) based on what I think is reasonable for the feature. I like to leave these ranges quite wide to provoke discussion around what they may get for the high price vs low, but at this stage we don’t make any commitments.
Armed with this data I setup a workshop with the client team to work through the list of must haves. Generally you will need half a day or so for this, or 2-3 shorter meetings. The aims of the workshop are twofold: agree an appetite for the feature and align on the information that I’ve written. I expect there to be some changes, new edge cases or otherwise new information that goes into the list, and often there are some follow-ups for me and/or the client to dig into. By the end of the workshop we should have agreed an appetite for 90% of the features and know what is needed to finalise the other 10%, which we either do via email or in one further call.
One really important goal of all this is to get the client onboard with the way we work. We don’t estimate the features upfront, I fundamentally don’t believe it is possible to do so accurately (is the phrase “accurate estimate” an oxymoron?). We all know that technology is complicated and the one thing we can be 100% sure of is that things will change (normally a lot) during the build.
This is why we use appetite and not estimates. We want the whole team aligned on the question: how can we get this feature done within the agreed appetite? It’s only possible if all parts of the team pull together: engineers, QA, design, client, etc. We want to foster creative thinking amongst us all, to get the best result for our client without breaking their budget or sucking up a massive overrun ourselves.
How does your agency approach this? I’d love to hear via twitter, @adamf321, or in the comments below.
It’s been almost a year and a half since the new WordPress editor “Gutenberg” was released officially as the default WordPress editor. However, before this release, we had the option to install Gutenberg as a plugin so we could test it and start getting familiar with it.
As any new major release, it introduced some challenges to WordPress editors and developers (mainly developers), besides some bugs and usability issues.
Unfortunately, it was not well received by some developers (myself included) who preferred to continue working with the Classic Editor, so a new plugin started catching on: The Classic Editor plugin, which at the time of writing this post, has more than 5 million of active installations. Fair enough for all those websites which were already using the good old Classic Editor. The Classic Editor plugin became a lifesaver for those websites.
By the way, this plugin is an official WordPress plugin, so it was released by the WordPress team precisely to avoid breaking those websites’ content with the new Gutenberg functionalities. They also announced:
This plugin will be fully supported and maintained until at least 2022, or as long as is necessary.
Once again, fair enough for all those sites using the Classic Editor, but what will happen after that date? What if the plugin developers decide to stop offering support to this plugin? Well, then it will be time to move away from the “good old times” and welcome the present.
Going back to “the rejection stage”, and talking about myself, I didn’t want to switch immediately to Gutenberg as soon as it was released, not for being afraid of change (well, actually I was afraid of that huge change)… anyway, I was aware there were many latent bugs and compatibility issues at the time it was released. In my opinion, the release of Gutenberg was too premature, I would have wait a bit for it to be more mature, more robust, and with fewer bugs.
Some of the most notorious issues when it was released were its lack of accessibility support, performance issues, some editor UX issues, lack of support on other plugins (like WooCommerce), etc.
In fact, Matt Mullenweg (who was leading the release process), in his presentation during the WordCamp US right after the launch of Gutenberg, did acknowledge that mistakes were made during the launch, and that he had learned the lesson so that these mistakes will hopefully not be repeated.
I heard (and read) many people saying that Gutenberg was a bad move and that it should never have been integrated to WordPress core as its default editor. But I think some of these feelings were also encouraged by personal reasons: fear of change, having to change the way to build custom themes, what would happen to the custom fields, the need to learn ReactJS to be able to build custom blocks, some people (like me) had already a defined workflow to build entire sites with Advanced Custom Fields and to offer a great edition experience with reusable fields, repeater fields, flexible content fields, etc. “Now, with Guteneberg, we’ll have to change everything we’ve built during these last years”, we said.
I know many people was really excited about Gutenberg, I actually was, but at the same time I felt worried about starting using it on my new projects, and more than excitement, the Gutenberg release caused anxiety and uncertainty to some (or most) of us. Instead of an improvement, I felt it would introduce a risk to my new projects, I really felt it like an Alpha release instead of a Production release.
And there are tons of additional reasons why many people felt this way and this is why I called this first chapter “Rejection”.
But what’s next? Sometimes after a rejection comes a less rejective or acceptance stage, but, do you think this was the case of my experience with Gutenberg? Judging by the title of this post, we could say yes, however accepting something not always means embracing it or adapting to it. If you want to know what happened after this rejection stage, and if I embraced it or just kept doing my job with my good old structured way to build websites, then be in the loop and wait for my next article: Chapter 2!
We’ve been building, supporting, and maintaining WordPress since when we were founded in 2006. Through those years we’ve explored nearly a dozen hosting providers such as GoDaddy, MediaTemple, Linode, WP Engine, AWS, SiteGround, and Flywheel to name a few. Given the fact that our clients rely on us to make the best technical decisions for them as possible it’s imperative we make the right ones. Pantheon has overdelivered on our expectations as a provider and have raised the bar on what a provider can and should deliver. While Pantheon offers many benefits I’ve listed what I feel are the most important points below.
From support to uptime, to performance we’ve been able to consistently rely on Pantheon for keeping both their product up and operational as well as the sites we host with them while also being thoughtful around how our account is managed.
99.9% Historical Uptime
Support has always been responsive, knowledgeable and has assisted to resolve issues on repeated occassions.
Account management has always assisted our team to help work through advanced technical issues and escalate when needed.
We rarely experience issues with their web portal which enables our team to work efficiently on client products.
A Superior Product
The Pantheon dashboard and site management product is far superior to what we’ve seen in the market. The tools in place help Nolte to work more efficiently resulting in less time managing DevOps and more time focusing on how we deliver value to our clients and the users of their products.
Structured and flexible development, testing, ad-hoc, and production environments
Custom Upstreams to allow for enforced standards, one click updates, and customization
Ability to automation and build on top of the platform through automation and APIs
Application Monitoring via New Relic
Git and version control on all environments
Site & WordPress Performance
Site performance is of the utmost importance to us as the majority of the products we manage have some level of business critically inherent to the sites. Given the fact that performance has a direct business impact and our mission is to help our clients grow this needs to be top of mind.
A Global CDN (Content Delivery Network)
Advanced Page Caching
Redis & Varnish Caching
SSD Underlying Infrastructure
Highly Performant PHP & Tuned MySQL Implementations
This benchmark has some great insights marking Pantheon #1 against many top hosts.
Our clients rely on us to keep their products, users, and businesses secure and Pantheon helps ensure that we meet their expectations.
Container based secure infrsastructure
Best data safeguard practices
Automated updates, security monitoring, backups and retention
Denial of service and network intrusion protection
Authentication best practices (SAML/SSO/2FA)
Secure code and database access
Secure data centers
Learn more about Pantheon’s security practices here
We’ve tested the reliability of Pantheon’s support team on several occassions and they’ve continued to meet or exceed our expectations. The support for agencies extends to training on how to use the tools, group virtual trainings and a host of other resources. The standard site support team has been awesome and provides expert level support through agents that really get WordPress and Pantheon’s product. We’ve saved thousands in resource costs from this point alone.
Some areas where the Pantheon suporrt has overdelivered:
Live training for Nolte’s engineering team
Co-creation of open-source tools that help us build better products faster
Occasional engineering support on products
Detailed technical instruction related to product optimization
Balancing agility with standardization is often a difficult task. In our experience with many of the previous partners we’ve found that they are usually good at one focus area of hosting such as content sites that scale or WooCommerce. These were difficult lessons but helped us to identify the right type of partner for the types of work we do. We’ve put Pantheon to the test with all types of WordPress sites such as:
Headless sites with multi-tenant front-ends
Highly trafficked WooCommerce / WordPress sites
Mobile applications powered by a WordPress JSON API
Have questions around how we’ve gotten the most out of the Pantheon platform or looking for more info on how to build a solution on top of the platform we’d love to hear more. Leave a comment or contact us.
We all know that data is a major success factor for modern businesses. Many of the systems we build for our clients and ourselves involve loading data from various sources, and it’s usually imperative that these loads produce complete and consistent datasets. After all incorrect data can be worse than no data at all.
In this article I’m going to talk about the key points I discuss with my team when we’re planning any data loading work. In my experience, any data load that doesn’t consider these factors has a high risk of running into issues at some point…
Batch vs Stream
First let’s define these terms:
Batch: a batch job runs periodically and processes a batch of data at once.
Stream: a stream “listens” to a data source and updates your dataset in (near) realtime.
The first question I ask is about the expectations for the end use(s) of this data. Is it expected to be updated in real time or would a periodic update work best? For example, P&L closing calculations for a bank would be expected each morning for the previous day’s trading, based on the previous day’s closing prices. However, the bank’s traders need to see prices in real time, looking at the previous day’s price or even prices which are stale by minutes is not going to allow them to make good decisions.
The next thing to consider is what is possible with the data source? Taking the above examples, meaningful P&L calculations have to be based on the marked price and number of other data sources, they cannot be calculated in real time. Stock prices are available in real time from the exchanges.
How do we ensure nothing is missed?
For batch processes that pull data from an API we need some way of knowing what has changed. Fortunately many APIs provide a way of querying with an updatedFrom parameter, so if we store the latest updated timestamp of the previous batch we can query for anything that’s been updated since. You could also do this based on the time the job runs, eg if you job runs daily at midnight you can pull in anything that was updated since the beginning of the previous day.
For batches that read data from a file produced by the source system, we are somewhat reliant on the source to provide the correct set of data in each file. However, there may be some “sense checks” we can run, such as checking the dates of the entries look right. It is our job to ensure that all files are processed, I recommend archiving processed files (eg by moving them to a different folder) or changing the filename (eg by adding .imported to the name) once all data from the file has been successfully loaded. This is also going to make your life easier if something goes wrong!
For stream jobs I advise using a robust pub/sub queue system, such as MQ, GCP Pub/Sub or Laravel Queues. There are lots of options here, the right one depends largely on the platform you are using and the volume of data you intend to process. The queue will ensure that each transaction is processed successfully or it will do something on failure (eg notify you).
The next thing to consider is that your data load must be re-runnable. This means that if it runs more than once for a particular item then it won’t cause a problem, such as a failure or duplicated data. This can happen for a number of reasons, eg a batch job fails part way through, some of the source data was found to be incorrect and needs to be reloaded manually or the data processor processed a queue item but failed before notifying the queue of the successful run (top tip: make sure you leave yourself an easy way to run your job manually, including ways to specify constraints such as the date range to process). Two broad approaches to this are:
Check first: it may be feasible to first check if an item exists before inserting or updating a record, eg by performing a lookup on the primary key. This is often necessary in transactional systems where duplicates will cause issues, but in say a data warehousing situation it may not be efficient to do this.
Deal with it: design your system in such a way that in can accept duplicate records. In the data warehousing example we can do this by adding a load timestamp to each record and building a view that expose the most recent version only (check out this example for BigQuery using window functions).
Logging & Alerting
Be kind to yourself. Do you want to spend your nights and weekends guessing what went wrong with your job? Do you want to be the person who had the whole company looking at incorrect data for 3 months due to a silent failure (I’ve been there 🙁 )?
As you are building your load job, you need to think hard about what information you’d like to have when you need to debug it. Data processing jobs can be notoriously hard to debug because they may be complex and often take significant amounts of time to run. So make sure you log what happened as much as is feasible. What files where loaded and when? How many records were loaded? What steps were completed? If you can save the data source for a period of time, eg by storing it in a cloud bucket object, this will help you too (next top tip: make sure you have data rotation policy setup, the last thing you want is for jobs to fail due a lack of disk space or for your cloud costs to keep going up and up forever).
All this logging is great, but what good is it if you are never told when something has gone wrong? You need to have an alerting system setup, which could be as simple as configuring your queue to say retry 3 times and then alert you. You may need to build in an email or other integration (for emails you should use a transactional email API like PostMark to ensure emails don’t end up in spam). The key here is to identify when a failure occurs, eg catch exceptions then log and alert or check for return codes of API calls and log/alert if they are not as expected. You can also setup alerts based on expectations about the data, eg is it ever expected that 0 records are loaded on any run?
The more you log (without breaking performance or logging sensitive date such as PII or passwords!) and alert the better. I can’t stress how important logging and alerting is. It’s the part I most often see people “leaving to the end” (and then never getting round to) and it’s almost inevitable that something will go wrong at some point, don’t be that person!
I wrote this post as I found myself having similar conversations with my engineering team every time we build something like this. With the few key pointers above you should be in good stead.
Do comment below or find me on twitter if you have any thoughts or questions.
According to best-selling business author Cal Newport and many other field experts, one of the biggest challenges in today’s society is staying focused. Every day, we have countless emails, Slack messages, Whatsapp, Jira tickets, tweets, notifications, the list goes on…
We need a to-do list to process our to-do lists; we need a break to process all of that information. There is a common misconception about multitasking: we think it’s efficient to do a lot of tasks at the same time; in reality, we are delivering ordinary or sub-par results. Working individuals need to go deeper to achieve notable outcomes, and deep concentration is the key to create new values and improve skills.
Before engaging with how to best perform “deep work” as a team, we spent about 60% of our time doing shallow and ordinary work. Now, we’ve examined the problems of shallow work, and created processes that encourage deeply focused, highly effective work here at Nolte. Read on to learn more about how to instill this focus within yourself or your team.
What is Shallow Work?
Shallow Work put simply, is distracted work. According to Evernote’s blog, we unlock our iPhones an average of 80 times daily and rack up more than 4.7 hours actively engaged with our mobile devices or instant messaging apps each day.
When we check our emails and instant messaging apps every minute, we experience a fear of missing out, or “FOMO” as the kids call it. This illustration of FOMO helps me and our team, remember to let go of the constant demands of being “plugged in” 24/7. Let go of the FOMO, and embrace experiencing the moment you’re in, which will allow you to work, play, and live more productively— plus, you’ll enjoy yourself more.
“Non Cognitively demanding, logical-style tasks, are often performed while distracted. Their efforts tend to no create much new value in the world because they are easy to replicate.” – Carl Newport.
Rules to Deep Work
From experience working at Nolte, I’ve come up with a set of rules to practice deep work when we’re doing our projects:
1. Use network communications wisely.
According to Newport’s book, if we stop an important task to attend to some other task, we could need about 20 minutes to recover our full concentration in the main task. If we stop every five minutes because we get a new push notification, email, or ping, we will produce only ordinary results for our customers, bringing back the shallow work practice.
When you are trying to create new value, and want to accomplish something or improve your results in your current job:
Avoid instant message applications
Put your phone in no-disturb/night/sleep mode, and
Disable distracting mobile phone notifications
Try to go deep with your full attention at least for 25 minutes with NO interruptions. Need some help to practice this? Try the Pomodoro Technique.
If you still can’t focus, then you can try something that has helped me with focus. It’s an innovative interactive game called Forest App. The less you use your phone, the more the app rewards you, through the visual tool of planting and growing trees in a forest. It’s a great starting point for people who find their phones addicting, and it’s helped me in my professional journey.
2. Avoid multitasking when possible
When you try to do two or more different things at the same time you are going to discover the fraud that is the multitasking approach; certainly, you can simulate “good” progress in your duties, but you are not delivering the best quality that you could deliver if you go deep. There are quite a few apps that give you the option to go full screen or offer you a “distraction-free” mode. Utilizing these tricks and tools, when available, is a great way to foster focused working, and great idea development.
At Nolte, we use Jira Boards to assign tasks for every person in the company. It’s not a surprise that the best practices of the Agile methodology suggest that you should only have one task in the “in progress” column because, think about it— our minds are not designed to multitask. We have might have two eyes and two hands, but we only have one brain to process what we are doing. By working on one thing at a time, we can create more opportunities for deep work as a team.
As Newport said, to remain valuable in our global economy, you must master the art of quickly learning complicated things. This task requires deep work. If you don’t cultivate this ability, you’re likely to fall behind as technology advances.
Digital tools are getting more complex day by day and it’s hard to follow their pace. However, this is something we must keep learning and practicing. Only then, we’ll be able to use new, advanced tools to their full potential. So keep learning, keep doing complex things, cultivate the power of concentration and you will be rewarded with new discoveries every day about your job and your capacities.
At Nolte, we trust in our team and its capabilities to choose the right time to go deep, and we enjoy the results of this process: delivering meaningful and unique products to our clients.
So, what is more important to you? Your team’s goals and aspirations, or the meme messaging on your Whatsapp group? Remember, the goal of deep focus is not to ignore every opportunity to communicate, but to choose the right time to do it. Give yourself time to do both, and you’ll find you need to do them simultaneously will disappear.