End-to-end (E2E) testing is gaining popularity within the WooCommerce community since the core development team released an E2E testing suite earlier this year. Essentially, the suite allows developers to test whether a store is working as expected and will play a significant role in ensuring changes don’t break anything.

E2E didn’t go unmentioned at WooConf recently. If you didn’t make it to the conference yourself, check out Brian Richards’ excellent talk Test Everything – or just set your money on fire.

In this post, I want to introduce you to E2E for WooCommerce – what it is, the core E2E test suite, and the benefits of running E2E tests.

What is End-to-End Testing?

End-to-end testing is a technique used to test whether the flow of an application right from start to finish is behaving as expected.

But how does this relate to WooCommerce? And what is the start or finish of your WooCommerce store?

An alternative way to think about E2E testing is “Automated Browser Testing”. It’s an approach for automating the use of customer (i.e. user) facing pages on a website using a web browser. The tests perform actions on those pages as a user and then verifies that those actions work as expected.

From a WooCommerce point of view, an example of a user could be a customer purchasing a product or a store owner creating a new product.

To help visualize what these tests might look like, check out the example below of some tests to verify that the “Add New Product” functionality is working as expected for a store.

What Are the Benefits of Running E2E Tests?

E2E testing (in the context of WooCommerce) provides a way for store owners and developers to verify that their stores are working properly without hiccups or errors. It reduces the need to manually perform repetitive, time consuming tests – a massive time saver!

The key benefits of E2E tests include:

• Increased confidence that your store is functioning as expected
• Decreased repetition – once written, the tests can be performed again and again
• Reduced errors/inconsistencies that occur with manual tests
• Saves you time (and money)
• Automation – depending on your chosen solution, tests may be scheduled or executed as part of continuous integration
• Increased testing will result in higher quality software/releases
• Depending on your chosen solution, it may allow you to test using a range of different browsers

How WooCommerce Uses E2E Tests

Since version 3.0 of core, the WooCommerce GitHub repository has included an E2E test suite.

The initial suite aimed to provide a “Core Testing Checklist”. Working through the checklist was a very manual process, so the ability to now automate this process is a big improvement, not to mention a time saver.

The tests can also be useful for extension developers (and store owners) to help verify that updates don’t break core WooCommerce functionality.

The WooCommerce developer team has published an article explaining exactly what its goals are and how to get set up. The post goes over how to contribute to the test suite and how you can add E2E tests to extensions.

The Current WooCommerce E2E Tests

The current suite doesn’t provide 100% coverage of the core checklist, but the end goal is to have E2E tests written for every item on their checklist.

The ones highlighted in green are the tests WooCommerce developers have written so far:

As you can see they’re already testing:

• Admin operations, like saving settings and creating coupons and products
• Basic cart flow, like adding / removing products from the cart and updating quantities
• Checking cart totals and subtotals; and
• They also include tests to test the entire checkout process is operating correctly

Setting up E2E Tests in WooCommerce

To set up the WooCommerce E2E test suite and run them on your local machine, you will need:

• A local copy of a WooCommerce store running WooCommerce v3.0 or greater – you’ll need the GitHub version of WooCommerce includes the test suite.
• The Twenty Twelve or Twenty Fifteen default WordPress themes or the Storefront theme.
• An import of the WooCommerce dummy data using the WordPress importer
• Install the required node modules within the WooCommerce directory
• Create a JSON file containing your E2E test configuration – a sample file is available in the `tests/e2e-tests/config/` directory)
• Have the Chrome browser installed

Detailed setup instructions can be found here.

Running WooCommerce Tests

Once set up, running the tests is as simple as running the Grunt e2e-tests command from within your WooCommerce plugin directory.

From here, you can sit back and relax while your computer does all the hard work.

The E2E tests use webdrivers to whip up an instance of a browser (be it Safari, Chrome, IE or even a headless browser like PhantomJS). The WooCommerce test suite uses the Chrome browser.

This is screencast of what running the WooCommerce E2E actually looks like. It provides a good example of how quick they run and also some of the actions they perform

As you can see, it starts off by testing some cart page operations.

Writing E2E Tests

One of the cool things WooCommerce has done is to make the tests extendable for developers.

The test suite is written in Chai JS. If you aren’t familiar with this language or need a refresher, there are some useful tutorials to get you started here.

These docs go over the basics, like setting up the tests and also cover how to write E2E tests for admin pages and front-end pages for WooCommerce.

The E2E Test Suite’s Limitations

The current E2E test suite is not without its limitations.

First off, as you no doubt gather from what we’ve covered so far, the setup isn’t easy. The majority of store managers I know would not be able to pull the latest WooCommerce files from GitHub, let alone be able to install node modules.

Another limitation is that the WooCommerce E2E tests utilize WooCommerce dummy products. Importing the dummy product isn’t an option for those who may not have a test site and can’t import the dummy products into a live store.

You also need to use a specific theme, i.e. Twenty Twelve, Twenty Fifteen or Storefront.

The tests aren’t automated on a recurring schedule and the errors that come back aren’t very helpful or readable, which makes it difficult to diagnose what caused the problem and how to fix it.

Still, despite these difficulties, the E2E tests in WooCommerce core are a fantastic start and are only going to get better. I’m super excited to see where the core team takes them.

Note: This post was adapted from a presentation Robot Ninja developer Matt Allan gave at the WooCommerce Vancouver meetup on 7 June, 2017 about E2E Testing with WooCommerce. If you’re interested in learning more about testing or would like someone on our team to give a presentation at your Meetup or WordCamp, get in touch.