Dramatic, Chromatic and Co: The Newcomers on the (Test) Horizon

State of Testing 2020

While reading the new State of JS 2020 Report I noticed some newcomers in the field of testing, which I haven't heard about before.

To get an updated overview I have presented some (for me) new testing tools

Playwright 🎭

Playwright is for many fans of Puppeteer the new star in the field of automated testing.

But what makes this node-based testing framework so special?
At first glance, it looks a lot like Puppeteer (accordingly, it is easy to migrate from Puppeteer).
Playwright can therefore be understood as the successor to Puppeteer. The development team developed Pupeteer for Chrome back then and now the same team is working on Playwright for Microsoft.

However, there is a crucial difference between the two:
Playwright is crossbrowser compatible; that is, the framework supports Firefox and Webkit (Safari) in addition to Chrome, and thus represents a massive advantage over its (unofficial) predecessor.

Personal assessment

For fans of Puppeteer, Playwright is absolutely unavoidable. But also for newcomers the tool should be quite interesting, because it builds on the learned knowledge of the predecessor and exceeds it in many areas clearly in usabilty. Those who want to use Puppeteer should therefore rather rely on Playwright in the future.

Headless Recorder 📹⏺️

This browser extension for Chrome allows to record the current interactions with a website and generates code for Puppeteer/Playwright Test, which you can simply copy into a corresponding JS test file and it can be executed (if the required dependencies are given in the project, of course).

Especially for newcomers but also veterans who want to quickly write simple action inputs, the tool is a good help to generate the base code for the actual test logic.

One of my biggest concerns with such tools was and is that automatic code generation tools usually don't necessarily generate the kind of selectors you would use manually. Most of the time, the focus here is on readability and maintainability.
Best practice is to use data attributes like 'data-cy-*' or similar.
Theoretically, the extension also supports the preferential use of these data-attributes, but this did not work in my tests, so that the manual adjustment of the selectors may eat up more time than writing the code manually.

Personal assessment

Currently the tool is more gimmick than real timesaver for me, at least as long as the shortened selectors don't work properly.

But I think, if you use Playright or Puppeteer, you should keep an eye on this tool: With a little more development time, it could become a pretty good timesaver. In any case, the potential is already there.

Testing Library 🐙🧪

This DOM Testing Library tries to solve one of the main problems of testing: Unlike quite a few E2E testing frameworks, Testing Library eschews implementation details almost entirely.

In other words, instead of using a component with data from the real application, we populate it with mocked data for testing.
This has a significant advantage: our tests work even if our texts, data, etc. change.
Among other things, this allows components to be factored without having to adjust the tests.

The tool evolved from React Testing Library and now offers support for all frameworks like Angular, Vue and Svelte and also integrates with various testing libraries like Cypress, Jest, Puppeteer or Nightwatch.

Personal assessment

Since Testing Library can be used as a complement to such testing frameworks as just mentioned, I think you can definitely give the tool a try. Whether the concept works, however, probably has a lot to do with the project setup.
After all, the JS community seems to be very enthusiastic about it! 😉

Chromatic (Storybook) 📸📕

The last Testing Tool is a bonus in a way, since it only appears indirectly in the statistics as part of Storybook.

Chromatic is a classic visual regression testing tool, like Percy or BackstopJS.
However, with one big feature: Chromatic integrates directly with an existing Storybook project and thus automatically has the perfect use-case for Visual Regression Testing: **A (Storybook) UI patternlab as the basis of Visual Regression Testing.

This has immense advantages over testing the real application, first and foremost that the data is mocked out-of-the-box, meaning its content should not change regularly, which is often not something you can say about a live application.

Personal assessment

Chromatic and Storybook seem like the perfect team. Since Chromatic (like Percy) offers 5000 snapshots per month for free, there's little reason not to give it a try. And depending on the size of the application, subscribing to the SaaS solution can significantly improve quality assurance, especially since the tool can be integrated into CI pipelines so that it can easily become a permanent part of the workflow. So: anyone who uses Storybook should definitely give it a try!

Conclusion 🧐

A lot is happening on the testing horizon this year, or rather last year. This time the word "horizon" really fits a lot, because many tools appear for the first time in the JS annual report and directly achieve high popularity.
I find the latest developments in the field of quality assurance incredibly exciting and even if many of the new tools are still in their infancy, I think they are well worth a look.
As always, just try them out! What else do we have 'git' for 😉😋 ^^



Introduction to Playwright | QA Camp - YouTube

Headless Recorder - Chrome Web Store

checklyhq.com - headless recorder docs

Introduction | Testing Library

What Is React Testing Library? | LevelUpTuts - YouTube

Chromatic demo - YouTube