Manual Testing vs. Automated Testing: Which Best Suits Your Project?

Why are we discussing manual testing vs. automated testing? Good question. I’ll preface by saying this article will not discuss the general importance or necessity of testing. At SEP, it is an integral part of our development process. However, the methods and extent to which our client or we decide to test can vary greatly. You could write a book about the different testing tools, methods, and processes; others have undertaken this chore.

However, I have focused on a smaller scope and related some of my experiences with automation vs. manual testing. Both have pros and cons, and I will not focus on those here, although some of those attributes and deficiencies will be apparent.

When Automated Testing Doesn’t Deliver

My work on a recent project involved testing a medical application that communicated with an insulin pump and a continuous glucose monitor (CGM). The complexity of using automation to test an application communicating with remote devices is exceedingly difficult, if not impossible, and cost prohibitive (i.e. expensive!). Therefore, our client decided to take the approach of using manual testing.

Jira XRAY was selected as the testing tool. XRAY provides a format and environment for managing the tests. Tests are written in an “Action,” “Data,” and “Expected Results” format. For example:

Action:
Enter username and password.

Data:
Username: johndoe
Password: P@$$w0rd!

Expected Result:
The home page is displayed.

Once a series of these test steps have been written, the tests can be executed. XRAY permits the “tester” to systematically step through each test step, reporting a pass or fail condition. A project manager can oversee the test suite results (a group of tests) and see the test history.

Finding Project Success with Manual Testing Tool, Gherkin

Alternatively, another recent project utilized a manual testing tool called Gherkin. A mobile banking application that SEP developed for a client began with the intent to use automated testing. However, after about a month, it became apparent that automated testing would be time and cost prohibitive.

We discovered that objects on this mobile app were nearly impossible to identify or find the object path. So, we abandoned automated testing in favor of adopting manual testing, and we chose to use Gherkin as our testing tool.

Gherkin permitted us to return to the automated testing approach if the testing technology caught up with the development software. We never did return to automated testing, but the Gherkin served the project well. Gherkin is simply a “Given,” “When,” and “Then” conditional method of writing the tests.

Here is an example:

Scenario: Logging into Home screen
Given there is a Username entry
And there is a Password entry
When I enter “johndoe” as the Username
And I enter “P@$$w0rd!” as the password
Then the Home page “url” is displayed 

This tool works well for manual testing but is easily adapted to automated testing. For example, the “When” and “Then” conditions from above might look like this:

When (“I enter “{string} as the Username”), (username: string) => {
	cy.type (username);
And (“I enter “{string} as the Password”), (password: string) => {
	cy.type (password);
Then (“the Home page “{string} is displayed”), (url: string) => {
	cy.url (“url”);

The Advantages of Automated Testing in Web Applications

I am currently using automated testing on another financial tool in development for a client. For this project, we’re using Gherkin with the Cypress automated tool to test a web application and generated .pdf files.

The advantages of automated testing are numerous:

• faster run-time
• tests execution at any given time testing on multiple platforms
• reusable tests scripts
• data driven tests
• maximum coverage
• and more

For these reasons, the team chose to undertake automated testing.

Web applications tend to lend themselves to automated testing much more readily than a mobile app, from my experience.

Automated Testing for Mobile Still Has Obstacles

Although the technology for testing mobile applications is improving, there continue to be obstacles. I foresee that there will need to be massive technological advances for automated testing to be practical for mobile apps that are mutually dependent on other devices. I have enjoyed testing in both environments and look forward to new and interesting adventures in testing.

You might also like:

• Lessons Learned in Automated UI Testing
• Task 6: Map Out What Your Exploratory Testing Looks Like
• Powerful Visual Testing with Applitools
• How to set up Xray in Jira for Test Case Management