Selenium and Other Competitor Comparison

70% of test automation projects fail, we help you succeed

Most companies interested in test automation will purchase one of the traditional test-automation platforms or elect to use Selenium. Sadly. 70% of these test automation projects will fail within six months. ZenVerify solves the primary reason these projects fail. Plus, we give you a clear path forward to continuously improve the quality of your product. And, all without having to hire consultants, trainers, or more test automation experts.

How we solved the test automation Achilles Heel

The test automation projects have a tendency of becoming too expensive. Ironically, this happens because of the human component in QA automation.

With previous generation products, each test case becomes a silo with its own copy of widgets. If a single widget changes, you have to find and update all the test cases that use that widget. Pretty soon this becomes a very expensive process. This is why test automation experts advocate building a ‘test framework’ where widgets are referenced and not copied.

Building such Test frameworks is inherently a complex task and involves skilled human capital. This remains a manual process with current crop of QA Automation platforms. Soon enough, the QA Automation requires a full-fledged development team with its own architects, developers and testers. The cost starts to go up exponentially and that is when most automation projects are stopped.

With ZenVerify, we eliminate the need of manually building the Test Framework. As you record the test cases using our Resource Editor, we are building the Test Framework under the covers. When you edit an existing Test Case, we simply make changes within the Test Framework.

We had to rethink the test automation from ground up, and included several such innovative solutions within ZenVerify foundation. ZenVerify allows a QA Automation project to move at much greater speed, with higher accuracy, and requiring fewer and less skilled resources. This yields much better results while keeping the costs within budget.

10x Productivity Gain over Selenium

Selenium is Open Source, is “free”, and is considered Gold Standard for test automation. However, the TCO (Total Cost of Ownership) is very high as we explained in last section. A large number of very high skilled resources are required to use Selenium. Moreover, it is an API and one needs to build additional features to manage the Automation. ZenVerify not only fixes the test automation Achilles Heel, it also includes a number of usability features to fast track test automation. If we deploy the same number of resources with ZenVerify, we easily get 10x or more productivity gain.

5x Cost-Effective over Traditional Commercial Platforms

A number of products have attempted to solve the Test Automation for past several years. Some have been more successful than others. However, in our opinion none of the existing platforms address the fundamental issue. The so-called record-and-playback platforms start strong and the first few test cases are recorded easily. However, as soon as you start editing those test cases, you need to get down to scripting (a.k.a. coding). In fact, you encounter the same “silo” issues then fall down to manually building a “framework”. It is true that these platforms provide many features to record, execute and manage the test cases. However, these still require large number of high level of resource skillset. To achieve the same level of productivity, ZenVerify would be 5x more cost-effective.

Total Cost of Ownership Comparison

As you see from the below graph, the startup costs of Selenium are very low because it is “free”. However, these are not zero as you still need to hire a team to use Selenium. Then we the project grows, the resource costs keep on getting higher and higher.

The traditional commercial test automation platforms have very high startup cost because of licensing and resources, then the costs keep getting higher with time.

ZenVerify startup costs are about the same as Selenium because of lower resource needs. Then pretty soon, its TCO becomes even cheaper then Selenium.

Total Cost of Ownership - Cumulative (Typical Medium Size** Project)

NOTE: TCO includes the following:

  1. License: Required number of licenses will be different for each platform
  2. Training: Training costs are estimated
  3. Services: Various platforms require number of resources with different skillset levels to accomplish same goal

** 10-15 QA Professionals

Feature Comparison

FeaturesSeleniumTypical Commercial ProductZenVerify
Test Case Construction
Skillset RequiredHighHighLow
Construct 'test framework'ManualManualAutomated
Construct 'object repository'ManualManualAutomated
Effort to handle complex web pagesHighHighMedium
Test Case Execution
Skillset RequiredMediumMediumNone
Effort to create execution environmentsHighHighNone
Effort to execute on multiple environmentsMediumMediumLow
Organization
Concurrent support for multiple versionsComplexComplexOTB
Support for Test Plans or Test SuitesProgramaticOTB OTB
Reporting
“At-a-glance” reports for Project ManagersNoNoOTB
Execution path screenshotsProgramaticOTBOTB
Collaboration
Test assets sharing and collaborationComplex ComplexOTB
Test result sharing and collaborationComplexComplexOTB
Collaboration among global teamsComplexComplexOTB
Defect RecreationComplexComplexOTB
Integration
Connect with CI environmentsComplexOTBEasy
Connect with defect tracking systemsComplexOTBEasy
API based integrationsComplexComplexOTB
Database integrationProgramaticOTBOTB
Flexibility
Adding plugins to defect tracking systemsComplexComplexNoEasy
Adding plugins for CI ToolsComplexComplexNoEasy
Adding plugins for ALM ToolsComplexComplexNoEasy
Adding plugins for DatabasesComplexComplexNoEasy
Custom roles and privilegesComplexComplexNoEasy
Custom business rulesComplexComplexNoEasy
Integrating with specific business processesComplexComplexNoEasy