How to Enable 100% Test Automation for Agile, DevOps, and Continuous Integration

SANTA CLARA, CA. - Tuesday July 30, 2019

Background

With the advent of Agile and DevOps, new releases that need to go through the testing cycle can come out weekly, or on an even more frequent basis.  Depending on the duration of the test cycle, multiple releases might be going through the cycle at the same time; increasing the complexity of defect tracing, debugging, and resources needed.

Continuous Integration/Continuous delivery tries to address these challenges by automating the whole cycle: source code submittal would automatically trigger a new release build, a new testing environment would be provisioned loaded with the new release, regression testing performed, releasing the test environment and sending out results automatically.

Challenge and Old Methodology

One major challenge for test automation is how to perform time-shift testing to test date & time sensitive logic for month-end, quarter-end, year-end processing, billing cycles, policy management cycles, regulation date changes, etc.  The test environment must be running at those specific trigger dates/times to confirm if the logic is correct. 

The traditional way is to change the system clocks.  This is typically a lengthy manual process and can take hours or days per time change while the tester stays idle waiting, normally testers don’t have the admin privileges to do this. 

Example

Let’s look at a a regression test for example.  For each time change, all users must first logoff from the SAP application server and the Oracle database server.  Then the SAP administrator needs to shutdown SAP.  After that, the Oracle administrator can shutdown Oracle.  This is followed by the system administrator who can reset the system clocks for the SAP application server and the Oracle database server.  Then the Oracle administrator can restart Oracle and lastly, the SAP administrator can restart the SAP application server.   Now imagine those administrators are all different personnel from three different organizations and each action requires a help desk change request. Since the order of operations must be followed, the next help desk request can’t be submitted until the previous one is finished.  Then imagine there are multiple time changes per regression test. Now you can see why one time change could take two days on average!  

This also makes the administrators’ job more challenging as they become the bottleneck for testing!  A lot of the time, they have to work during after-hours or weekends to perform the shutdown/restart and change the system clocks so that testing can start or continue.

The Time Machine Solution

A software virtual clock solution is the enabler to address time-shift testing and make test automation one hundred percent achievable.  With a software virtual clock, applications and database can see the virtual time without needing to change the system clocks.  As long as the virtual dates are jumping forward, there is no need to shut down and restart the application and database!  As a result, no administrators’ assistance is needed; testers can set their own virtual clock and continue testing without waiting for each time change.  Better yet, the whole thing can be fully automated by incorporating the virtual clock change into the test script itself.

There can be other challenges for testing automation and continuous integration.  For example, when a brand-new test environment is provisioned when a new release comes out, the time travel target needs to be able to be defined dynamically.  Also the license must be available, if someone needs to call the vendor to get a license key, then full automation is broken, as well.

There is a software virtual clock solution that exists today that can enable full testing automation to support Agile, DevOps, and Continuous Integration on premise or in the Cloud.  Please feel free to check out the solution here and some success stories at here.

About Time Machine

Time Machine saves ten-fold or more on resources and costs associated with temporal testing of applications. It enables you to finish your large scale software projects on time and under budget. Some other usage scenarios of Time Machine include Cloud migration, time zone adjustment, data aging, Big Data analysis and online training. Time Machine is the original date and time simulation software for Unix and Windows since 1997 and it is Cloud ready. Thousands of customers worldwide utilize Time Machine successfully across all industries on a daily basis; including 47 of the Fortune 100 companies.