rs-dispatch

Continuous Testing Vision (Triangulum Project - Service Virtualization)

Table of Contents

1. What Is Continuous Testing Vision (CTV)
   • Need for Shift Left testing approach
2. How Does CTV Help In Engineering Productivity
3. What Is a Test Double
   • What is a Stub
   • What is a Mock
   • Difference Between a Stub and a Mock
   • Demsifying the Internals of Stubbing and Mocking
   • What is Service Virtualization
4. Brief Comparision between Stubs Mocks and Virtual Services
5. Brief Comparision of various Mocking and Service Virtualization Tools
6. Choice of Tools for CTV SV Project Implementation
7. More insights on Broadcom DevTest
8. DevOps Pipeline as Code Implementation

What Is Continuous Testing Vision (CTV)

To embrace the massive organizational shift in releasing quality and reliable software to the end users at a faster pace embracing the full potential of agile methodologies, It is mandatory to build a strong Continuous Testing pipeline to achieve Shift Left testing.

Need for Shift Left testing approach:

Shift-left testing is important because it helps to prevent the following types of harm due to late testing:

• Testers may be less involved in initial planning, often resulting in insufficient resources being allocated to testing.
• Defects in requirements, architecture, and design remain undiscovered while significant effort is wasted implementing them.
• Debugging (including identifying, localizing, fixing, and regression testing defects) becomes harder as more software is produced and integrated.
• Encapsulation impedes white-box testing, reducing code coverage during testing.
• There is less time to fix defects found by testing, thereby increasing the likelihood that they will be postponed until later increments or versions of the system.

This creates a “bow wave” of technical debt that can sink projects if it grows too large and especially in the era of Micro Services.

Below image highlights our goal of building continuous testing with shift-left approach in our CI/CD pipelines.

📝 Note: The master and feature branch illustrated in the above image is for information purpose and doesn't bias towards any branching strategy

How Does CTV Help In Engineering Productivity

Continuous Testing Vision is all about integrating various testing methodologies in the Release pipeline to acheive the goals of

1. Decreasing the Lead Time for Changes (to One Day)
2. Increasing the Deployment Frequency (Multiple Deployments Per Day)
3. Minimizing Change failure rate (<0.15%)

What Is a Test Double

In automated testing usage of objects that look and behave like their production equivalents, but are actually simplified is a Test Double. This reduces complexity, allows to verify code independently from the rest of the system and sometimes it is even necessary to execute self validating tests at all. A Test Double is a generic term used for these objects.

Majorly used types of Test Doubles

• Stubs
• Mocks
• Virtual Services

What is a Stub

Stub provides hard-coded answers to calls done during the test. It’s an object, in most cases, responding only to what was programmed in for test purposes, nothing else. We can say that stub overrides methods and returns needed for test values. The purpose of a stub is to prepare a specific state of your system under the test.

Primarily, we use it wherever the test suite is uncomplicated, and having hard-coded values is not a problem. Additionally, both developers and testers use it. But we can not share it mainly for compatibility concerns occurring due to hard-coding of resources, deployment dependencies, and platforms.

Let us take an example of a game under development with Play, Exit, Up, and Down Arrow buttons. When game development is in progress, we have to verify user interfaces (Play, Exit, Up, and Down arrows), we shall implement some dummy actions. For example, on clicking on the Play button, a dummy action shall take place. Additionally, we can do the implementations as shown below:

What is a Mock

Mock is a part of your test that you have to set up with expectations. It’s an object pre-programmed with expectations about calls it’s expected to receive. The purpose of a mock is to make assertions about how the system will interact with the dependency. In other words, mock verify the interactions between objects. So, you don’t expect that mock return some value (like in the case of stub object), but to check that specific method was called.

Difference between a Stub and a Mock

A stub is an object that returns a hard-coded answer. So it represents a specific state of the real object. Mock, on the other hand, verifies if a specific method was called. It’s testing the behavior besides returning data to the question or call. The idea is stub returns hardcoded data to the question or call and mock verifies if the question or call was made with data response.

Demsifying the Internals of Stubbing and Mocking

In our example application, we have a class that reads a customer from the database and forms their full name.
Below is the code for the customer:

@Entity
	public class Customer {
		@Id
		@GeneratedValue(strategy = GenerationType.AUTO)
		  private long id;
		  private String firstName;
		  private String lastName;
		//...getters and setters redacted for brevity...
	}
 

and below is our business class:

	public class CustomerReader {
		@PersistenceContext
		  private EntityManager entityManager;
		  
    public String findFullName(Long customerID){
			   Customer customer = entityManager.find(Customer.class, customerID);
			   return customer.getFirstName() +" "+customer.getLastName();
		  }
	}

In the above example, Our business class reads the customer info from the database through EntityManager.

How to Test this Class ?

On the first thought, the solution would be to pre-fill a real database with customers and run this test against it. This is problematic for a lot of reasons. It creates a hard dependency on a running database, and also requires an extra step to create the test data. For this example, you could make it work, but in a real-world application, this would not be practical at all.

A better solution would be to use an in-memory database. This would solve the speed problem, but it would suffer from the extra step of pre-filling the database with custom test data. Again, in a real-world application, this can get quickly out of hand, especially when multiple tests also change the initial state of the database.

The best solution for a true unit test is to completely remove the database dependency. We will stub the database connection instead, and “fool” our class to think that it is talking to a real EntityManager, while in reality, the EntityManager is a Mockito stub. This way, we have complete control over what is returned by the database connection without having to deal with an actual database.
Basic Unit Test for the above code is:

public class CustomerReaderTest {

		@Test
		public void happyPathScenario(){
			 Customer sampleCustomer = new Customer();
			 sampleCustomer.setFirstName("Susan");
			 sampleCustomer.setLastName("Ivanova");
		
			 EntityManager entityManager = mock(EntityManager.class);
			 when(entityManager.find(Customer.class,1L)).thenReturn(sampleCustomer);
		
			 CustomerReader customerReader = new CustomerReader();
			 customerReader.setEntityManager(entityManager);
		
			 String fullName = customerReader.findFullName(1L);
			 assertEquals("Susan Ivanova",fullName);
		}
}

We are had coding the return values which our EntityManager is supposed to return "Susan Ivanova" , which have been otherwise retrieved from the connecting to the database. Now are able to test the class with out connnecting to the actual dependency.

Now, Below code is slightly improvised to ensure our class has multiple external dependencies which is the case with many real time applications today.

public class LateInvoiceNotifier {

		private final EmailSender emailSender;
		private final InvoiceStorage invoiceStorage;
	
		public LateInvoiceNotifier(final EmailSender emailSender, final InvoiceStorage invoiceStorage){
			this.emailSender = emailSender;
			this.invoiceStorage = invoiceStorage;
		}
	
		public void notifyIfLate(Customer customer)
		{
			if(invoiceStorage.hasOutstandingInvoice(customer)){
				emailSender.sendEmail(customer);
			}
		}
	}

The above class has two external dependencies, and we use constructor injection this time around. It checks for late invoices of customers and sends them an email if an invoice is late.

In a real system, the InvoiceStorage class is actually a web service that connects with an external legacy CRM system which is slow. A unit test could never make use of such a web service.

The EmailSender class is also an external system from a third-party mass email provider. So, we will stub it as well.

However, as soon as you try to write a unit test for this class, you will notice that nothing can really be asserted. The method that we want to test – notifyIfLate – is a void method that cannot return anything.

But when it comes to a system or class which has two or more external dependencies, thats when the Stubbing can't be a solution. Majority of enterprise grade applications are no so simple in nature and with the modularity with agility, stubbing can't be used for a component testing. Moreover hardcoding values for a request is only thing supported by stubbing. But when it comes to enterpsie grade applications,

• 📢 When the application becomes moderately complex and expects a variety of return values, Hardcoding each one of them becomes a nightmare and is unmanageable.
• 📢 When the class is actually a web service, A stub can never make use of such a webservice or evaluate the web service calls.
• 📢 Stubbing only does state testing and doesn't support interaction testing while mocking does.

What is Service Virtualization

Challenges with mocking are, Mocking functions focus on very specific scenarios and contexts and simulate behavioral responses to fulfill a certain development need. This can be very useful for isolating a specific component from the rest of the application and for performing unit tests. In general, mocking is a static implementation and state-less (responses do not have contextual information), and requires 100% manual configuration. Developers/testers must go through the arduous act of creating new interfaces for each use case. Mocks are not re-usable, not flexible and tough to automate. Mocking is not very robust, and ultimately a simplistic approach that can waste a lot of time.

Service Virtualization, on the other hand, can emulate the full implementation of the service and provides a more realistic, fully functioning simulated environment that is more robust than just mocking. For example, API virtualization provides you with the capability of quickly recording virtual API responses. These responses can then be used to create other scenarios by linking them to a dataset. Virtualization tools also allow developers to switch between a virtual environment and real asset, and test under a variety of conditions. Furthermore, Virtualization can help you do load tests, performance tests, failure tests, security tests, evaluate integrations, and more. Components can be used throughout production and testing environments, especially early in the software development lifecycle. Another benefit is that these tests can also be automated. ** Mocking simulates individual classes while service virtualization can simulate the entire network of services.**

In the micro service world with complex applications, Service Virtualization is the go to solution but comes with a cost, complexity and hard to find skilled resources

Brief Comparision between Stubs, Mocks and Virtual Services

	Response Source	Data Variety	Invocation Verification	Created By	State Perseverance
Stub	Hardcoded Data	Less	Same Process/Program	Developers	No
Mock	Data Set (JSON)	Medium	Same Process and HTTP	Developers	Yes
Virtual Service	Recorded Data	High	HTTP and Many other Protocols	Testers	Yes

	When to Use	When Not to Use	Support Protocols
Stub	When Test data needed is not complex	Not good for large suits of tests with complex data	Same Process/Program
Mock	Applicable in most of the cases	When the micro services are communicating over multiple protocols other than HTTP	Same Process and HTTP
Virtual Service	Large scale applications that have many APIs. Need support for protocols other than HTTP. To test non-functional requirements like latency, throughput and more	Agile teams in small to medium size. When the applications are loosely coupled and ideally modular	HTTP and Many other Protocols

Brief Comparision of various Mocking and Service Virtualization Tools

📝Below image illustrates the difference between a Stub, Mock and a Virtual Service

After a thorough research of various available open source and paid tools, considering the wider open source community, wider adaption, extended support for languages and the problems they solve besides compliance with multiple frameworks, the below list of tools are identified for implementation of Continuous Testing Vision

Mockito	WireMock	BlazeMeter Mock Services	DevTest
Freeware	Freeware	Licensed	Licensed
Mockito provides the mock implementation of the method/object	Provides simulation for HTTP based APIs	Mock Services only support HTTP Protocols	Full Fledged support for virtualizing almost any service via wide range of protocols
Mockito is designed Unit Testing	WireMock is designed for component testing and integration testing	Functional, Regression and Performance Testing	Functional, Regression and Performance Testing
Used in Development stage of SDLC	During early API Integration	When In-house and Third party systems are unavailable, TDM dependency and during load issues	When In-house and Third Party systems are unavailable, TDM dependency and during load issues.
Support for all Object Oriented Languages (Java, PHP) and Python, Flex, JavaScript, Scala, TypeScript	Supports HTTP based API calls	Supports HTTP(S) only as of now. BlazeMeter can integrate with Broadcom's DevTest	Supports 30+ Protocols

Choice of Tools for CTV SV Project Implementation

After a thorough research of the available mocking tools,

The choice of Mocking framework is Mockito for the following reasons:

• Simple, Clean API
• Large Community support
• Extensions solve some difficult edge cases
• Verification and exceptions are clean and flexible
• Ease of use
• Approximately 80% of the DevTeams who work with OOPS languages already use Mockito for their Unit Testing.
• Supported languages (native and through extensions): Java, Python, Flex, Javascript, Scala, Perl, Objective C, Perl, PHP, TypeScript

The choice of basic open source Service Virtualization tools is Wiremock for the following reasons:

• Has both fluent expressions (DSL) & configuration using Json
• Easy to write with extensive options of methods to perform,validate,extend a stub.
• They had given equal importance to configuration mocks in lines with fluent expressions.
• Documentation is quite good & most liked github project.
• Dockerized versions of WireMock is available to spwan up ephimeral containers

The choice of full fledged licensed Service Virtualization tool is Broadcom DevTest for the following reasons:

• Full stack API testing
• Component-level performance testing
• Codeless testing ability
• Integration with existing application development processes and test management tools
• Widest multiprotocl support for DevTest Available
• Supports dynamic strings and data support with random test data

More insights on Broadcom DevTest

With Broadcom DevTest, we can achieve

• Parallel development and testing in a DevOps environment
• Shift left to test early and test more in the software lifecycle
• Widest multi-protocol support available
• Simulation of observed behaviors, stateful transactions, and performance scenarios
• Realistic ways to emulate application testing
• Ease of administration

Development Testing Team Deployments

Easily store and launch test cases with CA Application Test as actionable assets alongside source code management (SCM), requirements management, build and issue tracking of test management tools.

Functional Testing Team Deployments

Run tests across the whole stack to validate APIs at every layer of a complex, multi-tiered application, allowing for complex workflows and the stateful behavior needed to achieve genuine reuse of test assets.

Virtualize anything and test anything

Out of the box, DevTest with CA Application Test and CA Service Virtualization features the broadest and deepest multi-data protocol support across front-end, middleware, and back-end technologies.

• Common web services protocols: HTTP, HTTPS, REST, SOAP, XML, JSON
• ESB/middleware protocols: WebSphere® MQ, WebSphere Native, Standard JMS, Tibco JMS, Rabbit MQ
• Mainframe protocols: CICS Link, CICS Transaction Gateway (CTG), IMS Connect, DRDA, Copybook
• ERP protocols: SAP—RFC/Jco, Idoc/Jco
• Database protocols: JDBC
• Financial protocols: SWIFT, EDI/X12
• Proprietary: TCP (Raw Socket), Java™, Scriptable (JSR-223 compliant), Request Manager, (Data-desensitizer)

Creation and Utilization of Virtual Services in DevTest

The DevTest portal enables users to create API tests quickly and easily—either by entering the data or by pasting it into the test request and response fields. This RR data can also be imported in bulk from one or more request-response file pairs, and then edited in the portal.

DevOps Pipeline as Code Implementation

🙋‍♂️🙋‍♂️🙋‍♂️