Skip to main content

End-to-End Testing with Sandboxes

Overview

This guide explains how you can set up end-to-end testing for your microservices running in Kubernetes using Signadot Sandboxes. First, there is a description of how sandboxes can help with E2E testing and following that, there are concrete details of how you can set up sandbox environments to run tests in an end-to-end environment.

Background

As development teams iterate on microservices independently, it is important to ensure that relationships between services are healthy with each change being made. End-to-end testing ensures this by exercising flows as an end user would use the application.

When such testing is deferred in the development lifecycle, many issues are discovered late causing long feedback loops and wasted development time in debugging and resolving issues. The goal of modern E2E testing as is for the other types of testing is to make high-quality feedback available to developers early in the development lifecycle as opposed to letting them be discovered by developers / QA teams after the fact.

How E2E Testing works with Sandboxes

Sandboxes provide a lightweight test environment that can be spun up quickly in the context of the real cluster and dependencies to run E2E tests.

Shown above is a simplified view of a microservices stack. In this example, we assume that the E2E testing scenario will be using a REST API that is exposed externally via the Kubernetes Ingress shown at the top, in order to exercise and verify changes elsewhere in the microservices stack. This type of test can give us high degree of confidence in the overall application behavior.

Unlike traditional environments which are expensive to set up, we can use Sandboxes to set up E2E test environments even for a single commit to a microservice in our stack without incurring a large infrastructure or maintenance cost. An E2E environment that enables testing each commit to svcA will look as follows:

This behavior is accomplished by making use of a combination of header propagation and sandbox routing to ensure that requests from the Sandbox Endpoint follow a path through the microservices stack that allows us to test the forked version of svcA.

The benefits of using this approach of sandboxes over other approaches for E2E testing are:

  1. Tests run a real environment that closely resembles production.
  2. Each sandbox environment spins up in seconds and at nominal infrastructure cost as it only deploys the minimal set of services that have changed which means critical E2E testing can be done before PRs are merged.

For testing more complex scenarios that involve management of stateful components like databases, message queues, etc, refer to the reference documentation on sandbox resources.

How to set up E2E Testing for a Microservice

This section details the actual methodology of building a sandbox specification for running E2E tests on each Pull Request. Taking the same scenario described in the previous section, the steps below explain the workflow for how you can build a sandbox specification, manually test it using Sandbox endpoints, and then automate this entire workflow within your CI/CD pipeline.

tip

If you have a sandbox specification already and are looking for a way to set it up in your pipeline for each pull request, refer to the CI Integration Guide.

Pre-requisites

Typically, the connected Kubernetes cluster would already be running the baseline version of services that are up-to-date with the master branch so that the latest versions of dependencies can be used by each Sandbox environment that is set up.

Writing a Sandbox Specification

info

Note that this requires that header propagation and sandbox routing are set up in your microservices stack, at least in all services that participate in the request path.

Continuing to use the above example, the first step is to write a sandbox specification that can be used to create a Sandbox for E2E testing. In this case, to test a change to svcA in an E2E environment, the simplest possible specification is shown below.

name: my-test-sandbox
spec:
cluster: my-cluster # name of cluster (as per app.signadot.com)
description: sandbox for running e2e tests when svcA has changed
forks:
- forkOf:
kind: Deployment # With respect to the baseline environment, this
namespace: default # forks (clones & customizes) svcA only into this sandbox.
name: svcA
customizations:
images: # this image will be applied to the forked version
- image: repo/image:abcdef # of svcA.
endpoints:
- name: my-endpoint # a host endpoint that points to the ingress url.
host: api.ingress.url
port: 443
protocol: https

Once you have configured the sandbox specification for svcA, you can test it by creating the above sandbox as specified on the Kubernetes cluster using the CLI.

% signadot sandbox apply -f my-test-sandbox.yaml

Created sandbox "my-test-sandbox" (routing key: ...) in cluster "...".

PREVIEW ENDPOINT TYPE URL
my-endpoint fork https://my-endpoint--my-test-sandbox.preview.signadot.com

Testing Sandbox Endpoints

Once you have created the sandbox and obtained the endpoint, you can exercise it manually to ensure that our configuration is correct. The endpoint URL that is returned above is authenticated and points to the forked version of svcA and can be used to run tests against it using an API key that you can retrieve from the Signadot Dashboard at https://app.signadot.com.

curl -H "signadot-api-key: ${SIGNADOT_API_KEY}" \
'https://my-endpoint--my-test-sandbox.preview.signadot.com'

Once your test is successful, to delete your sandbox, you can use the following command:

% signadot sandbox delete -f my-test-sandbox.yaml

Using your own Ingress Endpoints (Advanced)

In some cases, there may be specific ingress-based routing that may be difficult to realize via Sandbox Endpoints. This section explains how you can use your Ingress URLs for testing with Sandboxes in these cases.

A Sandbox Endpoint of type fork or host is responsible for authenticating and proxying requests into the cluster after setting certain context (headers) to each request that identifies the Sandbox, known as the routing key. This routing key is used by the routing layer within the cluster to ensure that the requests are automatically re-routed to the appropriate forked version of a workload if applicable.

In order to use your own Ingress URLs, you can pass this routing key as a header within the request. Different supported header formats are shown in supported headers and passing one of those headers is sufficient to end up with the same service-level routing behavior as Sandbox Endpoints.

Integrating with CI

Once you have verified that the Sandbox Endpoint tests the new workload as expected, you can use the sandbox specification you created above and set up CI automation such that a new environment is created automatically for each pull request commit.

Next Steps

This guide described a basic sandbox environment that can be set up to write and run E2E tests in a microservices stack. Building on these fundamental constructs, you can build sandbox environments that use sandbox resources and support ephemeral stateful components associated with a sandbox or fork multiple services within one sandbox to test more complex scenarios.