Integration tests with Sitecore ASP.NET SDK

The Sitecore ASP.NET Core SDK¹ is used for building headless applications with modern .NET - and Sitecore has really embraced the concepts and made a seamless SDK that honors the principles for modern Microsoft ASP.NET development. Model binding is “just” extending the standard ASP.NET binders and enabling Sitecore SDK is done through app registration of middlewares and is using dependency injection.

This also allows us to take benefit of the features from the ecosystem and features from Microsoft. One of the nice utils is that Microsoft provides a way to run integration tests that acts as if a request was made to the application, but running in-process so this can very easily fit in existing test tooling without the need for testcontainers or similar.

Sitecore ASP.NET SDK show the (initial) road

The SDK² also have several nice integration tests, howevere there is a little difference.

• The SDK is currently using the older and obsoleteTestserver, we will create integration tests with the newer WebApplicationFactory³.

• The integration tests in SDK switches the layout service to REST based. The consequence is that any customizations to GraphQL services will be skipped. We want to have an integration test that is as close to production site as possible and hereby also use GraphQL for layout requests.

Setup integration test with mocked layout response

To create an integration test with WebApplicationFactory it is “just” a test with your favorite test framework. I will in the following use xUnit, but it could be anything. WebApplicationFactory allows you to point to a Program implementation and hereby use all registrations of middlewares and dependency injection etc.

However, we want one difference compared to the live site, we want to “mock” the response from the GraphQL endpoint, “what will be rendered for this reponse layout response”. With WebApplicationFactory we can add additional service registration and configuration, and hereby configure the underlying HttpHandler:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19

internal class MyWebApplicationFactory : WebApplicationFactory<Program> {
    public MockHttpMessageHandler MockClientHandler { get; } = new();

    protected override void ConfigureWebHost(IWebHostBuilder builder)
    {
        ConfigureSitecoreLayoutResponseMocking(builder);
        base.ConfigureWebHost(builder);
    }

    private void ConfigureSitecoreLayoutResponseMocking(IWebHostBuilder builder)
    {
        builder.ConfigureServices(services =>
        {
            var handler = MockClientHandler;
            services.Configure<SitecoreGraphQlClientOptions>(cfg => cfg.HttpMessageHandler = handler);
            services.Configure<GraphQLHttpClientOptions>(cfg => cfg.HttpMessageHandler = handler);
        });
    }
}

Note: For this Configure task to work, we need one little change in the SDK, I have added a Pull Request⁴ so hopefully, this will soon be available for you.

The message handler is just a class that allows us to add responses:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10

[ExcludeFromCodeCoverage]
internal class MockHttpHandler : HttpMessageHandler
{
    public Queue<HttpResponseMessage> Responses { get; } = new();

    protected override Task<HttpResponseMessage> SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
    {
        return Task.FromResult(Responses.Dequeue());
    }
}

With this configured TestApplicationFactory we can write a test that make a request to the site and when the SDK requests the GraphQL endpoint, we can mock the response.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19

public class ErrorHandlingShould
{
    [Fact]
    public async Task ShowStaticPage_WhenServerErrorInProductionMode()
    {
        var factory = new MyWebApplicationFactory();
        factory.MockClientHandler.Responses.Enqueue(new HttpResponseMessage
        {
            StatusCode = System.Net.HttpStatusCode.BadGateway,
        });
        var client = factory.CreateClient();

        var response = await client.GetAsync("/", CancellationToken.None);
        var actual = await response.Content.ReadAsStringAsync();

        Assert.Equal(System.Net.HttpStatusCode.InternalServerError, response.StatusCode);
        Assert.Contains("This is a nice static 500 error page", actual);
    }
}

The client allows us to make request to our in-process webserver and hereby mimicing the clients/users of the website.

Here we are testing the actual error handling in the application acting as if Experience Edge is not working. We can even test how is the error handling working in different environments, eg. when using app.UseDeveloperExceptionPage() etc.

Mock actual GraphQL response

The really interesting part is the actual layout handling and mocking a positive scenario.

We need to return a valid GraphQL response that can be deserialized by the Sitecore SDK.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29

[Fact]
public async Task ShowEmptyPage_WhenLayoutResponseIsAvailable()
{
    var factory = new MyWebApplicationFactory();
    var layoutResponse = """
            {
                "data": { "layout": { "item": { "rendered": { 
                    "sitecore": { 
                        "context": { "language": "en" },
                        "route": { "fields": {}, "displayName": "My demo site 123" }
                    }
                } } } }
            }
            """;
    factory.MockClientHandler.Responses.Enqueue(new HttpResponseMessage
    {
        Content = new StringContent(
            layoutResponse,
            System.Text.Encoding.UTF8,
            "application/json")
    });
    var client = factory.CreateClient();

    var response = await client.GetAsync("/", CancellationToken.None);
    var actual = await response.Content.ReadAsStringAsync();

    Assert.Contains("<title>My demo site 123</title>", actual);
    Assert.Equal(System.Net.HttpStatusCode.OK, response.StatusCode);
}

Here we are testing that with this very simple layout response, the DefaultController can get the mapped Layout model and it is used to set the title of the page (that is the implementation in the xmcloud-start-dotnet⁵ - we know that in real life the title tag is more complicated than that…)

More complex layout responses, actual placeholders and components

The layout response is a bit complex and crafting those json responses by hand will easily become time-consuming and error prone. However, we can reuse the models from the SDK and serialize this to our mocked response.

Here is a very simple model bound view:

@model SampleComponentModel;
<div class="component">
    <h2 asp-for="@Model.Title"></h2>
</div>

We can test this with:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58

public async Task RenderTitle_WithRawLayoutResponse()
{
    var factory = new MyWebApplicationFactory();
    var layoutResponseContent = new SitecoreLayoutResponseContent
    {
        Sitecore = new()
        {
            Context = new()
            {
                Language = "en"
            },
            Route = new()
            {
                Placeholders = new()
                {
                    { "headless-main", [
                        new Component {
                            Name = "Sample",
                            Fields = new() {
                                { "title", new TextField("test title") }
                            }
                        }
                        ]
                    }
                }
            }
        }
    };
    var graphQlResponse = new
    {
        data = new
        {
            layout = new
            {
                item = new
                {
                    rendered = layoutResponseContent
                }
            }
        }
    };
    var layoutResponse = Encoding.UTF8.GetString(JsonSerializer.SerializeToUtf8Bytes(graphQlResponse, new JsonSerializerOptions().AddLayoutServiceDefaults()));

    factory.MockClientHandler.Responses.Enqueue(new HttpResponseMessage
    {
        Content = new StringContent(
            layoutResponse,
            Encoding.UTF8,
            "application/json")
    });
    var client = factory.CreateClient();

    var response = await client.GetAsync("/", CancellationToken.None);
    var actual = await response.Content.ReadAsStringAsync();

    Assert.Equal(System.Net.HttpStatusCode.OK, response.StatusCode);
    Assert.Contains("<h2>test title</h2>", actual);
}

The SitecoreLayoutResponseContent is just the layout, but with GraphQL we have the flexibility of requesting other details etc. but also the complexity, there is no free lunch.

The GraphQL to request layout is (as mentioned in Query Examples for XM Cloud):

1
2
3
4
5
6
7

query {
  layout(site: "experienceedge", routePath: "/", language: "en") {
    item {
      rendered
    }
  }
}

Hereby the GraphQL response will include the data.layout.item.rendered structure before the actual rendered layout response.

More in-depth validation of response html

Pure string comparison of a full html document is a bit tough and when something goes wrong, the error messages is often not very helpfull, so often it make sense to use a bit more clever parsing of the response document. Sitecore SDK is already using HtmlAgilityPack so in the following I will use that (it is actually only used for handling markup for editing image fields so it might be completely removed in the future, but you can of course still use that or an alternative library in your tests).

1
2
3
4
5

var html = new HtmlDocument();
html.LoadHtml(actual);

var lnk = html.CreateNavigator().Select("//a").OfType<HtmlNodeNavigator>().Single();
Assert.Equal("Go to blog", lnk.InnerXml);

As always, refactor to make your code (and tests) easily understandable

For sure there will be another developer working with code later on - or even ourself coming back weeks, months, years later remembering almost nothing about this… It is always important to make your code easily understandable.

Long blocks of raw json or nested object structures can get hard to read. I therefore prefer to create a few helper methods to create those structures in a more readable way.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19

var factory = new MyWebApplicationFactory();
factory.MockClientHandler.Responses.Enqueue(
    new LayoutResponseBuilder()
        .WithPlaceholder("headless-main", p =>
            p.WithComponent(c =>
            {
                c.Name = "SampleBtn";
                c.Fields = new()
                {
                    { "Title", new TextField("test title") },
                    { "Button link", new HyperLinkField(new () {
                        Href = "https://balle.dev",
                        Text = "Go to blog",
                        Title = "On my blog you can see more details"
                    }) }
                };
            })
        ).ToResponseMessage()
    );

Hey, what is the namespace…

All the code listed above including details such as helper methods, namespaces etc. - and a fully working example is available at my GitHub repository Sitecore-ASP.NET-SDK-IntegrationTest

But integration tests are so slow

The inner and outer feedback loop should be as fast as possible, if tests takes too long there is a high chance that developers will not run the tests and hereby potentially break, get obsolete, annoy instead of create value and being counter-productive. Similar if the tests are unreliable, failing due to reasons that are not related to the code then the tests are counter-productive.

With the integrated test server and using static layout responses, we can have fairly fast tests that are not relying on external resources but still test the full application stack.

While the pure unit-tests are here executed in 4-60 ms the integration tests are executed in the range 160-762 ms - on my laptop. The tests are fully isolated so they can be executed in parallel

Watch out

While the benefit is that the full application stack and pipeline is used, it is also the downside. Whenever adding any cross-cutting concern it can effect all of your integration tests.

Eg. when using multisites, a middleware is added and an initial request for site collection is performed. In normal usage this response will be cached but the integration tests are executed as a “cold start” so you need to handle this request. It could eg. be to give the actual site collection response in all tests, add it to the WebApplicationFactory factory so it is only done once - or register a new implementation of ISiteCollectionService in your tests to avoid the call.