At QBS Group we are promoting the use of Application Insights. We do that not because we are nerds and love new technology, but also because of simple home economics.

Before the introduction of Application Insights last year, we would often see long-running support tickets between our partners and Microsoft to troubleshoot problems with Business Central. Application Insights changed that almost overnight by giving partners and even customers the option to see where problems are caused.

Although we did not do any exact math, we feel it is safe to say partners can save hundreds, if not thousands of euro’s by using Application Insights.

What exactly does it cost and how do I monitor it?

But what does Application Insights cost per tenant or per ISV Solution in AppSource and do the economics still apply?

Running an Application Insights resource in Azure is not free of charge. The first 5GB per month is paid for by Microsoft with a retention period of 90 days. This sounds like it should suffice and in most cases it does. However, some of our partners reported to us that they had examples where they had much more data than that which resulted in a cost of tens, even hundreds of euros per month to host Application Insights.

Together with Microsoft, we investigated the issues, and we feel we should share these experiences with you.

Executive summary

As an executive summary, the conclusion is that having an Application Insights instance that starts charging you for a Business Central tenants telemetry is a warning of problems as such. An average tenant should not produce enough telemetry to cause the invoicing process in Azure to start.

So, what can trigger Application Insights to explode, what can you do about it and lastly, how can you avoid having to pay hundreds of euros for an overflow of telemetry cause by, for example, an App from AppSource with programming errors.

We investigated a few instances that we have access to which were large. A few tenant instances, an AppSource ISV instance and an Embed App instance.

How to analyze the cost

To analyze which events cause the problem we used this KQL query that Microsoft provided and that they shared on GitHub now too.

traces

| where timestamp > ago(30d)

| project eventId = tostring( customDimensions.eventId )

| summarize count() by eventId

For now, we will ignore AL000EAV because Microsoft confirmed it as a bug. I will get back to that at the end of the blog.

We need to focus on the other events, RT0004, RT0019, RT0009 and RT0020. This means (Pre) Open Company and Web Service calls.

In fact, RT0019 is called 160.000 times which is the equivalent of once per second. In other words, someone is polling something, and the solution is probably to implement an event driven system like Azure Event Grid. The culprit here was a partner customization.

Let’s look at another example.

A whopping 67 million counts into a Web Service from Business Central. In fact, this happened on a few days with the 30 days period generating 50GB of telemetry on one single day generating a 300+ euro invoice for this ISV.

In this case we narrowed the problem to Azure Blob Storage called millions of times to ping if a file exists.

What does this cost exactly?

Don’t kill your own system

The examples we found that were overflowing Application Insights can all be related to software that behaves in a polling mechanism used for interfaces leading to excessive amounts of web service calls both inbound and outbound.

We believe Business Central is not designed for this and with the Azure world, better options exist that are event-driven rather than timer driven. Logic Apps and Azure Event Grid are just two examples that we will share more examples of in the near future.

Avoid large bills

Lastly, I promised to explain how to avoid getting large invoices for Application Insights. Your customers may install an App from AppSource that results in a cost on your tenant without you knowing.

Also, let’s be honest, a bug by Microsoft may also cause a lot of records in your systems and it would be fair not to have to pay the same company for that.

The solution is to cap the amount of data and setup email alerts in a case large number of events are created in the system.

For a Business Central tenant, we recommend capping on 50MB per day and raising an alert by email if one event id is created more than 5000 times per day. Of course, there can be exceptions to this rule.

ISV’s can limit to 1GB per day to start and let the system grow over time if the number of (hopefully paying) customers grows. We recommend ISV’s to look at Application Insights on a daily bases and monitor delta’s rather than just large numbers. You can also raise alerts if you see a new tenant id coming in which probably means someone installed your solution from AppSource.

You can read how to set this up on Microsoft Docs or by registering to the QBS Academy Application Insights Class.

Useful links:

Microsoft  BCTech AppInsights

Microsoft BCTech AppInsights FAQ

A question that was raised more frequently than others lately is, how to use the Business Central API from C# in Service-to-Service scenarios?

Within the Business Central community, C# is the most frequently used language to code Azure Functions and Windows Services using DotNET core.

There are a few choices to make when deciding how to design your code and we’ll discuss the most important ones being security and structure of your classes.

Security

When you connect to Business Central using the API you have two choices. OAuth with tokens or Basic with a Web Service Access Key.

Using OAuth is the preferred solution which is promoted by Microsoft. The problem is that OAuth is only supported when the software calling into Business Central has the ability to provide a user interface.

This user interface is required to show a window to login and return the access token.

Business Central currently does not support OAuth in service-to-service scenarios except for the API that is handling the service tier itself.

That leaves you with the choice of going back to Basic authentication with a Web Service Access Key or to use Azure as the middle man.

Most examples that you can find online of authentication use the latter option and therefor this blog is about the other option that is hard to find information on.

Simplicity is the core value of Business Central and even though customers using Business Central have an Office 365 tenant, not everyone has their Azure tenant configured.

Using Azure as the middle man for security may be the easy way out for a developer, it requires each user of your solution to set this up in their Azure tenant.

The advice is to use Basic Authentication for Service-to-Service connections until Microsoft is finished with a final solution for OAuth in these scenarios.

Classes

There are two options when you design your solution to structure your classes. You can have them generated based on the metadata of the API endpoint or you can design your own classes.

Both options have pros and cons and the most important reason why the example repository creates its own classes is again simplicity and to explain clearly how the API works rather than working through a pile of generated code that will leave 95% unused.

In real life implementations, many developers will choose to generate the classes based on the endpoint, especially if the API has strong versioning like the one provided by Microsoft.

A strong exception would be if you only need a few API methods implemented or if the API is a custom API for a project and subject to change.

If changes to the API happen on a frequent basis, which is not recommended but happens often in real-life scenarios, you can be more flexible in your error handling if you define your classes yourself.

Rest Client

To send data back and forth your code needs a Rest Client and, in the example, we use the one from RestSharp. This is a very popular NuGet package. Another frequently used Rest Client is the Simple Client (Simple.OData.Client) which is also available as NuGet package.

Json Handling

Json is not native to C#. It was created for JavaScript and in order to use it in C# you can use either NewtonSoft or the new System.Text.Json from Microsoft. Both are created by the same person who works for Microsoft now.

You will most likely find out that both libraries have scenarios where they work and fail.

NewtonSoft Linq supports the use of JArray and JObject in a similar way as AL has the native JsonArray and JsonObject although you will find the NuGet libraries much more flexible and powerful.

Value

Each result that you get from Business Central has a wrapper with a tag called [Value]. In almost all cases you won’t need this in your code. The function ConvertToDataSet converts the result of an API call into a Json Array without the [Value] wrapper.

The API Endpoint

When Business Central is hosted by Microsoft the API endpoint has up to 6 variables that determine things like the Sandbox you are using (if any), the Tenant ID (obsoleted by OAuth) and the details about the API version, and group.

Learning more

To work effectively with the Business Central API it is important to have in-depth knowledge of Json and Rest. QBS Group offers a dedicated workshop to gain this knowledge.

Resources

Link to Github: https://github.com/qbsgroup/QBS.API.Example
https://www.kauffmann.nl/2021/01/07/deprecation-of-basic-auth-for-saas-has-been-postponed-to-2022/
https://www.kauffmann.nl/2020/09/14/service-to-service-authentication-for-automation-apis-in-business-central/
https://www.youtube.com/watch?v=dp_mzqzNvPA&t=1553s