What’s the Business Central Universal Code Initiative?

For a while now, Microsoft has been emphasizing the move to the Cloud. We’ve had their ‘Cloud-first strategy’ and then more recently, the ‘Business Central Publisher Program’ targeting ISVs to convince and even force them to adopt the ‘Cloud-first strategy’. Microsoft forces ISVs to put their solutions on AppSource, even if those ISVs are only serving on-premises customers right now. They’ve even gone so far as to announce possible fees in the future if you won’t adopt the Cloud-first strategy. And now, with the Business Central Universal Code initiative, they are targeting everyone to adopt this strategy.

Basically, with this new initiative, selling extensions not targeted for online use to new Business Central on-premises customers could require you to license two additional modules:

• Module “Implemented code is not in extensions”
  When the Microsoft Business Central base application is modified, customers will need to license this module.
• Module “Implemented code is not cloud-optimized”
  When code is developed in extensions, but they are not cloud-optimized, customers will need to license this module

The cost of these modules (read: fees) will start at $0 per full user (both Essentials and Premium) per year and will increase over time. Here is the timeline Microsoft is currently aiming for.

And what about those increasing fees?

Recurring fee(s) Timeline	Implemented code is not in extensions	Implemented code is not cloud-optimized
2022	$0	$0
2023	$75	$0
2024	$125	$75
2025	$250	$175
2026 onwards	TBD	TBD

Note: Customers who licensed Dynamics NAV or Dynamics 365 Business Central on-premises before April 1, 2022 will not be required to license the non-Universal Code modules in their current configuration. They can add, remove users and first- and third-party functionality without the need to license previous mentioned new modules.

More information about Business Central Universal Code

What’s Microsoft’s Goal?

To put it very simply, Microsoft would like to create a Business Central SaaS world where every (potential) customer can get any (ISV) solution they need to be able to run their organization on Business Central SaaS. Therefore, Microsoft wants you to create solutions that will work in Business Central SaaS. It’s the future for Microsoft Partners and customers in the Dynamics 365 (Business Central) community, and Microsoft wants to see everyone collaborating on this. Microsoft tries to motivate us by charging fees to those who do not cooperate.

The benefits of the Universal Code initiative, according to Microsoft:

• Universal Code can be implemented everywhere, in Business Central online, in on-premises or Partner-hosted environments.
• By listing your Universal Code on Microsoft AppSource, you extend your sales force, reach more customers, and connect directly with decision-makers.
• By bundling your connected Microsoft cloud services with Universal Code, you generate customer stickiness.
• Universal Code frees up capacity to improve our mutual offerings and we will be able to serve more customers.

Let’s Get Technical about Universal Code

From Q4 2022 onwards, technical enforcement will be introduced in Business Central so that only ‘Universal Code’ is allowed as additional code in customer environments. You need to move to full extensions only, meaning no more code customizations in Base App. That means in your app.json you should use: “target” : “Cloud”, so your code will work both in SaaS and in on-premises environments.

To make your solutions compatible with Business Central SaaS you need to meet the requirements of SaaS, such as:

• No usage of .Net Interop (please think about Azure Functions)
• No file system (please think about Azure Blob Storage, Cosmos etc.)
• No direct access to SQL
• Web client only
• No Base App Source Code modifications
• Lack of events
• Printing

This means you not only have to think about moving your code to AL, but also about how to overcome the limitations or requirements of Business Central SaaS. You really need to think outside the box and see the opportunity offered by the whole Microsoft Platform instead of just focusing on the Business Central app itself.

You really need to have a look at Azure (Functions, Storage, SQL, Apps, etc.) which offers multiple solutions in the stack to help you meet the requirements of Business Central SaaS. And yes, that requires other skills you probably don’t have in your organization yet.

Conclusion, Impact and Calls to Action

So, are you shocked, or do you need to make use of this initiative? Let’s hope not, because that would mean you have missed all the Microsoft news in the last 3 or 4 years.

Microsoft has shared their vision for the direction of Business Central SaaS.  It should be the easiest-to-use business application suite, delightful, efficient, “video-game” fast, powerful, simple to discover and quick to onboard. All of this should be available for every (potential) Business Central customer, and they want you to collaborate on that by making your solutions available for SaaS (too).

Hopefully you are already there, or at least almost there, but if not, please rethink your organization’s approach or even the right to exist in the next few years. If you really would like to embrace the cloud you need to think outside the ERP box. It’s not a single solution, but just one part of the platform you offer to your customers.

• Rethink your solution: Get rid of legacy customizations that are included in Business Central nowadays. Think about Buy or Build. If something already exists on AppSource, why are you going to invest in creating a similar app?
• Embrace Azure: Have a look at your solutions through the eyes of an Azure Solutions Architect. Train your team in Azure (Blob, SQL, Logic Apps, Functions and Cosmos to start with).
• Pick your battles: Use Horizontal ISV Solutions. Don’t develop something which already exists. Team up with other Partners. Outsource development and maintenance of your apps.
• Search for Value: Only invest in solutions where you are really adding value to Business Central and your vertical. Have a look at the QBS Marketplace. Tell your PSM that you would like to have a chat with one of our Architects.

 

I would like to close with a quote of Marije Brummel (Azure Solutions Architect);

“No one needs to be left behind!”

One of the highest-scoring FAQs from partners and customers moving to Business Central is about direct access to the SQL Server database in combination with the maximum database size.

When all you have is a hammer, everything looks like a nail. Since Dynamics NAV in the last ten years only ran on SQL Server this is what most partners know. Large customers often buy CPU licenses and since they have a SQL Server this is what they often start using as a Data Warehouse too.

The cloud equivalent of SQL Server is Azure SQL and this is fully supported by Business Central. In fact, this is what Microsoft uses for hosting the SaaS platform.

One might be tempted to use Azure SQL then also for cloud data storage, but this may not always be the best choice. On Azure, we have more than just a hammer.

Azure Cosmos Database is a good alternative in Azure if you are looking to store large amounts of data from different sources that you want to use in a Web Shop or other Web-Based solution.

The question that rises is then, what would this cost, and how do I send information back and forth between Cosmos and Business Central.

The pricing model for Cosmos is different from Azure SQL in a sense that you pay for real throughput and storage. The storage model is cheaper than the model for Azure SQL.

Cosmos is also auto-scale and auto indexed even though I expect that most of the workload from SMB customers will not be affected by this a lot.

Cosmos vs. Azure Data Lake

In this article we discuss Cosmos as an alternative to Azure SQL. Cosmos is not the only alternative and in the next articles, we will also discuss Azure Data Lake and other blob-based solutions.

Working with Cosmos & Business Central

As you may expect from my profile the main topic of this blog is the technology required to get data to move from Business Central to Cosmos. For this there are two options and I’ll describe both and share code examples.

In both cases we are going to use the Cosmos Rest API.

The AL way

The first option and probably the option of choice by experienced AL developers is to call the API directly from Business Central. This is a straightforward HTTP request where we send json information to a Cosmos container. The thing that may require to fiddle with a bit is the security tokens, but this has been laid out in the example code. You can find code examples on our GitHub.

The C# way

Another option which will most likely be the preferred option by developers who are new to our community is to call the Business Central REST API from an Azure Function and push that data into Cosmos using some C# code. This is also part of the examples you can find on GitHub.

Best of both worlds

Both options can also be combined. This depends on how you want to schedule the synchronization. Scheduling can be done using a Logic App that synchronizes data every x minutes, or hours, or you can push the Azure Function from the Business Central Job Queue. Alternatively, you can even call the Azure Event Grid from a Business Central Webhook and use that to transform data from Business Central to Cosmos.

The Azure Function also allows data to be stored from other data sources such as a SOAP endpoint.

Pitfalls and tips

You can store all kinds of data in Cosmos, based on containers just like Azure Blob Storage.

Cosmos is a no SQL database so it does not force you to follow schemas.

The consequence of the lack of schema’s is that it becomes the responsibility of the owner of the database to make sure data in a container is consistent and all json files follow the same structure.

As an example, let’s assume you have created a container with Item Inventory by date and you want to add location code as a new field. You need to update old records with the location code or create a new container with fresh data.

The example code provided on the GitHub contains a Version No. field and some extra Metadata about the json stored in the container. This allows you to have multiple versions of a record in the same container and store, for example, items and resources in the same container. This can have an advantage if you need an aggregation across Business Central entities in one container.

Because throughput is the main driver for cost in Cosmos it can also be beneficial to duplicate data across containers if there is a clear difference in data requirements across certain BI dashboards. For example, certain dashboards only require a customer name, where other dashboards require post codes or city.

Since data is stored in json format you can store parent and child information in one container, such as Sales Headers and Lines.

Creating a new Cosmos Instance & Database

One of the reasons Cosmos can be interesting for the SMB market and Business Central users is that it provides a free entry level of 25GB and 1000ru. This should suffice for an average Datawarehouse combining data from Business Central and other entities.

API

You can choose any API you are familiar with, but the examples on our GitHub are based on Core SQL. This syntax is relatively close to Transact SQL that most Business Central power users are familiar with.

 

Serverless or Provisioned

Probably serverless since we are having very incremental workloads. If we don’t use Cosmos, we only pay for storage.

Account Name

The account name is a generic name, used for all databases, just like an Azure SQL Server.

Other options may be left default or may vary based on your organizational backup and security policies and are not part of the scope of this document.

Creating a Database and a Container

In our code examples, the Database and Container are created by the Azure Function in C# if you try to write to a Database and/or Container that does not exist. The AL code does not contain examples of creating a Database or Container automatically. If you go for AL only you need to either write this code or create them manually via the Azure Portal.

 

GitHub
https://github.com/qbsgroup/QBS.BusinessCentral.Cosmos
https://github.com/qbsgroup/QBS.AzureFunction.Cosmos

 

At QBS Group we are promoting the use of Application Insights. We do that not because we are nerds and love new technology, but also because of simple home economics.

Before the introduction of Application Insights last year, we would often see long-running support tickets between our partners and Microsoft to troubleshoot problems with Business Central. Application Insights changed that almost overnight by giving partners and even customers the option to see where problems are caused.

Although we did not do any exact math, we feel it is safe to say partners can save hundreds, if not thousands of euro’s by using Application Insights.

What exactly does it cost and how do I monitor it?

But what does Application Insights cost per tenant or per ISV Solution in AppSource and do the economics still apply?

Running an Application Insights resource in Azure is not free of charge. The first 5GB per month is paid for by Microsoft with a retention period of 90 days. This sounds like it should suffice and in most cases it does. However, some of our partners reported to us that they had examples where they had much more data than that which resulted in a cost of tens, even hundreds of euros per month to host Application Insights.

Together with Microsoft, we investigated the issues, and we feel we should share these experiences with you.

Executive summary

As an executive summary, the conclusion is that having an Application Insights instance that starts charging you for a Business Central tenants telemetry is a warning of problems as such. An average tenant should not produce enough telemetry to cause the invoicing process in Azure to start.

So, what can trigger Application Insights to explode, what can you do about it and lastly, how can you avoid having to pay hundreds of euros for an overflow of telemetry cause by, for example, an App from AppSource with programming errors.

We investigated a few instances that we have access to which were large. A few tenant instances, an AppSource ISV instance and an Embed App instance.

How to analyze the cost

To analyze which events cause the problem we used this KQL query that Microsoft provided and that they shared on GitHub now too.

traces

| where timestamp > ago(30d)

| project eventId = tostring( customDimensions.eventId )

| summarize count() by eventId

For now, we will ignore AL000EAV because Microsoft confirmed it as a bug. I will get back to that at the end of the blog.

We need to focus on the other events, RT0004, RT0019, RT0009 and RT0020. This means (Pre) Open Company and Web Service calls.

In fact, RT0019 is called 160.000 times which is the equivalent of once per second. In other words, someone is polling something, and the solution is probably to implement an event driven system like Azure Event Grid. The culprit here was a partner customization.

Let’s look at another example.

A whopping 67 million counts into a Web Service from Business Central. In fact, this happened on a few days with the 30 days period generating 50GB of telemetry on one single day generating a 300+ euro invoice for this ISV.

In this case we narrowed the problem to Azure Blob Storage called millions of times to ping if a file exists.

What does this cost exactly?

Don’t kill your own system

The examples we found that were overflowing Application Insights can all be related to software that behaves in a polling mechanism used for interfaces leading to excessive amounts of web service calls both inbound and outbound.

We believe Business Central is not designed for this and with the Azure world, better options exist that are event-driven rather than timer driven. Logic Apps and Azure Event Grid are just two examples that we will share more examples of in the near future.

Avoid large bills

Lastly, I promised to explain how to avoid getting large invoices for Application Insights. Your customers may install an App from AppSource that results in a cost on your tenant without you knowing.

Also, let’s be honest, a bug by Microsoft may also cause a lot of records in your systems and it would be fair not to have to pay the same company for that.

The solution is to cap the amount of data and setup email alerts in a case large number of events are created in the system.

For a Business Central tenant, we recommend capping on 50MB per day and raising an alert by email if one event id is created more than 5000 times per day. Of course, there can be exceptions to this rule.

ISV’s can limit to 1GB per day to start and let the system grow over time if the number of (hopefully paying) customers grows. We recommend ISV’s to look at Application Insights on a daily bases and monitor delta’s rather than just large numbers. You can also raise alerts if you see a new tenant id coming in which probably means someone installed your solution from AppSource.

You can read how to set this up on Microsoft Docs or by registering to the QBS Academy Application Insights Class.

Useful links:

Microsoft  BCTech AppInsights

Microsoft BCTech AppInsights FAQ

A question that was raised more frequently than others lately is, how to use the Business Central API from C# in Service-to-Service scenarios?

Within the Business Central community, C# is the most frequently used language to code Azure Functions and Windows Services using DotNET core.

There are a few choices to make when deciding how to design your code and we’ll discuss the most important ones being security and structure of your classes.

Security

When you connect to Business Central using the API you have two choices. OAuth with tokens or Basic with a Web Service Access Key.

Using OAuth is the preferred solution which is promoted by Microsoft. The problem is that OAuth is only supported when the software calling into Business Central has the ability to provide a user interface.

This user interface is required to show a window to login and return the access token.

Business Central currently does not support OAuth in service-to-service scenarios except for the API that is handling the service tier itself.

That leaves you with the choice of going back to Basic authentication with a Web Service Access Key or to use Azure as the middle man.

Most examples that you can find online of authentication use the latter option and therefor this blog is about the other option that is hard to find information on.

Simplicity is the core value of Business Central and even though customers using Business Central have an Office 365 tenant, not everyone has their Azure tenant configured.

Using Azure as the middle man for security may be the easy way out for a developer, it requires each user of your solution to set this up in their Azure tenant.

The advice is to use Basic Authentication for Service-to-Service connections until Microsoft is finished with a final solution for OAuth in these scenarios.

Classes

There are two options when you design your solution to structure your classes. You can have them generated based on the metadata of the API endpoint or you can design your own classes.

Both options have pros and cons and the most important reason why the example repository creates its own classes is again simplicity and to explain clearly how the API works rather than working through a pile of generated code that will leave 95% unused.

In real life implementations, many developers will choose to generate the classes based on the endpoint, especially if the API has strong versioning like the one provided by Microsoft.

A strong exception would be if you only need a few API methods implemented or if the API is a custom API for a project and subject to change.

If changes to the API happen on a frequent basis, which is not recommended but happens often in real-life scenarios, you can be more flexible in your error handling if you define your classes yourself.

Rest Client

To send data back and forth your code needs a Rest Client and, in the example, we use the one from RestSharp. This is a very popular NuGet package. Another frequently used Rest Client is the Simple Client (Simple.OData.Client) which is also available as NuGet package.

Json Handling

Json is not native to C#. It was created for JavaScript and in order to use it in C# you can use either NewtonSoft or the new System.Text.Json from Microsoft. Both are created by the same person who works for Microsoft now.

You will most likely find out that both libraries have scenarios where they work and fail.

NewtonSoft Linq supports the use of JArray and JObject in a similar way as AL has the native JsonArray and JsonObject although you will find the NuGet libraries much more flexible and powerful.

Value

Each result that you get from Business Central has a wrapper with a tag called [Value]. In almost all cases you won’t need this in your code. The function ConvertToDataSet converts the result of an API call into a Json Array without the [Value] wrapper.

The API Endpoint

When Business Central is hosted by Microsoft the API endpoint has up to 6 variables that determine things like the Sandbox you are using (if any), the Tenant ID (obsoleted by OAuth) and the details about the API version, and group.

Learning more

To work effectively with the Business Central API it is important to have in-depth knowledge of Json and Rest. QBS Group offers a dedicated workshop to gain this knowledge.

Resources

Link to Github: https://github.com/qbsgroup/QBS.API.Example
https://www.kauffmann.nl/2021/01/07/deprecation-of-basic-auth-for-saas-has-been-postponed-to-2022/
https://www.kauffmann.nl/2020/09/14/service-to-service-authentication-for-automation-apis-in-business-central/
https://www.youtube.com/watch?v=dp_mzqzNvPA&t=1553s