Manage large volumes of data with asynchronous integration | OnActuate
 In A-How To

In this OneByte article, we dive into how to manage large volumes of data using asynchronous integration with representational state transfer (REST) API in Microsoft Dynamics 365. This approach is ideal when you don’t need real-time data updates and want to avoid system slowdowns. 

Why choose asynchronous integration?

When dealing with large data sets, synchronous integration can bog down your system by waiting for each transaction to complete. Asynchronous integration, on the other hand, allows you to process data in the background, keeping your system running smoothly. 

Two types of asynchronous integration

Microsoft Dynamics 365 offers two types of asynchronous application programming interface (API) integrations: 

  • Data Management Package API – covered in this article. 
  • Recurring Integration API – to be discussed in a follow-up article. 
How to Use Data Management Package API

The Data Management Package API allows for smooth data integration using data packages. It works with both cloud and on-premises deployments. 

Key Considerations: 

  • Scheduling: Handled outside of Dynamics 365. 
  • Format: Only supports data packages. 
  • Transformation: Managed externally. 
  • Protocols: Uses REST. 
  • Authorization: OAuth 2.0 for secure access. 
Authorization

The data management framework’s package API uses OAuth 2.0 to authorize access. The API must be called by using a valid OAuth access token.

manage large volumes data with asynchronous integration

Follow these steps to export the Data Management Package API

Create the export project in Dynamics 365 Finance & Operations (D365FO). 

 

ExportToPackage: Used to initiate the export of a data package 

GetExportedPackageUrl: Used to retrieve the URL of the data package that was exported 

The URL generated in the previous step will be used to download the data package. It contains the Manifest file, Package Header file and files for data entities in the project. 

Below is the execution history of the export project, showing the history or status that is maintained in D365 FO as a result of the above steps. 

manage large volumes data with asynchronous integration

Process flow diagram 

Follow these steps to import a Data Management Package API 

Create the import project in D365FO and download the data package.

Once the data package is downloaded, update the customer group file and vendor group file with new records. 

The download package 

 

GetAzureWriteUrl is used to get a writable blob URL. The method includes a shared access signature (SAS) token that is embedded in the URL. You can use this method to upload a data package to the Azure Blob storage container. 

manage large volumes data with asynchronous integration

Push Package: Used to push the package to the blob 

manage large volumes data with asynchronous integration

ImportFromPackage: Used to initiate an import from the data package that is uploaded to the blob storage 

Check the job history in the import project. 

manage large volumes data with asynchronous integration

Process flow diagram 

 

Best Practice Tip

Using asynchronous integration for large data volumes prevents system slowdowns and ensures smooth operation. It’s the recommended approach for non-real-time data processing. 

Explore more recommendations based on different scenarios to understand when to use synchronous and asynchronous integrations. 

About the expert
Chirag Sethi, Senior Consultant, OnActuate

Chirag is a D365F&Sc Microsoft Dynamics certified technical lead with 13 years of experience in application designing, development, integration of D365 F&Sc and D365 BC and NAV implementations. He has led teams in multiple E2E implementations in both D365 F&Sc and NAV. He has assisted businesses in addressing their requirements, providing solution/functional design documents, guiding development, testing, deployment teams in delivering final solutions.

Chirag’s certifications include the MB-700 Microsoft Exam for Microsoft Dynamics 365: Finance and Operations Solution Architect and MB-800 Microsoft exam for Microsoft Dynamics 365 Business Central Functional Consultants.

 The “OneByte” blog series are technical how-to articles by OnActuate’s expert consultants covering Microsoft Business Applications (Microsoft Dynamics, Power Platform) and related technologies. Have a topic you’d like to learn more about? Email us at info@onactuate.com! 

 

Our website uses cookies to personalize and enhance your experience and improve the delivery of information to you. By clicking "Accept", you agree to our use of cookies. Click “Learn more” to read our Privacy Policy

Policy violations D365 expense mobile appPolicy violations D365 expense mobile app