Difference between revisions of "Data Import Service"

Revision as of 15:11, 22 December 2020

The Data Import Service is a interface consisting of a web service, hosted on an IIS service, which takes data pushed to Ubik and imports it efficiently. The heart of the service is a shared module called the Import Mechanism, which can be used to import data from whatever source they come, easing future interface developments. The system is highly customizable via Ubik Plugins to fit most use cases.

In detail, the service consists of three major parts:

• Web Service
  • Customizable Authentication System
• Customizable Import System

Web Service

The Web Service contains the externally callable endpoint, containing the type reference of the import object in the URL and the data in the body:

This request also expects some Authentication data, which are completely dependent on the customized implementation provided via plugins.

Authentication System

There is currently no default implementation, so in order to get the Web Service running, one plugin containing a custom ImportValidator has to be provided. It contains a single method to override, Validate, which contains the headers of the incoming requests to validate the Authorization header. There is also a possibility to set the outgoing headers for the response, for example, to return a token that makes authentication easier next time.

Import System

The Import System is the heart of the entire system. It is responsible for creating Ubik objects out of raw data, via proxies. In its ideal use case, the interface runs the following steps:

• Creating ImportObject derivate out of the raw data
• Loading ImportProcessor derivate supporting the type of ImportObject
• Validating the ImportObject with the <ImportProcessor>
• Writing the ImportObject to a new or existing Proxy instance
• Saving the Proxy

At this time, the data are successfully saved in Ubik and the interface returns a status and object information. However, there's some more stuff going on in the background:

• Async bulk-import of Proxy to create Ubik object
• Once finished and new items are queued, a new bulk-import will launch. If no import is queued, the customizable post processing method will be called.