Data Ingestion Service (DIS)

Various data sources such as logs, the Internet of Things, social media feeds or geotracking generate data continuously. In some business scenarios, analyzing this data via big data analyses generates added value. However, one of the main challenges is transferring the never-ending data flows to the platform on which they are to be analyzed. Data ingestion services are designed for this task. They allow data to be recorded more quickly, i.e. immediately after production, in order to generate real-time analyses as well as a range of other functions.

With the Data Ingestion Service (DIS), the Open Telekom Cloud offers a module from the MapReduce suite as a standalone service. The DIS ensures that large quantities of data are available in the Open Telekom Cloud in a very short time. Based on Kafka, HBase and Redis, the service offers users the freedom to use the most suitable tool for analysis and allows the streams to be designed using the console. Common streams enable up to 50 partitions with a reading speed of 2 MB/s, while up to 10 partitions with 10 MB/s are possible using advanced streams. Long-term data storage can be implemented using Object Storage.