Data Lake Insight (DLI) new minor version contains following new features:
Data Lake Insight (DLI) new Minor Version upgrade containing new features
- DLI decouples the permissions of the dli_admin_agency agency, meaning that the DLI management plane and the DLI tenant plane can have different permissions for this agency.
- Tenant-plane metrics can now be integrated with Prometheus.
- DLI now supports displaying SQL execution plans for SQL statements that are billed based on the amount of data scanned.
- Logs now display the number of rows read from and written to tables.
- Optimized the SPARK CTE syntax.
- When exporting query results to a CSV file, you can now specify an escape character to avoid garbled text when opening the file in Excel.
- Released Spark 3.3 notebook image on the live network.
- Reduced CBC service tickets using common error codes.
- The tenant plane can access OBS through VPCEP.
- Enhanced Spark Driver monitoring, including Full GC monitoring.
- ·DLI elastic resource pools now support 16 to 64 CUs.
- Hive text format tables now support multi-character delimiters.
Here some insights regarding a cost saving feature introduced in point: “DLI elastic resource pools now support 16 to 64 CUs” and how to get benefit of it.
Before you could create a Ressource Pool with a minimal size of 64 CU or bigger.
In the new minor version, you may choose Basic or Standard. Basic is for test/development scenarios where the minimal Compute Units(CU) size is now b 16 CU and Standard starting by 64 CU for production.
After you have created a Resource Pool you may be able in the new version to Create Queues inside a Resource Pool. Please keep in mind that without a Resource Pool you may be not able to Create Queues in the new version.
With this new feature customers will reduce their cost at a maximal of one quarter by selecting 16 CU instead of 64 in the past.
Further information can be found in the Data Lake Insight (DLI) area of the Help Centers.