Load-based elastic resource pool, queue, and job scaling

• SQL jobs can preempt resources based on loads (graceful scaling).
• SQL queues can preempt resources based on loads.
• In a SQL queue, the number of CUs applied for by the pod in the QuotaManager information can be customized.

Automatic scaling of general-purpose queues based on job priorities and resource loads

• The job waiting list of the general queue is sorted by time and priority.
• The ClusterAgent component of the cluster can collect backpressure metrics of Flink jobs.
• The ClusterAgent can collect the number of Spark executors.

Permissions on DLI database tables can be granted by tag

• Permissions on DLI database tables can be granted and authenticated by tag.
• APIs that support granting permissions by tag are added.
• On the DLI console, permissions on DLI tables can be granted by tag.

Fließende Skalierung von elastischen Ressourcenpools

• Elastische DLI-Ressourcenpools können problemlos skaliert werden.
• Allgemeine Warteschlangen können in elastische Ressourcenpools migriert werden.
• Verschiedene Flavors in elastischen Ressourcenpools können zugewiesen und freigegeben werden.

DLI can connect to multiple catalogs

• dli-catalog-client connects to DLI catalogs using independent configuration items.
• Spark images connect to DLI catalogs.
• The Spark Extension plugin connects to DLI catalogs using new configuration items.