In this article you will read about,

• content of the performance and price/performance analysis covering AWS, Microsoft Azure, Google Cloud, and Open Telekom Cloud,
• which VMs in the market provide the best performance,
• and an analysis of key parameters such as storage, bandwidth, database, and encryption/decryption speed.

We are amidst the cloud era. Cloud becomes the new essential for almost any business. Now that the basic advantages are known, many enterprises find their highest priorities in security, compliance, and financial governance of the Public Cloud. They begin to have a look at the details to find optimal solutions for their demands.

Financial governance, for example, is not just about orchestration and FinOps. Cost-effective cloud deployment starts with Infrastructure as a Service (IaaS) performance and thus with the selection of the best Virtual Machines (VMs) and storage for the specific requirement. But only an independent analysis of the various cloud offerings can reveal the true technical capabilities.

In July, T-Systems commissioned the independent cloud analyst Cloud Mercato with a performance and price/performance study. Like in the last year, Cloud Mercato tested a set of comparable 16 vCPU VMs and block storage offered in the European market from European data centers. Overall, the VMs of the Open Telekom Cloud show excellent performance and price/performance values. The following VMs were considered.

Additionally, Cloud Mercato put some a, n and z variety of AWS VMs under the microscope and found out that Open Telekom Cloud c, s and m VM’s still outperform AWS’s a, n and z series. New to the study are analyses of the OpenSSL encryption/decryption speed and performance of relational database services.

To collect a comprehensive performance value, the benchmark suite Geekbench 5 was used. Most instance types in the basic test panel of comparable 16 vCPU VMs delivered a Geekbench performance value of about 8k for multi-threading. That means that Google Cloud and Azure improved the performance compared to the last year, while AWS delivered nearly the same performance as 2020. The test also revealed that Open Telekom Cloud’s VMs deliver a performance plus of more than 50 percent (about 12k Geekbench performance value) on average. Only one of the additional AWS VMs (m6i) came close to that value.

While Cloud Mercato did not observe a development in the CPU performance of AWS flavors, the measured bandwidth shows a clear performance gain. Introducing the n-tagged network-optimized VMs (c5n, m5n, r5n) obviously increased the local bandwidth between machines in the same data center. Iperf 3 with TCP mode and a number of threads equal to CPU showed a network bandwidth of about 24 Gbps.
With the value, AWS nearly reaches, but is still behind Open Telekom Cloud‘s c4 VM standard level. 

OpenSSL encryption is one of the most important challenges for compute power as it supports the function of a high level of security and compliance for users processing data in the cloud. Thus, performance of services also depends on the ability of VMs to encrypt and decrypt data. How many bytes can be encrypted/decrypted in a second using a 16 vCPU VM? Cloud Mercato found out using OpenSSL speed.

OpenSSL is a lightweight opensource tool provided with the OpenSSL command-line tool. It analyzes the encryption/decryption through different algorithms and block sizes. Its output delivers a good picture of how much a system is able to perform with the different encryption systems available in the industry.

There are huge differences in the basic test panel using a block size of 16.384 Byte and aes-256 cbc encryption: Performance ranges from Azure’s Standard E16 with about 220.000 Bytes encrypted per second to Open Telekom Cloud’s s3 that delivers more than 10-fold performance with 2.7 million.

The performance benchmark is only one side of the coin. The flip side is the price. But which cloud pricing is “high”, which cloud performance “low”? Both parameters must be considered for a realistic view. How much performance do you get for one cent? Cloud Mercato calls this ratio the price/performance value.

The analysts conclude their study with a price/performance comparison of the tested flavors –with huge differences as well. But some results continue to be true: An Open Telekom Cloud VM leads the field delivering about 15 percent additional value for money compared to the next best offer in the market. But hyperscalers’ value for money improves.

And the analysis also reveals: A large “fleet” of specific VMs might theoretically be the best choice for specific use cases, but there are general purpose VMs that can tackle specific workloads with even better performance and cost.  

Interested in the price/performance data? You want to know about RAM performance and RDS performance? Check out the Cloud Mercato benchmark report and download it for free.