MSU Cloud Benchmark 2020
conducted by Moscow State University Graphics & Media Lab Video Group
Find the best encoding service for your task
Video group head: | Dr. Dmitriy Vatolin |
Project head: | Dr. Dmitriy Kulikov |
Measurements, analysis: |
Dr. Mikhail Erofeev,
Egor Sklyarov, Anastasia Antsiferova |
Contents
- About
- Comparison stages
- Participating services
- Previous comparison results
- Contribute
- Contact
- Subscribe to receive notifications about our reports
About
We perform independent comparison of cloud-based video-encoding services. This kind of encoding tools is quickly gaining popularity and selecting one for your tasks is quite difficult. In our experience, encoding performance may entirely differ from service to service. The results of our previous comparison showed that in the case of cloud services that provide the same quality, the file-size difference can reach 100% and the cost per minute difference is 700%.
Comparison main points
-
Encoding time and objective videos quality are measured
We perform a series of measurements to estimate average time needed for transcoding. Subjective comparison and objective algorithms are used for estimating the resulting video quality -
Two encoding use cases: default encoding options and options provided by service maintainers
This year we will perform two types of tests: encode test videos using service default encoding settings and using the settings recommended by service maintainers for our scenario. If no settings are provided we will do our best to find and use optimal encoding settings (for example, see "Tunned Qencode" in the previous comparison report). If a service does not have an option to vary encoding settings, it will not be presented in the second test case -
FullHD videos of different type and content (including UGC, vertical and noisy videos, etc.)
5-10 FullHD videos will be used for tests. We will perform two types of measurements: the main case involved encoding into FullHD, and the additional case will include transcoding into several resolutions (the results will be published in the report appendix) -
Cross-checking is performed by industry experts and independent users
We ask the users of participating services to encode several videos from test set to validate the results -
Anonymous accounts are used for testing
For fair comparison, we use accounts which are not connected to us as the comparison organizers. This is done to prevent the services reserving separate servers/rules for comparison test videos processing
Comparison Stages
1. Application
February, 18 - April, 10 |
Application for participation is open for all cloud encoding services and users!
Use the form to tell us which services you want to see in this comparison |
|
|
||
2. Measurements | We encode comparison videos in each service and measure encoding performance | |
|
||
3. Verification | We conduct independent cross-checking of obtained results. You can become a contributor and verify comparison results. If you have an account in any of cloud encoding services, we will provide test sequences and pay for encoding time which needed for measurements | |
|
||
4. Release | PDF and HTML reports are published on this page. There will be two versions (free and enterprise). All contributors will receive full (enterprise) report |
List of services to be compared
Cloud Encoding Service | Supported standards | |
1 | Alibaba | H.264, HEVC |
2 | Amazon Elastic Transcoder | H.264 |
3 | AWS Elemental MediaConvert | H.264, HEVC |
4 | Coconut | H.264, HEVC |
5 | Qencode | H.264, HEVC |
6 | Zencoder | H.264, HEVC |
7 |
We are open to add other services to the comparison. Please use the form below if you have any suggestions. Which cloud encoding services you want to see in the comparison? |
We are looking for people who have an account in any cloud encoding services and able to use it for verificating encoding results. If you want to participate in our comparison, contact us using this form or via cloud@compression.ru
Download previous report
Video Transcoding Clouds Comparison 2019
Download free full version ![]() ![]() |
|
Take part in cloud video-encoding services comparison
We ask users of cloud encoding/transcoding services to take part in this comparison by encoding several videos to compare the results with our measurements. Approximate timeline of verification measuremets for contributors:
If you have an account in any cloud encoding service and want contact us usint the form below. We are ready to compensate the cost of encoding.
Contact Information
![]() |
![]() compression.ru |
in cooperation with |
![]() |
Lomonosov MSU
Graphics & Media Lab (Video Group) |
![]() |
Dubna State University | ||
![]() |
Institute for Information Transmission Problems RAS |
Subscribe to report updates
Materials about MSU Codec Comparison
Call for cloud encoding comparison participation 2020 See all MSU Video Codecs Comparisons
MSU video codecs comparisons resources:
- Introduction to Video Codecs Comparison
- Lossless Video Сodecs Comparison 2004 (October 2004)
- MPEG-4 SP/ASP Video Codecs Comparison (March 2005)
- JPEG 2000 Image Codecs Comparison (September 2005)
- First Annual MPEG-4 AVC/ H.264 Video Codecs Comparison (January 2005)
- Second Annual MPEG-4 AVC/H.264 Video Codec Comparison (December 2005)
- Subjective Comparison of Modern Video Codecs (February 2006)
- MPEG-2 Video Decoders Comparison (May 2006)
- WMP and JPEG2000 Comparison (October 2006)
- Third Annual MPEG-4 AVC/H.264 Comparison (December 2006) (All versions for free!)
- Lossless Video Codecs Comparison 2007 (March 2007)
- Fourth Annual MPEG-4 AVC/H.264 Comparison (December 2007) (All versions for free!)
- Options Analysis of MPEG-4 AVC/H.264 Codec x264 (December 2008)
- Fifth MPEG-4 AVC/H.264 Comparison (May 2009) (All versions for free!)
- Sixth MPEG-4 AVC/H.264 Comparison (May 2010)
- Seventh MPEG-4 AVC/H.264 Comparison (May 2011)
- Eighth MPEG-4 AVC/H.264 Comparison (May 2012)
- Ninth MPEG-4 AVC/H.264 Comparison (Dec 2013)
- Tenth Video Codec Comparison (HEVC) (Oct 2015)
- Eleventh Video Codec Comparison (HEVC) (Aug 2016)
- Twelfth Video Codec Comparison (HEVC) (Aug 2017)
- Thirteen Video Codec Comparison (HEVC) (Aug 2018)
- Fourteen Video Codec Comparison (HEVC) (Sept 2019)
- Cloud Encoding Servoces Comparison 2019 (Dec 2019)
- Codec Analysis for Companies:
Other Materials
Video resources:
Server size: 8069 files, 1215Mb (Server statistics)
Project updated by
Server Team and
MSU Video Group
Project sponsored by YUVsoft Corp.
Project supported by MSU Graphics & Media Lab