River publishers

Extensions to HEVC

As with H.264/AVC, additions/extensions to HEVC include 4:2:2 and 4:4:4 formats, higher bit depths (10 and 12), scalable video coding (SVC), (References are listed in section 5.10 Summary) 3D/stereo/multiview coding and screen content coding. See [E23, E25, E34, E39, E260, E326, SCC48]. Several proposals related to SVC have been presented at the Shanghai meeting of HEVC (ITU-T/ISO, IEC) held in Oct. 2012. The reader is referred to the poster session [E27] MA.P2 “High efficiency video coding”, IEEE ICIP 2012, Orlando FL, Sept. - Oct. 2012 and the special issue on emerging research and standards in next generation video coding” IEEE Trans. CSVT, vol. 22, Dec. 2012 [E45]. Several papers from this special issue are cited as references here [E61 - E63, E67, E73, E74, E76-E79, E83 -E85, E87 -E93, E120]. See also [E321, E325, E326]

These papers cover not only the various aspects of HEVC, but also ways to reduce the implementation complexity and the reasons behind the various tools and techniques adopted in this standard. The justification for initial consideration of some tools such as adaptive loop filter, MDDT, ROT etc. and subsequent elimination of these tools in the standard is also provided in some of these papers.

Some of these tools are reconsidered in the next generation video coding (NGVC), beyond HEVC. Also IEEE Journal of selected topics of signal processing has published a special issue on video coding: HEVC and beyond, vol.7, December 2013. [E80]

These additions/extensions have been standardized in 2015.

6. Profiles and levels

Three profiles (Main, Main10 and Main Still Picture – intra frame only) are listed in Annex A in the final draft international standard (FDIS) (January 2013) [E125]. ITU-T study group 16 has agreed to this first stage approval formally known as Recommendation H.265 or ISO/IEC 23008-2. Main profile is limited to YC_bC_r 4:2:0 format, 8 bit depth, progressive scanning (non interlaced) with spatial resolutions ranging from QCIF (176x144) to 7640x4320 (called 8Kx4K). Figure 5.12 [E23] lists the spatial resolutions ranging from SD (NTSC) to super Hi-Vision/ultra HD video. In the main profile 13 levels are included in the first version (Table 5.7) [E61, E99].

Fig. 5.12 Future visual applications and demands. (NHK and Mitsubishi have developed a SHV-HEVC hardware encoder) [E23]

Table 5.7 Level limits for the main profile in HEVC [E61] © 2012 IEEE

7. Performance and computational complexity of HEVC encoders

Correa et al [E84] have carried out a thorough and detailed investigation of the performance of coding efficiency versus computational complexity of HEVC encoders. This investigation focuses on identifying the tools that most affect these two vital parameters (efficiency and complexity). An invaluable outcome is the trade off between the efficiency and complexity useful for implementing complexity-constrained encoders.

Additionally, the development of low-complexity encoders that achieve coding efficiency comparable to high-complexity configurations is a valuable resource for industry. This combination of low-complexity and high efficiency can be achieved by enabling Hadamard ME, asymmetric motion partitions and the loop filters instead of computationally demanding tools such as ME. Their analysis included the three coding tools (non square transform, adaptive loop filter and LM chroma) which have been subsequently dropped from the draft standard [E60].

8. System layer integration of HEVC

Schierl et al describe the system layer integration of HEVC [E83]. The integration of HEVC into end-to-end multimedia systems, formats, and protocols (RTP, MPEG-2 TS, ISO File Format, and dynamic adaptive streaming over HTTP - DASH) is discussed. Error resiliency tools in HEVC are also addressed. Many error resilience tools of H.264/AVC such as FMO, ASO, redundant slices, data partitioning and SP/SI pictures (Chapter 4) have been removed due to their usage. They suggest that the use of error concealment in HEVC should be carefully considered in implementation and is a topic for further research. They conclude with a description of video transport and delivery such as broadcast, IPTV, internet streaming, video conversation and storage as provided by the different system layers. See reference 24 listed in this paper i.e., T. Schierl, K. Gruenberg and S. Narasimhan, “Working draft 1.0 for transport of HEVC over MPEG-2 Systems”, ISO/IEC SC29/WG11, MPEG99/N12466, February 2012. See other references related to various transport protocols.

9. HEVC lossless coding and improvements [E86]

Zhou et al [E85] have implemented lossless coding mode of HEVC main profile (bypassing transform, quantization and in-loop filters – Fig. 5.13 [E85]) and have shown significant improvements (bit rate reduction) over current lossless techniques such as JPEG2000, JPEG-LS, 7-Zip and WinRAR. (See Appendix F) They improve the coding efficiency further by introducing the sample based intra angular prediction (SAP).

Figure 5.13 HEVC encoder block diagram with lossless coding mode [E85] (See Fig. 5.5) © 2012 IEEE. See also [E194, E196]

Cai et al [E114] also conducted the lossless coding of HEVC intra mode, H.264 High 4:4:4 Profile intra mode, MJPEG 2000 and JPEG-LS using a set of recommended video sequences during the development of HEVC standard. Their conclusion is that the performance of HEVC has matched that of H.264/AVC and is comparable to JPEG-LS and MJPEG 2000. Similar tests on lossy intra coding show that HEVC high10, H.264/AVC HP 4:4:4 and HEVC MP have similar performance. However, MJPEG 2000 outperforms the former three in low bit scenario, while this advantage is gradually compensated and finally surpassed by the former three as the bit rate increases. Several other interesting papers on performance comparison of these and other standards are listed in [E114]. Wige et al [E147] have implemented HEVC lossless coding using pixel-based averaging predictor. Heindel, Wige and Kaup [E194] have developed lossless compression of enhancement layer (SELC) with base layer as lossy. Wang et al [E196] have implemented lossy to lossless image compression based on reversible integer DCT.

Horowitz et al [E66] have conducted the informal subjective quality (double blind fashion) comparison of HEVC MP (reference software HM 7.1) and H.264/AVC HP reference encoder (JM 18.3) for low delay applications. Table 5.14 describes the comparison results for test sequences encoded using the HM and JM encoders. They conclude that HEVC generally produced better subjective quality compared with H.264/AVC at roughly half the bit rate with viewers favoring HEVC in 73.6% trials.

Table 5.14. Subjective viewing comparison results for sequences encoded using the HM and JM encoders [E66] © SPIE 2012

To reinforce these results, x264 the production quality H.264/AVC encoder is compared with eBrisk Video (production quality HEVC implementation). See Table 5.15 for the comparison results. Viewers favored eBrisk encoded video at roughly half the bit rate compared to x264 in 62.4% trials. These tests confirm that HEVC yields similar subjective quality at half the bit rate of H.264/AVC. Chapter 9 – Compression performance analysis in HEVC – by Tabatabai et al in [E202] describes in detail the subjective and objective quality comparison of HEVC with H.264/AVC. Tan et al [4] have presented the subjective and objective comparison results of HEVC with H.264/AVC and conclude that the key objective of the HEVC has been achieved i.e., substantial improvement in compression efficiency compared to H.264/AVC.

Table 5.15. Subjective viewing comparison results for sequences encoded using the eBrisk and x264 encoders [E6] © 2012 SPIE

10.