Video coding standards k. R. Rao, Do Nyeon Kim J. J. Hwang Springer 2014
High efficiency video coding (HEVC)
Yüklə 0,84 Mb.
|
- Bu səhifədəki naviqasiya:
- Introduction
5 High efficiency video coding (HEVC)ABSTRACTHEVC the latest video coding standard is presented. Comparison with H.264/AVC (Chapter 4) is cited. The focus is on overview of HEVC rather than a detailed description of tools and techniques that constitute the encoder. A plethora of projects listed at the end challenges the implementation and further research related to HEVC. Keywords: HEVC, JCTVC, unified intra prediction, coding tree unit, prediction unit, transform unit, SAO, coefficient scanning, HM software, lossless coding.
This chapter details the development of HEVC by the joint collaborative team on video coding (JCT-VC).
The Joint Collaborative Team on Video Coding is a group of video coding experts from ITU-T Study Group 16 (VCEG) and ISO/IEC JTC 1/SC 29/WG 11 (MPEG) created to develop a new generation video coding standard that will further reduce by 50% the data rate needed for high quality video coding, as compared to the current state-of-the-art advanced video coding (AVC) standard (ITU-T Rec. H.264 | ISO/IEC 14496-10).This new coding standardization initiative is being referred to as High Efficiency Video Coding (HEVC). In ISO/IEC it is called MPEG-H Part2. VCEG is video coding experts group and MPEG is moving picture experts group. Tan et al [SE4] have presented the subjective and objective results of a verification test in which HEVC is compared with H.264/AVC. The conclusions in [SE4] are repeated here: “These tests show that the bit rate savings of 59% on average can be achieved by HEVC for the same subjective quality compared with H.264/AVC. The PSNR-based BD-rate average over the same sequences was calculated to be 44%. This confirms that the subjective quality improvements of HEVC are typically greater than the objective quality improvements measured by the method that was primarily used during the standardization process of HEVC. It can therefore be concluded that the HEVC standard is able to deliver the same subjective quality as AVC, while on average (and in the vast majority of typical sequences) requiring only half or even less than half of the bit rate used by AVC”. ITU-T Rec. H.264 | ISO/IEC 14496-10, commonly referred to as H.264/MPEG-4-AVC, H.264/AVC, or MPEG-4 Part 10 AVC (Chapter 4) has been developed as a joint activity within the joint video team (JVT). The evolution of the various video coding standards is shown in Fig. 5.1. P.S.: H.265 and recent developments in video coding standards (Seminar presented by Dr. Madhukar Budagavi on 21 Nov. 2014 in the Dept. of Electrical Engineering, Univ. of Texas at Arlington, Arlington, Texas )
Cisco VNI Global IP Traffic Forecast 2014-2019 predicts that a) Globally IP video traffic will be 80% of all IP traffic (both business and consumer) by 2019 b) By 2019 more than 30% of connected flat-panel TV sets will be 4K and c) Traffic from wireless and mobile devices will exceed traffic from wired devices by 2016”. HEVC is designed to address the increased video traffic and increased resolution such as the 4K and 8K video.
Fig. 5.1 Evolution of video coding standards (AOM: alliance for open media) Fig.5.1 Video coding standardization (courtesy Dr. Nam Ling, Sanfilippo family chair professor, Dept. of Computer Engineering, Santa Clara University, Santa Clara, CA, USA) [E23], AOM: Alliance for open Media The JCT-VC is co-chaired by Jens-Rainer Ohm and Gary Sullivan, whose contact information is provided below.
Additional information can be obtained from http://www.itu.int/en/ITU-T/studygroups/com16/video/Pages/jctvc.aspx JCT-VC has issued a joint call for proposals in 2010 [E7] • 27 complete proposals submitted (some multi-organizational) • Each proposal was a major package –lots of encoded video, extensive documentation, extensive performance metric submissions, sometimes software, etc. • Extensive subjective testing (3 test labs, 4 200 video clips evaluated, 850 human subjects, 300 000 scores) • Quality of proposal video was compared to AVC (ITU-T Rec. H.264 | ISO/IEC 14496-10) anchor encodings • Test report issued JCTVC-A204/ N11775 • In a number of cases, comparable quality at half the bit rate of AVC (H.264) • Source video sequences grouped into five classes of video resolution from quarter WVGA (416 x 240) to size 2560 x 1600 cropped from 4k x 2k ultra HD (UHD) in YCbCr 4:2:0 format progressively scanned with 8bpp. •Testing for both “random access” (1 sec) and “low delay” (no picture reordering) conditions
Figures 5.2 and 5.3 show results averaged over all of the test sequences; in which the first graph (Figure 5.2) shows the average results for the random access constraint conditions, and the second graph (Figure 5.3) shows the average results for the low delay constraint conditions. The results were based on an 11 grade scale, where 0 represents the worst and 10 represents the best individual quality measurements. Along with each mean opinion score (MOS) data point in the figures, a 95% confidence interval (CI) is shown. 
Figure5.2. Overall average MOS results over all Classes for Random Access coding conditions [E7]. 
Figure5.3. Overall average MOS results over all Classes for Low Delay coding conditions [E7]. A more detailed analysis performed after the tests, shows that the best-performing proposals in a significant number of cases showed similar quality as the AVC anchors (H.264/AVC ) at roughly half the anchor bit rate [E25, E61, E99]. More recently, Tan et al [SE4] present the subjective and objective results of a verification test in which the performance of HEVC Mail profile is compared with H.264/AVC High profile using video test sequences at various bit rates and resolutions. They conclude that the HEVC is able to deliver the same subjective quality as H.264/AVC while on average requiring only half or even less than half the bit rate used by the latter. [SE4] describes in detail, the methodology, test procedures, test sequences, comparison metrics and other factors in arriving at the conclusions. The technical assessment of the proposed technology was performed at the first JCT-VC meeting held in Dresden, Germany, 15 - 23 April 2010. It revealed that all proposed algorithms were based on the traditional hybrid coding approach, combining motion-compensated prediction between video frames with intra-picture prediction, closed loop operation with in-loop filtering, 2D transform of the spatial residual signals, and advanced adaptive entropy coding. As an initial step toward moving forward into collaborative work, an initial Test Model under Consideration (TMuC) document was produced, combining identified key elements from a group of seven well performing proposals. This first TMuC became the basis of a first software implementation, which after its development has begun to enable more rigorous assessment of the coding tools that it contains as well as additional tools to be investigated within a process of "Tool Experiments (TE)” as planned at the first JCT-VC meeting. P.S.: Detailed subjective evaluations in mobile environments (smart phone/iPad, Tablet), however, have shown that the user (observer) experience is not significantly different when comparing H.264/AVC and HEVC compressed video sequences at low bit rates (200 and 400 kbps) and small screen sizes [E124, E210]. Advantages of HEVC over H.264/AVC appear to increase dramatically at higher bit rates and high resolutions such as HDTV, UHDTV etc. One of the most beneficial elements for higher compression performance in high-resolution video comes due to introduction of larger block structures with flexible mechanisms of sub-partitioning. For this, the TMuC defines coding units (CUs) which define a sub-partitioning of a picture into rectangular regions of equal or (typically) variable size. The coding unit replaces the macroblock structure (H.264) and contains one or several prediction unit(s) (PUs) and transform units (TUs). The basic partition geometry of all these elements is encoded by a scheme similar to the quad-tree segmentation structure. At the level of PU, either intra-picture or inter-picture prediction is selected. The paper “Block partitioning structure in the HEVC standard”, by I.-K. Kim et al [E93], explains the technical details of the block partitioning structure and presents the results of an analysis of coding efficiency and complexity. • Intra-picture prediction is performed from samples of already decoded adjacent PUs, where the different modes are DC (flat average), horizontal, vertical, or one of up to 28 angular directions (number depending on block size), plane (amplitude surface) prediction, and bilinear prediction. The signaling of the mode is derived from the modes of adjacent PUs. • Inter-picture prediction is performed from region(s) of already decoded pictures stored in the reference picture. This allows selection among multiple reference pictures, as well as bi-prediction (including weighted averaging) from two reference pictures or two positions in the same reference picture. In terms of the usage of the motion vector (quarter pixel precision), merging of adjacent PUs is possible, and non-rectangular sub-partitions are also possible in this context. For efficient encoding, skip and direct modes similar to the ones of H.264/AVC (chapter 4) are defined, and derivation of motion vectors from those of adjacent PUs is made by various means such as median computation or a new scheme referred to as motion vector competition. At the TU level (which typically would not be larger than the PU), an integer spatial transform similar in concept to the DCT is used, with a selectable block size ranging from 4×4 to 64×64. For the directional intra modes, which usually exhibit directional structures in the prediction residual, special mode-dependent directional transforms (MDDT) [E51] are employed for block sizes 4×4 and 8×8. Additionally, a rotational transform (See P.5.13) can be used for the cases of block sizes larger than 8×8. Scaling, quantization and scanning of transform coefficient values are performed in a similar manner as in AVC. At the CU level, it is possible to switch on an adaptive loop filter (ALF) which is applied in the prediction loop prior to copying the frame into the reference picture buffer. This is an FIR filter which is designed with the goal to minimize the distortion relative to the original picture (e.g., with a least-squares or Wiener filter optimization). Filter coefficients are encoded at the slice level. In addition, a deblocking filter (similar to the deblocking filter design in H.264/AVC) [E73] is operated within the prediction loop. The display output of the decoder is written to the decoded picture buffer after applying these two filters. Please note that the ALF has been dropped in the HEVC standard [E235, E61, E99]. In the updated version, in loop filtering consists of deblocking [E73, E209, E283, E357, E378] and sample adaptive offset (SAO) filters (Fig.5.4). See [E87, E111, E373] about SAO in the HEVC standard. The TMuC defines two context-adaptive entropy coding schemes, one for operation in a lower-complexity mode, and one for higher-complexity mode. A software implementation of the TMuC has been developed. On this basis, the JCT-VC is performing a detailed investigation about the performance of the coding tools contained in the TMuC package, as well as other tools that have been proposed in addition to those. Based on the results of such Tool Experiments (TE), the group will define a more well-validated design referred to as a Test Model (TM) as the next significant step in HEVC standardization. Specific experiments have been planned relating to a tool-by-tool evaluation of the elements of the current TMuC, as well as evaluation of other tools that could give additional benefit in terms of compression capability or complexity reduction in areas such as intra-frame and inter-frame prediction, transforms, entropy coding and motion vector coding. Various ad hoc groups (AHGs) have been set up to perform additional studies on issues such as complexity analysis, as listed below: Ad Hoc Coordination Groups Formed •JCT-VC project management •Test model under consideration editing •Software development and TMuC software technical evaluation •Intra prediction •Alternative transforms •MV precision •In-loop filtering •Large block structures •Parallel entropy coding In summary, the addition of number of tools within the motion compensated (MC) transform prediction hybrid coding framework (adaptive intra/inter frame coding, adaptive directional intra prediction, multiple block size motion estimation, SAO filter [E87, E111, E373], in loop deblocking filter [E73, E209, E283, E357, E378], entropy coding (CABAC, see [E67, E68]), multiple frame MC weighted prediction, integer transforms from 4x4 to 32x32 [E8, E74], Hadamard transform coding of dc coefficients in intra frame coding introduced in H.264/AVC (Chapter 4) and various other tools as enumerated below has shown further gains in the coding efficiency, reduced bit rates, higher PSNR etc. compared to H.264/AVC. Thus HEVC holds promise and potential in a number of diverse applications/fields and is expected to eventually overtake H.264/AVC.
Note b) and c) have been dropped as they contribute very little to coding efficiency at the cost of substantial increase in complexity. Other directional transforms that were proposed, like directional DCT [E112] and mode dependent DCT/DST [E113], were also dropped due to the same reason. Recently steerable DCT (SDCT) and its integer approximation INTSDCT that can be steered in any direction are proposed [E392]. The authors state “SDTC allows to rotate in a flexible way pairs of basis vectors and enables precise matching of directionality in each image block, achieving improved coding efficiency”. They have implemented SDCT in image coding and compared with DCT and DDCT. They also suggest a possible implementation of SDCT and INTSDCT in a video compression standard such as HEVC. See the projects P.5.262 thru P.5.267. Both in loop deblocking and SAO filters reduce blocking and ringing artifacts. However, in HD and UHD videos, false contour is a major artifact due to unbalanced quantization in smooth areas. Huang et al [E393] have identified the cause for false contours and have developed effective method for detection and removal of false contours in both compressed images and videos while preserving true edges and textures. It is interesting to note that some of these tools such as rotational transforms are reconsidered in the Next Generation Video Coding (NGVC) (beyond HEVC) [BH2]. Other tools considered in NGVC (proposed earlier in HEVC) are:
References and figures cited here are from [BH2]. P.S: The introduction to JCT-VC is based on the paper by G.J. Sullivan and J.-R. Ohm published in applications of digital image processing XXXIII, proc. of SPIE vol. 7798, pp. 7798V-1 through 7798V–7, 2010. Paper title is “Recent developments in standardization of high efficiency video coding (HEVC)”[E7]. For the latest developments in HEVC, the reader is referred to an excellent review paper: G.J. Sullivan et al, “Overview of high efficiency video coding (HEVC) standard”, IEEE Trans. CSVT, vol. 22, pp. 1649-1668, Dec. 2012 [E61]. Also keynote speeches on HEVC [E23, E25, E99]. Also tutorials on HEVC (see the section on tutorials). Also see HEVC text specification draft 8 [E60]. An updated paper on HEVC is G.J. Sullivan et al, “Standardized extensions of high efficiency video coding (HEVC)”, IEEE Journal of selected topics in signal processing, vol. 7, pp. 1001-1016, Dec. 2013 [E160]. Another valuable resource is the latest book: V. Sze, M. Budagavi and G.J. Sullivan (editors) “High efficiency video coding (HEVC): algorithms and architectures”, Springer, 2014 [E202]. Another valuable book is M. Wien, “High Efficiency Video Coding: Coding Tools and Specification”, Springer, 2015. (See section on books on HEVC),
|
