Video coding standards k. R. Rao, Do Nyeon Kim J. J. Hwang Springer 2014

BEYOND HEVC:

[BH1] J. Chen et al, “Coding tools investigation for next generation video coding based on HEVC”, [9599 – 47], SPIE. Optics + photonics, San Diego, California, USA, 9 – 13, Aug. 2015.

[BH2] A. Alshin et al, “Coding efficiency improvements beyond HEVC with known tools,” [9599-48], SPIE. Optics + photonics, San Diego, California, USA, 9 – 13, Aug. 2015. References [2] through [8] describe various proposals related to NGVC (beyond HEVC) as ISO / IEC JTC – VC documents presented in Warsaw, Poland, June 2015. Several projects can be implemented / explored based on [BH2] and these references.

[BH3] A. Alexander and A. Elina, “Bi-directional optical flow for future video codec”, IEEE DCC, Sun Bird, Utah, March-April 2016.

[BH4] X. Zhao et al, “Enhanced multiple transform for video coding”, IEEE DCC, Sun Bird, Utah, March-April 2016.

Projects on BEYOND HEVC:

[BH-P1] In [BH2] several tools (some of these are straight forward extensions of those adopted in HEVC) are considered in NGVC (beyond HEVC). Some of these are increasing CU and TU sizes, up to 64 adaptive intra directional predictions, multi – hypothesis probability estimation for CABAC, bi-directional optical flow, secondary transform, rotational transform and multi-parameter intra prediction. Go through [BH2] in detail and evaluate the performance of each new tool, for all – intra, random access, low delay B and low delay P (see Table 1). Compare the computational complexity of each tool with that of HEVC.

[BH-P2] See [BH-P1]. Evaluate performance impact of enlarging CU and TU sizes (see Table 2). Also consider computational complexity.

[BH-P3] See [BH-P1]. Evaluate performance impact of fine granularity Intra prediction vs HEVC with enlarged CU and TU sizes (see Table 3). Also consider computational complexity.

[BH-P4] See [BH-P1]. Evaluate performance impact of multi-hypothesis probability estimation vs HEVC with enlarged CU and TU sizes (see Table 4). Also consider computational complexity.

[BH-P.5] See [BH-P1]. Evaluate performance impact of bi-directional optical flow vs HEVC with enlarged CU and TU sizes (see Table 6). Also consider computational complexity.

[BH-P6] See [BH-P1]. Evaluate performance impact of implicit secondary transform vs HEVC with enlarged CU and TU sizes (see Table 7). Also consider computational complexity.

[BH-P7] See [BH-P1]. Evaluate performance impact of explicit secondary transform vs HEVC with enlarged CU and TU sizes (see Table 9). Also consider computational complexity.

[BH-P8] See [BH-P1]. Evaluate performance impact of multi-parameter Intra prediction vs HEVC with enlarged CU and TU sizes (see Table 11). Also consider computational complexity.

[BH-P9] See [BH-P1]. Evaluate Joint performance impact of all tested tools on top of HEVC (see Table 6). Also consider computational complexity.

[BH-P10] Similar to [BH2], several coding tools (some of them were considered earlier during the initial development of HEVC) are investigated for NGVC based on HEVC [BH1]. These tools include large CTU and TU, adaptive loop filter, advanced temporal Motion Vector Prediction, cross component prediction, overlapped block Motion Compensation and adaptive multiple transform. The overall coding performance improvement resulting from these additional tools for classes A through F Video test sequences in terms of BD-rate are listed in Table 3. Class F sequences include synthetic (computer generated) video. Performance improvement of each tool for Random Access, All Intra and low-delay B is also listed in Table 4. Evaluate the performance improvement of these tools [BH1] over HM 16.4 and verify the results shown in Table 3.

[BH-P11] See [BH-P10]. Evaluate the performance improvement of each tool for All Intra, Random Access and Low-delay B described in Table 4. Test sequences for screen content coding (similar to class F in Table 3) can be accessed from http://pan.baidu.com/share/link?shareid=3128894651&uk=889443731 .

[BH-P12] See [BH-P10] and [BH-P11]. Consider implementation complexity as another metric, as these additional tools invariably result in increased complexity. Evaluate this complexity for all classes (A through F) – see Table 3 and for All Intra, Random Access and low-delay B cases (see Table 4). See the papers below related to proposed requirements for Next Generation Video coding (NGVC).

1. ISO/IEC JTC1/SC29/WG11, “Proposed Revised Requirements for a Future Video coding Standard”, MPEG doc.M36183, Warsaw, Poland, June. 2015.

2. M. Karczewicz and M. Budagavi, “Report of AHG on Coding Efficiency Improvements,” VCEG-AZ01, Warsaw, Poland, June 2015.

3. J. –R. Ohm et al, “Report of AHG on Future Video Coding Standardization Challenges,” MPEG Document M36782, Warsaw, Poland, June. 2015.

[BH-P13] In [BH1] for larger resolution video such as 4K and 8K, 64x64 INT DCT (integer approx.) is proposed to collaborate with the existing transforms (up to 32x32 INT DCT) and to further improve the coding efficiency. Madhukar and Sze developed unified forward and inverse transform architecture (2D-32x32 INT DCT) for HEVC (see IEEE ICIP 2012) resulting in simpler hardware compared to implementation separately i.e., forward and inverse. See also Chapter 6 HEVC transform and quantization by Budagavi, Fuldseth and Bjontegaard in [E202]. Implement similar uniform forward and inverse transform architecture for 2D-64x64 INT DCT and evaluate the resulting simpler hardware compared to implementation separately i.e., forward and inverse.

[BH-P14] Refer to Figs. 6.7 and 6.8 of chapter 6 cited in [BH-P13]. These QM are based on visual sensitivity of the transform coefficients. Similar QM have been proposed / adopted in JPEG, MPEG – 1,2,4, AVS china etc. Also see the book by K.R. Rao and P. Yip, “Discrete Cosine Transform”, Academic press 1990, wherein the theory behind developing the QM is explained. See also related references at the end of chapter 6 in [E202]. The QM matrices (Fig 6.8) for (16x16) and (32x32) transform block sizes are obtained by replicating the QM for (8x8) transform block size. These extensions are based on reducing the memory needed to store them. Develop QM for (16x16) and (32x32) transform block sizes independently based on their visual sensitivities (perception).

[BH-P15] See [BH-P14]. Develop QM for (64x64) INTDCT reflecting the visual perception of these transform coefficients. Again refer to the book by K.R. Rao and P. Yip, “Discrete Cosine Transform”, Academic press 1990.

[BH-P16] See [BH-P13] and [BH-P14]. In Fig.6.2 (page 149) of chapter 6 [E202] (32x32) INTDCT, (4x4), (8x8), (16x16) INTDCTs are embedded. Develop (64x64) INTDCT wherein the smaller size INTDCTs are embedded. Is it orthogonal? What are the norms of the 64 basis vectors?

[BH-P17] See “Y. Sugito et al, "A study on addition of 64x64 transform to HM 3.0.," Joint Collaborative Team on Video Coding (JCT-VC) of ITU-T SG16 WP3 and ISO/IEC JTC1/SC29/WG11, JCTVC-F192, Torino, Italy, July, 2011” which is reference [15] in [BH2]. Is the (64x64) INTDCT orthogonal? Embedding property?

[BH-P18] See [BH3]. Theoretical and implementation aspects of bi-directional optical flow which is part of JEM1 (Joint Exploration Model) See [2] in this paper are developed. Evaluate this MV refinement in terms of MC prediction compared to the ME process adopted in HEVC based on some standard test sequences. Sum of absolute MC prediction errors for various block sized can be used as the comparison metric.

[BH-P19] In [BH4] enhanced multiple transform (EMT) is proposed and implemented on top of the reference software using various test sequences. See Table 4. This scheme is compared with HM-14.0Main 10 for all intra (AI) and random access (RA) configurations. Verify these results and extend the simulation to low delay case. Use the latest HM software.