Joint Collaborative Team on Video Coding (jct-vc) of itu-t sg16 wp3 and iso/iec jtc1/SC29/WG11

5.3RCE3: Intra coding methods for screen content

JCTVC-N0036 RCE3: Summary Report of HEVC Range Extensions Core Experiment 3 on Intra Coding Methods for Screen Content [L. Guo]

(Reviewed Mon. 29th, Track A (David Flynn).)

A summary of RCE3 on Intra coding methods for Screen Content for HEVC Range Extensions is reported. Three methods have been evaluated based on the CE description in JCTVC-M1123.

The CE performed tests, covering two subject areas:

• 
  Palette coding (two methods, tests one and two), where for each block, a dictionary (palette) of pixel values is transmitted, and the block consists of indices into the dictionary.
  
• 
  Intra motion compensation (an intra picture block copying operation), with two subtests three and four that respectively disable and enable limiting the vertical vector to within the current CTU.
  

No combination tests have been performed.

All methods tested only offer benefits for screen content material, with either no gain or minor losses for non-screen content.

Complexity assessments are reported for one method (Test 1).

For the palette coding methods, the main difference between the two methods is the size of the dictionary, the use of a pixel prediction method and the entropy coding method. Test 2 seems to report higher gains than test 1, but has a higher runtime cost.

5.3.1RCE3 summary and general discussion

5.3.2RCE3 primary contributions

JCTVC-N0205 RCE3: Results of test 3.3 on Intra motion compensation [D.-K. Kwon, M. Budagavi (TI)]
JCTVC-N0247 RCE3: Results of Test 3.1 on Palette Mode for Screen Content Coding [L. Guo, M. Karczewicz, J. Sole (Qualcomm)]
JCTVC-N0287 RCE3 Test 2: Multi-stage Base Color and Index Map [W. Zhu (BJUT), J. Xu (Microsoft), W. Ding (BJUT)]

5.3.3RCE3 cross checks

JCTVC-N0104 RCE3: Cross-verification of Test 3.1 [S. Lee, C. Kim (Samsung)] [late]
JCTVC-N0125 RCE3: Cross-check of Test 3.2 [X. Wei, J. Zan (Huawei)] [late]
JCTVC-N0126 RCE3: Cross-check of Test 3.3 [X. Wei, J. Zan (Huawei)] [late]
JCTVC-N0326 Cross-check report for RCE 3.1 [X. Wang, Z. Ma, M. Xu (Huawei)] [late]
JCTVC-N0327 Cross-check report for RCE 3.3 [X. Wang, Z. Ma, M. Xu (Huawei)] [late]

6Non-CE Technical Contributions

6.1Range extensions

6.1.1General

6.1.2RCE1 related (inter-component decorrelation)

JCTVC-N0266 Non RCE1: Inter Color Component Residual Prediction [W. Pu, W.-S. Kim, C. Chen, L. Guo, J. Sole, M. Karczewicz (Qualcomm)]

(Reviewed in Track A (GJS) 27th p.m.)

In the 13th JCT-VC meeting, seven experiments were included in RCE1 to study inter-component decorrelation methods. This proposal presented an asserted improvement of RCE1 Experiment 3. In the proposed method, the chroma residual is predicted using the scaled luma residual signal. The scaling factor is signalled for each TU. Compared to Experiment 3 of RCE1, a wider range of scaling values are allowed in order to improve the prediction performance. A flag is signalled to switch on/off the method. In case of intra coding, the method is allowed only when DM mode is used as chroma prediction mode. For common coding conditions, the average BD-rate of the method is reportedly -20.2%, 2.1%, -1.1% for Y, U, V, respectively. For the screen content sequences, the BD-rate is reportedly -23.0%, -13.7%, -13.8% for Y, U, V, respectively.

This is an ILP technique. For RGB coding, using only positive coefficients is best. However, for YCbCr, negative are suggested to be allowed. The proposal increases the alpha range accordingly. The alpha does not need to be computed at the decoder side – it is signalled directly. Thus there is lower complexity for the decoder than for a method in which the decoder computes the correlation.

The encoder uses calculation, not exhaustive testing, so the encoder search is not a big issue.

A QP offset can be used or not.

Significantly improved gain was reported for YUV 4:4:4: AI/RA/LB gain was reported as 1.5%/0.5%/0.3% for luma, and 6-8% for chroma. For screen content, the corresponding gains were 6.9%/6.0%/5.9% for luma, and 8-10% for chroma.

The results were cross-checked.

The signalling is at the TU level. Thus, tere is substantial overhead for signalling. The proponent however indicates that TU level operation is best. He has tested others, but found TU level to be best.

To convert luma resolution to chroma resolution for 4:2:2, the (encoder and) decoder decimates the signal.

The technique has been tested for 4:2:2 (and 4:4:4), but not 4:2:0.

The sent information includes, for u and v, the magnitude and sign of alpha.

Some participants suggested strong consideration for use of this technique in a profile supporting 4:4:4 and were leaning toward adoption. Deeper study was certainly supported. However, a couple of participants thought the technique was too complex for the provided benefit (esp. for the encoder).

One participant remarked saying there are other better things we can to in the case of SCC.

The perceptual effects have not been investigated.

The proponent said that when using YCbCr coding, the QP offset idea discussed in the context of RCE1 should be applied.

It was remarked that there is a need to confirm the timescale of the RExt work.

Plan further study in CE.
JCTVC-N0366 Cross-check for JCTVC-N0266: Non-RCE1: Inter Color Component Residual Prediction [K. Sharman, N. Saunders, J. Gamei (Sony)] [late]
JCTVC-N0223 In-loop Chroma Enhancement for HEVC Range Extensions [J. Dong, Y. Ye, Y. He (InterDigital)]

(Reviewed Sun. 28th p.m. Track A (GJS).)

This contribution proposes an in-loop chroma enhancement filtering scheme to HEVC Range Extensions, which is performed after SAO and before the reconstructed picture is added into the DPB. It aims at enhancing the chroma planes of a reconstructed picture, which, if used as a reference picture, also improves the accuracy of future MCP for chroma components. Specifically, a reconstructed chroma pixel is enhanced by adding an appropriate offset obtained by applying the chroma enhancement filters, which usually have high-pass characteristics, on the surrounding luma pixels. By doing this, the chroma edges lost during compression are well restored using the high frequency components from the corresponding luma plane. Experimental results based on common test condition JCTVC-L1006 show that the average {Y, U, V} gain over three color formats (i.e., RGB 4:4:4, YUV 4:4:4, YUV 4:2:2) is {0.0%, -3.0%, -4.9%}, {0.0%, -1.4%, -3.2%}, {0.0%, -0.6%, -1.7%}, {0.2%, -6.0%, -8.4%}, {0.0%, -3.6%, -6.8%}, {-0.1%, -4.6%, -6.7%}, and {-0.2%, -2.7%, -5.1%} for AI-MT, AI-HT, AI-SHT, RA-MT, RA-HT, LB-MT, and LB_HT, respectively.

The original idea of this chroma enhancement filtering scheme was proposed to SHVC to enhance the chroma planes of an ILR picture (JCTVC-L0059 and JCTVC-M0183), and in this contribution is extended to the HEVC Range Extensions single layer coding as an in-loop chroma enhancement filtering scheme.

It was noted that this could be applied as a post-filter, and this was the topic addressed in JCTVC-N0224.

However, no comparison was provided of the effectiveness of the technique to the same test sequences when applying in-loop versus out-of-loop processing.

It was also noted that Wiener filtering within a single component rather than across components (as done in this contribution) can provide a benefit, and would be less complex from the decoder perspective. However, no comparison was provided of the effectiveness of the technique in such a manner.

It was also noted that ALF could be considered somewhat similar in spirit.

The tested technique was constrained to be a high-pass filter. It was asked whether this constraint harms performance, and commented that it does not.

Line buffering was commented to be an issue.

Subjective viewing was suggested if this is to be studied further.

It was noted that in the RA Main Tier case, there was luma degradation (0.6%) that seemed significant enough to potentially offset the chroma improvement (6%).

The technique was not tested on screen content.

For further study.

Presentation to be uploaded.
JCTVC-N0359 Non-RCE1: Cross-check of JCTVC-N0223 In-loop Chroma Enhancement for HEVC Range Extensions [L. Guo (Qualcomm)] [late]
JCTVC-N0368 Non-RCE1: Chroma intra prediction with mode-dependent reduced reference [K. Kawamura, T. Yoshino, S. Naito (KDDI)] [late]

(Reviewed Sun. 28th p.m. Track A (GJS).)

This contribution proposes the two chroma prediction method with reduced references which predicts chroma samples by using linear combination of luma samples for non 4:2:0 format. When a block size is large, a load of the parameter derivation process for each transform unit is reduced by using limited reference samples. One method utilizes fixed reduction pattern, while the other utilizes luma-intra mode-dependent reduced pattern. The Y BD-rate gains of HE Main / High / Super-High tiers are 1.8% / 1.5% / 1.0% for all intra conditions of YUV4:4:4. Compared with the top of TU-based chroma prediction in JCTVC-N0227, the Y BD-rate loss is less than 0.1% for whole AI conditions while the number of reference samples are limited.

The result for natural content YCbCr is reportedly similar to LMC.

The asserted benefit is to reduce the complexity of the coefficient derivation (in both encoder and decoder) relative to TU-based LMC, by the reduced reference usage.

6.1.3RCE2 related (prediction and coding for transform skip)

Non-RCE2 BoG (R. Joshi)

JCTVC-N0137 Non-RCE2: Golomb-Rice parameter initialization for transform-skip and transquant-bypass modes [V. Kolesnikov, C. Rosewarne, M. Maeda (Canon)]

JCTVC-N0350 Non-RCE2: Cross-verification of JCTVC-N0137, Golomb-rice parameter initialization for transform-skip and transquant-bypass modes [R. Cohen (MERL)] [late]
JCTVC-N0181 Non-RCE2: Rice Parameter Initialization [M. Karczewicz, J. Sole, R. Joshi (Qualcomm)]
JCTVC-N0232 Non-RCE2: Rice parameter update method [J. Min, S. Lee, C. Kim (Samsung)]
JCTVC-N0325 Non-RCE2: Cross-verification of JCTVC-N0232 Rice parameter update method [L. Guo (Qualcomm)] [late]
JCTVC-N0281 Non-RCE2 Rice parameter extension for transform-skip blocks [S. H. Kim, K. Misra, A. Segall (Sharp)]
JCTVC-N0333 Non-RCE2 Cross-check of N0281 (Rice parameter extension for transform-skip blocks) [J. Min, S. Lee (Samsung)] [late]

JCTVC-N0042 Non-RCE2: Restriction on the Residual DPCM block size [J. Sole, R. Joshi, M. Karczewicz (Qualcomm)]
JCTVC-N0279 Cross-check of JCTVC-N0042 on Restriction on the Residual DPCM block size [E. François (Canon)] [late]
JCTVC-N0072 RCE2-related: Variants of simplified sample-based weighted prediction [P. Amon, A. Hutter (Siemens), E. Wige, A. Kaup (Universität Erlangen-Nürnberg)]

JCTVC-N0364 Non-RCE2: A cross-verification report of N0072 [R. Joshi (Qualcomm)] [late] [miss]
JCTVC-N0075 Non-RCE2: Complexity reduction for inter residual DPCM in lossless coding [M. Naccari, M. Mrak (BBC)]
JCTVC-N0293 Cross-check of complexity reduction for inter residual DPCM in lossless coding (JCTVC-N0075) [J. Xu (Microsoft)] [late]
JCTVC-N0079 Non-RCE2: Simplified sample based intra prediction for lossless coding [J.Zhu, W. Zheng, K. Kazui (Fujitsu)]
JCTVC-N0367 Non-RCE2: cross-check of simplified sample based intra prediction for lossless coding (JCTVC-N0079) [K. Kawamura, S. Naito (KDDI)] [late]
JCTVC-N0080 Non-RCE2: Skip of neighbouring samples filtering in intra prediction for lossless coding [J. Zhu, K. Kazui (Fujitsu)]
JCTVC-N0100 Non-RCE2: Unified lossless residual coding [Y. H. Tan, C. Yeo (I2R)]
JCTVC-N0321 Cross-check for JCTVC-N0100: Non-RCE2: Unified lossless residual coding [M. Naccari, M. Mrak (BBC)] [late]
JCTVC-N0137 Non-RCE2: Golomb-Rice parameter initialization for transform-skip and transquant-bypass modes [V. Kolesnikov, C. Rosewarne, M. Maeda (Canon)]
JCTVC-N0350 Non-RCE2: Cross-verification of JCTVC-N0137, Golomb-rice parameter initialization for transform-skip and transquant-bypass modes [R. Cohen (MERL)] [late]
JCTVC-N0176 Non-RCE 2: On sample adaptive intra prediction for oblique modes in lossless coding [H. Chen, A. Saxena, F. Fernandes (Samsung)]
JCTVC-N0319 Crosscheck of sample adaptive intra prediction for oblique modes in lossless coding (JCTVC-N0176) [D.-K. Kwon (TI)] [late]
JCTVC-N0177 Non-RCE 2: On sample adaptive intra prediction for oblique modes in lossy coding [A. Saxena, H. Chen, F. Fernandes (Samsung)]
JCTVC-N0363 Non-RCE2: A cross-verification report of N0177 [R. Joshi (Qualcomm)] [late]
JCTVC-N0181 Non-RCE2: Rice Parameter Initialization [M. Karczewicz, J. Sole, R. Joshi (Qualcomm)]
JCTVC-N0222 Non-RCE2: Results for combination of methods [J. Sole, R. Joshi, L. Guo, M. Karczewicz (Qualcomm)]
JCTVC-N0353 Cross-check for JCTVC-N0222, Non-RCE2: Results for combination of methods [M. Naccari, M. Mrak (BBC)] [late]
JCTVC-N0232 Non-RCE2: Rice parameter update method [J. Min, S. Lee, C. Kim (Samsung)]
JCTVC-N0325 Non-RCE2: Cross-verification of JCTVC-N0232 Rice parameter update method [L. Guo (Qualcomm)] [late]
JCTVC-N0258 Non-RCE2: Extension of TU-Based Inter RDPCM [C. Pang, J. Sole, R. Joshi, M. Karczewicz (Qualcomm)]
JCTVC-N0342 Cross-check of JCTVC-N0258 on TU-Based inter RDPCM extension [H. Yang (Huawei)] [late]
JCTVC-N0281 Non-RCE2 Rice parameter extension for transform-skip blocks [S. H. Kim, K. Misra, A. Segall (Sharp)]
JCTVC-N0333 Non-RCE2 Cross-check of N0281 (Rice parameter extension for transform-skip blocks) [J. Min, S. Lee (Samsung)] [late]
JCTVC-N0288 Non-RCE2: Transform skip on large TUs [X. Peng, J. Xu (Microsoft), L. Guo, J. Sole, M. Karczewicz (Qualcomm)]
JCTVC-N0335 Cross-check for JCTVC-N0288: Non-RCE2: Transform skip on large TU [M. Naccari, M. Mrak (BBC)] [late]
JCTVC-N0167 Transform skip based on minimum TU size [Kwanghyun Won, Seungha Yang, Byeungwoo Jeon (SKKU)] [late]
JCTVC-N0289 Cross-check of transform skip based on minimum TU size (JCTVC-N0167) [J. Xu (Microsoft)] [late]
JCTVC-N0113 Cross Residual DPCM for HEVC lossless coding [Yung-Lyul Lee, Sung-Wook Hong]

6.1.4RCE3 related (intra coding for screen content)

JCTVC-N0169 Non-RCE3:Template-based palette prediction [Wenjing Zhu, Haitao Yang (Huawei)]
JCTVC-N0351 Cross-check of template-based palette prediction (JCTVC-N0169) [J. Xu (Microsoft)] [late]
JCTVC-N0206 Non-RCE3: Intra motion compensation with variable length intra MV coding [D.-K. Kwon, M. Budagavi (TI)]
JCTVC-N0348 Non-RCE3: Cross-check of Intra Motion Compensation in JCTVC-N0206 [W.-S. Kim (Qualcomm)] [late]
JCTVC-N0235 Non-RCE3: base color merging for MBCIM [J. Xu, A. Tabatabai (Sony)]
JCTVC-N0323 Non-RCE3: Cross-check of JCTVC-N0235 base color merging for MBCIM [L.Guo (Qualcomm)] [late]
JCTVC-N0249 Non-RCE3: Modified Palette Mode for Screen Content Coding [L. Guo, M. Karczewicz, J. Sole, R. Joshi (Qualcomm)]
JCTVC-N0332 Non-RCE3: Cross-check of N0249 (Modified Palette Mode for Screen Content Coding) [J. Min, S. Lee (Samsung)] [late]
JCTVC-N0254 Non-RCE3: Pipeline Friendly Intra Motion Compensation [C. Pang, J. Sole, L. Guo, R. Joshi, M. Karczewicz (Qualcomm)]
JCTVC-N0360 Cross-check of JCTVC-N0254 Table 10 and 13 on pipeline friendly Intra motion compensation [J. Xu (Sony)] [late]
JCTVC-N0376 Non-RCE3:Cross-check of Table 11 and 14 from N0254 ( Pipeline Friendly Intra Motion Compensation) [J. Min, S. Lee (Samsung)] [late]
JCTVC-N0377 Non-RCE3: Cross-check of N0254 on Table11 and Table14 (Pipeline Friendly Intra Motion Compensation) [X. Wei, J. Zan (Huawei)] [late] [miss]
JCTVC-N0256 Non-RCE3: 2-D MV Supported Intra Motion Compensation [C. Pang, J. Sole, L. Guo, R. Joshi, M. Karczewicz (Qualcomm)]
JCTVC-N0340 Non-RCE3: Cross-check of N0256 (Intra Motion Compensation with 2-D MVs) [J. Min, E. Alshina (Samsung)] [late]
JCTVC-N0285 Non-RCE3: Intra motion compensation for screen contents [J. Min, M. W Park, S. Lee, C. Kim (Samsung)]
JCTVC-N0341 Non-RCE3: Crosscheck for JCTVC-N0285 Intra motion compensation for screen contents [C. Pang (Qualcomm)] [late]
JCTVC-N0231 AHG 8: Intra mode coding for screen contents [J. Min, S. Lee, C. Kim (Samsung)]
JCTVC-N0322 AHG 8 Cross-check for JCTVC-N0231: Intra mode coding for screen contents [M. Naccari, M. Mrak (BBC)] [late]

6.1.5Transforms and transform coefficient coding

JCTVC-N0138 AHG5: Square transform deblocking for 4:2:2 [C. Rosewarne, V. Kolesnikov, M. Maeda (Canon)]
JCTVC-N0192 AHG 5: 32x32 Scaling List Derivation for Chroma [K. Sharman, N. Saunders, J. Gamei (Sony)]

6.1.6Intra prediction

JCTVC-N0143 On Mode Dependent Intra Smoothing for Range Extension [G. Laroche, C. Gisquet, T. Poirier (Canon)]
JCTVC-N0349 Cross-check of Mode Dependent Intra Smoothing in JCTVC-N0143 [W.-S. Kim (Qualcomm)] [late]
JCTVC-N0183 Non-RCE 2: Enhanced angular intra prediction for screen content coding [H. Chen, A. Saxena, F. Fernandes (Samsung)]
JCTVC-N0358 Cross-check for JCTVC-N0183 Non-RCE2: Enhanced angular intra prediction for screen content coding [M. Naccari, M. Mrak (BBC)] [late]

6.1.7High bit depth

JCTVC-N0142 AHG18: On 16-bits support for Range Extensions [E. François, J. Taquet (Canon)]
JCTVC-N0188 AHG 5 and 18: Internal Precision for High Bit Depths [K. Sharman, N. Saunders, J. Gamei (Sony)]
JCTVC-N0369 AHG 5 and 18: Cross-check of Internal Precision for High Bit Depths (JCTVC-N0188) by Sony [C. Rosewarne, M. Maeda (Canon)] [late] [miss]
JCTVC-N0189 AHG 5 and 18: Entropy Coding Compression Efficiency for High Bit Depths [K. Sharman, N. Saunders, J. Gamei (Sony)]
JCTVC-N0338 Cross-check report for JCTVC-N0189: AHG 5 and 18: Entropy Coding Compression Efficiency for High Bit Depths [Seung-Hwan Kim, Andrew Segall (Sharp)] [late] [miss]
JCTVC-N0190 AHG 5 and 18: Entropy Coding Throughput for High Bit Depths [K. Sharman, N. Saunders, J. Gamei (Sony)]
JCTVC-N0201 SAO extension for higher bit-depth coding [Alexis Tourapis (??)]
JCTVC-N0253 Cross-check of JCTVC-N0201 on SAO extension for higher bit-depth coding [J. Xu, A. Tabatabai (Sony)] [late]
JCTVC-N0275 AHG18: Modified scaling factor for transform-skip blocks to support higher bit depths greater than equal to 14 [S. H. Kim, K. Misra, A. Segall (Sharp)]
JCTVC-N0336 Cross check AHG 5 and 18: Entropy Coding Throughput for High Bit Depths [Wei Pu, Woo-Shik Kim] [late] [miss]

6.1.8Lossless and screen content coding related contributions

JCTVC-N0115 On RGB to YCbCr conversion for screen contents [A. Minezawa, S. Sekiguchi, T. Murakami (Mitsubishi)] [late]

6.1.9Other

See also N0145 regarding chroma format.

JCTVC-N0141 AHG5: on chroma QP for HEVC Rext [E. François, C. Gisquet, G. Laroche, P. Onno (Canon)]
JCTVC-N0295 AhG5: Cross-check of Chroma QP for HEVC RExt in JCTVC-N0141 [W.-S. Kim (Qualcomm)] [late]
JCTVC-N0116 AHG5/AHG8: RGB4:4:4 video coding using HEVC multi-view extensions [A. Minezawa, S. Sekiguchi, T. Murakami (Mitsubishi)] [late]
JCTVC-N0148 AhG8: Guided Image Filtering for Screen Content Coding [T. Vermeir (Barco), J. De Cock, G. Van Wallendael, S. Van Leuven (Ghent University - iMinds)]
JCTVC-N0246 AHG5: Modified SAO for range extensions [S.-T. Hsiang, C.-M. Fu, Y.-W. Huang, S. Lei (MediaTek)]
JCTVC-N0331 AHG 5: Cross-check of N0246 (Modified SAO for range extensions) [J. Min, E. Alshina (Samsung)] [late]
JCTVC-N0261 AhG5: Memory Bandwidth Reduction for HEVC Rext [W.-S. Kim, J. Sole, M. Karczewicz (Qualcomm)]
JCTVC-N0310 Cross check of Memory Bandwidth Reduction for HEVC RExt (JCTVC-N0261) [G. Laroche (??)] [late]
JCTVC-N0263 AhG5: Deblocking Filter in 4:4:4 Chroma Format [W.-S. Kim, J. Sole, M. Karczewicz (Qualcomm)]
JCTVC-N0320 Crosscheck of JCTVC-N0263 on deblocking filter in 4:4:4 chroma format [D.-K. Kwon (TI)] [late]
JCTVC-N0292 RExt: Fidelity adaptive coding mode [D. Flynn, N. Nguyen, D. He (RIM)]
JCTVC-N0309 Signalling of chroma sampling filter [K. Kazui (Fujitsu), T. Chujoh(Toshiba)] [late]