Joint Collaborative Team on Video Coding (jct-vc) of itu-t sg16 wp3 and iso/iec jtc1/SC29/WG11

5.2RCE2: Prediction and coding techniques for transform-skip blocks

5.2.1RCE2 summary and general discussion

JCTVC-N0035 RCE2: Summary report on HEVC Range Extensions Core Experiment on prediction and coding techniques for transform-skip and transform-bypass blocks [R. Joshi, P. Amon, R. Cohen, S. Lee, M. Naccari] [late]

(Reviewed in Track A Sun. 28th p.m. (GJS).)

This is a summary report on HEVC Range Extensions Core Experiment 2 on prediction and coding techniques for transform-skip and transform-bypass blocks. The core experiment investigated prediction and coding techniques for transform-skip and transform-bypass blocks. The core experiment consisted of subtests that investigated update of the Rice parameter for level coding, rotation, flipping or other scan changes and replacement of specific intra prediction modes. Performance of the proposed methods as well as their combinations was evaluated for lossy and lossless configurations based on the test conditions and sequences described in JCTVC-M1122.

The CE contains:

• 
  Subtest A: Rice parameter modifications (3 methods).
  
• 
  Subtest B: Rotation, flipping, or scan changes.
  
• 
  Subtest C: Replacement of specific intra prediction modes (RDPCM for intra for lossy, RDPCM for inter for lossy and lossless, replacing DC mode with something else, and sample-adaptive weighted prediction in place of the planar mode).
  
• 
  Subtest D: Combinations within subtest C.
  

In subtest A, the focus is on performance for lossless (and near-lossless), intra, and screen content. Especially lossless. A.3 seems to perform a little bit better than A.2. A.1 has higher complexity than A.2 and A.3 (but more gain). The complexity impact (esp. for A.2 and A.3) is relatively low.

It was remarked that Rice parameter changes can be especially helpful with higher bit depth. It was noted that we could have profile-based restrictions – e.g., for CABAC changes to only be in effect for profiles with higher bit depths.

It we take action on this, it would probably be A.3.

For subtest B, there are three variants.

In subtest B, the focus is on performance for screen content for lossy coding (and it also helps on lossless). B.2 is simpler than the other two (but has less gain).

The lossy is only at 4x4.

It is also suggested to consider only applying the modification to smaller blocks (perhaps the same block size for which lossy coding applies).

It was suggest to focus on the variant with the extra context.

Lossless:

Techniques B.1 and B.3 do better here than technique B.2 – both for lossy and lossless.

Lossy:






It was agreed that some action should be taken on this topic.

It was noted that it is proposed to enable transform skip for non-4x4 block sizes, and it is proposed to enable RDPCM for lossy coding. Some participants suggested considering the interactions.

Subtest C considers RDPCM for intra for lossy, RDPCM for inter for lossy and lossless, replacing DC mode with something else, and sample-adaptive weighted prediction in place of the planar mode).

Regarding CU versus TU level operation, it was suggested that CU level is easier for the encoder. However, TS is a TU level decision and it was suggested to keep the decision at the TU level.

C.1 is only for inter, both for lossy and lossless. C.2 extends our current RDPCM to the lossy case (for intra). So these are essentially independent. Combining C.1 and C.2 is subtest D.1.

C.1 shows benefit (in YUV) only for SC – and only for LD, because it is only for inter, and the bits are allocated mostly to intra. There might be gain if applied to other block sizes – but 4x4 only for now.

C.2 is also only beneficial for SC.

If C.2 is in the design, C.1 might have no added complexity other than syntax flag parsing on some architectures.

Decision: Adopt C.2. Also adopt C.1. (This was pending that C.1 is not the only change to CABAC parsing, and it isn't.)

(Further discussion Monday 29th p.m. (GJS).)

Decision: Need an enable flag for each feature. There should be an enable/disable flag at the SPS level for RDPCM – one flag for intra and one for inter.

It was noted that RDPCM disables sign data hiding for the blocks that use that mode.

C.3 has a 4-pixel neighbourhood average with focus on lossless (more complex), replacing the planar mode.

This is interesting, but no action taken (since this is only for lossless and has significant complexity).

C.4 is a DC mode replacement, using a parallelized neighborhood-based average. It is both for lossless and lossy, and somewhat higher complexity than RDPCM (esp. for encoder in lossy case).

Primarily for screen content.

On top of C.2, the gain of C.4 is less – about 1% in AI for YUV 444.

No action due to reduced benefit relative to added complexity when considered on top of C.2.

In N0207, B.3 (with extra context) was tested on top of C2+C1, and the gain was mostly (but not completely) additive.

In N0222, C2 + C1 + A.3 + B.1 + TS8x8 + N0181 (Rice param change) was tested.

Discussion of combinations to test and further discuss relative to C2, for Class F and YUV 444 (Discussed Thu 1st (GJS)):

• 
  + C1 = D1 (RDPCM stuff, already have these results):
  

• 
  + A.3 (Rice parameter stuff) Conclusion: Essentially additive.
  
• 
  + special context Conclusion: Essentially additive.
  
• 
  B variations (mirroring/flipping)
  
  • 
    B.1 rotation of residue (lesser of the more complex two)
    
  • 
    B.2 swapping of vert/horiz scans (simplest)
    
  • 
    B.3 mirroring of the scan (most complex)
    

Decision: D1 + context + B.1 (4×4 only).

Defer A.3 to further study CE (with other stuff).

5.2.2RCE2 primary contributions

JCTVC-N0044 RCE2 Test B.1: Residue rotation and significance map context [J. Sole, R. Joshi, M. Karczewicz (Qualcomm)]
JCTVC-N0052 RCE2 subtest C.2: Extension of residual DPCM to lossy coding [R. Joshi, J. Sole, M. Karczewicz (Qualcomm)]
JCTVC-N0053 RCE2 subtest C.4: Replacement of DC mode for transform-skip and transform-bypass blocks [R. Joshi, J. Sole, M. Karczewicz (Qualcomm)]
JCTVC-N0071 RCE2: Simplified sample-based weighted prediction (Test C.3) [P. Amon, A. Hutter (Siemens), E. Wige, A. Kaup (Universität Erlangen-Nürnberg)]
JCTVC-N0073 RCE2: Description and report for Test D.3 [E. Wige, A. Kaup (Universität Erlangen-Nürnberg), P. Amon, A. Hutter (Siemens), R. Joshi, J. Sole, M. Karczewicz (Qualcomm)]
JCTVC-N0074 RCE2: Experimental results for Test C.1 [M. Naccari, M. Mrak (BBC)]
JCTVC-N0083 RCE2 subtest D.1: Extension of intra and inter residual DPCM to lossy coding [R. Joshi, J. Sole, M. Karczewicz (Qualcomm), M. Naccari, M. Mrak (BBC)]
JCTVC-N0096 RCE2: Test B.2: Swapping of scan order for transform skip/bypass [R. Cohen, A. Vetro (MERL)]
JCTVC-N0101 RCE2 Test A.3: Simplified update of the coefficient level Rice parameter [J. Sole, M. Karczewicz (Qualcomm)]
JCTVC-N0175 RCE2 Test B.3: Mirroring of Coefficients and significance map context [R. Weerakkody, M. Mrak (BBC)]
JCTVC-N0207 RCE2: Combination of Mirroring of Coefficients (Test B.3), Significance map context (Test B.1) and RDPCM (Test D.1) [R. Weerakkody, M. Mrak, M.Naccari (BBC)]
JCTVC-N0228 RCE2: Results for Test A.2 Rice parameter update [M. Budagavi (TI)]
JCTVC-N0268 RCE2 Test A.1: Rice parameter signaling for transform-skip blocks [S. H. Kim, A. Segall (Sharp)]
JCTVC-N0354 Non-RCE2: Results for combination of Mirroring of non-transformed residual blocks (Test B.3) with other methods [R. Weerakkody, M. Mrak, M. Naccari (BBC)] [late]

5.2.3RCE2 cross checks

JCTVC-N0076 RCE2: Cross-check for Test B.2 [M. Naccari, M. Mrak (BBC)]
JCTVC-N0097 RCE2: Cross-verification of JCTVC-N0044, Test B.1: Residue rotation and significance map context [R. Cohen (MERL)]
JCTVC-N0098 RCE2: Cross-verification of JCTVC-N0073, Description and report for Test D.3 [R. Cohen (MERL)]
JCTVC-N0099 RCE2: Cross-verification report for subtest B.3 (N0175) on mirroring of coefficients [R. Joshi (Qualcomm)]
JCTVC-N0103 RCE2: Cross-verification of Test C.1 [S. Lee, C. Kim (Samsung)] [late]
JCTVC-N0105 RCE2: Cross-check of Test A.2 on coefficient level Rice parameter update (JCTVC-N0228) [J. Sole (Qualcomm)]
JCTVC-N0123 RCE2: Cross-check of Test A.2 [X. Wei, J. Zan (Huawei)] [late]
JCTVC-N0136 RCE2: Cross-check of Sub-test D.1 (JCTVC-N0083) by the BBC and Qualcomm [C. Rosewarne, M. Maeda (Canon)] [late]
JCTVC-N0156 RCE2: Crosscheck for Test C.3 [W. Zhang, Y. Chiu (Intel)] [late]
JCTVC-N0164 RCE2: Cross-check results for Test C.4 (JCTVC-N0053) [E. Wige, A. Kaup (Universität Erlangen-Nürnberg), P. Amon, A. Hutter (Siemens)]
JCTVC-N0328 Cross-check report for RCE2 C2 [J. Ye, Z. Ma, M. Xu (Huawei)] [late]
JCTVC-N0329 Cross-check report for RCE2 A1 [J. Ye, Z. Ma, M. Xu (Huawei)] [late]
JCTVC-N0330 Cross-check report for RCE2 B3 [J. Ye, Z. Ma, M. Xu (Huawei)] [late]
JCTVC-N0337 RCE2: Cross-check report for Test A3 [S.-H. Kim, A. Segall (Sharp)] [late]
JCTVC-N0370 A Cross-Check report of N-0354 [H. Chen, A. Saxena, F. Fernandes (Samsung)] [late]
JCTVC-N0365 Cross-check for JCTVC-N0207: RCE2: Combination of Mirroring of Coefficients (Test B.3), Significance map context (Test B.1) and RDPCM (Test D.1) [K. Sharman, N. Saunders, J. Gamei (Sony)] [late]