International organisation for standardisation organisation internationale de normalisation

4.5CE5: Transform skipping

4.5.1Summary

4.5.1.1.1.1.1.1.1JCTVC-H0035 CE5: Summary Report of Core Experiment on Transform Skipping [M. Mrak, J. Sole, J. Xu, A. Saxena (CE coordinators)]

Two types of transform skipping were studied:

• 
  Transform skipping on intra coded blocks, namely Residual Scalar Quantization (RSQ), presented in JCTVC-E145, and
  
• 
  Transform skipping on inter coded blocks, namely Transform Skip Mode (TSM), presented in JCTVC-G1003.
  

Test a: Quantization (adapting the scaling factors). Without this, there are slight losses by transform skipping. When invoked, the gain is 0.4% for RA and 0.7% in LD (excluding class F); higher in class F (1.8/4%).

When using quant matrices (and compared to the case of using default matrices without TS), the gain is 0.7/0.8% for RA and LD.

Encoder runtime increase by TS is approximately 30-35%.

Test b: If restricted to 4x4 blocks, encoder runtime increase is 8%, and the gain is 0.2/0.2% for RA and LD cases.

Test c: Mode dependent scans: no gain demonstrated.

Test d: Coding the last coefficient in bypass mode does not affect coding efficiency for default conditions, and decreases bit rate by approximately 0.2% for class F. Note: For screen content (class F), a large number of blocks (up to 70%) uses transform skipping.

Question: Are the horizontal and vertical modes useful? These were reported to be used in majority of cases for camera content (approximately 50% no skip, 20% vert/hor, 10% full skip). For screen content, approximately 30% no skip, and percentage full skip increases.

Test e: Withdrawn.

Test f: Intra coding (RSQ): Only a choice of 2D or no transform, and the usual prediction modes and mode-dependent scans are used. The deblocking filter is disabled when one of the blocks is fully skipped. No gain was shown for common test conditions, 8% gain for class F in AI-HE mode. If modified contexts are used for the skip mode, 0.2% gain was measured in common conditions, and 10% in class F. If the deblocking filter is used as conventionally applied, the gain in class F all-intra is reduced to 6.5%.

Test g: Combination of tests a-d. Compared to HM, 0.4/0.6% gain for common test conditions, 2.1/4.7% for class F.

Transform skipping does not give a significant benefit under common testing conditions, and would be a specific tool for screen content (class F) – note: Class F sequences may be less challenging for the inter case.

A thorough investigation of the complexity impact is missing. Proponents argue that complexity is decreased but in fact it is increasing the decoder complexity (as it adds additional logic for switching the transform and the scaling for de-quantization).

Some concern was expressed for hardware implementations, particularly about the horizontal/vertical options. Frequent switching of the de-quantization operation is also of concern.

For inter coding, the version which restricts transform skipping to 4x4 blocks (only changing transform and scaling) seems to be most beneficial, but this still requires hor/vert switching and a change of scaling/dequant.

In regard to related non-CE contributions: H0061 uses only full skip for inter, H0141 is reducing the complexity for scaling/dequant.

It was said that if the transform is fully skipped, the change in dequantization can be implemented by a shift operation (further clarification was requested whether this resolves the concern above).

The relationship with lossless coding was discussed. In that case, the 2D transform would be skipped, but with no change in de-quantization (just skipping it) – this would be more acceptable.

A request was also made to investigate subjective quality.

No support was expressed by other companies except proponents, and some concern was expressed about the complexity impact and that the benefit of this tool is highly sequence dependent.

Conclusion: The additional complexity implication of the encoder and decoder would not justify the benefit it gives for specific class F sequences.