6.3CE3: Intra prediction and mode coding (38)
Contributions in this category were discussed Wednesday 11 July 1140–1800 (chaired by GJS).
JVET-K0023 CE3: Summary Report on Intra Prediction and Mode Coding [G. . Van der Auwera, J. . Heo, A. . Filippov]
The goal of CE3 is to study intra prediction tools including mode coding for the VVC standard. There are 7 sub-tests defined. In CE3.1 tests are included targeting DC, planar, directional, and additional modes. CE3.2 tests reference sample filtering, interpolation, across boundary filtering and prediction sample filtering. CE3.3 targets intra mode coding, such as most probable mode list variations. In CE3.4 cross-component linear model and variations are tested. CE3.5 focuses on multi-reference line intra prediction tests and CE3.6 on non-linear intra prediction. In CE3.7 tests are performed regarding bidirectional intra prediction.
The following is the list of defined sub-tests in CE3:
-
CE3.1: Intra modes
-
CE3.2: Intra filtering and interpolation
-
CE3.3: Intra mode coding
-
CE3.4: Cross-component linear model (CCLM)
-
CE3.5: Multi-reference line intra prediction
-
CE3.6: Non-linear intra prediction
-
CE3.7: Bidirectional intra prediction
The CE3 description originally defined 75 tests of which 7 were withdrawn and one redefined. This document summarizes the objective results (BD-rates, runtimes), cross-check reports and related input contributions.
Comments from cross-checkers were copied verbatim into the summary report.
CE3.1 on ‘Intra modes’
Test#
|
Short description
|
Doc. #
|
1.1.1
|
Use 129 directional modes for all blocks
|
JVET-K0060
(Qualcomm)
|
1.1.2
|
Use variable number of directional modes (33, 65, or 129) depending on block size comparison with two SPS thresholds
|
1.2.1
|
DC mode with only shift operators
|
JVET-K0211
(Panasonic)
|
1.3.1
|
Wide-angle prediction
|
JVET-K0046
(Nokia)
|
1.4.1
|
Usage of line-based intra prediction mode
|
JVET-K0049
(HHI)
|
1.4.2
|
Fast Line-based intra prediction mode
|
1.4.3
|
Constrained Line-based intra prediction mode
|
1.5.1
|
Unequal weighted planar prediction (UWP)
|
JVET-K0055
(ARRIS, Sharp)
|
CE3.1: ‘All Intra Main10’ results
|
|
All Intra Main10 – Over VTM1.0
|
All Intra Main10 – Over BMS1.0
|
Test#
|
Description
|
Y
|
U
|
V
|
EncT
|
DecT
|
Y
|
U
|
V
|
EncT
|
DecT
|
1.1.1
|
Use 129 directional modes for all blocks
|
−1.30%
|
−1.43%
|
−1.42%
|
119%
|
101%
|
0.04%
|
−0.10%
|
−0.13%
|
102%
|
100%
|
1.1.2
|
Use variable number of directional modes (33, 65, or 129) depending on block size
|
−1.44%
|
−1.45%
|
−1.48%
|
116%
|
101%
|
−0.14%
|
−0.13%
|
−0.18%
|
101%
|
101%
|
1.2.1
|
DC mode with only shift operators
|
0.01%
|
0.04%
|
0.01%
|
100%
|
99%
|
0.01%
|
0.09%
|
0.02%
|
100%
|
99%
|
1.3.1
|
Wide-angle prediction
|
−0.28%
|
−0.38%
|
−0.42%
|
106%
|
101%
|
−0.23%
|
−0.23%
|
−0.30%
|
102%
|
98%
|
1.4.1
|
Usage of line-based intra prediction mode
|
−2.34%
|
−2.15%
|
−2.47%
|
293%
|
120%
|
−0.70%
|
−0.99%
|
−1.15%
|
125%
|
105%
|
1.4.2
|
Fast line-based intra prediction mode
|
−2.00%
|
−1.83%
|
−2.09%
|
164%
|
112%
|
−0.35%
|
−0.55%
|
−0.57%
|
106%
|
102%
|
1.4.3
|
Constrained line-based intra prediction mode
|
−1.58%
|
−1.74%
|
−1.98%
|
146%
|
110%
|
−0.26%
|
−0.39%
|
−0.49%
|
105%
|
102%
|
1.5.1
|
Unequal weighted planar prediction (UWP)
|
−0.20%
|
−0.03%
|
−0.06%
|
101%
|
103%
|
−0.53%
|
−0.53%
|
−0.53%
|
100%
|
102%
|
Focus on 1.2.1: There is a division by width + height, which is bad for non-square blocks. This proposal fixes that and has no coding loss, which seems ripe for action pending review of non-CE inputs K0122 and K0400. [Resolved per notes elsewhere]
Focusing on 1.4.1: This has the most coding gain in the table, but has an increase in encoder complexity (requiring a line-by-line operation similar to a 1-D transform). Test 1.4.2 has a reduced-complexity encoding with the same syntax and decoding process. It has an increase in decoding complexity and from the encoder perspective, there is an extra switch to evaluate. It was commented that 1.4.1 has substantial complexity.
Focusing on 1.3.1: It was commented that there is no significant complexity associated with that. It does have an extra switch bit and two variations to support in the decoder side. There are non-CE inputs on that. Those should be reviewed.
It was noted that these are AI results, whereas the impact of these techniques is roughly cut in half for RA conditions.
Focus on the BMS: It was suggested that we should consider adopting the 67 modes that are in the BMS. This question was deferred to after consideration of the mode coding.
CE3.2 on ‘Intra filtering and interpolation’
Test #
|
Short Description
|
Doc. #
|
2.1.1
|
Bilateral reference sample filter
|
JVET-K0061
(Qualcomm)
|
2.2.1
|
4-tap cubic filter for extending reference samples
|
JVET-K0062
(Qualcomm)
|
2.2.2
|
Bilinear interpolation for extending reference samples
|
JVET-K0211
(Panasonic)
|
2.3.1
|
Interpolation filter selection between 6-tap cubic and 4-tap Gaussian filter based on block size
|
JVET-K0062
(Qualcomm)
|
2.3.2
|
Combine tests 2.2.1 + 2.3.1
|
2.3.3
|
Interpolation filter selection between 4-tap cubic and 4-tap Gaussian filter based on intra prediction mode and block size
|
JVET-K0097
(LGE)
|
2.3.4
|
Interpolation filter selection between 6-tap cubic and 4-tap Gaussian filter based on intra prediction mode and block size
|
2.4.1
|
Simplified position dependent intra prediction combination (PDPC)
|
JVET-K0063
(Qualcomm)
|
2.5.1
|
Bilinear interpolation for projection and smoothing after projection
|
JVET-K0211
(Panasonic)
|
2.6.1
|
6-tap combined filter without reference sample smoothing
|
JVET-K0165
(ETRI)
|
2.7.1
|
Bilateral reference sample filter
|
JVET-K0043
(HHI)
|
2.8.2
|
Mode dependent de-ringing filter based on bitstream flag
|
JVET-K0066
(Huawei)
|
2.9.1
|
Intra boundary filters
|
JVET-K0240
(MediaTek)
|
2.10.1
|
Multiple 4-tap filter
|
JVET-K0179
(Samsung)
|
2.11.1
|
Multi-combined intra prediction
|
JVET-K0180
(Samsung)
|
CE3.2: ‘All Intra Main10’
|
|
All Intra Main10 – Over VTM1.0
|
All Intra Main10 – Over BMS1.0
|
Test #
|
Description
|
Y
|
U
|
V
|
EncT
|
DecT
|
Y
|
U
|
V
|
EncT
|
DecT
|
2.1.1
|
Bilateral reference sample filter
|
0.03%
|
−0.02%
|
−0.04%
|
100%
|
101%
|
−0.13%
|
−0.16%
|
−0.17%
|
100%
|
99%
|
2.2.1
|
4-tap cubic filter for extending reference samples
|
−0.05%
|
−0.08%
|
−0.11%
|
102%
|
100%
|
−0.01%
|
−0.04%
|
−0.03%
|
100%
|
99%
|
2.2.2
|
Bilinear interpolation for extending reference samples
|
−0.05%
|
−0.08%
|
−0.08%
|
101%
|
100%
|
0.00%
|
0.01%
|
−0.03%
|
100%
|
99%
|
2.3.1
|
Interpolation filter selection between 6-tap cubic and 4-tap Gaussian filter based on block size
|
−0.57%
|
−0.39%
|
−0.43%
|
107%
|
103%
|
−0.44%
|
−0.48%
|
−0.53%
|
101%
|
101%
|
2.3.2
|
Combine tests 2.2.1 + 2.3.1
|
−0.63%
|
−0.49%
|
−0.48%
|
107%
|
103%
|
−0.46%
|
−0.58%
|
−0.58%
|
102%
|
100%
|
2.3.3
|
Interpolation filter selection between 4-tap cubic and 4-tap Gaussian filter based on intra prediction mode and block size
|
−0.54%
|
−0.41%
|
−0.46%
|
106%
|
103%
|
−0.40%
|
−0.41%
|
−0.43%
|
106%
|
104%
|
2.3.4
|
Interpolation filter selection between 6-tap cubic and 4-tap Gaussian filter based on intra prediction mode and block size
|
−0.58%
|
−0.39%
|
−0.43%
|
107%
|
102%
|
−0.44%
|
−0.57%
|
−0.57%
|
101%
|
101%
|
2.4.1
|
Simplified position dependent intra prediction combination (PDPC)
|
−0.97%
|
−0.14%
|
−0.01%
|
109%
|
107%
|
−1.35%
|
−0.94%
|
−0.81%
|
103%
|
105%
|
2.5.1
|
Bilinear interpolation for projection and smoothing after projection
|
−0.01%
|
−0.11%
|
−0.13%
|
102%
|
100%
|
0.02%
|
0.01%
|
−0.04%
|
100%
|
99%
|
2.6.1
|
6-tap combined filter without reference sample smoothing
|
−0.55%
|
−0.01%
|
0.02%
|
108%
|
102%
|
−0.39%
|
−0.11%
|
−0.16%
|
101%
|
101%
|
2.7.1
|
Bilateral reference sample filter
|
−0.22%
|
−0.11%
|
−0.14%
|
101%
|
101%
|
−0.24%
|
−0.20%
|
−0.24%
|
101%
|
102%
|
2.8.2
|
Mode dependent de-ringing filter based on bitstream flag
|
−0.40%
|
−0.26%
|
−0.32%
|
116%
|
100%
|
−0.40%
|
−0.37%
|
−0.39%
|
115%
|
100%
|
2.9.1
|
Intra boundary filters
|
−0.75%
|
−0.69%
|
−0.75%
|
104%
|
102%
|
−0.87%
|
−0.55%
|
−0.59%
|
103%
|
106%
|
2.10.1
|
Multiple 4-tap filter
|
−0.41%
|
−0.19%
|
−0.16%
|
112%
|
103%
|
−0.13%
|
−0.11%
|
−0.16%
|
103%
|
102%
|
2.11.1
|
Multi-combined intra prediction
|
−0.12%
|
−0.04%
|
−0.06%
|
196%
|
100%
|
0.01%
|
0.04%
|
0.00%
|
212%
|
101%
|
Focus on 2.4.1: This has some complexity, but it was suggested that the gain (relative to VTM: 0.97% in AI, 0.48% in RA) is worth that. Some participants suggested instead using the intra boundary filter approach of 2.9.1, which is asserted to be less complex. In spirit, PDPC is a merging of boundary filtering with the intra prediction process into a single formula. Decision: Adopt PDPC (per K0063).
Focus on 2.3.3: It was remarked that 2.3.1 and 2.3.4 are conceptually similar and there may be some relationship with 2.6.1. Basically it was remarked that 2.3.3 appears to be a good starting point by having a shorter filter. A participant said they had experimented with 2.3.3 combined with PDPC and there was not a diminishment of the gain. It was agreed to further study this in a CE.
Dostları ilə paylaş: | 
