Test 1: Intra block copy and Inter signalling unification (JCTVC-T0094)
In JCTVC-S0113 and JCTVC-S0302, the IBC mode is signalled without intra_bc_flag but reusing inter signalling by adding the current picture to the reference picture list(s) instead. With the shared signalling, IBC can be differentiated from the conventional inter mode by checking the selected reference picture.
On PU IBC on/off
Test 2: Intra block copy signalled at PU level (JCTVC-T0071)
In JCTVC-S0121, the result for PU level IBC on/off on top of SCM-2.0 was provided. In this test, the performance of PU level IBC on/off is tested on SCM-3.0.
On IBC merge/skip mode
Test 3.1: IBC merge mode using HEVC merge candidates (JCTVC-T0040)
In JCTVC-S0081, merge candidates for IBC are constructed according to inter mode. In this test, the performance of this method is tested on SCM-3.0.
Test 3.2: IBC merge mode using default candidates (JCTVC-T0073)
In JCTVC-S0123, some default IBC merge candidates are added to the merge list when the merge list is not full. In this test, the performance of this method is tested on SCM-3.0.
On IBC deblocking
Test 4.1: On deblocking for screen content coding (JCTVC-T0039)
Note: 4.1 is not a harmonization
In this test, the boundary strength rule is altered as follows:
else, apply inter rules to select between BS = 1 or BS = 0.
Test 4.2: Inter deblocking for IBC (JCTVC-T0095)
In JCTVC-S0112, IBC is treated as inter mode during the deblocking process. Moreover, in order to keep the deblocking module unchanged, after an IBC block is decoded and before the deblocking process, it is converted to be a uni-directionally predictive inter block with the BV being its MV and the current picture being the reference picture.
On Vector coding
Test 5.1: Vector binarization (JCTVC-T0089)
Note: 5.1 also changes the current MVD coding of HEVC. None of the tests has used the current MVD coding method for BVD, as it was asserted this would gve considerable loss
In this test, first a context coded flag is coded to indicate MVD is zero. When MVD is not zero, another flag may be coded to indicate if absolute level of MVD is greater than N. If MVD is greater than N, the remaining absolute level is coded using bypass Exp-Golomb codes. One flag is coded to indicate sign of the vector.
(Test 5.2: IBC BV coding for small PUs withdrawn)
Test 5.3: Block vector coding for IBC (JCTVC-T0090)
In this test following aspects have been tested separately and jointly. a) One flag is added to specify whether the vector is predicted or not. If not predicted, vector is coded without prediction. b) Reducing redundancy of vectors with respect to their sign and absolute values.
The combination of 5.1+5.3 is tested and results are provided.
Implicitly, it enables merge/skip, bi-prediction from inter and intra and mixed prediction (sub-CU inter/intra) which explains some of the gain.
It is furthermore pointed out that two macros are turned on that partially contribute to the gain. (0.5–0.7%). Related to checking the correct decision for mixed prediction, involving 3 components in the vector search.
With regard to de-blocking, when constraint intra prediction flag is on, IBC goes back to intra, which could cause a contradiction. The same would be the case with test 2, which basically treats IBC as inter mode.
Restriction to 4x8 is asserted to lead to loss, which is however compensated by the other benefits.
Several experts expressed opinion that it would be desirable to achieve a true harmonization, where the current inter mode of v1 (including MV coding, binarization etc.) is used as is. It was also expressed that the advantage of such a unification would even outweigh a certain loss in compression.
Continue CE in this regard. In this context, it should also be considered to use an efficient way of coding the integer-pel MV of IBC, e.g. by signalling in quarter-pel and using the adaptive MV resolution flag at slice level. It is expressed during the discussion that a specific modification for IBC just shifting the MV after the decoding would be more acceptable than modifying the binarization or CABAC.
Test 2a/b gives information about sub-PU IBC (with intra/inter DBF). Compared to test 1, it uses 4x4 still. This also unveils that inter deblocking is giving benefit for IBC
Test 4 tests de-blocking (where the inter de-blocking 4.3 performs better than the specifically modified version 4.1)
Test 5.1 modifies the vector binarization for both MV and BV coding. It is however not a complete unification, using different context models for MV and BV. For camera-captured content, a small loss is observed in some cases (0.1% in LD)
Test 8 (5.1) uses the same context model and same binarization from 5.1 for BV and MV. It is however still performing a shift of MV as from Test 1.
The common understanding is that
Either a full unification is achieved (in which context an MV shift operation when used for IBC is acceptable, but MV coding different from v1 would not be desirable, and modificaton of AMVP/merge would also not be desirable), or
Design IBC as “third mode” beyond intra and inter prediction, and then with the usual tradeoff complexity/performance. This should branch off at the CU level (which would not qualify test 2).
With regard to merge (test 3) two concerns are raised a) it may be not well parallelizable due to 4x4 blocks and 5 candidates and b) some of the gain could be achieved by encoder only modification (see 6.1 T2).
It is requested to report results of the unified solution (as described above). If resonable (expectation is that the gain compared to current SCM would still be around 1% for RA and LD) this is likely to be adopted. See further notes under JCTVC-T0227.
Some of the other methods investigated in CE2 also give additional gain compared to SCM, however
Test 3 (4x4 merge) may have some implications wrt parallelization, and some of the gain could also be achieved by encoder-only (test 6.1T2)
Test 4.2 (using inter deblocking for IBC) obviously gives benefit, but would be implicit in the unified solution
Test 5.1 would be undesirable because it changes the MV coding (would need to be modified such that it only changes the BV coding, such that 5.3 would better); 5.3 could be another add-on over current SCM, but only gives 0.5%
Test 6.1T1 uses 5 candidates for BVP, which also raised some complexity concerns
One expert points outremarked that unification with inter would have potential further advantages such as combination with weighted prediction.
Due to the decision on unification (see notes for JCTVC-T0227), it was agreed that other aspects of CE2 did not need follow-up action.
4.2.2CE2 primary contributions (10)
JCTVC-T0039 CE2 Test 4.1: On deblocking for screen content coding [C. Rosewarne, M. Maeda (Canon)] JCTVC-T0040 CE2: results of test 3.1 and 6.1 [B. Li, J. Xu (Microsoft)] JCTVC-T0071 CE2: Test 2 – Intra BC signalled at PU level [X. Xu, S. Liu, S. Lei (MediaTek)] JCTVC-T0073 CE2: Test 3.2 – Intra BC merge mode with default candidates [X. Xu, T.-D. Chuang, S. Liu, S. Lei (MediaTek)] JCTVC-T0075 CE2: Test 7 – Combination of Test 1 and Test 3.2 [X. Xu, T.-D. Chuang, S. Liu, S. Lei (MediaTek)] JCTVC-T0089 CE2: Test 5.1 and Test 8.1 on vector binarization [K. Rapaka, M. Karczewicz, C. Pang, V. Seregin (Qualcomm)] JCTVC-T0090 CE2: Test 5.3 and Test 8.2 on block vector (BV) coding for IBC [K. Rapaka, M. Karczewicz, V. Seregin (Qualcomm), K. Miyazawa, A. Minezawa, S. Sekiguchi (Mitsubishi)] JCTVC-T0094 CE2 Test1: Intra block copy and inter signalling unification [C. Pang, Y.-K. Wang, V. Seregin, K. Rapaka, M. Karczewicz (Qualcomm), X. Xu, S. Liu, S. Lei (MediaTek), B. Li, J. Xu (Microsoft)] JCTVC-T0095 CE2 Test 4.2: Inter deblocking for intra block copy [C. Pang, V. Seregin, M. Karczewicz (Qualcomm)] JCTVC-T0096 CE2 Test 6.2: Intra block copy block vector prediction with line buffer [C. Pang, V. Seregin, M. Karczewicz (Qualcomm)]
4.2.3CE2 cross checks (12)
JCTVC-T0046 CE2: Cross-check of Test 1 on intra block copy and inter signalling unification [A. Minezawa, K. Miyazawa, S. Sekiguchi (Mitsubishi)] JCTVC-T0083 Crosscheck of CE2 Test 3.1 (JCTVC-T0040) [X. Xu, S. Liu (MediaTek)] JCTVC-T0084 Crosscheck of CE2 Test 6.1 (JCTVC-T0040) [X. Xu, S. Liu (MediaTek)] JCTVC-T0085 Crosscheck of CE2 Test 6.2 (JCTVC-T0096) [X. Xu, S. Liu (MediaTek)] [late] JCTVC-T0100 Crosscheck of CE2 Test 4.1 on intra block copy deblocking [C. Pang (Qualcomm)] [late] JCTVC-T0143 CE2 Test2: Crosscheck of Intra Block Copy Signalled at PU Level (JCTVC-T0071) [W. Zhang, L. Xu, Y. Chiu (Intel)] [late] JCTVC-T0144 CE2 Test5.3: Crosscheck of Block Vector Coding for IBC (JCTVC-T0090) [W. Zhang, L. Xu, Y. Chiu (Intel)] [late] JCTVC-T0155 Cross-check of ‘CE2 Test 4.2: Inter deblocking for intra block copy’ (JCTVC-T0095) by Qualcomm [C. Rosewarne, M. Maeda (Canon)] [late] JCTVC-T0156 Cross-check of test 5.1 of ‘CE2: Test 5.1 and Test 8.1 on vector binarization’ (JCTVC-T0089) by Qualcomm [C. Rosewarne, M. Maeda (Canon)] [late] JCTVC-T0158 CE2: Cross-check of test 3.2 (JCTVC-T0073) [B. Li, J. Xu (Microsoft)] [late] JCTVC-T0159 CE2: Cross-check of test 7 (JCTVC-T0075) [B. Li, J. Xu (Microsoft)] [late] JCTVC-T0170 CE2: Cross-check of Test 8.1 (JCTVC-T0089) and Test 8.2 (JCTVC-T0090) [P. Onno (Canon)] [late]