Joint Collaborative Team on Video Coding (jct-vc) of itu-t sg16 wp3 and iso/iec jtc1/SC29/WG11

4.2CE2: Adaptive loop filtering

4.2.1Summary

JCTVC-I0022 CE2: Summary report of Core Experiment on Adaptive Loop Filtering [T. Yamakage, M. Budagavi, I. S. Chong, M. Narroschke, Y.-W. Huang]

Subtest a: Signalling APS mode vs. interleaved mode. Problem: Current HM signals the LCU related ALF data in the slice header, which introduces latency.

In APS mode, 1 or 16 sets of filter coefficients are transmitted once per picture (coefficients determined from previous picture):

• 
  Option 1 – interleaving (i.e. 1 set of filter coefficients encoded with LCU data)
  
• 
  Option 2 – 1 or 16 sets with APS, enabling/select with LCU data
  
• 
  Option 3 – same as 2, but additional option to encode 1 set of individual coefficients with LCU.
  

Option 1 is always using the same filter

Options 2 and 3 can be operated in RA and BA mode

Complexity-wise, Option 1 and option 2 RA are similar (using same filter without switching within LCU). BA mode decides about the filter for each 4x4 block. Overall gain of BA mode vs. RA mode is small (0.2-0.3% average), but the gain is higher for RA configuration and particularly class A.
Subtest b: Simplification of ALF signalling

• 
  Remove run prediction (from JCTVC-I0173): No loss
  
• 
  Remove explicit K table signaling and use implicit K table Explicit k table signalling(from JCTVC-I0248): Small loss
  
• 
  Remove alf_pred_flag and no prediction between filter coefficient setsPrediction of coefficients (from JCTVC-I0038, two versions within filter or between filters): No loss
  
• 
  Remove alf_nb_pred_luma_flag and noprediction of the (N-1)-th coefficient from other coefficients of a filter (from JCTVC-I0039). (Prediction of center coefficient cannot be skipped without loss.)
  
• 
  QP dependent quantization of filter coefficients (from JCTVC-I0241): Small loss
  

Recommendation: Adopt all simplifications from subtest 3. (include doc numbers from CE summary) Decision: Agreed.
Subtest c: Number of classes in BA mode

CE2.c.1-1 studied the reduction of number of filter coefficients in non-normative means (i.e., encoder restriction).

CE2.c.1-2 studied the reduction of number of filter coefficients in normative means (i.e., specification change). In addition, additional results by reducing computation for filter classification are provided.

CE2.c.2 studied the reduction of number of filter coefficients in normative means (i.e., specification change), where the number is adaptively selected from 1, 4, or 6.

Note: The test was done for HM6 which is hybrid RA/BA – subtest c as investigated in CE does not need consideration, as this is not a relevant case.
Subtest d: Number of regions in RA mode

Note: The test was done for with an HM5 like signalling. The results are therefore not fully conclusive. The main benefit would be reduction of cache size, but losses.

No gain for RA with 64 regions. 16 regions should be kept.
Subtest e: Depth of LCU division for filtering control

It is found that LCU-level control is sufficient. Decision: Adopt LCU-level control (JCTVC-I0213).

Subtest f: Line memory reduction fix at independent slice and tile boundaries

Suggested to skip the VB padding at LCU boundaries that are at tile boundaries (as it is done already at the picture boundary).

Adopt This was agreed to potentially be valuable if it is confirmed that a visual problem exists at tile boundary; otherwise do we would not adopt it and also consider to remove the case of different processing at picture boundaries, as it would be desirable to have identical LCU processing steps everywhere. (Nno action on this according to notes under ALF viewing report I0585.)
Subtest g: Encoder complexity reduction for LCU-based optimization

Only applies to Option 1 or 3 of subtest a. Loss of 0.1-0.3%. This cCould be interesting for a true low-latency mode (non-normative).

Subtest h: Reuse luma filter coefficients in chroma filtering for LCU-based optimization

Only applies to Option 1 or 3 of subtest a.

Loss in luma (0.1-0.2%) but gain in chroma (0.7%+). In total, marginal shift of BD performance.
Subtest i: Combination subtests

All signalling and signal processing modification in JCTVC-H0066 weare tested. These modifications include combination of CE2.a.1, CE2.b.1-4, CE2.c.1, CE2.e.1 and CE2.h.1 conceptually.

No additional information can could be drawn from this.
Option 1 (LCU based) vs. Option 2

Ppro O1: less dependency, better error resilience

Ccon O1: Worse coding performance, additional memory for filter coefficients (filter coefficients are CABAC coded, potential throughput problems)

Ppro O2: Better coding performance, particular better support of frame optimization

Ccon O2: see pro O1

Problem of secure network transmission of APS? Can we know if the APS was lost? This must be resolved in the context of HL high-level syntax.

Decision: Adopt option 2

Preliminary decision: O2 with RA mode is the best solution that is currently available.

Subjective tests were performed: Current Main Profile RA & LDB, vs. same with ALF/O2RA on Class B + A/E, QP 37 QP32

Expert viewing was done with non-proponents, comparison made on a 5-grade scale (worse – slightly worse –equal – slightly better – better.

In later plenary discussion, about the ALF visual quality test results, it was remarked that:

• 
  4 of the 40 test cases had non-overlapping confidence intervals.
  
• 
  The Riverbed sequence accounted for two of the visually significant cases in the ALF visual testing, and has some special characteristics
  
• 
  Problems in the deblocking filter design (e.g. see discussion of value in table) might cause ALF to show more benefit than it is likely to have if the deblocking filter is improved.
  

Decision: O2 with RA mode is the best solution that is currently available. RA mode will be in draft 7, BA mode for further study, not in the draft 7 text. No BA/RA flag.

It was requested to pPerform subjective tests of: the cCurrent Main Profile RA & LDB, vs. same with ALF/O2RA on.

Class B + A/E, QP 37 QP32

Experts viewing with non-proponents, comparison made on a 5-grade scale (worse - slightly worse –equal – slightly better – better.

See results of viewing in BoG report I0585.

The draft text is found in JCTVC-I0157 CD option 2 RA variant (revision r1) and further improved in JCTVC-I0603.

JCTVC-I0603 Cleanup of ALF CD text [C.-Y. Tsai, C.-Y. Chen, C.-M. Fu, Y.-W. Huang, S. Lei (MediaTek), T. Yamakage, T. Itoh, T. Watanabe, T. Chujoh (Toshiba), I. S. Chong, M. Karczewicz (Qualcomm)] [late]

JCTVC-I0603 is a text derived from I0157 with some further editorial improvements suggested by the CD editor.

Decision: Adopt as new ALF text in draft 7. I0603 still needed to be modified by the adoption of I0346.

After fFollow-up discussion (in context of US58 in DoC) – clipping of coefficients at the decoder is not necessary as a normative encoder constraint is also imposed.

It is also noted that the independent enabling of ALF for luma and chroma at the slice level should be further investigated (does not give gain, and would be desirable to use a unified mechanism with SAO, where chroma is enabled dependent on luma enabling) – further study in the an AHG was requested.

4.2.2Contributions

Test 1

JCTVC-I0249 CE2.a.1: Signalling mode change from slice header mode to interleaving mode [I. S. Chong, M. Karczewicz (Qualcomm), C.-Y. Chen, C.-Y. Tsai, C.-M. Fu, Y.-W. Huang, S. Lei (MediaTek), T. Yamakage, T. Itoh, T. Watanabe, T. Chujoh (Toshiba)]
JCTVC-I0243 CE2.a.1: Cross-verification [P. Chen, W. Wan (Broadcom)]
JCTVC-I0042 CE2.a.2: Additional signalling of picture-level ALF coefficients on top of CE2.a.1 (LCU interleaved signalling) [T. Yamakage, T. Itoh, T. Watanabe (Toshiba), C.-Y. Chen, C.-Y. Tsai, C.-M. Fu, Y.-W. Huang, S. Lei (MediaTek), I. S. Chong, M. Karczewicz (Qualcomm)] [late]
JCTVC-I0222 Crosscheck of CE2.a.2: Additional signalling of picture-level ALF coefficients on top of CE2.a.1 (JCTVC-I0042) [M. Budagavi (TI)] [late]
JCTVC-I0173 CE2: Results of CE2.b.1, CE2.g.1, and CE2.h.1 [C.-Y. Chen, C.-Y. Tsai, C.-M. Fu, Y.-W. Huang, S. Lei (MediaTek)]
JCTVC-I0248 CE2.b.2: Removing explicit K table signalling [I. S. Chong, M. Karczewicz (Qualcomm), C.-Y. Chen, C.-Y. Tsai, C.-M. Fu, Y.-W. Huang, S. Lei (MediaTek), T. Yamakage, T. Itoh, T. Watanabe, T. Chujoh (Toshiba)]
JCTVC-I0089 CE2: Results for CE2.b.1, CE2.b.2, CE2.c.1, CE2.h.1, CE2.i.1 [A. Fuldseth (Cisco)]
JCTVC-I0038 CE2.b.3: Necessity of alf_pred_flag [T. Yamakage, T. Itoh, T. Watanabe (Toshiba)]
JCTVC-I0039 CE2.b.4 Necessity of alf_nb_pred_luma_flag [T. Yamakage, T. Itoh, T. Watanabe (Toshiba)]
JCTVC-I0497 CE3: Cross-check of CE2.b.3 & CE2.b.4 (JCTVC-I0038 & JCTVC-I0039) [K. Ugur, O. Bici (Nokia)] [late]
JCTVC-I0040 CE2.b.5: Qp dependent quantization of ALF coefficients [T. Yamakage, T. Itoh, T. Watanabe (Toshiba)]
JCTVC-I0241 CE2.b.5: Cross-verification of QP dependent quantization of ALF coefficients (JCTVC-I0040) [P. Chen, W. Wan (Broadcom)]
JCTVC-I0212 CE2.c.1 Reduction of number of filter classes in ALF BA mode, by changing class mapping table and using HM6.0 encoding algorithm [P. Lai, F. C. A. Fernandes (Samsung)]
JCTVC-I0250 CE2.c.1: Reducing number of filters of BA (Encoder only) [I. S. Chong, Karczewicz (Qualcomm)]
JCTVC-I0365 CE2.c.1: Cross-check of reducing number of filters of BA (JCTVC-I0250, JCTVC-I0212) [T. Ikai (Sharp)]
JCTVC-I0098 CE2.c.1-1: Test of maximum number of BA filters of 10 [T. Yamakage, T. Itoh, T. Watanabe (Toshiba)]
JCTVC-I0054 CE2.c.2: Reduction of number of filter classes in BA mode with having flexible directional features [M. Matsumura, S. Takamura, A. Shimizu (NTT)]
JCTVC-I0064 CE2 C.2: crosscheck on reduction of number of filter classes in BA mode (JCTVC-I0054) [K. Sugimoto, A. Minezawa, S. Sekiguchi (Mitsubishi)]
JCTVC-I0041 CE2.d.1: Number of regions in RA mode [T. Yamakage, T. Itoh, T. Watanabe (Toshiba)]
JCTVC-I0240 CE2.d.1: Cross-verification of Number of regions in RA mode (JCTVC-I0041) [P. Chen, W. Wan (Broadcom)]
JCTVC-I0213 CE2.e.1 Reduction of depth of LCU division for filtering control [P. Lai, F. C. A. Fernandes (Samsung)]
JCTVC-I0099 CE2.e.1: Cross-check of reduction of depth of LCU division for filtering control (maximum CU division depth of 2) [T. Yamakage, T. Itoh, T. Watanabe (Toshiba)]
JCTVC-I0223 Crosscheck of CE2.f: Line memory reduction fix at tile boundaries (JCTVC-I0167) [M. Budagavi (TI)] [late]
JCTVC-I0100 CE2.i.1: Cross-check of combination test of CE2.a.1, CE2.b.1-4, CE2.c.1, CE2.e.1 and CE2.h.1 [T. Yamakage, T. Itoh, T. Watanabe (Toshiba)]
JCTVC-I0167 CE2: Line memory reduction fix at tile boundaries [S. Esenlik, M. Narroschke, T. Wedi (Panasonic)]

According toAfter reviewing visual testing results, no action was taken on this.