Joint Collaborative Team on Video Coding (jct-vc) of itu-t sg16 wp3 and iso/iec jtc1/SC29/WG11

4.2SCE2: Key pictures and single loop decoding (7)

4.2.1SCE2 summary and general discussion

Discussed Wed 23rd afternoon (GJS).

JCTVC-O0032 SCE2 Summary Report [M. Wien, K. Rapaka, X. Xiu]

Three variants proposed.

Base layer must use CIP (will hurt base layer).

Tested for SNR case only.

Questioning of the value of SNR scalability.

One participant commented that CIP harms coding efficiency and that it would effectively be necessary to decode both layers anyway, e.g., in case of the need to perform loss concealment by down-switching.

A question asked is what is the impact of the approaches if the decoder is designed to use multi-loop anyway.

BoG (K. Ugur) was asked to study and help present the topic later (incl. related non-CE and maxTid approach).

See notes on BoG O0345.
JCTVC-O0345 BoG Report on single loop decoding and key pictures in SHVC [K. Ugur]

Discussed Sat. p.m. GJS.

The BoG met on 24 October 2013 at 11:30 to discuss single loop decoding and key picture concepts for SHVC. The proposals were analyzed with respect to the following points:

• 
  Coding efficiency and complexity characteristics.
  
• 
  Profiling considerations for SNR scalability and single loop operation
  
• 
  Burden on multi-loop encoder/decoder that doesn’t operate in single loop fashion
  
  • 
    Access to unfiltered samples (in variant supporting single-loop decoding unless base layer is required to disable deblocking)
    
  • 
    TMVP problem (discovered by cross-checker, asserted minor fixable by proponent)
    

It was remarked that we might need a special profile for a single-loop approach that requires low-level modifications (as in JCTVC-O0145), since our intent has been to have an approach that can be built without low-level changes.

There is also an approach already supported in the standard that supports single-loop operation without low-level changes by forbidding ILP for certain temporal sub-layers. This has a more substantial loss for the enhancement layer decoder – unless the base layer non-key pictures are removed from the bitstream, in which case it actually has gain.

JCTVC-O0162 is more of an AVC-like single-loop design. It requires access to the base layer residual and to the unfiltered base layer samples and full-resolution motion field. It also requires the base layer and enchancement layer to reference the same pictures.

Conclusion: A new phase of work as development of additional profile(s) would need to be justified and approved to adopt such a scheme that requires low level changes. This is not within our current plan of work.

JCTVC-O0227 has a different scheme – not a single-loop decoding approach. It allows base layer pictures to refer to enhancement layer pictures, whereas a base layer decoder that operates only on base layer data would generate somewhat degraded picture quality due to drift (e.g., 4% on average). The enhancement layer decoder would actually produce improved quality (e.g., 3%) – although only in the case where temporal scalability is in use. Relative to operation without temporal scalability, e.g., in low-delay use, this scheme does not provide a benefit.

The text changes are asserted to be relatively small (e.g. less than a page of text changes).

It was remarked that in this scheme the BL decoder needs access to the pictures (all of them) in the EL DPB.

It was commented that the interaction of this with rate control may reveal problems that would not be seen with fixed-QP testing.

Further study is encouraged – e.g. on distribution and perceptibility of drift, encoder drift control, relationship to temporal scalability, interaction with rate control, DPB access feasibility.

4.2.2SCE2 primary contributions

JCTVC-O0145 [SCE2] Key picture concept and single loop decoding [C. Feldmann, M. Wien (RWTH Aachen University)]

See BoG report JCTVC-O0345.

JCTVC-O0162 SCE2: Single-loop decoding based SNR scalability for SHVC [X. Xiu, Y. Ye, Y.-W. He, Y. He (InterDigital)]

See BoG report JCTVC-O0345.

JCTVC-O0227 SCE2 : On key pictures in SHVC [K. Rapaka, J. Chen, V. Seregin, Y.-K. Wang, M. Karczewicz (Qualcomm)]

This is focused on key pictures without single loop functionality.

See BoG report JCTVC-O0345.

4.2.3SCE2 cross checks

JCTVC-O0146 [SCE2] Cross check for Qualcomm JCTVC-O0227 [Christian Feldmann, Mathias Wien (RWTH Aachen University)] [late]
JCTVC-O0169 SCE2: Cross-verification of key picture concept and single loop decoding from Aachen [X. Xiu, Y. Ye (InterDigital)] [late]
JCTVC-O0231 SCE2 : Cross check for Interdigital JCTVC-O0162 [K. Rapaka (Qualcomm)]

4.3SCE3: Inter-layer filtering (5)

4.3.1SCE3 summary and general discussion

Initially reviewed Wed 23rd. evening (JRO).

JCTVC-O0033 SCE3: Summary report of SHVC Core Experiment on Inter-layer Filtering [J. Chen, E. Alshina, J. Dong, M. Sychev]

4.3.2SCE3 primary contributions

JCTVC-O0078 SCE3: performance and complexity test for cross-color inter-layer filter [X. Li, J. Chen, M. Karczewicz (Qualcomm), E. Alshina, A. Alshin(Samsung), J. Dong, Y. Ye, Y. He (InterDigital)]

Additional processing step applied after upsampling of chroma components: The chroma components of inter layer reference picture are enhanced by adding an offset which is derived by applying a high pass filter on the base layer luma component. 8-tap non-separable filter, adapted region wise, up to 16 rectangular regions

Parameters signaled at slice header (currently only would allow 1 slice per picture); different way of signalling (a lower-level adaptation parameter set) would be necessary.

Relation with tile/wavefront processing? Implication on multi-core implementation? Conflict might apply in case of on-the-fly upsampling

Question: Impact on coding? Low latency possible? In current implementation, picture wise processing is applied.

Has it been checked for visual artifacts? Not systematically, but it is reported that some visual inspections did not unveil problems such as boundary artifacts.

Gains:

• 
  Spatial scalability average: −0.8% (Y) / −11.7% (U) / −21.7% (V)
  
• 
  SNR scalability average: −0.5% (Y) / −10.7% (U) / −19.8% (V)
  

Initial assessment: 4 multiplications per enhancement layer luma sample, moderate additional memory.

Opinion of some experts: The method gives a reasonable gain versus complexity tradeoff for the case of spatial scalability. Other experts do not agree with this assessment.

M. Zhou and Mediatek to further investigate complexity impact for hardware implementation, and discuss offline with proponents.

More proper way of signaling should be investigated and further discussed.

(Further reviewed Sat 26th afternoon JRO)

M. Zhou reported back on investigation, for decoder

• 
  increase of HW complexity of upsampler 25-30%
  
• 
  no additional line buffer
  

for encoder:

• 
  access the chroma samples once for computing parameters, once for interpolation processing
  
• 
  extra functional blocks
  

Non-negligible impact on encoder and decoder complexity

Request made by last meeting about potential visual artifacts not satisfied

Request made by last meeting about studying one-pass encoder not satisfied

Too many reservations by other experts - no action

JCTVC-O0163 SCE3: Inter-layer prediction modes based on base layer sharpness filter [M. Sychev, V. Anisimovskiy, S. Ikonin (Huawei)]

Generates second inter-layer reference which is adaptively sharpened after upsampling, by

• 
  hor/ver gradient and gradient magnitude
  
• 
  Blurring filter on gradient magnitude
  
• 
  Displacement vector calculation (based on hor/ver difference in gradient map, optical-flow like)
  
• 
  Displacement used to determine the position from which bilinear interpolation (1/16 pel accuracy) is performed.
  

Parameters sent in SPS: gradient threshold Tr, ShD in displacement vector estimation

• 
  Spatial scalability: −1.0%(Y)/ −2.3% (U) /−2.3%(V),
  
• 
  SNR scalability: −0.5%(Y)/ −1.6% (U) / −1.4%(V).
  

Computation and memory complexity seem to be non-negligible (decoder run-time increase reported 15-20%, which may not be fully precise).

Sequential steps, but at pixel level, latency may therefore be minor (around 10 lines?)

Gain is unequally distributed (people on street alone gives >4%)

Several experts express opinion that the gain vs. complexity tradeoff does not justify to consider the proposal for adoption.

4.3.3SCE3 cross checks

JCTVC-O0151 SCE3: Cross-Check of test 3.1 region based inter-layer cross-color filtering (N-0229) in SCE 3 [V. Anisimovskiy, M. Sychev (Huawei)]
JCTVC-O0282 SCE3: Verification of performance and complexity assessment for sharpening inter-layer filter [E. Alshina (Samsung)] [late]