Of itu-t sg16 wp3 and iso/iec jtc1/SC29/WG11

6.4.8HLS for hybrid scalability (3 – some deferred)

Basic questions:

• 
  Should the HEVC profile/tier/level conformance specification include the base layer within its scope?
  
• 
  Should we specify encapsulation of AVC in HEVC?
  
• 
  Should we specify encapsulation of HEVC in AVC?
  

Do we need anything from the framing system other than decoded pictures to reference for ILP?

Topic deferred for further study and parent body consideration. Parent bodies are continuing consideration. For further study.

JCTVC-O0143 Specification text and profile for SHVC with AVC base layer [J. Boyce (Vidyo)]

This contribution proposes specification text to support an AVC base layer for SHVC, and proposes a new Hybrid Scalable Main profile. The proposed draft text changes provided are based upon SHM3 in JCTVC-N1008_v3. While specifically proposed for SHVC, the changes could also apply to MV-HEVC and other HEVC layered extensions. The proposal includes an option for encapsulation of AVC NAL units inside HEVC NAL units, or allows external means, e.g. the systems layer, to be used to aggregate the AVC and HEVC layers.

The draft thus far has a flag in the VPS for using an AVC base layer and that is all.

Specification text is proposed to include support for an AVC base layer for SHVC, and could also be considered for use in MV-HEVC and the other HEVC layered extensions. It has been updated from a related previous contribution JCTVC-N0050. The proposal includes an option for encapsulation of AVC NAL units inside HEVC NAL units, or allows external means, e.g. the systems layer, to be used to aggregate the AVC and HEVC layers.

The proposed changes are summarized as follows:

• 
  The decoding process is modified for the AVC base layer (when nuh_layer_id equal to 0 and avc_base_layer_flag equal to 1) to reference decoding using the Rec. ITU-T H.264 | ISO/IEC 14496-10 and defines HEVC variable values PicOrderCntVal and PicOutputFlag
  
• 
  An encapsulation NAL unit type value (ENC_NUT) is added to enable (but not require) encapsulation of AVC NAL units in HEVC NAL units
  
• 
  The NAL unit type syntax is modified for an encapsulated AVC base layer (when nal_unit_type equal to ENC_NUT) to refer to the Rec. ITU-T H.264 | ISO/IEC 14496-10 nal_unit(NumBytesInNalUnit) syntax function
  
• 
  The VPS extension syntax is modified to add an avc_base_profile_level_idc syntax element when avc_base_layer_flag equal to 1.
  
• 
  A Scalable Main profile that requires avc_base_layer_flag shall be equal to 0.
  
• 
  A Hybrid Scalable Main profile is defined that requires avc_base_layer_flag shall be equal to 1.
  

It was suggested that an encapsulation should use two NAL unit types rather than one, such that VCL NAL units are mapped to VCL NAL units and non-VCL NAL units are mapped to non-VCL NAL units. Another suggestion was for the whole AVC AU to be considered VCL from the HEVC perspective.

It was also commented that there are rules about determining the boundaries of access units.

The AVC HRD would operate on the encapsulated date without the framing information (i.e., the HEVC NUH).

When the encapsulation is used, there would need to be a de-encapsulation stage inserted prior to operation of the AVC decoding process. Thus, the bitstream would not be (directly) backward compatible to the AVC specification.

Note that O0190 proposes an alternative approach of encapsulating HEVC within an AVC bitstream. That approach would require modification of the AVC specification.

Multiplexing at the systems level is a third approach, and that may be the most fundamental approach to enable.

It was suggested that we could define an informative annex that describes use of one of the "unspecified" NUTs for the encapsulation, and write it in a way that could easily be converted to normative.