Joint Collaborative Team on Video Coding (jct-vc) of itu-t sg16 wp3 and iso/iec jtc1/SC29/WG11

6.3HL syntax for range extensions and single-layer HEVC coding (4)

JCTVC-N0155 HLS: Thumbnail Support in HEVC [C. Wang, C. Wang, W. Zhang, Y. Chiu (Intel)]

This contribution proposes a modification to the syntax of current HEVC in order to embed thumbnail video into HEVC bitstreams. A thumbnail video, which is a sequence of snapshots with smaller resolution, is required in many video applications to give a preview of the video content. To support this feature, three modifications are applied to the high level syntax of HEVC. Firstly, a new VCL NAL unit type is proposed to contain the coded thumbnail picture. Secondly, a new SEI message is added to transmit auxiliary information, e.g. the period of presentation of the thumbnails. Finally, a reserved bit in profile_tier_level() structure is changed into a flag to signal the on/off of the thumbnail video in a bitstream. These modifications reportedly do not violate the conformance requirement of current HEVC, and provide a flexible method to enclose the thumbnail information into regular HEVC bitstream.

All-intra VCL NALU proposed.

Thumbnail alignment to POC suggested.

It was commented that auxiliary pictures seems like a good alternative approach, perhaps accompanied by SEI.

Further study was encouraged, particularly focused on the auxiliary pictures approach.

JCTVC-N0269 HLS: Non-significant slice segments with tiles for single layer HEVC extensions [C. Auyeung (Sony)]

The motion-constrained tile sets SEI message in Range Extensions text specification: Draft 3 can be used to signal that a CVS is comprised of one or more regions of interest in the form of independently decodable motion-constrained tile sets. A user can interactively select and decode a motion constrained tile set without decoding the other tiles. This contribution proposes syntax and semantics that support the extraction of independently decodable motion-constrained tile sets to form a new bitstream without transcoding at the CTU level for streaming of the regions of interest in applications such as interactive UHDTV application, dynamic high-quality zoom-in application, interactive on-demand, e-learning, smart surveillance, and etc.

Prior relevant idea M0277 and M0046.

Proponent describes replacing regions outside an ROI set of tiles with blank/filler coded picture regions.

As proposed, this would require a new profile.

It was remarked that alternatives to non-backward-compatible syntax may be feasible for this. An SEI message can indicate that some slices would decode as empty.

An analogy was drawn with the pseudo-monochrome indicator at the SPS/VUI level.

Further study is encouraged.

JCTVC-N0063 REXT/MV-HEVC/SHVC HLS: Auxiliary picture layers [M. M. Hannuksela (Nokia)]
This contribution proposes to specify new scalability dimension in the VPS extension syntax for a layer carrying auxiliary pictures. It is asserted that the proposed approach:

• 
  Enables to indicate CPB HRD parameters separately for the primary pictures (i.e., the base layer) and for the entire bitstream;
  
• 
  Enables to the bitstream extraction process to extract the bitstream containing the primary pictures only;
  
• 
  Can be used in systems functionalities, such as session negotiation, as the presence of auxiliary pictures is indicated through the VPS; and
  
• 
  Does not require any new VCL NAL unit type(s).
  

Revision 1 discusses the following options on syntax structures for including the type and characteristics of auxiliary pictures. Options 1 and 2 are relevant regardless of whether auxiliary pictures are carried on specific layers (as proposed in this contribution) or within new VCL NAL unit type(s). Option 3 is suggested in this contribution for the proposed auxiliary picture layer design.

• 
  Indicating the type and characteristics of auxiliary pictures in the SPS extension.
  
• 
  Indicating the type and characteristics of auxiliary pictures in an SEI message.
  
• 
  Indicating the type of auxiliary pictures in the VPS extension and the characteristics in an SEI message.
  

Revision 2 includes specification text for chroma enhancement auxiliary pictures proposed in JCTVC-N0145.

It was remarked that the scheme is quite interesting and seems like a good idea. It was agreed to consider this as a starting point for strong consideration and further development in AHGs on SHVC HLS and RExt. The proponent indicated that software could be provided and this was encouraged.

JCTVC-N0077 AHG 5: On support for alpha channel in HEVC [M. Naccari, M. Mrak (BBC)]

Alpha channel signals are used in professional (studio) video coding applications and their usage may become popular also for frame composition at the receiver side. Briefly, the alpha channel complements the information contained in the main bitstream by providing, for each image pixel, its degree of transparency. To support the coding and embedding of alpha channel in the range extensions of the HEVC standard, this contribution introduces the concept of auxiliary picture. An auxiliary picture is a picture which is sent together with the primary coded picture and is compressed and managed as a monochrome picture coded with the same coding tools specified in the syntax of the HEVC standard. In this document, proposed definitions, syntax and semantics for the proposed auxiliary pictures are introduced first and then different options, to address different application requirements, are proposed and discussed.

Proposes use of LTRPs for repetition.

Proposes thresholding values for opaque/transparent determination. For further study in AHGs on SHVC HLS and RExt.

6.4HL syntax in SHVC and 3D extensions (67)

6.4.1Generic HLS issues (2)

(Reviewed Thu 25th plenary)

JCTVC-N0135 MV-HEVC/SHVC HLS: Extended maximum number of layers [B. Choi, Y. Cho, M. W. Park, J. Y. Lee, H. Wey, C. Kim (Samsung)] [late]

(Reviewed Thu. 25th plenary)

This is a follow-up proposal of JCTVC-M0164. To support having more than 64 layers, proposes to use three of the reserved bits in slice headers for extra bits of nuh_layer_id. Approximately 500 layers can be represented with extra 3 bits of nuh_layer_id.

Primarily motivated by "super multiview" application for displays with many views.

Question: Would such an application use one layer per view?

Suggestion to put additional bits into slice header, further investigation necessary whether this is a good place. The current proposal suggests 15 additional bits for layer_id, which according to some experts' opinion might be excessive.

It was remarked that the proposed syntax requires parsing the PPS before being able to access the bits of the extended layer ID, which may be undesirable.

It was noted that we have substantial syntax freedom for non-base layers, although we are constrained by compatibility for layer ID zero.

Near-term profiles likely will not need to support many layers, but it is important to provide extensibility in terms of the number of layers. Some way of allowing extension is highly desirable. Compatibility with existing decoders would be desirable, e.g. for decoding a subset of views.

Another possibility would be to assign one value of layer_id as “reserved for extension”.

Further offline discussion about best way to achieve this

Not clear whether there is any need for immediate action w.r.t. the MV-HEVC draft – likely not.

It was remarked that JCT3V-E0092, JCT3V-E0223 and JCT3V-E0224 are related (not submitted as JCT-VC contributions), where it is suggested to put additional bits into parameter sets. These proposals will also be registered as JCT-VC docs, and were discussed in the context of generic HLS issues.

Presentation not uploaded.

See BoG report.

JCTVC-N0355 / JCT3V-E0092 3D/MV-HEVC HLS: Extending the supported number of layers [K. Suehring, G. Tech, R. Skupin, T. Schierl (FhG HHI)] [late]

(Reviewed Thu. 25th plenary)

This contribution proposes an extension mechanism for layer identifiers to increase the number of supported layers in MV-HEVC and 3D-HEVC. The range of nuh_layer_id is extended by an additional syntax element within the NAL units. The concept of so-called layer clusters allows using the existing extraction processes to select groups of related layers as proposed during the 4th meeting. The syntax has been modified slightly to ensure a backward compatible base layer and to be aligned with MV-HEVC Draft 4.
Again focused on "super multiview" with many views.

Proposes to signal, in VPS, a number of extra bits of layer ID.

It was commented that just having reserved fields might be adequate from a syntax perspective.
It was commented that some form of syntax involving "if( layer_id != 0)" branch in the syntax could be used.

It was commented that using layer ID equal to 63 as an escape code indication could be an alternative way to deal with the layer ID range.

Having an extended NUH for use with some particular profile was also discussed.

It was remarked that having a view-subset decoding capability for a lower-capability decoder is desirable. The proponent suggested that having a "clustering" of views to indicate which subset to decode is also desirable.

It was generaly agreed that support for some extensibility to more views with subset capabilities would be desirable, but we don't want to burden "mainstream" decoders with significant extra work to accomplish that, and we don't want the standard to contain purely "hypothetical syntax" that is normatively forbidden to be used.

See BoG report.

JCTVC-N0244 / JCT3V-E0075 MV-HEVC/SHVC HLS: Cross-layer POC alignment [Y. Chen, Y.-K. Wang, A. K. Ramasubramonian (Qualcomm)]

(Reviewed Thu. 25th plenary)

In this proposal, a mechanism is proposed in order to ensure that the POC values all pictures of each access unit are the same even when it is allowed that access units for some pictures are IRAP pictures with NoRaslOutputFlag equal to 1 while others are not. Draft text was provided.

The contribution proposes a "poc_reset_flag" syntax flag.

When set to 1, the flag changes the POC of the previously-decoded pictures of the same layer in the DPB, by subtracting an offset from their POC values. It was remarked that this has loss resilience implications (when the picture which causes the POC reset is lost), and further study of that aspect was encouraged (e.g. to add some SEI message to improve the detection and handling of lost pictures).

Base layer decoder would work different from a single-layer (version 1) decoder (which is likely not critical).

Handling of long term pictures? Is claimed to be solved.

See BoG report and notes elsewhere.

JCTVC-N0356 / JCT3V-E0223 3D/MV-HEVC HLS: Dependency signaling for extending the supported number of layers [G. Tech, K. Suehring, R. Skupin, Y. Sanchez, T. Schierl (HHI)] [late]

(related to JCTVC-N0355 / JCT3V-E0092).  3V

JCTVC-N0357 / JCT3V-E0224 3D/MV-HEVC HLS: Flexible layer clustering for extending the supported number of layers [G. Tech, K. Suehring, R. Skupin, Y. Sanchez, T. Schierl (HHI)] [late]

(related to JCTVC-N0355 / JCT3V-E0092).  3V

JCTVC-N0267 / JCT3V-E0087 MV-HEVC/SHVC HLS: On changing of the highest layer ID across AUs and multi-mode bitstream extraction [Y.-K. Wang, Y. Chen (Qualcomm)]

(Reviewed Thu. 25th plenary)

This document discusses allowing changing of the highest value of nuh_layer_id across AUs within a CVS (which is currently allowed, due to an adoption intended to allow ARC), and proposes a multi-mode bitstream extraction process. The contribution raises a number of issues relating to this topic.

Related to N0110.

Question: Does “higher layer” usually mean equal or higher resolution? Currently, yes. However, in the case of using the “single_layer_for_non_irap_flag” it might also make sense to allow prediction of lower resolution from higher resolution, which would require defining decimation filters for prediction.

See BoG report.