Joint Collaborative Team on Video Coding (jct-vc)

5.12.3Hypothetical reference decoder

JCTVC-I0277 HRD Buffering for Bitstreams Starting with CRA Picture [S. Deshpande, A. Segall (Sharp)]

This document proposes a modification to the buffering period SEI message and to the HRD specification for bitstreams starting with CRA pictures when leading pictures may be present. It is asserted that the proposed HRD modification can reduce the initial buffering latency when starting playback at CRA picture. Additionally it was reported that with the proposed modification HRD will not underflow when leading pictures after a CRA picture are discarded.

In the discussion, there seemed to be some terminology confusion between the definition of "overflow" and "underflow" – possibly relating to encoder versus decoder perspective.

There was also some confusion over whether removal of data can cause a problem in the VBR case.

The proponent asserted that the removal of some data from the bitstream can cause both overflow and underflow. Whether this was actually the case or not was also not clear to everyone in the discussion.

However, aside from this, there was agreement that an issue can exist, at least in some modes of HRD operation.

One suggestion was to make it the responsibility of the system that drops the data to ensure that the HRD will operate properly (e.g. by not dropping the data or by adjusting the initial_cpb_removal_delay and initial_cpb_removal_delay_offset as necessary).

The proposal was to send additional initial_cpb_removal_delay and initial_cpb_removal_delay_offset parameters in the BP SEI message that are specifically for that case.

Editorially, it would be important to strictly obey the decoder perspective (e.g. not presume the existence of some larger set of data from which some non-present data was removed).

It was remarked that, with such a scheme, it would be necessary to have some indication of what the decoder needs to use to initialize the HRD. I0278 would provide such an indication. Another suggestion was to put a flag into the BP SEI message itself.

If there is an indication that leading pictures are not present, it seems that there is no way to fully test the values of the initialization parameters. Some specification language might be needed for this case, such as saying that the values must be something that could be hypothetically conforming if some leading pictures were present.

It was asserted that this initialization would, at least ordinarily, reduce the start-up latency of the bitstream buffering.

A participant remarked that in VBR mode it might not be necessary to have such data, because t_ai,earliest might prevent the problem.

A suggestion was to put the parameters into a different SEI message, and have the discarding system element be responsible for using that data to replace the data of the BP SEI message.

It was agreed that, in principle this proposal was supported for adoption, together with having some indication of which initialization to use, subject to appropriately-drafted text.

For the indication, it seems likely that we would want to use the NUH.

Decision: Adopted (together with the I0278 NAL unit type indication).
JCTVC-I0333 AHG11: Proposed text for sub-picture based CPB operation [K. Kazui, J. Koyama, S. Shimada, A. Nakagawa (Fujitsu)]

This contribution proposes text for the sub-picture based CPB operation in JCTVC-H0215.

It was asked how it is determined which bits are removed from the CPB in each removal operation.

It was remarked that if the coded data uses a certain type of partitioning, that might provide a way.

Another suggestion was for it to be specified that the bits removed from the CPB are removed in units of NAL units until that NAL units containing a certain needed number of CTBs (specified by a syntax element num_ctbs_in_subpicture_minus1 + 1) have been removed.

The proposed text does not say which data (called "decoding unit") is removed from the CPB at each point.

It was asked how the data flows into the CPB – an arrival time was proposed to be specified for each decoding unit.

The proponent indicated that decoder conformance would not be affected by this – only bitstream conformance.

See notes relating to I0349 and I0588.

JCTVC-I0349 Sub-picture-level low-delay CPB behavior [Y.-K. Wang, Y. Chen (Qualcomm)]

This proposal is similar to I0333. It has a definition of a decoding unit that is a NAL unit.

Several participants indicated that it was desirable not to attempt to fetch decoding units that are smaller than NAL units.

For discussion of both proposals:

It was suggested that if such a scheme is specified, the associated parameters would be provided in addition to the conventional parameters. This view seemed generally supported.

Aspects to work out:

• 
  decoding units that may be multiple VCL NAL units
  
• 
  making this scheme be something that is defined to operate in addition to the existing scheme
  

It was suggested for the proposed text to be studied and for these aspects to be worked on, and then to discuss the subject further during the meeting.

See notes relating to I0333 and I0588.

JCTVC-I0588 Sub-picture based CPB operation [Y.-K. Wang, Y. Chen (Qualcomm), K. Kazui, J. Koyama, S. Shimada, A. Nakagawa (Fujitsu)] [late]

This was submitted to address the issues noted above in the notes for I0349 (and I0333).

This document proposes a sub-picture based coded picture buffer (CPB) behaviour to achieve reduced codec delay in an interoperable manner. In the proposal, CPB removal may be carried out either at the access unit level or sub-picture level. When operating at the access unit level, complete access units are removed from the CPB. When operating at the sub-picture level, decoding units containing one or more slices are individually removed from the CPB. Sub-picture level CPB removal timing information may be signalled in addition to access-unit-level CPB removal timing information. When CPB removal timing information at both the access unit level and sub-picture level are present, the decoder may choose to operate its decoding process at either an access unit level or sub-picture level.

The proposed changes were included in the attachment of this document.

Decision: Adopted. (One "shall" removed in group discussion.)