Joint Collaborative Team on Video Coding (jct-vc) of itu-t sg16 wp3 and iso/iec jtc1/SC29/WG11

5.12High-level syntax and slice structure

5.12.1Adaptation parameter sets and parameter set referencing [done]

JCTVC-I0465 Signalling of quantization matrices in SPS and PPS [M. Zhou (TI)] [late]

There is a concern over potential loss of QM data.

Two alternatives to the current APS scheme were

• 
  Just as in AVC (optional presence in SPS with optional over-ride in PPS – i.e. just remove them from the APS)
  
• 
  Some partial update mechanism for PPS
  

It was discussed whether there is a need for frequent update of QMs.

Decision: Take QMs out of APS and put into SPS/PPS as in AVC.

Suggestion: Also remove DF control parameters from APS. Put them in PPS with optional override in SH (and a flag in the PPS to indicate whether the override is possible in the SH). Decision: Agreed.

At the moment, it does not appear that there is a need for APS partial update.

JCTVC-I0046 APS partial update through conditional replacement [S. Wenger, J. Boyce (Vidyo)]

Specific to APS partial update.

JCTVC-I0061 Simplified APS partial update through referencing [M. Li, P. Wu (ZTE), S. Wenger, J. Boyce (Vidyo), D. Flynn (BBC)]

Specific to APS partial update.

JCTVC-I0081 APS partial update – APS update with one or multiple references [Hendry, B. Jeon (LG)]

Specific to APS partial update.

JCTVC-I0083 Error resilience improvements for APS partial updates [Hendry, B. Jeon (LG)]

Specific to APS partial update.

JCTVC-I0082 APS partial update – APS buffer management [Hendry, B. Jeon (LG)]

Mostly about APS partial update. However, it was suggested that the proposal could apply without partial update.

The contribution proposes an alternative to the current method of constraining the ID value in order to constrain the PS storage capacity for the decoder. In the proposed scheme, there is a model of a limited decoder buffer capacity and a some rule for establishing which subset of the PSs are stored.

The proposal is essentially to provide a form of loss detection.

JCTVC-I0189 On APS referring and updating [N. Ouedraogo, E. Francois (Canon)]

On APS partial update, a priority flag for APS, and random access for APSs.

This was cClosely related to I0082, with a signalling of the validity of the APS in terms of the number of pictures.

The contribution aAlso proposes a priority flag to indicate "droppable" APS, such that when nal_ref_flag = 0 for an APS, it could only be referred to by non-reference pictures.

Alternatively, it was suggested that temporal_id could be used for this, which would be easier for a "middle-box" to use as an association of APSs (or PPSs to) with layers. It was remarked that such a concept was also proposed in I0330. This seemed like the desired behaviour to the group. This provides a better solution than the proposed one.

JCTVC-I0069 APS error resilience and partial updating [M. M. Hannuksela, S. M. Gopalakrishna (Nokia)]

On APS partial update and loss resilience for APSs.

This proposal has a similar concept of loss detection and buffering of APSs as in I0082 and I0189. It was remarked that this is analogous to the maximum POC difference constraint.

It was remarked that such a scheme would have a problem for splicing, as the range in which the APS IDs would fall after the splice point is not under the control of the encoder.

An alternative loss detection suggestion was to send a PS checksum in an SEI message. This would not have the splicing issue. It was remarked that this checksum idea had actually been previously proposed for AVC, but in that case it was thought that since the PSs are relatively small, it did not seem necessary, as they could be repeated. This argument would not be valid for APSs.

Definition of an SEI message could even be done later, and we could define one for AVC SPSs and PPSs as well. That idea seems worth further study.

JCTVC-I0572 Crosscheck for APS Error Resilience and Partial Updating (JCTVC-I0069) [S. Deshpande (Sharp)] [late]
JCTVC-I0338 On parameter sets [Y.-K. Wang, Y. Chen, G. Van der Auwera (Qualcomm), P. Wu (ZTE)]

Proposes a "GPS" concept for parameter set cross-referencing for APS partial update and other purposes.

The GPS would contain the GPS ID, the APS ID, the PPS ID, and the SPS ID. The SH would contain only the GPS ID.

The other purposes are:

• 
  Enabling sharing of PPSs with different SPSs (to reduce the number of necessary PPSs)
  
• 
  Reduce the number of bits in the SH.
  

It was remarked that loss of a GPS would be a problem, and that the content of the GPS might need to change frequently (which would make out-of-band signalling undesirable).

For further study.

JCTVC-I0067 On APS reference restriction for random access [A. Minezawa, K. Sugimoto, S. Sekiguchi (Mitsubishi)] [late]

The issue is if APS data is sent before a CRA or IDR is sent and then used after that, and suggests that this should be prohibited.

Decision (Ed.): Just put an informative note in the text to point out the issue (Y.-K. Wang and G. J. Sullivan volunteered to provide the text).

5.12.2Random access [first pass done]

See also the recovery point SEI message proposal I0044, HRD for CRA pictures I0277 and APS reference restriction for CRA pictures I0067.

JCTVC-I0133 POC signalling for CRA picture [B. Choi, J. Kim, J. Park (Samsung)]

(Review chaired by Y.-K. Wang.)

This contribution pProposes to signal POC MSBs for CRA pictures, in the slice header.

It is sufficient to have relative POC relationship in the decoding process.

The usefulness of the proposal depends on discussion the consideration of proposals for signalling of LTRPs in the SPS (I0076 and I0340).

After decision discussions on those proposals, those other two proposals would not be adopted in their original version. The existing design appears to work OK without explicit POC MSBs. No action taken.

JCTVC-I0275 On Leading Pictures [S. Deshpande, A. Segall (Sharp)]

(Review chaired by Y.-K. Wang.)

This document contribution proposes explicit signalling of leading pictures that follow a CRA picture in decoding order and precede it in output order. A NAL unit type is proposed for slices of leading pictures that follow a CRA picture (LFC). It is asserted that this LFC NAL Unit type can allow a server / network node to easily discard leading pictures when sending a bitstream starting at the corresponding CRA picture. The proposed LFC NAL unit type does not require changes to the decoding process of pictures.

Currently, bitstreams are conforming when the leading pictures are present. However, the bitstream may not be conforming when the leading pictures are not present, as the CPB may overflow.

A use case mentioned was seeking in either streaming or local playback, wherein it is easier to check whether a NAL unit belongs to a leading picture of the CRA picture at the new starting point.

The ISO base media file format (ISOBMFF) includes signalling of CRA picture type of random access points and the associated leading pictures. But there may be streaming contents not based on ISOBMFF.

Decision: Adopted.
JCTVC-I0278 Signalling of CRA Pictures [S. Deshpande, L. Kerofsky, A. Segall (Sharp)]

(Review chaired by Y.-K. Wang.)

PThis contribution proposes using a new NAL unit type (or a flag in the slice header) to indicate whether leading pictures associated with a CRA picture is may be present.

The use case seems to depending on the decision on I0277.

Adopted per notes in section on I0277.
JCTVC-I0404 CRA pictures with broken links [G. J. Sullivan (Microsoft)]

(Review chaired by Y.-K. Wang.)

Main idea: :Proposes an indication of a so-called "broken-link" CRA (BLC) picture, even when it is in the middle of the bitstream, the decoding process for leading pictures associated with the starting CRA picture is applied for decoding of the leading pictures for the BLC picture in the bitstream.

Comment: Why not discard the leading pictures when splicing? For "lazy" splicing. 

1) To signal the indication use a new NAL unit type for the BLC picture.
Decision: Adopt the above proposal piece(s).
2) To impose aA restriction that any particular picture can only be a leading picture (or a non-decodable picture) associated with at most one CRA picture.

Decision: Adopt
3) And To allow a BLC picture can to activate a new SPS (and consequently can to change spatial resolution), and other details like inclusion of output_prior_pics_flag – text availability was requested to be checked for this.le?

Decision: Adopt activation of new SPS with BLC.

4) To relax the constraints of an IDR picture – such that it can behave like a BLC picture;, then we would still use the IDR term but with the proposed BLC behavior. No text was initially available on this.

Further study was requested on the desirability of changing the IDR concept or ratherversus keeping BLC separate

5) To decouple the POC relationship from the decodability for of pictures that follow a CRA picture and BLC in decoding order and to explicitly indicate the decodability of a picture.

Decision: Adopt, but there may be some editorial issues on the definition of TFD.

Further rReview of the contribution was chaired by Jens-Rainer OhmO:

By relaxing constraints to IDR, the leading picture behaviour seemed no longer needed.

A nNew NAL type 15 of TFD picture (tagged for discard) is introduced as from another (Sharp I0275) proposal.

NA new terminology “RAP” is introduced which subsumes IDR and CRA. What isThe difference between IDR and CRA with the suggested modification of IDR? is that IDR discards the TFD pictures during ordinary bitstream decoding, whereas these would only be discarded for random-access purposes in the CRA case.

One expert mentions suggested that there should not be TFD pictures when the stream starts with a CRA.

Q: Can itIt was asked whether this can be used for a channel change in a continuous stream, and whether it is the case? Is it that that any pictures with NAL units available along with an IDR RAP (unless they are TFD) can be output? . (Iit was mentioned that it could happen that pictures appear which are decodable, but in output order would need to be presented before the IDR.).

Several experts point outsuggested that the definition of TFD (3.111 of the presented draft text) may not be fully complete when used in context of CRA.

About item 4) above, it is mentionedwas suggested that the change of IDR definition may also have impact on how currently systems currently operate .on IDR pictures.
In a later plenary review, a participant said that the broadcast industry would not be interested in the broken link CRA use. In the last review on Monday 7 May, it was suggested to further study whether this would be useful for broadcast.
JCTVC-I0236 TLA picture restriction [R. Sjöberg, J. Samuelsson (Ericsson)]

(Review chaired by Y.-K. Wang.)

The contribution pProposes to add a restriction such that w: When the current picture is a TLA picture, there shall be no reference picture included in the reference picture set with temporal_id greater than or equal to the temporal_id of the current picture.
Decision (Ed.): Adopted (this is a text bug fix, not a change of design intent – it corrects the specification of the intended behavior of TLA pictures).