Organisation internationale de normalisation

6.2SEI messages

6.2.1.1.1JVT-Z040 ( Prop 2.2) [A. A. Rodriguez, J. Au (SciAtl/Cisco)] Prop SEI message to convey suitable splice points in the bitstream

This contribution proposed a new SEI message that would convey information about a potential splice point in the bitstream located N access units subsequent to (in bitstream order) the location of the SEI or the current access unit.
A "Suitable splicing point" of order M was defined as a point in the bitstream at which some specific number of pictures M is present in the DPB that are ready for output at consecutive clock ticks, prior to any gap in output times. For example, if the DPB contains F = 5 pictures, and if the bitstream is cut at that point, and it is a suitable splicing point with M = 2, there would be 2 pictures with consecutive output times and each of the other three pictures would have one of the following two properties:

• 
  There is a "time gap" after the output times of the M pictures and before the output time of the other picture arrives, or
  
• 
  The other picture has an output time that precedes the output times of the M pictures.
  

The proposed SEI message identifies a "suitable splice point" and conveys the value of M, N, and F.

Question: Fixed frame rate assumption? Response: Yes.
Question: What is the need for providing advance notice in the bitstream of these properties that are to be fulfilled at a later point in the bitstream? Response: Motivation is to reduce complexity and delay in the splicer, to enable "pre-conditioning" of the other stream that is to be spliced in, etc. Remark: Don't really understand that explanation.
Question: Why is an SEI message needed? Why can't the decoder/splicer just scan the bitstream and watch the picture properties to identify for itself the points in the bitstream that have these properties?
Remark: Need to consider the additive nature of the buffering in relation to the buffer capacity.
The contribution proposes to add flags/indicators whether the indicated "suitable splice point" immediately precedes an I or IDR.
Question: How is that aspect useful?
Question: Is there an assumption that the new data that would be concatenated after the "suitable splice point" would begin with an IDR picture? Response: Perhaps an all-Intra picture rather than an IDR picture.
Question: What to do with the other pictures that are waiting in the DPB if not followed by an IDR?
Response: 1) Send an MMCO to mark them "not used for reference", and 2) if they have not yet been output, perhaps this should not be indicated as a "suitable splice point".
Proposed syntax and semantics were shown. There were some other syntax elements in the proposal.
The proposal also includes signaling of some CPB properties.
Remark: As shown, the contribution may reflect a misconception about the meaning of PicOrderCnt (although the concept may be expressible in another way).
Remark: These proposals create "promises" that must be fulfilled later in the bitstream – but some of the "not yet fulfilled promises" may be broken by a splicing operation.
Remark: This is especially true for (e.g. older-generation) splicing equipment that has been designed without awareness of this proposed SEI message.
Response: We can specify that the promises go away upon encountering an IDR picture or end_of_stream NAL unit. Or we can specify not to make overlapping promises.
Remark: The proposal does not seem like it provides a complete solution to this "broken promises" problem.
Response: But we risk having other specifications developed that establish harmful constraints on bitstream or application behavior – resulting in reduced interoperability and loss of potential capability.
Remark: Managing CPB aspects is another aspect that needs study in this context.
Remark: Whenever the "M" is mismatched between the old stream and the new stream, the only thing the splicer can probably to is set no_output_of_prior_pics_flag to 1. A splice operation must manage both DPB output times and CPB output times (and decoding times, etc.). Is it really possible to produce seamless behaviour and/or maintain conforming behavior under some of these circumstances?
See further notes below.

6.2.1.1.2JVT-Z041 ( Prop 2.2) [A. A. Rodriguez, J. Au (SciAtl/Cisco)] Prop SEI message to control DPB output in non-seamless spliced bitstreams with end_of_stream

This contribution proposed a new SEI message that would convey information related to the output of DPB pictures at the splice point of non-seamless concatenated bitstreams. It was asserted that the proposed SEI message could serve as a tool to aid splicing devices, along with the end_of_stream NAL unit and no_output_of _prior_pics_flag. The proposed SEI message would be provided in the bitstream prior to the end_of_stream NAL unit to identify its location and specify the output behavior of non-previously output pictures in the DPB subsequent to the end_of_stream NAL unit.
Information that specifies the output behavior of each non-previously output DPB picture reportedly allows for outputting a picture, not outputting, or outputting the picture for a number of consecutive times prior to outputting the subsequent picture from the first bitstream.
It was asserted that the proposed SEI message could be signaled ahead with information that points to the location of the end_of_stream
Discusses gaps – non-consecutive (fixed frame rate) output times for pictures in the DPB.
There is currently no equivalent of MMCO for output marking – just no_output_of_prior_pics_flag. This is basically what is suggested in this contribution.
Remark: We need to consider the conformance implications – we have a standard that establishes requirements for conformance that we can't change (at least not easily/substantially).
Remark: Consider the situation where a "left-over" picture from prior to the splice point ends up with the same output time as a picture from the new spliced-in coded video sequence.
See further notes below.

6.2.1.1.3JVT-Z042 ( Prop 2.2) [A. A. Rodriguez, J. Au (SciAtl/Cisco)] Prop SEI message to forewarn location of end_of_stream

This contribution proposed a new SEI message that would identify the location of an end_of_stream NAL unit in the bitstream (some number of access units prior to its placement).
The end_of_stream NAL unit is the last NAL unit in the access unit that ends a bitstream. In some system environments, a new bitstream may immediately follow the access unit that ended the bitstream. The proposed SEI message would indicate that an end of stream NAL unit will appear in the bitstream at a position N access units after the location of the SEI message.
See further notes below.

6.2.1.1.4Discussion of JVT-Z040, JVT-Z041, and JVT-Z042 together

JVT Disposition: These contributions seem to open an topic likely to require action. But further study is needed to determine exactly what to do.
Related activities may be under way in ITU-T SG 9 (J.181, incoming LS, on cue messages and codec changes) and SCTE (SCTE 35, which is a corresponding specification, and other activities on "conditioning" – how to manage the operation) with some interest also found in DVB.
The JVT planned to conduct further study of this subject, and to establish an AHG in which to perform such investigations.