International organisation for standardisation organisation internationale de normalisation

18.MPEG-4 AVC File Format (14496-15)

18.1Topics

18.1.1ISO/IEC 14496-15:2010 AMD 2 Carriage of HEVC

This amendment of ISO/IEC 14496-15 specifies the storage format for HEVC (ISO/IEC 23008-2) video streams.

The storage of HEVC content uses the existing capabilities of the ISO base media file format but also defines extensions to support the following features of the HEVC codec.

• 
  Adaptation Parameter sets:
  decoding parameters that are supposed to change more frequently than coding parameters in picture parameter sets.
  
• 
  Temporal scalability sample grouping:
  a structuring and grouping mechanism to indicate the association of access units with different hierarchy levels of temporal scalability.
  

Temporal layer access sample grouping: a structuring and grouping mechanism to indicate the identification of access units as temporal layer access (TLA) samples.

18.1.2ISO/IEC 14496-15:2013 AMD 1 Enhanced carriage of HEVC and support of MVC with depth information

This amendment specifies the storage of video bitstreams consisting of multiple views and the associated depth, encoded based on Annex I of ISO/IEC 14496-10. The design is based on the MVC file format, which is specified in Clause 7 of ISO/IEC 14496-15, in a backwards-compatible manner. In the design, storage of the texture and depth of a particular view in either separate tracks or the same track is supported. The design also includes the signalling of various indications, such as the presence of texture and/or depth for each view, as well as whether the texture or depth component or both of a view is required for the presentation of another view. The amendment also adds the signaling (using HEVC video descriptor) to indicate use of HEVC low-delay coding mode in each access unit where the STD buffer management is performed using the HEVC HRD parameters

18.2Contributions

Number	Session	Title	Source	Dispositions
m32746	File Format	Summary of Voting on ISO/IEC 14496-15:201x/DCOR 1	SC 29 Secretariat	Refer w14440
m33083	File Format	Improvement on HEVC Tile Track for WD of 14496-15 AMD1	Mitsuhiro Hirabayashi Sally Hattori	Accepted w14328
m32951	File Format	Editorial improvements for HEVC AMD	[e-mail], Guido Franceschini	Accepted w14441
m33155	File Format	A Stream Access Point Sample Group for ISOBMFF	Miska M. Hannuksela Vinod Kumar Malamal Vadakital	Accepted w14328
m33156	File Format	File Format for Layered HEVC	Vinod Kumar Malamal Vadakital Miska M. Hannuksela	Accepted w14328
m33222	File Format	Tile Information in ISOBMFF	[e-mail], Cyril Concolato [e-mail], Jean Le Feuvre [e-mail], Franck Denoual [e-mail], Frederic Maze [e-mail], Eric Nassor	Accepted w14328
m33462	File Format	Study of AVC ES constraints in 14496-15		Accepted w14441

18.3Summary of discussions

18.3.1m33083 Improvement on HEVC Tile Track for WD of 14496-15 AMD1

We need to visit the assumption that a tile track would share the codecs sub-parameters of the main track, and see whether it’s true, and formally define the sub-parameters of hvt1. We could relay just these new values, or relay the values from the base stream with these new values replacing the base stream values. (We note that there are a number of typos – Tlie for Tile, Temproral for Temporal, etc.). We see some value in having the operational parameters for a tile track in that track, and not rely on the possible existence of an SEI in the base track’s configuration record. Into the WD.

18.3.2m33155 A Stream Access Point Sample Group for ISOBMFF

Thank you for the exploration of the SAP types. We note also that we have said we’d entertain a revision of sample groups that makes them media-type independent. We are concerned about constant_ind_flag and constant_sap_type in the parameter, but it basically allows the sampletogroup mapping to be de-interleaved. However, that leaves us with two ways to describe the track – interleaved and non-interleaved. There is also some concern over the requirement that the actual sap_type match the constant sap type promised – this may be fragile. The same applies to the independent flag. We accept the group with these two fields removed from the grouping_type_parameter. We prefer dependent_flag as for the non-layered case (which cannot have dependency) the flag will be 0. Into the next Part 12 amendment.

18.3.3m33156 File Format for Layered HEVC

Much discussion on output layers, and layers required for decoding; can we (apparently not) just mark each layer with its profile/level (taking into account the required other layers)? No, because the choice of output layers affects level, and there may be more than one choice. (Many more clarification questions.)

On the config record, it might be nice to have reserved fields where the profile/level were, so we can use the same parsing software. (The explanation of complete_representation could be improved.)

We used to document operating points in the base, but we cannot in the case that the base is AVC, so that’s why we need the oref track reference.

The recent decision to allow recursive extractors seems somewhat in tension with the restriction that an extractor extracts only one layer; the authors didn’t see the use case at time of writing but now do.

We note that this doesn’t yet define for the present all the sample groups, and doesn’t (yet) define the time-parallel meta-data.

Some concern that we can no longer (say, from DASH) say “decode and present this track (and what it pulls in)” as we need to identify the operating point. Do we need a ‘default operating point’ for a track, and maybe each track could identify which are its possible operating points?

We see these major changes:

1. 
   no profile/level etc. in the config record; can we have them back for decoding ‘the entire track’ (maybe operating at the ‘default operating point’, which could be signaled); we can then define the codecs MIME sub-parameters, which we need to do.
   
2. 
   restrictions on extractor/aggregator; we agree we should lift the extractor restriction. do we need aggregrators if we don’t have sample groups and maps? Yes, they can help skipping entire layers.
   
3. 
   IRAP signaling; we agree to put the new SAP sample group in the ‘audio’ part 12 amendment, clearly defining the extra bit and layered coding support as used only in layered coding, with zero there otherwise. Do we need the material in the current WD? We agree to put both into the next WD.
   
4. 
   editorial changes; can we put M-HVC and S-HVC together (but still separate from MVC+D)? Do we like the change from ‘S’/’M’ to ‘L’?
   
5. 
   track content box and operating point information box are new, and the oref track reference
   
6. 
   changed the handling of non-output layered samples; but the old text covered (in an ugly way) the case that there might or might not be an output from this sample, depending on choice of layer/operating point. 8.4.9 can’t apply in this case. More study needed.
   

Do we need the variants like AVC for base layers with or without extractor/aggregator? Perhaps we could simplify and say that base layers don’t use them? No, we want to be able to assemble base layers from tile tracks. We need hev2/hvc2, as in the working draft.

Do the tcon/oinf need to be mandatory? In some simple cases we might not need them. Can we phrase the conditional mandate clearly?

New WD needed (no, we can’t go to amendment).

18.3.4m33462 Study of AVC ES constraints in 14496-15

We think that it should be a ‘should’ and decoders must use the file-level signaling for the functions covered by these (i.e. they may be always ignored, even if there are no file-format structures).

18.3.5m32951 Editorial improvements for HEVC AMD

We’d like text that says how to set all these fields, and the one problematic flag should be changed to say ‘0’ means “don’t know”. We clarify that there is no requirement to put any SEI into the arrays. We also need to clarify the complete_representation flag.
Defect Report with a two-week editing period. Guido to start. We’d like people to scan for other corrections. In particular we do not recall why we have frame rate information in the ‘decoder configuration’.