Joint Video Experts Team (jvet) of itu-t sg 6 wp and iso/iec jtc 1/sc 29/wg 11

7.15360° Video related (65)

Contributions in this category were discussed XXday XX July XXXX–XXXX (chaired by XXX).

JVET-K0141 AHG8: 360°-based inter/intra prediction for cubemap projection [C.-H. Shih, J.-L. Lin, H.-C. Lin, S.-K. Chang, C.-C. Ju (MediaTek)]
JVET-K0142 AHG8: 360°-based in-loop filters for cubemap projection [S.-Y. Lin, L. . Liu, C.-H. Shih, J.-L. Lin, H.-C. Lin, S.-K. Chang, C.-C. Ju (MediaTek)]
JVET-K0466 Cross-check of JVET-K0142: AHG8: 360°-based in-loop filters for cubemap projection [P. . Hanhart (InterDigital)] [late]
JVET-K0183 AHG8: Face boundary filtering for 360° video [Xuchang Huangfu, Yule Sun, Bin Wang, Lu Yu (Zhejiang Univ.)] [late]
JVET-K0333 AHG8: Horizontal geometry padding for PERP [P. . Hanhart, Y. . He, Y. . Ye (InterDigital)]
JVET-K0404 AHG8: Selective In-loop filtering for 360 Video Compression [C. . Pujara, S. N. . Akula, A. . Singh, R. . Narayana, W. . Choi (Samsung)] [late]

7.16Extended colour volume related (0)

Contributions in this category were discussed XXday XX July XXXX–XXXX (chaired by XXX).

7.17High-levelL syntax (6)

Contributions in this category were discussed Sunday 15 July 1700–1800 (chaired by GJS and JRO).

JVET-K0155 AHG12: Flexible Tile Partitioning [Y. . Yasugi, T. . Ikai (Sharp)]

This contribution was discussed Sunday 1710 (chaired by GJS and JRO).

This contribution proposes a tile functionality that allows to split pictures into flexible partitioning tile, where the width or height of the unit of tile can be any multiplies of 4 (the minimum CU size), 8, 16, 32 and 64, i.e. smaller than a CTU.

In the proposed tile design, pictures would be split into constant-size CTUs as the conventional tile while the size of the right most and bottom most CTUs in tile boundary can be different from the constant CTU size. This flexible feature is asserted to provide better load balancing since all tile can be almost the same size in uniform spacing mode. It is also asserted that this feature is useful for 360 video sequences or frame packing sequences since the corresponding tile can fit the arbitrary face size.

The experimental results with this feature (unit=32) reportedly show that luma BD-rate coding losses on average are 0.66 % for All Intra and 0.99 % for Random Access under the common test condition (CTC) for SDR sequences.

The experimental results without this feature (i.e. HEVC like tile) reportedly show that luma BD-rate coding losses on average are 0.64 % for All Intra and 0.97 % for Random Access under the common test condition (CTC) for SDR sequences. In some cases, there was even some gain observed.

It was commented that the coding efficiency impact would be affected by the boundary handling in the design.

The contributor suggested that this is relevant to AHG12 (on parallelism).

2x2 composited images were shown as an example, where the tile boundaries could be set to align to the region boundaries.

Further study was encouraged.

JVET-K0408 Cross-check of JVET-K0155: AHG12: Flexible Tile Partitioning [A. . Wieckowski (HHI)] [late]
JVET-K0260 Flexible Tiles [R. . Sjöberg, M. . Damghanian, M. . Pettersson, J. . Enhorn (Ericsson)]

This contribution was discussed Sunday 1725 (chaired by GJS and JRO).

This contribution proposes to include tiles into VVC. The tile sizes are proposed to be signalled individually, either by copying the tile size from the previous tile size in decoding order or by one tile width and one tile height code word. This is reported to enable tile partition structures that are not restricted by the HEVC rule that tile boundaries must span across the entire picture.

The following HEVC tile properties are included in the proposal:

1. 
   That CABAC is initialized for each tile, flushed after each tile and that the tiles are byte aligned
   
2. 
   That tiles break predictions as in HEVC
   
3. 
   That the tile structure is specified in the PPS
   
4. 
   That tile pointers are mandatory for all tiles except the first in a slice
   
5. 
   That either all CTUs in a tile belongs to the same slice or all CTUs in a slice belong to the same tile
   
6. 
   That the initial QP of a tile is set as in HEVC
   
7. 
   That tiles are CTU aligned
   
8. 
   That tiles can only have rectangular shape
   

The following tile properties different to HEVC tiles are also proposed:

• 
  That tile boundaries are not required to span across the entire picture
  
• 
  That the number of tiles in a picture is signalled in the PPS
  
• 
  That the tile sizes are specified in subtile units to reduce the signalling bit cost
  
• 
  That tiles are specified individually in subtile raster scan order by their individual tile size either by copying the tile size from the previous tile size in decoding order or by one tile width and one tile height UVLC code word each
  

The contribution presents experimental results for an OMAF 360 video partitioning example using HM 16.18. Compared to a realization using HEVC tiles and slices, using tiles with the proposed method, an average BDR of -1.1% was reported under RA configuration using the three 8-bit VVC CfP 360 video sequences.

The proponent said that both tiles and slices should be supported.

A particular syntax is proposed, with a prediction of tile sizes and a signalling of a granularity of the tile boundaries.

It is proposed that there be an established maximum number of tiles per picture.

In HEVC, tiles have a minimum width of 256, and there may also be a limit on minimum height.

It was commented that the ability for the tile structure to change from picture to picture may cause problems for parallel decoders.

It was commented that partitioning the picture for processor load allocation may be different from partitioning for other purposes.

It was commented that if we really want to think about tiles as a parallelization tool, this could get into profile/level constraints discussions that may be premature.

JVET-K0300 Design goals for tiles [M. . M. . Hannuksela, A. . Zare, M. . Homayouni, R. . Ghaznavi-Youvalari, A. . Aminlou (Nokia)]

This contribution proposes that the VVC tile design should enable

• 
  Encoding of motion-constrained tile sets (MCTSs) that are more efficient than HEVC MCTSs in terms of rate-distortion penalty;
  
• 
  Avoiding visible MCTS boundaries with as small processing cost as possible;
  
• 
  Intra block copy across tiles for enabling prediction from one constituent frame to another for frame-packed stereoscopic video, provided that intra block copy is adopted as a tool in VVC;
  
• 
  Extracting VCL NAL units of a subset of MCTSs from one VVC bitstream and reposition them to another VVC bitstream without VCL NAL unit modifications.
  

The proposed design goals are asserted to make VVC tiles suited for viewport-dependent 360° streaming.

These design goals are proposed to be used in evaluating merits of technical contributions and to be included as mandates of an appropriate JVET ad-hoc group.

It was commented that a more fair comparison for some of the illustrated cases would use tiles that are not motion-constrained tiles.

The proposal also suggested supporting MCTS reordering and rewriting functionality.

This was further discussed Tuesday 1250 (GJS & JRO). It was agreed to establish an AHG on segmentation of a picture into coded regions, to investigate tiles, slices, etc., and what they can be used for and what additional functionalities for such regions may be beneficial beyond what has been done in past standards.

JVET-K0325 On High Level Syntax Starting Point [S. . Deshpande, B. . Choi (Sharp)]

This was reviewed in JVET plenary Tuesday 1200 (GJS & JRO).

A high-level syntax starting point was proposed for VVC that includes a NAL unit structure, a sequence parameter set, a picture parameter set, and a slice header. Initial NAL unit header and NAL unit types were also proposed.

It was commented that this appears to be an appropriate basic “trimmed down” approach. It includes the concepts of IRAP-vs-nonIRAP, NAL units, SPS, PPS, SEI, end of sequence, end of bitstream, and “slice”. Specification of access unit delimiters and filler data were not included.

It was commented that the traditional concept of a slice may not be needed, although it is likely that we would have some collection of CTUs that is not a whole picture (e.g., tiles). Using the term “slice” does not necessarily imply a traditional slice.

Decision: Adopted.

JVET-K0403 A Flexible Syntax Framework for VVC [S. . Deshpande, F. . Bossen, A. . Segall (Sharp)] [late]

This was reviewed in JVET plenary Tuesday 1230 (GJS & JRO).

This document proposes a framework for supporting a syntax structure in VVC. The core motivation for the proposal is to create syntax groups that may be skipped without parsing. This is asserted to allow decoders to extract syntax of interest without having to parse other syntax element groups. Here, the emphasis is on the signalling of high-level syntax in parameter sets, though it is asserted that the framework could be applicable to other signalling uses in the VVC standard.

The scheme is equivalent to the current method of carrying multiple SEI messages in one NAL unit.

It was commented that such a scheme could be only partially used - e.g., only for extension parameters.

It was commented that such schemes had been previously considered at some point - e.g., for groups of extension data.

A contributor said this would allow encoders to send the syntax they care about first, without needing to send other parameters earlier.

It was commented that if the order of the syntax is allowed to be changed, dependencies between syntax elements would need to be carefully considered, as “race conditions” would be possible.

It was asked whether this is needed at this stage, and suggested to just keep it in mind for further development.

This adds some overhead for type codes and length values.

It was commented that the lack of such a scheme has not really been a problem thus far.

Other than VUI, the syntax in the parameters sets is (at least generally) essential to enabling decoding of the bitstream (or for capability identification, in the case of the profile/tier/level syntax).

It was commented that the granularity of the synax has an effect on the usefulness - e.g., it doesn’t seem appropriate to wrap a single flag in this.

Several participants expressed interest in the scheme, and it should be considered in further work, but seems not necessary to use at this stage.