Of itu-t sg16 wp3 and iso/iec jtc1/SC29/WG11

5.3Non-normative: encoder optimization, decoder speed improvement and cleanup, post filtering, loss concealment, rate control, other information (0)

See section 3.4.

6Plenary discussions, joint meetings, BoG reports, and summary of actions taken

6.1Project development

Joint meetings are discussed in this section of this report. Additional notes on the same topics may appear elsewhere in this report and in the reports of the other organizations involved in these joint discussions. Joint discussions were held on Tuesday 0900–1000, Tuesday 1000–1030, and Tuesday 1400–1500, as recorded below.
Joint meeting on evaluation of projections, Tuesday 0900–1000, of JCT-VC, JVET, VCEG, MPEG Systems

Y.-K. Lim (MPEG Systems chair) said the MPEG Systems subgroup did some work on compression performance of projection formats

J. Boyce presented a summary JCTVC-Z0050 / JVET-E0137 as an update on JCT-VC and JVET 360° video activities.

The current ERP syntax suggestion in JCT-VC when at the time of this discussion:

omnidirectional_projection_indication( payloadSize ) {	Descriptor
omnidirectional_projection_indication_cancel_flag	u(1)
if( ! omnidirectional_projection_indication_cancel_flag) {
geometry_type	u(6)
sub_geometry_flag	u(1)
projection_type	u(8)
if (geometry_type = = 1 && sub_geometry_flag = = 1                 && projection_type = =  1) {
yaw_center	i(16)
pitch_center	i(16)
roll_center	i(16)
yaw_range	u(16)
pitch_range	u(16)
}
omnidirectional_projection_information_persistence_flag	u(1)
}
}

The “geometry_type” and “projection_type” syntax elements are essentially placeholders for extensibility.

Where to express the semantics was discussed. It was commented that the MPEG Systems group was considering not to do this in the CICP standard, but rather in another new standard. It was noted that the syntax needs can go beyond just a “codepoint”.

The status of JVET work on 360 video coding tools was reviewed. This included discussion of the “360Lib” software that has been developed.

Observations in the summary presentation included:

• 
  Significant variation in objective metric calculations, per sequence, per metric, per projection format
  
• 
  WS-PSNR (E2E) seems to be the single best metric from JVET-D1030, as it is the only one that computes E2E
  
  • 
    Additional E2E metrics were recommended to be added to CTC for next meeting
    
• 
  Reduction in PSNR due to projection conversion doesn’t seem correlated to reduction in PSNR due to coding
  
• 
  No subjective testing has been done
  
  • 
    There is particular concern about the subjective quality of discontinuous edges
    
    • 
      This would be masked by average PSNR values
      
  • 
    Further study was highly encouraged
    

Regarding the subjective issue, it was noted that some issues – especially “seam” artefacts, may be very significant subjectively, but have little effect on objective measures.

No clear conclusions seemed able to be reached (at this time) on the relative coding efficiency merits of different projection schemes.

Getting some subjective testing done is necessary. It was agreed that using a dynamic viewport (unknown in advance, not moving too rapidly) is a way to do that.

It was noted that the software has a mapping of samples back to a sphere, which seems to be what would be needed for a precise specification.

Agreed: Developing and conducting such a testing is needed as quickly as possible. The goal is to develop a testing methodology by the next meeting and complete evaluation of various schemes by the meeting after that.
Joint meeting on SEI messages and ITU-R liaison, Tuesday 1000–1030 JCT-VC, VCEG, MPEG Requirements and Video

There is an ILS from ITU-R on full-range scaling, submitted to both orgs. SG16 will reply with a scaling description per how this was specified in AVC and HEVC as before the last meeting.

Planned SEI messages were discussed and agreed:

• 
  Regional nesting
  
• 
  MCTS extraction
  
• 
  Content colour volume
  
• 
  Omnidirectional 360° equirectangular projection – to coordinate this with MPEG Systems
  

Showcases had been provided for the functionality.

Regarding AVC

• 
  A proposal to add a CRI SEI message to AVC had been submitted to both orgs – yes, it was agreed to proceed with this.
  
• 
  Omnidirectional 360° equirectangular projection – it was also planned to include this in AVC.
  

Joint meeting Tuesday 1400–1500 JCT-VC on projection extensibility, JCT-VC, VCEG, MPEG R, S, V

Topics of initial focus in the discussion:

• 
  Omnidirectional SEI message extensibility.
  
• 
  Hypothetical CRI or TMI SEI message extensibility.
  

The MPEG Systems chair said there was a desire to specify “region-wise packings” as well as “projections”, where a packing would involve some “chopping up” and rearranging, with a syntax that is capable of expressing the packing in a generalized manner.

The planned output draft text for OMAF from this meeting will only consider ERP.

An MPEG input contribution m39947 proposed a parameterized projection description. It is based on the concepts of mapping an image texture on a mesh, as is common in graphics. An example of decomposing a sphere into triangles to approximate ERP was shown. The projection is defined in terms of three elemens: 1) vertex coordinates, 2) texture coordinates, 3) how the vertices are arranged to form the mesh.

Variation of ERP quality was shown based on the resolution of mesh that is used to approximate the sphere.

There are currently five projection formats other ERP than that are under study in MPEG Systems.

Beyond ERP and the final list of these, and packing syntax, MPEG Systems will consider the proposal to define a generalized projection.