International organisation for standardisation organisation internationale de normalisation

Explorations

1. Coding of screen content

At this meeting, new test material focusing on mixed content, graphics and motion was presented. Several contributions on screen content coding show that bit rate savings of 35% compared to HEVC Rext 4.1 on pure screen content are possible. The group prepared N14089 Draft requirements for future extensions of HEVC in coding screen content as well as N14090 Draft call for proposals for coding of screen content.
Several solutions for screen content coding were proposed:

• 
  4:4:4 8 bit profile as defined in range extensions
  
• 
  New profile HEVC 4:4:4 8 bit screen content coding
  
• 
  Frame packing arrangements
  
• 
  Auxiliary frame
  

Frame packing arrangements are certainly the least preferred method.
The input of several National Bodies was processed. The responses are gathered in N14084 Response to NB Comments on Screen Content Coding and HEVC Profiles.
While a CfP jointly with VCEG is envisioned by some to be issued at the 107^th meeting, the lack of consensus on test sequences, anchors, rates and the 4:4:4 profile in the range extensions under ballot make it unlikely that a final call can be issued or evaluations can be done prior to the 108^th meeting. N14081 AHG on coding screen content and medical visual content is charged with preparing a call for proposals and the related evidence.


Figure 1 Artifacts when coding screen content

2. Coding of medical visual content

Medical images require typically subjectively or mathematically lossless coding. At this point, the Requirements subgroup recommends to wait for the performance evaluation of the current range extensions. In case these extensions do not fulfil the requirements defined in N14088 Draft Requirements for future extensions of HEVC in medical visual content, this exploration will continue with the currently available test material. N14081 AHG on coding screen content and medical visual content will monitor the situation.

3. Compact descriptors for video search

CDVS works on describing contents of an image. This activity wants to extend content description to video. Applications are in video retrieval, computer vision, automotive, security and other application where many video streams need to be evaluated (). Related technology within MPEG can be found in MPEG-7 and video signatures.

Figure 2: The upper part of the diagram shows the “Analyze-Then-Compress” (ATC) paradigm. That is, sets of video features are extracted from raw frames end encoded before transmission resulting in lwo bandwidth communications. This is opposite to traditional “Compress-Then-Analyze” (CTA) paradigm, in which video features are extracted close to complex visual analysis.
N14093 Compact Descriptors for Video Search summarizes the context and goals of the exploration. N14094 Compact Descriptors for Video Search: Objectives, Applications and Use Cases reflects thoughts towards applications and requirements. Requirements are outlined in N14095 Draft Requirements for compact descriptors for video search. The adhoc group N14080 AHG on compact descriptors for video search will refine applications and requirements. Furthermore, a timeline needs to be defined. Since the work will require annotated video, members are encouraged to start working on test material.

4. Full gamut content distribution

Several film studios currently master movies for digital cinema and DVD separately since the colour space of the cinema is much larger than the colour space of a regular TV set. The industry would like to master just one version of the content using the xyz colour space. In order to adapt to the different displays and projectors, transformation hints are desirable which instruct the terminal how to scale the colours of the content to the capabilities of the display. Deploying video in such a way would also allow consumers to actually benefit from a wider colour gamut of a new TV screen.
At this meeting, the meaning of XYZ, High Dynamic Range (HDR) and EOTF was clarified. Use cases were specified. In particular the B2C application is sometimes rate sensitive. At this point, any evaluation method is completely unclear. Off-site visits to labs having extend color gamut and HDR display capabilities are planned for the 107^th meeting. The requirements subgroup hopes to come to a decision on whether to move forward at the 107^th or 108^th meeting. At this point, there is no commitment to provide appropriate test material.
It is not clear how much HEVC will be affected in case of coding xyz data instead of YUV data. The adhoc group N14078 AHG on Support of HDR XYZ Color Space and HDR will gather test material and evidence on the coding performance of HEVC. It is also charged with further developing N14096 Draft Requirements for Full Gamut Content Distribution. Concerns of National Bodies were addressed in N14101 Response to the JNB comments on supporting XYZ color space and N14102 Response to UKNB Comment on High Dynamic Range.
Liaison statements were sent to various organisations: N14097 WG11 Liaison to SMPTE, N14098 WG11 Liaison to DVB, N14099 WG11 Liaison to ITU-R WP6C, and N14100 WG11 Liaison to ARIB.

5. Additional Support for Coding of Interlaced Video in HEVC

At this point, HEVC is capable of coding interlaced material. SEI-messages allow for signalling of interlaced material. Compared to AVC using sequence adaptive frame/field (SAFF) coding, HEVC saves 30% in bitrate for the same objective video quality. If the same subjective video quality is required, HEVC saves 50% of the bitrate compared to AVC. At this point, the reference software does not properly implement SAFF.

Experiments by one company show that relatively small changes to HEVC (colour motion vector scaling, picture-adaptive frame/field coding (PAFF) …) can reduce the bitrate by 8% to 10% as shown using selected interlaced material. Before arriving at any conclusion, further study of the HEVC encoder for interlaced material is required. Currently two tests sequences are available.

There is significant contention in the group on why additional support of interlaced coding in HEVC is required given that displays are progressive. According to the supporter of additional support for interlaced coding, deinterlacing of the material prior to sending it to the consumer using equipment at the service provider creates quality and resource problems while deinterlacing at the consumer side avoids these issues.

At this meeting no subjective evidence on performance benefits were provided. The group agrees that viewing of interlaced material on a progressive display without deinterlacer is inadequate. Since the reference software for AVC does not support interlaced material correctly at this point as well as a SAFF-capable HM-decoder is missing, showing evidence at the next meeting will require major work by the proponents.

A joint meeting with VCEG revealed that VCEG has no interest in coding of interlaced material.

Work continues in the N14082 AHG on Study of interlace coding in HEVC. N14091 Requirements and use cases on interlace video under consideration and N14092 Initial Draft Call for Proposals for Interlaced Video Coding – quality progress towards a CfP, however subjective evidence has to be shown prior to issuing a CfP.

6. 3DV Exploration

Free Viewpoint TV was the vision that drove the development of many different 3D video coding extensions. It is now time to take back a step and see where the future of 3D will go. Ultra-multiview displays and holographic displays are currently under development. They will require further extensions of current multiview technology or new codecs due to the very different nature of the source material. For interaction and navigation purposes, modern human computer interfaces need to be developed. The adhoc group N14079 AHG on FTV (Free-viewpoint Television) will work on this long-term exploration. Initial requirements and use cases are gathered in N1404 Use Cases and Requirements on Free-viewpoint Television (FTV). The group is working on evidence as described in N14105 Description of Exploration Experiments on Free-viewpoint Television (FTV).

7. User description

A standardized user description may enable customers to have easier access to services. An ontology and schemas need to be standardized. Information for user description includes attributes, activities, interests, status, history, medical data, etc. User Descriptions are used in MPEG-7, MPEG-21, MPEG-M, and MPEG-V.
MPEG decided at the 105^th meeting to issue N13879 Call for Proposals on MPEG User Description (MPEG-UD). At this meeting MPEG received five responses only. These responses provide an uneven coverage of the different requirements. No input from a major service provider was received. The exploration phase ended at this meeting. The Systems subgroup may develop a useful standard in the future.

6. 
   – Systems report
   

Source: Young-Kwon Lim, Chair

1.