International organisation for standardisation organisation internationale de normalisation

MPEG-H

MPEG-H will be the new umbrella for the High Efficiency Video Coding and the related Systems and Audio standards. The vision for MPEG-H was defined at the 99^th meeting in N12312 Context and Objectives of MPEG-H.
MPEG foresees the initial standards of MPEG-H to include video coding (HEVC), advanced 3D audio coding and MPEG Media Transport. The Definition of graphics and user interfaces will come later.

1. MPEG-H Part 2: HEVC

At the Requirements, plenary concern was voiced regarding the current practice of issuing a new baseline video coding standard about once every 10 years. Given the short lifetime of a mobile device, this industry might be in need of faster progress with lower performance improvements compared to MPEG-2, AVC and HEVC. Incremental improvements of MPEG standards would also ensure that MPEG is the leading provider of coding technology at any point in time. An improvement of at least 15% might be a reasonable differentiator for the mobile market. Further discussion of this issue is required.
Still Picture Coding

HEVC-Intra coding might be used as the Advanced Image Coder that JPEG requires. The capabilities of HEVC-Intra coding were discussed. Evaluation of the HEVC performance will be done jointly with JPEG. Access to the Hannover and Brussels ftp-sites will be exchanged in order to use MPEG and JPEG test material. 16 bit test material is missing at this point. N14086 Test conditions for HEVC still picture coding performance evaluation describes the envisioned evaluation. This evaluation was delayed and it was envisioned to take place prior to the 108^th meeting. However, only objective evaluation was done up to now. It is the conclusion of the experts that HEVC outperforms JPEG2000 using objective measures. For 4:2:0 gains of about 20% are observed. In rare cases of YCC 4:4:4 images, HEVC might be 3% less efficient than JPEG 2000 while typical gains are in about 10% for this format. RGB can be compressed 15% more efficiently. The adhoc group N14505 AHG on still image coding using HEVC will continue to work on this topic with a focus on the subjective evaluation. JPEG is informed by means of the liaison statement N14506 Liaison to WG1,

Scalable Extensions

The French National Body reminded MPEG that scalability with respect to bit depth and color gamut are also within the scope of the HEVC Scalable Extensions. MPEG discussed the necessary technology and profiles. MPEG confirms its commitment to these extensions in the High efficiency video coding (HEVC) scalable extension Draft specification.

Screen Content Coding

In response to the joint call for coding of screen content, 7 proponents submitted technology. For pure screen content MPEG expects coding gains of about 50% compared to HM 13 RExt 6. The JCT-VC is charged with developing the standard. This was discussed in a joint meeting with VCEG as well and is mentioned in N14496 Liaison Statement to ITU-T SG 16 on video coding collaboration. Results of the CfP evaluation can be found in N14399 Results of CfP on Screen Content Coding Tools for HEVC. N14520 Standardization plan for HEVC extensions for screen content coding is sent to VCEG.

Profiles

Several profiles related to 4:4:4 were discussed based on input from National Bodies and members. The contention is about 8 bit and 10 bit profiles based on the RExt currently under ballot and the impact of the profiles on potential Screen Content profiles. At this meeting consensus on a profile for 8 bit was reached. This profile does not contain tools specifically targeted at Screen Content. It is expected that the standardization of Screen Content Coding will results in additional profiles for 4:4:4 video at 8 and potentially 10 bits. The definition of other profiles is mentioned in the JCT-VC report.

5. MPEG-DASH

In order to catch up with ongoing work in the Systems Subgroup, Requirements for MPEG-DASH were extended to include also support for spatial and layered arrangement of visual content.

An additional request to enforce content ownership and licensing arrangements was discussed. MPEG-DASH will not itself be extended to enforce such rights. However, MPEG-DASH supports encrypted content and an additional requirement for providing hooks for technology outside of MPEG to enforce rights on content was added to N14511 Requirements on HTTP Streaming of MPEG Media.

6. Explorations

   1. Compact Descriptors for Video Analysis (CDVA)

The previous exploration Compact Descriptors for Video Search (CDVS) was renamed to Compact Descriptors for Video Analysis (CDVA). The new name better represents the envisioned activity which will go beyond object recognition required for example in broadcast applications. Especially for automotive and security applications, object classification is of much more relevance than object recognition (). Related technology within MPEG can be found in MPEG-7 and video signatures.



Figure 2: The upper part of the diagram shows the “Analyze-Then-Compress” (ATC) paradigm. That is, sets of video features are extracted from raw frames end encoded before transmission resulting in low bandwidth communications. This is opposite to traditional “Compress-Then-Analyze” (CTA) paradigm, in which video features are extracted close to complex visual analysis.
N14508 Requirements under consideration for Compact Descriptors for Video Analysis and N14507 Compact Descriptors for Video Analysis: Objectives, Applications and Use Cases summarize the context and goals of the exploration. New test data was received at this meeting. The licensing arrangements will be clarified until the next meeting. A tool for supporting the annotation of test data was provided to MPEG. N14509 Compact Descriptors for Video Analysis is meant for informing the public about this ongoing exploration. The adhoc group N14503 AHG on compact descriptors for video analysis will refine applications and requirements. Furthermore, a timeline needs to be defined.

2. Free Viewpoint TV

Free Viewpoint TV was the vision that drove the development of many different 3D video coding extensions. It is now time to take back a step and see where the future of 3D will go. Super-multiview displays and holographic displays are currently under development. They will provide horizontal as well as vertical parallax. Hence, we need further extensions of current multiview technology, which assumes a linear camera arrangement, in order to accommodate for more general camera arrangements for future displays. For interaction and navigation purposes, modern human computer interfaces need to be developed.
The adhoc group N14502 AHG on FTV (Free-viewpoint Television) will work on this long-term exploration. The group is working on evidence as described in N14512 Description of Exploration Experiments on Free-viewpoint Television (FTV). At the 109^th meeting, a seminar on FTV as outlined in N14513 Plan of FTV seminar at 109^th MPEG meeting will be organized.

3. High dynamic range and wide colour gamut content distribution

Several film studios currently master movies for digital cinema and DVD separately since the colour space of the cinema is much larger than the colour space of a regular TV set. The industry would like to master just one version of the content using the xyz colour space as one example of a Wide Color Gamut (WCG) colour space. Furthermore, future TV will use High Dynamic Range (HDR) displays. In order to adapt to the different displays and projectors, transformation hints are desirable which instruct the terminal how to scale the colours of the content to the capabilities of the display. Furthermore, the signal needs to be deployed with an amplitude resolution accommodating HDR. Deploying video in such a way would also allow consumers to actually benefit from a WCG and HDR of a new TV screen.
At this point, tools for supporting bit depth scalability as well as WCG scalability are available. An appropriate profile definition is on its way.

At this meeting, one organization provided five WXG and HDR test sequences for the standardization work within MPEG. More sequences were promised. Two displays were made available for viewing content at the meeting. It was noticed that there were significant quality differences and it will be difficult to agree on subjective evaluation procedures given the challenges due to the current display technologies. A joint meeting with JPEG on subjective evaluation is planned for the 109^th meeting.

For B2B application like contribution, a bitrate of 25 Mbit/s is sufficient coding HDR WCG content using HEVC. For B2C applications like Blu-ray or broadcast, there might be a need for further improvements in coding efficiency.
Liaison letters N14515 and N14516 were sent to numerous organizations. WG11 would also like to establish a liaison with Blu-ray Disc Association (N14517).
The adhoc group N14501 AHG on Support of HDR XYZ Color Space and HDR will gather further test material and evidence on the coding performance of HEVC. It is also charged with further developing N14510 Draft Requirements and Explorations for HDR /WCG Content Distribution and Storage. MPEG plans to reach an agreement until October on whether new technology needs to be called for.

4. Processing and Sharing of Media under User Control

In the actual use of MPEG technologies, there are many contexts that require the processing of the media to be private. Examples of this are searching an encrypted audio visual database with an encrypted query (Figure 3); identify a spoken keyword in a private conversation, e.g. encrypted audio; removal of identification clues from media such as video, audio or speech; sharing multimedia content in a limited context, e.g. make a picture available to a limited list of persons or for a limited time. For details, please see N145144 Use Cases for Processing and Sharing of Media under User Control. The Requirements subgroup and the adhoc group N14504 Adhoc Group on Processing and Sharing of Media under User Control will continue to investigate whether MPEG can provide support for these application scenarios. The AhG is also charged with investigating whether other standardization bodies are working in this area.

Figure 3 Media Processing with Privacy

5. Screen Content Sharing Application Format

N14519 Draft Requirements for MPEG Adaptive Screen Content Sharing Application Format combines technologies for transport, coding and composition in order to enable the description of contents of computer screens based on components like background, window, video etc.

.

6. 
   Storage of traceable media signatures
   

As ISOBMFF (14496-12) in combination with Common Encryption (23001-7) becomes increasingly utilized for the distribution of high-value content, there is a growing need to support coding tools to aid with the identification of the player model from which content has been copied without authorization. A common identification capability is known generically as “forensic marking”. A forensic marking system utilizes “variances” in the sample data that in aggregate are unique to the playback environment (Figure 4). Since this concept is important to ensure the use of MPEG technologies for the distribution of contents, N14520 Draft Requirements for a Framework for sample variants in ISOBMFF is a first effort to define a potential standardization activity.

Figure 4 Block-diagram of a Variant Processing

7. Additional Support for Coding of Interlaced Video in HEVC

At this point, HEVC is capable of coding interlaced material. SEI-messages allow for signalling of interlaced material. Compared to AVC using sequence adaptive frame/field (SAFF) coding, HEVC saves 30% in bitrate for the same objective video quality. If the same subjective video quality is required, HEVC saves 50% of the bitrate compared to AVC. At this point, the HEVC reference software does not properly implement SAFF. This issue has been raised for several meetings now. Furthermore, rate control for coding interlaced video appears to be broken. Unfortunately, nobody undertook a serious effort to mitigate these short comings of the reference software.

Since several meetings, experiments by one company show that relatively small changes to HEVC (colour motion vector scaling, picture-adaptive frame/field coding named PAFF …) can reduce the bitrate by 8% to 15% as shown using selected interlaced material. At this meeting, video sequence were shown on consumer type displays. A comparison of HEVC with and without the proposed tools at the same bitrate did not show sufficient evidence to warrant an extension of HEVC with coding tools targeted specifically at interlaced contents.

6. 
   – Systems report
   

Source: Young-Kwon Lim, Chair