International organisation for standardisation organisation internationale de normalisation

Output documents:

No.	Title	TBP	Available
10857	Applications and Requirements of 3D Video Coding	N	09/07/03
10720	Description of Exploration Experiments in 3D Video Coding	N	09/07/03

5Explorations – High-Performance Video Coding

MPEG has concluded that video bit rate (when current compression technology is used) will go up faster than the network infrastructure ability to carry it economically, both for wireless and wired networks. Therefore a new generation of video compression technology that has sufficiently higher compression capability than the existing AVC standard in its best configuration (e.g., the High Profile), would be needed, in particular expecting

• 
  Performance improvements in terms of coding efficiency at higher resolution,
  
• 
  Applicability to entertainment-quality services such as HD mobile, home cinema and Ultra High Definition (UHD) TV.
  

After first results had been reported to MPEG which indicated that compression technology giving higher compression performance than AVC might exist for such application cases, a more rigorous evaluation phase was started by issuing a Call for Evidence on new high-performance video compression technology in April 2009. A set of test sequences and rate points were defined, and anchor results were produced with an AVC High Profile codec (JM reference software) for comparison. A total of 9 responses were received following this Call, and comparisons were made based on subjective quality assessment.

Submitters were encouraged (but not required) to submit results for all test cases, and submitters were required to provide results for all sequences in any given class. Source test sequences consisted of progressive scan material with 4:2:0 color sampling and 8 bits per sample for each component sample. The classes of video sequences were:

1. 
   Cropped areas of size 2560x1600 taken from the following sequences (frame rates unchanged): Traffic (4096x2048/30), People on Street (3840x2160/30), Park Joy (3840x2160/50).
   
2. 
   1920x1080@24 frames per second: ParkScene, Tennis, Kimono1
   
3. 
   832x480@30 frames per second (WVGA): Mobisode 2, Keiba, Flowervase
   
4. 
   Downsampled versions to half spatial resolution (416x240@30fps, WQVGA) of sequences in class C)
   

The evaluation of high quality use cases, as expected in the case of HVC, requires the participation of video coding experts as viewing subjects. Three independent experts (i.e., experts who were not contributors to the Call for Evidence) participated in the viewing tests. Display of sequences was arranged such that the original video sequence was shown first, followed by (in random order) the AVC anchor and the new technology under test for the same test case and bit rate. Considering that experts are able to immediately recognise the original video, it was considered not necessary to ask them to evaluate it (as the original should always be graded at the maximum vote) but just to score the two coded clips versus the original. The grading scale was on a scale from 0 to 10. In addition to the test demonstration material for subjective viewing, proponents were required to submit an input contribution with documentation of PSNR values (at least the average of frame PSNR for each sequence and encoding point, separate for luma and chroma components) and Bjøntegaard Delta-Rate and Delta-PSNR measurements.

Subjective testing at the highest rate points of each test case was not performed, firstly to reduce the effort, but also as the result of observing that the quality produced by the AVC anchors, at least for some cases, was already very high (close to transparent) and thus no substantial increase in quality relative to the anchor would be possible to achieve in such cases. Detailed results can be found in N 10721. The results indicated that, for a considerable number of test cases, a significant gain over the AVC High Profile encoding could be achieved, with improvements of approximately a magnitude of 2 on the 0 to 10 scale (corresponding to a full unit step on an MOS scale) not being single events. Gains were clearly highest for class B, but even for classes C and D it could be concluded that subjective improvement had been demonstrated. Class A results also showed improvement, but the results for this class may be biased due to the fact that only two sequences were tested, and those were only captured with a static camera and also showed some noise superposition. The best rate improvements that were reported based on the Bjøntegaard Delta-Rate measurement by single proposals were around 20% over all cases in all 4 classes, 30% for the entire class B, and even more for single sequences.
It was therefore concluded that evidence exists of the existence of compression technology that might – after further development – significantly outperform AVC High Profile. As a consequence, it was decided to proceed by preparing and issuing a formal Call for Proposals, with responses expected early in 2010. A draft of the CfP (N 10722) was produced as a non-public output document from the London meeting. The draft CfP definition included the following noteworthy aspects relative to the content of the Call for Evidence:

• 
  Defining new test sequences, as some of the Call for Evidence appeared too simple or inappropriate for subjective viewing, defined now by rate points
  
• 
  Five test classes: A-D as before, E new
  
  • 
    A: 2560x1600 cropped from 4Kx2K
    
  • 
    B: 1920x1080p 24/50-60 fps
    
  • 
    C: 832x480 WVGA
    
  • 
    D: 416x240 WQVGA
    
  • 
    E: 1280x720p 50-60 fps
    

In particular:

• 
  Proposals must meet target rates (no 5% margin)
  
• 
  Two AVC anchor configurations specify additional constraints on delay and random access to which the proposals should match
  
• 
  The purpose of these anchors is different than in the Call for Evidence, they should be expected to serve as a kind of "lowest quality reference" in subjective rating
  
• 
  Additional constraints: No preprocessing, no postfilter, no rate control loop
  

The Draft CfP can be further refined under the AHG mandate. Updates of Conditions in October should however be accompanied by a sufficient justifying rationale.

During the London meeting, two joint meeting sessions with VCEG were held, to inform VCEG about MPEG’s progress in the Call for Evidence and to discuss potential collaboration in the preparation of the CfP and subsequent standard development. Remarks made during these discussions were recorded in N 10871. The Draft Call for Proposals (N 10722) was sent to VCEG as a liaison statement attachment to serve as a basis for further discussions.

Responses to Call for Evidence as reviewed in Sunday AHG

m16570	Response to Call for Evidence in HVC: Decoder-side Motion Estimation for Improved Prediction Usage of forward motion estimation previously used for frame interpolation. Interpolated frame is used as additional reference frame. Bi-directional refinement. BD gains: Traffic -8.64/+0.34; People on Street -6.18/+0.29;Park Joy -1.10/+0.03; Park Scene -4.03/+0.15; Tennis +0.37/-0.01; Kimono -4.86/+0.15. Averag 5.3% rate gain for 4K vs 2.84% for HD.	Sven Klomp Jörn Ostermann
m16574	Response to Call for Evidence in HVC: Integration of Efficient Coding Tools Add several coding tools: Quadtree-based adaptive loop filter (Wiener filter switched on/off for variable block sizes); Internal bit-depth increase (reducing rounding errors in interpolation, de-blocking and weighted prediction); Adaptive Quantization matrix selection (macroblock based; 4 different for P, 2 different for B); Bidirectional intra prediction (weighted average of two uni-directional modes); extended macroblock size. Total average for all 4 classes 20% BR reduction Same picture structure as anchor data. Traffic 25, PoS 12, PJ 19 (avg 19) ParkS 15, Tennis 28, Kimono 30 (avg. 25) Mobisode 27, Keiba 19, FlowV19 (avg. 22) Class D 17/11/16 (avg. 15).	Taksehi Chujoh Tomoo Yamakage
m16592	Samsung's response to Call for Evidence on High-Performance Video Coding Same tools as in the report for the Busan workshop. Gain reported only for class B only: Park scene -21, Kimono 33, Tennis 36; average 30; Initial assessment that computational complexity would increase about 1.6 over AVC (measured decoding time) Also shows that tools can be switched off, i.e. low-complexity mode possible.	Woo-Jin Han JeongHoon Park Ken McCann
m16597	Response to Call for Evidence on High-Performance Video Coding Extended MB size 32x32, different motion partitions than AVC (“hook shaped”, remove small partitions), 16x16 DCT, modified de-blocking, adaptive Wiener filter: Gain 6.5% for class A, 5.7% for class B Current implementation only for P pictures, more gain expected for B Traffic -10/+0.39; PoS -0.64/0.02; PJ -8.1/0.23 ParkS -0.07/2.08; Tennis -6/0.26; Kimono -13/0.43 Some issues for possible CfP: More test material; higher quality than 4:2:0	Shun-ichi Sekiguchi Kazuo Sugimoto Ping Wu Robert A. Cohen Kohtaro Asai Tokumichi Murakami
m16600	Response to Call for Evidence in High-performance Video Coding: KTA-based Simulation Results of Class B and Class C Uses new tools Extended block sizes, quadtree-based Wiener filter, 16x16 transform, modified de-blocking Kimono -28/1.09; ParkSc -15/0.58; Tennis -30/1.15; Flower -12/0.51; Keiba -13/0.6; Mobisode -11/0.37 Average class B -25/0.94 C -12/0.49	Keiichi Chono Kenta Senzaki Hirofumi Aoki Junji Tajime Yuzo Senda
m16628	Response to Call for Evidence in HVC: A Hybrid Coding Based Codec Larger compensation blocks, transform 16x16, bit-depth increase; enhanced interpolation & loop filter (QT based Wiener); second prediction (intra) of MC residual Class B -9/0.35; C -7/0.27; D -2/0.11	Lu Yu Xiaolin Shen Peng Li Xingguo Zhu Shangwen Li Siwei Ma Li Zhang Xin Zhao Xiaoming Li Xiaozhen Zheng
m16629	LG’s Response to MPEG’s Call for Evidence 32x32 MB, several tools adapted to that basis. Adaptive filter. Smallest MC unkit 8x8. New tool partial skip mode. Intra/inter mixed mode in one MB. Intra MB types over all sizes 4…32, New mixed intra mode allowing I4 and I8 prediction in one MB. New transforms 16x16, 16x8, 8x16. BALF, HPIF and MVC used from KTA. Coding structure same as in anchors. Comparison only for 237 frames. Class B average -25/1 dB	Byeongmoon Jeon JungSun Kim Joonyoung Park Seunguk Park
m16661	Response to Call for Evidence in HVC: Hybrid Video Coding with ETP and DMVD Use as additional tools displacement intra prediction Class B average around 4% BR reduction (7% in RD optimized mode that is said to be inferior in terms of subjective quality), Class C 7% reduction. Park scene at one point (lowest in Park Scene) does not	Steffen Kamp Johannes Ballé Mathias Wien
m16684	Response to Call for Evidence in HVC: New High-Quality Deblocking Loop Filter New de-blocking filter (claimed to improve de-blocking while preserving texture). BR savings only around 2%, but it is claimed that most effect is subjective.	Geert Van der Auwera Yeong Taeg Kim

Other:

m16563	Selection of New Perceptual Objective Video Quality Metric Based on DVQ (Watson) – „VQM Plus model“. Watson DWT (SpaTemp sensitivity based on DCT, masking, SSIM). No results shown based on conpression, no comparison against results of subjective testing. No decision to be taken based on the evidence that the contribution brings (only some artificially constructed artifacts).	Lazar Bivolarski
m16705	Algorithm Complexity Metric	Lazar Bivolarski
m16719	1080p,WVGA,WQVGA video coding test sequences Various indoor, outdoor and sports scenes shot by Sony F35 cine camera. Performed initial coding using JM 15. Proposal for target bitrates: 512 WVGA/30 and WQVGA/60,, 768 for WVGA/50/60, 2/3/4 Mbit/s for 720p, 1080/24p and 1080/50/60p, respectively. Usual conditions „for developing, testing and promulgating standards“. Could also be provided in 4:4:4 10 bit, also longer clips could be made available.	Akira Fujibayashi Boon Choong Seng TK Tan
m16763	1080p/50, 720p/50 video coding test sequences Sequences previously proposed in Lausanne (captured by a Sony HDC-1500), now with post-processing (de-noising). Same sequences available in various sizes and frame rates. Has blurring increased due to the de-noising? Usual conditions „for developing, testing and promulgating standards“.	Adi Kouadio Hans Hoffmann Massimo Visca
m16764	BBC 1080p50 test materials for HVC study Sequences capured by Sony HDC-1500, available in 1080p/50 and 720p/50. „Artificial“ scenes with high amount of detail, but some with low amount of motion. Longer sequences would be available. Conditions: Only for MPEG and research (does not match with the current copyright statement)..	Thomas Davies
m16663	Reliability metric for BD measurements Based on overlap range of test and anchor PSNR plots.	Rickard Sjöberg Kenneth Andersson Andrey Norkin

Output documents: