International organisation for standardisation organisation internationale de normalisation

3TE1: Decoder-side motion vector derivation

3.1.1.1.1.1.1.1.1JCTVC-C115 TE1: Summary report of TE1 activity [Y.-J. Chiu (Intel), Y.-W. Huang (MediaTek), M. Wien (RWTH Aachen Univ.), H. Yu (Huawei)]

This contribution summarized the results and various contributions relating to TE1. The contributions and results in this area are further discussed below in this section.

3.1.1.1.1.1.1.1.2JCTVC-C097 TE1: Huawei report on DMVD improvements in TMuC [M. Yang, S. Lin, D. Wang, J. Zhou, H. Yu (Huawei)]

The goal of this Tool Experiment (TE1) was to measure the performance of Decoder-side Motion Vector Derivation (DMVD) techniques in TMuC software. There are two categories in the TE1, one is TE.1a (DMVD Inter Prediction) and the other is TE.1b (DMVD Direct Mode). This contribution continued the work on the Template Matching based DMVD (TM-DMVD) and Spatial and Temporal Direct Mode (STDM) techniques. STDM and TM-DMVD work together in the TE.1a, and STDM works alone in the TE.1b. Both techniques had been implemented into the TMuC 0.7 software and tested following the common test conditions in JCTVC-B300 and JCTVC-B301. This document described these techniques in detail, which included algorithm description, software implementation, coding performance, and complexity evaluation and analysis. According to the test results, the average bit rate savings for the random access and low delay configurations were 3.90% and 2.43%, respectively in TE1.a, and 2.9% and 0.5%, respectively in TE1.b.

It was noted that some of the tests for Class A and Class B sequences in the TE1.a random access configuration were not finished yet, and the authors indicated that final results would be updated when these became available. As of the time of writing this report, the latest uploaded version of the contribution still said that the results were not yet finished.

TE1.a

• 
  Results for Random Access were not finished yet when reviewed
  
• 
  Current decoder runtime increase to 230% for RA, 300% for LD.
  
• 
  "low complexity" method presented by last meeting not implemented in TMuC yet – estimate to bring complexity increase to 160%.
  

TE1.b

• 
  Current decoder runtime increase to 118% for RA, 129% for LD.
  
• 
  Encoder runtime increase negligible
  
• 
  AMVP was disabled in all TE1
  

3.1.1.1.1.1.1.1.3JCTVC-C128 TE1: Cross-checking of DMVD result from Huawei [Y.-J. Chiu, L. Xu, W. Zhang, H. Jiang (Intel)]

This contribution was a cross check of the results of JCTVC-C097 on TE1.a, reporting matching results as far as the results that were available.

3.1.1.1.1.1.1.1.4JCTVC-C125 TE1: Spatio-temporal direct mode [S. Sekiguchi, Y. Itani (Mitsubishi Electric)]

This contribution was a cross check of the results of JCTVC-C097 on TE1.b, reporting matching results.

3.1.1.1.1.1.1.1.5JCTVC-C138 TE1.a: Implementation report of refinement motion compensation using DMVD on TMuC [M. Ueda (JVC)]

In TE1 activity, the tool experiment targets at exploring the performance and value of Decoder-side Motion Vector Derivation (DMVD) and at analyzing the interaction of DMVD with other tools under consideration for HEVC.

This proposal focused on the tool "Refinement Motion Compensation (RMC) using DMVD" which belongs to TE1.a: DMVD Inter Prediction. The feature in RMC is that bi-predictive MC blocks can be obtained by transmitting only one motion vector using inter-reference DMVD.

In this contribution, the implementation of the basic RMC tool on TMuC0.7 software and result of evaluating the performance were reported. The simulation results reportedly showed that the proposed technique provides 1.8% BD-rate gain for the random access case and 0.0% for the low delay case as tested against the TE1 common anchor under TE1 test conditions.

• 
  Decoder runtime increased to 138%, encoder runtime to 146% for RA case
  
• 
  The method seems not to be working for bi-predictive extrapolation currently
  
• 
  Results were relative to TMuC anchors without AMVP
  

3.1.1.1.1.1.1.1.6JCTVC-C266 TE1: Cross-check result of DMVD proposal from JCTVC-C138 (JVC/Kenwood) [W.-J. Chien, P. Chen, M. Karczewicz (Qualcomm)]

The results of JCTVC-C138 were reportedly confirmed in terms of coding performance and speed.

3.1.1.1.1.1.1.1.7JCTVC-C124 TE1: Implicit direct vector derivation [Y. Itani, S. Sekiguchi (Mitsubishi Electric)]

This contribution reported results of performance verification on implicit direct vector derivation technique that was approved as a part of TE1 at the last (Geneva) meeting. The evaluated technique provides a new design of B-skip and B-direct modes with locally adaptive direct MV derivation relying on a decoder-side decision. A performance evaluation using TMuC software was conducted according to the test conditions defined in TE1, and the results reportedly showed 1.8% coding gain on average for RA case (and up to 3.5% at maximum gain). The increase of decoding time was about 10% relative to the TE1 anchor, and the encoding time was reportedly almost same as that of the anchor. This contribution also provided notes on normative text in the form of modification of the TMuC specification.

• 
  For LD case, 0.4% BR reduction on average
  
• 
  Decoder time increases 13% for RA case and 5% for LC case; no significant increase in encoder run time
  
• 
  Comparison against anchors without motion vector competition and without CU merging (skip/direct on)
  

3.1.1.1.1.1.1.1.8JCTVC-C024 TE1: Cross-check result of DMVD proposal JCTVC-C124 (Mitsubishi) [T. Yokoyama, K. Nakamura (Hitachi)]

This contribution confirmed the results of JCTVC-C124 w.r.t. performance and encoding/decoding time.

3.1.1.1.1.1.1.1.9JCTVC-C127 TE1: Report of self derivation of motion estimation improvement in TMuC [Y.-J. Chiu, L. Xu, W. Zhang, H. Jiang (Intel)]

This contribution presented the techniques of decoder-side motion vector derivation (DMVD) side to increase the coding efficiency of B pictures. With the motion vector (MV) information self derived at the video decoder side, the transmission of these MVs from the video encoder side to the video decoder side is skipped. The proposed self derivation of motion estimation (SDME) techniques were reported to be friendly to parallel implementation by utilizing on the temporal correlation among the available samples in the previously-decoded reference pictures, instead of operating on the previously reconstructed samples of the neighboring area of the current picture, which poses an inherent decoding ordering causal relationship among decoding blocks of current picture. Experiments reportedly demonstrated an overall 4.8% BD bit rate improvement for the test of random access (high efficiency) category under the TE1 test conditions with 73% increase in encoding time and 150% increase in decoding time increase on TMuC 0.7 Reference Software platform. A fast version of the direct-candidate simplified SDME reportedly demonstrated an overall 3.1% BD bit rate improvement with a 26% reduction in encoding time and 18% increase in decoding time when compared to the TE1 anchor. Another version of the rounded-candidate simplified SDME had reportedly demonstrated an overall 3.7% BD bit rate improvement with 10% increase in encoding time and 13% increase in decoding time when compared to the TE1 anchor. Also, the proposed SDME reportedly demonstrated 2.7% BD bit rate improvement over the anchor of TE12 under the random access test category (high efficiency) on a shorter version of the common test video sequences defined for TE12. A description of the normative syntax change in the form of TMuC specification was included.

In the discussion, it was remarked that the reduction of encoding time is not specific to DMVD, but could be achieved similarly by appropriate encoder optimization (avoid unnecessary checking of intra modes).

3.1.1.1.1.1.1.1.10JCTVC-C098 TE1: Cross-checking of DMVD results from Mitsubishi and Intel [S. Lin, D. Wang, M. Yang, H. Yu (Huawei)]

The results of JCTVC-C124 w.r.t. performance and encoding/decoding time were reportedly confirmed.

The results of JCTVC-C127 w.r.t. performance and encoding/decoding time (x-check for the fast modes not done) were reportedly confirmed.

3.1.1.1.1.1.1.1.11TE1 summary discussion

Aspects remarked upon in the discussion of TE1 included the following:

• 
  Current TE1 settings were with AMVP (motion competition) off and CU merging off.
  
• 
  Gains were reported of 4.8% BR reduction average.
  
• 
  New methods were proposed with significantly less decoder complexity than before.
  
• 
  Results should be compared against the default TMuC settings (which probably brings the gains down by 2-2.5%).
  
• 
  Some of the methods seem to reach a region of more moderate increase in decoding complexity.
  

It was suggested to continue investigation and experiment work on this topic.