International organisation for standardisation organisation internationale de normalisation

7.4MVC Compression Tools

7.4.1MVC coding tool benefit study

7.4.1.1.1JVT-Y044 ( Prop 2.2/3.1) [T. Dong, P. Pandit, P. Yin, C. Gomila (Thomson)] Study of MVC coding tools

The current MVC JD does not contain any new coding tools. The JMVM so far has included two new coding tools: illumination compensation and motion skip. This contribution presented updated experiment results, comparing simulcast with JMVM without new coding tools, and JMVM with new coding tools using JMVM 5.0.
It was asserted that, compared with simulcast, JMVM without new coding tools achieves about 17% bit rate savings on average, while JMVM with new coding tools achieves about 24% bit rate savings on average (an addition 9% savings relative to JMVM not using new tools), when using JMVM common test conditions.
The contribution recommends that the JVT adopt the new tools that are in the JMVM into the JD and create two profiles: a “simple” profile to support only high level syntax and an “efficiency” profile that supports new coding tools.
Remark: May be too early to determine profiles.
Question: How long would the complete specification of these tools require in the text specification? Perhaps 40 pages (currently 13 pages).
Presentation? Uploaded later.

7.4.2Illumination compensation & related subjects

7.4.2.1.1JVT-Y033 (Prop 2.2/3.1) [G. H. Park, M. W. Park, Y. L. Lee, D. Y. Suh, K. Kim (KHU)] Simplif B- and P-SKIP modes on MVC illum comp

In this contribution, a modified method of B- and P-SKIP modes on illumination compensation was proposed. Compared to JMVM 5.0, this method reportedly shows almost the same performances as the existing JMVM method while reducing the complexity of decoding processes and memory bandwidth.
The motivation of this contribution is simplification.
Some participants expressed support for the proposal, suggesting that it simplifies decoder operation. Another participant indicated that there might not be a real simplification here, partly because some parts of the computation can be shared between different parts of the decoder processing.
It was observed that this is not a change to the basic effectiveness of the IC coding tool, which is a more important topic for our current focus of effort.
The contribution reports on a strange case: ic_flag=1 & DVIC=0. Does this indicate a bad encoder mode decision or should this be restricted? Proposes changes to derivation process for DVIC.
The proposal reports having the same coding efficiency, but reduced complexity. Complexity analysis: numerical results were based on number of times that IC was invoked, as well as arithmetic operations. A substantial savings was reported, but there appears to be a tradeoff. Some clarification is needed.
It as noted that the IC tool is not currently part of the MVC JD specification.
Remark: There might be alternative solutions.
The case of ic_flag = 1 and DVIC = 0 case is agreed to have a problem.
JVT decision: Adopted into JMVM as a bug fix, since there is a need to fix the problem.
Presentation? Available.

7.4.2.1.2JVT-Y038 ( Prop 2.2) [J. Huo, H. Yang, Y. Chang (Xidian U.), S. Lin, S. Gao, L. Xiong (Huawei)] MVC illum & color compensation

In this proposal, a linear compensation model was used to compensate illumination and color mismatch for MVC. Model parameters were estimated using statistical linear regression analysis. For the illumination compensation, it operates on the MB-level due to there being local luminance variation in pictures. The color compensation is based on slice level, reportedly because the color inconsistency between pictures is mainly introduced by the diversity of cameras. It was asserted that the simulation results show that this method can improve coding efficiency – especially for the non-rectified video sequences.
The contributor had not yet finished the full test – instead, they used only a small number of frames of each sequence: the average improvement relative to illumination compensation in the work thus far was reported to be 1-2%.
Question: More complex than illumination compensation? Yes, but similar complexity to ordinary weighted prediction.
The previous meeting contribution JVT-X055 lacked comparison to illumination compensation. This contribution integrates the proposed color compensation method with existing illumination compensation.
Compared to IC, gains on the order of 1~2% for 5 test sequences were reported.
Similar complexity as weighted prediction. More complex than IC.
Further study may be performed to investigate the behavior on the full set of test data.

7.4.3Motion skip operation

7.4.3.1.1JVT-Y036 ( Prop 2.2) [S. Lin, S. Gao, H. Yang, L. Xiong (Huawei)] RDV based MVC motion skip

In this proposal, a motion skip mode for MVC based on a regional disparity vector (RDV) was proposed. The proposed method uses an RDV instead of a global disparity vector (GDV) to indicate the corresponding macroblock position in the picture of a neighbouring view, and the RDV is not represented in the bitstream. Comparing with JMVM 5.0 with MotionSkipMode enabled, the average improvement is reported to be a 2.3% bit rate reduction (best result 4.3%). Complexity is reported to be somewhat increased.
Remark: The GDV bit cost is negligible.
Remark: Two-pass encoding operation is needed to determine the GDV in an encoder, whereas that appears not to be needed for this proposed method? So the encoder complexity may be reduced by this method? This sounds like an advantage of this proposal relative to the current GDV technique.
Remark: Proposal introduces dependency between RDV’s that might impact single loop decoding.
Proponent suggested to continue investigation of the technique. Such further study is encouraged.

7.4.3.1.2JVT-Y037 ( Prop 2.2) [H. Yang, J. Huo, Y. Chang (Xidian U.), S. Lin, S. Gao, L. Xiong (Huawei)] Inter-view motion skip MVC with fine motion

In this document, an inter-view motion skipped multi-view video coding scheme with fine motion matching was proposed. Compared with the existing motion skipped coding scheme in JMVM 5.0, several improvements were reportedly introduced. First, the precision of motion information used for motion skipped coding was asserted to be improved. Second, it was asserted that the optimal motion information is selected from a set of high-precision motion information candidates to maximize coding gains. For each inter-view reference picture, the set of high-precision motion information candidates were obtained within a predefined search window, the center of the search window was inferred by global disparity. Next, the candidates from all the inter-view reference pictures were compared with each other in term of rate-distortion performance, and the optimal one was reportedly selected. If the motion skip mode outperforms other coding modes, the side information about the motion information selection would be transmitted to the decoder. Compared with JMVM 5.0 with MotionSkipMode enabled, the average improvement for all the 8 test sequences was reported to be 0.14 dB gain or 3.5% bitrate reduction (the best results were 0.31 dB gain or 6.3% bit rate reduction).
The contribution proposes to use global disparity vectors at 8x8 resolution rather than 16x16, which allows derivation of motion information from corresponding blocks in neighboring views with a finer scale.
No change to syntax, only changes to semantics to specify disparity vectors at the 8x8 level.
Question: Were the simulations done using common conditions? Not quite, but similar.
Simulations were done with slight modifications to the prediction structure defined in common conditions to allow inter-view prediction for anchor and non-anchor frames in P and B-views.
Compared to existing motion skip mode, up to 0.31 dB / 6.32% bitrate reduction for Akko&Kayo, average 0.14 dB / 3.53% bitrate reduction for all 8 sequences. Compared to JMVM reference, up to 0.689 dB / 14.25% bit rate reduction for Rena; average 0.38 dB / 8.44% bitrate reduction for all sequences except for Breakdancers and Uli (simulation still running).
Complexity: No increase at decoder; small increase in encoder.
Remark: It would be nice to know how much is contributed by each aspect of the proposal.
The proponent suggested a CE to explore the gains obtained by different aspects of the proposal.
Source code availability? Will be provided to CE partners.
Disposition: Create a CE based on this.

7.4.3.1.3JVT-Y069-V ( Info 2.0/3.1) [J. Luo (Thomson)] Verif of JVT-Y037 Huawei prop JVT-Y070

This document reports verification results for the proposal by Huawei as described in document JVT-Y037 “Inter-view Motion Skipped Multi-view Video Coding with Fine Motion Matching”. The decoding executables, coded bitstreams for views, encoder configuration files and simulation results were provided by Huawei. For Akko&Kayo, Ballroom, Breakdancers, Exit, Race1, Rena and Uli, the decoding and its results were reportedly verified.
The contributor received encoder configuration file and view coded bitstreams of Akko&Kayo, Ballroom, Breakdancers, Exit, Flamenco2, Race1, Rena and Uli. They were reported to have been decoded successfully, with details as follows:

• 
  For Akko&Kayo, Ballroom, Breakdancers, Exit, Race1 and Rena, The verification results were reported to be perfectly matched with the ones provided.
  

• 
  For Flamenco2, the contributor only received coded bitstreams for QP 22, 27 and 37. These verification results were reported to be perfectly matched with the ones provided.
  

• 
  For Uli, the bit rates were matched, but the PSNR was not matched. It was reported that this was due to using differing original YUV files from those of the Uli sequence as used by Huawei. According to Huawei, this could be a bug in their own tool that was used to convert the original image files to the source YUV sequence.
  

The source code was not checked.

7.4.3.1.4JVT-Y058 ( Prop 2.2/3.1) [Y.-J. Jeon, H.-S. Koo, B.-M. Jeon (LG)] Motion skip issues in MVC

In this contribution, some motion skip issues raised at the last meeting were presented. At the Geneva meeting, some issues about motion skip that were reportedly not clear were raised and an ad-hoc group ‘JMVM coding tools’ was established. This contribution is asserted to clean up these issues as responses of the mandates of the ad-hoc. The issues this contribution deals with are as follows: 1) Simplification of GDV (Global Disparity Vector) calculation, 2) Signaling in anchor and/or non-anchor pictures with respect to inter-view dependency of non-anchor pictures, 3) Motion skip for multiple inter-view reference pictures, and 4) Simulation of motion skip by disabling motion skip in some non-anchor pictures of which inter-view prediction is forced not to be used according to view dependency information in SPS. This contribution also presented experimental result of motion skip for multiple inter-view reference pictures. Simulation results reportedly showed that bit-saving is obtained for non-anchor pictures having multiple inter-view reference pictures.
JVT-X024 was proposed at the last meeting, with a favorable disposition provided follow-up work was done. This was reported to have taken place.
JVT decision: Adopt JVT-X024 simplification of GDV (Global Disparity Vector) calculation.

JVT decision: Adopt fix to condition of sending global_disparity_mb.

JVT decision: Adopt text fix to mismatch between JMVM text & software reported for motion_skip_flag.
Proposes to allow sending multiple GDVs for motion skip in cases of multiple inter-view reference pictures. Average savings on the B views is reported as 3.7%. Gain as percentage of total bit rate would be substantially smaller.
Remark: This is not common conditions, and is in response to a comment pointing out that there was no way to handle this case in the design. The coding gain is not the main point here. This is asserted to be essentially a bug fix – no design change.
JVT decision: Adopt sending multiple GDVs for motion skip in cases of multiple inter-view reference pictures.
Additional information including simulation of motion skip by disabling motion skip in some non-anchor pictures reportedly shows that the motion skip design provides a net 2.16% average gain on common conditions.
The following notes should be redundant with the above notes: The contribution deals with several fixes to motion skip tool.

1) Simplification of GDV (global disparity vector) calculation. Adopted as presented in JVT-X024.

2) Signaling in anchor and/or non-anchor pictures with respect to inter-view dependency of non-anchor pictures. Adopted as bug fix.

3) Motion skip for multiple inter-view reference pictures. Resolves an ambiguity in the spec and software when multiple inter-view references. Adopted.

Another set of essentially-redundant notes: First issue: Simplification of motion skip that was already presented last meeting and up for further study. Adopted. Second issue: Change signaling of motion skip (condition when to send motion skip, text alignment with software) adopted. Third issue: Derive motion skip from multiple reference pictures. This is response to a question from last meeting where the behaviour of the tool under non-common testing conditions needed to be investigated. In principle, this is a bug fix of the software. Gain of 3.69% on average is reported (only for B views). Adopted.

7.4.3.1.5JVT-Y059-V ( Info 2.0/3.1) [J. Luo (Thomson)] Verif JVT-Y058 motion skip

This document reported verification results for the proposal by LG as described in document JVT-Y058. The decoding executables, assembled coded bitstreams and source code were provided by LG. For Akko&Kayo, Ballroom, Breakdancers, Exit, Race1, Rena and Uli, the decoding and its results were reportedly verified.
The contributor received decoding executables, source code and assembled coded bitstreams of Akko&Kayo, Ballroom, Breakdancers, Exit, Race1, Rena and Uli. They were reported to have been decoded successfully. It was reportedly observed that the verification results were matched with the ones provided in the file JVT-Y058_a.xls.
The source code was not checked.

7.4.4Single-loop decoding

7.4.4.1.1JVT-Y053 ( Prop 2.2.1/3.1) [Y. Chen, Y.-K. Wang, M. M. Hannuksela (Nokia)] Single loop decode and motion skip in JMVM

In this contribution, high level syntax changes for supporting of single-loop decoding were proposed.
Coding efficiency gains of the motion skip tool for both the single-loop decoding scheme and the multiple-loop decoding scheme (i.e. current JMVM with motion skip enabled and illumination compensation disabled) were also reported. The simulation results reportedly show that motion skip benefits the single-loop decoding scheme with an average bit-saving of around 7.5% for the test sequences defined in the common test conditions. However, for the multiple-loop decoding scheme, only around a 2.3% bit-rate savings is reported. After reporting the results, some potential improvements to motion skip are discussed.
Full decoding is performed only for target views. From non-target views, only intra and motion vectors (for skip cases) are used. Anchor pictures are fully decoded. As a result, no loop needs to be run for the non-target views. Proposal includes a request for a flag that indicates whether a view will be used for motion skip reference, and another flag that indicates whether single loop decoding of a particular view is possible. The reported average bit rate saving comparison, performed against other methods that allow single-loop decoding, was as follows: relative to Simulcast 29.7%, relative to anchor-only prediction 7.5%. The advantage of motion skip in multiple-loop decoding was reported to be 2.25%; and the loss compared to multi-loop decoding was reported to be only around 1-2% average (with motion skip on).
The authors proposed 1) to adopt the high-level syntax changes to the JMVM, 2) to keep the motion skip tool in the JMVM, and 3) to set up a core experiment to study possible improvements of the motion skip tool, aiming at better performance in both single-loop decoding and multiple-loop decoding scenarios.
Some reported issues are resolved by actions reported above in relation to JVT-Y058.
Question: What is the loss of single-loop decoding relative to multi-loop decoding with

• 
  Motion skip off? Answer: Didn’t think about that.
  
• 
  Motion skip on? Answer: Roughly 1% loss on average (details in accompanying spreadsheet).
  

7.4.4.1.2JVT-Y042 ( Prop 2.2/3.1) [P. Pandit, P. Yin, T. Dong, C. Gomila, H. Koo, Y. Jeon, B. Jeon (Thomson & LG)] MVC single-loop decoding

This contribution proposed a single loop decoding mode for MVC. It asserted that for some applications where only certain views need to be decoded, complet reconstruction of the reference view (i.e. reconstructed sample data) is not essential. Only certain syntax elements and data from those views can instead be inferred and used for the target view. This can reportedly have the advantage of saving memory and complexity. In particular, it was proposed that a reference view should not be completely reconstructed, such that only parsing its syntax is sufficient. The target view may then use this parsed information. The motion skip mode (already supported in JMVM) is used as the main coding tool to support single loop decoding in this contribution. Finally, RD results are presented without using reconstructed reference view pictures for inter-view prediction of non-anchor pictures.
The proposal is similar to JVT-Y053. Allows inter-view prediction only for anchor pictures. It also makes use of the reported loss, relative to Single Layer, of 1.64 % (0.04 dB). An SPS Flag is proposed to switch single-loop decoding on or off (a different solution than JVT-Y053).
Question: What is justification to make it switchable? Response: Depends on the application (e.g. for M-view display all views need to be decoded).
Syntax proposed is less flexible, but simpler than what is proposed by Nokia in JVT-Y053.

7.4.4.1.3JVT-Y060-V ( Info) [Y.-L. Lee (Sejong U.)] Verif JVT-Y042 MVC single-loop decoding

This contribution reports verification of JVT-Y042 "Single Loop Decoding for MVC". The results of JVT-Y042 were reportedly confirmed.
The proponents provided the decoder source code, bitstreams of the modified JMVM 5.0 software (corresponding to JVT-Y042), configuration files and simulation results.
The software provided by the proponents was reportedly cursorily reviewed (not closely checked), compiled, and run.
The results of the provided SW were reported to be the same as those reported in JVT-Y042.

7.4.4.1.4General discussion on single-loop decoding

Suggestion: Why bother having a switch – mainly for experimental purposes for now.
JVT decision: Adopt single-loop decoding into JMVM using syntax switch proposed in JVT-Y042.

JVT decision: Adopt flag in slice header to omit motion skip flag at lower layer.

Plan CE on potential improvements of motion skip.