Of itu-t sg16 wp3 and iso/iec jtc1/SC29/WG11

6.3Motion hook in version 1 (3)

JCTVC-L0177 Hook on temporal motion vector prediction for MV-HEVC [Y. Lin, X. Zheng, J. Zheng (HiSilicon)]

MV-HEVC Hook on temporal motion vector derivation is proposed with two modifications to HEVC. The first modification affects only the case when the collocated picture has the same POC value as the current picture (which cannot occur in a version 1 bitstream). A second modification is proposed to collocated reference picture selection and processing. For testing this in MV-HEVC configurations, the two modifications reportedly provide 1.8% average bit rate saving for 3-view coding, and the coding gain for use with single dependent view reportedly reaches up to 4.3%.

Both aspects of the proposal seem to be intended for cases that cannot occur (or that we would not want to occur) in a version 1 bitstream, so it seems that this would not need to be specified in version 1. No action.

JCTVC-L0400 Cross-check of JCTVC-L0177: Hook on temporal motion vector prediction for M-HEVC [L. Zhang (Qualcomm)] [late]
JCTVC-L0257 Temporal motion vector prediction hook for efficient merge mode in MV-HEVC [Y. Chen, Y.-K. Wang, V. Seregin, L. Zhang, M. Karczewicz (Qualcomm), K. Ugur, M. M. Hannuksela (Nokia)]

In the context of multiview or 3DV coding, reference index equal to zero in merge mode may correspond to the reference picture in the same view, while the motion vector (MV) of the collocated PU may point to an inter-view reference picture which is marked as long-term. In this case, TMVP candidate is considered as unavailable. To address this issue, it is proposed that in this case the motion vector is still available, with a changed target reference index (which is non-zero). For multiview video coding (MV-HEVC), the proposed method reportedly provides about 0.94% average bit rate saving for the all the views and 2.5% bit rate saving for the non-base views. As the proponent asserts that only high-level syntax changes are allowed for MV-HEVC, the change is proposed for HEVC version 1.

For the stereoscopic case, the overall gain was estimated at 0.6%.

Several options to address the multiview coding scenario were discussed in the contribution.

If the collocated block has a MV that refers to an LTRP and reference index 0 corresponds to an STRP, the TMVP candidate of the merge mode is currently marked as unavailable. It is proposed to instead define a TMVP candidate (e.g. specifying the candidate's reference index in the SH).

It was remarked that the software for the LTRP case handling is probably not correct at this point, and that we do not currently have conformance bitstreams to test this.

The cross-check status of some presented options did not seem entirely clear, although one of the proposed variations had previously been proposed and cross-checked.

It was remarked that at this point we do not know whether JCT-3V would actually want the proposed change.

Among the presented "options", the one called option 2 seemed the most reasonable candidate for discussion.

In the initial review of this in JCT-VC, the group was not inclined to take action on this (or L0177), pending some expression of interest by JCT-3V.

Joint discussion between JCT-VC and JCT-3V (Fri. 18 Jan. 0900 hours)

At the opening of a joint discussion between JCT-VC and JCT-3V (Fri. 18 Jan. 0900 hours), the JCT-3V requested consideration of "option 2" of L0257.

Enabled at the slice level by a flag: Conditioned on the flag, an alternative reference index would be sent. Although the contribution discussed specifying this but prohibiting it from being used in the version 1 profiles. In discussions, it was agreed that it would be unlikely to be actually implemented in version 1 decoders if it is normatively prohibited in version 1 bitstreams. So if it is specified in version 1, it should be allowed to be used in version 1 bitstreams to ensure that conformance would be required in decoders and would be tested in version 1 decoders.

It was asked why the presence of the flag would not be gated by a flag in the SPS or PPS. It was agreed that having a gating flag in the PPS would be the desired scheme. It was noted that if such a flag is used, it would actually not be necessary to have an additional flag in the slice header.

During TMVP in merge mode, instead of always using 0 as the target reference index for some combination of long-term and short-term usage, the value sent in the slice header would be used.

It was asserted that this would provide additional flexibility for use with ordinary LTRPs of version 1 and could have a benefit in such usage. The complexity impact would be two additional checks in the TMVP derivation process.

For study, a participant requested to see the software impact in the latest reference software.

The green and yellow text in L0257 was asserted to describe the discussed option (although with the prohibition of usage and without the gating flag).

A version showing just this particular part of the text clearly without being intermixed with other variations in other colour highlight colours was requested to be uploaded.

For the test model text, presumably we would just (for now) add a statement saying that the encoder does not use the feature.

Issue #925 of the reference software was asserted to be an existing problem in the software, and it was asked whether 1) the bug is fixed in the tested software, and 2) whether the comparison used in evaluation of the effectiveness of the scheme had confounding results due to this (or some other) bug.

It was commented that some (perhaps basically all) of the achievable gain from this might alternatively be achievable by using reference picture list construction (with a high-level-only normative change) or reference picture list reordering (non-normatively) to place the desired picture at reference index 0.

The cross-check status was also requested to be clarified.

The JCT-VC was asked to discuss this further after availability of software and text and pending clarification of cross-check and bug fix relationship.

Further discussion in JCT-VC

A revision (-v4) of L0257 was made available for the "option 2" variant that had been discussed. Software and text were included. Test results (with a cross-check submitted as JCTVC-L0447) addressing the issue raised regarding issue #925 as noted above were provided. The proponent indicated that the test results of the new testing were negligibly different from what had previously been described.

A participant said that the text had some issues. The proponent indicated that there was an error in the provided text that had not been in a previous version (resulting from attempted editorial improvement), and that the design intent was clear. Another revision (-v5) of L0257 was made available to address this (replacing two "if"s with "when"s).

To recap, the results were roughly as follows:

• 
  For multiview video coding (MV-HEVC), the proposed method reportedly provides about 0.9% average bit rate saving for the all the views and 2.5% bit rate saving for the non-base views.
  
• 
  For the steroscopic case, the overall gain was estimated at 0.6%.
  

It was asked what happens if the slice_temporal_mvp_enable_flag is not present.

Ultimately, the desire for stability in the low-level coding design, along with concern about the maturity of the proposal, led to no action being taken on the proposed "motion hook".

JCTVC-L0447 Cross-check report of Temporal motion vector prediction hook for efficient merge mode in MV-HEVC (JCTVC-L0257) [O. Nakagami, T. Suzuki (Sony)] [late]

The cross-check used the same (HTM-based) as what was provided. The cross-checker studied in the -v3 text (same as -v4), and the cross-checker reported that the implementation was consistent with the text.