International organisation for standardisation organisation internationale de normalisation

MFC (3)

14.1.1.1.1.1.1.1.161JCT3V-F0096 Editorial Improvement of Multi-resolution Frame Compatible Stereo Coding [P. Yin, Y.-K. Wang, G.J. Sullivan]

ISO/IEC ballot: WG11 M31336.

Included a case of support for multi-view profile where it is identical with stereo high progressive.

Plan: Prepare DoC (in case of generic USNB comment, explain which issues have been improved or corrected).

14.1.1.1.1.1.1.1.162JCT3V-F0097 AHG13: MFC Conformance Testing Update [P. Yin, T. Lu, H. Ganapathy, T. Chen, W. Husak (Dolby), D. Tian (MERL)]

ISO/IEC ballot: WG11 M31332.

One more stream for PAFF/MBAFF included.

Plan: Prepare DoC (in case of generic comments, explain which issues have been improved or corrected)

14.1.1.1.1.1.1.1.163JCT3V-F0098 AHG13: MFC Reference Software Update [P. Yin, T. Lu, H. Ganapathy, T. Chen, W. Husak (Dolby), D. Tian (MERL)]

ISO/IEC ballot: WG11 M31333.

Only editorial improvements.

Plan: Prepare DoC.

17. High-level syntax (78)

    1. MV-HEVC/3D-HEVC (12)

       1. Camera parameters

14.1.1.1.1.1.1.1.164JCT3V-F0136 Comments on camera parameters in 3D-HEVC [Y. Chen, V. Thirumalai (Qualcomm)]

Currently, the depth for one view is sent for both texture and depth. The document discusses whether it is safe to use the same camera parameters for texture and depth. It is claimed that same camera parameters can be used if the resolution is the same for texture and depth. If the resolutions are different, a scaling factor can be applied to scale and offset parameters (later if such need appears). It is therefore proposed to signal the camera parameters only for texture. There is an editor’s note about the depth camera parameters overwriting the texture camera parameters.

A question was asked what happens if the camera parameters are sent in the slice header. It was mentioned that it might be desirable to have the same constraint in the VPS and in slice header. The proponent answered that the update in the slice header is a separate issue. One participant liked the idea but did not think the adoption is needed at this meeting. There is another proposal (F0045) that supports similar functionalities.

It was agreed to have an offline discussion between the proponents of F0045, and the editor (Gerhard) to harmonize the proposals.

Further discussion on Thurs 10/31: There is agreement to specify the camera parameters in a loop of view order index rather than layer ID. Also, when both texture and depth are present as coded pictures, the camera parameters are sent for texture. When only texture or depth are present for a view, then only the camera parameters are associated with only that view.

Decision: Adopt (harmonized F0136/F0045).

14.1.1.1.1.1.1.1.165JCT3V-F0045 3D-HEVC HLS: Constraints on camera parameter signaling [Y.-L. Chang, Y.-W. Chen, J.-L. Lin, Y.-P. Tsai, S. Lei (MediaTek)]

In the current 3D-HEVC, camera parameters of a texture layer can be replaced with those of a depth layer of the same view, which may cause wrong depth-to-disparity conversion. In this contribution, a pair of missing braces related to camera parameters coded in the video parameter set (VPS) is first added. Then two methods are proposed to solve the ambiguity problem of the camera parameter signaling for layers with the same view order index.

For each view, the cp_present_flag can be sent for both texture and depth.

The depth layer of a view can override the texture layer of a view, in terms of camera parameters.

An editorial fix is reported:

Decision (BF): Add missing brackets in the loop related to the camera parameter signaling.

The first method is to allow at most one layer per view with its cp_present_flag `equal to 1 and to transmit camera parameters in the slice segment header extension only when cp_present_flag of the layer and cp_in_slice_segment_header_flag are both equal to 1.

In VPS, for each view, only one set of camera parameters are present, for whichever layer comes first.

So the cp_in_slice_segment_layer_flag controls all layers, however, the presence of the slice header level camera parameters are proposed to be dependent on cp_presesnt_flag, which is view specific.

Only when both cp_in_slice_segment_layer_flag and cp_present_flag are 1, the slice header level camera parameters can be sent.

The second method is to force the camera parameters to be the consistent for all layers of the same view.

In VPS extension, this method allows redundant camera parameter sets to be sent for texture and depth layers of the same view by requiring both sets to be identical.

In slice header, also redundant camera parameter sets can be sent for the texture and depth layers of the same view within the same access unit.

Three possible ways of sending camera parameters in the picture level.

1. Allow duplications or redundancy for texture and depth of the same view within the same access unit.

2. Send just to one texture (depth) picture of a view component, and allow slice header prediction (inheritance).

3. Consider other places for camera parameters.

It was commented that if the bits used for camera parameters are not significant, the current solution in 3D-HEVC (solution 1) may be acceptable.

Further study encouraged for this aspect if new evidence is provided.

14.1.1.1.1.1.1.1.166JCT3V-F0082 3D-HEVC HLS: On slice-level camera parameter signaling [Y.-L. Chang, Y.-W. Chen, J.-L. Lin, Y.-P. Tsai, S. Lei (MediaTek)]

This proposal targets at the same problem as the F0045: the relation between cp_in_slice_segment_layer_flag and cp_present_flag.

The first solution is the same as in F0045 for the condition of the slice header level camera parameters.

The second solution changes the cp_in_slice_segment_layer_flag to be view specific and put this as a condition of the presence of slice header level camera parameters.

The flexibility of supporting the additional flexibility (one view has update of the camera parameters while another does not have) seems to be a bit hypothetical, but no harm.

Several experts express preference of the second solution.

Decision: Adopt the second solution: the cp_in_slice_segment_layer_flag to be view specific and used as a condition of the presence of slice header level camera parameters.

2. Other slice header related topics

14.1.1.1.1.1.1.1.167JCT3V-F0044 3D/MV-HEVC HLS: HEVC compatible slice segment header in 3D-HEVC [K. Zhang, J. An, X. Zhang, J.-L. Lin, S. Lei (MediaTek)]

It has been discussed whether the additional syntax elements in slice header should be put in slice header extension of under the condition of nuh_layer_id unequal to 0.

Similar topics have been discussed during the SHVC/MV-HEVC development. The main reasons JCT-VC/3V decide to take the latter approaches are:

1. the extension bytes in slice header are (also) considered for additional info. in extensions for base layer;

2. The design of the slice header extension, is byte aligned and require a syntax element indicating the length of the extension (which is ue(v)) coded. There is also a gating flag in PPS to control the presence of the slice header extension.

However, it is reported that camera parameters are put in the slice header extension, which are not consistent with the presence of other syntax elements (for nuh_layer_id >0) in slice header.

Camera parameters are not required for the base layer (view), so it might be desirable to align the positions (or the presence conditions) of the camera parameters and other syntax elements (such as slice_ic_enable_flag).

Decision: Adopt the proposal to move the camera parameters from slice header extension to some place before the slice header extension in slice header under the condition of nuh_layer_id unequal to 0.

If changes are to be made, the MVCompatibleFlag should also be part of the condition.

3. DLT

Regarding the presence of DLT:

1. Whether it is good to potentially enable it in a picture level:

There were proposals of putting DLT in slice header, but probably it is not necessary to do slice level signaling for DLT at this stage.

Enabling certain level of flexibility might be desirable.

It was also reported that it might be possible to do prediction from parameter set to slice header with differential coding.

Decision: Move DLT signaling (DLTs for multiple layers) from VPS to PPS.

One question is whether the DLT tables of multiple views should be put in one PPS or separate PPSs (one per view).

Since in SHVC/MV-HEVC, SPS/PPS instances can be shared by multiple layers, if we put DLT in one PPS, it would be possible for multiple layers to use the same PPS.

2. If inter-layer DLT prediction is desirable:

There are multiple proposals addressing this issue. So the current preference is to enable the inter-layer DLT if the cost is acceptable.

14.1.1.1.1.1.1.1.168JCT3V-F0131 Depth Lookup Table Coding for 3D-HEVC [Y. Chen, H. Liu (Qualcomm), Y. Cai, S. Ma (PKU), M. Li, P. Wu (ZTE)]

The proposal includes two aspects single view coding and inter-view coding. In the first approach for single view coding, in the incremental table to signal the entranced to be removed. In the second method, the removal entry list is not signaled in with incremental table but as a loop of flags. The overall reduction of the bitrate is about 40%. The results for the single view method are 15% bits needed compared to the bits needed for the current design.

It was claimed by the proponent of a competing proposal that the current design has a problem because it depends on the bit depth, which is sent in the SPS. This issue would need to be fixed if the proposal is adopted. It was mentioned by another expert that the issue can be fixed and is not a major concern if the proposal is adopted.

It was also claimed by a proponent of a competing proposal that it is not clear what is the worst case number for the arbitrary DLT sequence.

Decision: Adopt the first part of the proposal (by changing the first syntax element from u(v) to ue(v) if DLT is present in VPS) and offline discussion with the proponent of F0138.

14.1.1.1.1.1.1.1.169JCT3V-F0217 Cross-check of Depth Lookup Table Coding for 3D-HEVC (JCT3V-F0131) [S. Yoo, S. Yea (LGE)] [late]

14.1.1.1.1.1.1.1.170JCT3V-F0138 3D-HEVC HLS: Inter-view Updating Mechanism for Coding of Depth Lookup Table (Delta-DLT) [F. Jäger (RWTH Aachen University)]

The proposal suggests updating mechanism for DLT coding. For the dependent views, an update regarding to the base view is coded by signaling only the differences between the look-up tables.

The approach results in 40% DLT rate compared to the current configuration.

A multi-range configuration also needs to code the ranges in which the DLT values are updated. The proponent suggest adopting the single range scheme, since the gains are similar and the

Offline discussion with the proponents of F0131 for evaluation of the coding performance on top of the adopted changes to the single-view coding and possible harmonization.

Further discussion on Thurs 10/31: With interview updating, further reduction of 15% in bits for DLT on top of F0131 single view DLT coding. Simple text modifications required to realize F0138, no change to syntax, only semantics. Revised semantics harmonized with F0131.

Decision: Adopt (inter view update).

14.1.1.1.1.1.1.1.171JCT3V-F0225 AHG7: Cross-check on Updating Mechanism for Coding of Depth Lookup Table (Delta-DLT) (JCT3V-F0138) [Jacek Konieczny (Huawei)] [late]

14.1.1.1.1.1.1.1.172JCT3V-F0139 AHG7 Related: Differential coding method for DLT in 3D-HEVC [M. Li, P. Wu(ZTE), K. Zhang, J. An, S. Lei (MediaTek)]

The proposal uses has two methods. The first method is similar to the one proposed in F0131. There is flag, which can be used to indicate which method is used. The second method is used to cap the worst case scenario which can be probably caused by the first method. The approach reaches similar gains that the approach in F0131.

First method is similar to F0131 (but the latter is assessed to be more mature w.r.t. syntax and semantics), see notes there.

Decision: Adopt the second part with the simplified syntax removing the difference between the minimum and the maximum DLT value.

A flag has to be used indicating whether the coding method of F0131 or F0139 method 2 is used.

The flag (residual_dlt_index_flag) as proposed in F0167 has been discussed.

This flag enables/disables the tool in F0159 when DLT is used.

Since the DLT now has been moved to the PPS, so the problem as identified in F0167 might be less serious.

In general, it sounds the flexibility of this flag (in terms of trading the efficiency with the complexity in a per slice basis) is favorable.

Further discussion in BoG on parameters to enable tools in HLS. See notes under F0256. It was agreed in the BoG to not take action on the particular slice level flag discussed above.

14.1.1.1.1.1.1.1.173JCT3V-F0168 Crosscheck on ZTE's proposal JCT3V-F0139 [Peng Lu, Lu Yu (Zhejiang University)] [late]

4. 3D specific MV-HEVC HLS proposals

14.1.1.1.1.1.1.1.174JCT3V-F0169 MV-HEVC HLS: depth representation information SEI message for auxiliary pictures [M. M. Hannuksela (Nokia)]

This contribution proposes the depth representation information SEI message for auxiliary pictures. The contribution is technically identical to the SEI message option presented in JCT3V-E0102. The JCT-3V decision on JCT3V-E0102 was, quoting JCT3V-E_Notes_d8.doc: “Decision: Adopt (and emphasize need further coordination with JCT-VC on auxiliary picture structure and constraints)”.

It is reported that this proposal depends on the adaptation status of JCT3V-F0031.

The motivation to use the depth representation information SEI message of MVC+D as a basis for the proposal was that the SEI message is used for a similar purpose in MVC+D and hence apparently contains all the necessary information. Furthermore, the syntax of the SEI message was followed as much as possible.

The main differences compared to the proposed syntax and the depth representation information SEI message of MVC+D can be summarized as follows:

1. 
   The SEI message documents the properties of all depth views in the bitstream. The proposed syntax structure is specified in a manner that it documents the properties of a single view/picture. The scalable nesting SEI message can include the proposed SEI message and indicate which auxiliary layers it applies to.
   
2. 
   The Z-axis reference view was not included in the proposed syntax and the units for the Z values are not specified, because these values are meaningful only relative to extrinsic camera parameters and at the moment there is no mechanism to indicate the extrinsic camera parameters in MV-HEVC.
   

The proposed SEI is always present in the scalable nesting SEI, such the association to a specific layer (view) is known. Note that this SEI concerns only one layer, instead of multiple layers, as in the counterpart SEI in MVC+D.

It seems that both changed bullets are simplifications compared with the similar SEI in MVC+D.

It is noted that the signalling of disparity values using arbitrary floating point values might be simplified based on integer values.

A general comment was given that the camera parameters are not currently supported in MV-HEVC. If that is supported later, the z-axis might still be useful to be present in this SEI message.

Decision: Adopt this SEI message in MV-HEVC, under the condition of JCT3V-F0031 being adopted.

14.1.1.1.1.1.1.1.175JCT3V-F0156 MV-HEVC HLS: Indication of sampling grid position [M. M. Hannuksela (Nokia), Y. Yan (USTC)]

This contribution studies alternatives for arranging the processing chain of downsampling to vertically half resolution for output on a polarizing stereoscopic display and MV-HEVC encoding/decoding of stereoscopic content with inter-view prediction. It concludes that coding vertically half-resolution sequences improved the RD performance for polarizing displays compared to the MV-HEVC anchor settings by about 3% in BD bitrate. Furthermore, the contribution concludes that the use of interleaved sampling grid positions between views improved the BD bitrate by 0.2 %-unit on average and by 2.5 %-unit at maximum relative the use of aligned sampling grid positions. It is proposed to include vertical sampling grid position information in the VPS extension.

The information sounds interesting. More accurate grid positions may introduce averagely 0.2% coding improvements, based on the measurement which considers the PSNR values of the packed frame.

In case of pre-coded content, it may be needed to have two versions of such information.

It is also commented maybe such a case may be relevant to frame-packing arrangement SEI message.

It might be possible in some situations, when the clients are not known, such information might not be always helpful. A feedback channel from the displays might need to be assumed.

For active shutter glasses, it might have some limitations.

This proposal only makes sense when the encoder knows the display characteristics at the receiver, which may occur in conversational systems. However, systems level metadata could be used in this case.

In other types of systems, the signaling could help select the appropriate upsamping filters, but the grid position and offset could be constrained by application standards.

No action.

2.