International organisation for standardisation organisation internationale de normalisation

Non-normative contributions

4.2.1.1.1.1.1.1.128JCT3V-C0039 A New Hybrid View Synthesis Method for View Interpolation and Extrapolation [I. Koreshev, M. T. Pourazad, P. Nasiopoulos (UBC & TELUS)] [late]

This contribution presents a hybrid view synthesizing approach for synthesizing virtual camera views. In this report the proposed method is used to both interpolate and extrapolate virtual views from either one or more available camera views respectively. The performance of the proposed method is evaluated using a series of subjective evaluations of synthesized videos and compared with that of the existing VSRS method. The experimental results indicate that the subjective quality of the synthesized 3D views using this approach is higher than the synthesized views generated by the view synthesis reference software.

Hybrid solution between hole-filling and warping. Warped picture used in a secondary hole-filling step.

Subjective results reported that in particular show advantages in case of extrapolation (e.g. picture boundary).

Subjective viewing to be organized (help by N. Stefanofski).

Potentially include as alternative synthesis method (additional to VSRS).

A viewing session was held Wednesday 16:00. 3 sequences (Ballons, Kendo, GTFly) were shown.

Comparisons were made against VSRS 3.5.

Approximately 8 experts participated in the viewing and confirmed the following observations:

Remarkable difference on Balloons (hybrid method able to repair artefacts), Kendo hard to distinguish, GTFly was also better with hybrid method.

Extrapolation examples were also shown, where the difference was even more observable.

The software is currently written in Matlab, but the proponents expressed willingness to re-write the same algorithm as C code until the next meeting. The willingness to put this under the same software licensing header used currently in ATM and HTM was also expressed. This should be possible, as it is reported that no external libraries are required.

This contribution to the next meeting would be very welcome, and would likely be included as alternative synthesis method.

4.2.1.1.1.1.1.1.129JCT3V-C0090 JCT3V – Improvement of the rate control for 3D multi-view video coding [W. Lim, D. Sim, I. Bajić]

At the previous two meetings, we have proposed a rate control algorithm for multi-view video coding (JCT3V-A0151, JCT3V-B0030). Those contributions presented a rate control scheme for multi-view video coding based on the unified rate-quantization (URQ) model for HEVC described in JCTVC-H0213 and JCTVC-I0094. In the contributions, there were two primary algorithms for the extended views: (1) initial frame level QP setting of the first frame of a GOP of an extended view, and (2) inter-view MAD prediction based on the depth map. Since the last meeting in Shanghai, we have fixed the problem related to the consideration of frame boundary in MAD prediction. In the present contribution we also describe how our proposed method can be used with any rate control scheme, not necessarily URQ. To evaluate the proposed scheme, 3D-HTM4.0.1, which uses hierarchical QP setting, is employed for the constant bit rate (CBR) case. We found that the accuracy of our proposed multi-view rate control was over 99%, which came with a 0.03dB of average PSNR improvement compared to the URQ model-based rate control scheme.

Results on quality fluctuation? No.

In general, the results indicate that the scheme works. Loss in PSNR average about 1.3 dB.

Several experts expressed opinion that this would be beneficial to include (not in CTC), but it is suggested to rather wait until migration of the HTM software into a newer HM version is finished.

9. Quality assessment

4.2.1.1.1.1.1.1.130JCT3V-C0032 A human visual system based 3D video quality metric [A. Banitalebi-Dehkordi, M. T. Pourazad, P. Nasiopoulos (UBC & TELUS)]

This contribution proposes a full-reference Human-Visual-System based 3D video quality metric. In this report, the presented metric is used to evaluate the quality of compressed stereo pair formed from a decoded view and a synthesized view. The performance of the proposed metric is verified through a series of subjective tests and compared with that of PSNR, SSIM, MS-SSIM, VIFp, and VQM metrics. The experimental results show that HV3D has the highest correlation with Mean Opinion Scores (MOS) compared to other tested metrics.

Generation of “cyclopean view” from stereo images.

frequency weighting in DCT domain.

SSIM-like measure after IDCT.

Results reported (Spearman, Pearson) show advantage, but other measures seem to fall behind the values reported previously by EPFL.

Training and test sets were different.

Further investigation of the metric should be performed in the context of the Qualinet initiative (proponents are associated with Qualinet).

4.2.1.1.1.1.1.1.131JCT3V-C0202 AHG9: Correlation analysis between MOS data collected on stereoscopic and autostereoscopic displays [Tomasz Grajek, Olgierd Stankiewicz, Krzysztof Wegner (Poznan Univ.)] [late]

This document tries to answer the question whether subjective evaluation can be performed only on one type of 3D monitor or various types of displays should be considered. To achieve this, MOS data obtained during MPEG evaluation of responses to CfP have been analysed. Correlation was analysed based on charts where on one axis MOS obtained on the stereoscopic monitor while on the other axis MOS obtained on the autostereoscopic monitor for Class A (a), Class C (b) and Class A & C (c) sequences were plotted, respectively. Pearson corr. of approx. 0.92 and Spearman corr. of approx. 0.95 were found. Therefore, it is concluded in the contribution that it is worth considering to perform 3D subjective tests on one 3D monitor only (huge time and effort savings).

6. Plenary Discussions and BoG Reports

   1.