International organisation for standardisation organisation internationale de normalisation

7.6MVC with depth

7.7JMVM analysis and optimization

7.7.1.1.1JVT-Y022 ( Info 2.2) [R. He, G. Jiang, M. Yu (Ningbo U.)] Determining decoding path in MVC

As mentioned in WG 11 document M13121, while compression efficiency is important in multi-view video system, many applications of multi-view video (e.g., free viewpoint video) reportedly also require the ability to randomly access different views from the compressed bitstream. To perform view switching in real-time, efficient random access is reportedly essential. For this purpose, when users access one frame of a multi-view video sequence randomly, it is reportedly necessary for the multi-view video system to be able to determine how to find and decode the corresponding frame quickly and efficiently. An algorithm was proposed in M13121to reduce the number of frames that a decoder must process to perform random access. To further improve the random access performance of the multi-view video system; a concept of a frame level matrix of prediction structure was presented, based on which a calculation method of the decoding path was presented. An example based on a hierarchical B picture prediction structure was used to compare the proposed algorithm with the algorithm in M13121. It was asserted that the proposed algorithm can obtain a better decoding path, so it was asserted to effectively reduce the decoding complexity and improve random access performance in the multi-view video system.
A non-normative decoder implementation speed optimization technique provided as information.

7.7.1.1.2JVT-Y023 ( Info 2.2/3.1) [G. Jiang, Y. Yang, M. Yu (Ningbo U.)] Evaluation of random access of MVC schemes

Random access on both temporal and spatial dimensions was proposed as a basic requirement for future multiview video applications. However, evaluation on random accessibility is asserted to be a critical problem in research and development of multiview video systems, while discussions and research had reportedly not been sufficient in the previous JVT meetings. This contribution provided a “universal definition” for random accessibility for multiview video systems through graph theory and stochastic methods. Although some evaluation methods were proposed by other works to measure random accessibility in some ways, the outcome scores in these methods are asserted to be equipment-dependent. In this contribution, the proposed definition and evaluation function for random accessibility is reportedly a general method measured by graph theory and stochastic procedures. Experiments were implemented, and it was asserted that the parameters of random accessibility will change according to the corresponding complexity of the MVC scheme. Furthermore, the proposed methods and functions can reportedly be used as evaluation tools to assess the performance of existing MVC schemes and the needs of future multiview video applications.

7.7.1.1.3JVT-Y026 ( Info 2.0) [M. Yu, Z. Peng, G. Jiang (Ningbo U.)] Statistical analysis of macroblock mode selection in JMVM

In the Joint Multiview Video Model (JMVM), the macroblock mode of each macroblock is reportedly decided by full searching of all modes. Excellent rate-distortion performance is reportedly achieved in this manner, but at the expense of a large computational complexity. This contribution reports statistical analysis results based on the JMVM reference software with regard to macroblock mode selection. The statistical results reportedly show that the proportions of selected macroblock modes are not equal. Most of the macroblocks are reportedly encoded with the SKIP macroblock mode, while the Inter1616 mode is reported to be the second most common. These two modes reportedly account for more than 58% of all macroblock mode selections. The motion-skip mode, which reportedly greatly improves rate-distortion performance, reportedly plays an important role in MVC and is reportedly applied in significant percentages for all test sequences. The proportions of other inter macroblock modes, such as Inter168, Inter816, Inter88 and Inter88Frext, are reported to be less than those of the SKIP and Inter1616 macroblock modes. Although the Intra modes are least frequently selected of all modes, they are reported to be indispensable to video encoding. It is asserted that these statistical analysis results may be helpful for designing fast macroblock mode selection algorithms.

7.7.1.1.4JVT-Y027 ( Info 2.2) [M. Yu, Q. Xu, G. Jiang, Z. Peng (Ningbo U.)] Bandwidth distortion model for MVC in interactive system

Traditionally, rate distortion performance with respect to all views is used to evaluate the encoding efficiency of multiview video coding (MVC) schemes, and some comparative study of MVC prediction structures has been given. Under this criterion, the Hierarchical B Pictures (HBP) structure, which benefits from the carefully designed prediction structure in the temporal domain as well as the spatial domain, is regarded as one of the most efficient MVC schemes. Even though the coding efficiency is substantially improved by using temporal and interview correlations, it is reportedly desired for the server that only partial compressed data will be interactively selected and transmitted to the client due to the limit of network bandwidth and limited resources of clients, and the fact that the user only watches several specific viewpoint views during a relatively long period instead of needing all viewpoint views, and the number of views which should be transmitted is relative to the user’s selection, the type of display and the view dependence of the coding scheme. From this point of view, rate distortion analysis only reflects the memory needed for storing the total bitstream of the encoded multiview video signal. It is intuitive that view dependency enables a coding scheme to achieve high coding efficiency because more interview correlations can be exploited, however, it is asserted that this will also result in more viewpoints that have to be transmitted even though some of them are not directly selected by the user (being involved only in the predication of the needed views). Therefore, the traditional rate distortion, referred to as the encoding rate distortion model, just evaluates the “static performance” of an MVC scheme in coding efficiency but not the “dynamic performance” associated with the user’s interactive selection when used for an interactive multiview video system. A transmission bandwidth distortion model was suggested for evaluating multiview video coding (MVC) schemes in an interactive multiview video system in this contribution, instead of the traditional encoding rate distortion model at the encoder, and comparative results of some typical MVC prediction structures under this suggested evaluation model were also given.

7.7.1.1.5JVT-Y028 ( Info 2.2/3.1) [Y. Yang, M. Yu, G. Jiang (Ningbo U.)] Dependency func for perf analysis of MVC

Compression related requirements in future multiview video applications reportedly have close relationships with prediction dependency structures of multiview video coding (MVC) schemes. Evaluation of the complexity of prediction dependency structures of MVC schemes will reportedly be very important for assessing both the needs of future multiview video applications and other performance issues in multiview video systems. This contribution reported on an investigating of these issues and defines a complexity measure for prediction dependency structures of MVC schemes. The contribution defined a view dependency function based on graph theory to describe the complexity and coupling degree of an MVC scheme. Particular MVC schemes were investigated by the proposed methods. The dependency function was further applied to evaluate random accessibility and decoding effort.