International organisation for standardisation organisation internationale de normalisation
Yüklə 4,35 Mb.
|
| 3D-AVC HLS (10):
Note that some proposals relating to SEI messages may be also applicable to 3D-HEVC design. Parameter Sets
In this proposal, it is required that a non-base view texture or depth uses a different SPS than the SPS used by the base view texture. Some extensions in the SPS further indicate the control of enabling/disabling of some new coding tools. To use the same structure for depth and texture SPSs, the order of the view components is signalled in the SPSs. One loop is used to indicate the order of depth, with respect to its corresponding texture, e.g., to support TDDDTT, wherein T indicates a texture view component and D indicates a depth view component. A question was asked on whether there is a substantial advantage of having a very flexible coding order in terms coding efficiency – it seemed that there was no such advantage. It was commented that this signalling has no real impact for the MVC-based 3DV decoder. For an AVC-based 3DV decoder, it is not more useful than just identifying, for each view, whether the first view component is texture or depth. It was mentioned that for each view a flag can be signalled to indicate whether the texture view component or the depth view component is the first one in the decoding order within an access unit. So the T0D0D1D2T2T2 structure can be simplified to T0D0D1T1D2T2. The group decision was to introduce a flag for each view to indicate the decoding order, e.g. T0D0D1T1D2T2, in the SPS of EHP. It was agreed that slice header syntax clean-ups can be achieved based on the order which is denoted as e.g. T0D0D1T1D2T2 in SPS. Editorial MB-level syntax improvements can be achieved based on such an indication in the SPS.
This proposal introduces a syntax structure called Camera Parameter Sets (CPS) to transmit coded camera parameters. It was said that the proposed method has better error resiliency and supports random access and that these parameters have a different requirement of time delay. The CPS concept is proposed again here as previously, with some modifications. It achieves a similar functionality as a Depth Range Update (with nal_unit_type 16) in the current WD. And the purpose of the CPS is to replace the Depth Range Update (DRU). Camera parameters are further categorized into subsets. Prediction between CPS subsets is enabled. Prediction within a CPS subset is also enabled. A CPS_id syntax element is added in slice header, which is similar to an APS id in HEVC. Additionally, an index to the subset of the CPS is further signalled in the slice header. Regarding efficiency, for the current ATM, 56K bits are needed for GT_fly, while 45K bits are needed in the current proposal. Slightly more bits are required for other sequences compared to the ATM. Software has been developed, but not made available with the contribution. The proposal was not cross-checked. The saved bits are distributed to all the access units of a coded video sequence. The total bit savings was said to be not significant. Prediction between CPSs is disabled. In the current ATM design, prediction between DRUs is enabled. When present, the DRU applies only to the current access unit. One aspect of this proposal is to remove the depth range signalling from the SPS; the same concept was proposed by Sony. It is said that the camera parameters of multiple access units are first processed before the first access unit is coded. It was remarked that the proposal looks interesting and requested for more details to be figured out before the next meeting. Cross checking of the software is also encouraged to better understand this proposal. Further study of the topic was recommended.
This proposal is similar to the one proposed by the same authors in the HEVC category for the same topic. The purpose is to support more flexible slice header prediction. The syntax elements in a slice header are grouped to 4 slice header sub-groups. The source of the reference view component for each of the 4 sub-groups, from which a slice header can be predicted, is defined in SPS. Note that multiple sources are defined in the SPS. For a given source of a sub-group, it can be “not predicted”. The reference view is not directly signalled. It was commented that it might be a bit complicated to use such a flexible design. It was agreed that no changes in SPS are needed and three sub-groups of the slice header syntax elements can be identified, each of which has a reference view component, texture or depth signalled in the slice header: reference picture list construction related syntax elements, reference picture marking syntax elements, and most of the remaining syntax elements.
A more flexible slice header prediction scheme was proposed in this contribution. In each slice header, the reference view component is directly signalled, with a delta of view order index. If the delta is not zero, a flag is sent to indicate whether the reference view component is texture or depth. It was commented that the current WD design may not be efficient in terms of predicting reference picture list construction related syntax elements. This might imply that it might be beneficial to group the slice header syntax elements into two groups: one having the syntax elements for the reference picture list construction, and one with most of the remaining syntax elements. It was commented that the prediction of reference picture list construction (and maybe also reference picture marking) can be added on top of this proposal. It is possible to have multiple source reference view components signalled in the slice header; thus no modifications would be needed in the sequence parameter set. It was decided to have a harmonized design for slice header prediction in EHP. A comment was made that this concept might be new and thus not preferable to the HP profile. Other participants mentioned that this technique, as a concept, was not new and has been stable in the ATM software since the first release of this software. A comment was also made that the idea cannot be extended to the prediction among the texture views in HP. Further study was recommended regarding whether to enable the slice header prediction of depth in HP.
Reference picture list modification is proposed to handle the view synthesis reference. In addition, the proposed reference picture list modification syntax elements also indicate the source of the view, which is used for view synthesis prediction. An example is given that when two views are coded and one view in the middle is to be coded, one may need to signal whether the left one or right one of these two views is used to warp and generate the synthesized view. It was commented that the reference picture list modification syntax doesn’t have to be the placeholder for the source of the view synthesis picture. It was proposed that EHP should contain only one VSP picture: to leave it open and further study the benefit of having multiple VSP pictures. Two aspects can be discussed separately:
After further discussion, it was agreed to adopt the proposal. SEI messages
Two different types of depth maps are proposed. One bIt was proposed to indicate the type; further extensibility might be needed. The current depth acquisition SEI message allows all camera parameters to use inter-prediction. It was proposed to disable the prediction of camera parameters, either between views, or across the pictures in a given view. For GT_fly, a minor efficiency loss is identified. It was commented that the reducing prediction may lead to very small loss of the similar signalling inside the coded. Regarding moving num_frames to a depth acquisition_info() syntax table: num_frames indicates the duration of the camera parameters. It was noted that num_frames now may be assigned for different subsets of the camera parameters and it was proposed to make it applicable to all the parameters. This was agreed. Regarding all_frames_equal_flag: a more flexible design is proposed, noting that some camera parameters never change, while some camera parameters can change. This was a more flexible design when at least some camera parameters change. This was agreed. It was asked, if disparity is signalled, what is the referencing view corresponding to a disparity value? It is signalled. The proponent mentioned this might be useful. This may be less important compared to depth related info. signalled in SEI. We need to define the meaning of the depth values. This seems to be a valid problem, and further study is needed, related to depth_representation_type. SEI messages for stereo adaptation There was a merged proposal in m25256 from Ericsson and Zhejiang University to have merged syntax design in SEI. This included the following:
It was suggested that the functionality seems to be needed but we don’t yet have a clear harmonized syntax design. One scheme is aiming at carrying the information related to acquisition, while the other is related to content, thus requiring picture level signalling. It is agreed after potential adoption, a showcase is to be provided. It was commented that the syntax and semantics specifics in m25256 are not in such good shape. Further study was therefore recommended – planning to adopt m25256 after the showcase is provided and a clean syntax proposal is provided.
It was proposed to signal stereo pairs, which might contain a synthesized view. The current CTC allows only negative disparity, but not positive disparity. An indication of the suitable display configuration was proposed, to signal which view of a pair is chosen and to which position a virtual view is generated. It was noted that the disparity can be scaled based on the physical width of the display and the picture resolution (width) of the display. Associated information is proposed to be added in the VUI.
It was asked whether the proposal could be put in SEI. It was suggested this is more suitable in VUI than SEI. It was asked whether there is an implication that there is a most suitable display. It was commented that one set of info. is not sufficient for heterogeneous devices.
This proposal tries to address the issue of stereo content adaptation. The same principle applies to MVD content. The baseline is to be scaled based on the screen width. Translation and sensor shift values are said to both be inversely proportional to the width of the display. It was proposed to add the following in VUI:
It was commented that in real applications, such high precision parameters might not be needed. The sensor shift can be calculated from the depth acquisition SEI. It is assumed that sensor shift is already introduced during content production. It was commented that the proposed SEI may be combined in the depth acquisition SEI. A concern was raised that putting similar info. in both VUI and SEI might not be needed.
It was proposed to signal a recommend range of the position of the virtual view. Another aspect of the proposal is that the range can be signalled in the 3DVC view scalability information SEI. A flag for each left/right view indicates whether the range is within the left/right view. A mechanism is proposed to enable the reuse of the range by referring to an operation point. It was commented that the semantics might need improvement. It was commented that this is very dependent on the 3D renderer. It is suggested that how the parameters can be generated should be shown.
A zero_parallax syntax element was proposed that specifies that the depth value of zero parallax is provided in an SEI message. A baseline distance is also proposed to be placed in an SEI message.
An SEI message to indicate sub-bitstream profile/level was proposed. This proposal was similar to the scalability info. SEI message in the scalable extension of MPEG-4 part 10 AVC. Separate presence flags for profile and level were proposed to indicate whether the profile and level syntax elements are present. So it is possible that a profile indicator is not present but a level indicator is present for an operation point. Even when two operation points have the same profile/level, they are still signalled for both. The level signalled in the SPS is defined in a way that it is applicable only to a specific layer, but not to the whole operation point. The level of an operation point is signalled in the proposed SEI message. The major motivation of having the level signalling scheme redefined is that the level signalled in SPS might not be applicable to more than one view. This problem can be solved by removing the profile/level info from the SPS. This might be dependent on the VPS design. Further study was encouraged.
Signalling View Synthesis RDO (VSO) related information. The motivation of this proposal is that the view synthesizer of VSO is different from the view synthesizer of a specific device. A flag is used to indicate whether VSO is used. Further indications are introduced to indicate to which extent the encoder depends on VSO. It was commented that such additional indications are most likely not useful, as they are not easy to use for a 3DV render. It was further discussed whether the flag is needed. It was suggested that even it is helpful, it can be present only in an SEI message. It was commented that an encoder typically doesn’t provide any guarantee to the quality of the coded video. So to signal this flag, even in an SEI message, might be quite different from what we typically do. It was commented that the need of the flag may be related to the more generic discussions on whether VSO is always beneficial in the end-to-end 3DV system.
Yüklə 4,35 Mb. Dostları ilə paylaş:
|