The Joint Collaborative Team on Video Coding (jct-vc) of itu-t wp3/16 and iso/iec jtc 1/ sc 29/ wg 11 held its tenth meeting during 11-20 July 2012 at the City Conference Centre (ccc), a k. a

JCTVC-J0536 AHG9: Mental cross-check of 2D compatible frame packing arrangement SEI (JCTVC-J0070) [M. Arena (??)] [late]
JCTVC-J0073 AHG9 High-Level Syntax: Graceful Handling of Frame-Packed Pictures by Ignorant Decoders [M. M. Hannuksela (Nokia)]

Among other things documents JCTVC-I0058 and JCTVC-I0072 discussed the compatibility of stereoscopic frame-packed content on decoders and renderers, which do not parse or process the frame packing arrangement SEI message. This contribution continues exploring similar topics and proposes the following modifications:

1. 
   Any number output cropping rectangles can be provided in the sequence parameter set. If no cropping rectangle is provided in the sequence parameter set, one cropping rectangle which equals to the extents of the decoded picture is inferred.
   
2. 
   A target output cropping rectangle, indicated by TargetPicCropIdx, is either specified by external means, or, when not specified by external means, TargetPicCropIdx is equal to 0.
   
3. 
   The cropping rectangle of TargetPicCropIdx is applied for the picture that is output from the decoder.
   
4. 
   For side-by-side (frame_packing_arrangement_type equal to 3), top-bottom (4) and “tiled” (7) frame packing types, the number of cropping rectangles has to be at least two and TargetPicCropIdx equal to 0 causes output of constituent frames of a single view only.
   
5. 
   When a constituent frame of a frame-packed decoded picture is output on a conventional two-dimensional display, upsampling is typically required, e.g. from half vertical or horizontal resolution to full resolution, to produce a picture for a uniform pixel grid. VUI indications to assist in such upsampling are proposed in the contribution indicating the horizontal and vertical spacing between samples when displayed on a uniform sampling grid.
   

Item 5 could be resolved by using sample aspect ratio; de facto in side-by-side or top-bottom frame compatible the effective ratio in both pictures is 2:1 or 1:2.

Several experts questioned item 4, whether more than one cropping rectangle is necessary in SPS, because any additional cropping window would be required to be assigned with some semantics. Also w.r.t. to conformance testing of normative decoder behaviour, it is desirable to have only one cropping window, as otherwise it would be necessary to define conformance for each of the windows. It is only necessary to have a small subset of conformance streams that test whether a decoder supports the cropping functionality – this is sufficient for the backward compatibility of devices which do not know the FC SEI message.

(Further discussion chaired by Y.-K. Wang.)

Suggestion: When the FPA SEI message is used, and the FPA type is such that at least one of the views is a coherent rectangle (e.g. side by side), the cropping rectangle in the SPS should be defined to output only one view.

It was commented that the FPA SEI semantics would need to change with this approach, and that perhaps a different SEI message should be defined since the semantics would be different.

Why to have the cropping window? One reason is that the area of the decoded picture be output may not be aligned with the minimal coding block size, and consequently padding may be needed.

It was discussed whether it is needed to have normative cropping window behavior. It was suggested that the the cropping window behavior may be specified in an informative way.

Suggestion: The one normative cropping window in SPS should cover one view only. Additional cropping windows should be specified by either SEI messages or external means.

It is expected that version 1 encoders and decoders would all understand the FPA SEI message.