International organisation for standardisation organisation internationale de normalisation

14TE12

14.1TE12: Overall TE12 summary

14.1.1.1.1.1.1.1.1JCTVC-C225 TE12: Summary of evaluation of TMuC tools in TE12 [K. McCann (Samsung/Zetacast)] (missing prior, uploaded first day)

This contribution was a summary of tool experiment 12 (TE12) on exploring the performance of individual tools in the Test Model under Consideration (TMuC). Detailed results were reported by the TE12 participants in individual input documents.

The work of TE12 was very substantial, and it was a very important effort.

The incremental contribution of each individual tool in the TMuC is highly dependent on the settings of the other tools in use; in effect, it is a multi-dimensional non-linear system. The starting points used in TE12 were the default configurations of the TMuC specified in JCTVC-B300, since these represent the current best estimate of realistic use cases.

Each tool in the TMuC was tested individually as follows.

• 
  Some tools are on/off in nature (e.g. ROT). This type of tool was tested by being turned on (if off in the default configuration) or off (if on in the default configuration) unless it has a direct alternative (see below).
  
• 
  Some tools have direct alternatives either within the TMuC itself or else added to the software for comparison purposes (e.g. PIPE and the LCEC entropy coding within the TMuC, with CABAC for comparison). This type of tool was tested by being replaced by each direct alternative in turn (e.g. PIPE was tested against both LCEC and CABAC).
  
• 
  Some tools can have minimum or maximum limits defined (e.g. Large Transform block size). This type of tool was tested by having either the minimum or maximum limit made stricter as appropriate (e.g. maximum transform size being reduced from 64x64 to 32x32).
  

Tools for which there is no alternative in the TMuC (e.g. inter transforms) were not tested.

In most cases, only one tool at a time was exercised by giving it an alternative setting from that of the default configuration. However, some limited testing of combinations of tools was also performed, in order to improve understanding of the interaction between tools (e.g. between ROT and MDDT).

All of the tools listed in the Tool Experiment 12 test plan were evaluated, with the exception of one encoder-only test on CABAC-based RDO on/off. In most cases the tests used TMuC version 0.7 and subsequent versions of the TMuC released by the Software AHG. However, three of the tests used company-specific branches: PU-Based Merging (tested with TMuC 0.8-hhi-bugfix), Enhanced CIP (tested with 0.7-BBC branch) and ALF default/3-input ALF (tested with 0.7-panasonic branch).

Detailed results were reported by the TE12 participants in individual input documents.

It was anticipated that the results of this work would be used to guide improvements in default operating points. In addition, the TE work provides insight on how the various tools contribute to the overall performance of the TMuC and hence helped inform the process of forming the first Test Model. However, it should be noted that the various tools in TMuC interact in a complex non-linear manner, so that careful interpretation of the results is required.

A total of 70 documents directly relating to TE12 were registered. The categories were as follows:

1. 
   Unit definitions
   
2. 
   Motion representation
   
3. 
   Intra-frame prediction
   
4. 
   Transforms
   
5. 
   Filtering
   
6. 
   Entropy coding
   
7. 
   Other decoder tools
   
8. 
   Encoder-only tools
   

Most subtests used TMuC 0.7, while some used 0.7.x (which would include some bug fixes that do not affect the default configuration). All of the 0.7.x versions should be able to be well compared to the distributed anchors. Some variants are noted below:

• 
  PU merging testing used "0.8 HHI bug fix" software;
  
• 
  Note that a special patch was used in Enhanced CIP on/off testing as discussed in JCTVC-C213 with cross-check JCTVC-C297, which are not classified here as TE12.
  
• 
  "0.7 Panasonic branch" was used for 3-input ALF tests (e.g., JCTVC-C214).
  
• 
  LCEC and PBIC tests used 0.7.4.
  
• 
  CABAC based RDO on/off encoder-only evaluation was not tested.
  

14.1.1.1.1.1.1.1.2JCTVC-C320 TE12: Summary of results of evaluation of TMuC tools in TE12 [K. McCann (TE coordinator)] (late registration)

Due to the large effort and large number of contributions involved in TE12, the original summary report for the TE (JCTVC-C225) did not contain a summary of the results. Such a report of results was prepared during the meeting and was reported in this contribution. Although the effort involved in the preparation of this contribution was greatly appreciated, there was insufficient time to properly review the document at the meeting due to the late arrival of the document.

14.2TE12 TMuC coding unit definitions

14.2.1.1.1.1.1.1.1JCTVC-C029 TE12: Report on 64x64 versus 32x32 maximum coding unit size for low complexity configuration [P. Pandit (Harmonic)]

This contribution presented results for the Max CU size experiment under TE12 for the low complexity configuration. Simulations reportedly showed that when using CU32 size, for Intra only case the average BD-Bit rate gain is about 0.2%, for Random access case the average BD-Bit rate loss is about 7% and for low delay case the average BD-Bit rate loss is about 8.8%. Some comments on the visual quality were also made. Cross verification of the results was reportedly done with Motorola.

• 
  The tested cases included LC configuration for intra, low delay & random access
  
• 
  Visual comparison was done for QP 37
  

14.2.1.1.1.1.1.1.2JCTVC-C044 TE12: Results for experiments on max CU size, RDOQ and AIS [F. Bossen, T. K. Tan (NTT DoCoMo)]

Other aspects of this contribution are discussed in another section below.

This contribution reported the simulation results for Tool Experiment 12 as described in JCTVC-B312r5. Three sets of experiments were simulated. The experiments are the comparison of the maximum coding unit (Max CU) size of 32 and 64, the comparison of rate distortion optimized quantization (RDOQ) on and off, and the disabling of the adaptive intra smoothing (AIS) with the default set to always off.

• 
  The tested cases included HE configuration for intra, low-delay & random access
  
• 
  Decoder run time was 2.5x longer without 64x64 (the reason being related to adaptive scanning in MDDT)
  
• 
  With 32x32 and smaller only, there was marginal loss in intra-only, around 3% average for the inter cases
  
• 
  Most of the gain was in classes B and E (as was the case in JCTVC-C029)
  

14.2.1.1.1.1.1.1.3JCTVC-C259 TE12: Results for experiments on Max CU size [Y. Yu, K. Panusopone, L. Wang, S.-T. Hsiang, F. Ishtiaq (Motorola)]

Test Experiment 12 was set up at the second JCT-VC meeting. TE12 aimed to explore the performance of individual tools in the Test Model under Consideration (TMuC). As part of TE12, the CU size was tested to decide the best maximum CU size between 32 vs 64. The simulation results were cross checked with NTT Docomo and Harmonic. From the simulation results, it was found that CU size 64 is consistently better than CU size 32. Therefore, the contributor recommended that a maximum CU size 64 should be used in the TMuC.

The results mostly confirmed the findings of JCTVC-C029 and JCTVC-C044 (a different TMuC version was used, with small deviations in PSNR)

It was suggested to possibly restrict the intra case to 32x32 (note: there were currently only 5 prediction directions used in intra 64x64).