The Joint Collaborative Team on Video Coding (jct-vc) of itu-t wp3/16 and iso/iec jtc 1/sc 29/wg 11 held its second meeting during 21-28 July, 2010 at the itu-t premises in Geneva, ch

JCTVC-B110 [C. Auyeung, A. Tabatabai (Sony)] Separable adaptive loop filter

In this proposal, separable and non-separable loop filters were compared. To provide better trade-off of complexity and subjective quality, this document proposes to allow the encoder and decoder to switch between non-separable filters and separable filters. In particular for the hierarchical B coding structure, non-separable filters were applied to filter the I pictures and the separable filters were applied to filter the P and B pictures.

This document proposes a tool-experiment to compare the performance of non-separable and separable loop filters.

The proposal was tested under test conditions established in the in-loop filtering AHG.

The software context was KTA2.6r1 with quadtree adaptive loop filtering, with support for separable filtering added.

If all pictures are coded separably versus all pictures coded non-separably, there are reportedly substantially more artifacts in the separable case. If only the I picture is filtered non-separably, this effect is reportedly substantially mitigated (although not entirely eliminated).

A participant asked whether it might be adequate to use non-separable filtering for an I picture and not perform the adaptive loop filter for the other pictures. This may merit further investigation.