International organisation for standardisation organisation internationale de normalisation

5Non-CE Technical Contributions

5.1Clarification and Bug Fix Issues

5.1.1.1.1.1.1.1.1JCTVC-H0486 Suggested figures for the HEVC specification [C. Fogg (Harmonic)]

This contribution was a suggested editorial improvement – consideration was delegated to the editor.

5.2HM settings and common test conditions

No contributions were noted to be specific to this subject area.

5.3HM coding performance

See also JCTVC-H0116 and JCTVC-H0022.

5.3.1.1.1.1.1.1.1JCTVC-H0063 Coding efficiency comparison between HM5.0 and JM16.2 based on PQI, PSNR and SSIM [Y. Zhao (Zheijang Univ.)]

(Random-access configuration test only.)

This contribution compared the compression efficiency of HM5.0 and JM16.2 (the AVC codec generating the AVC anchors in the CfP) under the "random-access, high efficiency" test conditions. Three objective quality metrics (PSNR, SSIM, and PQI) were used to evaluate video quality and thus to evaluate the average BD bit rate saving over 15 sequences in class A to D. The three metrics reportedly each measure around a 40% BD bit rate reduction on average HM5.0 for relative to JM16.2 when using the "random-access, high efficiency" configuration.

AVC anchors from CfP were used. Compared to common test conditions for HM, the lowest QP was removed (and a high QP was included) to get a better overlap between the curves. Several experts suggested that it would be a better comparison to remove the lowest rate points for AVC to get a more useful range of matching acceptable visual qualities.

PQI and SSIM provided very similar numbers as PSNR – and it was questioned whether they are really better matching the subjective visual quality.

The BD bit rate performance of HM anchors after the QP adjustment was reported as follows:

Class	Sequence	BD bit rate performance of HM anchors
		PSNR	SSIM	PQI
A	S01_Traffic	-40.27%	-38.66%	-37.31%
	S02_PeopleOnStreet	-32.72%	-34.32%	-31.68%
B	S03_Kimono1	-44.61%	-45.73%	-38.92%
	S04_ParkScene	-34.60%	-35.47%	-35.33%
	S05_Cactus	-39.18%	-37.06%	-39.63%
	S06_BasketballDrive	-45.49%	-48.26%	-41.53%
	S07_BQTerrace	-51.57%	-52.79%	-50.51%
C	S08_BasketballDrill	-41.86%	-41.88%	-46.04%
	S09_BQMall	-40.51%	-40.11%	-44.43%
	S10_PartyScene	-37.14%	-36.62%	-54.58%
	S11_RaceHorses	-38.40%	-39.87%	-50.59%
D	S12_BasketballDrill_416	-31.01%	-34.27%	-36.87%
	S13_BQMall_416	-50.64%	-41.88%	-49.15%
	S14_PartyScene_416	-31.25%	-27.58%	-36.55%
	S15_RaceHorses_416	-28.21%	-29.18%	-32.54%
Average		-39.16%	-38.91%	-41.71%

5.3.1.1.1.1.1.1.2JCTVC-H0360 Comparison of Compression Performance of HEVC Working Draft 5 with AVC High Profile [B. Li (USTC), G. J. Sullivan, J. Xu (Microsoft)]

This contribution is a further study of the relative objective (i.e. PSNR-based) compression performance of HEVC and AVC High Profile. It builds upon the prior work reported in JCTVC-G399, updating the results by using the latest available reference software (JM-18.2 and HM-5.0) and configuration common conditions. Experimental results reportedly show that, when compared with JM-18.2:

1. 
   for the high-efficiency all-intra configuration, HM-5.0 can save about 23% in bit rate;
   
2. 
   for the high-efficiency random-access configuration, HM-5.0 can save about 33% in bit rate; and
   
3. 
   for the high-efficiency low-delay configuration, HM-5.0 can save about 41% in bit rate.
   

It is noted that these results are based on PSNR measurements, which are an imperfect substitute for subjective quality assessment, and that the subjective performance of HEVC may actually be generally somewhat better than its objective (PSNR) performance may indicate.

A combined weighted (with 6:1:1 weighting) average YUV PSNR measure was used for the quality measurement.

The summary test results are shown in the tables below.

Table for All-intra case

	Y	U	V	YUV
Class A	-24.5%	-21.9%	-16.8%	-23.8%
Class B	-24.0%	-27.6%	-24.5%	-24.5%
Class C	-20.9%	-25.9%	-26.9%	-22.2%
Class D	-17.0%	-20.9%	-22.0%	-18.1%
Class E	-28.4%	-19.7%	-24.4%	-27.4%
Average Without F	-22.6%	-23.8%	-23.6%	-23.0%
Class F	-24.3%	-23.2%	-24.7%	-25.0%

Table for Random access case

	Y	U	V	YUV
Class A	-33.3%	-31.4%	-29.6%	-33.0%
Class B	-41.9%	-43.9%	-39.6%	-42.0%
Class C	-32.1%	-36.5%	-37.1%	-33.1%
Class D	-30.1%	-34.6%	-36.2%	-31.3%
Class E
Average Without F	-35.0%	-37.8%	-36.7%	-35.6%
Class F	-27.5%	-30.2%	-32.0%	-28.6%

Table for Low delay case

	Y	U	V	YUV
Class A
Class B	-46.3%	-46.9%	-47.0%	-46.5%
Class C	-38.5%	-44.4%	-45.1%	-39.7%
Class D	-35.3%	-43.3%	-43.4%	-36.7%
Class E	-49.9%	-46.1%	-49.7%	-49.7%
Average Without F	-42.3%	-45.2%	-46.2%	-43.0%
Class F	-35.1%	-40.7%	-42.2%	-37.0%

The reported results were basically similar to what was reported at the preceding meeting in JCTVC-G399.

The biggest difference from last time was probably due to using 8b vs. 10b encoding for the HE configuration.

5.3.1.1.1.1.1.1.3JCTVC-H0562 Informal Subjective Quality Comparison of Low-Delay Compression Performance of HEVC Working Draft 5 with AVC High Profile [F. Kossentini, N. Mahdi, H. Guermazi, M. Horowitz, S. Xu (eBrisk Video Inc.), B. Li (University of Science and Technology of China), G. J. Sullivan, J. Xu (Microsoft Corp)]

(Low-delay configuration test only.)

This contribution presented the results of an informal subjective quality comparison between the current state of the HEVC draft standard and the AVC High Profile (HP) for low-delay configurations, using five Class B (1080p) and Class E (720p) video sequences. More specifically, the study compares the high efficiency, low delay configuration of the HEVC reference software against a similarly configured AVC HP JM encoder. These tests were suggested to be complementary to the formal tests performed by AHG22, by focusing on low-delay rather than random-access configurations.

The encoding methodology used here was reportedly as described in JCTVC-H0360 (very similar to that previously used for JCTVC-G399), since that methodology was reportedly designed to reflect relevant application scenarios and to enable a fair comparison to the maximum extent feasible, i.e. using comparable quantization settings, reference frame buffering, etc. Relative to JCTVC-G399, the HEVC software version was updated to version HM 5.0, the AVC software version was updated to version JM 18.2, and the JM configuration was updated to maximally reflect the non-normative aspects of the HM 5 common test configuration.

When viewing HEVC encoded video side-by-side with AVC video encoded at approximately two times the HEVC average bit rate per sequence, in approximately 75% of the test cases and for four of the five tested encoding pairs, the (mostly non-expert) viewers reportedly indicated either that they had no preference, or that they preferred the HEVC encoded video. For three of the five encoding pairs, the number of people who expressed a preference for the HEVC encoding was substantially greater than (by a ratio of 1.7 to 1 or more) the number of people who expressed a preference for the AVC encoding (and in one of the two remaining cases, the number differed by only 1). The selection of which encoding was displayed on which side for the side-by-side viewing was established in a randomized double-blind fashion. It is suggested that the results generally confirm that the HEVC design (as represented by HM 5.0) exhibits a substantial improvement in compression capability beyond that of AVC (as represented by a similarly-configured JM 18.2) for low-delay video applications.

Target and Achieved Bit Rates for the Test Bitstreams: