International organisation for standardisation

5.2.4MPEG-D Unified Speech and Audio

Open Issues

The Chair identified the following open issues:

• 
  Selection process
  
• 
  Initial selection
  
• 
  Refined selection
  

Wednesday Discussion

Werner Oomen, Philips, presented a spreadsheet of combined results of the item selection listening test data. This presented three columns showing three pooling of data:

• 
  all mono tests
  
• 
  all stereo tests
  
• 
  all tests
  

For each test for which there is listening data, the analysis counted how many times a given item satisfied at least one of the following criteria:

• 
  min(AMR-WB+)
  
• 
  min(HE-AAC v2)
  
• 
  max(abs(diff(AMR-WB+ - HE-AAC v2)))
  
• 
  min(VC)
  

If that count is greater than one, the item receives a “check,” and the total number of “checks” is displayed in the spreadsheet. If the spreadsheet column is 1, then the process is done. If it is 2, then the item in “1” is removed from consideration and the process is repeated. This iterative process is repeated for 3, 4, etc. The VC score is the best of HE-AAC-V2 and AMR-WB+ for a given item as pooled over all listeners in a test. The Chair noted that this is not as defined in the CfP, but hopefully is good enough for this selection process.

Pierrick Phillipe, France Telecom, presented a spreadsheet of combined results of the item selection listening test data. The listed all items and presented, for each item, mean scores for three score statistics:

• 
  AMR-WB+ - HE-AAC V2
  
• 
  VC
  
• 
  Max or Min of AMR-WB+ and HE-AAC V2
  

Werner Oomen’s data analysis was extended to cover all cases of mono, stereo and pooling of mono and stereo results and also to consider two pooling of listener data for a given test: within test sites and for all test sites. After additional discussion it was agreed to use the pooling over all test sites.

The Chair slightly re-formated the data analysis to use conditional formatting to highlight the items selected. Using this technique, approximately 6 items per content category were selected, as shown below in the rightmost column. This selection will be discussed with the objective of reducing them to 12 items in total.

Tests grouped over bitrate	M+S					M only					S only						Select
	1	2	3	4		1	2	3	4		1	2	3	4	5	6
es01_s	2	4	5	6		2	4	4	4		0	0	1	2	2	2	x
es02_s	2	2	5	5		1	1	3	3		1	1	2	2	2	2	x
es03_s	1	1	1	2		1	1	1	1		0	0	0	1	2	2
te19	1	2	3	4		1	1	2	2		0	1	1	2	2	2
KoreanM1	1	2	3	3		1	1	2	2		0	1	1	1	2	2
louis_raquin_15	1	3	4	4		1	3	4	4		0	0	0	0	1	2	x
Arirang_speech	1	4	4	6		1	2	2	4		0	2	2	2	2	2	x
Green_speech	2	3	4	4		2	2	2	2		0	1	2	2	2	2
Wedding_speech	3	4	4	6		1	2	2	4		2	2	2	2	2	2	x
te1_mg54_speech	3	4	6	6		2	3	4	4		1	1	2	2	2	2	x
carrot_speech	0	0	0	0		0	0	0	0		0	0	0	0	0	0
noodleking	0	1	2	2		0	0	1	1		0	1	1	1	1	1
te16_fe49	1	1	2	4		1	1	2	3		0	0	0	1	1	1
twinkle_ff51	0	4	4	5		0	2	2	3		0	2	2	2	2	2	x
phi1	0	1	2	4		0	1	2	3		0	0	0	1	2	2
phi6	4	4	4	4		2	2	2	2		2	2	2	2	2	2	x
speechOverMusic_1	0	2	4	4		0	0	2	2		0	2	2	2	2	2	x
speechOverMusic_2	2	2	2	3		1	1	1	1		1	1	1	2	2	2
speechOverMusic_3_s	0	0	1	3		0	0	1	2		0	0	0	1	1	1
speechOverMusic_4_s	0	0	2	2		0	0	0	0		0	0	2	2	2	2	x
speechOverMusic_5_s	1	1	1	1		0	0	0	0		1	1	1	1	1	1
Alice	1	3	3	4		0	2	2	3		1	1	1	1	2	2
dora	1	2	2	3		1	2	2	3		0	0	0	0	0	1
lion	6	6	6	6		4	4	4	4		2	2	2	2	2	2	x
HarryPotter	3	4	6	6		3	3	4	4		0	1	2	2	2	2	x
salvation	2	3	3	3		1	1	1	1		1	2	2	2	2	2	x
trilogy	1	2	3	3		1	2	3	3		0	0	0	0	0	0
brahms	1	1	2	2		1	1	2	2		0	0	0	0	0	0
sc03	0	0	0	0		0	0	0	0		0	0	0	0	0	0
dongwoo	0	0	2	3		0	0	1	2		0	0	1	1	1	1
te09_s	1	3	3	4		1	3	3	4		0	0	0	0	0	0	x
te15_s	1	3	4	5		1	3	3	4		0	0	1	1	1	2	x
phi2_s	0	0	1	3		0	0	1	2		0	0	0	1	2	2
phi3_s	0	2	2	3		0	2	2	3		0	0	0	0	0	0
phi4_s	1	2	3	3		0	1	2	2		1	1	1	1	2	2
phi5_s	0	0	0	1		0	0	0	0		0	0	0	1	1	1
phi7	3	6	6	6		3	4	4	4		0	2	2	2	2	2	x
Music_1_s	3	5	5	5		2	3	3	3		1	2	2	2	2	2	x
Music_2	1	1	1	2		0	0	0	1		1	1	1	1	1	1
Music_3	1	2	2	4		0	0	0	2		1	2	2	2	2	2	x
Music_4_s	0	0	2	3		0	0	2	2		0	0	0	1	2	2
Music_5_s	2	2	3	3		1	1	2	2		1	1	1	1	1	1
Wedding_music	0	1	1	1		0	1	1	1		0	0	0	0	0	0