International organisation for standardisation organisation internationale de normalisation

Exploration: Audio Synchronization

Takehiro Sugimoto, NHK, presented

m32189

Cross-check report of audio synchronization

Takehiro Sugimoto, Masayoshi Ohnishi

NHK experts visited the Sony laboratory for this investigation for the purpose of characterising the performance of the Sony technology.

The synchronization technology worked up to approximately -20 dB SNR at the second screen microphone.

When synchronization was possible, the synchronization was always within 32 ms.

Masayuki Nishiguchi, Sony, presented

m32236

Cross Check Report of Audio Synchronization Experiment

Shusuke Takahashi, Akira Inoue, Masayuki Nishiguchi, Toru Chinen

The contribution reviewed the workplan’s experimental setup, in which the “second screen” presentation is synchronized with a “first screen” presentation:

• 
  A fingerprint is calculated from the first screen content
  
• 
  The fingerprint is transmitted with second screen content. No changes is required in the first screen content or transmission
  
• 
  Using the transmitted fingerprint, the second screen can align its presentation to that of the first screen
  

The synchronization technology worked up to approximately -10 dB SNR at the second screen microphone for all first screen content signals and all interfering signals.

The proposal in the contribution is slightly different from the proposal from the previous meeting, in that it is proposed to have an Audio Elementary Stream that carries just the first screen content fingerprint and an associated time stamp.

Encoder:

Decoder:

The presenter gave a demonstration of the synchronization technology.

Discussion

The Chair noted that in e.g. live production, the second screen content would have to “read” the master clock associated with the first screen. Christof Fersch, Dolby, noted that there might be a need to also synchronize video media, and that there already might be good ways to do this within MPEG-2 Systems.

Sony requests that the following be standardized

• 
  MPEG-4 Audio Elementary Stream audio object type
  
• 
  Syntax of binary representation of fingerprint
  
• 
  Fingerprint extraction algorithm
  
• 
  Normative output format to express time difference (time delay to be applied to second screen content)
  

The Chair noted that this activity can be kept as an exploration, however he could schedule a joint meeting with Systems to get additional information from Systems experts.

4.