International organisation for standardisation organisation internationale de normalisation

8Task group activities

8.1Joint meetings

8.1.1MXM and Audio on MPEG eXtensible Middleware (MXM) and Audio reference software

Information on the MXM project can be found at mxm.wg11.sc29.org.

Phillippo Chiariglione, SmartRM, presented an overview of the MXM APIs. The categories of APIs are as follows:

• 
  Creation
  
• 
  Decoding
  
• 
  Rendering
  
• 
  Editing
  

MXM provides an API the permits “black box” modules that play audio-visual encoders and decoders. The APIs are defined under the “C++ APIs” on the MXM website. Whenever possible, MXM provides a standardized wrapper for existing multimedia APIs. Some of these may be to media-centric (i.e. decode an AccessUnit) or platform-centric (i.e. change the audio volume). One could then construct interface code below the MXM API to provide access to e.g. the ffmpeg media coding tools and also the unique media coding tools found in the MPEG Audio Reference Code repository (e.g. SAOC or USAC).

This was elaborated in an email circulated to interested experts, which is quoted here:

The APIs to access media-related technologies are part of the MediaFramework MXM Engine. The APIs are divided into the following categories:

• 
  Creation: APIs to create an object (e.g. create a compressed audio stream from an uncompressed one, create a LASeR object having certain characteristics, etc.)
  
• 
  Access: APIs to access an object (e.g. decode a compressed video stream, etc.)
  
• 
  Rendering: APIs to render an object (e.g. render a 2D or 3D graphics object, an audio stream, etc.)
  
• 
  Editing: APIs to edit an object (e.g. add an elementary stream to an existing multiplexed content, transcode an elementary stream from one format to another, etc.)
  

The possible types of "Objects" on which the APIs shall allow operating are the following:

• 
  Elementary Streams:
  
  • 
    Image
    
  • 
    Audio
    
  • 
    Video
    
• 
  Muxed Content
  
• 
  Graphics-related
  
  • 
    Scene
    
    • 
      2D/layout
      
    • 
      3D/scene graph
      

• 
  Graphics primitives
  
  • 
    2D graphics
    
  • 
    3D graphics
    

The type of input that MXM would like to receive from the various Audio/Video/System/Graphics experts in MPEG is of the following type

• 
  feedback on the classification of APIs and "Objects"
  
• 
  feedback on the Creation (http://tinyurl.com/kody4o) and Access (http://tinyurl.com/km55s6) APIs already specified in MXM
  

• 
  input for defining the remaining APIs currently not covered yet
  

The Audio Chair suggested that audio experts could propose a revised MXM audio API that will cover, at a minimum, the following:

• 
  instantiate a new audio encoder or decoder
  
• 
  encode an audio segment to an AccessUnit or bitstream segment
  
• 
  decode an AccessUnit or bitstream segment to get an audio segment
  
• 
  free resources associated with the audio encoder or decoder
  

The Audio Chair will take the lead in drafting a contribution to the next MPEG meeting on a proposed MXM audio API.

There was discussion on meda-data, specifically on “signal-derived” meta-data. Examples would be audio program level and meta-data required to implement dynamic range control. The Audio Chair volunteered to be a contact point for further discussions on this topic.