Video coding standards k. R. Rao, Do Nyeon Kim J. J. Hwang Springer 2014

Post – HEVC activity

Both MPEG and VCEG have established AHGs (ad hoc groups) for exploring next generation video coding.

Grois et al (See item 7 under tutorials) have suggested as follows:

Focusing on perceptual models and perceptual quality, and perceptually optimized video compression provision: http://www.provision-itn.eu

PROVISION is a network of leading academic and industrial organizations in Europe including international researchers working on the problems with regard to the state-of-the-art video coding technologies.

The ultimate goal is to make noteworthy technical advances and further improvements to the existing state-of-the-art techniques of compression video material.

AV1 codec

Alliance for Open Media (AOM) http://tinyurl.com/zgwdo59

Website: http://www.streamingmedia.com/Articles/Editorial/What-Is -.../What-is-AV1-111497.aspx

AOM (charted members Amazon, Cisco, Google, Intel Corporation, Microsoft, Mozilla, and Netflix). In April 2016, ARM, AMD and NVIDIA joined the alliance to help ensure that the codec is hardware friendly and to facilitate and accelerate AV1 hardware support.

AOM has nearly finalized a codec called AV1 based on CISCO’s Thor, Google’s VP10 and Mozilla’s DAALA with following goals:

• 
  Interoperable and open
  
• 
  Optimized for the web [e-mail]
  
• 
  Scalable to any modern device at any bandwidth
  
• 
  Designed with a low computational footprint and optimized for hardware
  
• 
  Capable of consistent, highest-quality, real-time video delivery
  
• 
  Flexible for both commercial and non-commercial content, including user-generated content.
  

AV1 is to be shipped sometime between Dec. 2016 and March 2017 and is positioned to replace Google’s VP9 and to compete with HEVC. It is designed to be royalty free.

In terms of makeup, the Alliance members enjoy leading positions in the following markets:

• 
  Codec development - Cisco (Thor) Google (VPX), Mozilla (Daala)
  
• 
  Desktop and mobile browsers - Google (Chrome), Mozilla (Firefox), Microsoft (Edge)
  
• 
  Content - Amazon (Prime), Google (YouTube), Netflix
  
• 
  Hardware co-processing - AMD (CPUs, graphics), ARM (SoCs, other chips), Intel (CPUs), NVIDIA (SoC, GPUs)
  
• 
  Mobile - Google (Android), Microsoft (Windows Phone)
  
• 
  OTT - Amazon (Amazon Fire TV), Google (Chromecast, Android TV)
  

The Alliance is targeting an improvement of 50 percent over VP9/HEVC with reasonable increases in encoding and playback complexity. One focus is UHD video, including higher bitrate, wider color gamut, and increased frame rates, with the group targeting the ability to play 4K 60fps in a browser on a reasonably fast computer. The base version of the codec will support 10-bit and 12-bit encoding, as well as the BT.2020 color space. Another focus is providing a codec for WebRTC (Real-Time Communications), an initiative supported by Alliance members Google and Mozilla, and similar applications including Microsoft's Skype.

P.S: This material on AV1 codec is collected from Streaming Media Magazine, “What is AV1”, Jan Ozer, Posted on June 3, 2016.

AV1-P1 As AV1 is designed to be a royalty free codec, gather all details about this codec (website, steaming media etc) and implement using standard test sequences. Compare its performance with HEVC using the standard criteria such as PSNR, BD bitrate, BD PSNR, SSIM, implementation complexity etc.

AV1-P2 See AV1-P1 Based on the performance comparison, explore any changes in the functionalities (transform/quantization/ME-MC, entropy coding, intra-inter and other modes) of the codec that can result in further in improving AV1’s compression performance with negligible increase in complexity.

AV1-P3 See AV1-P1 Can AV1 codec be applied to SCC? If so compare its performance with HEVC SCC extension.

AV1-P4 See AV1-P1 Explore how AV1 Codec can be modified for scalable video coding similar to HSVC.

AV1-P5 See AV1-P1 Explore how AV1 Codec can be modified for 3D-Multiview coding similar to HSVC.

AV1-P6 Performance comparison of SDCT with DCT and DDCT is investigated in [E392]. In AV1 INTDCT is used. Replace INTDCT with INTSDCT and implement the performance comparison based on various test sequences including 4K and 8K and different block sizes. See the Tables and Figures in [E392].

AV1-P7 See [E393]. Apply the FCDR to AV1 codec both as a post processing operation and as an in loop (embedded) operation. Implement the projects for AV1 similar to those described in P.5.268 and P.5.269.

Real Media HD (RMHD)

https://www.realnetworks.com/realmediaHD

Real Media HD, successor to its RMVB (Real Media Variable Bitrate) video codec.

Real Networks claims that RMHD codec has achieved a reduction of more than 30% bit rate compared to H.264/AVC, while achieving the same subjective quality.

Performance comparison of RMHD with H.264, H.265 and VP9 is also described.

RMHD-P1 While the RMHD encoder/decoder details are not known, develop comparison data similar to those shown in charts 1-7. Hopefully, RMHD software can be accessed.

SMPTE

SMPTE ST 2042-1:200x 28/09/2009

Page 9 of 131 pages

Foreword

SMPTE (the Society of Motion Picture and Television Engineers) is an internationally recognized standards developing organization. Headquartered and incorporated in the United States of America, SMPTE has members in over 80 countries on six continents. SMPTE’s Engineering Documents, including Standards, Recommended Practices and Engineering Guidelines, are prepared by SMPTE’s Technology Committees. Participation in these Committees is open to all with a bona fide interest in their work. SMPTE cooperates closely with other standards-developing organizations, including ISO, IEC and ITU.

SMPTE Engineering Documents are drafted in accordance with the rules given in Part XIII of its

Administrative Practices.

VC-1

Video coding 1 is based on Windows Media Video (WMV9) developed by Microsoft. See [VC-1,8] cited below for details.

References

VC-1, 1.H. Kalva and J.-B. Lee, “The VC-1 VIDEO CODING STANDARD”, IEEE Multimedia, vol. 14, pp. 88-91, Oct.-Dec. 2007.

Vc-1, 2. J.-B. Lee and H. Kalva, “The VC-1 AND h.264 VIDEO COMPRESSION STANDARDS FOR BROADBAND VIDEO SERVICES”, SPRINGER, 2008.

VC-1, 3 VC-1 Software : http://www.smpte.org/home

VC-1, 4 Microsoft website - VC-1 Technical Overview

http://www.microsoft.com/windows/windowsmedia/howto/articles/vc1techoverview.aspx#VC1ComparedtoOtherCodecs

VC-1, 5 VC-1 Compressed video bitstream format and decoding process (SMPTE 421M-2006), SMPTE Standard, 2006.

VC-1, 6 J.-B. Lee and H. Kalva, "An efficient algorithm for VC-1 to H.264 video transcoding in progressive compression," IEEE International Conference on Multimedia and Expo, pp. 53-56, July 2006.

VC-1, 7 S. Srinivasan and S. L. Regunathan, “An overview of VC-1” Proc. SPIE, vol. 5960, pp. 720–728, 2005.

VC-1, 8 S. Srinivasan et al, “Windows Media Video 9: overview and applications” Signal Processing: Image Communication, Vol. 19, pp. 851-875, Oct. 2004.
WMV-9 (VC-1)

WMV9 of Microsoft
Windows Movie Maker, in Windows Vista or XP can be used, but with limited functionality. It can be used to convert a WMV file to AVI format, for example.
For more precise control of encoding parameters, you can use the Windows Media encoder tool, freely available at http://www.microsoft.com/windows/windowsmedia/forpros/encoder/default.mspx. It is an encoding-only tool.
Source code for such tools is not available.
You can also use commercial video editing tools , such as the ones listed at

http://www.microsoft.com/windows/windowsmedia/forpros/service_provider/software/default.aspx#encoding

Source code for VC-1 (WMV9) http://store.smpte.org/VC-1-Test-Material-p/vc-1.htm

Please note: We must receive your completed VC-1 License Agreement Form before we can ship your VC-1 CD-ROM.  You can download this form from the SMPTE Website at:  http://www.smpte.org/standards/.   After you’ve completed the form please fax back to SMPTE 914-761-3115 or email a scanned copy to [e-mail] Our Price: $450.00

1. 
   The below link had a lot of information on audio and video codecs. 
   http://wiki.multimedia.cx/index.php?title=Main_Page
   
2. 
   The link below has a VC-1 reference decoder. It is not the reference decoder but a free one!
   http://ffmpeg.mplayerhq.hu/download.html
   
3. 
   Also, Google search provided the reference code for VC 1 - version 6 which might be outdated now and may not be legal to use (I am not sure on this).  http://www.richardgoodwin.com/VC1_reference_decoder_release6.zip
   
4. 
   
   

VC-2 STANDARD

The VC-2 standard specifies the compressed stream syntax and reference decoder operations for a video compression system. VC-2 is an intra frame video compression system aimed at professional applications that provides efficient coding at many resolutions including various flavors of CIF, SDTV and HDTV. VC-2 utilizes wavelet transforms that decompose the video signal into frequency bands. The codec is designed to be simple and flexible, yet be able to operate across a wide range of resolutions and application domains.

The system provides the following capabilities:

• 
  Multi-resolution transforms. Data is encoded using the wavelet transform, and packed into the bitstream subband by subband. High compression ratios result in a gradual loss of resolution. Lower resolution output pictures can be obtained by extracting only the lower resolution data.
  
• 
  Frame and field coding. Both frames and fields can be individually coded.
  
• 
  CBR and VBR operation. VC-2 permits both constant bit rate and variable bit rate operations. For low delay pictures, the bit rate will be constant for each area (VC-2 slice) in a picture to ensure constant latency.
  
• 
  Variable bit depths. 8, 10, 12 and 16 bit formats and beyond are supported.
  
• 
  Multiple color difference sampling formats. 444, 422 and 420 video are all supported.
  
• 
  Lossless and RGB coding. A common toolset is used for both lossy and lossless coding. RGB coding is supported either via the YCoCg integer color transform for maximum compression efficiency, or by directly compressing RGB signals.
  
• 
  Wavelet filters. A range of wavelet filters can be used to trade off performance against complexity. The Daubechies (9,7) filter is supported for compatibility with JPEG2000. A fidelity filter is provided for improved resolution scalability.
  
• 
  Simple stream navigation. The encoded stream forms a doubly-linked list with each picture header indicating an offset to the previous and next picture, to support field-accurate high-speed navigation with no parsing or decoding required.
  
• 
  Multiple Profiles. VC-2 provides multiple profiles to address the specific requirements of particular applications. Different profiles include or omit particular coding tools in order to best match the requirements of their intended applications. The Main profile provides maximum compression efficiency, variable bit rate coding and lossless coding using the core syntax. The Simple profile provides a less complex codec, but with lower compression efficiency, by using simple variable length codes for entropy coding rather than the arithmetic coding used by the Main profile. The Low Delay profile uses a modified syntax for applications requiring very low, fixed, latency. This can be as low as a few lines of input or output video. The Low Delay profile is suitable for light compression for the re-use of low bandwidth infrastructure, for example carrying HDTV over SD-SDI links. The High Quality profile similarly provides light compression with low latency and also supports variable bit rate and lossless coding.
  

Scope

This standard defines the VC-2 video compression system through the stream syntax, entropy coding, coefficient unpacking process and picture decoding process. The decoder operations are defined by means of a mixture of pseudo-code and mathematical operations.

VC2-P1 Compare the performance of VC2 lossless coding with other lossless coding techniques described in JPEG/JPEG-LS, JPEG-XR, JPEG-XT, JPEG2000, HEVC, H.264, MPEG-2 and DIRAC-PRO based on standard comparison metrics.

VC2-P2 Performance comparison of SDCT with DCT and DDCT is investigated in [E392]. In VC2 replace wavelets with INTSDCT and implement the performance comparison based on various test sequences including 4K and 8K and different block sizes. See the Tables and Figures in [E392].

VC1-P3 See [E393]. Apply the FCDR to VC1 codec both as a post processing operation and as an in loop (embedded) operation. Implement the projects for VC1 similar to those described in P.5.268 and P.5.269.

Legacy Codec

LC1. J. Chen et al, “ Efficient Video Coding Using Legacy Algorithmic Approaches,” IEEE Trans. on Multimedia, vol. 14, pp. 111-120, Feb. 2012.

LC-P1. See [LC1] The authors have developed a video codec based on the traditional transform predictive coding aided by motion compensation followed by the QM-coder. While its performance is similar to that of H.264/AVC, the main advantage is its royalty few feature. Implement this legacy codec using various test sequences including ultra HD TV resolution.

PSNRAVG

PSNRAVG is a weighted average of luminance (PSNRY) and chrominance (PSNRU and PSNRV) PSNR components. All involved test sequences are in 4:2:0 color format, for which PSNRAVG is computed as (de facto standard)

PSNRAVG = (6 × PSNRY + PSNRU + PSNRV)/ 8.

Since PSNRAVG also takes the impact of the chrominance components into account, it is supposed to provide more reliable results than the conventional PSNRY metric in the cases when the luminance and chrominance components have dissimilar RD behaviors. See reference below:

B. Li, G. J. Sullivan, and J. Xu, RDO with Weighted Distortion in HEVC, document JCTVC-G401, ITU-T/ISO/IEC Joint Collaborative Team on Video Coding (JCT-VC), Geneva, Switzerland, Nov. 2011.

ACKNOWLEDGEMENTS

The graduate students in Multimedia Processing Lab., University of Texas at Arlington, Arlington, Texas have been extremely helpful in all aspects of updating/revising this chapter including adding additional projects and additional references. Special thanks go to M. Budagavi, Samsung Research Lab, T. Richter, University of Stuttgart, Stuttgart, Germany, W. Gao, Peking University, Peking, D. Mukherjee, Google Inc., D. Grois, HHI –Fraunhofer Heinrich Hertz Institute, G.J. Sullivan, Microsoft Inc., T. Borer, BBC, UK, Z. Wang, Univ. of Waterloo, Waterloo, Canada, Enrico Magli, Dept. of Electronics and T1elecommunications, Politecnico di Torino, Italy and P. Topiwala, FastVDO for providing various resources in this regard.