Video coding standards k. R. Rao, Do Nyeon Kim J. J. Hwang Springer 2014

JPEG XT Projects:

The JPEG XT suite of standards described in the paper [JXT1] can lead to several projects. Review this paper in detail and implement these standards. Please access the reference software.

JXT-P1. JPEG Privacy & Security: With social media having a huge impact on every individual, it disrupts their privacy. Hence, protecting the privacy and security is becoming very important, not only because of the social media, but also because images/ meta-data use cloud for storage purposes over private repositories. Encrypting the images and providing access to only the authorized person can help in maintaining the privacy of the image/ meta-data.
JXT-P2. Recoding Image Editing Operations: While editing any image or text, only the original data and the final output after editing is usually saved. But instead, if every intermediate step is also recorded, it gives a possibility to revert the data to the original image or any of the intermediate steps. Saving every step while developing a special effect (sharpen, emboss, smooth) can help to develop new, faster and easier effects.
JXT-P3. In the conclusion section the authors [JXT6] state

“The encoding or decoding hardware can be in fact designed based on a pair of existing JPEG coding chips, as shown in Figure 2, resulting in a minimal hardware change in the existing hardware infrastructure without influencing its real-time performances.” Implement the encoding and decoding hardware.

JXT-P4. See [JXT7] As part of JPEG XT future extensions, JPEG privacy and security, Shah has developed an algorithm to encrypt and decrypt the images using C++ and OpenCV libraries. This algorithm performs well for protecting an individual’s identity by blurring the image. By resolving the compatibility issues between the OpenCV libraries and JPEG XT code this algorithm can be incorporated in the JPEG XT code tom provide the security feature automatically. Explore this.
JXT-P5 See JXT-P4. Shah suggests that the amount of blurring can be changed according to the user preference by changing the kernel size. This gives the user full control over how the image should be seen by others. Explore this in detail and draw some conclusions.
JXT-P6 See JXT-P4. Several other tasks also are proposed as future work in Chapter of this thesis.

Investigate and implement these tasks.

JXT-P7 See [JXT10], [JXT11] and [JXT12]. Richter has proposed some optimization techniques to improve the legacy JPEG in terms of rate distortion performance close to JPEG-XR (See Fig. 5 in JXT10). Using the software described in [JXT 11] and [JXT12] confirm the enhanced JPEG rate distortion performance shown in [JXT10].
JXT-P8 JPEG-XT standard defines a normative decoding procedure to reconstruct an HDR image from two JPEG regular code streams named the base layer (visible to legacy decoders) and an extension layer and how to merge them together to form one single image. See also JXT1.The standard does not however, define the encoding procedure and leaves large freedoms to the encoder for defining the necessary decoder configuration. Mantiuk, Richter and Artusi [JXT8], explored the whole space of possible configurations to achieve the best possible R-D performance. Review this paper in detail and verify the performance results described in the figures [JXT8].

JPEG PLENO

JP1. T. Ebrahimi, “Towards a new standard for plenoptic image compression”, IEEE DCC, 29 March- 1 April 2016. Abstract of this valuable paper is reproduced below:

Abstract:
JPEG format is today a synonymous of modern digital imaging, and one of the most popular and widely used standards in recent history. Images created in JPEG format now exceeds one billion per day in their number, and most of us can count a couple, if not more JPEG codecs in devices we regularly use in our daily lives; in our mobile phones, in our computers, in our tablets, and of course in our cameras. JPEG ecosystem is strong and continues an exponential growth for the foreseeable future. A significant number of small and large successful companies created in the last two decades have been relying on JPEG format, and this trend will likely continue.

A question to ask ourselves is: will we continue to have the same relationship to flat snapshots in time (the so-called Kodak moments) we call pictures, or could there be a different and enhanced experience created when capturing and using images and video, that could go beyond the experience images have been providing us for the last 120 years? Several researchers, artists, professionals, and entrepreneurs have been asking this same question and attempting to find answers, with more or less success. Stereoscopic and multi-view photography, panoramic and 360-degree imaging, image fusion, point cloud, high dynamic range imaging, integral imaging, light field imaging, and holographic imaging are among examples of solutions that have been proposed as future of imaging.

Recent progress in advanced visual sensing has made it feasible to capture visual content in richer modalities when compared to conventional image and video. Examples include Kinect by Microsoft, mobile sensors in Project Tango by Google and Intel, light-field image capture by Lytro, light-field video by Raytrix, and point cloud acquisition by LIDAR (Light Detection And Ranging). Likewise, image and video rendering solutions are increasingly relying on richer modalities offered by such new sensors. Examples include Head Mounted Displays by Oculus and Sony, 3D projector by Ostendo and 3D light field display solutions by Holografika. This promises a major change in the way visual information is captured, processed, stored, delivered and displayed.
JPEG PLENO evolves around an approach called plenoptic representation, relying on a solid mathematical concept known as plenoptic function. This promises radically new ways of representing visual information when compared to traditional image and video, offering richer and more holistic information. The plenoptic function describes the structure of the light information impinging on observers’ eyes, directly measuring various underlying visual properties like light ray direction, multi-channel colors, etc.
The road-map for JPEG PLENO follows a path that started in 2015 and will continue beyond 2020, with the objective of making the same type of impact that the original JPEG format has had on today's digital imaging starting from 20 years ago. Several milestones are in work to approach the ultimate image representation in well-thought, precise, and useful steps. Each step could potentially offer an enhanced experience when compared to the previous, immediately ready to be used in applications, with potentially backward compatibility. Backward compatibility could be either at the coding or at the file format level, allowing an old JPEG decoder of 20 years ago to still be able to decode an image, even if that image won’t take full advantage of the intended experience, which will be only offered with a JPEG PLENO decoder.
This talk starts by providing various illustrations the example applications that can be enabled when extending conventional image and video models toward plenoptic representation. Doing so, we will discuss use cases and application requirements, as well as example of potential solutions that are or could be considered to fulfill them. We will then discuss the current status of development of JPEG PLENO standard and discuss various milestones ahead. The talk will conclude with a list of technical challenges and other considerations that need to be overcome for a successful completion of JPEG PLENO.

JP2. M.D. Angelo et al, “Correlation plenoptic imaging”, Physical Review Letters, vol. 116, no.22, pp. 223602, Jun. 2016.

Abstract is reproduced below.

Plenoptic imaging is a promising optical modality that simultaneously captures the location and the propagation direction of light in order to enable tridimensional imaging in a single shot. However, in classical imaging systems, the maximum spatial and angular resolutions are fundamentally linked; thereby, the maximum achievable depth of field is inversely proportional to the spatial resolution. We propose to take advantage of the second-order correlation properties of light to overcome this fundamental limitation. In this paper, we demonstrate that the momentum/position correlation of chaotic light leads to the enhanced refocusing power of correlation plenoptic imaging with respect to standard plenoptic imaging.

JP-P1 Please review the abstract in [JP1]. Focus on the research towards developing the JPEG PLENO standard with backward compatibility with legacy JPEG. Review the technical challenges and consider the application requirements. This is a challenging research.

JP-P2 Please review [JP2] in detail. In the conclusions and outlook, the authors state that the merging of plenoptic imaging and correlation quantum imaging has thus the potential to open a totally new line of research. Explore this.

LAR-LLC

LL1. Y. Liu, O. Deforges and K. Samrouth, “LAR-LLC: A low complexity multiresolution lossless image codec,” IEEE Trans. CSVT, vol. 26, pp. 1490-1501, Aug 2016.

Abstract:

It compares the performance of LAR-LLC lossless codec with JPEGXR, JPEG2000, JPEGLS and LJPEG (lossless JPEG) in terms of compression ratio, encoding and decoding speeds (see Tables I-IV). LAR-LLC has much less encoding and decoding speeds (less complexity) compared to the JPEG series. It also supports spatial scalability similar to JPEG2000.

LL-P1. See [LL1]. In Table II the bitrates (bpp) for lossless color image (24 bpp) codecs are shown, for various test images. Implement the JPEG lossless codecs and compare with LAR-LLC.

LL-P2. HEVC has lossless coding in intra (AI) profile. Implement this and compare with Table II.

LL-P3. Develop Tables similar to Tables V and VI for HEVC AI profile (lossless coding).

LL-P4. In the paragraph before VII conclusion, the authors state that the LAR-LLC currently runs in a mono thread configuration, but has potential parallel processing ability in multithreading and suggest how this can be implemented. Explore this and evaluate how the multithreading configuration can further reduce the LAR-LLC encoding/decoding speeds for lossless coding.

LL-P5. Extend the lossless image coding scheme to VP10. See Tables I to IV in [LL1].

LL-P6. See LL-P4. Extend the lossless image coding scheme to AVS2 – intra coding only.