Real-time Control of MPEG-4 Coders

Local supervisor:

Associate professor Dr. Ing. José Ignacio Ronda Prieto

Supervisor at TU-HH:

Professor Dr.-Ing. Otto Lange

Abstract

The increasing demand on high quality digital video broadcasting systems has driven the development of several coding standards, MPEG-4 being the most recent. As these standards only define the bit-stream syntax, such essential features of the coder as controlling the output bit-rate have to be implemented individually.

An important problem arising here is the optimization of the bit-rate-distortion trade-off, since in general the resulting bit-stream has to meet the requirements of a communication channel with limited bandwidth.

The new video compression standard for multimedia applications MPEG-4 (currently being developed by the ISO) poses new problems regarding the control of the encoding process, due to its object-oriented nature. In this paradigm the scene is encoded in terms of multiplexed streams, each of which conveys information of an object, corresponding to a visual entity considered as a semantic unit. Important facts about those objects are that they have a time-varying arbitrary shape, and the frame rate can vary between them.

Here is given an excerpt of the whole project papers to give an idea about theirs contents.

• Introduction
• Objectives
• Outline of Report

Introduction

For some years a demand exists to realise multimedia communication. This demand led to institute committees and groups for developing needed coding standards. One of these is the MPEG with its successful development of the multi-industry MPEG-1 and MPEG-2 standards.

MPEG-4, the most recent standardisation project of MPEG (Motion Pictures Expert Group), combines some of the typical features of other MPEG standards with new ones, to meet the still increasing demand on high quality digital video broadcasting systems. Its main aim is to provide a coding scheme for small bandwidth channels, as used for video conferencing, picture-phone, and HDTV. Traditional coding technics are not able to target such low bit-rates, provided that a minimum quality criterion is pinpointed. The necessary improvements are only possible if the coding process takes into account the typical characteristics of natural picture sequences: Several objects exist inside the picture scene, which contain a shape and which can move independent from each other.

The first step of an object-oriented video coding is the segmentation of the picture scene in, the content of the scene conformed, sensible areas. The logical components of the pictures then are not any longer pixels or rectangular blocks, but arbitrarily shaped picture-segments.

Objectives

All MPEG standards only define the syntax of the encoded bit-stream to guarantee a univocal decoding. Therefore, essential features of the coder, such as controlling the output bit-rate, have to be implemented individually, and in a way that the subsequent decoding process is not disturbed.

In Figure the bit-rates of two objects of the weather sequence are shown. The plot on the left shows the objects which represent the background, a weather forecast map. The plot on the right shows the object which represents the forecaster. Well seen is the moment in this scene, beginning with frame 200, when the forecaster steps aside which leads to an increasing bit-rate for both VOPs.

Bit-rate of first object of the weather sequence, in each frame quantized with best quality.

Caused by the in-depended movement of the objects and their different complexities, each ones bit-rate has its own characteristic. Nevertheless, in case of overlapping objects an interrelation takes place. Therefore, a kind of regulation for the bit-rate of each object, as well as for the whole bit-rate is advisable.

Within this master-thesis work, new algorithms have been designed and tested for the real-time regulation of the MPEG-4 video coding process. Whereas the so far developed regulation systems are either block based or related to the whole bit-stream, these algorithms are giving wide coverage to the object concept and its implications.

The algorithms were integrated and proved in the MPEG-4 VM (Verification Model) version 3.1 (actual in November 1996), which is part of the MoMuSys (MObile MUltimedia SYStems) project. The activities of MoMuSys are closely linked with the development of ISO (International Standardisation Organization) MPEG-4. Thus MoMuSys can be seen as a European initiative to push the development of standardisation, mainly in MPEG-4, but also in the field of mobile multimedia communications.

Outline of Report

This section will give a synopsis about the structure of this report and the contents of the several chapters and sections.

2. summarizes the MPEG standards and explains the concept of the MPEG-4 encoder.
3. gives a general introduction to object-based rate-distribution systems. By picking up the parameters bit-rate and distortion, several algorithms for such a system will be developed, each one regulating the bit-stream with other objectives.
4. introduces the traditional regulative system for the coder bit-rate: The controlled buffer or rate-control system. It is shown, how it can be connected to the rate-distribution system, and that this constitutes in a better quality.
5. presents three stochastic models, for predicting the parameters bit-rate and distortion, which depend on the used quantization parameters in the coding process.
6. finally, draws conclusions about the previous three chapters, giving suggestions when and how the items for the overall real-time control system should be used. Moreover, some future directions for a further development are collected.

1. offers numerical examples for some of the rate-distribution algorithms, explained in Chapter.
2. holds out pictorial results of all the rate-distribution algorithms.
3. introduces the three video source sequences that have been used to test the developed algorithms and rate-control systems.

Finally, the report is ended by an extensive glossary, a bibliography, and an index.

Last but not least, this project was used and is mentioned in the bibliography part in a research and conference by the Siemens company.