Video Codec Design for Real Time Applications

Abstract:

Multimedia industries are witnessing a rapid evolution toward integrating

complete systems on a single chip. The most important characteristic of multimedia
applications is the rapid growth of the complexity of the design. As an example, the new
videophone and teleconferencing standard (H.263+) is much more complex than the
previous H.263 and H.261 standards, and the continued evolution of video coding
standards from MPEG-1 and MPEG-2 to MPEG-4 [3] is accompanied by a significant
complexity increase with user flexibility. The conventional approach for system-level
design and integration of video processing technology, combined with the level
complexity and extremely short product development cycles, cannot found robust, cost-
effective solutions simply by implementing large parts of the system functionality in
software running on application-specific instruction set processor (ASIP) cores.

The project focuses on the system-level modeling of image and video processing
systems. Based on the models given, image and video compression and decompression
standards (e.g. H.263 and MPEG-4) will be implemented in a formal, consistent and
extensible framework with well-defined and optimized processing primitives. The models
of computation and an implementation of image and video processing algorithm will also
help automate partitioning of the design into hardware and software.

Video Codec Structure

The Video Codec is split into two programs Encoder and Decoder. Both of them
are organized into three layers according to the functionality, interfaces, and the coding
strategy followed.

Block diagram of Encoder:

Input

Top Layer

Raw frame Recon. frame

Intermediate Layer ME, MC

Blocks
Residue frame Recon. Residue frame
Encoder
Bottom Layer
Blocks

Bit stream

This diagram explains the flow of data and control in the Encoder. The decoder has a
similar structure with the data flowing the reverse direction.
 Top Layer: This layer serves as interface between user and the actual encoder. It
loads the input into frames and feeds them to the lower layer sequentially. This layer also
performs the necessary initializations (such as Loading Huffman tables). The input
sequence is read and passed as frames to the lower layer. The very first frame is forced to
be INTRA. The subsequent frames are by default directed to be coded as P frames. The
lower layer may dynamically decide to code such a frame as an I-frame according to
various parameters (scene change / INTRA refresh).
Intermediate Layer: This Layer performs the frame level manipulations and also
takes care of the frame-level and macro block-level decision making in the encoder. This
layer performs the Motion Estimation and Compensation or removes 128 from the frame
according to whether the frame is to be coded as an I-frame or a P-frame. The residue
frame is input to the frame-encoder layer. The reconstructed residue frame received from
the encoder layer is used to reconstruct the current frame for future prediction.
Bottom Layer: This layer performs the actual coding. The frame is coded
according to the set of parameters which are passed by the higher layer. These parameters
are set by the higher layer according to the coding strategy formed. The bits generated are
put in the bit stream. One way to modify the encoder to perform better is to keep track of
the bits generated per macro block and adaptively change the quantization step size in
order to keep the bit rate to a desired value. In this case it is necessary to include the
Quantizer information in the macro block header. This is a possible extension to the
existing implementation.