Essentials of Error-Control Coding Techniques

Essentials of Error-Control Coding Techniques

Hideki Imai

Faculty of Engineering, Yokohama National University, Kanagawa, Japan

Table of Contents

Cover image

Title page

Copyright

Contributors

Preface

Chapter 1: Introduction

Publisher Summary

1.1 Digital Techniques and Error-Control Coding

1.2 Error-Control Coding in Communications

1.3 Error-Control Coding in Computers

1.4 Error-Control Coding in Audio-Video Systems

1.5 Future Prospects of Error-Control Coding

Part I: Coding Theory

Chapter 2: Principles of Error Detection and Correction

Publisher Summary

2.1 Basic Concepts

2.2 Principles of Error Detection and Correction

2.3 Finite Fields

2.4 Linear Codes

Chapter 3: Block Codes

Publisher Summary

3.1 Cyclic Codes

3.2 BCH Codes

3.3 Majority Logic Decodable Codes

3.4 Burst-Correcting Codes

3.5 Methods for Modifying and Combining Codes

Chapter 4: Convolutional Codes

Publisher Summary

4.1 Introduction

4.2 Threshold Decoding

4.3 Maximum-Likelihood Decoding

Part II: Applications

Chapter 5: Design Technique for an Error-Control Scheme

Publisher Summary

5.1 Code Performance

5.2 Decoder Design Technique

Chapter 6: Applications to Communication Systems

Publisher Summary

6.1 Introduction

6.2 Applications to Satellite Communications

6.3 Applications to Broadcasting

6.4 Applications to Other Communication Systems

Chapter 7: Applications to Computer Systems

Publisher Summary

7.1 Introduction

7.2 Applications to Logic Circuits

7.3 Applications to Semiconductor Memories

7.4 Applications to Magnetic Storage Devices

7.5 Application to Optical Disk Systems

Chapter 8: Applications to Audio-Video Systems

Publisher Summary

8.1 Introduction

8.2 Applications to the Audio Recorder

8.3 Applications to Digital Video Systems

8.4 Conclusion

Index

Copyright

Copyright © 1990 by Academic Press, Inc.

All Rights Reserved.

No part of this publication may be reproduced or transmitted in any form or by any means, electronic or mechanical, including photocopy, recording, or any information storage and retrieval system, without permission in writing from the publisher.

Academic Press, Inc.

San Diego, California 92101

United Kingdom Edition published by

Academic Press Limited

24–28 Oval Road, London NW1 7DX

Library of Congress Cataloging-in-Publication Data

Essentials of error-control coding techniques / edited by Hideki Imai.

p. cm.

ISBN 0-12-370720-X (alk. paper)

1. Error-correcting codes (Information theory) 2. Computer arithmetic and logic units. I. Imai, Hideki, date.

QA268.E87 1990

005.7′2–dc20

89-17513

CIP

Printed in the United States of America

90 91 92 93 9 8 7 6 5 4 3 2 1

Contributors

Numbers in parentheses indicate the pages on which the authors’ contributions begin.

Eiji Fujiwara(171),     Department of Computer Science, Tokyo Institute of Technology, 2-12-1 Ohokayama, Meguro-ku, Tokyo 152, Japan

Tsuneo Furuya(171),     CS Group, Sony Corporation, Osaki Branch, 2-10-14 Osaki, Shinagawa-ku, Tokyo 141, Japan

Yasuo Hirata(103),     Research and Development Laboratories, KDD, 2-1-23 Naka-Meguro, Meguro-ku, Tokyo 153, Japan

Hideki Imai(1),     Faculty of Engineering, Yokohama National University, 156 Tokiwa-dai, Hodogaya-ku, Yokohama 240, Japan

Tohru Inoue(75),     Information Systems and Electronics Development Laboratory, Mitsubishi Electric Corporation, 5-1-1 Ohfuna, Kamakura 247, Japan

Kunimaro Tanaka(269),     Mitsubishi Electric Corporation, Industrial Electronics and Systems Development Laboratory, 1-1 Tsukaguchi-Honmachi 8-Chome, Amagasaki, Hyogo 661, Japan

Osamu Yamada(103),     The Institute of Space and Astronautical Science, Sagamihara-City, Kanagawa 229, Japan

Takahiro Yamada(11, 39, 61),     Jet Propulsion Laboratory, MS 301-280, California Institute of Technology, 4800 Oak Grove Drive, Pasadena, California 91109

Mitsuhiko Yashiro(171),     System Design Section, Magnetic System Design Department, Fujitsu Ltd., 1015 Kami-Odanaka, Nakahara-ku, Kawasaki 211, Japan

Preface

In recent years, error-control coding has been used extensively in digital communication systems and computer systems because of its cost-effectiveness in achieving efficient, reliable digital transmission and storage. The use of error-control coding techniques has in fact become integral to the designing of modern communication and computer systems, especially new media systems. Some systems could never have been realized without the application of error-control coding; the compact disk player is one example. A working knowledge of error-control coding techniques is now a necessity for anyone involved in the design of digital communications or digital storage systems.

This book presents error-control coding techniques with an emphasis on the most recent applications. It is written for engineers who use or build error-control coding equipment. Many examples of practical applications are provided, enabling the reader to obtain valuable expertise for the development of a wide range of error-control coding systems. Necessary background knowledge of coding theory (the theory of error-correcting codes) is also included so that the reader is able to assimilate the concepts and the techniques.

The authors of this volume are leading researchers in the field of error-control coding and engineers experienced in designing error-control coding equipment. One of the authors is the designer of the error-control scheme for the compact disk player and another designed the error-control coding used in the teletext systems of NHK (Japan Broadcasting Corporation). Due to this superior combination of active research and experienced engineering, this book provides the most useful approach to the application of error-control coding to actual systems.

This book is divided into two parts. Chapters 2–4 Chapter 3 Chapter 4 provide the reader with the fundamental knowledge of the coding theory that is necessary to understand the material in the latter part. These chapters are basically descriptive, and every attempt has been made to present the reader a fundamental understanding of the subject.

Chapters 5–8 Chapter 6 Chapter 7 Chapter 8 are devoted to the practical applications of error-control coding in various fields and explain how to design cost-effective error-control coding systems. Many examples of actual error-control coding systems are described and evaluated.

The reader will obtain not only a comprehensive view of the practical applications of error-control coding techniques but also the knowledge essential for constructing efficient error-control coding systems, as well as an understanding of the fundamental concepts of the coding theory. The book is particularly suited for the engineer striving to master the practical applications of error-control coding. It is also suitable for use as a graduate text for an advanced course in coding theory.

1

Introduction

Hideki Imai

Publisher Summary

This chapter focuses on error-control coding. Errors are classified as one of three types, namely, random errors, burst errors, and byte errors. Random errors occur independently on each symbol. Burst errors occur intensively in a period of data. Byte errors occur in a small block of data. In some cases, mixtures of two of the three types can occur. For each type of error, there exist codes that can effectively detect and/or correct errors of that type. However, a code designed to detect or correct one type of error cannot necessarily be effective for other types. Error-control coding has become an essential part in most of the digital communication and recording systems. As the amount of digital data that must be transmitted or stored reliably increases, error-control coding is becoming more and more important. Because of advances in solid-state electronics technology, codes with large error-correction capability and high efficiency are widely used. In particular, coding systems that employ two codes combined together to obtain better performance, for example, concatenated codes, are often employed.

1.1. Digital Techniques and Error-Control Coding

1.2. Error-Control Coding in Communications

1.2.1. Applications to Satellite Communications

1.2.2. Applications to Broadcasting

1.2.3. Applications to Other Communication Systems

1.3. Error-Control Coding in Computers

1.3.1. Applications to Logic Circuits

1.3.2. Applications to Semiconductor Memory Systems

1.3.3. Applications to Magnetic Recording Systems

1.3.4. Applications to Optical Disk Systems

1.4. Error-Control Coding in Audio-Video Systems

1.4.1. Applications to Audio Systems

1.4.2. Applications to Video Systems

1.5. Future Prospects of Error-Control Coding

Bibliography

1.1 Digital Techniques and Error-Control Coding

As of 1989, the number of communication or audio-video systems that treat voice or image signals as digital data is rapidly increasing. The material that computers process or store is digital data as well. One of the important characteristics of digital signals is that they are more reliable in a noisy environment than analog signals. Since the detector for digital data may only decide whether each symbol is a 0 or a 1, digital symbols can often be detected perfectly, provided the noise is weak.

However, when the noise is not weak, the detector may make an erroneous decision, that is, it may decide that a symbol is a 1 although it was originally a 0. But if the data are coded, that is, some appropriate check (redundant) symbols are annexed to the data symbols, the decoder can correct or detect certain errors. Thus, when a signal is represented as digital data, we can make the signal detection more reliable by adding check symbols to the data symbols. This technique is called error-control coding.

A vast amount of research has been done so far on the theory of error-control coding. But it is not until recently that this technique has been applied widely to digital communication and storage systems. Today, error-control coding is used in many digital systems, and its role is becoming more and more important. This is because error-control codes have become easy to implement owing to advances in solid-state electronic technology and because the amount of digital data that must be transmitted or stored reliably has greatly increased. In some areas, error-control coding is essential to practical system design.

When choosing an appropriate error-control code for a digital communication or recording system, a number of factors must be considered. The first of these is the type of errors that occur in the system. Errors are classified as one of three types, namely, random errors, burst errors, and byte errors. Random errors occur independently on each symbol. Burst errors occur intensively in a period of data. Byte errors occur in a small block of data. In some cases, mixtures of two of the three types may occur. For each type of error, there exist codes that can effectively detect and/or correct errors of that type. However, a code designed to detect or correct one type of error will not necessarily be effective for other types.

The next things to be considered are the required error-correction or -detection capability, the allowable number of redundant symbols, and the size, complexity, and speed of the decoder. Naturally, a trade-off among the preceding must be made. This is shown conceptually in Fig. 1.1. The vertical axis represents the error-correction (or -detection) capability, while the transverse axis represents the code rate, which is related inversely to the amount of redundancy required. The closer to the origin the point in Fig. 1.1 is, the better the code is.

Fig. 1.1 General characteristics of codes.

Generally, a code with large error-correction capability has low code rate. This relationship is indicated by the two lines in Fig. 1.1 representing two classes of codes, namely code classes A and B. Since code class B has a code with higher code rate than code class A for a given error-correction capability, code class B is more efficient in that it is less redundant and thus has less waste than code class A. However, efficient codes usually need a complicated decoder which operates at a relatively low speed. Thus, if a simple or fast decoder is required, efficiency must be sacrificed to a certain extent. Figure 1.1 also shows the limit of theoretically realizable codes. We cannot have a code beyond this limit no matter how large a decoder we install. Therefore, the problem of selecting a code boils down to finding a point in Fig. 1.1 that fulfills the requirements for error-correction capability, code rate, size, and speed of the decoder. Table 1.1 lists typical error-control codes.

Table 1.1

Typical Error-Control Codes

In addition to the selection of the code, we must choose among three error-control schemes: (1) correct any errors at the decoder according to a given rule (FEC: forward error correction), (2) request a retransmission of erroneous data block (ARQ: automatic repeat request), (3) estimate the true value of erroneous data from correct data by utilizing the statistical characteristics of the data. This is determined by the required reliability of the decoded data, the allowable delay time for reception of the decoded data, the amount of hardware that can be put at the sending and receiving ends, and so on.

In the following sections, we derive the requirements for error-control coding and show examples of codes actually used in the areas of communications, computers, and audio-video systems.

1.2 Error-Control Coding in Communications

At first, error-control coding was studied as a means to improve the bit error performance of digital communication systems, but not until quite recently has it become an important part of the digital communication systems.

1.2.1 Applications to Satellite Communications

In satellite communications, the channel noise can be regarded as additive white Gaussian; thus the errors are mostly random. Since the transmitter power and the size of the antennas of a spacecraft are limited, it is desirable to use a code with large error-correction capability to compensate for the low signal-to-noise ratio. This is because we can reduce the required transmitting power per bit ratio to obtain a given bit-error rate by using an error-correcting code. However, since the available bandwidth for transmission is also limited, the code rate must not be too low. Furthermore, for a system where the transmission speed is very high, we cannot use a code that needs a complicated decoder.

Self-orthogonal convolutional codes, which are decoded by a simple decoder, were mainly used for satellite communications. However a ½ rate convolutional code with Viterbi decoding, which produces larger error-correction capability, is often adopted now. High-rate BCH codes are also used in some cases. High-rate convolutional codes with Viterbi decoding are being applied.

1.2.2 Applications to Broadcasting

In teletext, which transmits digitalized characters and figures overlapped with a TV signal, the most important type of error is a burst caused by an impulsive noise. Thus a code with large error-correction capability is required. But, since decoding is performed in a television set, the decoder must be small. As a result, a different set cyclic code, which can be decoded by a relatively simple circuit, has been selected for the Japanese teletext.

1.2.3 Applications to Other Communication Systems

In space communication that transmits observed data from spacecraft to ground, most of the errors are random. Since the received signal is usually weak, large error-correction capability is needed, especially for deep space communications. However, the bandwidth limitations are not strict, and we can install a large decoder at ground stations. Therefore, codes with large error-correction capability are used for space communications. Convolutional codes with sequential decoding used to be employed, but convolutional codes with Viterbi decoding are mainly used as of the late 1980s. In the case where the signal-to-noise ratio is particularly low, a concatenated code with Reed—Solomon and convolutional codes is often adopted.

In mobile communications, burst errors owing to multipath fading are dominant. Since the bandwidth available to one channel is strictly limited, the code rate must be high. And since the decoder is on the vehicle, we cannot use a code that requires a large decoder. Therefore, burst-correcting codes decoded by a simple decoder are suitable for mobile communications. In some systems, BCH codes are used in conjunction with ARQ.

1.3 Error-Control Coding in Computers

Applications of error-control coding to computer systems have also been studied for a long time. In the computer field of the late 1980s, error-control codes are being used in most storage devices.

1.3.1 Applications to Logic Circuits

In order to detect or correct errors in logic and arithmetic circuits of computers, codes based on partially check codes and Hamming codes are used. And codes especially suited to detect and/or correct errors in logic and arithmetic circuits are being developed.

1.3.2 Applications to Semiconductor Memory Systems

Errors that occur in semiconductor memory systems can be regarded as either random errors or byte errors. The speed of the operations for semiconductor memory systems is very high, and therefore the decoders must be extremely fast. Also the number of redundant symbols cannot be very large. As a result, a class of single-error-correcting and double-error-detecting (SEC-DED) codes has been widely used for memory systems. These codes are constructed on the basis of Hamming codes, but the decoder for these codes is smaller and can be operated at a higher speed than that of Hamming codes. In some systems, single-byte—error-correcting (SbEC) codes or single-byte—correcting and double-byte—error-correcting (SbEC-DbEC) codes are used.

1.3.3 Applications to Magnetic Recording Systems

In magnetic recording systems, burst errors caused by defects of the device or dust sometimes occur. Therefore burst-correcting codes have been employed, while Reed—Solomon-like codes are of growing importance. In magnetic-disk systems, Fire codes (among the most well-known burst-correcting codes) are being used, while SbEC-DbEC codes based on Reed—Solomon codes with interleaving have been implemented recently. In magnetic-tape systems, burst-correcting codes derived from cyclic codes were being used; however, SbEC codes based on Reed—Solomon codes have been adopted lately. In mass-storage systems, interleaved SbEC codes are being used.

1.3.4 Applications to Optical Disk Systems

In optical disk systems, both random and burst errors occur and error rate of the device is relatively high. Therefore, codes with large error-correction capability are required. As a result, multiple coded Reed—Solomon codes in conjunction with interleaving have been adopted.

1.4 Error-Control Coding in Audio-Video Systems

Digital audio or video systems using pulse coded modulation (PCM) have recently come into practical use. Most of such systems were developed utilizing error-control coding as an essential technique. As of 1989, audio-video systems may be the most active area in applying error-control coding.

1.4.1 Applications to Audio Systems

Since the error rate of devices is high and both random and burst errors occur in digital audio systems, large error-correction capability is required. However, since the correlation between adjacent data is relatively high for audio signals, we can estimate the correct value of erroneous data by using the values of the data before and after the erroneous data. Miscorrection by the decoder causing a click noise must be strictly avoided. Therefore, it is desirable to estimate the correct values of data that are likely to be miscorrected as described previously, instead of correcting any errors at the decoder. Doubly coded Reed—Solomon or cyclic codes with interleaving are often used.

1.4.2 Applications to Video Systems

Requirements for error-control codes for video systems are the same as those for audio systems, except that (1) the probability of miscorrection does not need to be as small as that for audio systems and (2) the processing speed must be much higher. Doubly coded Reed—Solomon or cyclic codes with interleaving are also used for video systems.

1.5 Future Prospects of Error-Control Coding

Error-control coding has become an essential part in most of the digital communication and recording systems. As the amount of digital data that must be transmitted or stored reliably increases, error-control coding is becoming more and more important.

Because of advances in solid-state electronics technology, codes with large error-correction capability and high efficiency are widely used. In particular, coding systems that employ two codes combined together to obtain better performance, e.g., concatenated codes, are often employed. Error-control systems with much larger error-correction capability will be developed further.

The need for data transmission or data processing at a higher speed is growing rapidly. Recently fast decoders utilizing parallel processing techniques have been implemented. The development of faster decoders will be the most important problem in applying error-control coding to future digital systems.

The theory of error-control coding has been developed as a mathematical theory. However, as a practical technology, the study of error-coding has only just begun. In the future, techniques to apply error-control coding as well as theories to maintain more powerful codes will be extensively studied and developed.

Bibliography

Blahut, R. E.Theory and Practice of Error Control Codes.. Massachusetts: Addison-Wesley, Reading, 1983.

Clark, G. C., Jr., Cain, J. B.Error-Control Coding for Digital Communications.. New York: Plenum, 1981.

Glover, N., Dudley, T. Practical Error Correction Design for Engineers,, 2nd ed. Data Systems Technology Corp., 1988.

Lin, S., Costello, D. J., Jr.Error Control Coding: Fundamentals and Applications.. New Jersey: Prentice-Hall, Englewood Cliffs, 1983.

Michelson, A. M., Leveseque, A. H.Error-Control Techniques for Digital Communications.. New York: Wiley, 1985.

Peterson, W. W., Weldon, E. J., Jr. Error-Correcting Codes,, 2nd ed. Massachusetts: MIT Press, Cambridge, 1972.

Rao, T. R.N., Fujiwara, E.Error-Control Coding for Computer Systems.. New Jersey: Prentice-Hall, Englewood Cliffs, 1989.

Vans tone, S. A., van Oorschot, P. C.An Introduction to Error Correcting Codes with Applications.. Massachusetts: Kluwer, Hingham, 1989.

Viterbi, A. J., Omura, J. K.Principles of Digital Communication and Coding.. New York: McGraw-Hill, 1979.

Wiggert, D.Codes for Error Control and Synchronization.. Massachusetts: Artech House, Dedham, 1988.

Part I

Coding Theory

Outline

Chapter 2: Principles of Error Detection and Correction

Chapter 3: Block Codes

Chapter 4: Convolutional Codes

2

Principles of Error Detection and Correction

Takahiro Yamada

Publisher Summary

This chapter discusses principles of error detection and correction. To detect or correct errors on digital communications channels, error-control coding techniques are employed. There are two types of encoding methods. The codeword is a binary sequence of length n (n > k), which is denoted by x = (x1, x2, … xn) where xi = 0 or 1. The first is block encoding, where each codeword is generated from one block of k message symbols. So the block encoder retains no information about a message block after an encoding operation for the block is completed. The second is convolutional encoding, where each codeword is generated from several consecutive message blocks. In general, a convolutional encoder consists of shift registers to retain message symbols and logical circuits to produce output symbols. The codeword from the encoder is transmitted on the channel. The channel can be a storage device in some applications. Decoding methods are classified into two classes: (1) hard decision decoding—where the receiver decides whether each transmitted symbol is a 0 or a 1 and the received vector is a binary sequence and (2) soft decision decoding—where the receiver generates some analog information on each received symbol and the decoder recovers the message from the analog or quantized received vector y.

2.1. Basic Concepts

2.1.1. Model of a Digital Communications System

2.1.2. Hamming Distance and Minimum Distance

2.1.3. Random Errors and Burst Errors

2.2. Principles of Error Detection and Correction

2.2.1. Principles of Error Detection

2.2.2. Principles of Error Correction

2.2.3. Optimum Decoding Methods

2.3. Finite Fields

2.3.1. Elements of Finite Fields

2.3.2. Vector Representation of Elements

2.3.3. Primitive Polynomials and Minimal Polynomials

2.4. Linear Codes

2.4.1. Definition of Linear Codes

2.4.2. Decoding Linear Codes

2.4.3. Some Examples of Linear Codes

Bibliography

2.1 Basic Concepts

2.1.1 Model of a Digital Communications System

In order to detect or correct errors on digital communications channels, we employ error-control coding techniques. Figure 2.1 is a conceptual diagram of a digital communications system with error-control coding.

Fig. 2.1 Model of digital communications system.

In Fig. 2.1, the message to be transmitted is a sequence of binary symbols of length k, which is denoted by u = (u1, uuk) where ui = 0 or 1. The message is fed into an encoder to generate a codeword. The codeword is a binary sequence of length n (n > k), which is denoted by x = (x1, xxn) where xi = 0 or 1. The mapping of the encoder is chosen so that certain errors can be detected or corrected at the receiving end. The number of symbols is increased by this mapping from k to n. The ratio k/n is called the code rate and is denoted by R.

There are two types of encoding methods. The first is block encoding, where each codeword is generated from one block of k message symbols. So the block encoder retains no information about a message block after an encoding operation for the block has been completed. The second is convolutional encoding, where each codeword is generated from several