IEEE TRANSACTIONS ON COMMUNICATIONS, VOL. 48, NO.

1, JANUARY 2000 1

Joint Source-Channel Decoding for Variable-Length Encoded Data

by Exact and Approximate MAP Sequence Estimation
Moonseo Park, Student Member, IEEE, and David J. Miller, Member, IEEE

Abstract—Joint source-channel decoding based on residual
source redundancy is an effective paradigm for error-resilient
data compression. While previous work only considered fixed-rate
systems, the extension of these techniques for variable-length
encoded data was recently independently proposed by the authors
and by Demir and Sayood. In this letter, we describe and compare
the performance of a computationally complex exact maximum a
posteriori (MAP) decoder, its efficient approximation, an alterna-
tive approximate decoder, and an improved version of this decoder
suggested here. Moreover, we evaluate several source and channel
coding configurations. The results show that our approximate
MAP technique outperforms other approximate methods and
provides substantial error protection to variable-length encoded
data.
Index Terms—Hidden Markov models, joint source-channel
decoding, MAP estimation, residual redundancy, variable length
codes.
I. INTRODUCTION
and standard forward-error correction (FEC). However, FEC
may be bandwidth-inefficient when the channel conditions are
fairly mild. A variable-rate extension of JSC decoding could
thus potentially replace FEC under mild conditions, and for
noisier channels, it could possibly be used with FEC to reduce
the coding rate and to extend the range of conditions under
which the bit stream is adequately protected.
However, the nature of variable-rate systems greatly compli-
cates the estimation problem at the decoder. Whereas, under
certain statistical assumptions, optimal maximum a posteriori
(MAP) sequence estimation can be efficiently achieved in the
fixed-rate case by dynamic programming on a trellis, this is not
possible, in general, in the variable-rate case. Recently, several
different variable-rate approaches were proposed. In [6], a com-
putationally complex exact MAP decoding method and an effi-
cient approximation were both suggested. In [2], a different ap-
proximate method was independently proposed. In [13], another
method was proposed specifically for memoryless sources. Re-

M OTIVATED by Shannon’s source-channel separation

theorem, source and channel coder designs are tra-
ditionally considered independently. However, Shannon’s
cently in [5], a new MAP decoding method was proposed. This
approach does not constrain the length of the decoded symbol
sequence and is thus best suited for the case where the number
theorem effectively assumes that the source coder removes
of transmitted symbols is unknown at the decoder. Here, as in
all redundancy and the channel coder corrects all errors. In
[2] and [6], we focus on the more typical situation where the
practical systems, there is often residual redundancy in the
number of transmitted symbols is known (e.g., the fixed-row
source encoder’s output [12], quantified as excess rate [9], and
length of an image) or specified by side information.
attributable to constraints on complexity and delay that may
In this letter, we describe the methods in [6] and [2] and
limit performance. Thus, one can often improve performance
identify a potential advantage of the method in [6]. We then
by considering the source and channel designs jointly [1]. In
suggest an improvement to [2]. Simulation results will confirm
this work, we focus on joint source-channel (JSC) methods for
both the performance advantage of [6] and the effectiveness of
decoding, based on residual source redundancy [8], [12]. In
the improvement to [2]. In addition to comparing JSC decoding
this approach, redundancy is used as a form of implicit channel
methods, we also investigate fundamental design choices
protection by the decoder, which acts as a statistical estimator
within a differential pulse-code modulation (DPCM) coding
of the transmitted sequence. This approach provides error
environment. Our results indicate the following ranges of
correction capability and has been found to achieve substantial
channel conditions: 1) where variable-rate coding is preferred
benefits for very noisy channels [4], [8], [12].
over fixed-rate coding; 2) where given JSC decoding, FEC is
Until very recently, work on JSC decoding methods focused
not required; and 3) where JSC decoding in conjunction with
almost exclusively on systems that use fixed-length coding, even
particular types of FEC is most effective. In Section II, we
though variable-length coding (VLC) is widely used. Typically,
discuss exact and approximate MAP decoding for VLC data.
VLC bit streams are made channel-robust through packetization
In Section III, we present our experimental results. The letter
concludes with suggestions for future work.
Paper approved by E. Ayanoglu, the Editor for Communication Theory and
Coding Application of the IEEE Communications Society. Manuscript received
May 21, 1998. This work was supported in part by National Science Foundation
Career Award IIS-9624870. This paper was presented in part at the IEEE Inter- II. EXACT AND APPROXIMATE MAP DECODING
national Conference on Acoustics, Speech, and Signal Processing, Phoenix, AZ,
March 1999. We will consider several configurations, including the two
M. Park was with the Department of Electrical Engineering, Pennsylvania
State University, University Park, PA 16802 USA. He is now with Globespan, shown in Fig. 1, with source encoding/quantization followed
Inc., Red Bank, NJ 07701 USA. by either: a) Huffman coding and then binary convolutional en-
D. J. Miller is with the Department of Electrical Engineering, Pennsylvania coding (BCE); or b) nonbinary convolutional encoding (NCE)
State University, Rm. 227-C EE West Bldg., University Park, PA 16802 (e-mail:
miller@perseus.ee.psu.edu). and then Huffman coding [2]. Consider a fixed-length sequence
Publisher Item Identifier S 0090-6778(00)00496-7. of information symbols , , with the
0090–6778/00$10.00 © 2000 IEEE
2 IEEE TRANSACTIONS ON COMMUNICATIONS, VOL. 48, NO. 1, JANUARY 2000
(a)
(b)
Fig. 1. Communications system models considered in this letter, based on (a) BCE and (b) NCE.
symbol alphabet of size . In case a), we take to be the quan-
tization index sequence output by the source encoder, and in b),
we take it to be the symbol sequence output by the nonbinary
convolutional encoder. In either case, each index is Huffman
coded, resulting in an encoded bit stream , a connected path through the directed graph. However, in each
, with the length varying for each block of sym-
bols. For decoding, in case a), we propose a decoding (inner)
of the convolutional code followed by a JSC decoding (outer).
In case b), we use JSC decoding followed by NCE decoding
(a symbol-to-quantization index mapping). In this section, we
focus on the JSC decoding, common to both configurations. The
JSC decoder, which is assumed to know the length based on
transmitted side information, receives the corrupted bit stream
.1 Its objective is to estimate given . As
in previous work [4], [8], [12], we model residual redundancy by
assuming the random process is first-order Markovian, based
on the conditional probabilities and the
initial probabilities . We will also assume that the
channel in Fig. 1 is binary symmetric, with bit-error rate (BER)
. Moreover, in Fig. 1(a), we will find it useful to model the effec-
tive channel formed by the BCE encoder-channel-BCE decoder
cascade as also being memoryless and binary symmetric. While
the true effective channel will in general have memory, the mem-
oryless assumption allows us to directly apply the JSC decoding
method from [6]. Also, the decoder based on this (somewhat
inaccurate) model will be seen to achieve good performance.
An extension of [6] based on more accurate modeling can be
identified as a future direction.2 Given our modeling assump-
tions, the decoder objective addressed here is to estimate the se-
quence of MAP probability, i.e., to realize the sequence MAP
rule . For the fixed-rate case, se-
quence MAP decoding can be posed as a search for the optimal
path through an -state trellis, efficiently implementable via
dynamic programming [8]. In the variable-rate case, the optimal
sequence MAP decoder can also be achieved by dynamic pro-
gramming as proposed in [6]. However, the problem cannot be
represented on a trellis, but rather it must be posed on a more
complicated directed graph, due to the lack of synchronization
between the information symbols and the corrupted, received
bits.
case a), r is the output of the convolutional decoder.
1In
2A Markovian assumption, based on a finite number of channel states, is more
reasonable than the memoryless BSC for modeling the effective channel. If the
channel and source are both assumed Markovian, then the (source, channel)
pairs are likewise Markovian. Thus, one can develop extensions of the sequence
estimation methods considered here, based on this (somewhat) more complex
statistical model.
Two alternative graph representations proposed in [6] and [2]
are shown, respectively, in Figs. 2 and 3, for and a partic-
ular set of codewords. In both figures, the small boxes represent
possible decoder states, with a decoded sequence specified by
case, the states have different meanings. In the bit-constrained
case [6] (Fig. 2), the horizontal axis represents bit “time,” with
each state at “time” representing a particular set of symbol
sequences that exhaust bits. The sequences in the set are char-
acterized by having a common length and a common termi-
nating symbol. The enclosed number within a state box is the
length of the symbol sequence. In the symbol-constrained case
[2] (Fig. 3), the horizontal axis represents symbol “time,” with
each state at “time” representing a particular set of symbol se-
quences of length . The sequences in this set are characterized
by having a common terminating symbol and by exhausting the
same number of bits. The enclosed number within a state is the
length of the associated bit sequence. The statistical modeling
assumptions guarantee that for both graph representations, one
can compute (which differs from
by the constant ) by summing metric
contributions associated with each branch along the decoded
path. Each branch contributes a term of the form
, with the second proba-
bility based on the BER and the Hamming distance between
the Huffman codeword for and its corrupted version
, a subsequence of bits from
. Here, is the length of the bit sequence required to de-
code up to symbol . A dynamic programming algorithm for
realizing the exact MAP rule was suggested in [6] for the struc-
ture in Fig. 2. This approach could be applied to the graph in
Fig. 3 as well. However, in both representations, the number
of states grows with the sequence length and can become very
large, making exact MAP decoding computationally imprac-
tical.3 This complexity necessitated the development of approx-
imate schemes, proposed independently in [6] and [2], based,
respectively, on the structures in Figs. 2 and 3.
There is no strong reason to favor either graph structure for
exact MAP decoding. However, as we indicate in this section,
the structures differ markedly when one considers approximate
schemes. For the graph in Fig. 2, consider the set of states within
any dashed box. In each such set, all states exhaust the same
number ( ) of bits and terminate a symbol subsequence at bit
using the same decoded symbol. Such sets are defined at all
3As an example, for T = 256 (an image row length), N = 8, B = 429, and
a particular set of VLC codewords, the number of states for the bit-constrained
graph structure will reach 1365.
IEEE TRANSACTIONS ON COMMUNICATIONS, VOL. 48, NO. 1, JANUARY 2000
Fig. 2. Bit-constrained directed graph representation for VLC decoding/MAP sequence estimation in [6].
Fig. 3. Symbol-constrained directed graph representation for VLC
decoding/approximate MAP sequence estimation in [2].
bit lengths. Our approximate method applied to this graph [6]
is similar to dynamic programming in that it involves a “for-
ward” operation for propagating and accumulating metrics and
saving pointers to previous states, and a “traceback” operation
to find the best state sequence. However, whereas in standard
dynamic programming this forward operation effectively saves,
at each time, the best partial sequence terminating in each state,
the approximate method reduces complexity by saving only one
sequence for each set of states belonging to the same dashed set.
The saved sequence is the one best in the sense of a posteriori
probability. For each dashed set, this approach compares symbol
sequences of different lengths and then prunes all but one. The
sequence pruning is achieved by removing, from each dashed
set, all but one state (the one indexed by the length of the op-
timal sequence). Thus, a state reduction operation is performed
within the Viterbi algorithm’s forward step, for each dashed set,
at each bit time.
Let us consider an example of state reduction for the graph in
Fig. 2. Denote the th state in the th dashed set at bit length
by . Then, for the first dashed set at , the
method will choose between and .4 The saved
state is the one which terminates the symbol sequence of MAP
(log) probability. Note that this operation at bit length prunes
states and branches from the graph, thus restricting the candi-
date symbol sequences (and states) considered at subsequent bit
lengths . Using the same example, if is chosen
over at , then is removed from the graph,
4Here, s (1; 1) corresponds to the state with symbol length 4 and s (1; 2)
to the state with symbol length 3. The states within dashed sets are enumerated
from the top down.
3
along with all branches emanating from this state. When sub-
sequently performing state reduction for the first dashed set at
, symbol sequences that include the branch from
to will not be considered as candidates.5
While it is not implemented this way, it may be useful to think
of our approximate method [6] in terms of the following equiv-
alent procedure: 1) perform state reduction on the graph up to
bit time and 2) perform optimal (MAP) sequence estimation
on the resulting reduced graph using the Viterbi algorithm. A
more detailed pseudocode description of both our approximate
and exact MAP algorithms can be found in [6].
At this point, we make two observations about our approxi-
mate method. First, although the cardinality of each dashed set
grows with bit length, the number of retained states for each
dashed set remains constant, at one. Thus, the complexity of the
method remains manageable for increasing bit length. Second,
we emphasize that our state reduction rule is based on a MAP
criterion. To be precise, consider the th state in the th dashed
set at bit length . Let denote the symbol
sequence with MAP probability from the set of sequences
terminating in this state. Here, is the graph “carved
out” by state reduction through bit length . The explicit
notational dependence of on indicates that the can-
didate symbol sequences are restricted to those that trace paths
through the reduced graph . Given this notation, state
reduction for set at bit length realizes the bit-constrained
MAP rule: , with
and the subset of states in the dashed
set consistent with the reduced graph . The state
hypotheses all correspond to symbol sequences of different
lengths, but they all try to “explain” the same observed data
sequence . To summarize, our “bit-constrained” method
performs state reduction consistent with a MAP criterion and
finds the symbol sequence that is MAP-optimal on the resulting
reduced graph via dynamic programming.
Now, consider approximate decoding for the symbol-con-
strained graph in Fig. 3 [2]. Again, the dashed sets group
together states that terminate symbol sequences using the same
decoded symbol. However, now, rather than exhausting the
same number of bits, all states in any dashed set exhaust the
same number of symbols. The method in [2], similar to [6],
0
5Moreover, if a removed state at bit length n 1 is the only source of branch
connections for some possible state at bit length n, then this state will not be
created. For example, if s (1; 1) is removed, then s (1; 1) will not be created.
4 IEEE TRANSACTIONS ON COMMUNICATIONS, VOL. 48, NO. 1, JANUARY 2000
involves a “forward” operation and a “traceback.” Similar to
[6], this approach approximates the Viterbi “forward” step
by saving, at each symbol length, only the best state from
each dashed set. Thus, the number of saved states remains
constant for increasing symbol length, with the complexity
of the resulting method again manageable. Clearly, the
methods from [6] and [2] implement quite similar operations.
Thus, we might expect them to perform roughly the same.
However, there are potential problems with the approach
in [2], as next indicated. Note that whereas state reduction
in [6] compares symbol sequences of different lengths that
exhaust the same number of bits, the method in [2] compares
symbol sequences of the same length that exhaust different
numbers of bits. This suggests that state reduction in [2]
might be equivalent to a “symbol-constrained” MAP rule:
. Here, is the
symbol sequence length, with the received bit sequence
associated with the th state in the th dashed set. Again,
is the subset of states in the dashed set consistent with the
reduced graph . Also, is the sequence of
symbols with MAP probability from the set of sequences that
are consistent with the state-reduced graph and that terminate
in state . We emphasize that unlike the previous
bit-constrained case, the bit sequences associated with
states in the same dashed set all have different lengths. The
significance of this observation is next seen.
It is not obvious which state reduction rule—bit-constrained
or symbol-constrained MAP—is to be preferred. However, a
difficulty with [2] is that it only implements an approximation
to the symbol-constrained MAP rule. In particular, note that
for both decoder structures (Figs. 2 and 3), the accumulated
metric used for state reduction consists of only two terms:
the log likelihood of a symbol sequence starting at symbol
time 1 and ending at some symbol time , ,
and the conditional log likelihood of a received bit sequence
. However, the log a posteriori probability
is . Thus, for
both approximate decoder structures, the term is
omitted. In the bit-constrained case, this does not introduce any
suboptimality in the state reduction rule, since all competing
states from the same dashed set involve the same
received bit sequence. However, omitting this term does intro-
duce suboptimality in the symbol-constrained case, since each
state from the same dashed set corresponds to a bit sequence
with a different length. Thus, to achieve MAP state reduction in
the symbol-constrained case, each candidate requires
a different term , omitted in [2]. Moreover,
one cannot easily correct this problem in a precise way, since
calculation of requires at least as much computation
as the exact MAP rule itself.6 As a practical way to improve the
method from [2], we assume all bit sequences of a given length
are equally likely, i.e., we let . We then
6Although one may use either the graph structure in Fig. 2 or the one in Fig. 3
to calculate P [r ], the probability of a bit sequence of length n, it is both simpler
and more intuitive to consider the structure in Fig. 2. For this structure, P [r ]
can be computed by first recursively calculating forward variables (see [11]) for
all states in the graph up to and including bit length n. The sum of the forward
variables at length n gives the required probability.
modify the symbol-constrained approach to include this term
in the accumulated metric used for state reduction. Next, we
consider the effect of this modification, and more generally,
evaluate the performance of all of the JSC decoders discussed
here.
III. RESULTS
We tested the various methods for DPCM coding of images
over a binary symmetric channel. The choice of DPCM allows
comparison with [2] and provides a simple framework for eval-
uating decoding techniques and basic system configurations,
rather than focusing on absolute image coding performance.
We also note that DPCM is often practically used for coding
low-frequency transform and subband signals. We considered
both variable- and fixed-rate systems, with and without FEC,
with the FEC based on BCE or NCE. In [2], a system composed
of NCE-based channel encoding and JSC decoding was com-
pared with a more conventional system based on BCE and (stan-
dard Viterbi) channel decoding. There, the NCE-based system
was found to achieve superior performance. However, in the
BCE-based scheme, no JSC decoding was used. Here, for the
BCE-based scheme, we use both inner (bit level, Viterbi based)
decoding and outer (symbol level, JSC) decoding.
For BCE, we chose a rate 2/3 convolutional encoder with con-
straint length 2 as in [10]. This was applied to the Huffman-
coded indices associated with a six-level quantizer. In this case
[Fig. 1(a)], rather than using , the JSC decoder used the ef-
fective BER of the concatenated BCE-channel-Viterbi decoder
system. For NCE, we matched the code to the (six-level) quanti-
zation to increase residual redundancy in the sequence, as in ear-
lier work [2]. We thus chose an NCE with 6 input symbols and
12 output symbols.7 Denote the quantization index at symbol
time by . Then, the NCE output is given
by the equation
if
otherwise.
Our particular choices for BCE and NCE were made so that the
different systems would achieve comparable bit rates, allowing
useful comparisons.
For variable rates, in conjunction with BCE and Viterbi-based
(inner) decoding, we implemented the bit-constrained approxi-
mate MAP (BC-AMAP) (outer) JSC decoder [6]. This method
is denoted BC-AMAP + BCE. For variable rates, in con-
junction with NCE, we implemented the exact MAP decoder
(denoted MAP + NCE) [6], the bit-constrained approximation
(BC-AMAP + NCE) [6], the symbol-constrained method
(SC-AMAP + NCE) [2], and its modification suggested here
(MSC-AMAP + NCE). We also implemented a 14-level vari-
able-rate quantizer with decoding solely based on BC-AMAP,
i.e., without channel coding (denoted BC-AMAP~FEC). For
fixed-rate DPCM, we used an eight-level quantizer with: 1)
conventional DPCM decoding (denoted FLC); 2) channel-opti-
mized scalar quantization (COSQ) [3]; and 3) sequence-based,
7We chose an ordered mapping of quantization levels to indices, with an in-
creasing sequence of integer indices assigned to the sequence of quantization
levels ordered from smallest to largest.
IEEE TRANSACTIONS ON COMMUNICATIONS, VOL. 48, NO. 1, JANUARY 2000
approximate MMSE decoding (denoted FLC + SAMMSE) [4].
For variable rates, we chose a block length of .8 The
side information associated with specifying the bit length for
each block was estimated and included in the overall bit rate.
For each variable-rate system, the side information was 0.1
bits/sample and the Huffman code rate was 2.9 bits/sample.
Thus, all the systems (both variable and fixed rate) achieved a
rate of roughly 3 bits/sample in coding the 256 256 Lenna
image.9
Before presenting results, we discuss one important MAP de-
coding detail that was applied in [6] but not discussed there.
Note that the decoding must produce a valid symbol sequence,
i.e., one of length consistent with the bit length . To en-
sure a valid decoded sequence, we need to constrain the graph
to exclude the creation of states not associated with any valid se-
quences. We sketch how this is done in the bit-constrained case
(a similar approach applies for the symbol-constrained one).
First, before decoding a sequence of bits, we compute upper
and lower bounds on the number of symbols valid for bit length
, for each . A symbol sequence at bit length
is valid if it can be extended to a sequence of symbols at
length . For example, for Fig. 2, at , the lower
bound is and the upper bound is . There are sev-
eral ways to find the upper and lower bounds for each . One
approach involves a consideration of the codeword sequences
that satisfy the bounds—the lower bound will be achieved by
using the longest codeword as much as possible, while the upper
bound will be achieved by using the shortest codeword as much
as possible. Based on these respective strategies, it is possible
to identify the lower and upper bounds for each in a simple,
low-complexity fashion. Given the upper and lower bounds, we
are ready to decode, building up the graph iteratively. The states
added at bit are determined based on the existing graph up
to bit and on the given VLC codewords. States are only
added if they fall within the symbol length bounds.
Fig. 4 shows the SNR of several decoding techniques. We are
interested in comparison of the following: 1) the variable-rate
JSC decoding methods; 2) the NCE-based schemes with
the BCE-based method; 3) JSC decoding with and without
FEC; and 4) variable and fixed-rate coding. Comparing all
the JSC methods used with NCE, we find that while exact
MAP decoding achieves the best results, the more practical
BC-AMAP + NCE approach, with computational complexity
8Our choice T = 64 was judiciously made to limit error propagation. Further,
this choice may be convenient for image coding applications since the image
row length is often a multiple of 64. For this value of T , B lies between 81 and
192 for the BCE case.
9There are two sources of error in the estimates for the variable-rate systems,
which should be noted as the following. 1) We used the entropy of the side infor-
mation, which slightly underestimates the true rate; however, the upper bound

on expected codelength is also 0.1 bits/sample. 2) We assumed the side in-
formation is received error-free. To ensure this, suppose we use a rate 1/3 code
for protection. This amounts to roughly 0.3 bits/sample overhead. Since none of
the fixed-rate systems can achieve fractional rates, we did not attempt to more
closely match the bit rates for the fixed- and variable-rate systems (if we were
to instead use vector quantization for fixed rates, then for fair comparison, we
would need to do so for the variable-rate systems as well). However, if this were
done, in Fig. 4, the performance gains of the variable-rate systems at low BER’s
would likely be somewhat reduced, and the advantage of the fixed-rate systems
at high BER’s would likely be somewhat enhanced. Note further that 1) and 2)
should have a negligible effect on comparisons solely involving the variable-rate
systems.
5
Fig. 4. SNR versus channel BER performance of several techniques.
roughly forty times smaller, also achieves good performance.
Both methods substantially outperform SC-AMAP + NCE
from [2]. BC-AMAP + NCE achieves a 5.7-dB gain at
and a 7.1-dB gain at . This confirms the advantage
of the bit-constrained method [6] discussed in Section II. The
modification of SC-AMAP + NCE to include the term
substantially improves its performance. MSC-AMAP + NCE
is 4.4 dB better than SC-AMAP + NCE at and 6.0 dB
better at .
In comparing all the curves, except exact MAP and COSQ,
we can identify different BER regimes where particular config-
urations are preferred. For the low BER range, BC-AMAP~FEC
achieves the best results. Thus, here, JSC decoding for vari-
able rates provides adequate protection without wasting bits
on FEC. Moreover, in this regime, variable-rate coding is pre-
ferred over fixed-rate coding. For the low-to-medium BER
range ( ), the best performance is achieved
by inner BCE and outer JSC decoding. From this result and
those in [2], we can infer that inner convolutional decoding
in conjunction with outer JSC decoding is a more effective
scheme than standard convolutional coding without outer JSC
coding. Moreover, in this BER regime, we again find that
a variable rate scheme outperforms the fixed rate ones. For
medium-to-high BER (epsilon belongs to [0.01, 0.05]), the best
performance is achieved by combining NCE with BC-AMP
JSC coding. Thus, here, NCE is preferred over BCE. Finally,
for high BER ( ), we have the expected result that
fixed-length coding (with MMSE-based JSC decoding [4]) out-
performs the variable-rate systems.
For very high BER, COSQ outperforms other methods be-
cause the COSQ quantization levels were redesigned for every
BER, while for other systems, these levels were designed only
for the error-free channel. If we also optimize our JSC decoding
scheme (both the quantization levels and the allocation between
source and channel coding) for a particular BER, we can im-
prove performance. In particular, for , if we use only
a three-level quantizer and rate 1/2 BCE with the BC-AMAP
decoder, this approach outperforms the COSQ system as shown
in the figure.
6 IEEE TRANSACTIONS ON COMMUNICATIONS, VOL. 48, NO. 1, JANUARY 2000
IV. CONCLUSION
We have described several methods for extending JSC de-
coding to VLC data. In comparing these techniques, we found
the approximate MAP method from [6] to achieve the best
results, in comparison with alternative approximate decoders.
Our experiments identified BER regimes where fundamentally
different system configurations (fixed rate, variable rate, FEC
using either BCE or NCE, no explicit channel coding) are
preferred. In very recent work, we have developed an alter-
native decoder for variable-rate systems based on a minimum
mean-squared error criterion [7]. Future work may address
extensions, including bit allocation strategies and alternative
source and channel models. We may also apply the new JSC
methods to other compression environments, such as subband
coding.
ACKNOWLEDGMENT
The authors would like to thank Dr. R. Van Dyck from Penn-
sylvania State University for a suggestion that helped to improve
the paper.
REFERENCES
[1] E. Ayanoğlu and R. M. Gray, “The design of joint source and channel
trellis waveform coders,” IEEE Trans. Inform. Theory, vol. IT-33, pp.
855–865, Nov. 1987.
[2] N. Demir and K. Sayood, “Joint source/channel coding for variable
length codes,,” in Proc. Data Comp. Conf., Snowbird, UT, 1998, pp.
139–148.
[3] A. J. Kurtenbach and P. A. Wintz, “Quantizing for noisy channels,” IEEE
Trans. Commun., vol. COM-17, pp. 291–302, Apr. 1969.
[4] D. J. Miller and M. Park, “A sequence-based approximate MMSE
decoder for source coding over noisy channels using discrete hidden
Markov models,” IEEE Trans. Commun., vol. 46, pp. 222–231, Feb.
1998.
[5] A. H. Murad and T. E. Fuja, “Joint source-channel decoding of vari-
able-length encoded sources,” in Proc. Information Theory Workshop,
Killarney, Ireland, June 1998.
[6] M. Park and D. J. Miller, “Decoding entropy-coded symbols over noisy
channels by MAP sequence estimation for asynchronous HMMs,” in
Proc. Conf. Information Sciences and Systems, Princeton, NJ, Mar.
1998, pp. 477–482.
[7] , “Improved joint source-channel decoding for variable-length en-
coded data using soft decisions and MMSE estimation,” in Proc. Data
Comp. Conf., Snowbird, UT, Mar. 1999, p. 544.
[8] N. Phamdo and N. Farvardin, “Optimal detection of discrete Markov
sources over discrete memoryless channels—Application to combined
source channel coding,” IEEE Trans. Inform. Theory, vol. 40, pp.
186–193, Jan. 1994.
[9] N. Phamdo, F. Alajaji, and N. Farvardin, “Quantization of memoryless
and Gauss–Markov sources over binary Markov channels,” IEEE Trans.
Commun., vol. 45, pp. 668–675, June 1997.
[10] J. G. Proakis, Digital Communications. New York: McGraw-Hill,
1995, p. 472.
[11] L. Rabiner and B. H. Juang, Fundamentals of Speech Recogni-
tion. Englewood Cliffs, NJ: Prentice Hall, 1993, pp. 335–337.
[12] K. Sayood and J. C. Borkenhagen, “Use of residual redundancy in the
design of joint source/channel coders,” IEEE Trans. Commun., vol. 39,
pp. 838–846, June 1991.
[13] K. P. Subbalakshmi and J. Vaisey, “Optimal decoding of entropy coded
memoryless sources over binary symmetric channels,” in Proc. Data
Comp. Conf., Snowbird, UT, 1998, p. 573.