Basics of Wavelets

References: I. Daubechies (Ten Lectures on Wavelets; Orthonormal Bases of Compactly Supported Wavelets) Also: Y. Meyer, S. Mallat Outline: 1. Need for time-frequency localization 2. Orthonormal wavelet bases: examples 3. Meyer wavelet 4. Orthonormal wavelets and multiresolution analysis

1.

Introduction

Signal:

fig 1 Interested in frequency content of signal, locally in time. E.G., what is the frequency content in the interval [.5, .6]?

Standard techniques: write in Fourier series as sum of sines and cosines: given function defined on P P as above: 0 B
_ " +! " +8 cos 8B1P ,8 sin 8B1P # 8"

+8 ,8 constants) " P +8 ( .B 0 B cos 8B1P P P " P ,8 ( .B 0 B sin 8B 1P P P (generally 0 is complex-valued and +8 ,8 are complex numbers). FOURIER SERIES: Consider function 0 B defined on P P. Let P# P P square integrable functions 0 P P (
P

.B l0 # Bl _

where complex numbers. Then P# forms a Hilbert space. Basis for Hilbert space: " " cos 8B 1 P sin 8B1P P P R "
_

(together with the constant function "#P).

These vectors form an orthonormal basis for P# length 1).

(constants "P give

1. Recall the complex form of Fourier series: Equivalent representation: Can use Euler's formula /3, cos , 3 sin ,. Can show similarly that the family " 38B1P / #P
8_

8_

" 3 cos 8B1P sin8B1P #P #P 8_ is orthonormal basis for P# Function 0 B can be written 0 B " -8 98 B
_ 8_ P

where -8 98 0 ( where 98 B 8>2 basis element

" 38B1P #P /

.B 98 B0 B

2. Recall derivation of the Fourier transform from Fourier series: We start with function 0 B on P P:

fig 2 0 B " -8 98 B " -8 /38B1P #P

_ _ 8_ 8_

Let 08 81P; let ?0 1P; let -08 -8 #1#P ?0. Then:

0 B " -8 /38B1P #P
_

" -8 #P /3B08
8_ _

8_ _

" -8 #P?0/3B08 ?0 .
8_

" #1

" " -08 /3B08 ?0 #1 8_

8_ _

" -8 #1#P?0/3B08 ?0
_

Note as P _ we have ?0 !, and

-08 -8 #1#P?0 ( ( (
P P P

.B 0 B 98 B #1#P ?0 #P?0 #1 #1 /38B1P /38B1P

.B 0 B .B 0 B

P P P

P " .B 0 B /3B08 ( #1 P

#P1P

Now (informally) take the limit P _ The interval becomes P P _ _ We have " 0 B #1 " -08 /3B08 ?0
_

8_

_ " 3B0 ( -0/ . 0 P _ #1 _

Finally, from above

P " -0 .B 0 B /3B0 #1 (P _ " .B 0 B /3B0 . ( P _ #1 _

Thus, the informal arguments give that in the limit, we can write

_ " 0 B -0 /3B0 . 0, #1 (_

where -0 (called Fourier transform of 0 ) is

_ " -0 .B 0 B /3B0 ( #1 _

(like Fourier series with sums replaced by integrals over the real line). Note: can prove that writing 0 B in the above integral form is valid for arbitrary 0 P # _ _.

3. FREQUENCY CONTENT AND GIBBS PHENOMENON Consider Fourier series of a function B 0. E.g. if 0 B lBl B
2 1 . 5 1 0 . 5 3 2 1 0 . 5 1 1 . 5 2 1 2 3

0 B which is discontinuous at

the first few partial sums of the Fourier series of 0 look like this: 5 terms: %sin B %sin $B %sin &B 1 $1 &1

2 1 . 5 1 0 . 5 3 2 1 0 . 5 1 1 . 5 2 1 2 3

fig 3 10 terms of Fourier series: %sin B %sin $B %sin &B %sin "!B 1 $1 &1 "!1
2 1 . 5 1 0 . 5 3 2 1 0 . 5 1 1 . 5 2 1 2 3

20 terms:

2 1 . 5 1 0 . 5 3 2 1 0 . 5 1 1 . 5 2 1 2 3

40 terms:
2 1 . 5 1 0 . 5 3 2 1 0 . 5 1 1 . 5 2 1 2 3

fig 3 Note that there are larger errors appearing near the singularity" Specifically: overshoot of about 9% of the jump near singularity no matter how many terms we take! In general, singularities (discontinuities in 0 B or its derivatives) cause high frequency components so that the Fourier series 0 B " -8 /38B #1
_ 8_

has large -8 for n large (bad for convergence). But notice that the singularities are in only one point, but cause all the -8 to be too large. Wavelets deal with the problem of localization of singularities, since they are localized. Advantages of Fourier series: Frequency content" displayed in sizes of the coefficients +5 and ,5 . Easy to write derivatives of 0 differential equations) in terms of series (and use to solve

Fourier series are a natural for differentiation. Equivalently, sines and cosines are eigenvectors" of the derivative operator . .B . Disadvantages: Usual Fourier transform or series not well-adapted for time-frequency analysis (i.e., if high frequencies are there, then we have large +5 and ,5 for 5 "!!. But what part of the function has the high frequencies? Where B !? Where # B $? Possible solution: Sliding Fourier transform -

fig 4
9

Thus first multiply 0 B by a window 1B 5B! , and then take a look at the Fourier series or take Fourier transform: look at (
P

.B 0 B 145 B (

.B 0 B 1B 5B! /34=B -45

Note however: the functions 145 B 1B 5B! /34=B are not orthonormal like sines and cosines; do not form a nice basis as in Fourier series; need something better. 4. The wavelet transform Try: Wavelet transform - first fix an appropriate function 2B.

Then form all possible translations by integers, and all possible stretchings" by powers of 2: 245 B #4# 2#4 B 5 (#4# is just a normalization constant)

10

fig. 5: 2#B and 2%B $ Let -45 ( .B0 B 245 B If 2 chosen properly, then can get back 0 from the -45 : 0 B " -45 245 B
45

These new functions and coefficients are easier to manage. Sometimes much better.

Advantages over windowed Fourier transform: Coefficients -45 are all real For high frequencies (4 large), the functions 245 > have good localization (they get thinner as 4 _; see above diagram). Thus short lived (i.e. of small duration in B) high frequency components can be seen from wavelet analysis, but not from windowed Fourier transform. Note 245 has width of order #4 , and is centered about 5#4 diagram earlier). (see

DISCRETE WAVELET EXPANSIONS:

11

Take a basic function 2B (the basic wavelet);

fig 6 let 245 B #4# 2#4 B 5 Form discrete wavelet coefficients: -45 ( .B 0 B 245 B 0 245 . Questions: Do the coefficients -45 characterize 0 ? Can we expand 0 in an expansion of the 245 ? What properties must 2 have for this to happen? How can we reconstruct 0 in a numerically stable way from knowing -45 ? We will show: It is possible to find a function 2 such that the functions 245 form such a perfect basis for the functions on . That is, the functions 245 are orthonormal:
12

245 24w 5w ( 245 B24w 5w B.B ! unless 4 4w and 5 5 w And any function 0 B can be represented by the functions 245 : 0 B " -45 245 B
45

So: just like Fourier series, but the 245 have better properties (e.g., they are non-zero only on a small sub-interval, i.e., compactly supported)

5.

A SIMPLE EXAMPLE: HAAR WAVELETS

Motivation: suppose we have a basic function 9B = 1 if 0 B 1 basic pixel". 0 otherwise

We wish to build all other functions out of this pixel and translates 9B 5

fig 7: 9 and its translates Linear combinations of the 9B 5: 0 B #9B $9B " #9B # %9B $

13

5 4 3 2 1 1 1 2 3 1 2 3 4 5

fig 8: a linear combination of 9B 5 [Note that any function which is constant on the integers can be written in such a form:] Given function 0 B, approximate 0 B by a linear combination of 9B 5:

fig 9: approximation of 0 B using the pixel 9B and its translates. Define Z! = all square integrable functions of the form 1B "+5 9B 5
5

= all square integrable functions which are constant on integer intervals

14

fig 10: a function in Z! To get better approximations shrink the pixel

fig 11: 9B, 9#B, and 9## B

fig 12: approximation of 0 B by translates of 9#B. Define Z" = all square integrable functions of the form
5

1B "+5 9#B 5

all square integrable functions which are constant on all halfintegers

15

fig 13: Function in Z" Define Z# = sq. int. functions 1B "+5 9## B 5
5

= sq. int. fns which are constant on quarter integer intervals

2 2

fig 14: function in Z# Generally define Z4 = all square integrable functions of the form 1B "+5 9#4 B 5
5

16

= all square integrable functions which are constant on #4 length intervals [note if 4 is negative the intervals are of length greater than 1].

17

2. Haar Wavelets, General Theory

1. The Haar wavelet Now define the desired wavelet <B " if ! B "# " if "# B " ! otherwise

2 1 . 5 1 0 . 5 1 0 . 5 0 . 5 1 1 . 5 2 0 . 5 1 1 . 5

fig 15: <B Now define family of Haar wavelets by translating:

2 1 . 5 1 0 . 5 1 0 . 5 1 1 . 5 2 1 2 3 4 5 6

fig 16 <B & <!& and stretching:

18

4 3 2 1 0 . 5 1 2 3 4 $# 0 . 5 1 1 . 5 2

fig 17: # In general:

<#$ B ( <$(

<45 #4# <#4 B 5 Show Haar wavelets are orthogonal, i.e., <45 <4w 5w ( if 4 4w or 5 5 w (i) if 4 4w , 5 5 w : <45 <4w 5w !
_

.B <45 B<4w 5w B !

because <45 ! wherever <45w ! and vice-versa. (ii) if 4 4w : <45 <4w 5w !

because:

19

integral <45 <4w 5w is !.

fig 18

2. Can any function be represented as a combination of Haar wavelets? [A general approach:] Recall: Z4 square int. functions of form "+5 9#4 B 5
5

square int. functions constant on dyadic intervals of length #4 [note if j is negative the intervals are of length greater than 1:] Z" = functions constant on intervals of length 2 Z# functions constant on intervals of length 4

20

2 2

fig. 19: function in Z4 4 # We have: (a) Z# Z" Z! Z" Z# Z$

[i.e., piecewise constant on integers piecewise constant on half-integers, etc.]

Fig. 20: Relationship of the nested spaces Z4 (b) Z8 !

8

(only ! function in all spaces)

21

[if a function is in all the spaces, then it must be constant on arbitrarily large intervals must be everywhere constant; also must be square integrable; so must be 0]. (c) Z8 is dense in L2 ()
8

[i.e. the collection of all functions of this form can approximate any function 0 B] Proof: First consider a function of the form 0 B = ;+, B. Assume that + 5#8 +" , and , j#8 ," , where +" ," "#8 .

fig 21: Relationship of + , with 5#8 and j#8 . Let 1B ;5#8 j#8 B .Z4 .
4

Then m0 1m ( .B 0 1# area under 0 1 ##8

22

fig 22: area under 0 1 Since 8 is arbitrary, 0 1 can be made arbitrarily small. Thus arbitrary char. functions 0 can be well-approximated by functions 1B -Z3
3

Now if 0 B is a step function:

4

2 2

fig 23: step function We can write 0 B "-3 ;+3 ,3 B linear combination of char. functions
3

So by above argument, step functions 0 can be approximated arbitrarily well by 1 -Z4 Now step functions are dense in P# (see R&S, problem II.2), so that -Z4 must be dense in P#
4 4

(d) 0 B Z8 0 #B Z8"
23

[because a function constant on intervals of length 28 when shrunk is constant on intervals of length #8" ] (e) 0 B Z! 0 B 5 Z!

[i.e. translating a function by an integer does not change that it is constant on integer intervals] (f) There is an orthogonal basis for the space V0 in the family of functions 9!5 9B 5 where 5 varies over the integers. This function 9 is (in this case) 9 = ;!" B. 9 is called a scaling function. Definition: A sequence of spaces Z4 together with a scaling function 9 which generates Z! so that (a) - (f) above are satisfied, is called a multiresolution analysis. 3. Some more Hilbert space theory Recall: Two subspaces Q" and Q# of vector space Z are orthogonal if every vector A" Q" is perpendicular to every vector A# Q# . Ex: Consider Z P# 1 1. Then let Q" 0 B 0 B "+8 cos 8B
_ 8!

be the set of Fourier series with cosine functions only. Let Q# 0 B 0 B ",8 sin 8B
_ 8"

be the set of Fourier series with sin functions only.

24

Then if 0" ! +8 cos 8B Q" and if 0# ! ,8 sin 5B Q# , then using

_ _ 8" 5"

usual arguments: 0" 0# "+8 ,5 cos 8B sin 5B !

_ 8"

Thus Q" is orthogonal to Q# . Recall: A vector space Z is a direct sum M" Q# of subspaces Q" Q# if every vector @ Z can be written uniquely as a sum of vectors A" Q" and A# Q# . Z is an orthogonal direct sum Q" Q# if the above holds and in addition Q" and Q# are orthogonal to each other. Ex: If Z $ , and Q" B C plane B C ! B C Q# D axis ! ! D D then every vector B C D Z can be written uniquely as a sum of B C ! Q" and ! ! D Q# , so that Z is an orthogonal direct sum Q" Q# Ex: Z P# 1 1. Then every function 0 B can be written uniquely as 0 B "+8 cos 8B ",8 sin 5B
_ _ 8! 5"

[note first sum in Q" and second in Q# ] Thus P# Q" Q# is an orthogonal direct sum. Note: not hard to show that Q" = even functions in P# Q# = odd functions in P#

25

[thus L2 is an orthogonal direct sum of even functions and odd functions] Theorem 1: If Z is a Hilbert space and if Q" Q# and Z Q" Q# , i.e., a @ Z b 73 Q3 s.t. @ 7" 7# , then Z Q" Q# is an orthogonal direct sum of Q" and Q# Pf: In exercises. Note: no assumption of uniqueness of @3 necessary above. Def: If Z [" [# is an orthogonal direct sum, we also write [" Z [ # Recall: Given a subspace Q Z , Q vectors which are perpendicular to everything in Q @ Z @ A a A [ Ex: If Z $ and [ B-C plane, then [ D -axis Ex: If Z P# then if [ even functions, [ odd functions. Pf. exercise Recall (R&S, Theorem II.3): Given a complete inner product space Z and a complete subspace Q , then Z is an orthogonal direct sum of Q and Q [# Z [ "

4. Back to wavelets: Recall: Z4 functions constant on dyadic intervals5#4 5 "#4 Z# Z" Z! Z" Z# Since Z! Z" , there is a subspace [! Z" Z! such that Z! [! Z"

26

fig 24: Relationship of Z! and [! Z! as the B-C plane and [! as the D axis ... Z! [! Z" $ Similarly define [" Z # Z " Generally: [4" Z4 Z4" Then relationships are: Z# Z" Z! Z" Z# [# [" [! [" Also note, say, for Z$ : Z$ Z # [ # Z" [ " [ # Z! [ ! [ " [ # Z" [" [! [" [# Thus if @$ Z$ , we have:
27

@$ @# A# @" A " A # @! A! A" A# @" A" A! A" A# with @3 Z3 and A3 [3 [successively decomposing the @ into another @ and a A]. In general @$ @8 " A5
# 58

(1)

Now let 8 _ . Since all vectors in above sum orthogonal, we have (see exercises): m@$ m m@8 m " mA5 m#
# # # 58

Thus " mA5 m# m@$ m#

#

58

a 8, so ! mA5 m# _
#

5_

Lemma: In a Hilbert space L , if A5 are orthogonal vectors and the sum ! mA5 m# _, then the sum ! A5 converges.
5 5

28

Pf:

We can show that the sum ! A5 forms a Cauchy sequence by noting

R 5"

if R Q : ! m"A5 "A5 m# m" A5 m# " mA5 m# R Q _

R Q R R 5" 5" 5Q " 5Q "

Thus we have a Cauchy sequence. The sequence must converge (L is _ complete), and so ! A5 exists.
5"

From above: @8 @$ " A5

# 58 #

Letting 8 _ get

! A5 . Thus vectors @8 @8 8 _ @$
5_

have

limit as 8 _ @8 @_ But notice @8 Z8 Z8" Z8# @8 Z8 Z8" Z8# , Z5

_ 58

Thus @_ intersection of all Z8 's @_ ! (by condition (b) on spaces Z4 ). Thus taking the limit as 8 _ in (1) @$ @8 " A5
# 58

(1)

get

29

@$ " A5
# 5_

So by definition of direct sum: Z$ [# [" [! [" [# : [5

# 5_

i.e., every vector in Z$ can be uniquely expressed as a sum of vectors in the [4 . Further this is an orthogonal direct sum since [4 's orthogonal Generally: Z8 [# [" [! [" [# [8" Now note P# Z$ Z$ Z% Z% Z$ [$ Z% Thus comparing above get Z$ [$ Z% . Similarly Z% [% Z& So Z$ [$ [% Z& . Generally: : [5
8" 5_

30

 Z$ [$ [% [& [8 Z8" . Letting 8 _ and using same arguments, we see that the Z8" components go to ! as 8 _ , so that

Z$ [$ [% [& Thus: P# Z$ Z$ [# [" [! [" [# [Thus every function in P# can be uniquely written as a sum of functions in the [4 's]. Thus: Theorem: Every vector @ P# _ _ can be uniquely expressed as a sum " A4 where A4 [4
_

4_

Conclusion - relationship of Z4 and [4 :

5. What are the [4 spaces? Consider [! .

31

Claim: [! E functions which are constant on half-integers and take equal and opposite values on half of each integer interval
4 3

2 1

fig 24: Typical function in E Proof: Will show that with above definition of E, Z! E Z " , and that Z! and E are orthogonal. Then it will follow that E Z" Z ! [ ! , First to show Z! and E are orthogonal: let 0 Z! and 1 E Then 0 looks like:

fig 24: 0 B Z! 1B [! Thus 0 1 (

_

0 B1B.B (

"

"

"

"

0 B1B.B

32

since 0 B1B takes on equal and opposite values on each half of every integer interval above, and so integrates to ! on each interval. Thus 0 and 1 orthogonal, and so Z! and E are orthogonal. Next will show that if 0 Z" , then 0 0! 1! , where 0! Z! and 1! E (which is all that's left to show). Let 0 Z" . Then 0 is constant on half integer intervals:
8 6 4 2 4 2 2 4 6 8 2 4

fig 25: 0 B Z" Define 0! to be the function which is constant on each integer interval, and whose value is the average of the two values of 0 B on that interval:

fig 26: 0! B as related to 0 B. Then clearly 0! B is constant on integer intervals, and so is in Z! . Now define 1! B 0 B 0! B

33

8 6 4 2 4 2 2 4 6 8 2 4

fig 27: 1! B 0 B 0! B Then clearly 1! takes on equal and opposite values on each half of every integer interval, and so is in E. Thus we have: for 0 B Z" 0 B 0! B 1! B, where 0! Z! and 1! E. Thus Z" Z! E by Theorem 1 above. Thus E Z" Z! [! , so E [! . Thus [! functions which take on equal and opposite values on each half of an integer interval, as desired. Similarly, can show: [4 = functions which take on equal and opposite values on each half of the dyadic interval of length #4 and are square integrable:
8

2 2

fig 28: typical function in [4 4 "

34

6 What is a basis for the space [4 ? Consider [! = functions which take equal and opposite values on each integer interval What is a basis for this space? Let <B " " if ! B "# . if "# B "

Claim a basis for [! is <B 5_ 5_ . Note linear combinations of <B 5 look like: 1B #<B $<B " #<B # #<B $.
4

2 1

fig 29: graph of 1B. I.E., linear combinations of translates <B 5 functions equal and opposite on each half of every integer interval. Can easily conclude: [! functions in P# equal and opposite on integer intervals functions in P# which are linear combinations of translates <B 5.

35

Also easily seen translates <B 5 are orthonormal. Conclude: <B 5 form orthonormal basis for [! . Similarly can show #"# <#B 55 form orthonormal basis for [" . ### <## B 55 form orthonormal basis for [# . Generally, #4# <#4 B 5_ 5_ form orthonormal basis for [4 . Define <45 B #4# <#4 B 5. Recall every function 0 P# can be written 0 "A 4
4

where A4 [4 . But each A4 can be written

5

A4 "+5 <45 B

[note 4 fixed above replace +5 by +45 since need to keep track of 4]. so: 0 ""+45 <45 B.
4 5

Furthermore we have shown the <45 orthonormal. Conclude they form orthonormal basis for P# .

7. Example of a wavelet expansion: Let 0 B B# ! if ! B " . Find wavelet expansion. otherwise

36

8. Some more Fourier analysis: Recall Fourier transform (use = instead of 0 for Fourier variable): 0 B
_ " s 0 =/3B= . = ( #1 _ _ " .B 0 B /3B= . ( #1 _

s 0 =

[earlier had s 0 = -= ] Write s 0 = Fourier transform of s 0 = Y 0 9. Plancherel theorem: Plancharel Theorem: (i) The Fourier transform is a one to one correspondence from P# to itself. That is, for every function 0 B P# there is a unique P# function which is its Fourier transform, and for every function s 1= P# there is a unique P# function which it is the Fourier transform of. (ii) The Fourier transform preserves inner products, i.e., if s 0 is the FT of 0 s = s and s 1 is the FT of 1, then 0 1= 0 B 1B. (iii) Thus s =m# m0 Bm# m0 Now for a function 0 P# 1 1, consider the Fourier series of 0 , given by " -5 /35B
_

5_

37

The above theorem has analog on 1 1. Theorem below follows immediately from fact that /38B #1_ 8_ form orthonormal basis for # P 1 1 Plancharel Theorem for Fourier series: (i) The correspondence between functions 0 P# 1 1 and the coefficients -5 of their Fourier series is a one to one correspondence, if # we restrict !-5 _. That is, for every 0 P# 1 1 there is a unique series of square summable Fourier coefficients -5 of 0 such that !l-5 l# _ Conversely for every square summable sequence -5 there is a unique function 0 P# 1 1 such that -5 are the coefficients of the Fourier series of 0 . (ii) Furthermore, ! m-5 #
5 " # #1 m0 Bm 5 5

38

3. General Wavelet Constructions

1. Other constructions: Suppose we use another pixel function 9B:
2 1 . 5 1 0 . 5

1 0 . 5

fig 30: another pixel function Can we use this to build approximations to other functions? Consider linear combination: #9B $9B " #9B # 9B $
4 3 2 1 1 1 2 3 1 2 3 4 5

fig 31: graph of linear combination of translates of 9 Note we can try to approximate functions with other pixel functions. Question: Can we repeat the above process with this pixel (scaling) function? What would be the corresponding wavelet?
39

Assumptions: l9Bl has finite integral and ' 9B.B !. More general construction: As before define Z! all P# linear combinations of 9 and its translates: 0 B) " +5 9!5 B +5 0 P#
5

(2)

with 9!5 B 9B 5 and Z" 0 B " +5 9"5 Bl +5 0 P#

5

(3)

9"5 B #"# 9#B 5 etc. We want the same theory as earlier. [Note Z! no longer piecewise constant functions] Recall condition (d) 0 B Z8 0 #B Z8" This is automatically true by definition of Z8 , since if 0 B Z! , then 0 has the form of an element of (2). Then 0 #B has form of an element of (3), and 0 #B Z" . Similarly can be shown that (d) holds for any pair of spaces Z8 and Z8" of above form. 2 Some basic properties of F.T.:

40

Assume that s 0 Y 0 Then (a) Y 0 B -= /3=- s 0 = "s (b) Y 0 -B - 0 =- Proofs: Exercises. 3. Orthogonality of the 9's: Another property of Z4 (f) The basis 9B 5 for Z! is orthogonal, i.e. 9B 5 9B j ! for 5 j Not automatic. Let Y 0 F.T. of 0 s 0 =. Require a condition on 9 of the following sort: if 5 j, then (note use = as Fourier variable) ! 9B 5 9B j Y 9B 5 Y 9B j Thus conclude if 7 ! !( (
#1 _

s = /3=j 9 s= /3=5 9 (
_

s =l# . = /3=5j l9

s =l# . = /37= |9 (
#1

_ !1

%1 _

#1

!1

s=l# . = /37= l9

" (
8_

8"#1

s =l# . = /37= l9

8#1

41

" (
_ 8_

#1

s = #81l# . = /37= l9
#1

37=

8_

s = #81l# . = " l9
_

8_

[since we can show that the integral of the absolute sum converges because _ s = #81l# . = absolutely integrable; see exercises] ! l9 s = #81l# on ! #1 is in P# because it has Conclude function ! l9
_ 8_

square summable Fourier coefficients (in fact they are ! if 7 !). Further s = #8 1 l# is #1- periodic in =, and has a Fourier series ! l9
_

8_

8_

s = #8 1 l# " -7 /37= " l9

_ _ 7_

where -7 And
_ _ " " 37= s # s=l# . = -! l9=l . = ( / ( l9 #1 _ #1 _ 7! _ #1 " 37= " s l9= #81l# . = ! ( / #1 ! 8_

if 7 !

_ " " |9 Bl# .B ( #1 _ #1

Thus
_ " # s " l9= #81l " -7 /37B # 1 8_ 7_ _

42

This condition equivalent to orthonormality of 9B 5. V0 V1 Recall the condition (a) Z! Z" What must be true of 9 for this to hold in general? This says that every function in Z! is in Z" . Thus since 9B Z! , it follows 9B Z" , i.e. 9B linear combination of translates of # 9#B = " 25 9"5 B
5

(4)

[recall normalization constant # is so we have unit P# norm] Ex: If 9B Haar wavelet, then 9B 9#B 9#B " " " 9"! B 9 B # # ""

9"5 B #"# 9#B 5

2"! 9"! B 2"" 9"" B

43

2 . 5 2 1 . 5 1 0 . 5

fig 32: 9B 9#B 9#B " Thus in this case all 2's are ! except 2"! and 2"" ; 2"!
" #

2""

" #

Note in general that since this is an orthonormal expansion,

# " 25 m9Bm# _ 5

4. What must be true of the scaling function for (1) above to hold? Thus in general we have:

9B " 25 9"5 B lim

_ 5_

R _

5R

" 25 9"5 B
R

(3)

in P# norm. Denote " 25 9"5 B JR B

R

5R

44

Specifically, m9B " 25 9"5 Bm !

R 5R

[recall Y is Fourier transform] Corollary of Plancherel Theorem: Corollary: The Fourier transform is a bounded linear transformation. In particular, if the sequence of functions JR B converges in P# norm, then Y lim JR = lim Y JR =
8_ R _

in P# norm, i.e., Fourier transforms commute with limits. Thus since _ sums are limits and Y is linear: Y " 25 9"5 B " 25 Y 9"5 =
_ _ 5_

O_

[i.e., Y commutes with _ sums] s=. Then generally: Let Y 9= 9 Y 945 = Y #4# 9#4 B 5= #4# Y 9#4 B 5= [recall dilation properties of Fourier transform earlier] #4#
" #4

Y 9B 5=#4

[recall translation by 5 pulls out an /3=5 ] #4# /3=5# Y 9B=#4

4 s =#4 #4# /3=5# 9 4

45

Specifically for 4 ":

Y 9"5 = # /3=5#

"s 9=# #

Recall (3): 9B " 25 9"5 B

_ 5_

Fourier transforming both sides: s = Y 9B 9 Y

_

" 25
5_

5_

" 25 9"5 B
_

(5)

" 35=# s / 9=# #

Define 7=# " 25

_ 5_

" 35=# / #

(6)

note 7 is #1- periodic Fourier series of 7=# given above.

# Note 7= P# ! #1, since ! 25 _. 5

Thus by (5): s = 7=# 9 s=# 9 with 7 a #1-periodic P# function. [Note: This condition exactly summarizes our original demand that Z! Z" !] Note if Z! Z" , then it follows (same arguments) that Z" Z# , and Z4 Z4" in general.

46

5. Some preliminaries: Given a Hilbert space L and a closed subspace Z , for 0 L write 0 @ @ where @ Z and @ Z Definition: The operator T defined by T 0 T @ @ @ is the orthogonal projection onto Z . Note T is a bounded linear operator (see exercises). Easy to check that mT m " if T ! (see exercises). Ex: Z $ T B C D B C ! is the orthogonal projection onto the BC plane. T B C D ! ! D orthogonal projection onto D axis.

Ex: Z P# 1 1 is the even functions. Then for 0 P# T 0 B 0even B 0 B 0 B #

47

(see exercises). 6. How to construct the wavelet? Recall we have now given conditions on the scaling function: Condition (a) Z# Z" Z! Z" Z# Z$

is equivalent to: (i) s = 7! =#9 s=# 9

where 7! is a function of period #1. Condition (f) There is an orthogonal basis for the space V0 in the family of functions 9!5 9B 5 is equivalent to: s = #15l# " l9
5

(ii) Condition (b) Z8 !

8

" #1

can also be shown to follow from (ii) as follows: Proposition: If 9 P# and satisfies (ii), then Z4 !
4

Proof: Denote G- to be compactly supported continuous functions. Let 0 Z4 Let % ! be arbitrarily small. By arguments as in problem
4

G- with II.2 in R&S, G- is dense in P# , so that there exists an 0 m % m0 0

48

with m m denoting P# norm. Let T4 orthogonal projection onto Z4 Then since 0 Z4 : m mT4 0 T4 0 m mT4 0 0 m m0 0 m % m0 T4 0 Thus by triangle inequality m mT4 0 m % mT4 0 m m0 m m0 T4 0 Z4 we have Since T4 0 "-45 945 B T4 0
5

(7)

where -45 945 0 (recall 945 B_ 5_ is an orthonormal basis for Z4 ). Thus if m0 m_ supl0 Bl
B

m# "l-45 l# "l945 0 l# mT4 0

5 5

B .B "( 945 B 0
5

is supported in V V ] [assuming 0 #
4

m# " ( m0 " _ VV 5

l9# B 5l.B
4

[using Schwartz inequality +B,B m+Bmm,Bm] m # "( #4 m0 _

5

VV

"# .B (

l9#4 B 5l# .B

VV

m# #V " ( #4 m0 _
5

l9#4 B 5l# .B

VV

49

C#4 B5

m# #V ( m0 _

l9Cl# .C

WV4

[where WV4 5 5 #4 V 5 #4 V (note we replaced 5 5 in the union) assuming 4 large and negative, so #4 V " # Note that then the 5 sum becomes a sum over disjoint intervals after the change of variables above, and we therefore replace a sum over 5 by a union over these intervals, as above]
# m0 m# ! _ #V ( ;WV4 Cl9Cl .C 4 _

by the dominated convergence theorem, since if C , ;WV4 C 4 ! _ Thus by (7), we have for 4 large and negative and all % ! m mT4 0 m % mT4 0 m 2% . m0 m m0 T4 0 Thus m0 m ! and 0 ! Condition (c) Z8 is dense in L2 ()
8

also follows from (ii): Proposition: If 9 P# and satisfies (ii), then Z4 P#

4

Proof: Similarly technical proof. Condition (d) 0 B Z8 0 #B Z8" is automatic from the definition of the Z8 . Condition (e) 0 B Z! 0 B 5 Z!

50

is also automatic from definition. Thus we conclude: Theorem: Conditions (i) and (ii) above are necessary and sufficient for the spaces Z4 and scaling function 9 to form a multiresolution analysis. Thus if (i), (ii) are satisfied for 9 and we define the spaces Z4 as usual, the spaces will satisfy properties (a) - (f) of a multiresolution analysis.

Recall: orthonormality of translates 9B 55 is equivalent to: (ii) Rewrite (ii): s=# 15l# ! l7! =# 15l# l9
5 " #1

s = #15l# " " l9 #1 5

" #1

s= w 15l# ! l7! = w 15l# l9

5

[=w =#]

s= w 15l# =! l7! = w 15l# l9

5 even 5 odd

s= w 15l# ! l7! = w 15l# l9

s= w 1 #5l# !l7! = w 1 #5l# l9

5 5

7! periodic

s= w #15l# l7! = w 1l# "l9 s= w 1 #15l# l7! = w l# "l9

5 5

s= w 1#5 "l# ! l7! = w 1#5 "l# l9

by (ii)

l7! = w l#

" #1

l7! = w 1l#

" #1 .

This implies that

51

l7! = w l# l7! = w 1l# "

(8)

What about wavelets? Recall we define [4 Z4" Z4 . We now know that 945 B form basis for Z4 . The wavelets <45 will form basis for [4 .

52

4. More on General Constructions

1. What are <45 ? [Recall norms and inner products of functions are preserved when we take Fourier transform. Let's take FT to see.] Note if we find [! Z" Z! , then we will be done. [Let's look at Fourier transforms of functions in these spaces:] Note that if 0 Z! then 0 B " +5 9B 5 " +5 9!5 B
5 5

(9)

gives by F.T.:

s s= 70 =9 s= 0 = " +5 Y (9!5 B) "+5 /35= 9

5 5

(10)

where

70 = "+5 /35=
5

is a 21 periodic P# ! #1 function which depends on 0 . In fact reversing argument shows (9) and (10) are equivalent. Similarly can show under Fourier transform that 1 Z" equivalent to: s=# 1= 71 =# 9 s with 71 some other #1 periodic function on P# ! #1 Notice functions 70 and 71 both have period #1 (look at their Fourier series). Also note above steps are reversible, so equation (10) implies (9) by reverse argument. Thus: (11)

53

s=# 0 Z" s 0 70 =# 9 Recall: we want to characterize 0 [! ; such an 0 has the property that 0 Z" and 0 Z! . Now note: s !5 0 Z! 0 9!5 a 5 s 0 9 ( !(
_ _

s s = . = ! 0 = /3=5 9
#17"

s s= . = "( 0 = /3=5 9
7 #1

s s= 0 = /3=5 9

#1 7

"(
7

s s= #17 . = 0 = #17/35=#17 9

#1

s= #17 . = /35= " s 0 = #17 9

7

where above identities hold for all 5 . Hence [viewing sum as some function of =] s= #17 ! "s 0 = #17 9
7

Thus: s= #17 ! !s 0 = #17 9

7

s= #17#7! = #17#9 s= #17# !70 = #17#9

7

s=/# 7) 7! =# 179 s=# 17 !70 =# 179

7

54

s=# 17 ! + ! 70 =# 17 9
7 even 7 odd

s=# 17 7! =# 179 s=# #177! =# #179 s=# #17 ! 70 =# #179

7

s=/#+1+#17 ! 70 =# 1 #179
7

s=# 1 #17 7! =# 1 #179 s=# #17 9 s=# #17 70 =#7! =#! 9

7

s=# #17l# 70 =#7! =#! l9

7 7 7

s=# 1 #179 s=/# 1 #17 70 =/# 17! =# 1! 9

s=# 1 #17l# 70 =/# 17! =# 1! l9

" #1

70 =#7! =# (3)

70 =# 17! =# 1

" #1

70 = w 7! = w 70 = w 17! = w 1 !

Thus (note 7! = w and 7! = w 1 cannot vanish together) let = w = 70 = 1 7! =

70 = where

7! = 1 -= 7! = 1

(12)

- = and so -= is #1 periodic. Also,

70 = 1 7! =

55

- = - = 1
combining fractions and using $

70 = 1 7! =

70 = # 1 7! = 1

(13)

!.

Define / #= -= /3= Then / #= #1 -= 1 /3=1 -=/3= /31 -=/3= / #= so / has period #1. s=# -=#7! =# 1 9 s=# Thus s 0 = 70 =#9 s=# / = /3=# 7! =# 1 9 Thus we define the wavelet <B by its Fourier transform: s = /3=# 7! =# 1 9 s=# < Thus s s= 0 = / =< Going back in Fourier transform, we would get (compare with how we got s s=) 0 = 70 =9 0 B " +5 <B 5
5

(14)

(15)

where +5 are coefficients of the Fourier series of / = i.e., / = "+5 /35=

5

To justify process of Fourier transformation as above, need to also show that the coefficients +5 are square summable (i.e. !l+5 l# _), since we do not
5

56

know whether Fourier transform properties which we have used in getting (15) are valid otherwise. Note since +5 are coefficients of Fourier series of / , we just need to show / is square integrable on ! #1 (recall this is equivalent to the +5 being square summable). To show that / is square integrable, note that with 70 as in (0): _
use 70 P# !#1

'!#1 . = l70 =l#

by (12)

'! '1 . = l-=l# l7! = 1l#

1 #1

'!#1 . = l-=l# l7! = 1l#

[substitute = w = 1 in second integral; then rename = w = again] (

1

. = l-=l l7! = 1l (
# #

. = l-= 1l# l7! = #1l#

[recall that by periodicity l7! = #1l# l7! =l# and use (13)] =(
1

. = l-=l# l7! = 1l# l7! =l# ( (

1

use (8)

. = l-=l#

! 1

. = l/ #=l#

= w # = " #1 # ( . = l/ =l # !
#1 Thus we have that _ '! . = l/ =l# so that / is square integrable, as desired.

57

This was only thing left to show <#B 5 span [! . Wish to show also orthonormal. Use almost exactly the same argument as was used to show the same for 9B 5:
(14) s = #15l# use s=# 15l# "l< "l7! =# 15 1l# l9

[now break up the sum into even and odd 5 again and use the same method as before] s=# 15l# ! + ! |7! =# 15 1l# l9
5 even 5 odd

s=# 1 #5l# ! l7! =# 1 #5 1l# l9

5

s=# 1 #5 "l# !l7! =# 1 #5 " 1l# l9 s=# 1 #5l# l7! =# 1l# !l9 s=# 1 #5 "l# l7! =#l# !l9
5 using (ii) above again 5 5

(16)

l7! =# 1l# l7! =#l#

" #1

" #1

By same arguments as used for 9B 5, it follows by (16) <B 5 orthonormal This proves our choice of < gives a basis for [! as desired. Specifically, <!5 B <B 5 form an orthogonal basis for [! (in fact can show their length is " so they are orthonormal).

58

In same way as for 9, can show immediately that since functions in [4 are functions in [! stretched by factor #4 , the functions <45 B #4# <#4 B 5 form a basis for [4 (4 fixed 5 varies). Since P# direct sum of the [4 spaces, conclude functions # <45 B_ 45_ over all integers 4 and 5 form orthonormal basis for P . Conclusion: If we start with a pixel function 9B, which satisfies s (=) 7! =#9 s =# (with 7! some #1-periodic function) (i) 9 (ii) !l9= #15l#
5 " #1

then the set of spaces Z4 form a multiresolution analysis, i.e., satisfy properties (a) - (f) from earlier. Further, if define function <B with Fourier transform: s = /3=# 7! =# 1 9 s=# < (here 7! is from (i) above), then <45 B #4# < #4 B 5. form orthonormal basis for P# Next we'll construct some wavelets (17)

2. Additional remarks: Note further that transform (17) has another interpretation without Fourier

Recall the two scale equation:

59

9B "25 9"5 B
5

Also then we have (see eq. (5)) that if 7! = "

5

25 35= / #

then: s = 7! =#9 s=#. 9 Then we have from 17:

_ 3=/# " " s s (=/2) < (= ) = / 25 /35=#1 9 # 5_

_ 3=/# "

"
_

" s (=/2) 25 /351 /35=# 9 # 5_

" s (=/2) 25 "5 /35+"=# 9 # 5_

Inverse Fourier transforming: s = <B Y " < _ " s (=/2) " 25 "5 Y " /35"=# 9 # 5_ "
_

"
_ _

" 25" "5" 9"5 B " 15 9"5 B

5_ 5_ _

25" "5" ##9#B 5 # 5_

25 "5 #9#B 5 " # 5_

60

where 15 2"5 "5" 2"5 "5" and (recall) 25 defined by

standard form

2 "5 "5"

9B "25 9"5 B
5

3. Some comments on the scaling function: Recall s = 7! =# 9 s=# 9 from earlier. This stated that the Fourier transform of 9 and its stretched version are related by some function 7! =#, where 7! is a periodic function of period #1.

Lemma: The Fourier transform of an integrable function is continuous. Proof: exercise Assumption: 9B (the scaling function) is integrable (i.e., its absolute value has a finite integral).
_ Fact: Under our assumptions, it can be shown that '_ .B 9B " [proof is an exercise]

Consequence: A consequence of the above assumption is that the Fourier s = satisfies: transform 9 s ! 9
_ _ " " " 3!B .B 9 B / .B 9B ( ( #1 _ #1 _ #1

Now recall we had s = 7! =#9 s=# 9 for some periodic function 7! . Replacing = by =# above: (18)

61

s =# 7! =%9 s=% 9 Plugging into (18): s = 7! =#7! =%9 s=% 9 Now taking (18) and replacing = by =%, and then plugging into (19): s = 7! =#7! =%7! =)9 s=) 9 Continuing this way 8 times, we get: s = 7! =#7! =%7! =) 7! =#8 9 s=#8 9 or:
4 8 s = s 9 $ 7! =# 9=# 8 4"

(19)

s = 9 s =#8 9 $ 7! =#4
8 4"

(20)

s is continuous (above Now let 8 _ on both sides of equation. Since 9 assumption), we get s =#8 9 s 9 8 _ ! " #1

Since the left side of (20) converges as 8 _, the right side also converges. After letting 8 _ on both sides of (20):
_ s = 9 $ 7! =#4 s ! 9 4"

62

s = 9

" _ $7! =#4 #1 4"

Conclusion: If we can find 7! =), we can find the scaling function 9.

4. Examples of wavelet constructions using this technique: Haar wavelets: Recall that we chose the scaling function 9B and then we defined spaces Z4 From 9 we constructed the wavelet < whose translates and dilates form a basis for P# . Such constructions can be made automatic if we use above observations. Note first in Haar case: s = 9
" " " /3=B " " /3= " 3=B / .B ( #1 ! #1 3= ! #1 3= 3=

" !

if ! B " , otherwise

#1 = #

3=#

/3=# /3=# #3 #3

#1 = #

/3=# sin =#

For Haar wavelets we can find 7! = from: s = 7! =#9 s=# 9 so

63

7! =#

s = sin =# 9 " /3=% s =# sin =% # 9

" 3=% sin # =% / sin =% #

" 3=% # sin =% cos =% / sin =% # " 3=% / # cos =% # /3=% cos =%

Recall wavelet Fourier transform is: (4) s = /3=# 7! =# 1 9 s=# <

In this case s = /3=# /3=%1# cos =% 1# < [using cos =% + 1# cos =% cos 1# sin =% sin 1# sin =%] #1 = %3 /3=# sin2 =% #1 = % /3=% sin =%

Can check (below) this indeed is Fourier transform of usual Haar wavelet <, except the complex conjugate (which means the original wavelet is reflected about !, i.e., translated and negated, which still yields a basis for [! ). To check this, recall Haar wavelet: <B " " if ! B "# if "# B "

64

Thus:
_ " s = < <B /3=B .B #1 (_ "# " " ( <B /3=B .B #1 (! "# "# " " " 3=B / .B /3=B .B #1 (! #1 ("#

" /3=# " " /3= /3=# #1 #1 3= 3= 3= 3= #/3=# /3= " #1 3= #1 3= #1 3= # # /

3=#

3=#

/3=# /3=# #

#1 3= #

3=#

/3=# cos =#

#1 3= #

/3=# /3=# cos # =% /3=# /3=# " # sin# =%

[using cos #B " # sin2 B] #1 3=

% /3=# sin# =% #1 3=

65

#1 = %3

/3=# sin# =%

66

5. Constructing Wavelets
1. Meyer wavelets: another example Scaling function: " if l=l #1$ " s = if #1$ l=l %1$ 9 / 2$ cos 1 # 1 l =l " #1 ! otherwise [error in Daubechies $%1 instead of $#1 inside / ] where / is any infinitely differentiable non-negative function satisfying if B ! / B " if B " smooth transition in / from ! to " as B goes from ! to " and / B / " B ". !

fig 33: / B and / " B

67

s = of the Meyer scaling function fig 34: Fourier transform 9 Need to verify necessary properties for a scaling function: (i) s = #15l# "l9
5

" #1

21

To see this, consider the two possible ranges of values of =: (a) l= #15" l #1$ for some 5" In that case (see diagram above): s = # 1 5" 9 " #1 s= #15 ! if 5 5" 9

since if l= #15" l #1$, then l= #15l %1$ for 5 5" . Thus (21) holds because there is only one non-zero term in that sum. (b) #1$ = #15" %1$ for some 5" . In this case we also have %1$ = #1(5" " #1$ Also, for all values 5 5" or 5" ", can calculate that #15 %1$ %1$ so s = #15 ! 9 So sum has only two non-zero terms:

68

s = #15l# #1l9 s= #15" l# l9 s= #15" "l# #1 "l9 $ $ 1 1 cos# / l= #15" l " cos# / l= #15" "l " # #1 # #1 1 $ $ 1 cos# / = $5" " cos# / aa= #15" "bb " # #1 # #1 1 $ 1 $ cos# / = $5" " cos# / = $5" # # #1 # #1 1 $ 1 $ cos# / = $5" " cos# " / " = $5" # # #1 # #1 1 1 1 $ $ cos# / = $5" " cos# / = $5" " # #1 # # #1 1 $ 1 $ cos# / = $5" " sin# / = $5" " # #1 # #1 " Note that above l= #15" "l a= #15" "b, since quantity in parentheses always negative for our range of =. In next to last equality have used cos 1 # B sin B
5

Note since cases (a), (b) cover all possibilities for = (since they cover a range of size #1 for = #15" ), we are finished proving (21). Also need to verify: (ii) s = 7! =#9 s=# 9 for some #1-periodic 7! =#. Indeed, looking at pictures:

69

s = and 9 s =# (----) fig 35: 9 ratio of these two looks like:

fig. 36:

s =9 s =# #1 9 s = in #1 #1, we can define Note since ratio 9 7! =# if = #1 #1 Definition ambiguous when numerator and denominator are 0. Definition also ambiguous for = #1, #1 since numerator and denominator both !. So define 7! =# by periodic extension of above for all real =. s = to make it How to do that? Just add all possible translates of the bump 9 41-periodic: s = 9 s= #1 9 s 9=# 22

s =# #1 9 s = in the interval #1 #1 9s =9

70

s = %15 7! =# #1 " 9
5

Check:

s =# #1 "9 s= %159 s=# 7! =#9

5

s =9 s =# #1 9 s = 9 s (= %15 has no overlap with 9 s (=# if where we have used the fact that 9 5 ! [So we expect a full MRA.] 2. Construction of the Meyer wavelet Standard construction: s = /3=# 7! =# 1 9 s=# < s = #1#5 "9 s=# /3=# "9
5

s = #1 9 s= #19 s=# /3=# 9 [supports of 2d and 3d factors do not overlap for other values of 5 ; note s9 s since 9 s is real] 9

71

s = # 1 9 s = #1 and 9 s=# (dashed) fig 37: 9

s = # 1 9 s = #19 s=# fig 38: 9 Thus have 2 distinct regions: (a) For #1$ = %1$ we see in diagram that

72

s = #1 9 s = #1 9 s= #19 s=# /3=# < s = # 1 9 " 1 $ cos / l= #1l " #1 # #1 " 1 $ cos / = #1 " #1 # #1 " $ 1 cos / = # #1 # #1 " 1 $ cos " / " = # #1 # #1 " 1 $ cos " / = " #1 # #1 " 1 $ sin / = " #1 # #1 So by symmetry same is true in #1$ = %1$, so replace = by l=l above to get: s = /3=# < " 1 $ sin / l=l " for #1 # #1 we see #1$ l=l %1$

(b) For %1$ = )1$, #1$ =# %1$:

from

diagram

(note

s = #1 9 s = #1 9 s= #19 s=# /3=# < s =# 9 " 1 $ cos / =# " #1 # #1 " 1 $ cos / = " #1 # %1

73

Again by symmetry same is true in )1$ = %1$, so replace = by l=l: s = /3=# < Thus: " 1 $ cos / l=l " for #1 # %1 %1$ l=l )1$

$ /3=# sin 1 # / #1 l=l " , if #1$ l=l %1$ " $ s = < 1 3=# #1 / cos # / %1 l=l " , if %1$ l=l )1$ ! otherwise

s =l Fig. 39: The wavelet Fourier transform l<

Fig. 40: The Meyer wavelet <B

3. Properties of the Meyer wavelet

74

Note: If / is chosen as above and has all derivatives ! at 1#, can check s = is: that < infinitely differentiable (since it is a composition of infinitely differentiable functions, and one can check that all derivatives are ! from both sides at the break For example, the derivatives coming in from the left 1 at = #$ are: .8 s ! < = . =8 #1 =
$

and similarly .8 s + ! < = . =8 #1 =

$

(proof in exercises). supported (non-zero) on a finite interval Lemma: (a) If a function <B has 8 derivatives which are integrable, then the Fourier transform satisfies s =l O" l=l8 l< Conversely, if (23) holds, then <B has at least 8 # derivatives. s = has 8 integrable derivatives, then (b) Equivalently, if < l<Bl O" lBl8 s = has at least 8 # derivatives. Conversely, if (24) holds, then < Proof: in exercises. Thus: <B Decays at _ faster than any inverse power of B 24 23

75

 Is infinitely differentiable Claim: <45 B #4# <#4 B 5 form an orthonormal basis for P# . Check (only to verify above results - we already know this to be true from our theory): ( Pf: (
_ _

l<Bl .B (
#

s=l# . = " l<

s =l# . = l<

" $ # 1 . = sin / l=l " ( #1 #$1 |=| %$1 # #1 ( $ 1 . = cos# / l=l " %1 )1 # %1 $ |=| $

[getting rid of the | | and doubling; changing vars. in second integral] " $ # 1 . = sin / = " ( 1 #$1 = %$1 # #1 #( 1 $ . = cos# / = " 21 41 # #1 $ = $

" $ $ # 1 # 1 . = sin / = " # cos / = " ( 1 #$1 = %$1 # #1 # #1 " $ # 1 . = " cos / = " ( 1 #$1 = %$1 # #1

76

[letting =

$ #1 =

" = #1$= "]

" # 1 ( .=" cos# / = $ ! #

"# " # 1 # 1 ( .=" cos / = ( .=" cos# / = $ ! # # "# "# "# # 1 # 1 ( .=" cos / = ( .=" cos# / = "# $ ! # # !

[using / (= "# " / "# =]

"# "# # 1 # 1 ( .=" cos / = ( .=" cos# " / "# = $ ! # # !

"# "# # # 1 # 1 .= " cos / = .= " sin / "# = ( ( $ ! # # ! "# "# # # 1 # 1 .= " cos / = .= " sin / = ( ( $ ! # # ! "# # .=# " " ( $ !

="#=

To show in another way that they form an orthonormal basis, sufficient to show that for arbitrary 0 P# , " l<45 0 l (
_ # 45 _

l0 Bl# .B

[this is a basic analytic theorem].

77

Now note: " l<45 0 l "( .B <45 B0 B.B

_ _ # 45 45 #

s 45 = " ( . = s 0 =<
_ # 45

Note if <45 B #4# <#4 B 5 Then as usual: s 45 = #4# < s #4 = /3#4 5= < Plug this in above and can do calculation to show (we won't do the calculation): " l0 <45 l# (
_ 45 _

.B l0 Bl#

as desired. CONCLUSION: The wavelets <45 B #4# <(24 B 5 form an orthonormal basis for the square integrable functions on the real line.

4. Daubechies wavelets: Recall that one way we have defined wavelets is by starting with the scaling s B. Recall it satisfies: (pixel) function 9 s = 7! =#9 s =# 9

78

for all =, where 7! = is some periodic function. If we use 7! as the starting point, recall we can write s = 9 " _ $7 =#4 #1 4" ! (25)

Recall 7! is periodic, and so has Fourier series:

7! = " +5 /35= .
5

If 7! satisfies l7! =l# l7! = 1l# ", construction of wavelets and scaling functions.

then it is a candidate for

For Haar wavelets, recall 7! = /3=# cos =# so we could plug into (25) s , and then use previous formulas to get wavelet <B to get 9 If we start with a function 7! =, when does 25 lead to a genuine wavelet? Check conditions: (1) s = 9 " _ $7! =/#4 #1 4"

_ " 7 =#$7! =/#4 #1 ! 4= #

" _ $7! =/#4" 7! =# #1 j=" s =# 7! =#9 Recall this implies that Z4 Z4" where Z4 " +5 945 B "l+5 l# _
_ 5_ 5

(26)

(usual definition) with 945 B #4# 9#4 B 5

79

(2) The second condition we need to check is that translates of 9 orthonormal, i.e., s = #15l# "l9
5

" #1

If 7! = finite Fourier series " +5 /3=5 trigonometric polynomial

R 5R

there is a simple condition which guarantees condition (2) holds. Theorem (Cohen, 1990): If the trigonometric polynomial 7! satisfies 7! ! " and l7! =l# l7! = 1l# " (27)

(our standard condition on 7! , and also 7! = 0 for l=l 1$, then condition (2) above is satisfied by s = 9 Proof: Daubechies, Chapter 6. Since condition (1) is also automatically satisfied, this means 9 is a scaling function which will lead to a full orthonormal basis using our algorithm for constructing wavelets. Another choice of 7! is: 7! = " " $ $ $/3= $ $/#3= " $ /$3= ) " _ $7! =#4 #1 4"

(Fourier series with finite number of terms).

80

2 1 . 5 1 0 . 5 2 1 0 . 5 1 1 . 5 1 2

Fig 41: Real (symmetric) and imaginary (antisymmetric) parts of 7! = To check Cohen's theorem satisfied: (i) Equation (27) satisfied (see exercises). (ii) If 7! = Re 7! = 3 Im 7! =, l7! =l# lRe 7! =l# lIm 7! =l# ! for l=l 1$, as can be seen from graph above. So: conditions of Cohen's theorem are satisfied. In this case if we define scaling function 9 by computing infinite product (25) (perhaps numerically), and then use our standard procedure to construct wavelet <B, we get:

phi(t)

1.25

0.75

0.5

0.25

t 0.5 1 1.5 2 2.5 3

-0.25

81

psi_2(t)

1.5

0.5

t -1 -0.5 0.5 1 1.5 2

-0.5

-1

fig 42: pictures of 9 and < Note meaning of 7! : In terms of the original wavelet, this states 9B
" %

" $9#B $ $9#B " $ $9#B # " $9#B $

(see (26) above). Note this equation gives the information we need on 9, since it determines 7! =

82

6. Examples, applications
1. Other examples Note again it is possible to get other wavelets this way: If we demand 9B ##' 9#B )&% 9#B " "#% 9#B # "*' 9#B $ "%$% 9#B % !%' 9#B & ""! 9#B ' !!) 9#B ( !") 9#B ) !!% 9#B * Then this results with an 7! = 7! = ""$ %#( /3= &"# /#3= !*)/$= !!#/*3=
4

(28)

1 2

Fig 43: Real (symmetric) and imaginary parts of 7! ; note condition (ii) of Cohen's theorem is satisfied. Can check it satisfies condition (ii) of Cohen's theorem and resulting 9 is obtained: s = $7! =#4 . 9
_ 4"

83

It satisfies required properties (a) - (f) of a multiresolution analysis. Corresponding scaling function 5 9B and wavelet 5 <B are below

Fig 44: Scaling function and wavelet for the above 9 choice NOTE: Can show that if there is a finite number of terms on the right side of (28), then corresponding wavelet and scaling function are compactly supported. 2. Numerical uses of wavelets Note that once we have an orthonormal wavelet basis <45 , can write any function: 0 B " +45 <45 B
45

with +45 0 <45 . Numerically, can find +45 <45 0 using numerical integration to evaluate inner product With Daubechies and other wavelets, there are no closed form for the wavelets, so above integrations must be performed on the computer.

84

But there are very efficient methods of doing this: in order to get all the wavelets <45 into the computer, we just need to input one - all others are rescalings and translations of the original one. There are efficient algorithms to get coefficients +45 ; more details in Daubechies' book.

3. SOME GENERAL WAVELET BASES:

PROPERTIES

OF

ORTHONORMAL

Theorem: If the basic wavelet cannot be infinitely differentiable. (in particular, if differentiable). <

<B

has exponential decay, then

<

has compact support, then

<

cannot be infinitely

Proof: Daubechies, Chapter 5.

Compactly Supported Wavelets: So far we are able to get wavelets <45 B #4# <#4 B 5 which form an orthonormal basis for P# Note Haar wavelets had compact support. When will wavelets be compactly supported in general? Recall we assume that given basic scale space Z! , that we have scaling (pixel) function 9 such that 9B 55 form basis for Z! . Recall Z! Z " , 9B Z! 9B Z" # 9#B Z" # 9#B 5_ 5" form a basis for Z"

85

Recall since 9B Z" , we have for some choice of 25 : 9B " 25 # 9#B 5.

_ !

Constants 25 relate the space Z! to Z" . We will see that: Theorem: finitely many 25 ! <, 9 have compact support.

Proof: : Assume 9 has compact support. Then note since #9#B j are orthonormal, 2j ( #9#B j9B.B

! for all but a finite number of j

fig 45 Note 26 integral of product ! for all but finite number of j To prove : (rough sketch only) Assume that 25 are ! for all but a finite number of 5 . Then need to show 9B has compact support. s = Strategy of proof: look at 9 Recall we defined

86

7! = "
5

25 35= / #

Recall:
_ " s = $ 7! #4 =. 9 #1 4"

s(=) extends to an analytic function of = in whole From this show that 9 complex plane satisfying: s =l G" l=lQ /R lIm =l l9 for constants Q and R s is This implies by Paley-Wiener type theorems that 9B J " 9 compactly supported.

4. GENERIC PRESCRIPTION FOR COMPACTLY SUPPORTED WAVELETS: Start with finite sequence of numbers 25 (define how Z! will be related to Z" ) Construct 7! = "
5

25 35= / #

check that it satisfies Cohen's theorem conditions l7! =l ! for l=l 1$ and l7! =l# l7! = 1l# " Construct

87

" _ s = $ 7! #4 = 9 #1 4" Construct Fourier transform of wavelet by: s = /3=# 7! =# 1 9 s=#, < Take inverse Fourier transform to get <B wavelet

5. SOME FURTHER PROPERTIES OF WAVELET EXPANSIONS QUESTION: Do wavelet expansions actually converge to the function being expanded at individual points B? Assume that scaling function 9 is bounded by an integrable decreasing function. Then: Theorem: If 0 is a square integrable function, then the wavelet expansion of 0 0 B " +45 <45 B
_ 45

converges to the function 0 almost everywhere (i.e., except on a set of measure !). QUESTION: How fast do wavelet expansions converge to the function 0 ? ANSWER: That depends on how regular" the wavelet < is. particularly it depends exactly on the Fourier transform of <: Theorem: In . dimensions, the wavelet expansion 0 B " +45 <45 B
45

More

converges to a smooth 0 in such a way that the partial sum

88

4R 5

" +45 <45 B

differs from 0 B at each B by at most G #R = , iff s =l# l=l#=. . = _. ( l<

6. CONTINUOUS WAVELET TRANSFORMS Consider a function <B P# (i.e., < is square integrable), such that <B decays fast enough at _ (faster than "B# ), and such that (
_

<B .B !.

Then we can define an integral wavelet expansion (integrals instead of sums) using re-scalings of <B: Define rescaled functions <+, B l+l"# <+B ,. [note + "+ in definition of Daubechies] Here + , . Thus + measures how much < has been stretched (dilation parameter), and , measures how much < has been moved to the right (translation parameter). New point: dilation parameter + and translation parameter , can take on any real value. Now define wavelet expansions in this case (analogous to Fourier transform -- called wavelet transform): given 0 P# , we define the transform (assuming that < is real)

89

[ 0 +,, ( .B 0 B l+l"# <+B , ( .B 0 B <+, B <+, 0 How to recover 0 from [ 0 + ,? Claim: 0 B G ( where s=l# G " #1( . = l=l" l< Pf. of claim (sketch; details in Daubechies, Ch. 2): We will show that for any 1B P# , 1B 0 B 1B G ( To see this, note that 1B 0 B ( (
_ _ _

.+ ., [ 0 + , <+, B

_ _

.+ ., [ 0 + , <+, B

1B 0 B .B

.= s 1 = s 0 =

[use Plancherel Theorem for wavelet transforms] G ( ( .+ ., [ 1+ ,[ 0 + ,

90

 G ( ( .+ ., 1B <+, B[ 0 + ,B 1B G ( ( .+ ., [ 0 + ,<+, B as desired, completing the proof. Thus we know how to recover 0 B from [ 0 + , (analogous to recovering 0 B from s 0 = in Fourier transform). QUESTION: What sorts of functions are [ 0 + ,? For some choices of <, these are spaces of analytic functions.

7. Convolutions: Definition: The convolution of two functions 0 B and 1B is defined to be 0 B1B (

_

0 B C1C.C

Theorem 2: The convolution is commutative: 0 1 10 Proof: Exercise. Theorem 3: Specifically, The Fourier transform of a convolution is a product. Y 0 B1B #1 s 0 =1 s =

Proof: Exercise. Lemma 4: For any function 0 , Y 0 B s 0 = Proof: Exercise.

8. APPLICATION OF INTEGRAL WAVELET TRANSFORM: IMAGE RECONSTRUCTION (S. Mallat)

91

Dyadic wavelet transform: a variation on continuous wavelet transform. Now define new dilation only by powers of 2; arbitrary translations: <4, B #4 <#4 B , Define <4 B #4 <#4 B. (Still allow , to take all values, but restrict + #4 .) Define this dyadic (partially discrete) wavelet transform by: [ 0 4 , ( 0 B <4, B .B i.e., usual set of wavelet coefficients, except that , is continuous. Note: [ 0 4 , ( 0 B <4, B .B ( .B 0 B #4 <#4 B , ( .B 0 B <4 B , 0 <4 , (a convolution) where as above <4 B #4 <#4 B shrinking of < by a factor #4 . s(=) satisfies New assumption: Fourier transform <
_ s #4 =l# " . " l< #1 4_

92

Now: given 0 B, consider dyadic wavelet transform; + #4 only: Can show under our assumptions that can recover 0 in this case too: Recovery formula for 0 is: 0 B " [ 0 4 B<4 B
_ 4_

(convolution in variable B). It is easy to check that this is correct: if Y denotes Fourier transform: Y " [ 0 4 B<4 B Y
_ _

4_

4_

" 0 B<4 B<4 B

_

" Y a0 B<4 B<4 Bb

4_ _

s 4 = < s 4 = #1 " s 0 = <

4_ _

s #4 = < s#4 = . #1 " s 0 = <

4_ _

s#4 =l# #1 " s 0 =l<

4_

s#4 =l# s 0 = #1 " l<

_ 4_

s 0 =. QUESTION: Given 0 B, what sort of function is the wavelet transform [ 0 4 ,, as a function of 4 and , ?

93

Let Z the collection of possible functions [ 0 4 , collection of possible wavelet transforms. When is an arbitrary function 14 , a wavelet transform? Can check that 1 must satisfy a so-called reproducing kernel equation: 14 , is the wavelet transform of some function iff 14 , O14 , " <4 ,<j ,1j ,
_

j_

[this equation defines O1; note convolution is in ,.] Back to recovering 0 from wavelet transform: Thus we can recover 0 as a sum of 0 at different scales: 0 " [ 0 4 B<4 B.
_ 4_

Since < is a known function, we can recover 0 from the sequence of functions. Assume +B is a cubic B-spline:

Fig. 46: A cubic B-spline +B is a symmetric compactly supported piecewise cubic polynomial function whose transition points are twice continuously differentiable

94

Now let the wavelet be its first derivative: <B

. .B +B

Fig 47: <B

. .B +B is

the wavelet

Using the wavelet <B [ 0 # B [ 0 " B [ 0 ! B [ 0 " B [ 0 # B [ 0 $ B To see that these pieces of 0 represent 0 at different scales, look at example:

95

So: one can recover 0 from knowing the functions [ 0 4 B. This is a lot of functions. What advantage of storing 0 in such a large number of functions? We can compress the data. CONJECTURE: We can recover 0 not from knowing all of the functions [ 4 B, but just from knowing their maxima and minima. Meyer has proved this conjecture false strictly speaking certain choices of < (including the above derivative <B of the cubic spline). It has been proved true for another choice, the derivative of a Gaussian. <B . B# / .B

However, for either choice of < numerically it is possible to recover 0 B from knowing only the maxima and minima of the functions [ 4 B Numerical method:

96

Assume that we are given only the maxima and minima points of the function [ 4 B for each 4. How to recover 0 ? Given 0 , first take its wavelet transform; get [ 4 B. Define > set of all functions 14 B which have the same set of maxima and minima (in B) as [ 4 B for each 4. Z set of all 14 B which are wavelet transforms of some function of B.

Idea is: the true wavelet transform [ 0 4 B of our given function 0 B is in > (i.e. has the same maxima as itself) and is in Z (i.e., in the collection of functions which are wavelet transforms). Thus [0 > Z . intuitive picture:

fig 48 Thus if we know just the maxima of [ 0 4 B, we can try to find [ 0 4 B That is: 1. We know maxima of [ 0 4 B, so 2. know > all functions with same maxima as [ 0 4 B
97

3. Find [ 0 4 B as unique point in > which is also a wavelet transform, i.e., unique point in > Z :

Algorithm: 1. Start with only the maxima information about [ 4 B. Call Q the maxima information. 2. Make initial guess using function 1" 4 B which has the same maxima as [ 4 B . 3. Find closest function in Z set of wavelet transforms to 1" 4 B. Call this function 1# 4 B 4. Find closest function in > functions with same maxima as Q to 1# 4 B. Call this function 1$ 4 B 5. Find closest function in Z to 1$ 4 B; call this 1% 4 B. 6. Find closest function in > to 1% ; call this 1& 7. Continue this way: at each stage 4 find the closest function 14 to 14" in the space Z or > (alternatingly). Eventually the 14 4 B [ 0 4 B as desired. 4_

98

CONCLUSION: We can recover the wavelet transform [ 0 4 B of a function just by knowing its maxima in B. THE POINT: Compression. We can store the maxima of [ 0 using a lot less memory. APPLICATION: Compression of images:

Fig. 49

99

Fig. 50

9. Wavelets and Wavelet Transforms in Two Dimensions Multiresolution analysis and wavelets can be generalized to higher dimensions. Usual choice for a two-dimensional scaling function or wavelet is a product of two one-dimensional functions. For example, 9# B C 9B9C and scaling equation has form 9B C "256 #9#B 5 #C 6
56

Since 9B and 9C both satisfy the sclaing equation

5

9B "25 #9#B 5

we have 256 25 26 Thus two dimensional scaling equation is product of two one dimensional scaling equations.

100

We can proceed analogously to construct wavelets using products of one-dimensional functions. However, unlike one-dimensional case, we have three rather than one basic wavelet. They are: <M B C 9B<C <MM B C <B9C <MMM B C <B<C The generalization of the one-dimensional wavelet equation leads to the following relations: <M B C "156 #9#B 5 #C 6
M 56 MM

<MM B C "156 #9#B 5 #C 6

56

<MMM B C "156
56

MMM

#9#B 5 #C 6
MM

where 156 25 16 156

MM

15 26 and 156

15 16

We can generate two-dimensional scaling functions and wavelets using the functions ScalingFunction and Wavelet then taking the product. For example, here we plot the Haar wavelets in two dimensions. Various translated and dilated versions of the wavelets can be plotted similarly.

101

Fig. 51: Two dimensional Haar scaling function 9B C

Fig. 52: Haar wavelet <M B C

102

Fig. 53: Second wavelet <MM B C <M C B

Fig. 54: Third wavelet <MMM B C As example of another wavelet, here is so-called "least asymmetric wavelet" of order 8 in two dimensions

103

Fig. 55: Least asymmetric wavelet of order 8

104