Institutional Archive of the Naval Postgraduate School

Calhoun: The NPS Institutional Archive

DSpace Repository

Theses and Dissertations


1. Thesis and Dissertation Collection, all items

Acoustic noise removal by combining wiener and wavelet filtering techniques

Forney, Fredric D.

Monterey, California. Naval Postg

raduate School

Downloaded from NPS Archive: Calhoun

atthe | DUDLEY

WN] | ciseaRy

Calhoun is the Naval Postgraduate School's public access digital repository for

research materials and institutional publications created by the NPS community.

Calhoun is named for Professor of Mathematics Guy K. Calhoun, NPS's first appointed and published scholarly author.

Dudley Knox Library / Naval Postgraduate School 411 Dyer Road / 1 University Circle Monterey, California USA 93943

oe ate Pe ea ar “aby eh o ig t ; esi oh O se uneteaa sail SE ag ea tlt 1a Gud ta “= eer ee hd By CPteLY, ae adm " A as 1d ch te ade sf IAT A eos Prd is es . yh lag ays ni at 6 Ak 1 iat SP ne ls? R oer eee mile tis = arareenas teat, whanes ) cs can wane 5! Res nid teeth sctistiaa mes - 8. 06 pa ata atag | are pei 4 . ral seuss e , athe 7H ae tf tg Bot ty pre v8 ite Aah stein! re) 5 2 ous Piotseecaanden heb ‘pate : aves -oten 7 ows of 4s oarft Sinaia ie ois athe 42.8 0 6% al oF sigh tidal whe iat ad Fad HF ait a Me By parse See m3 iis ree! . Ae *


7 i! ins ae

eens i an id ‘a coe


wae siggy et is roth git dou bats " tet ve Ted tones gestae act has chee mgt J Re

gr 0% re waege alec al a@natae

aS Heh at

Pa hae eee saton m4 ‘a Ten pte ote tp sta. palite wn a {itiige Aevad ue Relor, i Pere goyeuee ae Te Vedas ae He eer a eh ial, eer Beats ut wens Nae ahah d,

hat oy ege sey dete tas sy Satan 3 a wae Fab Dec eMal ees See

eaia Se ea ee te Tih yt deka oS wha) Gilg2 ete cote Ms

ors ae bend 8 a0 wl a at ae 1 Ya det A Ooty gdeg platen eye tet

uns wo i re tg Nas ran nt gde pita te ytateday! 1pyy

ane Bae fa we C4 * > ° it agboeinisginie a Tass bac ches Ch

care "3 ais ob LOS hare ad yenee Satiaragniteeey

Hoe .

eee aes : he athe 3 a a Bane he ange 7g Reet acs ie ry seth ra eg P Se Nekahgahe Pet Tog ahd pa pen et pata

obetal sae te ta

. v vatehne ats % auledader vase eet a iat ata!

Nita dae ae i fabio

af ny Sphhhics

H as ae

+ mee ens 4

at 4 + fg lntstated padet® CS re * ata batedateds a acter td ie inte af ae wie tgieat sete» - eva bad eral ef 4 peas 2 igs aa - nary A een i is Be arte + a E , Ce ek Har ah Muted Made Qe biFe B ames 8 et LaSade tencla te otehee 1 So® eee em teh e sFe8s eeatedet it” A a ee MST ser Bie ey Ny Sakae wf Men eh 3a" 1s eae weg srylole'e aSeaedes ony 0 Babe te obst sete ut of Ny tat of Gane te DeMoivie iets vont Od gg pp evade! Seat aoe” watets etattede

pee dote i td 9% ee teh G04 wake 4 As wt

as sate $s



Cae ere Oe

ot erate bene®

rr ee ie Le "tk

rer Ag rs) “pear ~eed Te tetes tea oe yt 8 ee Seueee onade

ant a 3 nf .

= awals & Seted Bie *

‘aoe ares : taSeaeate © oa

eae 8 sea haty td A, i pene «=

. ns Pie

rae hf s entg nN bases = 2, e * hate of Tate ka a sor any fie aa

os a ee tergtee a8 gtyaee eee a” te 4 ¢ NM

Ca Behe 4 il ee

ae 2

oe ate . oor. 4 ig etree

re | . pene

. 4h yi ae Per a roe oem 148 rn ,." “uh 2 4A Rar ea ee


gna te ere’ os a se pare ie ee Oye a ge etant” Serene 2 UN aS ane eat tis POON had af bie

ry aa poe: Ser anes *, 1%e* - Ay a Ra oats aes sd CaN 1 SP ar ne qa am

ue agen ee

oF <4 ie ee id a Det

paneme cy

wa sl

Pie fa ry soncaoip ees

eta 7105 842g Be + ‘4 ea" “| oe ase PS ath Lokal de tet seers : a : a = zl

Pree yt he dh at “86 pachecty aA gre eg Oe. 8 be fe urate

ee bad

ae a Re « pe ST) Re eer : . ya ghg ath i voatety ata bk ae t) seryigterei Beer: pee a “7% ee LP etd aren meee Nt ae yay Cae nity Ait ite pakistan fa sass steet om git es at ait incur! ree seit Se fights rae ¥, pepe yong > a tie abe ae cee sto tate wer a hee i Ne PNT Atay an cabetarn secherg tary meet store B08! rs tte at aetiye Eder bho gurees

PP ak dee he Sa pa iL Sy Tie Baars Tie it acd bt ae a ihn Le es Pitan ear ahty rate coats fa “| ral mt al urhety tata we Me eap escnepeaetes iw eet ne cl etatensteadant at te? Be st ut a6 taegaree fet ea rages te pick Ter el ey Seat) stab ety" sghyegine hee Nitaeey

r Ae seh by ted binest " ak Sieee akong i; geuaze ba ps4 oe an sone 4 atewutts gents HY anette i ee et ts ; eh isa cee ie ae es SS eat ananassae on ete wate Rosod ey ace rep Piey ig ern get te) Se Bi be PRE be ah bot ts" A ney preter 74 y tt daptunaute ecerud wens areey” ee % . : fate al Seat i a astitey crea ee a H edtoee" erie Le a et rc te i ory Meee eee a & a) aie ess A che ahs aa Ae aaa atet Dy ay he aytets "4 eyee em, bade he ord ware fT. ; a ost ie ake A ‘Sescethe ee tye A amr Pine Sh Reet tt % seat wot erg tar Aa “ene wih a patel a eh4e ie = Hifi en ae eae a es Dita ipls fe seit! ot rT Detetatece tenure ae hearin ie *esetg Re ite = apys Pits PRL Ted ie, pe Ee ty tgeesbrene WZ teteteta wegen Seo ' OL aif ies tater HA whoa ayer pepe Mes me shee ta'n a lela dA i eoraresece cnr tgng-'* ar rereryet SW Serates eT ar, bp Sees tarig's meee ~ es ae etry oe i ries ence po A AR A ae a eR at fahy es eens i be tp siel ge are are efile ett ote? raga lay baal abe tes dite! aa vag ea) Lee tate pee 28 egw TIME STE Se = st wi 48 es ser ppacmaceae jo"s. io” tPA é FH < u iat meee . 7 Crit. tid = pray eee cigetley iets et inlet ames

ead sites inetses eee

whe "ase weeps Tar Ft Py ndagenig tees


Ait yiate, eee ster atad ! -y‘e" 4

se "2 eet se Dee

att ‘8 Ges oes op ne 2 ed ink

ve y re ed be stk Oe wid igi tees

De ee Me me Pirie. herds atere Paine > cecets pe ce) baum Spacey f sb aetsee 8 ebatae st itt, ie

eet Syagstewtty Aceee” : Uk aa Pity ope pera sere Beare oe i a ao ense





Fredric D. Forney, Jr. June 1998

| Thesis Advisor: Monique P. Fargues Co-Advisor: Ralph Hippenstiel

Approved for public release; distribution is unlimited.


| Public reporung burden for this collection of oeeaen is estimated to average | hour per response, mivalreiny: the ume for reviewing instruction, searching existing }} data sources, gathering and maintaining the data needed, and completing and reviewing the collection of information. Send comments regarding this burden estimate or any other aspect of this collection of information, including suggestions for reducing this burden, to Washington Headquarters Services, Directorate for

| Information Operations and eae ae Jefferson Davis ae Suite 1204, Arlington, VA 22202-4302, and to the Office of Management and Budget,




6. AUTHOR(S) Fredric D. Forney, Jr.



11. SUPPLEMENTARY NOTES The views expressed in this thesis are those of the author and do not reflect the official policy or position of the Department of Defense or the U.S. Government.

12a. DISTRIBUTION/AVAILABILITY STATEMENT 12b. DISTRIBUTION CODE Approved for public release; distribution is unlimited.

13. ABSTRACT (maximum 200 words)

This thesis investigates the application of Wiener filtering and wavelet techniques for the removal of noise from underwater acoustic signals. Both FIR and IIR Wiener filters are applied in separate methods which involve the filtering of wavelet coefficients which have been produced through a discrete wavelet decomposition of the acoustic signal. The effectiveness of the noise removal methods is evaluated by applying them to simulated data. The combined Wiener wavelet filtering methods are compared to traditional denoising techniques which include Wiener filtering and wavelet thresholding methods.


14. SUBJECT TERMS Wavelet Analysis, Wiener Filtering, Denoising, Acoustic 15. NUMBER OF Eiris. PAGES 148



Unclassified Unclassified Unclassified UL

NSN 7540-01-280-5500 Standard Form 298 (Rev. 2-89) i Prescribed by ANSI Std. 239-18 298-102

Approved for public release; distribution is unlimited.


Fredric D. Forney, Jr. Lieutenant Commander, United States N avy

B.S., United States Naval Academy, 1983 M.S., The George Washington University, 1991

Submitted in partial fulfillment of the requirements for the degree of




This thesis investigates the application of Wiener filtering and wavelet techniques for the removal of noise from underwater acoustic signals. Both FIR and IIR Wiener filters are applied in separate methods which involve the filtering of Wavelet coefficients which have been produced through a discrete wavelet decomposition of the acoustic signal. The effectiveness of the noise removal methods is evaluated by applying them to simulated data. The combined Wiener wavelet filtering methods are compared to more traditional denoising techniques

which include Wiener filtering and wavelet thresholding methods.

- —— | 2


_ LT INTRODUCTION ... 2.6.6. eee ence eee eee een eens l 1. OCEAN ACOUSTIC NOMS Bo vicar ee nin eect S A. NOISE SOURGES oo: vs. 55 5 cre ere ciel +

l. AMDICHEINOISE. .... 2:5. << << 400 cacee eee ee 4

a. Ultra-Low Band (<1 Hz) ....... Sing ep ie PI 4

b. Infrasonic Band (1 to 20 1iz).......... ee ee 4

Cc. Low Sonic Bana(20to Z0UTHz) «ee ss 5

d. High Sonic Band (200 to SU UUIZ) ee ee 5

e. Ultrasonie Band (> 50,000:HzZ)... . 7 eee. . ee 6

Ze SelONoise ec... . Sh es ee eee di


ee errr re we oh . fd eas Ske 8

c. WOU Sle) (C)} 15] 5 EE sj on coe eee 8

mime ei NEN FUMING gees 8 es ee ee oes no see a eo eee cs en Hi A. MODEIMDES@RIDTIOIN........:.:+-- +... ee. Os es 1]




A. Pelt 1 ecatibes Ee CIN Meena eee OE re ee ee eee ie


C. THE DISCRETE WAVELET TRANSFORM .................... 4)

D. IVETE RATES SwY.S EVES AUN SS ames sas ss ws See 30

Jel DAUBEGHITE SW Ay ECE ro IOS. ss. sds ae S/S)


B. TREES IOI Sees CO aaa ks se es ee 45

l. LOPES EUG CT lore oe ee ee 45

2. SIRE i itesO ld eee ei: ss << s « Se le 46

8. IVD BIG AN TESMNO| dae cepa wi sie... Semone aecy cee ee 46

4. Minimax Primeaple Vntesheld) . . “aiteicc.u as... sss sesso 47

>: Threshold Discussion ... et ee 47

pple ReUNe wey NORIO ER . eee tees ee ee aks eee ay A. M@IPTMIRATE SYSTEMS ANAIEYSIS 2...) . i scugenn 00 ee 60

B. AMPIAS ‘CANCELEA TION 9.) igs ee eee 66

Pe elimliicen te Nite tn WW AVERT TIGDIIR ox 2 eee ss in a 98S ee tie we 2 A. De AIEEE |) 6) 522 ook il 2) See rrr ane Ome 71



@F WIENERSHRINK. . peepee 719

VI. COMPARISON OF DENOISING METHODS .......................005. 85 A. WIENER FILTER ........ ... 0.0%... oS tee 86




E WIENERSHRINK = oi: esos Se ee. ee 88 iPemveAVELE |) PACKETS: 2... soe Seri ic..... lee ae 101 A WAVELET PACKET THEORY Sue a0 262 een eee 102


l. Short-time Wiener Filter 2... 245... 22 eee. 106

ip Wavelet Ehresholding .........iah.. 2. oe eee ee 106

3. FIR Witeper Wavelct Filter... 0: soe ne 106

4. IR Wiener Wavelet Milter... 0.22... ear see 107

Be Wavelet Packet [hresholdime =.) acer nen es: 107

6. BIR OW ienen Wavelet Packet Rilietc--pa es oe 6 ble a eee 108

7. IIR Wiener Wavelet Packet Filter ..... ics oe 108

8. SUMIRIARY 9)... eens I A Ee i ae 108

PCO CILUSIONS oi cn nt ee eile se ea eee He



Accurate analysis of ocean acoustic signals has proven to be a difficult task due to unwanted noise which is usually present. Significant effort has been directed to the problem of noise removal from underwater acoustic signals. In the 1940s, Norbert Wiener conducted extensive research in the area of linear minimum mean-square error (MMSE) filtering. Provided the spectral characteristics of an additive combination of signal and noise are known, the Wiener filter will operate in a linear manner to yield the best separation of the signal from the noise. Here, “best’”” means minimum mean-square error. The Wiener filter is today a well tested method of noise reduction.

Wavelet techniques are, conversely, a more modern approach to noise reduction, although their origin lies in the timeless methods developed by Fourier. Wavelet analysis decomposes the signal into a family of basis functions and provides two significant advantages over the traditional Fourier analysis. First, in wavelet analysis there is a wide variety of basis functions to choose from, and second, a multi-resolution capability is provided in the time-frequency domain which is critical to the identification and elimination of noise in a non-stationary environment. Wavelet techniques have proven to be a viable tool for the denoising of acoustic signals.

Recently, a question has been posed concerning the combination of these two invaluable techniques. The approach presented applies a discrete wavelet transform to the noisy signal. Next, rather than thresholding the wavelet coefficients, a Wiener filter is

applied to separate the noise from the signal. While this approach may intuitively seem

reasonable, there are problems associated with aliasing and perfect reconstruction of the denoised signal which must be considered. Given that the aliasing problem can be dealt with, the results still may not be superior to either Wiener filtering alone or wavelet-based techniques alone but the possibilities are certainly worthy of investigation.

In this thesis, an approach is presented which combines the Wiener filter, in both the causal finite impulse response (FIR) and the non-causal infinite impulse response (IR) forms, with wavelet-based techniques. The various noise removal methods are applied to simulated data which offers the advantage of providing ground truth to the analysis. Additionally, the signal-to-noise ratio (SNR) level can be easily modified to evaluate the effectiveness of the denoising algorithms. The combined Wiener wavelet based denoising methods demonstrate promise in the recovery of ocean acoustic signals from the noisy ocean environment in which ships operate. |

The problem of denoising underwater acoustic signals is addressed in nine additional chapters summarized as follows. Chapter II presents background information on the characteristics of ocean acoustic noise . In Chapter III the theory and application of Wiener filtering methods is discussed. Chapter IV presents the theoretical development of wavelet analysis and and mutirate systems. Standard wavelet threshold based denoising techniques are presented in Chapter V. FIR Wiener wavelet noise removal methods are developed in Chapter VI followed by IR Wiener wavelet methods in Chapter VI. A comparison of the various denoising methods is presented in Chapter VIII. Wavelet packet denoising techniques are applied to the problem of high frequency signals of short duration in Chapter

[X. Finally, conclusions are presented in Chapter X.


Though significant resources have been devoted to non-acoustic detection methods in the undersea warfare environment, sonar remains the primary and most reliable method of detection. However, as modern technology continues to enable significant reduction in radiated source levels the problem of early detection and classification of underwater acoustic signals gains greater significance. A possible solution lies in the area of improved digital signal processing and filtering methods which would allow detection and classification of signals at lower signal-to-noise ratios. The passive sonar equation states that the source level of the target minus the loss due to propagation through the medium, minus the sum of all interfering noises plus improvement by the spatial processing gain of the receiver, must be greater than the detection threshold for a sonar system to sense the presence of a target with a fifty percent probability of detection [1]

SL-TL>NL- DI+DT, (2.1) where: SL_ = Source Level of the target being detected passively

TL = Transmission Loss as the signal propagates to the detector

NL = Noise Spectrum Level of the ambient noise and self noise in the ocean

DI =Directivity of the detecting system

DT = Detection threshold for 50% probability of detection and all terms are in dB referenced to 1pPa.

A reduction in the noise spectrum level would certainly result in an improved

detection probability. The noise spectrum level includes both self noise and ambient noise.


1. Ambient Noise

Urick identifies ambient noise as that part of the total noise background observed with a nondirectional hydrophone which is not due to that hydrophone and its manner of mounting called “self-noise”, or to some identifiable localized noise source [2]. Ambient noise 1s produced by a variety of sources and may be found in the frequency range from 1 Hz to 100 KHz. The noise frequency range is typically divided into five frequency bands of interest, which are discussed next.

a. Ultra-Low Band (<1 Hz)

Though little is known about the exact contributors at the lower end of the spectrum, it 1s surmised that these sources include tides and hydrostatic effects of waves, Seismic disturbances, and oceanic turbulence [3]. Tides and waves cause hydrostatic pressure changes resulting in a low frequency, high amplitude contribution to the ambient noise spectrum. The constant seismic activity measured on land extends into the ocean environment causing low frequency, high amplitude contributions which add to those produced by tides and waves. Oceanic turbulence gives rise to varying dynamic pressures which are detected by pressure sensitive hydrophones.

b. Infrasonic Band (1 to 20 Hz)

This band has gained importance with the emergence of improved low frequency narrowband passive sonar systems. It contains the strong blade-rate fundamental

frequency of propeller-driven vessels and its accompanying harmonics. A steep negative

spectral slope of 10 dB per octave is common in the region from | to5 Hz. This slope goes positive from 5 to 20 Hz as shipping noise begins to become a more significant factor. In the absence of ship traffic this region continues to fall off and is more affected by wind speed.

C. Low Sonic Band (20 to 200 Hz)

Studies have shown that distant ship traffic is the dominant source of noise at 100 Hz and has a signineent effect in the low thc band [3]. In areas of low shipping intensity, wind speed continues to be the major factor just as it is in the infrasonic and high sonic bands. Thus, an area of heavy shipping such as the North Atlantic, where on average 1,100 ships are underway, will see a much greater effect than less traveled areas such as the South Pacific.

d. High Sonic Band (200 to 50,000 Hz)

The well known acoustician V.O. Knudsen conducted extensive studies in this band during World War II. In these studies, he was able to correlate noise with wind speed in the frequency range 500 Hz to 25 KHz. His results, published in 1948, are best summarized by the curves shown in Figure 2.1. These curves show a clear relationship between wind speed (or sea state which isn’t measured as accurately) and spectrum levels. Subsequent studies have shown the spectrum to be flatter in the range 200 to 800 Hz but have generally confirmed Knudsen’s results [4]. Crouch and Burt [5] have developed an expression to model the noise spectrum level in dB which is given as

NL(f) = Bf) + 20 n logy V, (2.2)

where NL is the noise spectrum level in dB referenced to 1 Pa at frequency f, B(f) is the

noise level at a wind speed of 1 knot at a particular frequency, n is an empirical coefficient,


Moderate: Light

Spectrum Level (dB re 1 micro-Pa}

10" 10 Frequency (kHz) Figure 2.1: Average Ambient Noise Spectra. From Ref. [6].

and V is the wind speed in knots. For n = 1 the noise level increases as 20 log,, V, and the noise intensity will increase as the square of the wind speed.

é. Ultrasonic Band (> 50,000 Hz)

At frequencies above 50,000 Hz thermal noise becomes the predominant contributor to the noise background. In 1952 Mellen [7] showed theoretically that the thermal noise of the molecules of the sea affects hydrophones and places a limit on their sensitivity at high frequencies. Based on principles of classical statistical mechanics he

formulated the following expression for the spectrum level NL in dB referenced to | Pa of

the thermal noise at frequency f in kHz given by NL(f) = -15 + 20 log, f- (2.3) Though measurements have been recorded in this band they have not conclusively substantiated the above expression due to excessive shipping noise in nearby ports. No measurements in this band in deep, quiet open ocean water appear to have been made until now. Zs Self Noise Self noise includes all noise created by the receiving platform and usually falls into one of two categories which include equipment noise and platform noise. Equipment noise includes electronic or thermal noise produced within the sonar electronic system. Platform noise is produced from the same sources as radiated noise except that the source of platform noise is the receiving platform vice the source or target platform. These sources include machinery noise, hydrodynamic noise, propeller noise and transients. Platform noise reaches the receiving transducer by a variety of methods including vibration via an all-hull path, all- water direct path, all-water back scattered path from volume scatterers, and all-water bottom reflected path [2]. Machinery noise occurs principally as low frequency tonals which are relatively independent of speed. Hydrodynamic noise which includes all sources of noise resulting from the flow of water past the hydrophone and its support and the outer hull structure of the vessel, becomes more significant as speed increases. At high speeds

propeller noise becomes the dominant contributor to self noise.


Properties of the water mass such as temperature, salinity, and density affect the transmission path of sound in water and therefore alter the signal received at the hydrophone. Additionally, the depth of water and bottom structure influence the path traveled and could result in multiple transmission paths between the source and receiver. The affects of absorption and attenuation cause the ocean to filter out the high frequency spectrum while passing the low frequency spectrum. Thus, ocean acoustic noise appears to occur more predominantly in the lower frequency region. CG NOISE MODEL

Modeling ocean acoustic noise as a stationary white Gaussian random process substantially simplifies the analysis and study of denoising. However, the variety of sources which contribute to self and ambient noise and the acoustic transmission characteristics of the ocean result in a noise contribution which is colored vice white in nature. A common method around this obstacle is to pre-whiten the acoustic signal.

Stationarity is another assumption which must be applied carefully. Certainly the Ocean environment is one in which sources of acoustic signals change regularly. Moreover, the water mass properties are in a constant state of flux. This hardly meets the criterion for Stationarity at first glance. However, on a time scale of several seconds, the ocean environment can indeed be considered stationary.

Finally, evaluation of ocean acoustic noise reveals that the assumption of a Gaussian

random process appears to hold true in most cases. An artificial computer generated white

Gaussian noise sample and its power spectral density are pictured in Figure 2.2. This assumption can be proved by comparing the histogram of a noise sample to the Gaussian probability density function with appropriate sample mean and variance or by checking a normal probability distribution plot of the noise sample as seen in Figure 2.3. Barsanti’s results have verified this assumption with many actual ocean acoustic signals [8]. Frack discusses several more involved methods of verifying this critical modeling assumption [9].

All noisy signals in this thesis are generated by adding white Gaussian noise with zero mean (as shown in Figure. 2.2) to the noise-free signal. When analyzing actual ocean acoustic signals a noise sample must be available to determine the statistical properties necessary to produce optimal filtering. A noise sample with statistical properties which accurately reflect the noise embedded in the noisy signal produces a more effective filter and amore accurate denoised signal. Frack discusses methods which can be used to model noise

for filtering purposes when adequate noise samples are not available [9].



Oo 100 200 300 400 500 600 700 800 900 1000 Sample

PSD of Noise

Amplitude (dB)

O O.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 Normalized Freq

Figure 2.2: Artificial White Gaussian Noise Sample.

Histogram of Noise and Superimposed Normal PDF

Normal Probability Plot of Noise

8:88 = 885 B 0.50 = es Bee | = —-2 —1 Oo 1 2 3 Data

Figure 2.3: Histogram and Normal Probability Plot of Artificial White Gaussian Noise Sample. The superimposed normal PDF has been scaled to match the histogram for 1024 samples.



Wiener filtering is one of the earliest methods used to separate noise from a desired signal. Developed by Norbert Wiener in the 1940's, this form of optimal filtering is used to denoise ocean acoustic signals with the objective of improving signal detection and classification. The FIR Wiener filter and its application to denoising is developed in this chapter. A. MODEL DESCRIPTION

The optimum linear discrete-time estimation model is illustrated below in Figure 3.1. The filter input signal x(n), consists of a signal s(m) and an additive noise w(n). The estimator is constrained to be a linear filter with impulse response h(n), which is designed to remove the noise while preserving the characteristics of the signal. The filter impulse response is obtained by minimizing the estimation error e(n), defined as the difference between the filter output and the desired signal d(n), which is taken as the original signal

s(n), for filtering applications.

, Desired Signal lige mesposié Discrete-Time ; 4 Filter E 7 a) + 4 7 *@)=s@)+wa) 4) y(n) = (2) + d(n) = s(n) ; Estimation Noise Error w(n) e(n) = s(n) - s(n)

Figure 3.1: Linear Discrete-Time Estimation Model.


A time-varying filter can be derived for nonstationary signals but the complications are significant [10]. A more tractable approach involves segmenting the input signal into blocks which can be considered stationary for the short duration of a few milliseconds to a few seconds. In developing the filter for this model, the input signal and noise spectral characteristics or equivalently, auto- and cross-correlation functions must be known. Although typically not known a priori, this information can be estimated from the data assuming that the noise can be isolated sometime during the observation interval.


Given the spectral characteristics of an additive combination of signal and noise, Wiener proposed a scheme which best separates signal from the noise [11]. His procedure involves solving for the “best” filter coefficients by minimizing the difference between the desired ideal noise-free signal and the filter output in a mean square error sense. The mean square error is most often used due to its mathematical simplicity, as it leads to a second order cost function with a unique minimum obtained by using optimal filter coefficients [10].

The FIR Wiener filter is constrained to length P with coefficients h, (0 < k < P-1). Its output depends on the finite input data record x(n) = [x(n), x(n-1), ......x(n-P+1)]’ and may be expressed as the convolutional sum:

P-1 §(n) = y(n) = s h *(k)x(n-k). (3.1) The mean square value of the error, e(n), between the desired output s(m), and the filter

output S(7), can then be expressed as:

P-1 o-=E{le(n)?} =E{Is(n)- > h*(k)x(n-k)P}. (3.2) k=0

The principle of orthogonality states that the optimal filter coefficients h(n), for n = 0,]1...., P-1, minimize the mean square error if chosen such that E{x(n-i) e'(n)} =0, fori = 0,1...., P-1, that is if the error 1s orthogonal to the observations [12]. The minimum mean square error can then be given by o*, = E{s(n) e'(n)}. Now applying the orthogonality principle results in the following set of equations: E{x(n-ie *(n)} | xn-i| 5 “(n) y+ h(k)x oh) | ="05" “forr=Onterre (B38) k=0 Equation (3.3) can also be expressed in terms of the auto-correlation function of the observations R,.(k) and the cross-correlation function between the signal and the observation sequence R,,(i) which leads to: x ieee?) (eeu), for i= 0,],,...,P-1. (3.4) k=0 Assuming the signal and noise are independent and have a zero mean, it can be shown that: R (A=R (A) ES) and R(A)=R (A) +R (4), where R,(k) and R,,(k) are the signal and noise correlation functions respectively. The set of discrete Wiener-Hopf equations for the causal stationary filter can be

expressed in matrix form as

Kee © _., (3.6) where R, = E{x(n)x "T(n)} (3.7) and r.. = E{s(n)x(n)} (3.8)

66 699 ow

an represents the reversal operator applied to a vector [12]. Equation (3.6) may be


solved for the filter coefficients using matrix methods. The minimum mean square error is

found using the second part of the orthogonality theorem:

o. = E{s(n)e*(n)} = | sn| s w-¥ h encr-b (3.9)

or again in terms of the correlation function: o. = R{(0) >? h*(k)R_(k). (3.10) The Wiener filter is now applied to a synthetically generated noisy sinusoidal signal of varying frequency with a SNR of 0 dB. The original signal and noisy signal, shown in Figure 3.2(a), are compared to the original signal and denoised signal in Figure 3.2(b). The standard measure used to compare denoised signals to the noise free original signal is a

modified Mean Squared Error (MSE), defined as:

N MSE = > a el (3.11)

1 norm(s) norm(y)

where s(7) is the noise free original signal, y(n) is the denoised output, and N is the length of the signal. The signals are energy normalized by dividing by their norms. This represents a denoised signal amplitude gain which is applied to account for losses incurred during the filtering process. This normalized MSE will be used throughout the thesis as a measure of performance. It is not normalized by the signal length. However, a standard signal length of 1024 or 16384 points is used exclusively for all denoising trials. The Wiener filtering results for various filter orders is depicted in Figures 3.3 and 3.4. A single-frequency noisy

sinusoidal signal (0 dB SNR) is denoised through ten trials and an average normalized MSE


is computed. This is repeated for test signals with normalized frequencies which range from 0.01 to 0.49 in steps of 0.01 and the results are displayed. The noise sample supplied to the Wiener filter is an independent noise sample in Figure 3.3 and the original noise sample used to create the noisy sinusoidal signal in Figure 3.4. Although the original noise sample is not practically available, it is shown for comparison purposes. A table comparing the MSE values averaged across the spectrum is shown following the Figures. The MSE performance improves for higher filter orders. Additionally, the MSE value is rather flat across the spectrum for a filter orders higher than four. These results are as expected for the Wiener filter and provide a benchmark standard of performance against which other denoising

methods can be compared.


Original Signal and Noisy Signal: SNR = OdB, Normalized Freq = O.1


—3 Ss 10 18 20 25 30 35 40 45 50 Time (sample number)

Original Signal and Wiener Filtered Signal: Filter Order = 12

3 2 a | <b> = = e ° << ~—1 —2 =—3 oO Ss 10 18 20 25 30 35 40 45 50 Time (sample number)

Figure 3.2: Wiener Filter (a) Original (dashed) and Noisy Sinusoidal Signal at normalized frequency=0.1, sampling frequency=1. (b) Original (dashed) and filtered signal.


Wienr Filter Results: SNR = 0dB


Filter Order

0.3 S us ©

3S 0.25 Cop) = cS i: 8) >



Oo 0.05 O.1 0.15 0.2 0.25 0.3