The focus of this thesis is the development of computationally efficient adaptive filter structures capable of operating in nonstationary and nonlinear environments, with application to the acoustic echo cancellation (AEC) problem in hands-free communication systems. The specific nonstationary environments considered are changing conditions in the acoustic environment resulting from the movement of people and objects, as well as changing operating conditions of hands-free devices. The particular nonlinear environments arise from the harmonic loudspeaker distortion in hands-free devices.

The linear and nonlinear echo path components for several hands-free systems are identified based on experimentally measured data collected under the nonstationary and nonlinear environments previously outlined. The subsequent analysis revealed that these components and their fluctuations are confined to specific time and frequency regions, which provides insight for achieving computational complexity reduction in echo cancellers (ECs) operating in these environments.

An oversampled subband EC structure based on a generalized gradient proportionate step size adaptive algorithm is proposed for AEC in fast changing environments. The structure's enhanced echo cancellation performance compared to fullband ECs is verified through computer simulations. Improvements in echo return loss enhancement (ERLE) of up to 7 decibels (dBs) are realized compared to an equivalent fullband structure, based on measured data from hands-free systems under changing conditions, while requiring decreased complexity.

AEC in the presence of nonlinear loudspeaker distortion is addressed by developing an oversampled subband EC structure based on adaptive Volterra filters. Experimental results validate the structure's equivalent nonlinear echo attenuation ability compared to its fullband counterpart, while requiring much lower complexity. Compared to a linear EC, the proposed structure achieves up to 7 dB higher ERLE with a similar computational cost.

For AEC in changing nonlinear environments, an oversampled subband EC structure with adaptive Volterra filters based on a generalized gradient proportionate variable step size adaptive algorithm is proposed. The structure improves echo cancellation performance by up to 5 dB higher ERLE with significantly reduced computational expense compared to an equivalent fullband structure, based on experimental results with measured hands-free system data.