• Skip to primary navigation
  • Skip to main content
  • Skip to primary sidebar

ALLSIGNALPROCESSING.COM

Learn signal processing online

  • Home
  • Courses
  • Courses
  • About
  • FAQ
  • My Account
  • Blog
  • News
  • Contact
  • Login
  • Logout
  • Get All Access

Solving Least-Squares Problems with Gradient Descent: the Least Mean-Square Algorithm

August 10, 2019 by

The least mean-square (LMS) algorithm solves the MMSE filter or least squares problem using a very simple iterative scheme. The current solution is updated by taking a step in the direction of the negative gradient of the instantaneous squared error.  You will find this very simple algorithm to be a powerful tool for MMSE filtering and solving least squares problems.

LMS has a 50 year history in signal processing and continues to find new applications.  For example, when solving very, very large least squares problems, it may not be possible to hold all the data in computer memory at once.  LMS and variants are used to obtain a low-cost iterative solution that requires relatively little memory.  LMS is also widely used in MMSE filtering problems to adapt or adjust the filter weights and maintain an optimal filter in response to changing characteristics of the input data.

Click here to purchase immediate access to this and every other (more than 120) lesson!

Primary Sidebar

Courses

  • Foundations

  • Time Domain LTI Systems

  • Fourier Series and Transforms

  • Sampling and Reconstruction

  • The DFT and Applications

  • The Z-Transform

  • Intro to Filter Design

  • IIR Filter Design

  • FIR Filter Design

  • Random Signal Characterization

  • Basis Representations of Signals

  • Estimation of Power Spectra and Coherence

  • Introduction to Signal Estimation and Detection Theory

  • MMSE Filtering and Least-Squares Problems

Copyright © 2023 ALLSIGNALPROCESSING.COM | Site Design by 3200 Creative

  • Terms of Service
  • Privacy Policy
  • Contact