Hey Jacob!! I was scouring the internet for this exact article, and lo-and-behold it's written by the legend I used to work with (you derived a lead-lag controller on a whiteboard when I was a co-op student off the top of your head). Thank you for the awesome read. I was thinking of trying to replace a low-pass butterworth filter with a kalman filter in a system with no known inputs but lots of disturbances and measurement noise, just like the example you chose. The discrete math is going over my head for now, I've been relying on diagrams and python state space / simulation libraries to give me my answers so far, but will be implementing it in a more real-time fashion soon and will definitely be referencing your article!
In recent years, Ive moved from Kalman to Sav Golay filters, which in many circumstances work as well but are computationally much simpler. They also have a few additional 'tricks' such as providing the first & second differential signal.
We use such filters in motion control applications.
In practical terms, I find often that when trying to optimise control for a particular piece of equipment that often the best estimate you have of performance over the next x periods is a moving average of that last 5-20x periods.
This gives you a known near to exact correct state. Then you trim this with your choice of predictive controller.
That's another very good question. I'm not sure where to draw the line for the very simplest, but for example, I'd be surprised if you could solve the ship problem[1] or even the lighthouse problem[2] with a constant-coefficient filter -- at least not unless it was initialized with a very good initial guess. The solution to those problems involves observations with coupled errors (α, β, and for the ship, v), and if you don't accurately keep track of their covariances, the filter output can easily diverge.
That suggests to me that using a steady-state Kalman gain wouldn't work there. But I haven't attempted it, so I'm hesitant to say it can't be done!
The math is a bit beyond me, but it sounds like Kalman filters are in some ways self-calibrating. So, perhaps the simplest example might be when a simple filter would work, except that one parameter needs to be adjusted on the fly, due to changing conditions?
I really like your writing and technical analysis. I found my way here via HN linking your "right half plane zeroes" post. To be honest, none of the mathematics landed with me, never having studied control theory and not having even attempted to work my way through your post, but the 40% or so that I grokked was quite interesting.
I'm a software developer with a wide variety of technical interests that definitely includes control theory. Who doesn't love inverted pendulum and PID controllers? I'm 20 years outside my CS degree and only discovered PID about 5-6 years ago. Basically, I'm at the level of, positive feedback bad, negative feedback good.
Anyway, I recently published a PID implementation, mainly as an exercise and also because I needed one for a different project. https://github.com/rickhull/device_control
I thought you might find it interesting, mainly for pedagogical purposes, and I'd be very receptive to any feedback you have, positive or negative, heh.
I'm still trying to wrap my head around Kalman filter, and that might be the next thing I tackle. I'm hoping it's relatively tractable like PID in terms of software implementation. I pored over the dang PID wikipedia article multiple times, and it always seemed way more complicated to understand or suggest an implementation, compared to the rather simple implementation I landed on.
I wouldn't worry too much about following the math. I try to aim my writing so that the text, pictures and videos convey the ideas without needing to read any of the equations. But I know that some people prefer to see the math, or might be trying to follow along while implementing something themselves, so I include it for them too.
And great work on implementing a PID controller! The main reason I set out to write this newsletter was to share the underlying mindset and intuition that control theory has helped me develop, which I find so useful in thinking about the world and everyday life. Interacting with PID control is a really good way to get a sense of the core teachings of control theory, and if everyone spent time playing with them then I probably wouldn't need to write anything at all!
Unfortunately I don't have any experience with Ruby so I'm kind of guessing a bit, but I think this looks like a great start. I'm curious what project are you using it for?
Here are some specific thoughts, if you don't mind me sharing them:
(1) When I'm looking at a PID controller the integrator implementation is the first place I look at, because it's the easiest place for subtle problems to creep up.
Since your @ki gain is tunable, this formulation could give you an unexpected surprise when you adjust the tuning on the fly. I recommend either using a pre-multiply formulation like this:
> @sum_error += @ki * @error * @dt
or else implement a "bump-free" tuning function where @sum_error gets recalculated whenever @ki is adjusted, such that the output stays continuous. Otherwise you'll get a step-change in your output proportional to the present value of @sum_error.
Also, I'd suggest clamping @sum_error, to avoid what's called "integrator windup". That's a condition where the controller output has hit its clamping limit, but the error is not zero, and @sum_error keeps counting up and up. Then when something changes in the system and your error swings the other way, you want the controller to respond quickly but it can't, because the integral term keeps it locked into maximum output until @sum_error eventually falls back down.
(2) The derivative term is implemented correctly, but when I implement a PID controller, I like to always add some low-pass filtering to it. The reason is because derivative term amplifies high-frequency noise in the system, so that when @kd is set as high as it needs to be to stabilize the system, the output contains lots of jittery high-frequency content. That might not matter when controlling something like a heater, but for some actuators it can result in excess noise, vibration, and wear.
Anyway, great work! I hope I'll be able to see it in action controlling something!
I'm in the middle of finalizing the integration of device_input as a dependency of driving_physics. Then, you'll be able to demo this stuff pretty easily if you're familiar with e.g. `git clone` and can get `ruby` on your system.
On the technical points, I bounced this off another practitioner and his comments pretty much echo yours. Hearing it again and from a different angle makes more sense now.
It's funny, I resisted the integral stuff because my version *seems* more elegant, particularly in the code structure and symmetry. And struggled to imagine the case where it mattered.
Also, because of my choice there, it felt right to cleave StatefulController from the PID specific stuff. But they are clearly intertwined.
So moving ki makes sense; I'd like to support "no surprise" tuning, and I have vague notions of very practical issues with sudden discontinuity.
On clamping sum_error, I'm still not sure. This doesn't seem to be a problem as I reset the sum when the error flips.
I hadn't heard this on the D term, and that explains my complaints. By the time kd is high enough to damp, it's oscillating at 1000 Hz / 2 as error drops to zero along with P and I, thus dominating the output.
I have some audio engineering background from university era, along witt vague notions of some DSP stuff. Certainly that was my intro to *-pass filters, purely in terms of audio. It blew my mind when I realized other types of signals are directly analagous and have noise, etc.
I'm wondering what my low pass filter looks like. A 2 tick moving average?
> On clamping sum_error, I'm still not sure. This doesn't seem to be a problem as I reset the sum when the error flips.
Oh, I totally missed that on the first read! I think that's a big problem, actually.
Resetting the sum does prevent runaway integrator growth but I'd be very concerned about the effect that it has on the integrator output. Even though the error is changing sign (and thus near zero), the sum could be arbitrarily large at the point where this happens -- and maybe it deserves to be large, because maybe the only reason the error crossed zero is because your integrator has finally accumulated enough to overpower a persistent disturbance acting against your system.
Picture the controller arm-wrestling some disturbance. It's a tight battle, but as time goes on the integrator gains strength and the error is getting wrestled down toward zero; your controller is about to win...
Abruptly clearing the integrator is like kicking the support out from underneath your controller right in its moment of triumph, and now all of the force is still pushing you back and your controller is suddenly doing nothing to resist (as both proportional and integral terms are zero). Your arm flies backward and you have to start the arm-wrestling process all over again.
> I resisted the integral stuff because my version *seems* more elegant
I totally get where you're coming from! I also like to modularize controllers into nice abstract units, but it's challenging because of some subtle interactions that come up in edge cases. Everything works fine in the base case, but then when you hit some speed limit, voltage limit, system reset, parameter adjustment, etc, suddenly the abstractions break down. It really makes elegance hard to achieve sometimes.
> I'm wondering what my low pass filter looks like. A 2 tick moving average?
That costs very little and can work quite well, so I'd start there! If you end up needing more smoothing than that, you can use the steady-state Kalman filter formula in this post (the equation after the sentence ending "a linear filter with constant coefficients"). That implements a generic first-order low-pass filter that can work for anything.
For your purpose, just replace Tₖ with self.derivative, μₖ with self.derivative_filtered_prev, and then use μₖ₊₁ as the filtered output of your derivative controller that gets passed to the controller output. Then you can tune the filter pole by adjusting Kₛₛ. (Of course, I'm sure you won't love having another parameter to tune.)
There, I've got a simulated motor with rotating mass, friction, and a torque curve. It stalls below 500 RPM and requires a starter motor to spin it up to 1000 RPM idle. What throttle % maintains that idle?
I started out with a comment "# we don't need no pid control" and then wrote a 7 line kludge of a P controller with epicycles. I couldn't remember how PID was implemented exactly but remembered the concept and a prior toy impl lost to the sands of time.
I pounded my head against the wikipedia article for too long, reviewed a couple impls that I found unsatisfying, and decided to write a clean, minimal, comprehensible implementation. It was originally within driving_physics, but I decided to spin it out as its own project.
Throttle goes 0-1 so that's my output clamp. Only friction slows the motor below 1500 RPM; above 1500 RPM is a crude engine braking effect at 20% of the nominal torque when off throttle.
I have spent far too long playing with tunables in seek of perfection. All of my tweaking has done little to achieve 100% perfect behavior, but it's all been at like 99.9%. So very robust and amazing performance.
I've settled on P clamp(-1, 1), I clamp(-0.5, 0.5) D clamp(-0.25, 0.25) as just being logical and robust. Surprisingly to me, it works really well with no clamps and default gain.
I love the technical comments and will respond separately.
Hey Jacob!! I was scouring the internet for this exact article, and lo-and-behold it's written by the legend I used to work with (you derived a lead-lag controller on a whiteboard when I was a co-op student off the top of your head). Thank you for the awesome read. I was thinking of trying to replace a low-pass butterworth filter with a kalman filter in a system with no known inputs but lots of disturbances and measurement noise, just like the example you chose. The discrete math is going over my head for now, I've been relying on diagrams and python state space / simulation libraries to give me my answers so far, but will be implementing it in a more real-time fashion soon and will definitely be referencing your article!
Hey Eric! Great to hear from you, I'm glad you enjoyed the article and I hope all is well!
Good series of articles.
In recent years, Ive moved from Kalman to Sav Golay filters, which in many circumstances work as well but are computationally much simpler. They also have a few additional 'tricks' such as providing the first & second differential signal.
We use such filters in motion control applications.
In practical terms, I find often that when trying to optimise control for a particular piece of equipment that often the best estimate you have of performance over the next x periods is a moving average of that last 5-20x periods.
This gives you a known near to exact correct state. Then you trim this with your choice of predictive controller.
I wonder what the simplest example would be of a system where a Kalman filter is worthwhile? When do we know that a fixed filter isn't good enough?
That's another very good question. I'm not sure where to draw the line for the very simplest, but for example, I'd be surprised if you could solve the ship problem[1] or even the lighthouse problem[2] with a constant-coefficient filter -- at least not unless it was initialized with a very good initial guess. The solution to those problems involves observations with coupled errors (α, β, and for the ship, v), and if you don't accurately keep track of their covariances, the filter output can easily diverge.
That suggests to me that using a steady-state Kalman gain wouldn't work there. But I haven't attempted it, so I'm hesitant to say it can't be done!
[1] https://jbconsulting.substack.com/p/the-kalman-filter-on-a-moving-target
[2] https://jbconsulting.substack.com/p/voiding-the-warranty-how-the-kalman
The math is a bit beyond me, but it sounds like Kalman filters are in some ways self-calibrating. So, perhaps the simplest example might be when a simple filter would work, except that one parameter needs to be adjusted on the fly, due to changing conditions?
Background: I'm a software engineer working on a musical instrument as a hobby, so I've been experimenting with sensors. I wrote up one experiment I did here: https://observablehq.com/@skybrian/a-sine-cosine-encoder-experiment?collection=@skybrian/digital-signal-processing
Hi Jacob,
I really like your writing and technical analysis. I found my way here via HN linking your "right half plane zeroes" post. To be honest, none of the mathematics landed with me, never having studied control theory and not having even attempted to work my way through your post, but the 40% or so that I grokked was quite interesting.
I'm a software developer with a wide variety of technical interests that definitely includes control theory. Who doesn't love inverted pendulum and PID controllers? I'm 20 years outside my CS degree and only discovered PID about 5-6 years ago. Basically, I'm at the level of, positive feedback bad, negative feedback good.
Anyway, I recently published a PID implementation, mainly as an exercise and also because I needed one for a different project. https://github.com/rickhull/device_control
I thought you might find it interesting, mainly for pedagogical purposes, and I'd be very receptive to any feedback you have, positive or negative, heh.
I'm still trying to wrap my head around Kalman filter, and that might be the next thing I tackle. I'm hoping it's relatively tractable like PID in terms of software implementation. I pored over the dang PID wikipedia article multiple times, and it always seemed way more complicated to understand or suggest an implementation, compared to the rather simple implementation I landed on.
Thank you for your kind words!
I wouldn't worry too much about following the math. I try to aim my writing so that the text, pictures and videos convey the ideas without needing to read any of the equations. But I know that some people prefer to see the math, or might be trying to follow along while implementing something themselves, so I include it for them too.
And great work on implementing a PID controller! The main reason I set out to write this newsletter was to share the underlying mindset and intuition that control theory has helped me develop, which I find so useful in thinking about the world and everyday life. Interacting with PID control is a really good way to get a sense of the core teachings of control theory, and if everyone spent time playing with them then I probably wouldn't need to write anything at all!
Unfortunately I don't have any experience with Ruby so I'm kind of guessing a bit, but I think this looks like a great start. I'm curious what project are you using it for?
Here are some specific thoughts, if you don't mind me sharing them:
(1) When I'm looking at a PID controller the integrator implementation is the first place I look at, because it's the easiest place for subtle problems to creep up.
> def integral
> (@ki * @sum_error).clamp(@i_range.begin, @i_range.end)
Since your @ki gain is tunable, this formulation could give you an unexpected surprise when you adjust the tuning on the fly. I recommend either using a pre-multiply formulation like this:
> @sum_error += @ki * @error * @dt
or else implement a "bump-free" tuning function where @sum_error gets recalculated whenever @ki is adjusted, such that the output stays continuous. Otherwise you'll get a step-change in your output proportional to the present value of @sum_error.
Also, I'd suggest clamping @sum_error, to avoid what's called "integrator windup". That's a condition where the controller output has hit its clamping limit, but the error is not zero, and @sum_error keeps counting up and up. Then when something changes in the system and your error swings the other way, you want the controller to respond quickly but it can't, because the integral term keeps it locked into maximum output until @sum_error eventually falls back down.
(2) The derivative term is implemented correctly, but when I implement a PID controller, I like to always add some low-pass filtering to it. The reason is because derivative term amplifies high-frequency noise in the system, so that when @kd is set as high as it needs to be to stabilize the system, the output contains lots of jittery high-frequency content. That might not matter when controlling something like a heater, but for some actuators it can result in excess noise, vibration, and wear.
Anyway, great work! I hope I'll be able to see it in action controlling something!
I'm in the middle of finalizing the integration of device_input as a dependency of driving_physics. Then, you'll be able to demo this stuff pretty easily if you're familiar with e.g. `git clone` and can get `ruby` on your system.
On the technical points, I bounced this off another practitioner and his comments pretty much echo yours. Hearing it again and from a different angle makes more sense now.
It's funny, I resisted the integral stuff because my version *seems* more elegant, particularly in the code structure and symmetry. And struggled to imagine the case where it mattered.
Also, because of my choice there, it felt right to cleave StatefulController from the PID specific stuff. But they are clearly intertwined.
So moving ki makes sense; I'd like to support "no surprise" tuning, and I have vague notions of very practical issues with sudden discontinuity.
On clamping sum_error, I'm still not sure. This doesn't seem to be a problem as I reset the sum when the error flips.
I hadn't heard this on the D term, and that explains my complaints. By the time kd is high enough to damp, it's oscillating at 1000 Hz / 2 as error drops to zero along with P and I, thus dominating the output.
I have some audio engineering background from university era, along witt vague notions of some DSP stuff. Certainly that was my intro to *-pass filters, purely in terms of audio. It blew my mind when I realized other types of signals are directly analagous and have noise, etc.
I'm wondering what my low pass filter looks like. A 2 tick moving average?
> On clamping sum_error, I'm still not sure. This doesn't seem to be a problem as I reset the sum when the error flips.
Oh, I totally missed that on the first read! I think that's a big problem, actually.
Resetting the sum does prevent runaway integrator growth but I'd be very concerned about the effect that it has on the integrator output. Even though the error is changing sign (and thus near zero), the sum could be arbitrarily large at the point where this happens -- and maybe it deserves to be large, because maybe the only reason the error crossed zero is because your integrator has finally accumulated enough to overpower a persistent disturbance acting against your system.
Picture the controller arm-wrestling some disturbance. It's a tight battle, but as time goes on the integrator gains strength and the error is getting wrestled down toward zero; your controller is about to win...
Abruptly clearing the integrator is like kicking the support out from underneath your controller right in its moment of triumph, and now all of the force is still pushing you back and your controller is suddenly doing nothing to resist (as both proportional and integral terms are zero). Your arm flies backward and you have to start the arm-wrestling process all over again.
> I resisted the integral stuff because my version *seems* more elegant
I totally get where you're coming from! I also like to modularize controllers into nice abstract units, but it's challenging because of some subtle interactions that come up in edge cases. Everything works fine in the base case, but then when you hit some speed limit, voltage limit, system reset, parameter adjustment, etc, suddenly the abstractions break down. It really makes elegance hard to achieve sometimes.
> I'm wondering what my low pass filter looks like. A 2 tick moving average?
That costs very little and can work quite well, so I'd start there! If you end up needing more smoothing than that, you can use the steady-state Kalman filter formula in this post (the equation after the sentence ending "a linear filter with constant coefficients"). That implements a generic first-order low-pass filter that can work for anything.
For your purpose, just replace Tₖ with self.derivative, μₖ with self.derivative_filtered_prev, and then use μₖ₊₁ as the filtered output of your derivative controller that gets passed to the controller output. Then you can tune the filter pole by adjusting Kₛₛ. (Of course, I'm sure you won't love having another parameter to tune.)
Updated here, I think I got everything: https://github.com/rickhull/device_control/blob/master/lib/device_control.rb#L165
Thanks for the pointers!
Something like this should work:
PREREQS:
# check the version of ruby and its package manager, gem
ruby --version
gem --version
# if these aren't available or not in PATH, figure out how to make them available
# install device_control (minimum v 0.3)
gem install device_control
# clone driving_physics
git clone https://github.com/rickhull/driving_physics
cd driving_physics
# run demo
ruby -I lib demo/pid_controller.rb
# when in doubt, press [Enter]
For the PID models, these are the Ziegler-Nichols models from wikipedia. 'some' and 'none' refer to overshoot. I got 'none' to converge at 1000 RPM
Try different setpoints and gain knobs. You can also bounce the RPM to wherever you need it.
Wow, great to hear back. First off, my larger project is https://github.com/rickhull/driving_physics
There, I've got a simulated motor with rotating mass, friction, and a torque curve. It stalls below 500 RPM and requires a starter motor to spin it up to 1000 RPM idle. What throttle % maintains that idle?
I started out with a comment "# we don't need no pid control" and then wrote a 7 line kludge of a P controller with epicycles. I couldn't remember how PID was implemented exactly but remembered the concept and a prior toy impl lost to the sands of time.
I pounded my head against the wikipedia article for too long, reviewed a couple impls that I found unsatisfying, and decided to write a clean, minimal, comprehensible implementation. It was originally within driving_physics, but I decided to spin it out as its own project.
Throttle goes 0-1 so that's my output clamp. Only friction slows the motor below 1500 RPM; above 1500 RPM is a crude engine braking effect at 20% of the nominal torque when off throttle.
I have spent far too long playing with tunables in seek of perfection. All of my tweaking has done little to achieve 100% perfect behavior, but it's all been at like 99.9%. So very robust and amazing performance.
I've settled on P clamp(-1, 1), I clamp(-0.5, 0.5) D clamp(-0.25, 0.25) as just being logical and robust. Surprisingly to me, it works really well with no clamps and default gain.
I love the technical comments and will respond separately.
Cheers!
Wow, a driving physics simulator is a very ambitious project! I hope to give it a spin soon. Good luck!