Re: [ng-spice-devel] FYI: backward Euler, trapezoidal, Gear
A few weeks ago Steve Hamm posted here a response to a question in
apparent oscillation using trapezoidal integration, that goes away
with Gear integration.
I present here another view, that really says the same thing.
What is really happening in so-called numerical integration, which is
really differentiation, is the conversion from continuous time,
S-domain, differential equations to discrete time, Z-domain,
difference equations.
In the S-plane, it is well known that the system is "stable" (the
response decays out with time) with poles (roots of the denominator)
on the left (negative real part), and is "unstable" (the response
grows with time) with poles on the right (positive real part). For
poles on the boundary, it remains steady. The "DC steady state"
point is (0,0).
Consider what happens when I take a first order system and move the
pole along the real axis. At the origin, the impulse response is a
DC voltage. For negative values, it decays. The more negative, the
faster it decays. At negative infinity, it decays instantly, so
response matches its input exactly. For positive values, it grows.
The more positive, the faster it grows.
If I have a conjugate pair of poles on the imaginary axis, I get a
steady state oscillation, with a frequency that increases as they
move away from the origin.
I hope the last three paragraphs were review, and you are wondering
why I bothered to repeat it here.
Now, let's consider the same discussion for the Z-plane. If you have
studied digital signal processing, this too will be review, but many
EE's have not.
In the Z-plane, the system is stable if the poles are inside a unit
circle, centered at the origin, and unstable if the poles are outside
the unit circle. The "DC steady state" point is (1,0).
This seems to say that the correct mapping from S to Z is to map the
S imaginary axis to the Z circle, such that the S left half is all
inside the Z circle, and the S right half is all outside, with the
S (0,0) mapping to the Z (1,0). This is what the trapezoid
rule does.
I mapped S to Z for this explanation. The concept can be further
clarified by mapping Z back to S. This is an exercise left to the
reader. :-)
Looking deeper, you will see that there is no perfect mapping, and
there is more to the story.
When I take a first order system and move its pole along the real
axis, for x > 1 (outside the circle) it grows. The more positive,
the faster it grows. So far, so good.
When I move its pole in the negative direction, beginning at (1,0),
the response begins decaying. As I move it more in the negative
direction, it decays faster, (ok so far) until I get to the origin
(0,0). At (0,0), the response instantly decays, so the output
reproduces the input. As I continue to move the pole negative, the
response comes back up again, so when it reaches (-1,0) it is again s
stady state. For positive real part, the samples are all the same
sign, which gives a nice decay. For negative real part, the samples
alternate in sign, giving the appearance of ringing at the "folding
frequency", and an oscillator at the folding frequency for the (-1,0)
point. Going more negative, it grows with time.
So, this is the problem with the trapezoid rule. Although strictly
it is stable, this ringing doesn't look like what the real continuous
time circuit does. You can move the mapping by changing the step
size, but this is not always a good idea. Typical circuits are
"stiff" which means the poles are spread over a wide range. Usually,
many of them can be ignored. With the trapezoid rule, if you attempt
to ignore one of these, you get this awful ringing. It would be nice
to have a method that allows you to ignore these poles without
getting junk as a result of it.
The key is to change the mapping, so that minus infinity in S maps to
the origin in Z. The simplest method that does this is Backward
Euler. BE maps the imaginary axis in S to a circle in Z that passes
through (1,0) and (0,0). So, moving the pole gives a reasonable
result.
Where BE loses is for the case of poles along the imaginary axis.
They are mapped to this smaller circle, that is entirely inside the
stability circle on the Z plane. This means "oscillators don't". It
also is "first order" in truncation error, while trapezoid is "second
order". This means you need smaller steps to get accuracy.
Actually, the lower order helps you when the error is high.
So, what if we don't map to a circle? Gear doesn't map to a circle.
For Gear-2 (a second order method) the S-imaginary axis ROUGHLY
maps to the unit circle close to the S-origin (Z (1,0)), but then it
diverges, and the S minus infinity maps to (0,0) as desired. This is
often a good compromise, but it still doesn't do oscillators well.
Oscillators tend to decay, but not as bad as in BE. Higher order Gear
methods do not do as well, and have really strange mappings.
All methods can be derived from polynomial interpolation. They use
combinations of the current value and past values, of the function
and derivatives. The methods differ in their choice if which, and
their weightings.
Runge-Kutta is a very poor choice, and has no place in circuit
simulation. The mapping places a large part of the S stable region
outside of the Z unit circle, mapping the stiff poles into the
unstable region. Besides that, it is harder to implement in this
context.
Nothing is perfect. That is why simulators provide a choice. My
rule: If you want accuracy, and can control the stepping
appropriately, use trapezoid. If you know it is a "stiff" problem,
use Backward Euler, and ignore truncation error. If you have a mixed
problem, you are not trying to build an oscillator or high Q
filter, and you do care about truncation error, use Gear.
Ideally, a simulator should give you a choice, component by
component, and provide a mechanism for excluding some from truncation
error time step control. Even more ideally, it should choose,
locally, based on time constants, rate of change, and truncation
error.
A few years ago, I taught a course that covered this. One of these
days I need to scan in my notes, and post them. Most of the notes
are done using the old pointed instrument technology.
Partial thread listing: