Welcome back.
Welcome back for week four of Modeling Complex Systems.
This week, we're gonna be looking
at continuous dynamical systems,
so instead of using discreet rules
that we update for every hour of every day
with some time window,
we're gonna decrease those time window,
so that we have infinitely many small steps
and effectively continuous time.
That's gonna open the entire toolbox
of calculus and differential equation
to help us study these systems.
So this is the bonus video,
that I originally referred to during week three
that was unfortunately recorded without sound,
so you're getting it in week four.
We're gonna be looking at some mathematical tools
that help us take those limits of very small steps.
The first tool we're gonna be looking at are Taylor series,
and here, I wanna highlight
like three important part of the name of this tool.
First of all, that it's a series,
so what that means is that
formally, it's gonna be a power series.
You just have to remember
that we're looking at polynomials essentially,
where we're gonna use the structure of a polynomial.
And this is a trick we're gonna use again
in a different context,
we're gonna use the structure of a polynomial,
where it's easy to take derivatives and all that
to store important pieces of information.
And here, we're gonna store information
about a key function, f of X,
which might simply be too complicated to study efficiently.
So if you have a function,
you wanna take a limit
and its behavior is sort of unknown,
it's a weird function,
then Taylor series can help you do that.
They're gonna help you summarize this function
in a much easier way.
And the last key part is that
it's really valid around one given point,
so if we wanna take a limit of something going to zero,
for example, well that's gonna be convenient,
'cause we know that for power series for polynomial,
if all coefficients go to zero,
well, that's easy to deal with.
We only care about the smaller order terms.
So that's really the key part.
So to summarize here,
we're gonna be looking for a polynomial,
such that the value of a function,
which I'm gonna write f of X0 and all derivatives,
which here, I'm gonna write f superscript n.
So f(1) would be the first derivative,
the slope of that function at X0.
F(2) would be the curvature of that function at X0
are respected or conserved or preserved
at x = X0, okay.
The insight really is that
we're looking at a function, f of X not f of X0, f of X,
that might be very complicated.
Let me just draw whatever, but it's a function,
sometimes very steep, sometimes not at all.
It's a weird function that might be hard to deal with,
and what we're saying is that,
if I look at a particular value, X0,
if I know where I am and I know the slope
and the curvature and all the derivatives,
I can get a really good local picture
of that function, right, even just the slope.
If you know where you are at X0
and you know in which direction you're going
and how steep you're going,
then it's easy to project your position,
like do a small step in delta X times the slope,
and you get your delta Y kinda thing, right.
So, that's the key insight,
that's what we're looking for here,
and often when people present Taylor series,
they're only gonna give you a formula.
It's almost like a recipe,
like apply this and you're gonna get a polynomial
that reflects your function but is easier to deal with.
The truth is it's super easy.
Looking it up on Wikipedia is easy,
but deriving the Taylor series is almost just as easy.
So we're looking for
an approximation of f of X,
so I'm gonna use this approximation here,
such that if I plug X of X0,
at least I want to to know that I get the right value.
So I'm gonna start with f of X0.
If I plug X0 here, I get the right value.
The problem is that right now, if I plug any other value,
I still get f of X0
'cause I don't have any derivatives right now.
So as I just said, one thing you can do,
lemme go back, back.
One thing you can do is just apply your derivatives,
the first derivative at X0,
so that's the slope of my function, times a change in X,
a delta X, to get a delta Y, right.
That's what the derivative means.
And now, if I plug anything other than X0,
I'm using this first order slope
to sort of project or extrapolate from my position,
so that's good.
It's also good because, if I take the derivative,
this is just a constant, so this goes away.
The derivative of this term is one.
So if I take the derivative of my polynomial here,
I know that I'm gonna get the right value here, f prime,
so we're doin' great.
If we take the second derivative though,
right now, we're at zero,
'cause we don't have any second order term,
so we wanna introduce a second order term,
oops, which is also gonna be evaluated at X0.
And now, this one is interesting.
Here, I've made it a second order term.
I respect the structure of my polynomial,
so I got this little square here, second power.
But if I take the derivative of f of X,
of my power series, what happens?
The first term goes away, it's a constant.
The second term gives me my f(1) for the first derivative.
I take a second derivative, it goes away.
This term that I just wrote though,
if I take two derivatives,
the two is gonna fall in the front, right,
so f prime prime or f(2) of x is gonna give me
two f(2) of X0, and that's it.
So, I'm getting twice the derivative now,
so really, I can just add 1/2 to control for that.
And now I could add all derivatives,
but if I have the term that's like an X(n),
then if I take n derivatives,
I'm gonna get n times n minus one times n minus two
for all the derivatives of these powers.
So the general form of this is really
just that you include the nth derivative with this form,
which you divide by n factorial,
to control for the fact that your derivatives
are adding factors in front
of the true slope that you want.
And that's it.
The sum of all f(n) of X0 times x minus X0 to the n,
normalize with this n factorial, that's our power series.
It's just a way to store the value and all the derivatives
in a way that makes sense,
and that if you derive your approximation,
you're gonna get the right function.
Here's an example of why that's useful.
So, we dealt a lot with small probabilities,
and I've played a little loose with probabilities,
with spelling probability, okay.
I need more coffee, it's early right now.
I've played a little loose with some of those terms.
Sometimes I would write, for example,
in the case of an SIS model,
I would do one step and I would say,
"Well, to know who's susceptible at time t plus delta t
"or t plus one steps, I take who's susceptible at time t,"
and then I've had multiple version,
but one of them was those that are still susceptible
are those that don't get infected.
So a contact doesn't infect
with probability one minus beta transmission rate
times delta t, the step that we're considering,
and I have I of t susceptible individuals.
And let's say it's a SI model, no recovery,
so this is fine, this is a complete model.
Now if delta, if beta times delta t,
the probability of transition is really small,
really what I can say is that
this is my one minus x to I of t, let's say,
and I can say like,
"Well lemme do a Taylor series around X equals zero,
"so that I'm gonna know what's the behavior
"for very small probabilities, for very small X."
And so well, if you evaluate that X equals zero,
we're gonna get, I'm just gonna write f of X,
I'm just gonna get one.
If I evaluate it at X0 equal to 2(0),
'cause I have one to some power, that's always one.
Now I take my first derivative,
and my first derivative is simply gonna be X times I of t.
And then usually, we would write,
we got some other terms in X square, and that's a big O,
meaning we have terms of order X square,
but because X is already really small,
X square is even smaller,
potentially so small that we ignore it, right.
So that's what we're gonna do.
And what this means is that this equation
could have been written
simply as S of t times one minus beta delta t I,
I'm running out of space here, I apologize.
Let's just go back here.
S of t times one minus beta delta t times I of t.
And sometimes I use this form because it's easier, right,
and the average is respected.
But the important part here is that
there is an approximation,
so let me just highlight it here.
This is approximately equal
to the simulations we would do with discrete time steps
and discrete number of infectious neighbors, okay.
There was one that was even harder
that showed up in a video we had.
This, I just copy and pasted it.
We had a cumulative distribution function
for our Poisson process, right,
and Poisson processes, like formally,
and that was part of the quiz in week three.
They're defined by a rate,
so by the expected time until the next event.
In fact, that's all they're defined by,
so they don't have any memory.
If all that matters is the expected rate
at which the next event occurs or the expected time,
this is, time and rate are just the same thing,
one over the other until the next event,
then it's a Poisson processes.
But it is a continuous time process.
And we were doing this cumulative distribution function
and then showing how,
which we expect is an exponential
or Poisson distribution of recovery time
or of me leaving a building.
And we took this derivative already,
so when time steps get really small, right.
We took this limit, not this derivative, this limit.
Well, so you could play with Taylor series,
doing something similar here.
The important part is that the delta t occurs twice, right,
so it is a complicated function.
And what's interesting here is that
there's one term that goes to one, and that's the term here,
but then the power goes to infinity.
So as time gets really small,
the probability of transition
during every time step is effectively zero.
So nothing happens, the argument goes to one,
but the number of times steps that we do becomes infinite.
And there's a second tool
that's really useful for dealing with that,
and that's the,
I apologize, I have a hard time saying this in English.
So it's the L'Hospital or the Hospital Rule.
Okay, and this is gonna be our example,
this CDF that we dealt with.
Fun fact, a bonus trivia fact as part of this bonus video,
you will also see this written as Hopital Rule.
Basically, in Old French,
we would add a lot of os and es,
that are now, in Modern French,
written with this circumflex
or little hat on the vowel.
So, it's the same in my name.
My last name, Dufresne, means from the Ash Tree,
but in Modern French, we would write so from, Du,
and then Frene, which is Ash Tree.
Okay, fun fact,
but let me go back to Hospitals Rule.
You can Google it with both spelling, that's my point.
So this works really well,
when you have this idea of two terms
that sort of fight each other, right.
When you take the limit, they fight each other.
And the rule is best written
when you're dealing with ratios.
So when you wanna apply this rule,
the first thing to do is to write it as a ratio,
and here, we're just gonna use
a little dumb trick to do that.
We're gonna take the exponential
of the natural logarithm
of our expression.
So taking the exponential of the natural logarithm
means doing nothing, right,
but here, that means that the limit of an exponential,
what you care about
is what the argument of the exponential is going to.
So we can say we're taking the exponential, the limit,
of a logarithm, then logarithm of something to some power,
you can take the power out,
and it applies a multiplicative factor.
So now we get L over delta t,
and then the logarithm, one minus L delta t.
And now we have what we want, so as delta t goes to zero,
the denominator here is delta t goes to zero,
so that makes the whole thing blow up.
But we also get the logarithm,
which goes to one as L times delta t goes to zero,
and logarithm of one goes to zero.
So we have one term, well, we have two terms going to zero,
the numerator and the denominator,
so they're sort of fighting.
And then what L'Hospital's Rule is saying is,
"Well, if you can look at how fast
"they're both going to zero, you're gonna know who win."
If the tops goes, if the numerator goes to zero faster,
the whole thing is going to zero.
If the denominator goes to zero faster,
the whole thing might blow up,
and then you're not only gonna know where they go
but how fast they're going.
So that's the idea.
So more formally saying what L'Hospital's Rule says is
when you have a ratio like this,
take the derivative of the numerator
and the derivative of the denominator,
and then reevaluate your limit.
So if I do exactly that here,
I'm simply gonna get the exponential of the limit
when delta t goes to zero.
The derivative of my denominator,
which is my first term here, is simply one delta t,
it's derivative as a function of delta t goes to one,
and then, I have minus L over
one minus L delta t here at the bottom.
So then if I apply my limit,
my denominator goes to one,
and I get my derivative of minus LL,
which is what we had in the previous video.
So those two rules,
sometimes they give you the same results,
well, they always give you the same result,
but basically is that,
sometimes one is gonna be easier to deal with.
Here we didn't have to do a lot of derivatives,
we just took the derivatives of logarithm term
and the derivative of a linear term,
so that was easy.
And dealing with a function,
like one minus X to some power over X,
would've been a little harder with Taylor series.
So Taylor series is the more general tool,
but L'Hospital's Rule is really good,
when you notice that there's the structure
of two terms fighting against each other.
So that's it for now.
In the next few videos for this week,
we're gonna be applying some of these rules
to show how discrete models and continuous models
are connected mathematically
in a way that's very quantifiable,
so that we're gonna be able to use discrete simulations
to compare with continuous equations and vice versa,
as long as we know
what approximations we're making mathematically.
So that's why these tools
are gonna be super useful going forward.
So, I'll see you in the next one.