Differential Equations

Table of Contents

Part 1A Michaelmas

This is the first term of the Mathematical Tripos you should be taking this with the other first term courses at the same time as Cambridge designed it this way for a good reason.

Course

This will be much longer than usual because we have to learn calculus before we can start learning differential equations. If you already know calculus then skip that part.

Schedule recommended books

  • J. Robinson An introduction to Differential Equations. Cambridge University Press, 2004
  • W.E. Boyce and R.C. DiPrima Elementary Differential Equations and Boundary-Value Problems. Wiley, 2004
  • G.F.Simmons Differential Equations (with applications and historical notes). McGraw-Hill 1991
  • D.G. Zill and M.R. Cullen Differential Equations with Boundary Value Problems. Brooks/Cole 2001

Strategy

  • The free book The Calculus Integral since we have to learn integration anyway for differential equations and convergence for other courses in this same term
  • This full course contains some of the Boyce book and also has boundary value problems and matches better with 1B content we'll be doing later
  • A problem bank with solutions
  • The SciML book and YouTube lectures to fill in the discrete differential equations content like recurrence relations/difference equations.
    • Teaches Julia for the computational projects we'll have to do later too.

Cambridge Lecture 1

We have access to the first Cambridge lecture here where he covers a review of calculus we're supposed to know. You can watch this after Calculus 101 if you want it doesn't matter.

First definition: the limit of the ratio f(a + h) - f(a) over h as h gets closer to 0 is the derivative. 'x naught' or x0 notation means the same thing as f(a) they both represent a specific point/input but this is arbitrary (any point) so it's \(f(x_0)\) or \(f(a)\). The h represents a tiny decimal that is tending towards h. Here are the LH and RH limits.

Lets go through the f(x) = |x| example. Plug that into the derivative equation for f(0) = |0| we get (|0 + h| - |0|)/h or |h|/h. However h is negative because it's approaching 'negative 0' from the left side where all the x values are negative so it's (h/-h) or -1.

Try an example for little-o where f(x) = x and g(x) = x2 so f(x)/g(x) is x/x2 = 1/x and if you crank inputs x to infinity then the limit is 0.

Try an example for big-O where f(x) = x and g(x) = x then f(x)/Mg(x) where M is some constant is x/Mx = M or their ratio differs by a constant M meaning g(x) bounds f(x) by a constant factor. He remarks the notation O(g(x)) means an entire class of functions. O(x) represents all linear functions, O(x2) all quadratic functions, O(1) all constant functions. The definition allows us to set the constant M to any value we want and is solely included to eliminate constants from big-O notation. If f(x) = 1000x and g(x) = x then f(x) is in O(x) you don't write O(1000x) as any arbitrary constant is included in the definition. This means O(x) + O(x) or O(2x) is still O(x).

Example 3 if f(x) = x2 then plugging it into the derivative equation:

  • \(\frac{(x + h)^2 - x^2}{h}\)
  • \(\frac{x^2 + 2xh + h^2 - x^2}{h}\)
  • \(\frac{2xh + h^2}{h}\)
  • = 2x + h then take the limit of h to zero
  • = 2x + 0

Since h is tending towards 0, before we take the limit we know the remainder term h2 is going to be so small it will approach 0 before we take the limit of h to zero so this is 2xh/h + o(h) though usually h2 is just ignored/junked. Try examples if h = .0001 then check what (.0001)2 is.

Some rules are introduced like chain/product rules. He says to look over the notes and see how big-O is used in the proofs which we will do but first we have to learn calculus.

Calculus 101

There's a sheet we have to try after called Practice Integrals.

Derivatives

Go to desmos.com and make a graph f(x) = x2 and notice this is not linear because it does not have the superposition property. A (line)ar line would be a straight line not a curved line meaning if you scaled it with multiplication such as 3(f(x)) and f(3x) the result would be the same. The input f(2) + f(2) and f(2+2) would also be the same if this were linear.

Let's try super zooming in on a point in that graph so all the surrounding points in it's neighborhood appear to be sitting on a straight line thus a linear approximation. Add the point (2, 4) and the point (2.001, 4.004) then zoom that point as I have done here.

When you step off the graph to the horizontal right direction of the x-axis by 0.001 you have to move up 0.004 in order to meet the graph again. Call these dx = 0.001 and dy = 0.004 then dy = 4dx or dy/dx = 4. This is the rise/run slope of the linear approximation we made and is the derivative of the function at that point.

Try more examples, plot (a,b) then (a + dx, b + dy) and super zoom to see where b should be moved to meet the graph:

  • (1.001, 1.002) dy = 2dx
  • (2.001, 4.004) dy = 4dx
  • (3.001, 9.006) dy = 6dx
  • (4.001, 16,008) dy = 8dx

In every case dy = (2x)dx or dy/dx = 2x. The derivative of the entire function f(x) = x2 is f'(x) = 2x where that ' tick denotes derivative or using our differentials notation \(\frac{dy}{dx}x^2 = 2x\)

To guarantee you will always have a straight line approximation these dx quantities were originally thought to be so small they are infinitesimal meaning dx = 0.001 but instead of 2 zeros you add 'infinitesimally many zeros' meaning dx has the property that it's extremely small but not actually zero. The product of two infinitesimals is zero since if you recall decimal multiplication you would shift the decimal point left adding an infintesimal amount of more zeros to an existing infintesimal and now it's just zero since it is indistinguishable from nothing. Adding two infinitesimals gives another infinitesimal since you aren't moving the decimal point. The problem with this definition is there does not exist any numbers with the property that when you multiply a non-zero quantity with another non-zero quantity it becomes zero. Thus real numbers and limits were invented so now we can pretend these infinitesimals are approaching zero and take their limit as 0.

That's really the story of derivatives in 2D or scalar calculus they are a way to approximate a non-linear function with this idea you can create enough tiny straight lines that are at all points tangent to the original function.

In vector or multi-variable calculus we now have 3D objects and their linear approximation is a super zoomed (x, y, z) point who's neighbors are now on a flat surface or an entire 2D plane as depicted in GeoGebra.

The truth about derivatives

In reality a typical continuous real function is nowhere differentiable not even using generalized derivatives like approximate or symmetric derivatives. A derivative doesn't need to be continuous either there is whole classes of them that are discontinuous almost everywhere and closed under addition but not under multiplication or composition and they exhibit a behavior which we will see later in matrix calculus.

If you want to learn the theory of derivatives there is no better book than Differentiation of Real Functions 2nd edition by Andrew M. Bruckner. We will take his analysis book in 1B.

Modern AI is using nonsmooth analysis. It is the study of differential properties of functions in a more generalized setting because almost every practically relevant function like the absolute value function or the max function are not differentiable. To understand this analysis we need to know functional analysis and the calculus of variations both which are taught in the Tripos eventually.

Integrals

Integration is the undo method for derivatives to recover a function from it's derivative using a covering relation. The definite integral is recovering a function for a specified range and produces a scalar that can represent the area underneath the original function in that range. In applications like probability and physics this has many interpretations and integration will come up in every future course.

Oddly enough we have to learn integration before we learn what a derivative truly is because how do you know the function you are looking at is a derivative? What characteristics does a derivative have? Newton's answer to this was to characterize a derivative by using an integral saying F is the integral of f if every derivative of F is f therefore f is a derivative and in many textbooks like Apostol's books you learn integration before differentiation.

The Calculus Integral

We are learning a generalized integral called the Newton integral which is really the Denjoy-Perron integral which is really the Henstock-Kurzweil integral. This book will help us learn highly abstract analysis later in the tripos like functional analysis and measure theory. We still learn the standard Riemann sums approximation just now we get to avoid lot's of nonsense like 'improper integrals' which don't exist and we won't waste our time learning an undergrad and high school integral that is tossed on the first day of grad instruction.

Obtain The Calculus Integral a free modernized calculus text from B.S. Thomson we have to meticulously work through the entire book completing the tripos example sheets for integration as we go.

TODO

Home