Basic Calculus in C (2024-07-10)

Numerical Methods of Differentiation

I just wrote this up because I was curious about how to code calculus stuff which I had never done before. This lead me to falling down a rabbit hole on numerical methods.

Anyone who has taken Calculus in school is familiar with some numerical methods of differentiation or integration. The basis of the definition of a derivative is itself an approximation:

\[ f'(x) = \lim_{{h \to 0}} \frac{f(x+h) - f(x)}{h} \]

A secant line where the distance between the two intersections is an infinitesimal.

I've recently learned that this is a class of method known as the Finite Difference Method where a derivative is calculated using the same difference quotient with a finite distance that is sufficiently small:

\[f'(x) \approx \frac{f(x+h) - f(x)}{h}\]

New to me is the Symmetric Difference Quotient which also approximates the slope, but by using two secant lines equidistant from x.

\[f'(x) \approx \frac{f(x+h) - f(x-h)}{2h}\]

The accuracy will be off by a proportion of h using the former method and by h^2 using the latter one.

The accuracy will also be affected by the choice of forward, back, or central difference, round-off error, or truncation error, which funnily enough is often notated using big-O.

Numerical Methods of Integration

Riemann Sums is the other numerical method that is often learned to introduce the concept of an integral. And has been in use since antiquity - like when Archimedes approximated the area of the circle through methods of exhaustion.

Riemann sums partition the area under a curve either from the left, right, or midpoint and if those partitions or slices are sufficiently small, you get a good approximation. This method utilizes rectangles:

\[\int_{a}^{b} f(x) , dx \approx \sum_{i=0}^{n-1} f\left( \frac{x_i + x_{i+1}}{2} \right) \Delta x\]

But can utilize other polygons such a trapezoids:

\[\int_{a}^{b} f(x) , dx \approx \frac{\Delta x}{2} \left[ f(x_0) + 2 \sum_{i=1}^{n-1} f(x_i) + f(x_n) \right]\]

There are a plethora numerical methods for both differentiation and integration that have been developed over the centuries that have their applications in engineering, manufacturing, physics, and computer science.

Numerical Methods in C

Since these methods discretize functions they are how one could create algorithms to perform differentiation or integration. For example:

// Forward difference method
double forward_difference(double x, double h) {
    return (f(x + h) - f(x)) / h;
}

// Backward difference method
double backward_difference(double x, double h) {
    return (f(x) - f(x - h)) / h;
}

// Central difference method
double central_difference(double x, double h) {
    return (f(x + h) - f(x - h)) / (2 * h);
}

These c function will approximate a derivative given some function.

How about integration:

// Trapezoidal rule function
double trapezoidal(double a, double b, int n) {
    double h = (b - a) / n;
    double sum = 0.5 * (f(a) + f(b));
    
    for (int i = 1; i < n; i++) {
        sum += f(a + i * h);
    }
    
    return sum * h;
}

This will calculate an integral given a number of sub-intervals.

Or this:

// Simpson's rule function
double simpsons(double a, double b, int n) {
    double h = (b - a) / n;
    double sum = f(a) + f(b);
    
    for (int i = 1; i < n; i++) {
        double x = a + i * h;
        if (i % 2 == 0) {
            sum += 2 * f(x);
        } else {
            sum += 4 * f(x);
        }
    }
    
    return sum * h / 3;
}

Which does the same according to Simpson's Rule.

In Conclusion

Numerical methods seem essential to most calculus applications in the real world, however it isn't emphasized at all in the university calculus series. I'm glad I read up on it. I may update this with more advanced methods translated to c.