
David G. answered 03/14/19
OChem, Calc, ACT/SAT/GRE-years of exp. w/all ages, former OChem TA@UCB
The ε-δ definition was the product of some collaboration, direct and indirect, but it's typically attributed to Cauchy. Ultimately, the problem we have in differential calculus is the idea of infinity, and the ε-δ definition is a way of formalizing that. It's funny you bring up circles, because Archimedes's proof has a lot in common with the ε-δ definition.
The idea is this: infinity means "we can always go further". Archimedes's proof of the area of a circle inscribes a regular polygon in and circumscribes a similar regular polygon around a circle, stating that the area of the circle is somewhere between the areas of the inscribed and circumscribed polygons. From there, we can narrow down the range of predicted areas for the circle by increasing the number of sides of the regular polygons. We can't literally do this forever, but we can "always go further", and the trend will continue in the same fashion (this is the critical point).
The ε-δ definition of a limit depends on differentiability. A function is differentiable in a region R if it is piecewise smooth (no matter how the function is broken up, the derivative is the same for each adjacent piece at the boundaries), continuous, and defined in that region. Differentiability allows us to state that, as we change the input, the output varies in a predicatble, consistent fashion.
So, The idea is that we can always make δ smaller, and ε will get smaller in a predictable way. Because this is a predictable process, we don't literally have to do it forever, we just have to say that we can.
This is why xsin(1/x) has no limit at x=0: we can keep making δ smaller, but ε will go up and down and up and down and up and down and never follow a consistent pattern, so we can't just say "we can keep doing this and get closer to the answer".