> On Apr 19, 2016, at 3:18 PM, Hendrik Boom <hend...@topoi.pooq.com> wrote: > > On Mon, Apr 18, 2016 at 08:04:32PM +0200, Jerzy Karczmarczuk wrote: >> Dmitry Pavlov wants "adult" numerics in Racket, and he adds: >> >>> - I need to take derivatives of equations that I wrote in my DSL, >>> symbolically, and have them converted to C too. >> >> And then, people (Robby Findler and John Clements) mentioned Siskind. Good. >> >> However, you should know that Jeffrey worked NOT on /symbolic/ >> derivatives, but mainly on the "automatic differentiation" stuff; >> this is numeric, but fast and exact (in the sense: no differential >> quotient approximations). There is plenty of valuable papers written >> by Jeffrey and Barak Pearlmutter... (But some are quite difficult). > > I am baffled how you can do differentiation numerically and get it exact.
I think that calling it “numeric” may call to mind something that is not exact (e.g., choose a small dx, take two nearby values, divide the difference by dx). This is definitely not exact. Automatic differentiation—and be forewarned, I’m going to biff some important detail or other—is instead an algorithm that computes the same answer as symbolic differentiation without having to take the extra step of actually constructing the formula. I think, however, that it is “morally equivalent” to symbolic differentiation. So, for instance, suppose you’re computing 3x^2+x. This might involve multiplying 3 by x by x and adding x. Suppose further that x=4. Automatic differentiation associates with each term computation a side computation that represents the derivative of that term, using an inductive computation. In this example, we start by multiplying 3 by x. If we know that the derivative of 3 is 0 and the derivative of x is 1 (we’re using the inductive hypothesis here), then for not much more than the cost of multiplying 3 by x we can also have (3 * 1)+(0 * 4), so we now know that the value of this term is 12 and that the derivative of this term is 3, and we’re ready for the next inductive step. Can I now just say “et cetera” ? I believe these are the basic ideas behind automatic differentiation. John -- You received this message because you are subscribed to the Google Groups "Racket Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to racket-users+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
signature.asc
Description: PGP signature
