[julia-users] Re: Hyper-Dual numbers
Cool. I took a quick look over the code, and it looks pretty good to me. You have an inline comment about wanting understand convert/promote better--one good place to look is promotions.jl in base: https://github.com/JuliaLang/julia/blob/master/base/promotion.jl#L158 This defines some fairly generic methods for +, -, *, /, etc. that mean "if there isn't a more specific method for this combination of argument types, try promoting the arguments to a common type before doing the operation". By defining convert and promote_rule, you are participating in this system. I didn't look carefully at all the promotions/conversions you define, but if you have the right ones, this will allow e.g. something like Hyper(0.0, 1.0, 1.0, 2.0) + Hyper(3//5, 1//2, 1//2, 2//7) to work without you ever having had to write code specific to the Rational type. As for the theory, it seems a little funny to me that this technique redundantly encodes the first derivative, and I can't see an advantage of using two variables that both square to 0 over using a single variable that cubes to 0. Since E1 and E2 are used symmetrically, suppose we make the change of variables e = E1 + E2. Now we have a single variable with e^2 = 2E1E2, and e^3 = 0, and you could go through everywhere making the replacements and end up saving yourself one field in your type, and a bit of redundant arithmetic. After the replacement, it's easier to see how to extend to higher derivatives too. If e^3 = 0 is a variable that helps us compute second derivatives, then f^4=0 is a variable that helps us compute third derivatives, g^5 = 0 helps us compute 4th derivatives, and so on. This is the theory behind the truncated power series in PowerSeries.jl Instead of talking about a lot of different variables with special properties, it's also possible to say that you're working modulo some polynomial. Dual numbers are equivalent to computing mod x^2, 2nd order truncated series are equivalent to computing mod x^3, 3rd order series to mod x^4 and so on. In this way of thinking, complex numbers are polynomials mod (x^2 + 1), and the split-complex numbers, which come up in the hyperbolic geometry of the plane, are polynomials mod (x^2 - 1). On Saturday, March 29, 2014 5:43:05 PM UTC-7, Rob J Goedman wrote: > > Hi, > > As a first 'jump into the fray' exercise I've attempted to translate > Jeffrey Fike's hyper-dual numbers code from c++ to Julia, more or less > following the DualNumbers package. > > The c++ code can be found at http://adl.stanford.edu/hyperdual/hyperdual.h. > The paper itself at > http://adl.stanford.edu/hyperdual/Fike_AIAA-2011-886.pdf . > > The Julia package can be found at: > https://github.com/goedman/HyperDualNumbers.jl.git . > > Of course, I'm pretty new at this so I'm sure there will be errors and > poor practices. So any feedback is appreciated. > > Also, I'm wondering if the type should be called Hyper or a better name > would be HyperDual. > > This work was triggered by the interesting threads around openPP, > TaylorSeries.jl, Calculus2, PowerSeries.jl (and at some time I hope > MCMC.jl). > > Rob J. Goedman > goe...@icloud.com > > > > >
[julia-users] Re: Hyper-Dual numbers
There is another method to calculate any order derivatives, very similar to this one, known as Multi-complex differentiation. This method can calculate derivatives up-to any order. http://www.autodiff.org/Docs/euroad/13rd%20EuroAd%20Workshop%20-%20Thierry%20Dargent%20-%20Using%20Multicomplex%20Variables%20for%20Automatic%20Computation%20of%20High-Order%20Derivatives.pdf http://delivery.acm.org/10.1145/217/2168774/a16-lantoine.pdf?ip=207.151.221.1&id=2168774&acc=ACTIVE%20SERVICE&key=6405B83BDA580DC2%2E4D4702B0C3E38B35%2E4D4702B0C3E38B35%2E4D4702B0C3E38B35&CFID=717380051&CFTOKEN=25446753&__acm__=1443489222_ee3f3d4c7ee41030aaba536e14d6a6ab I have used this in my field and it seems to work nicely. thanks, Nitin On Saturday, March 29, 2014 at 5:43:05 PM UTC-7, Rob J Goedman wrote: > > Hi, > > As a first 'jump into the fray' exercise I've attempted to translate > Jeffrey Fike's hyper-dual numbers code from c++ to Julia, more or less > following the DualNumbers package. > > The c++ code can be found at http://adl.stanford.edu/hyperdual/hyperdual.h > . The paper itself at > http://adl.stanford.edu/hyperdual/Fike_AIAA-2011-886.pdf . > > The Julia package can be found at: > https://github.com/goedman/HyperDualNumbers.jl.git . > > Of course, I'm pretty new at this so I'm sure there will be errors and > poor practices. So any feedback is appreciated. > > Also, I'm wondering if the type should be called Hyper or a better name > would be HyperDual. > > This work was triggered by the interesting threads around openPP, > TaylorSeries.jl, Calculus2, PowerSeries.jl (and at some time I hope > MCMC.jl). > > Rob J. Goedman > goe...@icloud.com > > > > >
Re: [julia-users] Re: Hyper-Dual numbers
Thank you very much Jason, great feedback. I will work through your suggestions and make the changes. In particular the extensibility argument/pattern is something I've been thinking about as well. Regards, Rob J. Goedman goed...@icloud.com On Mar 30, 2014, at 10:09 AM, Jason Merrill wrote: > Cool. I took a quick look over the code, and it looks pretty good to me. You > have an inline comment about wanting understand convert/promote better--one > good place to look is promotions.jl in base: > > https://github.com/JuliaLang/julia/blob/master/base/promotion.jl#L158 > > This defines some fairly generic methods for +, -, *, /, etc. that mean "if > there isn't a more specific method for this combination of argument types, > try promoting the arguments to a common type before doing the operation". By > defining convert and promote_rule, you are participating in this system. > > I didn't look carefully at all the promotions/conversions you define, but if > you have the right ones, this will allow e.g. something like Hyper(0.0, 1.0, > 1.0, 2.0) + Hyper(3//5, 1//2, 1//2, 2//7) to work without you ever having had > to write code specific to the Rational type. > > As for the theory, it seems a little funny to me that this technique > redundantly encodes the first derivative, and I can't see an advantage of > using two variables that both square to 0 over using a single variable that > cubes to 0. Since E1 and E2 are used symmetrically, suppose we make the > change of variables e = E1 + E2. Now we have a single variable with e^2 = > 2E1E2, and e^3 = 0, and you could go through everywhere making the > replacements and end up saving yourself one field in your type, and a bit of > redundant arithmetic. > > After the replacement, it's easier to see how to extend to higher derivatives > too. If e^3 = 0 is a variable that helps us compute second derivatives, then > f^4=0 is a variable that helps us compute third derivatives, g^5 = 0 helps us > compute 4th derivatives, and so on. This is the theory behind the truncated > power series in PowerSeries.jl > > Instead of talking about a lot of different variables with special > properties, it's also possible to say that you're working modulo some > polynomial. Dual numbers are equivalent to computing mod x^2, 2nd order > truncated series are equivalent to computing mod x^3, 3rd order series to mod > x^4 and so on. In this way of thinking, complex numbers are polynomials mod > (x^2 + 1), and the split-complex numbers, which come up in the hyperbolic > geometry of the plane, are polynomials mod (x^2 - 1). > > On Saturday, March 29, 2014 5:43:05 PM UTC-7, Rob J Goedman wrote: > Hi, > > As a first 'jump into the fray' exercise I've attempted to translate Jeffrey > Fike's hyper-dual numbers code from c++ to Julia, more or less following the > DualNumbers package. > > The c++ code can be found at http://adl.stanford.edu/hyperdual/hyperdual.h . > The paper itself at http://adl.stanford.edu/hyperdual/Fike_AIAA-2011-886.pdf . > > The Julia package can be found at: > https://github.com/goedman/HyperDualNumbers.jl.git . > > Of course, I'm pretty new at this so I'm sure there will be errors and poor > practices. So any feedback is appreciated. > > Also, I'm wondering if the type should be called Hyper or a better name would > be HyperDual. > > This work was triggered by the interesting threads around openPP, > TaylorSeries.jl, Calculus2, PowerSeries.jl (and at some time I hope MCMC.jl). > > Rob J. Goedman > goe...@icloud.com
Re: [julia-users] Re: Hyper-Dual numbers
Hi Nitin, Thanks for the note and references. The recent upgrade of ForwardDiff.jl certainly shows the value of these methods. Did you (re-)implement MultiComplexNumbers from the 2nd reference, or used one of the two (Fortran and Matlab) prototype modules? Did you use it from within Julia? Do you know the license that comes with the implementation you used? Sorry for all the questions! Regards, Rob > On Sep 28, 2015, at 18:11, Nitin Arora wrote: > > There is another method to calculate any order derivatives, very similar to > this one, known as Multi-complex differentiation. This method can calculate > derivatives up-to any order. > > http://www.autodiff.org/Docs/euroad/13rd%20EuroAd%20Workshop%20-%20Thierry%20Dargent%20-%20Using%20Multicomplex%20Variables%20for%20Automatic%20Computation%20of%20High-Order%20Derivatives.pdf > > http://delivery.acm.org/10.1145/217/2168774/a16-lantoine.pdf?ip=207.151.221.1&id=2168774&acc=ACTIVE%20SERVICE&key=6405B83BDA580DC2%2E4D4702B0C3E38B35%2E4D4702B0C3E38B35%2E4D4702B0C3E38B35&CFID=717380051&CFTOKEN=25446753&__acm__=1443489222_ee3f3d4c7ee41030aaba536e14d6a6ab > > I have used this in my field and it seems to work nicely. > > thanks, > Nitin > >> On Saturday, March 29, 2014 at 5:43:05 PM UTC-7, Rob J Goedman wrote: >> Hi, >> >> As a first 'jump into the fray' exercise I've attempted to translate Jeffrey >> Fike's hyper-dual numbers code from c++ to Julia, more or less following the >> DualNumbers package. >> >> The c++ code can be found at http://adl.stanford.edu/hyperdual/hyperdual.h . >> The paper itself at http://adl.stanford.edu/hyperdual/Fike_AIAA-2011-886.pdf >> . >> >> The Julia package can be found at: >> https://github.com/goedman/HyperDualNumbers.jl.git . >> >> Of course, I'm pretty new at this so I'm sure there will be errors and poor >> practices. So any feedback is appreciated. >> >> Also, I'm wondering if the type should be called Hyper or a better name >> would be HyperDual. >> >> This work was triggered by the interesting threads around openPP, >> TaylorSeries.jl, Calculus2, PowerSeries.jl (and at some time I hope MCMC.jl). >> >> Rob J. Goedman >> goe...@icloud.com >> >> >> >>
Re: [julia-users] Re: Hyper-Dual numbers
Just a note to avoid confusion: ForwardDiff.jl doesn't use complex numbers at all. It uses its own implementation of dual number/hyper-dual number "ensembles" for computing multiple partial derivatives simultaneously. There is no additional error term in ForwardDiff.jl's formulation. -- Jarrett On Tuesday, September 29, 2015 at 12:21:26 AM UTC-4, Rob J Goedman wrote: > > Hi Nitin, > > Thanks for the note and references. The recent upgrade of ForwardDiff.jl > certainly shows the value of these methods. > > Did you (re-)implement MultiComplexNumbers from the 2nd reference, or used > one of the two (Fortran and Matlab) prototype modules? Did you use it from > within Julia? Do you know the license that comes with the implementation > you used? > > Sorry for all the questions! > > Regards, > Rob > > > On Sep 28, 2015, at 18:11, Nitin Arora > > wrote: > > There is another method to calculate any order derivatives, very similar > to this one, known as Multi-complex differentiation. This method can > calculate derivatives up-to any order. > > > http://www.autodiff.org/Docs/euroad/13rd%20EuroAd%20Workshop%20-%20Thierry%20Dargent%20-%20Using%20Multicomplex%20Variables%20for%20Automatic%20Computation%20of%20High-Order%20Derivatives.pdf > > > http://delivery.acm.org/10.1145/217/2168774/a16-lantoine.pdf?ip=207.151.221.1&id=2168774&acc=ACTIVE%20SERVICE&key=6405B83BDA580DC2%2E4D4702B0C3E38B35%2E4D4702B0C3E38B35%2E4D4702B0C3E38B35&CFID=717380051&CFTOKEN=25446753&__acm__=1443489222_ee3f3d4c7ee41030aaba536e14d6a6ab > > I have used this in my field and it seems to work nicely. > > thanks, > Nitin > > On Saturday, March 29, 2014 at 5:43:05 PM UTC-7, Rob J Goedman wrote: >> >> Hi, >> >> As a first 'jump into the fray' exercise I've attempted to translate >> Jeffrey Fike's hyper-dual numbers code from c++ to Julia, more or less >> following the DualNumbers package. >> >> The c++ code can be found at >> http://adl.stanford.edu/hyperdual/hyperdual.h . The paper itself at >> http://adl.stanford.edu/hyperdual/Fike_AIAA-2011-886.pdf . >> >> The Julia package can be found at: >> https://github.com/goedman/HyperDualNumbers.jl.git . >> >> Of course, I'm pretty new at this so I'm sure there will be errors and >> poor practices. So any feedback is appreciated. >> >> Also, I'm wondering if the type should be called Hyper or a better name >> would be HyperDual. >> >> This work was triggered by the interesting threads around openPP, >> TaylorSeries.jl, Calculus2, PowerSeries.jl (and at some time I hope >> MCMC.jl). >> >> Rob J. Goedman >> goe...@icloud.com >> >> >> >> >>