Re: [julia-users] Re: Hyper-Dual numbers

[Jarrett Revels]

Just a note to avoid confusion: ForwardDiff.jl doesn't use complex numbers 
at all. It uses its own implementation of dual number/hyper-dual number 
"ensembles" for computing multiple partial derivatives simultaneously. 
There is no additional error term in ForwardDiff.jl's formulation.

-- Jarrett

On Tuesday, September 29, 2015 at 12:21:26 AM UTC-4, Rob J Goedman wrote:
>
> Hi Nitin,
>
> Thanks for the note and references. The recent upgrade of ForwardDiff.jl 
> certainly shows the value of these methods.
>
> Did you (re-)implement MultiComplexNumbers from the 2nd reference, or used 
> one of the two (Fortran and Matlab) prototype modules? Did you use it from 
> within Julia? Do you know the license that comes with the implementation 
> you used? 
>
> Sorry for all the questions!
>
> Regards,
> Rob
>
>
> On Sep 28, 2015, at 18:11, Nitin Arora  
> wrote:
>
> There is another method to calculate any order derivatives, very similar 
> to this one, known as Multi-complex differentiation. This method can 
> calculate derivatives up-to any order.
>
>
> http://www.autodiff.org/Docs/euroad/13rd%20EuroAd%20Workshop%20-%20Thierry%20Dargent%20-%20Using%20Multicomplex%20Variables%20for%20Automatic%20Computation%20of%20High-Order%20Derivatives.pdf
>
>
> http://delivery.acm.org/10.1145/217/2168774/a16-lantoine.pdf?ip=207.151.221.1=2168774=ACTIVE%20SERVICE=6405B83BDA580DC2%2E4D4702B0C3E38B35%2E4D4702B0C3E38B35%2E4D4702B0C3E38B35=717380051=25446753&__acm__=1443489222_ee3f3d4c7ee41030aaba536e14d6a6ab
>
> I have used this in my field and it seems to work nicely.
>
> thanks,
> Nitin
>
> On Saturday, March 29, 2014 at 5:43:05 PM UTC-7, Rob J Goedman wrote:
>>
>> Hi,
>>
>> As a first 'jump into the fray' exercise I've attempted to translate 
>> Jeffrey Fike's hyper-dual numbers code from c++ to Julia, more or less 
>> following the DualNumbers package.
>>
>> The c++ code can be found at 
>> http://adl.stanford.edu/hyperdual/hyperdual.h . The paper itself at 
>> http://adl.stanford.edu/hyperdual/Fike_AIAA-2011-886.pdf .
>>
>> The Julia package can be found at: 
>> https://github.com/goedman/HyperDualNumbers.jl.git .
>>
>> Of course, I'm pretty new at this so I'm sure there will be errors and 
>> poor practices. So any feedback is appreciated.
>>
>> Also, I'm wondering if the type should be called Hyper or a better name 
>> would be HyperDual.
>>
>> This work was triggered by the interesting threads around openPP, 
>> TaylorSeries.jl, Calculus2, PowerSeries.jl (and at some time I hope 
>> MCMC.jl).
>>
>> Rob J. Goedman
>> goe...@icloud.com
>>
>>
>>
>>
>>

Re: [julia-users] Re: Hyper-Dual numbers

[Rob Goedman]

Hi Nitin,

Thanks for the note and references. The recent upgrade of ForwardDiff.jl 
certainly shows the value of these methods.

Did you (re-)implement MultiComplexNumbers from the 2nd reference, or used one 
of the two (Fortran and Matlab) prototype modules? Did you use it from within 
Julia? Do you know the license that comes with the implementation you used? 

Sorry for all the questions!

Regards,
Rob


> On Sep 28, 2015, at 18:11, Nitin Arora  wrote:
> 
> There is another method to calculate any order derivatives, very similar to 
> this one, known as Multi-complex differentiation. This method can calculate 
> derivatives up-to any order.
> 
> http://www.autodiff.org/Docs/euroad/13rd%20EuroAd%20Workshop%20-%20Thierry%20Dargent%20-%20Using%20Multicomplex%20Variables%20for%20Automatic%20Computation%20of%20High-Order%20Derivatives.pdf
> 
> http://delivery.acm.org/10.1145/217/2168774/a16-lantoine.pdf?ip=207.151.221.1=2168774=ACTIVE%20SERVICE=6405B83BDA580DC2%2E4D4702B0C3E38B35%2E4D4702B0C3E38B35%2E4D4702B0C3E38B35=717380051=25446753&__acm__=1443489222_ee3f3d4c7ee41030aaba536e14d6a6ab
> 
> I have used this in my field and it seems to work nicely.
> 
> thanks,
> Nitin
> 
>> On Saturday, March 29, 2014 at 5:43:05 PM UTC-7, Rob J Goedman wrote:
>> Hi,
>> 
>> As a first 'jump into the fray' exercise I've attempted to translate Jeffrey 
>> Fike's hyper-dual numbers code from c++ to Julia, more or less following the 
>> DualNumbers package.
>> 
>> The c++ code can be found at http://adl.stanford.edu/hyperdual/hyperdual.h . 
>> The paper itself at http://adl.stanford.edu/hyperdual/Fike_AIAA-2011-886.pdf 
>> .
>> 
>> The Julia package can be found at: 
>> https://github.com/goedman/HyperDualNumbers.jl.git .
>> 
>> Of course, I'm pretty new at this so I'm sure there will be errors and poor 
>> practices. So any feedback is appreciated.
>> 
>> Also, I'm wondering if the type should be called Hyper or a better name 
>> would be HyperDual.
>> 
>> This work was triggered by the interesting threads around openPP, 
>> TaylorSeries.jl, Calculus2, PowerSeries.jl (and at some time I hope MCMC.jl).
>> 
>> Rob J. Goedman
>> goe...@icloud.com
>> 
>> 
>> 
>> 

[julia-users] Re: Hyper-Dual numbers

[Nitin Arora]

There is another method to calculate any order derivatives, very similar to 
this one, known as Multi-complex differentiation. This method can calculate 
derivatives up-to any order.

http://www.autodiff.org/Docs/euroad/13rd%20EuroAd%20Workshop%20-%20Thierry%20Dargent%20-%20Using%20Multicomplex%20Variables%20for%20Automatic%20Computation%20of%20High-Order%20Derivatives.pdf

http://delivery.acm.org/10.1145/217/2168774/a16-lantoine.pdf?ip=207.151.221.1=2168774=ACTIVE%20SERVICE=6405B83BDA580DC2%2E4D4702B0C3E38B35%2E4D4702B0C3E38B35%2E4D4702B0C3E38B35=717380051=25446753&__acm__=1443489222_ee3f3d4c7ee41030aaba536e14d6a6ab

I have used this in my field and it seems to work nicely.

thanks,
Nitin

On Saturday, March 29, 2014 at 5:43:05 PM UTC-7, Rob J Goedman wrote:
>
> Hi,
>
> As a first 'jump into the fray' exercise I've attempted to translate 
> Jeffrey Fike's hyper-dual numbers code from c++ to Julia, more or less 
> following the DualNumbers package.
>
> The c++ code can be found at http://adl.stanford.edu/hyperdual/hyperdual.h 
> . The paper itself at 
> http://adl.stanford.edu/hyperdual/Fike_AIAA-2011-886.pdf .
>
> The Julia package can be found at: 
> https://github.com/goedman/HyperDualNumbers.jl.git .
>
> Of course, I'm pretty new at this so I'm sure there will be errors and 
> poor practices. So any feedback is appreciated.
>
> Also, I'm wondering if the type should be called Hyper or a better name 
> would be HyperDual.
>
> This work was triggered by the interesting threads around openPP, 
> TaylorSeries.jl, Calculus2, PowerSeries.jl (and at some time I hope 
> MCMC.jl).
>
> Rob J. Goedman
> goe...@icloud.com 
>
>
>
>
>

[julia-users] Re: Hyper-Dual numbers

[Jason Merrill]

Cool. I took a quick look over the code, and it looks pretty good to me. 
You have an inline comment about wanting understand convert/promote 
better--one good place to look is promotions.jl in base:

https://github.com/JuliaLang/julia/blob/master/base/promotion.jl#L158

This defines some fairly generic methods for +, -, *, /, etc. that mean if 
there isn't a more specific method for this combination of argument types, 
try promoting the arguments to a common type before doing the operation. 
By defining convert and promote_rule, you are participating in this system.

I didn't look carefully at all the promotions/conversions you define, but 
if you have the right ones, this will allow e.g. something like Hyper(0.0, 
1.0, 1.0, 2.0) + Hyper(3//5, 1//2, 1//2, 2//7) to work without you ever 
having had to write code specific to the Rational type.

As for the theory, it seems a little funny to me that this technique 
redundantly encodes the first derivative, and I can't see an advantage of 
using two variables that both square to 0 over using a single variable that 
cubes to 0. Since E1 and E2 are used symmetrically, suppose we make the 
change of variables e = E1 + E2. Now we have a single variable with e^2 = 
2E1E2, and e^3 = 0, and you could go through everywhere making the 
replacements and end up saving yourself one field in your type, and a bit 
of redundant arithmetic.

After the replacement, it's easier to see how to extend to higher 
derivatives too. If e^3 = 0 is a variable that helps us compute second 
derivatives, then f^4=0 is a variable that helps us compute third 
derivatives, g^5 = 0 helps us compute 4th derivatives, and so on. This is 
the theory behind the truncated power series in PowerSeries.jl

Instead of talking about a lot of different variables with special 
properties, it's also possible to say that you're working modulo some 
polynomial. Dual numbers are equivalent to computing mod x^2, 2nd order 
truncated series are equivalent to computing mod x^3, 3rd order series to 
mod x^4 and so on. In this way of thinking, complex numbers are polynomials 
mod (x^2 + 1), and the split-complex numbers, which come up in the 
hyperbolic geometry of the plane, are polynomials mod (x^2 - 1).

On Saturday, March 29, 2014 5:43:05 PM UTC-7, Rob J Goedman wrote:

 Hi,

 As a first 'jump into the fray' exercise I've attempted to translate 
 Jeffrey Fike's hyper-dual numbers code from c++ to Julia, more or less 
 following the DualNumbers package.

 The c++ code can be found at http://adl.stanford.edu/hyperdual/hyperdual.h. 
 The paper itself at 
 http://adl.stanford.edu/hyperdual/Fike_AIAA-2011-886.pdf .

 The Julia package can be found at: 
 https://github.com/goedman/HyperDualNumbers.jl.git .

 Of course, I'm pretty new at this so I'm sure there will be errors and 
 poor practices. So any feedback is appreciated.

 Also, I'm wondering if the type should be called Hyper or a better name 
 would be HyperDual.

 This work was triggered by the interesting threads around openPP, 
 TaylorSeries.jl, Calculus2, PowerSeries.jl (and at some time I hope 
 MCMC.jl).

 Rob J. Goedman
 goe...@icloud.com javascript:



  

Re: [julia-users] Re: Hyper-Dual numbers

[Robert J Goedman]

Thank you very much Jason, great feedback.

I will work through your suggestions and make the changes.

In particular the extensibility argument/pattern is something I've been 
thinking about as well.

Regards,
Rob J. Goedman
goed...@icloud.com




On Mar 30, 2014, at 10:09 AM, Jason Merrill jwmerr...@gmail.com wrote:

 Cool. I took a quick look over the code, and it looks pretty good to me. You 
 have an inline comment about wanting understand convert/promote better--one 
 good place to look is promotions.jl in base:
 
 https://github.com/JuliaLang/julia/blob/master/base/promotion.jl#L158
 
 This defines some fairly generic methods for +, -, *, /, etc. that mean if 
 there isn't a more specific method for this combination of argument types, 
 try promoting the arguments to a common type before doing the operation. By 
 defining convert and promote_rule, you are participating in this system.
 
 I didn't look carefully at all the promotions/conversions you define, but if 
 you have the right ones, this will allow e.g. something like Hyper(0.0, 1.0, 
 1.0, 2.0) + Hyper(3//5, 1//2, 1//2, 2//7) to work without you ever having had 
 to write code specific to the Rational type.
 
 As for the theory, it seems a little funny to me that this technique 
 redundantly encodes the first derivative, and I can't see an advantage of 
 using two variables that both square to 0 over using a single variable that 
 cubes to 0. Since E1 and E2 are used symmetrically, suppose we make the 
 change of variables e = E1 + E2. Now we have a single variable with e^2 = 
 2E1E2, and e^3 = 0, and you could go through everywhere making the 
 replacements and end up saving yourself one field in your type, and a bit of 
 redundant arithmetic.
 
 After the replacement, it's easier to see how to extend to higher derivatives 
 too. If e^3 = 0 is a variable that helps us compute second derivatives, then 
 f^4=0 is a variable that helps us compute third derivatives, g^5 = 0 helps us 
 compute 4th derivatives, and so on. This is the theory behind the truncated 
 power series in PowerSeries.jl
 
 Instead of talking about a lot of different variables with special 
 properties, it's also possible to say that you're working modulo some 
 polynomial. Dual numbers are equivalent to computing mod x^2, 2nd order 
 truncated series are equivalent to computing mod x^3, 3rd order series to mod 
 x^4 and so on. In this way of thinking, complex numbers are polynomials mod 
 (x^2 + 1), and the split-complex numbers, which come up in the hyperbolic 
 geometry of the plane, are polynomials mod (x^2 - 1).
 
 On Saturday, March 29, 2014 5:43:05 PM UTC-7, Rob J Goedman wrote:
 Hi,
 
 As a first 'jump into the fray' exercise I've attempted to translate Jeffrey 
 Fike's hyper-dual numbers code from c++ to Julia, more or less following the 
 DualNumbers package.
 
 The c++ code can be found at http://adl.stanford.edu/hyperdual/hyperdual.h . 
 The paper itself at http://adl.stanford.edu/hyperdual/Fike_AIAA-2011-886.pdf .
 
 The Julia package can be found at: 
 https://github.com/goedman/HyperDualNumbers.jl.git .
 
 Of course, I'm pretty new at this so I'm sure there will be errors and poor 
 practices. So any feedback is appreciated.
 
 Also, I'm wondering if the type should be called Hyper or a better name would 
 be HyperDual.
 
 This work was triggered by the interesting threads around openPP, 
 TaylorSeries.jl, Calculus2, PowerSeries.jl (and at some time I hope MCMC.jl).
 
 Rob J. Goedman
 goe...@icloud.com