[julia-users] Re: MSys2 package source for BinDeps

[Tony Kelman]

No, it's a problem if the package you want to use happens to depend on 
libgcc or libstdc++ and was built with a different ABI or compiler version 
than what Julia was built with.

It's not that hard to write spec files and submit them to opensuse. The 
build service even makes new binaries for you automatically, and you can 
point WinRPM at your own personal project to use binaries before you've 
even submitted them to the official project. Msys2 is still all built by 
hand AFAIK, and I've had PR's sit for months there with zero 
acknowledgement whatsoever that their packaging is buggy or outdated in 
many cases.


On Tuesday, May 31, 2016 at 6:39:46 PM UTC-7, Charles Barto wrote:
>
> while I understand the possible ABI compatibility issues unless I'm 
> mistaken that should only effect julia programs if the package you are 
> directly depending on has a different ABI between msys2 and WinRPM (in 
> which case you should pick one (probably WinRPM) or prod the upstream repo 
> to update the library in question. Beyond that the only issue I can think 
> of is if you somehow managed to get a library that links statically against 
> microsoft's libc, However I know mingw always links with the (really, 
> really, old) MSVCRT.DLL, and there's no problem linking dynamically with 
> multiple Microsoft libcs.
>
> I agree that WinRPM is better where available, but this could be useful in 
> some cases (there are lots more packages in msys than there are in WinRPM).
>
> On Monday, May 30, 2016 at 1:31:50 AM UTC-4, Tony Kelman wrote:
>>
>> There are many variations in how you can build mingw-w64 gcc. C++ 
>> libraries break ABI on nearly every GCC version change, so you need to be 
>> careful to be consistent. There's also which exception handling model you 
>> use, which threading model you use, and other various configuration flags 
>> of how you build GCC, and even -D defines that you can set at compile time 
>> when building different libraries. The "abstraction library" that you're 
>> referring to is the posix layer itself used for msys2's bash and gmake - 
>> it's cygwin with minor patches. The mingw64 package set doesn't link 
>> against the posix layer, but if you try to load libraries even all compiled 
>> using nominally "mingw64" compilers, you can get in trouble if the subtle 
>> ABI configurations of the compilers used to build the various libraries 
>> don't match.
>>
>> The most reliable way to interoperate with MSYS2-packaged libraries would 
>> be to build and package Julia itself using MSYS2's toolchain. This should 
>> be possible but I don't think anyone has gotten very far with it recently, 
>> except by accident from not reading the caveats in README.windows.md. 
>> You'd then need to avoid libraries cross-compiled from Cygwin (which is 
>> what the Julia buildbots use) or most Linux distributions (which is what 
>> WinRPM uses, from opensuse) - which are almost all set up to be ABI 
>> compatible with one another, but won't necessarily match MSYS2.
>>
>> It should also be possible to read the pacman repository metadata 
>> directly and be more selective about what you download - WinRPM doesn't 
>> need a full installation of rpm for example. I think Yichao Yu had some 
>> initial bindings for libalpm but I haven't tested whether it works for 
>> MSYS2 packages. Last I checked I don't think MSYS2 is building libalpm as a 
>> shared library, which you'd need for Julia bindings.
>>
>>
>> On Sunday, May 29, 2016 at 9:01:44 PM UTC-7, Charles Barto wrote:
>>>
>>> I wrote it to get the postgres libraries (libpq and friends). What are 
>>> the ABI differences? I know msys2 itself uses an abstraction library but 
>>> the ming64 packages shouldn't have any ABI problems (although some packages 
>>> may be a different version than those in winRPM)
>>>
>>> On Saturday, May 28, 2016 at 8:20:40 AM UTC-4, Tony Kelman wrote:

 What package is in msys2 but not winrpm? This seems like a huge 
 download to avoid writing a spec file and submitting it to the opensuse 
 build service. Julia is also not using exactly the same ABI as msys2's 
 toolchain so this isn't going to be reliable for C++ dependencies, or on 
 32 
 bit.


 On Friday, May 27, 2016 at 11:51:13 PM UTC-7, Charles Barto wrote:
>
> Hello, I've recently needed to use some packages that were not 
> available in the WinRPM repositories but were in MSys2's mingw64 repo. I 
> naturally set out to build a package provider for MSys2, after some work 
> figuring out 
>
> The github repository is located at 
> https://github.com/barcharcraz/MSys2.jl and I'd love for a few people 
> to try it before I submit a pull request for METADATA.
>
> The build file will download and extract all of msys2 to a local 
> directory, and then update. It's a pretty large download (in the hundreds 
> of megabytes) so
> try not to build the package on a slow connection.
>
> 

[julia-users] Re: MSys2 package source for BinDeps

[Charles Barto]

while I understand the possible ABI compatibility issues unless I'm 
mistaken that should only effect julia programs if the package you are 
directly depending on has a different ABI between msys2 and WinRPM (in 
which case you should pick one (probably WinRPM) or prod the upstream repo 
to update the library in question. Beyond that the only issue I can think 
of is if you somehow managed to get a library that links statically against 
microsoft's libc, However I know mingw always links with the (really, 
really, old) MSVCRT.DLL, and there's no problem linking dynamically with 
multiple Microsoft libcs.

I agree that WinRPM is better where available, but this could be useful in 
some cases (there are lots more packages in msys than there are in WinRPM).

On Monday, May 30, 2016 at 1:31:50 AM UTC-4, Tony Kelman wrote:
>
> There are many variations in how you can build mingw-w64 gcc. C++ 
> libraries break ABI on nearly every GCC version change, so you need to be 
> careful to be consistent. There's also which exception handling model you 
> use, which threading model you use, and other various configuration flags 
> of how you build GCC, and even -D defines that you can set at compile time 
> when building different libraries. The "abstraction library" that you're 
> referring to is the posix layer itself used for msys2's bash and gmake - 
> it's cygwin with minor patches. The mingw64 package set doesn't link 
> against the posix layer, but if you try to load libraries even all compiled 
> using nominally "mingw64" compilers, you can get in trouble if the subtle 
> ABI configurations of the compilers used to build the various libraries 
> don't match.
>
> The most reliable way to interoperate with MSYS2-packaged libraries would 
> be to build and package Julia itself using MSYS2's toolchain. This should 
> be possible but I don't think anyone has gotten very far with it recently, 
> except by accident from not reading the caveats in README.windows.md. 
> You'd then need to avoid libraries cross-compiled from Cygwin (which is 
> what the Julia buildbots use) or most Linux distributions (which is what 
> WinRPM uses, from opensuse) - which are almost all set up to be ABI 
> compatible with one another, but won't necessarily match MSYS2.
>
> It should also be possible to read the pacman repository metadata directly 
> and be more selective about what you download - WinRPM doesn't need a full 
> installation of rpm for example. I think Yichao Yu had some initial 
> bindings for libalpm but I haven't tested whether it works for MSYS2 
> packages. Last I checked I don't think MSYS2 is building libalpm as a 
> shared library, which you'd need for Julia bindings.
>
>
> On Sunday, May 29, 2016 at 9:01:44 PM UTC-7, Charles Barto wrote:
>>
>> I wrote it to get the postgres libraries (libpq and friends). What are 
>> the ABI differences? I know msys2 itself uses an abstraction library but 
>> the ming64 packages shouldn't have any ABI problems (although some packages 
>> may be a different version than those in winRPM)
>>
>> On Saturday, May 28, 2016 at 8:20:40 AM UTC-4, Tony Kelman wrote:
>>>
>>> What package is in msys2 but not winrpm? This seems like a huge download 
>>> to avoid writing a spec file and submitting it to the opensuse build 
>>> service. Julia is also not using exactly the same ABI as msys2's toolchain 
>>> so this isn't going to be reliable for C++ dependencies, or on 32 bit.
>>>
>>>
>>> On Friday, May 27, 2016 at 11:51:13 PM UTC-7, Charles Barto wrote:

 Hello, I've recently needed to use some packages that were not 
 available in the WinRPM repositories but were in MSys2's mingw64 repo. I 
 naturally set out to build a package provider for MSys2, after some work 
 figuring out 

 The github repository is located at 
 https://github.com/barcharcraz/MSys2.jl and I'd love for a few people 
 to try it before I submit a pull request for METADATA.

 The build file will download and extract all of msys2 to a local 
 directory, and then update. It's a pretty large download (in the hundreds 
 of megabytes) so
 try not to build the package on a slow connection.

 After downloading the distribution package the build script updates the 
 repositories, installs the latest version of pacman, then upgrades all 
 packages twice.
 I need to do two calls to upgrade because when msys2 updates its 
 runtime you need to restart the shell.

[julia-users] filter() function edge case, possible bug.

[Anonymous]

Consider the code:

filter(n -> true, collect(1:0))

this returns a 0-element array.  However if I don't collect the iterator 
into an array:

filter(n -> true, 1:0)

I get the error 

Error: TypeError: typeassert: expected AbstractArray{Bool, N}, got 
Array{Int64, 1} in filter at array.jl:923

It seems like this should also return a 0-element array, since if the 
iterator is non empty, e.g.

filter(n -> true, 1:2)

I get a two the element array [1,2].

[julia-users] Parallel read and preprocess variable-length data files

[Joshua Jones]

I'm looking for pointers on the best practice for parallel reading and 
preprocessing several files of variable-length time series data. I have 
little familiarity with parallelization, so it feels like I'm missing 
something obvious. I've read the Julia documentation and tried several 
approaches based on it, but with little success.

All pids are on one machine. Each file consists of a fixed-length header + 
NX points of variable precision data (NX varies from file to file). NX is 
stored as a value in each fixed-length header. 

I've come up with two clumsy solutions; the problem(s) are described below. 
Note that, with no parallelization, execution time is ~400% of numpy, 
mostly due to file read speed. 

   1. *pmap file read + preprocess*. Somewhat slow due to overhead of 
   moving data; slightly slower than numpy.
   2. *shared array, parallelize read + downsample*. I can achieve a 20% 
   speedup vs. numpy, but I've only found two ways to make this work.
  - A two-pass approach: open each file twice in a pair of pmap-style 
  @sync loops. 
  - first pass, loop over each file: open, get NX, get timestamp, 
 close, return. 
 - NY = round(Int, sum(NX)*fs_ratio); xx = SharedArray(Float64, 
 (NY,)); set variables for e.g. time indexing.
 - second pass, loop over each file: open, seek to data start, read 
 data, downsample into shared array, close.
 - A single-pass approach with stat():
   - estimate shared array size by summing 
 (stat(file).size-header_size)/(smallest_precision) over each file.
 - initialize shared array of NaNs, loop over each file to read 
 data, delete the leftover NaNs 
  - this runs into memory problems. The lowest precision of some data 
 formats overestimates NX by 8x (e.g. Int4 vs Int32). *
 
It seems like the fastest approach (by far) would be to leave IOStreams 
open (two-pass approach) or to read and preprocess each file into its own 
SharedArray (one-pass approach).

   - With the former approach, I don't know how to pass IOStreams between 
   workers. Is this even possible? I've flailed with approaches like an 
   Array{IOStream,1} filled with values created in my parallel kernel, but I 
   get Bad file descriptor errors when I try to read from any of the 
   resultant IOStreams again. For any stream, isopen(stream) returns true, 
   isreadable(stream) returns true, but the handle is set to @0x00...000. I 
   don't know a workaround, or if one exists.
   - With the latter, if data are stored in a SharedArray created in my 
   parallel kernel, accessing from myid() with e.g. sdata(s) gives #undef 
   for every value of s. What I've read suggests this is by design (same 
   basic problem as https://github.com/JuliaLang/julia/issues/13802 ). What am 
   I doing wrong here?
   

Thanks in advance for any suggestions you can offer.


* Aside, mostly: is there a documented/accepted "fastest" way to read a 
signed twos complement 4 bit integer? 

[julia-users] Re: What do @_inline_meta and @_propagate_inbounds_meta do?

[Blake Johnson]

Those macros are used in abstractarray.jl because at the point in the 
boostrapping process where that file is loaded, we do not yet have access 
to @inbounds and @propagate_inbounds.

A meta node is a way to pass information about a method to the compiler.

--Blake

On Tuesday, May 31, 2016 at 5:42:28 PM UTC-4, Davide Lasagna wrote:
>
> Thanks Kristoffer. AFAIU these are not supposed to be used in user code. 
> However, the follow up question is what is a meta node?
>
> On Monday, May 30, 2016 at 5:21:02 PM UTC+1, Kristoffer Carlsson wrote:
>>
>> I believe they insert the corresponding Meta node into the AST before the 
>> actual macros (like @inline) have been defined.
>
>

[julia-users] Unable to launch remote workers

[Patrick Belliveau]

Hi all,
   I normally launch multiple julia workers across multiple nodes 
of my research group's cluster from the linux shell prompt using 

julia --machinefile 

where each line of the machine file contains the host name of a cluster 
node accessible via
   
   ssh 

This worked like a charm until the cluster was recently physicall moved. It 
runs CentOS 5.11. There weren't any upgrades to the cluster done after the 
move but I'm guessing some security setting may have been made. Now when I 
execute julia --machinefile, I get the error

   Master process (id 1) could not connect within 60.0 seconds.

I've checked that I can ssh to all the hosts in the machine file and launch 
julia locally from there. I also tried adding remote workers via 
addprocs(). This throws a different error:

   ERROR: connect: host is unreachable (EHOSTUNREACH)

My julia versioninfo output is:

Julia Version 0.4.6-pre+7
Commit 273b487* (2016-03-28 14:46 UTC)
Platform Info:
  System: Linux (x86_64-unknown-linux-gnu)
  CPU: Intel(R) Xeon(R) CPU   E5410  @ 2.33GHz
  WORD_SIZE: 64
  BLAS: libmkl_rt
  LAPACK: libmkl_rt
  LIBM: libimf
  LLVM: libLLVM-3.3

I'm not sure if this is purely a cluster sys admin problem or if I might 
have to change something in Julia to work with the cluster. If anyone has 
any tips or has run into this issue before, help would be greatly 
appreciated.

Thanks, Patrick

[julia-users] Re: Does anyone use BBEditTextWrangler-julia.plist?

[Tony Kelman]

Ah, okay. It has some issues as described in 
https://github.com/JuliaLang/julia/pull/16613#issuecomment-11629, and could 
use some work to clean it up, write a script to generate it automatically.

Re: [julia-users] package management

[Davide Lasagna]

Thanks! 

On Tuesday, May 31, 2016 at 11:24:50 PM UTC+1, Yichao Yu wrote:
>
> On Tue, May 31, 2016 at 6:16 PM, Davide Lasagna  > wrote: 
> > Hi, 
> > 
> > How do package developers write code that is specific to different julia 
> > versions? 
> > 
> > For example, is anything like this: 
> > if VERSION < v"0.5-dev" 
> > f(x) = code_that_runs_on_v0.4() 
> > else 
> > f(x) = code_that_runs_on_v0.5() 
> > end 
> > recommended or is there a better way? 
>
> If the feature is not available in Compat and can't easily be added, I 
> recommend just add the global branch that depends on julia version. 
> You can always naturally refactor out common code path as you go. 
>
> > 
> > My use case is a generated function that builds SubArrays. Between 
> v0.4.* 
> > and v0.5, the last parameter changed so I need to have code that builds 
> the 
> > correct code in different versions. 
> > 
> > Thanks, 
> > 
> > Davide 
>

Re: [julia-users] package management

[Yichao Yu]

On Tue, May 31, 2016 at 6:16 PM, Davide Lasagna  wrote:
> Hi,
>
> How do package developers write code that is specific to different julia
> versions?
>
> For example, is anything like this:
> if VERSION < v"0.5-dev"
> f(x) = code_that_runs_on_v0.4()
> else
> f(x) = code_that_runs_on_v0.5()
> end
> recommended or is there a better way?

If the feature is not available in Compat and can't easily be added, I
recommend just add the global branch that depends on julia version.
You can always naturally refactor out common code path as you go.

>
> My use case is a generated function that builds SubArrays. Between v0.4.*
> and v0.5, the last parameter changed so I need to have code that builds the
> correct code in different versions.
>
> Thanks,
>
> Davide

[julia-users] package management

[Davide Lasagna]

Hi, 

How do package developers write code that is specific to different julia 
versions? 

For example, is anything like this:
if VERSION < v"0.5-dev"
f(x) = code_that_runs_on_v0.4()
else
f(x) = code_that_runs_on_v0.5()
end
recommended or is there a better way? 

My use case is a generated function that builds SubArrays. Between v0.4.* 
and v0.5, the last parameter changed so I need to have code that builds the 
correct code in different versions. 

Thanks, 

Davide

Re: [julia-users] Re: Why these 2 functions generate very different llvm?

['Tobias Knopp' via julia-users]

100% agree. I did not wanted to imply that this is a bug but a "surprising 
slow pass" (at least for me). So absolutely thumbs up if there is an 
opportunity of optimizing this case!

Cheers,

Tobias

Am Dienstag, 31. Mai 2016 16:14:27 UTC+2 schrieb Stefan Karpinski:
>
> This isn't exactly a bug since the answer is correct, it's an optimization 
> that we don't yet do. (If all of those are bugs then every language 
> implementation has an infinite number of bugs.) But yes, fixing this is on 
> track. There's been a lot of recent work on Julia's IR which should make 
> this problem resolvable soon.
>
> On Tue, May 31, 2016 at 1:11 AM, 'Tobias Knopp' via julia-users <
> julia...@googlegroups.com > wrote:
>
>> It would be great if this is resolvable. It took me quite some time to 
>> determine that this is an issue in my own code. My mental model was that 
>> type stability is about the return type being statically dependent on the 
>> input parameter types. That a local binding has changed is quite a 
>> different thing.
>>
>> Cheers,
>>
>> Tobias
>>
>> Am Dienstag, 31. Mai 2016 02:14:21 UTC+2 schrieb Yichao Yu:
>>>
>>> On Mon, May 30, 2016 at 7:54 PM, Erik Schnetter  
>>> wrote: 
>>> > Yichao 
>>> > 
>>> > What is the known bug here: That Julia doesn't handle this more 
>>> efficiently, 
>>>
>>> The bug is that the type instability here is spurious. 
>>>
>>> > or that the variable `x` can change its type from Int to Float64? 
>>> > 
>>> > In this example, are there two variables `x` (the argument and the 
>>> > explicitly declared local variable), or should the declaration be an 
>>> error? 
>>> > 
>>> > -erik 
>>> > 
>>> > On Mon, May 30, 2016 at 7:47 PM, Yichao Yu  wrote: 
>>> >> 
>>> >> On Mon, May 30, 2016 at 7:23 PM, David P. Sanders  
>>>
>>> >> wrote: 
>>> >> > 
>>> >> > 
>>> >> > El lunes, 30 de mayo de 2016, 19:11:47 (UTC-4), FANG Colin 
>>> escribió: 
>>> >> >> 
>>> >> >> function t1(n::Int, x::Int, a::Float64) 
>>> >> >>x::Float64 = x 
>>> >> >>for i in 1:n 
>>> >> >> x += a 
>>> >> >>end 
>>> >> >> x 
>>> >> >> end 
>>> >> >> @time t1(10^6, 1, 1.0) 
>>> >> >> 
>>> >> >> 0.005445 seconds (1.00 M allocations: 15.259 MB) 
>>> >> > 
>>> >> > 
>>> >> > In t1, x changes type during the function, from Int to Float64, so 
>>> the 
>>> >> > function is type *un*stable, as shown by @code_warntype, 
>>> >> > and as suggested by the huge number of allocations. 
>>> >> > 
>>> >> > In t2, x is always a Float64, and the function is type stable. 
>>> >> > 
>>> >> >> 
>>> >> >> 
>>> >> >> 
>>> >> >> 
>>> >> >> 
>>> >> >> function t2(n::Int, y::Int, a::Float64) 
>>> >> >>x::Float64 = y 
>>> >> >>for i in 1:n 
>>> >> >> x += a 
>>> >> >>end 
>>> >> >> x 
>>> >> >> end 
>>> >> >> @time t2(10^6, 1, 1.0) 
>>> >> >> 
>>> >> >> 0.001044 seconds (6 allocations: 192 bytes) 
>>> >> >> 
>>> >> >> 
>>> >> >> 
>>> >> >> 
>>> >> >> The @code_warntype of the 2 functions are very similar. However, 
>>> the 
>>> >> >> llvm 
>>> >> >> code generated from t2 is a lot simpler. 
>>> >> > 
>>> >> > 
>>> >> > The @code_warntype of the two functions is very *different*. (This 
>>> is 
>>> >> > easier 
>>> >> > to see in the REPL than in the notebook, if 
>>> >> > that is the problem.) 
>>> >> > 
>>> >> >> 
>>> >> >> 
>>> >> >> Does it suggest that if we want to change the type of an argument, 
>>> we'd 
>>> >> >> better create a new variable? 
>>> >> 
>>> >> This is a known bug. Fortunately it's easy to catch with 
>>> code_warntype. 
>>> > 
>>> > 
>>> > 
>>> > 
>>> > -- 
>>> > Erik Schnetter  
>>> > http://www.perimeterinstitute.ca/personal/eschnetter/ 
>>>
>>
>

[julia-users] Re: What do @_inline_meta and @_propagate_inbounds_meta do?

[Davide Lasagna]

Thanks Kristoffer. AFAIU these are not supposed to be used in user code. 
However, the follow up question is what is a meta node?

On Monday, May 30, 2016 at 5:21:02 PM UTC+1, Kristoffer Carlsson wrote:
>
> I believe they insert the corresponding Meta node into the AST before the 
> actual macros (like @inline) have been defined.

Re: [julia-users] Re: Calculus.hessian result oddities

[Patrick Kofod Mogensen]

File an issue :)

On Tuesday, May 31, 2016 at 6:59:24 PM UTC+2, Boylan, Ross wrote:
>
> I believe Calculus.hessian does finite differences.  I had not expected it 
> to computer both h[i, j] and h[j, i], since they should be the same.  That 
> is, AFAIK they are the same mathematically; numerically some differences 
> dependent on the order of operations seem inevitable.
>
> The double calculation has the potential advantage of alerting you if the 
> numerical issues are significant, but returning a matrix that lacks the 
> expected symmetry seems a pretty high price.
> Ross
> --
> *From:* julia...@googlegroups.com  [julia...@googlegroups.com 
> ] on behalf of Kristoffer Carlsson [kcarl...@gmail.com 
> ]
> *Sent:* Tuesday, May 31, 2016 1:26 AM
> *To:* julia-users
> *Subject:* [julia-users] Re: Calculus.hessian result oddities
>
> Doesn't Calculu's Hessian just do finite difference? There will naturally 
> be floating point errors. 
>
> isposdef returns false if the matrix is not hermitian, under the hood, 
> potrf from Lapack is used which work with hermitian matrices.
>
> On Tuesday, May 31, 2016 at 7:04:04 AM UTC+2, Boylan, Ross wrote: 
>>
>> The hessian produced by Calculus.hessian has some properties that seem 
>> odd to me.  First, the resulting matrix is not symmetric (though it's 
>> really close).  Second, it is not positive definite even though the 
>> eigenvalues are all positive.  This might partly be the result of the first 
>> problem. 
>>
>> Comments?
>>
>>  julia> @time h6 = Calculus.hessian(RB.mylike, sol6[2])
>>  206.988247 seconds (2.43 G allocations: 83.830 GB, 15.48% gc time)
>>  10x10 Array{Float64,2}:
>> 
>>  julia> issym(h6)
>>  false
>>  julia> isposdef(h6)
>>  false
>>  julia> eigvals(h6)
>>  10-element Array{Float64,1}:
>>   3.27548e5
>>   41407.1
>>1873.44
>> 809.515
>> 385.469
>> 336.118
>> 269.82
>>  64.4228
>> 117.009
>> 122.845
>>  julia> [det(h6[1:i, 1:i]) for i in 1:10]  # all submatrix determinants 
>> are positive
>>  10-element Array{Any,1}:
>>   583.356
>> 2.93666e5
>> 3.76334e7
>> 4.26776e9
>> 3.17529e12
>> 6.53405e14
>> 1.12477e17
>> 8.54011e21
>> 3.75658e26
>> 6.65866e29
>>
>>  julia> showall(h6)
>>  [583.356378500377 -215.9519377704782 42.707753088841706 
>> -12.046905619427504 100.26418659571075 30.907721733245413 
>> 5.1999446295105445 -3522.0797654066855 4410.401341876791 29.123944464570087
>>   -215.95193777047822 583.3512842422355 -12.0619061856546 
>> 42.69860798380086 -82.01778794681003 -20.00373356894301 -8.643894790795585 
>> 1484.6195212852567 -3316.905502014349 -2.7622075180025965
>>   42.7077530888417 -12.061906185654598 131.30467049554812 
>> -44.14799847519552 16.748039679197223 -10.98960509547607 43.02830286169418 
>> 234.68906012694512 -1258.6622801992237 9.86994636336174
>>   -12.046905619427504 42.69860798380086 -44.147998475195514 
>> 131.35743705036847 -17.498616283805696 4.724998443749094 
>> -31.481387268916205 -95.390157895702 209.5245372890838 12.903636963625035
>>   100.26418659571075 -82.01778794681003 16.748039679197223 
>> -17.498616283805696 766.6651018177663 -20.766430168130857 
>> -2.164074090378522 610.206909770971 -6927.858446549729 -200.4409796490271
>>   30.907721733245413 -20.00373356894301 -10.98960509547607 
>> 4.724998443749093 -20.766430168130857 209.8283757716871 -60.222027409043555 
>> 280.3888402431943 -1638.62573982097 -158.60599205247166
>>   5.199944629510544 -8.643894790795585 43.02830286169418 
>> -31.4813872689162 -2.164074090378524 -60.222027409043555 204.34068072875016 
>> -3.631354965653126 314.50383342119943 30.12788292183477
>>   -3522.079765406686 1484.6195212852565 234.6890601269451 
>> -95.39015789570199 610.2069097709712 280.3888402431943 -3.631354965653125 
>> 102596.98982032725 -117630.58331133435 267.23733374838395
>>   4410.401341876791 -3316.9055020143487 -1258.6622801992235 
>> 209.5245372890838 -6927.858446549729 -1638.62573982097 314.5038334211995 
>> -117630.58331133435 265600.50878023007 5508.687364627178
>>   29.123944464570084 -2.762207518002596 9.86994636336174 
>> 12.903636963625033 -200.4409796490271 -158.6059920524717 30.12788292183477 
>> 267.23733374838395 5508.687364627178 2125.8954716959997]
>>
>> The largest absolute difference between symmetrical entries is < 5e-13.
>>
>> Ross Boylan
>>
>>

[julia-users] Re: Access attribute (field) using symbol (or string)

[Femto Trader]

I got it

getfield(v, :x)

Sorry

Le mardi 31 mai 2016 20:54:02 UTC+2, Femto Trader a écrit :
>
> Hello,
>
> I have a type like
>
> type V
> x::Float64
> y::Float64
> z::Float64
> end
>
> v = V(1.0,2.0,3.0)
>
> I read on http://docs.julialang.org/en/release-0.4/devdocs/reflection/
> that
> fieldnames(v)
> returns attributes of v as symbol
>
> I would like to access to field x using either :x symbol (or "x" string)
>
> I tried
>
> get(v, :x)
>
> but it raises
> ERROR: MethodError: `get` has no method matching get(::V, ::Symbol)
> Closest candidates are:
>   get(::ObjectIdDict, ::ANY, ::ANY)
>   get{K,V}(::Dict{K,V}, ::Any, ::Any)
>   get{K}(::WeakKeyDict{K,V}, ::Any, ::Any)
>   ...
>
>
> Any idea ?
>
> Kind regard
>
>
> PS: here is the context of such a question 
> https://github.com/JuliaStats/DataFrames.jl/issues/982
>
> using DataFrames
> a_V = [V(1.0,2.0,3.0), V(4.0,5.0,6.0)]
> println(a_V)
> df = DataFrame(myvar=a_V)
> │ Row │ myvar  │
> ├─┼┤
> │ 1   │ V(1.0,2.0,3.0) │
> │ 2   │ V(4.0,5.0,6.0) │
>
> # I'd prefer to have x, y, z as column
>
> df2 = DataFrame()
> #df2[:x] = map((val)->val.x, a_V)
> #df2[:y] = map((val)->val.y, a_V)
> #df2[:z] = map((val)->val.z, a_V)
> for fieldname in fieldnames(V)
> df2[fieldname] = map((val)->val.x, a_V)
> end
> println(df2)
>
>

[julia-users] Re: can't get pyjulia to work

[Páll Haraldsson]

On Friday, May 27, 2016 at 12:07:40 PM UTC, Nirav Shah wrote:
>
> I tried this for first time today and I see same exact error :( 
> I can run PyCall and it's version is 1.6.1+. So I assume problem is in 
> pyjulia
>

>From memory, in some of the issues mentioned:

You muse use/pin version 1.3.0 of PyCall.jl.

At least for now.. I do not know more, do not use pycall, and just tried 
PyCall in the past to experiment.

-- 
Palli.
 

>
>
> On Tuesday, March 1, 2016 at 7:40:03 AM UTC-8, Neal Becker wrote:
>>
>> From time-to-time, I get interested in trying out moving some of my work 
>> from python to julia.  Before I can even start, I need to be able to call 
>> from python to julia.  But I've never gotten pyjulia to work on 
>> linux/fedora 
>> (currently 23).  I've tried the fedora version of julia (0.4.3), and I've 
>> built my own julia today from master, and in both cases I get: 
>>
>> j = julia.Julia (jl_init_path='/home/nbecker/julia') 
>> ERROR: UndefVarError: dlpath not defined 
>>  in eval(::Module, ::Any) at ./boot.jl:267 
>>  [inlined code] from ./sysimg.jl:14 
>>  in process_options(::Base.JLOptions) at ./client.jl:239 
>>  in _start() at ./client.jl:318 
>> Traceback (most recent call last): 
>>   File "/home/nbecker/pyjulia/julia/core.py", line 238, in __init__ 
>> """]) 
>>   File "/usr/lib64/python3.4/subprocess.py", line 620, in check_output 
>> raise CalledProcessError(retcode, process.args, output=output) 
>> subprocess.CalledProcessError: Command 
>> '['/home/nbecker/julia/usr/bin/julia', '-e', '\n 
>> println(JULIA_HOME)\n 
>> println(Sys.dlpath(dlopen("libjulia")))\n ']' 
>> returned 
>> non-zero exit status 1 
>>
>> During handling of the above exception, another exception occurred: 
>>
>> Traceback (most recent call last): 
>>   File "", line 1, in  
>>   File "/home/nbecker/pyjulia/julia/core.py", line 244, in __init__ 
>> raise JuliaError('error starting up the Julia process') 
>> julia.core.JuliaError: error starting up the Julia process 
>>
>>

Re: [julia-users] Access attribute (field) using symbol (or string)

[Yichao Yu]

On May 31, 2016 2:54 PM, "Femto Trader"  wrote:
>
> Hello,
>
> I have a type like
>
> type V
> x::Float64
> y::Float64
> z::Float64
> end
>
> v = V(1.0,2.0,3.0)
>
> I read on http://docs.julialang.org/en/release-0.4/devdocs/reflection/
> that
> fieldnames(v)
> returns attributes of v as symbol
>
> I would like to access to field x using either :x symbol (or "x" string)
>
> I tried
>
> get(v, :x)

getfield

>
> but it raises
> ERROR: MethodError: `get` has no method matching get(::V, ::Symbol)
> Closest candidates are:
>   get(::ObjectIdDict, ::ANY, ::ANY)
>   get{K,V}(::Dict{K,V}, ::Any, ::Any)
>   get{K}(::WeakKeyDict{K,V}, ::Any, ::Any)
>   ...
>
>
> Any idea ?
>
> Kind regard
>
>
> PS: here is the context of such a question
https://github.com/JuliaStats/DataFrames.jl/issues/982
>
> using DataFrames
> a_V = [V(1.0,2.0,3.0), V(4.0,5.0,6.0)]
> println(a_V)
> df = DataFrame(myvar=a_V)
> │ Row │ myvar  │
> ├─┼┤
> │ 1   │ V(1.0,2.0,3.0) │
> │ 2   │ V(4.0,5.0,6.0) │
>
> # I'd prefer to have x, y, z as column
>
> df2 = DataFrame()
> #df2[:x] = map((val)->val.x, a_V)
> #df2[:y] = map((val)->val.y, a_V)
> #df2[:z] = map((val)->val.z, a_V)
> for fieldname in fieldnames(V)
> df2[fieldname] = map((val)->val.x, a_V)
> end
> println(df2)
>

[julia-users] Access attribute (field) using symbol (or string)

[Femto Trader]

Hello,

I have a type like

type V
x::Float64
y::Float64
z::Float64
end

v = V(1.0,2.0,3.0)

I read on http://docs.julialang.org/en/release-0.4/devdocs/reflection/
that
fieldnames(v)
returns attributes of v as symbol

I would like to access to field x using either :x symbol (or "x" string)

I tried

get(v, :x)

but it raises
ERROR: MethodError: `get` has no method matching get(::V, ::Symbol)
Closest candidates are:
  get(::ObjectIdDict, ::ANY, ::ANY)
  get{K,V}(::Dict{K,V}, ::Any, ::Any)
  get{K}(::WeakKeyDict{K,V}, ::Any, ::Any)
  ...


Any idea ?

Kind regard


PS: here is the context of such a 
question https://github.com/JuliaStats/DataFrames.jl/issues/982

using DataFrames
a_V = [V(1.0,2.0,3.0), V(4.0,5.0,6.0)]
println(a_V)
df = DataFrame(myvar=a_V)
│ Row │ myvar  │
├─┼┤
│ 1   │ V(1.0,2.0,3.0) │
│ 2   │ V(4.0,5.0,6.0) │

# I'd prefer to have x, y, z as column

df2 = DataFrame()
#df2[:x] = map((val)->val.x, a_V)
#df2[:y] = map((val)->val.y, a_V)
#df2[:z] = map((val)->val.z, a_V)
for fieldname in fieldnames(V)
df2[fieldname] = map((val)->val.x, a_V)
end
println(df2)

Re: [julia-users] Signal / slot (or publish-subscribe) libraries in Julia

[Femto Trader]

Here is my very basic implementation

#= 
A very basic signal / slot library

Usage:

function myslot01(args...; kwargs...)
println("myslot01 with $args and $kwargs")
end

function myslot02(args...; kwargs...)
println("myslot02 with $args and $kwargs")
end

function myslot03(a,b; x=-1, y=-1)
println("myslot03 with a=$a b=$b x=$x y=$y")
end


stop_modified = Signal()
connect(stop_modified, myslot01)
connect(stop_modified, myslot02)
connect(stop_modified, myslot03)
emit(stop_modified, 1, 2, x=1, y=2)
connected = is_connected(stop_modified, myslot01)
println("is_connected(stop_modified, myslot01): $connected")
disconnect(stop_modified, myslot01)
connected = is_connected(stop_modified, myslot01)
println("is_connected(stop_modified, myslot01): $connected")
emit(stop_modified, "a", "b")

=#

using DataStructures

typealias Slot Function

#=
ToDo: it will be possible to do this when Julia will accept anonymous 
function with keyword arguments (v>=0.5)

type Slot
f::Function
end
call(slot::Slot, args...; kwargs...) = slot.f(args...; kwargs...)

myslot01 = Slot((args...; kwargs...) -> begin
println("myslot01 with $args and $kwargs")
end)
=#

type Signal
slots::OrderedDict{Symbol,Slot}

Signal() = new(OrderedDict())
end

function emit(signal::Signal, args...; kwargs...)
for (symb, slot) in signal.slots
slot(args...; kwargs...)
end
end

function connect(signal::Signal, slot::Slot)
key = symbol(slot)
if !haskey(signal.slots, key)
signal.slots[key] = slot
else
error("slot $slot ever connected to $signal")
end
end

function is_connected(signal::Signal, slot::Slot)
symbol(slot) in keys(signal.slots)
end

function disconnect(signal::Signal, slot::Slot)
delete!(signal.slots, symbol(slot))
end





Le mardi 31 mai 2016 02:17:54 UTC+2, Steven G. Johnson a écrit :
>
> Reactive.jl?

[julia-users] Re: What is the difference between bitwise and logical operators?

[Ford Ox]

On Tuesday, May 31, 2016 at 8:24:43 PM UTC+2, Kristoffer Carlsson wrote:
>
> https://youtu.be/dK3zRXhrFZY?t=600


Thanks! 

[julia-users] Re: Does anyone use BBEditTextWrangler-julia.plist?

[Femto Trader]

I use it !!!

Le mardi 31 mai 2016 16:49:20 UTC+2, Tony Kelman a écrit :
>
> Agreed, but that can wait until someone wants to resurrect it and use it. 
> If such a person comes upon this thread some time in the future, the 
> removal PR was https://github.com/JuliaLang/julia/pull/16637, and you can 
> restore the state of the file from immediately before it was deleted at 
> https://github.com/JuliaLang/julia/blob/f75e74d212dab946e3b62dee8440083d64bd74be/contrib/BBEditTextWrangler-julia.plist.
>  
> I think an organization for various Julia editor support repos would be a 
> good idea if someone wants to do the migration work. I created a 
> placeholder but don't plan on doing that work in the immediate future, so 
> let me know if you'd like an invite to the placeholder organization.
>
>
> On Friday, May 27, 2016 at 7:59:17 PM UTC-7, Lyndon White wrote:
>>
>> I am not.
>>
>> Re: Generating lists exports automatically, a simple call to 
>> `names(Base)` would probably do it.
>> I'm doing more complex lists of exports in  
>> https://github.com/oxinabox/julia-vim-completions
>>
>> Julia reflection is strong enough, for this kinda thing.
>>
>>

[julia-users] Re: What is the difference between bitwise and logical operators?

[Kristoffer Carlsson]

https://youtu.be/dK3zRXhrFZY?t=600

Re: [julia-users] Bitstype

[Ford Ox]

I wanted to discard that question, since I wasn't aware, that today 
computers are usually byte addressable.

Re: [julia-users] Bitstype

[Stefan Karpinski]

Yes and yes.

On Tue, May 31, 2016 at 12:38 PM, Ford Ox  wrote:

> Why can bitstype be only multiples of eight?
> Will this ever change?
> If yes, will it be possible to create bitstype of any length (1, 3, 13...)?
>

RE: [julia-users] Re: Calculus.hessian result oddities

[Boylan, Ross]

I believe Calculus.hessian does finite differences.  I had not expected it to 
computer both h[i, j] and h[j, i], since they should be the same.  That is, 
AFAIK they are the same mathematically; numerically some differences dependent 
on the order of operations seem inevitable.

The double calculation has the potential advantage of alerting you if the 
numerical issues are significant, but returning a matrix that lacks the 
expected symmetry seems a pretty high price.
Ross

From: julia-users@googlegroups.com [julia-users@googlegroups.com] on behalf of 
Kristoffer Carlsson [kcarlsso...@gmail.com]
Sent: Tuesday, May 31, 2016 1:26 AM
To: julia-users
Subject: [julia-users] Re: Calculus.hessian result oddities

Doesn't Calculu's Hessian just do finite difference? There will naturally be 
floating point errors.

isposdef returns false if the matrix is not hermitian, under the hood, potrf 
from Lapack is used which work with hermitian matrices.

On Tuesday, May 31, 2016 at 7:04:04 AM UTC+2, Boylan, Ross wrote:
The hessian produced by Calculus.hessian has some properties that seem odd to 
me.  First, the resulting matrix is not symmetric (though it's really close).  
Second, it is not positive definite even though the eigenvalues are all 
positive.  This might partly be the result of the first problem.

Comments?

 julia> @time h6 = Calculus.hessian(RB.mylike, sol6[2])
 206.988247 seconds (2.43 G allocations: 83.830 GB, 15.48% gc time)
 10x10 Array{Float64,2}:

 julia> issym(h6)
 false
 julia> isposdef(h6)
 false
 julia> eigvals(h6)
 10-element Array{Float64,1}:
  3.27548e5
  41407.1
   1873.44
809.515
385.469
336.118
269.82
 64.4228
117.009
122.845
 julia> [det(h6[1:i, 1:i]) for i in 1:10]  # all submatrix determinants are 
positive
 10-element Array{Any,1}:
  583.356
2.93666e5
3.76334e7
4.26776e9
3.17529e12
6.53405e14
1.12477e17
8.54011e21
3.75658e26
6.65866e29

 julia> showall(h6)
 [583.356378500377 -215.9519377704782 42.707753088841706 -12.046905619427504 
100.26418659571075 30.907721733245413 5.1999446295105445 -3522.0797654066855 
4410.401341876791 29.123944464570087
  -215.95193777047822 583.3512842422355 -12.0619061856546 42.69860798380086 
-82.01778794681003 -20.00373356894301 -8.643894790795585 1484.6195212852567 
-3316.905502014349 -2.7622075180025965
  42.7077530888417 -12.061906185654598 131.30467049554812 -44.14799847519552 
16.748039679197223 -10.98960509547607 43.02830286169418 234.68906012694512 
-1258.6622801992237 9.86994636336174
  -12.046905619427504 42.69860798380086 -44.147998475195514 131.35743705036847 
-17.498616283805696 4.724998443749094 -31.481387268916205 -95.390157895702 
209.5245372890838 12.903636963625035
  100.26418659571075 -82.01778794681003 16.748039679197223 -17.498616283805696 
766.6651018177663 -20.766430168130857 -2.164074090378522 610.206909770971 
-6927.858446549729 -200.4409796490271
  30.907721733245413 -20.00373356894301 -10.98960509547607 4.724998443749093 
-20.766430168130857 209.8283757716871 -60.222027409043555 280.3888402431943 
-1638.62573982097 -158.60599205247166
  5.199944629510544 -8.643894790795585 43.02830286169418 -31.4813872689162 
-2.164074090378524 -60.222027409043555 204.34068072875016 -3.631354965653126 
314.50383342119943 30.12788292183477
  -3522.079765406686 1484.6195212852565 234.6890601269451 -95.39015789570199 
610.2069097709712 280.3888402431943 -3.631354965653125 102596.98982032725 
-117630.58331133435 267.23733374838395
  4410.401341876791 -3316.9055020143487 -1258.6622801992235 209.5245372890838 
-6927.858446549729 -1638.62573982097 314.5038334211995 -117630.58331133435 
265600.50878023007 5508.687364627178
  29.123944464570084 -2.762207518002596 9.86994636336174 12.903636963625033 
-200.4409796490271 -158.6059920524717 30.12788292183477 267.23733374838395 
5508.687364627178 2125.8954716959997]

The largest absolute difference between symmetrical entries is < 5e-13.

Ross Boylan

[julia-users] Re: What is the difference between bitwise and logical operators?

[Ford Ox]

As a side note.

Are add_int(), box() and unbox() implemented in c?
I was trying to find their declaration but was not successful.

[julia-users] Re: Using Julia for real time astronomy

[Páll Haraldsson]

On Monday, May 30, 2016 at 8:19:34 PM UTC, Tobias Knopp wrote:
>
> If you are prepared to make your code to not perform any heap allocations, 
> I don't see a reason why there should be any issue. When I once worked on a 
> very first multi-threading version of Julia I wrote exactly such functions 
> that won't trigger gc since the later was not thread safe. This can be hard 
> work but I would assume that its at least not more work than implementing 
> the application in C/C++ (assuming that you have some Julia experience)
>

I would really like to know why the work is hard, is it getting rid of the 
allocations, or being sure there are no more hidden in your code? I would 
also like to know then if you can do the same as in D language:

http://wiki.dlang.org/Memory_Management 

The most reliable way to guarantee latency is to preallocate all data that 
will be needed by the time critical portion. If no calls to allocate memory 
are done, the GC will not run and so will not cause the maximum latency to 
be exceeded.

It is possible to create a real-time thread by detaching it from the 
runtime, marking the thread function @nogc, and ensuring the real-time 
thread does not hold any GC roots. GC objects can still be used in the 
real-time thread, but they must be referenced from other threads to prevent 
them from being collected."

that is would it be possible to make a macro @nogc and mark functions in a 
similar way? I'm not aware that such a macro is available, to disallow. 
There is a macro, e.g. @time, that is not sufficient, that shows GC 
actitivy, but knowing there was none could have been an accident; if you 
run your code again and memory fills up you see different result.

As with D, the GC in Julia is optional. The above @nogc, is really the only 
thing different, that I can think of that is better with their optional 
memory management. But I'm no expert on D, and I mey not have looked too 
closely:

https://dlang.org/spec/garbage.html


> Tobi
>
> Am Montag, 30. Mai 2016 12:00:13 UTC+2 schrieb John leger:
>>
>> Hi everyone,
>>
>> I am working in astronomy and we are thinking of using Julia for a real 
>> time, high performance adaptive optics system on a solar telescope.
>>
>> This is how the system is supposed to work: 
>>1) the image is read from the camera
>>2) some correction are applied
>>3) the atmospheric turbulence is numerically estimated in order to 
>> calculate the command to be sent to the deformable mirror
>>
>> The overall process should be executed in less than 1ms so that it can be 
>> integrated to the chain (closed loop).
>>
>> Do you think it is possible to do all the computation in Julia or would 
>> it be better to code some part in C/C++. What I fear the most is the GC but 
>> in our case we can pre-allocate everything, so once we launch the system 
>> there will not be any memory allocated during the experiment and it will 
>> run for days.
>>
>> So, what do you think? Considering the current state of Julia will I be 
>> able to get the performances I need. Will the garbage collector be an 
>> hindrance ?
>>
>> Thank you.
>>
>

[julia-users] Bitstype

[Ford Ox]

Why can bitstype be only multiples of eight? 
Will this ever change? 
If yes, will it be possible to create bitstype of any length (1, 3, 13...)?

[julia-users] Re: Using Julia for real time astronomy

[Páll Haraldsson]

On Monday, May 30, 2016 at 12:10:39 PM UTC, Uwe Fechner wrote:
>
> I think, that would be difficult.
>
> As soon as you use any packages for image conversion or estimation you 
> have to assume that they use dynamic memory allocation.
>
> The garbage collector of Julia is fast, but not suitable for hard 
> real-time requirements. Implementing a garbage collector for hard real-time
> applications is possible, but a lot of work and will probably not happen 
> in the near future.
>
> Their was an issue on this topic, that was closed as "won't fix":
> https://github.com/JuliaLang/julia/issues/8543 
> 
>

Well, the "won't fix"-label was later taken off the issue.

Yes, the issue is still closed, but it's unclear to me what has changed 
with the GC, when. I know incremental GC was implemented at some point. No 
hard-real-time GC is available.

It would be cool to know of Julia in space so I gave this some thought..

I recall from MicroPython, that they claimed hard-real-time GC (also 
available for Java with Metronome), that is predictable pause times. I 
remember thinking, how can they do/claim that (and if I recall, didn't 
change the GC)? MicroPython is meant for microcontrollers (at the time only 
one), that has a known amount of memory. I can't locate the information I 
read at the time now, I think they where talking in megabytes range. Then 
worst case, you have to scan a fixed amount of memory, and the speed of the 
CPU is also known. Unlike with MicroPython, you will have an operating 
system (that is not real-time, but Linux can be configured as such, but 
caches are a problem..). Maybe if you can limit the RAM, or just how much 
Julia will try to allocate, it helps in the same way.

Anyway, you may not strictly need hard-real-time. I think, as always (in 
non-real-time/concurrent GC variants)?, the garbage collection only happens 
when you try to allocate memory and it is full. If you preallocate all 
memory and make sure no more is allocated, I can't see the GC being a 
problem (you can also disable it for some period of time).

Libc.malloc and free is also available with Julia..


[Possibly it helps to split your task into more than one process, having 
only one real-time? If you can have shared memory between two processes, 
would that help? Be careful with that.. I'm not sure it's a good idea or at 
least I need to explain it better..]



https://github.com/micropython/micropython/wiki/FAQ
"Regarding RAM usage, MicroPython can start up with 2KB of heap. Adding 
stack and required static memory, a 4KB microcontroller could start a 
MicroController, but hardly could go further than interpreting simple 
expressions. Thus, 8KB is minimal amount to run simple scripts."

https://forum.micropython.org/viewtopic.php?t=1778
"today I painfully learned, that uPy's automatic garbage collection can 
really mess up your 500Hz feedback control loop, since it takes forever 
(>1ms  :o :shock: :cry: )."


http://entitycrisis.blogspot.is/2007/12/is-hard-real-time-python-possible.html

http://stackoverflow.com/questions/1402933/python-on-an-real-time-operation-system-rtos


>
> Uwe
>
> On Monday, May 30, 2016 at 12:00:13 PM UTC+2, John leger wrote:
>>
>> Hi everyone,
>>
>> I am working in astronomy and we are thinking of using Julia for a real 
>> time, high performance adaptive optics system on a solar telescope.
>>
>> This is how the system is supposed to work: 
>>1) the image is read from the camera
>>2) some correction are applied
>>3) the atmospheric turbulence is numerically estimated in order to 
>> calculate the command to be sent to the deformable mirror
>>
>> The overall process should be executed in less than 1ms so that it can be 
>> integrated to the chain (closed loop).
>>
>> Do you think it is possible to do all the computation in Julia or would 
>> it be better to code some part in C/C++. What I fear the most is the GC but 
>> in our case we can pre-allocate everything, so once we launch the system 
>> there will not be any memory allocated during the experiment and it will 
>> run for days.
>>
>> So, what do you think? Considering the current state of Julia will I be 
>> able to get the performances I need. Will the garbage collector be an 
>> hindrance ?
>>
>> Thank you.
>>
>

[julia-users] Re: Does anyone use BBEditTextWrangler-julia.plist?

[Tony Kelman]

Agreed, but that can wait until someone wants to resurrect it and use it. 
If such a person comes upon this thread some time in the future, the 
removal PR was https://github.com/JuliaLang/julia/pull/16637, and you can 
restore the state of the file from immediately before it was deleted at 
https://github.com/JuliaLang/julia/blob/f75e74d212dab946e3b62dee8440083d64bd74be/contrib/BBEditTextWrangler-julia.plist.
 
I think an organization for various Julia editor support repos would be a 
good idea if someone wants to do the migration work. I created a 
placeholder but don't plan on doing that work in the immediate future, so 
let me know if you'd like an invite to the placeholder organization.


On Friday, May 27, 2016 at 7:59:17 PM UTC-7, Lyndon White wrote:
>
> I am not.
>
> Re: Generating lists exports automatically, a simple call to `names(Base)` 
> would probably do it.
> I'm doing more complex lists of exports in  
> https://github.com/oxinabox/julia-vim-completions
>
> Julia reflection is strong enough, for this kinda thing.
>
>

Re: [julia-users] Re: Why these 2 functions generate very different llvm?

[Stefan Karpinski]

This isn't exactly a bug since the answer is correct, it's an optimization
that we don't yet do. (If all of those are bugs then every language
implementation has an infinite number of bugs.) But yes, fixing this is on
track. There's been a lot of recent work on Julia's IR which should make
this problem resolvable soon.

On Tue, May 31, 2016 at 1:11 AM, 'Tobias Knopp' via julia-users <
julia-users@googlegroups.com> wrote:

> It would be great if this is resolvable. It took me quite some time to
> determine that this is an issue in my own code. My mental model was that
> type stability is about the return type being statically dependent on the
> input parameter types. That a local binding has changed is quite a
> different thing.
>
> Cheers,
>
> Tobias
>
> Am Dienstag, 31. Mai 2016 02:14:21 UTC+2 schrieb Yichao Yu:
>>
>> On Mon, May 30, 2016 at 7:54 PM, Erik Schnetter 
>> wrote:
>> > Yichao
>> >
>> > What is the known bug here: That Julia doesn't handle this more
>> efficiently,
>>
>> The bug is that the type instability here is spurious.
>>
>> > or that the variable `x` can change its type from Int to Float64?
>> >
>> > In this example, are there two variables `x` (the argument and the
>> > explicitly declared local variable), or should the declaration be an
>> error?
>> >
>> > -erik
>> >
>> > On Mon, May 30, 2016 at 7:47 PM, Yichao Yu  wrote:
>> >>
>> >> On Mon, May 30, 2016 at 7:23 PM, David P. Sanders 
>> >> wrote:
>> >> >
>> >> >
>> >> > El lunes, 30 de mayo de 2016, 19:11:47 (UTC-4), FANG Colin escribió:
>> >> >>
>> >> >> function t1(n::Int, x::Int, a::Float64)
>> >> >>x::Float64 = x
>> >> >>for i in 1:n
>> >> >> x += a
>> >> >>end
>> >> >> x
>> >> >> end
>> >> >> @time t1(10^6, 1, 1.0)
>> >> >>
>> >> >> 0.005445 seconds (1.00 M allocations: 15.259 MB)
>> >> >
>> >> >
>> >> > In t1, x changes type during the function, from Int to Float64, so
>> the
>> >> > function is type *un*stable, as shown by @code_warntype,
>> >> > and as suggested by the huge number of allocations.
>> >> >
>> >> > In t2, x is always a Float64, and the function is type stable.
>> >> >
>> >> >>
>> >> >>
>> >> >>
>> >> >>
>> >> >>
>> >> >> function t2(n::Int, y::Int, a::Float64)
>> >> >>x::Float64 = y
>> >> >>for i in 1:n
>> >> >> x += a
>> >> >>end
>> >> >> x
>> >> >> end
>> >> >> @time t2(10^6, 1, 1.0)
>> >> >>
>> >> >> 0.001044 seconds (6 allocations: 192 bytes)
>> >> >>
>> >> >>
>> >> >>
>> >> >>
>> >> >> The @code_warntype of the 2 functions are very similar. However,
>> the
>> >> >> llvm
>> >> >> code generated from t2 is a lot simpler.
>> >> >
>> >> >
>> >> > The @code_warntype of the two functions is very *different*. (This
>> is
>> >> > easier
>> >> > to see in the REPL than in the notebook, if
>> >> > that is the problem.)
>> >> >
>> >> >>
>> >> >>
>> >> >> Does it suggest that if we want to change the type of an argument,
>> we'd
>> >> >> better create a new variable?
>> >>
>> >> This is a known bug. Fortunately it's easy to catch with
>> code_warntype.
>> >
>> >
>> >
>> >
>> > --
>> > Erik Schnetter 
>> > http://www.perimeterinstitute.ca/personal/eschnetter/
>>
>

[julia-users] Re: Getting sequence of time and subset it

[akrun]

Many thanks for the solution and it works well with the example posted.   
But, if I have missing values in one of the columns, it throws an error.

   sdt2 = DataFrame(ID = 1:2, StartTime = DateTime(["4/13/2016 07:00", 
"4/13/2016 07:15"], "m/d/y H:M"),
   EndTime = DateTime(["4/13/2016 12:00", 
"4/13/2016 12:15"], "m/d/y H:M"),
   BreakTime1 = DateTime([NA, "4/13/2016 10:15"], 
"m/d/y H:M"),
   BreakTime2 = DateTime(["4/13/2016 12:00", 
"4/13/2016 12:15"], "m/d/y H:M")


  d1= DataFrame(id=Int64[], Time = AbstractString[])
for i in 1:nrow(sdt2), t in 
[sdt2[:StartTime][i]:Dates.Minute(30):sdt2[:EndTime][i]]
  if !(t in [sdt2[:BreakTime1][i], sdt2[:BreakTime2][i]])
  push!(d1, [i, Dates.format(t, "HH:MM")])
  end
   end

   MethodError: `convert` has no method matching convert(::Type{Int64}, 
::Array{Any,1})
This may have arisen from a call to the constructor Int64(...),
since type constructors fall back to convert methods.
Closest candidates are:
  call{T}(::Type{T}, ::Any)
  convert(::Type{Int64}, !Matched::Int8)
  convert(::Type{Int64}, !Matched::UInt8)

Is there any way to correct this?

Thanks.


 On Monday, May 30, 2016 at 9:22:24 PM UTC+5:30, Evan Fields wrote:
>
> I'm not 100% sure I understand your question, but let me give it a shot. 
>
> First thing is to note why you're getting that MethodError. It's from the 
> line
>
> println(si in sdt1[i, [:BreakTime1, :BreakTime2]])
>
> Define sdt1 as you do (I just copied into a Julia REPL) and set i = 1 to 
> for the first iteration of the for loop.
>
> Then:
>
> julia> sdt1[i, [:BreakTime1, :BreakTime2]]
> 1×2 DataFrames.DataFrame
> │ Row │ BreakTime1  │ BreakTime2  │
> ├─┼─┼─┤
> │ 1   │ 2016-04-13T10:00:00 │ 2016-04-13T12:00:00 │
>
> julia> si = 
> Dates.format([sdt1[i,:StartTime]:Dates.Minute(30):sdt1[i,:EndTime]], 
> "HH:MM")
> WARNING: [a] concatenation is deprecated; use collect(a) instead
>  in depwarn at deprecated.jl:73
>  in oldstyle_vcat_warning at abstractarray.jl:29
>  in vect at abstractarray.jl:32
> while loading no file, in expression starting on line 0
> 11-element Array{Any,1}:
>  "07:15"
>  "07:45"
>  "08:15"
>  "08:45"
>  "09:15"
>  "09:45"
>  "10:15"
>  "10:45"
>  "11:15"
>  "11:45"
>  "12:15"
>
> julia> si in sdt1[i, [:BreakTime1, :BreakTime2]]
> ERROR: MethodError: `start` has no method matching 
> start(::DataFrames.DataFrame)
>  in mapreduce_sc_impl at reduce.jl:194
>  in in at reduce.jl:377
>
> So si is of type Vector{Any} but happens to hold strings. sdt1[i, 
> [:BreakTime1, :BreakTime2]] is a DataFrame. What does it mean to ask if a 
> vector is in a DataFrame? That's what's happening in your println; the `in` 
> is just an infix operator of function in.
>
> 3-element Array{Int64,1}:
>  1
>  2
>  3
>
> julia> 1 in x
> true
>
> julia> in(1,x)
> true
>
> julia> in(5,x)
> false
>
> I assume you get the MethodError of no method start because in(a,b) 
> iterates over b, checking each element of b for equality to a. But 
> DataFrames are not iterable in this way. (This part just a guess; look it 
> up in the manual before telling your friends.)
>
> Anyway, to get output as from your R snippet, you could use a loop like 
> the following:
>
> julia> for i in 1:nrow(sdt1), t in 
> sdt1[:StartTime][i]:Dates.Minute(30):sdt1[:EndTime][i]
>if !(t in [sdt1[:BreakTime1][i], sdt1[:BreakTime2][i]])
>println("$i $(Dates.format(t, "HH:MM"))")
>end
>end
> 1 07:00
> 1 07:30
> 1 08:00
> 1 08:30
> 1 09:00
> 1 09:30
> 1 10:30
> 1 11:00
> 1 11:30
> 2 07:15
> 2 07:45
> 2 08:15
> 2 08:45
> 2 09:15
> 2 09:45
> 2 10:45
> 2 11:15
> 2 11:45
>
> I hope this helps!
>

[julia-users] Re: Quadratic equation solver ?

[Kristoffer Carlsson]

You would probably get more focused help 
in https://groups.google.com/forum/#!forum/julia-opt

On Tuesday, May 31, 2016 at 12:59:49 PM UTC+2, tann...@gmail.com wrote:
>
> which one is a good  solver(open source) for quadratic equation using jump 
> ? Thank you 
>

Re: [julia-users] Re: Quadratic equation solver ?

[Michele Zaffalon]

JuMP is a modeling language for linear and quadratic optimization problems,
see http://www.juliaopt.org/

On Tue, May 31, 2016 at 1:26 PM, Mosè Giordano 
wrote:

> which one is a good  solver(open source) for quadratic equation using jump
>> ? Thank you
>>
>
> What is jump?
>
> PolynomialRoots.jl is a
> package to find all roots (real and complex, they will have always Complex
> type) of polynomials of any degree, with a specialized function for
> quadratic equations.  Though, I don't know if this is what you're looking
> for.
>
> Bye,
> Mosè
>

[julia-users] Re: Quadratic equation solver ?

[Mosè Giordano]

>
> which one is a good  solver(open source) for quadratic equation using jump 
> ? Thank you 
>

What is jump?

PolynomialRoots.jl is a 
package to find all roots (real and complex, they will have always Complex 
type) of polynomials of any degree, with a specialized function for 
quadratic equations.  Though, I don't know if this is what you're looking 
for.

Bye,
Mosè

Re: [julia-users] Re: Double free or corruption (out)

['Bill Hart' via julia-users]

We are also suddenly getting crashes with 2.4.5. when running our (Nemo)
test suite. It says that some memory allocation is failing due to invalid
next size. I suspect there is a bug that wasn't there until the last few
days, since we were passing just fine on Travis. Though at this stage, I
haven't checked whether we are still passing on Travis.

Bill.

On 31 May 2016 at 12:52, Nils Gudat  wrote:

> Resurrecting this very old thread - after having been able to solve the
> model with no seg faults over the last couple of months, they have now
> returned and occur much faster (usually within 2 hours of running the code).
> I have refactored the code a little so that it hopefully will be possible
> for others to run it. Cloning the entire repo at
> http://github.com/nilshg/LearningModels, it should run when altering the
> path in
> https://github.com/nilshg/LearningModels/blob/master/NHL/NHL_maximize.jl
> to whatever path it has been cloned to.
>
> I'm running this code on a 16-core Ubuntu 14.04 machine with Julia 0.4.5
> installed an all packages on the latest tagged versions.
>
> On Tuesday, September 29, 2015 at 1:43:31 PM UTC+1, Nils Gudat wrote:
>>
>> The code usually segfaults after 2-5 hours, and is available at
>> http://github.com/nilshg/LearningModels, however I haven't written it up
>> in a way that is easy to run (right now it depends on some data not
>> included in the repo), so I'll have to restructure a bit before you can run
>> it. I'll try to do so today if I find the time.
>>
>

[julia-users] Re: What should Julia do on "Ubuntu compatibility mode in Windows 10"?

[lovebufan]

Just run Ubuntu version of Julia on Ubuntu mode on windows 10. 
Everything seems good.

On Sunday, April 3, 2016 at 3:27:04 AM UTC+8, Páll Haraldsson wrote:
>
> I guess just run as a regular Linux ELF binary..
>
> Since the news, and Windows Julia works, we can just ignore and run Julia 
> in non-Ubuntu mode..
>
> -- 
> Palli.
>
>

[julia-users] Quadratic equation solver ?

[tannirind]

which one is a good  solver(open source) for quadratic equation using jump 
? Thank you 

Re: [julia-users] Re: Double free or corruption (out)

[Nils Gudat]

Resurrecting this very old thread - after having been able to solve the 
model with no seg faults over the last couple of months, they have now 
returned and occur much faster (usually within 2 hours of running the code).
I have refactored the code a little so that it hopefully will be possible 
for others to run it. Cloning the entire repo at 
http://github.com/nilshg/LearningModels, it should run when altering the 
path 
in https://github.com/nilshg/LearningModels/blob/master/NHL/NHL_maximize.jl 
to whatever path it has been cloned to.

I'm running this code on a 16-core Ubuntu 14.04 machine with Julia 0.4.5 
installed an all packages on the latest tagged versions.

On Tuesday, September 29, 2015 at 1:43:31 PM UTC+1, Nils Gudat wrote:
>
> The code usually segfaults after 2-5 hours, and is available at 
> http://github.com/nilshg/LearningModels, however I haven't written it up 
> in a way that is easy to run (right now it depends on some data not 
> included in the repo), so I'll have to restructure a bit before you can run 
> it. I'll try to do so today if I find the time.
>

Re: [julia-users] keyword arguments and anonymous function - ERROR: syntax: "begin args... kwargs... end" is not a valid function argument name

[Mauro]

> How can I update to this version ?

You can download it (http://julialang.org/downloads/ nightly builds) or
build it yourself.  Best to have both 0.4 and 0.5 installed.  But note
that 0.5 is in development, i.e. not stable!

> Is it sure enough to do it ?

?

> If 0.5 is not released, when it will be ?

The release date is not known yet, presumably in the next few month.

> https://github.com/JuliaLang/julia/issues?utf8=%E2%9C%93=is%3Aissue+is%3Aopen+roadmap
> or
> https://github.com/JuliaLang/julia/issues?utf8=%E2%9C%93=is%3Aissue+is%3Aopen+0.5
>
> doesn't help
>
>
> Le mardi 31 mai 2016 09:49:38 UTC+2, Mauro a écrit :
>>
>> This works on 0.5, where anonymous functions are almost like normal
>> functions now.
>>
>> On Tue, 2016-05-31 at 09:26, Femto Trader > > wrote:
>> > Hello,
>> >
>> > I'm trying to define an anonymous function
>> >
>> >
>> > julia> (args...) -> begin
>> >println("hello anonymous func with args")
>> >end
>> > (anonymous function)
>> >
>> > works correctly
>> >
>> > A "classic" with keywords arguments can be define using
>> >
>> > julia> function f(args...; kwargs...)
>> >println("hello classic func with args and kwargs")
>> >end
>> > f (generic function with 1 method)
>> >
>> > But it doesn't work with keyword arguments
>> >
>> >
>> > julia> (args...; kwargs...) -> begin
>> >println("hello anonymous func with args and kwargs")
>> >end
>> > ERROR: syntax: "begin
>> > args...
>> > kwargs...
>> > end" is not a valid function argument name
>> >
>> >
>> > I don't understand if that's an issue or a misunderstanding from my
>> side.
>> >
>> > Any idea ?
>> >
>> >
>> > Kind regards
>>

Re: [julia-users] keyword arguments and anonymous function - ERROR: syntax: "begin args... kwargs... end" is not a valid function argument name

[Femto Trader]

How can I update to this version ?
Is it sure enough to do it ?
If 0.5 is not released, when it will be ?

https://github.com/JuliaLang/julia/issues?utf8=%E2%9C%93=is%3Aissue+is%3Aopen+roadmap
or
https://github.com/JuliaLang/julia/issues?utf8=%E2%9C%93=is%3Aissue+is%3Aopen+0.5

doesn't help


Le mardi 31 mai 2016 09:49:38 UTC+2, Mauro a écrit :
>
> This works on 0.5, where anonymous functions are almost like normal 
> functions now. 
>
> On Tue, 2016-05-31 at 09:26, Femto Trader  > wrote: 
> > Hello, 
> > 
> > I'm trying to define an anonymous function 
> > 
> > 
> > julia> (args...) -> begin 
> >println("hello anonymous func with args") 
> >end 
> > (anonymous function) 
> > 
> > works correctly 
> > 
> > A "classic" with keywords arguments can be define using 
> > 
> > julia> function f(args...; kwargs...) 
> >println("hello classic func with args and kwargs") 
> >end 
> > f (generic function with 1 method) 
> > 
> > But it doesn't work with keyword arguments 
> > 
> > 
> > julia> (args...; kwargs...) -> begin 
> >println("hello anonymous func with args and kwargs") 
> >end 
> > ERROR: syntax: "begin 
> > args... 
> > kwargs... 
> > end" is not a valid function argument name 
> > 
> > 
> > I don't understand if that's an issue or a misunderstanding from my 
> side. 
> > 
> > Any idea ? 
> > 
> > 
> > Kind regards 
>

[julia-users] Re: Calculus.hessian result oddities

[Kristoffer Carlsson]

Doesn't Calculu's Hessian just do finite difference? There will naturally 
be floating point errors.

isposdef returns false if the matrix is not hermitian, under the hood, 
potrf from Lapack is used which work with hermitian matrices.

On Tuesday, May 31, 2016 at 7:04:04 AM UTC+2, Boylan, Ross wrote:
>
> The hessian produced by Calculus.hessian has some properties that seem odd 
> to me.  First, the resulting matrix is not symmetric (though it's really 
> close).  Second, it is not positive definite even though the eigenvalues 
> are all positive.  This might partly be the result of the first problem.
>
> Comments?
>
>  julia> @time h6 = Calculus.hessian(RB.mylike, sol6[2])
>  206.988247 seconds (2.43 G allocations: 83.830 GB, 15.48% gc time)
>  10x10 Array{Float64,2}:
> 
>  julia> issym(h6)
>  false
>  julia> isposdef(h6)
>  false
>  julia> eigvals(h6)
>  10-element Array{Float64,1}:
>   3.27548e5
>   41407.1
>1873.44
> 809.515
> 385.469
> 336.118
> 269.82
>  64.4228
> 117.009
> 122.845
>  julia> [det(h6[1:i, 1:i]) for i in 1:10]  # all submatrix determinants 
> are positive
>  10-element Array{Any,1}:
>   583.356
> 2.93666e5
> 3.76334e7
> 4.26776e9
> 3.17529e12
> 6.53405e14
> 1.12477e17
> 8.54011e21
> 3.75658e26
> 6.65866e29
>
>  julia> showall(h6)
>  [583.356378500377 -215.9519377704782 42.707753088841706 
> -12.046905619427504 100.26418659571075 30.907721733245413 
> 5.1999446295105445 -3522.0797654066855 4410.401341876791 29.123944464570087
>   -215.95193777047822 583.3512842422355 -12.0619061856546 
> 42.69860798380086 -82.01778794681003 -20.00373356894301 -8.643894790795585 
> 1484.6195212852567 -3316.905502014349 -2.7622075180025965
>   42.7077530888417 -12.061906185654598 131.30467049554812 
> -44.14799847519552 16.748039679197223 -10.98960509547607 43.02830286169418 
> 234.68906012694512 -1258.6622801992237 9.86994636336174
>   -12.046905619427504 42.69860798380086 -44.147998475195514 
> 131.35743705036847 -17.498616283805696 4.724998443749094 
> -31.481387268916205 -95.390157895702 209.5245372890838 12.903636963625035
>   100.26418659571075 -82.01778794681003 16.748039679197223 
> -17.498616283805696 766.6651018177663 -20.766430168130857 
> -2.164074090378522 610.206909770971 -6927.858446549729 -200.4409796490271
>   30.907721733245413 -20.00373356894301 -10.98960509547607 
> 4.724998443749093 -20.766430168130857 209.8283757716871 -60.222027409043555 
> 280.3888402431943 -1638.62573982097 -158.60599205247166
>   5.199944629510544 -8.643894790795585 43.02830286169418 -31.4813872689162 
> -2.164074090378524 -60.222027409043555 204.34068072875016 
> -3.631354965653126 314.50383342119943 30.12788292183477
>   -3522.079765406686 1484.6195212852565 234.6890601269451 
> -95.39015789570199 610.2069097709712 280.3888402431943 -3.631354965653125 
> 102596.98982032725 -117630.58331133435 267.23733374838395
>   4410.401341876791 -3316.9055020143487 -1258.6622801992235 
> 209.5245372890838 -6927.858446549729 -1638.62573982097 314.5038334211995 
> -117630.58331133435 265600.50878023007 5508.687364627178
>   29.123944464570084 -2.762207518002596 9.86994636336174 
> 12.903636963625033 -200.4409796490271 -158.6059920524717 30.12788292183477 
> 267.23733374838395 5508.687364627178 2125.8954716959997]
>
> The largest absolute difference between symmetrical entries is < 5e-13.
>
> Ross Boylan
>
>

Re: [julia-users] keyword arguments and anonymous function - ERROR: syntax: "begin args... kwargs... end" is not a valid function argument name

[Mauro]

This works on 0.5, where anonymous functions are almost like normal
functions now.

On Tue, 2016-05-31 at 09:26, Femto Trader  wrote:
> Hello,
>
> I'm trying to define an anonymous function
>
>
> julia> (args...) -> begin
>println("hello anonymous func with args")
>end
> (anonymous function)
>
> works correctly
>
> A "classic" with keywords arguments can be define using
>
> julia> function f(args...; kwargs...)
>println("hello classic func with args and kwargs")
>end
> f (generic function with 1 method)
>
> But it doesn't work with keyword arguments
>
>
> julia> (args...; kwargs...) -> begin
>println("hello anonymous func with args and kwargs")
>end
> ERROR: syntax: "begin
> args...
> kwargs...
> end" is not a valid function argument name
>
>
> I don't understand if that's an issue or a misunderstanding from my side.
>
> Any idea ?
>
>
> Kind regards

[julia-users] keyword arguments and anonymous function - ERROR: syntax: "begin args... kwargs... end" is not a valid function argument name

[Femto Trader]

Hello,

I'm trying to define an anonymous function


julia> (args...) -> begin
   println("hello anonymous func with args")
   end
(anonymous function)

works correctly

A "classic" with keywords arguments can be define using

julia> function f(args...; kwargs...)
   println("hello classic func with args and kwargs")
   end
f (generic function with 1 method)

But it doesn't work with keyword arguments


julia> (args...; kwargs...) -> begin
   println("hello anonymous func with args and kwargs")
   end
ERROR: syntax: "begin
args...
kwargs...
end" is not a valid function argument name


I don't understand if that's an issue or a misunderstanding from my side.

Any idea ?


Kind regards

Re: [julia-users] Re: ANN: FileIO.jl, for loading and saving formatted files

[David van Leeuwen]

Hello, 

Picking up an old thread... I am looking for a way to trigger a specific 
constructor when loading an object from a JLD file.  

FileIO detects the files are JLD, and that is great, but it would be cool 
if it could trigger my constructor `MyModule.MyType(objectfromjld)` when
 - `MyModule` was specified in the addrequires while writing the file
 - `JLD.exists(io, "MyType")` (namely the object `objectfromjld`)

(The case here is that MyType contains a field that is largish and that can 
be precomputed from another field in the type, which is the want I want to 
store in the file). 

I've tried to write a `detectmytype()` in the spirit of, say, 
`FileIO.detecthdf5()`, but I am kind-of stuck here: 
 - the detect...() interface wants an IO object (I think)
 - I can find out if `MyType` exists from `jldopen(filename)`, but I can't 
find an interface for hdf5/jld for an IO object. 

Does anyone know how I could get this done?

Thanks, 

---david

On Wednesday, September 30, 2015 at 11:35:20 AM UTC+2, Tim Holy wrote:
>
> Probably best as an issue over at FileIO. 
>
> --Tim 
>
> On Wednesday, September 30, 2015 01:24:25 AM David van Leeuwen wrote: 
> > Hello Tim and Simon, 
> > 
> > I'd be very interested in this, for MFCCs.jl and perhaps 
> > GaussianMixtures.jl.  They currently use HDF5 and JLD, respectively, and 
> I 
> > don't think I've put in any magic. 
> > 
> >  - is it wise to put in magic, even with container formats such as HDF5? 
> > 
> >  - do you prefer questions about implementation on this list, the repo, 
> or 
> > some other way (or not at all)? 
> > 
> > Cheers, 
> > 
> > ---david 
> > 
> > On Tuesday, September 29, 2015 at 9:24:13 PM UTC+2, Tim Holy wrote: 
> > > Simon Danisch and I are pleased to introduce FileIO.jl 
> > > (https://github.com/JuliaIO/FileIO.jl), a package for centralizing 
> the 
> > > handling of formatted files. The package contains utilities for 
> querying 
> > > files/filenames by their extensions and/or magic bytes to deduce their 
> > > content. 
> > > It then allows package authors to "register" loaders/savers for 
> detected 
> > > file 
> > > formats. 
> > > 
> > > Some of the advantages of using FileIO are: 
> > > 
> > > - For users, this may reduce the searching required to answer "is 
> there a 
> > > julia package to load file X?" Now you can just try `load("X.ext")` 
> and 
> > > see 
> > > what happens---if the loader has been registered, it will try to find 
> the 
> > > right 
> > > package for you. 
> > > 
> > > - For developers, this allows you to decouple your package from 
> others. If 
> > > you 
> > > want to work with images, you will (once I tag a new release of 
> Images.jl) 
> > > no 
> > > longer need to make the Images package a dependency as long as you use 
> > > FileIO. 
> > > In your package code, you can say `load("myimage.png")` and everything 
> > > should 
> > > Just Work. 
> > > 
> > > - For users and developers, this reduces name conflicts among two 
> > > attractive 
> > > function names, `load` and `save`. If everyone imports those names 
> from 
> > > FileIO, there need be no conflicts. 
> > > 
> > > The package is relatively small and is fairly extensively documented. 
> > > 
> > > While there are not a large number of formats currently registered, 
> the 
> > > architecture already shows a lot of promise. For example, it can 
> recognize 
> > > our 
> > > .jld files as HDF5 files, something that escapes the well-tested (and 
> much 
> > > more 
> > > extensive) Unix `file` command. 
> > > 
> > > Best, 
> > > --Tim & Simon 
>
>