[julia-users] Re: Help to eliminate Calendar.jl dependence
timestamp questions for Tom: (a) is the most refined temporal resolution of your timestamps milliseconds (1/1_000 seconds)? (b) do you ever get multiple items that have identical timestamps? (If so, what is the most you have seen.) (c) do you want your timestamp to be an integer (best), a float (not a win), or a string? (If string, all digits or not.) These timestamps will be derived from UT not from localtime (I do the converting), to do otherwise invites various problems down the road. Are all time values given in your local timezone? The timestamp function(s) are somewhat configurable; still, I would like to ensure that your use case is covered. Regards On Wednesday, July 8, 2015 at 10:59:33 AM UTC-4, Tom Breloff wrote: I have some code which requires figuring out the number of seconds from the Epoch until midnight (local time) in order to quickly compute the local TimeOfDay. The reason is that I get passed a field which is seconds since Epoch, and I'd like to just subtract off the (cached) # seconds from Epoch--Midnight. Since I'm using a cached number, I don't care so much how long it takes to calculate. Right now I use both Dates and Calendar.jl, but I'm wondering if I can accomplish this without the dependency on Calendar.jl (which I currently use ONLY to get the hours offset between Eastern US and UTC). Is there a better way to write this function? function getHoursAdjustmentFromUTC(year::Integer, month::Integer, day::Integer) millisEST = *Calendar.ymd*(year, month, day, EST5EDT).millis millisUTC = *Calendar.ymd*(year, month, day, UTC).millis UInt64(round((millisEST - millisUTC) / (secondsInOneHour * millisInOneSecond))) end getEpochMillis() = UInt64(DateTime(1970,1,1).instant.periods.value) createUTCDateTimeFromSecondsSinceEpoch(secondsSinceEpoch::Integer) = DateTime(Dates.UTM(secondsSinceEpoch * millisInOneSecond + getEpochMillis())) # this is the function I care about... note that midnight refers to midnight local to Eastern US function calcSecondsEpochToMidnight(secondsSinceEpoch::Integer) dt = createUTCDateTimeFromSecondsSinceEpoch(secondsSinceEpoch) # get the hour adjustment using the Calendar module y = Dates.year(dt) m = Dates.month(dt) d = Dates.day(dt) hourAdjustment = getHoursAdjustmentFromUTC(y, m, d) millisMidnightUTC::UInt64 = DateTime(y, m, d).instant.periods.value millisMidnightEST::UInt64 = millisMidnightUTC + hourAdjustment * secondsInOneHour * millisInOneSecond return UInt64((millisMidnightEST - getEpochMillis()) / millisInOneSecond) end
[julia-users] Add a new dict with 2 different types into a new array in a structured dict
How can I add a dict with 2 different field types (1D array of float and
float) to an array nested in a dict that already exists?
The 'allmeas' structure already exists up to this point:
allmeas[measkey][manreps][num_manrep]
And I'm trying to add like this:
allmeas[measkey][manreps][num_manrep][autoreps] = Array{Dict{Any,Any},1}
num_autorep = 1
meas_1D_array = [1,2,3,4,5,6]
meas_single_value = 10
newmeas = {meas_1D_array= meas_1D_array ,meas_single_value=
meas_single_value}
allmeas[measkey][manreps][num_manrep][autoreps][num_autorep] = newmeas
The error I get is on the last line:
`setindex!` has no method matching setindex!(::Type{Array{Dict{Any,Any},1}},
::Dict{Any,Any}, ::Int64)
If I look at the types of each side, I get:
typeof(allmeas[measkey][manreps][num_manrep][autoreps]) is
Array{Dict{Any,Any},1}
typeof(newmeas) is Dict{Any,Any}
Re: [julia-users] is @inline redundant with fn(a::Int) = do_something(a) ?
If that's all there is to a definition it's pretty likely to get inlined, but you can write arbitrarily complex expressions, so it's not quite the same. I really don't think we should make using one-liner vs longer function form significant in terms of performance. That would encourage people writing awkwardly long one-liners for all the wrong reasons. On Mon, Aug 10, 2015 at 1:01 AM, Jeffrey Sarnoff jeffrey.sarn...@gmail.com wrote: Is there a reason to add @inline to functions that are defined using the form fn(x) = do_fn(x), or is it redundant in v0.4?
[julia-users] Re: julia 'stable' from git
Thanks! So something like git pull git checkout release-0.4 make should work... Any idea of when 0.4 will be out as stable (I know it is out as development)? Cheers F On Monday, 10 August 2015 16:50:12 UTC+3, Scott T wrote: In fact, I imagine there will be a release-0.4 branch instead of a tag, so you will probably be able to ignore the git fetch --tags and do something more like git checkout release-0.4. Anyway, my point being that quite a lot will change in v0.4 so it's best to run the upgrade yourself (and you can do it without deleting everything and starting again). There will definitely be instructions available! Scott On Monday, 10 August 2015 14:44:07 UTC+1, Scott T wrote: Hi Federico, I think that what you need is as simple as doing something like git fetch --tags git pull git checkout v0.4.0 make from within the git repo once version 0.4.0 is out. It's not quite magical, but you will probably want to push the button on the upgrade yourself instead of having it take you by surprise. Cheers, Scott On Monday, 10 August 2015 12:07:51 UTC+1, Federico Calboli wrote: Hi All, (assuming this is the right forum), according to the instructions here: https://github.com/JuliaLang/julia I can run the 'stable' release by cloning the git repo and then checking out the 0.3 release. Now, I presume the 0.4 release will eventually become 'stable', and so on and so forth. My lack of git-foo menas that I would then remove the whole /usr/local/julia directory and start from scratch with the new stable release. This is doable but I was wondering whether there is a way of tracking the stable release that will magically upgrade the whole thing to the next stable release when it is available. Cheers F
[julia-users] 0.4 used to load fast
You should try julia --precompile=yes . I never took the time to search why this was done, though.
[julia-users] Re: 0.4 used to load fast
Thanks. But this should be the default behavior, no? segunda-feira, 10 de Agosto de 2015 às 15:33:22 UTC+1, Simon Danisch escreveu: You should try julia --precompile=yes . I never took the time to search why this was done, though.
[julia-users] Re: julia 'stable' from git
Within the next month unless something unexpected happens, I'd say - you can follow discussion on the last few milestones here. https://github.com/JuliaLang/julia/milestones/0.4.0 Cheers, Scott On Monday, 10 August 2015 17:14:41 UTC+1, Federico Calboli wrote: Thanks! So something like git pull git checkout release-0.4 make should work... Any idea of when 0.4 will be out as stable (I know it is out as development)? Cheers F On Monday, 10 August 2015 16:50:12 UTC+3, Scott T wrote: In fact, I imagine there will be a release-0.4 branch instead of a tag, so you will probably be able to ignore the git fetch --tags and do something more like git checkout release-0.4. Anyway, my point being that quite a lot will change in v0.4 so it's best to run the upgrade yourself (and you can do it without deleting everything and starting again). There will definitely be instructions available! Scott On Monday, 10 August 2015 14:44:07 UTC+1, Scott T wrote: Hi Federico, I think that what you need is as simple as doing something like git fetch --tags git pull git checkout v0.4.0 make from within the git repo once version 0.4.0 is out. It's not quite magical, but you will probably want to push the button on the upgrade yourself instead of having it take you by surprise. Cheers, Scott On Monday, 10 August 2015 12:07:51 UTC+1, Federico Calboli wrote: Hi All, (assuming this is the right forum), according to the instructions here: https://github.com/JuliaLang/julia I can run the 'stable' release by cloning the git repo and then checking out the 0.3 release. Now, I presume the 0.4 release will eventually become 'stable', and so on and so forth. My lack of git-foo menas that I would then remove the whole /usr/local/julia directory and start from scratch with the new stable release. This is doable but I was wondering whether there is a way of tracking the stable release that will magically upgrade the whole thing to the next stable release when it is available. Cheers F
Re: [julia-users] Really need to change immutables
but the problem now is that *TS* is a memory owned by Julia and that will
be fatal when GMT's own memory cleaning functions try to free it and -
Julia crashes.
getting memory ownership right is absolutely critical when writing C
interop code. It is also a place where Julia gives you a large number of
tools (struct-compatible type layout, Ref, , Ptr, c_malloc, unsafe_load),
because there is no feasible way to detect the intent of the C code
automatically.
usually, there's is one big toplevel distinction driven by who will free
the memory: if C will be freeing the memory, then you must always have the
C code also allocating the memory (hopefully the library author provide
some sort of new / alloc / create function, although sometimes they rely on
the user calling malloc).
How do we do to change immutable members that are not pointers?
ccall goes to great lengths to make sure you don't accidentally do this,
because you semantically can't change a value, and Julia uses this to
implement many optimizations. And even if you managed to change one of the
instances of the value, Julia is free to delete / reuse the memory
containing that modification (or share it with something else that had the
same value) even before the call to `pointer_from_objref` returns.
Is that really impossible and I'm doomed to the path exemplified with
that C GMT_blind_change_struct() function?
ideally, C libraries that expect to manage a struct's memory also provide
accessor methods to read any of the structs important fields so that you
can always declare its type as Ptr{Void} in Julia. However, I realize this
is true of many C libraries.
On Mon, Aug 10, 2015 at 9:58 AM J Luis jmfl...@gmail.com wrote:
segunda-feira, 10 de Agosto de 2015 às 14:15:15 UTC+1, Isaiah escreveu:
https://github.com/JuliaLang/julia/pull/12113
So there is hope. Thanks
On Mon, Aug 10, 2015 at 8:11 AM, J Luis jmf...@gmail.com wrote:
Hi,
Let me try to explain my need to fight with immutables. I made another
round to my GMT wrapper (which is hopefully almost ready) and struggled
again with the immutables. The point is that GMT API structs are deeply
nested and need to be mirrored with immutables (otherwise Julia crashes).
Namely this one
https://github.com/joa-quim/GMT.jl/blob/master/src/libgmt_h.jl#L1445
so I first went with the advised solution of creating and filling the
type is julia and replace the pointees of the immutable type, which I did
here (see the commented code):
https://github.com/joa-quim/GMT.jl/blob/master/src/gmt_main.jl#L874
but the problem now is that *TS* is a memory owned by Julia and that
will be fatal when GMT's own memory cleaning functions try to free it and
- Julia crashes.
So if use a pure Julia solution I've no choice but to let memory leeks
and other house keeping to be done. Not good at all.
For the time being I ended up with this ugly C trick of sending to C
the job of changing some members (non-pointers) that need absolutely to
changed after the type has been created
http://gmt.soest.hawaii.edu/projects/gmt/repository/entry/branches/5.2.0/src/gmt_api.c#L7376
but besides being ugly I've no guarantee that I can convince my partners
to let that backdoor go into the GMT lib. I much lovelier trick would be if
this chunk only of that backdoor function would work.
else if (!strncmp (keyword, API_POINTER_, 12U)) {/*
Blindly change a pointer to a scalar. Irritatingly Julia ignores this */
if (!strcmp(keyword[12], UINT64))
*(uint64_t *)ptr = *(uint64_t *)what;
}
but the problem is that I am not able to get from Julia the pointers
address that would make that work. I tried with
pointer(S0.n_rows)
and
pointer_from_objref(S0.n_rows)
and others, but nothing. It seams to work in the C side (I can see it
with the debugger when I set in that piece of code) but than nothing
changes. The C struct doesn't change.
Is there a way to get the C pointer address from Julia?
So, I guess my question is:
How do we do to change immutable members that are not pointers?
Is that really impossible and I'm doomed to the path exemplified
with that C GMT_blind_change_struct() function?
[julia-users] Re: 0.4 used to load fast
On Monday, August 10, 2015 at 11:46:17 AM UTC-4, J Luis wrote: Thanks. But this should be the default behavior, no? Apparently it's due to a bug (affecting backtraces) that will be fixed in LLVM 3.5: https://github.com/JuliaLang/julia/commit/f98decceb6437fb160a5da2666fe7845f7f4f73c
Re: [julia-users] Really need to change immutables
usually, there's is one big toplevel distinction driven by who will free
the memory: if C will be freeing the memory, then you must always have the
C code also allocating the memory (hopefully the library author provide
some sort of new / alloc / create function, although sometimes they rely on
the user calling malloc).
Yes, the GMT_Create_Data (that ccals a C function with the same name) does
that.
https://github.com/joa-quim/GMT.jl/blob/master/src/gmt_main.jl#L852
https://github.com/joa-quim/GMT.jl/blob/master/src/gmt_main.jl#L852
Is that really impossible and I'm doomed to the path exemplified
with that C GMT_blind_change_struct() function?
ideally, C libraries that expect to manage a struct's memory also provide
accessor methods to read any of the structs important fields so that you
can always declare its type as Ptr{Void} in Julia. However, I realize this
is true of many C libraries.
The pointers are ok, my real problem are those 'scalars' like the n_rows in
http://gmt.soest.hawaii.edu/projects/gmt/repository/entry/branches/5.2.0/src/gmt_resources.h#L517
I only know the number of rows after having read the data and than the
immut is already created
On Mon, Aug 10, 2015 at 9:58 AM J Luis jmf...@gmail.com javascript:
wrote:
segunda-feira, 10 de Agosto de 2015 às 14:15:15 UTC+1, Isaiah escreveu:
https://github.com/JuliaLang/julia/pull/12113
So there is hope. Thanks
On Mon, Aug 10, 2015 at 8:11 AM, J Luis jmf...@gmail.com wrote:
Hi,
Let me try to explain my need to fight with immutables. I made another
round to my GMT wrapper (which is hopefully almost ready) and struggled
again with the immutables. The point is that GMT API structs are deeply
nested and need to be mirrored with immutables (otherwise Julia crashes).
Namely this one
https://github.com/joa-quim/GMT.jl/blob/master/src/libgmt_h.jl#L1445
so I first went with the advised solution of creating and filling the
type is julia and replace the pointees of the immutable type, which I did
here (see the commented code):
https://github.com/joa-quim/GMT.jl/blob/master/src/gmt_main.jl#L874
but the problem now is that *TS* is a memory owned by Julia and that
will be fatal when GMT's own memory cleaning functions try to free it and
- Julia crashes.
So if use a pure Julia solution I've no choice but to let memory leeks
and other house keeping to be done. Not good at all.
For the time being I ended up with this ugly C trick of sending to C
the job of changing some members (non-pointers) that need absolutely to
changed after the type has been created
http://gmt.soest.hawaii.edu/projects/gmt/repository/entry/branches/5.2.0/src/gmt_api.c#L7376
but besides being ugly I've no guarantee that I can convince my
partners to let that backdoor go into the GMT lib. I much lovelier trick
would be if this chunk only of that backdoor function would work.
else if (!strncmp (keyword, API_POINTER_, 12U)) {/*
Blindly change a pointer to a scalar. Irritatingly Julia ignores this */
if (!strcmp(keyword[12], UINT64))
*(uint64_t *)ptr = *(uint64_t *)what;
}
but the problem is that I am not able to get from Julia the pointers
address that would make that work. I tried with
pointer(S0.n_rows)
and
pointer_from_objref(S0.n_rows)
and others, but nothing. It seams to work in the C side (I can see it
with the debugger when I set in that piece of code) but than nothing
changes. The C struct doesn't change.
Is there a way to get the C pointer address from Julia?
So, I guess my question is:
How do we do to change immutable members that are not pointers?
Is that really impossible and I'm doomed to the path exemplified
with that C GMT_blind_change_struct() function?
Re: [julia-users] is @inline redundant with fn(a::Int) = do_something(a) ?
got it On Mon, Aug 10, 2015 at 12:38 PM, Stefan Karpinski ste...@karpinski.org wrote: If that's all there is to a definition it's pretty likely to get inlined, but you can write arbitrarily complex expressions, so it's not quite the same. I really don't think we should make using one-liner vs longer function form significant in terms of performance. That would encourage people writing awkwardly long one-liners for all the wrong reasons. On Mon, Aug 10, 2015 at 1:01 AM, Jeffrey Sarnoff jeffrey.sarn...@gmail.com wrote: Is there a reason to add @inline to functions that are defined using the form fn(x) = do_fn(x), or is it redundant in v0.4?
[julia-users] Creating a show method for type alias
Hi, I have implemented a basic immutable type with a type alias for a
vector of said type:
immutable CIGAR
OP::Operation
Size::Int
end
function CIGAR(op::Char, size::Int)
return CIGAR(Operation(op), size)
end
function convert(::Type{String}, cigar::CIGAR)
return $(cigar.Size)$(Char(cigar.OP))
end
function show(io::IO, cigar::CIGAR)
write(io, convert(String, cigar))
end
typealias CIGARString Vector{CIGAR}
function convert(::Type{CIGARString}, str::String)
matches = matchall(r(\d+)(\w), str)
cigarString = Vector{CIGAR}(length(matches))
@inbounds for i in 1:length(matches)
m = matches[i]
cigarString[i] = CIGAR(m[end], parse(Int, m[1:end-1]))
end
return cigarString
end
macro cigar_str(str)
return CIGARString(str)
end
I also want to define a show method for the alias CIGARString, so as it is
converted to a string that can be used with a show method:
function convert(::Type{String}, cigarString::CIGARString)
outString =
for cigar in cigarString
outString *= String(cigar)
end
return outString
end
function show(io::IO, cigarstr::CIGARString)
write(io, convert(String, cigarstr))
end
However the output a see on the REPL is:
*3-element Array{Bio.Align.CIGAR,1}:*
* 5M*
* 5N*
* 5M*
So, rather than the show method for CIGARString being called, the show
method for CIGAR is being called repeatedly for every element in the
CIGARString vector. How do I get Julia to use the show method I want for
CIGARString?
Thanks.
[julia-users] Re: 0.4 used to load fast
Ah okay, I suspected this. I wasn't convinced though. I compared the stack traces for precompiled=yes/no for the cases that really annoyed me and it seemed precompiled=no never gave better results. Guess I haven't hit the edge cases yet. But given these findings, I would greatly prefer precompiled=yes as the default, and if I'm unhappy with the stack traces I would want to switch to precompiled=no temporarily. I'll comment on the PR. Am Montag, 10. August 2015 16:17:17 UTC+2 schrieb J Luis: On Windows here. Some months ago the local build with MinGW used to start almost instantaneously. Now it takes almost 4 seconds again.
Re: [julia-users] Really need to change immutables
and will I not fall in the same case as the *TS* above? That is, when I do
the unsafe_store will that memory be owned by the C lib (and therefore safe
to be free by it)?
segunda-feira, 10 de Agosto de 2015 às 21:13:39 UTC+1, Jameson escreveu:
right, you can't mutate it directly, but you can make a new copy with the
intended changes, then use unsafe_store to write those changes back into
the struct.
On Mon, Aug 10, 2015 at 3:56 PM J Luis jmf...@gmail.com javascript:
wrote:
... amd I am using unsafe_load()
S0 = unsafe_load(unsafe_load(TTABLE.segment,1),1)#
::GMT.GMT_TEXTSEGMENT
https://github.com/joa-quim/GMT.jl/blob/master/src/gmt_main.jl#L860
but if I add this line after
S0.n_rows = 2
ERROR: type GMT_TEXTSEGMENT is immutable
in GMTJL_Text_init at C:\progs_cygw\GMTdev\GMT.jl\src\gmt_main.jl:861
segunda-feira, 10 de Agosto de 2015 às 20:47:27 UTC+1, J Luis escreveu:
segunda-feira, 10 de Agosto de 2015 às 20:15:57 UTC+1, Jameson escreveu:
My suggestion would be to mirror only the parts of those structs that
are part of the public API. Then use unsafe_load / unsafe_store to read /
modify / write the intended changes. The API authors were nice to put the
public fields at the top so that you can do this (many libraries don't /
can't do this). Then the real object will always be the Ptr to the one
allocated by GMT, and you only have temporary copies of them alive at any
time inside Julia.
But GMT_TEXTSEGMENT is part of the GMT API (hm, have to check why it
doesn't show up in http://gmt.soest.hawaii.edu/doc/latest/GMT_API.html)
Sorry, to be concrete, and repeating myself. I need to change n_rows of
http://gmt.soest.hawaii.edu/projects/gmt/repository/entry/branches/5.2.0/src/gmt_resources.h#L517
Can I do it?
BTW, I'm a developer of GMT (that's why I could create that backdoor
function that temporarily saved me on this), although the API was all
created by P. Wessel.
On Mon, Aug 10, 2015 at 2:53 PM J Luis jmf...@gmail.com wrote:
usually, there's is one big toplevel distinction driven by who will
free the memory: if C will be freeing the memory, then you must always
have
the C code also allocating the memory (hopefully the library author
provide
some sort of new / alloc / create function, although sometimes they rely
on
the user calling malloc).
Yes, the GMT_Create_Data (that ccals a C function with the same name)
does that.
https://github.com/joa-quim/GMT.jl/blob/master/src/gmt_main.jl#L852
https://github.com/joa-quim/GMT.jl/blob/master/src/gmt_main.jl#L852
Is that really impossible and I'm doomed to the path
exemplified with that C GMT_blind_change_struct() function?
ideally, C libraries that expect to manage a struct's memory also
provide accessor methods to read any of the structs important fields so
that you can always declare its type as Ptr{Void} in Julia. However, I
realize this is true of many C libraries.
The pointers are ok, my real problem are those 'scalars' like the
n_rows in
http://gmt.soest.hawaii.edu/projects/gmt/repository/entry/branches/5.2.0/src/gmt_resources.h#L517
I only know the number of rows after having read the data and than the
immut is already created
On Mon, Aug 10, 2015 at 9:58 AM J Luis jmf...@gmail.com wrote:
segunda-feira, 10 de Agosto de 2015 às 14:15:15 UTC+1, Isaiah
escreveu:
https://github.com/JuliaLang/julia/pull/12113
So there is hope. Thanks
On Mon, Aug 10, 2015 at 8:11 AM, J Luis jmf...@gmail.com wrote:
Hi,
Let me try to explain my need to fight with immutables. I made
another round to my GMT wrapper (which is hopefully almost ready) and
struggled again with the immutables. The point is that GMT API
structs are
deeply nested and need to be mirrored with immutables (otherwise
Julia
crashes). Namely this one
https://github.com/joa-quim/GMT.jl/blob/master/src/libgmt_h.jl#L1445
so I first went with the advised solution of creating and filling
the type is julia and replace the pointees of the immutable type,
which I
did here (see the commented code):
https://github.com/joa-quim/GMT.jl/blob/master/src/gmt_main.jl#L874
but the problem now is that *TS* is a memory owned by Julia and
that will be fatal when GMT's own memory cleaning functions try to
free it
and - Julia crashes.
So if use a pure Julia solution I've no choice but to let memory
leeks and other house keeping to be done. Not good at all.
For the time being I ended up with this ugly C trick of sending
to C the job of changing some members (non-pointers) that need
absolutely
to changed after the type has been created
http://gmt.soest.hawaii.edu/projects/gmt/repository/entry/branches/5.2.0/src/gmt_api.c#L7376
but besides being ugly I've no guarantee that I can convince my
partners to let that backdoor go into the GMT lib. I much lovelier
trick
RE: [julia-users] Re: 0.4 used to load fast
I agree. In my mind bad backtraces are a bug, certainly that is how I as a user experience them, and we wouldn’t want to make a buggy but fast behavior the default. From: julia-users@googlegroups.com [mailto:julia-users@googlegroups.com] On Behalf Of Tony Kelman Sent: Monday, August 10, 2015 1:16 PM To: julia-users julia-users@googlegroups.com Subject: [julia-users] Re: 0.4 used to load fast I commented on the commit, but I disagree with making worse backtraces the default. If you know what you're doing and want fast startup in exchange for worse backtraces, you can use --precompiled=yes (and we should probably document that more clearly), but for the sake of all of those users who don't know what they're doing, we can't make bad backtraces the default behavior. On Monday, August 10, 2015 at 10:49:51 AM UTC-7, Simon Danisch wrote: Ah okay, I suspected this. I wasn't convinced though. I compared the stack traces for precompiled=yes/no for the cases that really annoyed me and it seemed precompiled=no never gave better results. Guess I haven't hit the edge cases yet. But given these findings, I would greatly prefer precompiled=yes as the default, and if I'm unhappy with the stack traces I would want to switch to precompiled=no temporarily. I'll comment on the PR. Am Montag, 10. August 2015 16:17:17 UTC+2 schrieb J Luis: On Windows here. Some months ago the local build with MinGW used to start almost instantaneously. Now it takes almost 4 seconds again.
Re: [julia-users] Really need to change immutables
... amd I am using unsafe_load()
S0 = unsafe_load(unsafe_load(TTABLE.segment,1),1)#
::GMT.GMT_TEXTSEGMENT
https://github.com/joa-quim/GMT.jl/blob/master/src/gmt_main.jl#L860
but if I add this line after
S0.n_rows = 2
ERROR: type GMT_TEXTSEGMENT is immutable
in GMTJL_Text_init at C:\progs_cygw\GMTdev\GMT.jl\src\gmt_main.jl:861
segunda-feira, 10 de Agosto de 2015 às 20:47:27 UTC+1, J Luis escreveu:
segunda-feira, 10 de Agosto de 2015 às 20:15:57 UTC+1, Jameson escreveu:
My suggestion would be to mirror only the parts of those structs that are
part of the public API. Then use unsafe_load / unsafe_store to read /
modify / write the intended changes. The API authors were nice to put the
public fields at the top so that you can do this (many libraries don't /
can't do this). Then the real object will always be the Ptr to the one
allocated by GMT, and you only have temporary copies of them alive at any
time inside Julia.
But GMT_TEXTSEGMENT is part of the GMT API (hm, have to check why it
doesn't show up in http://gmt.soest.hawaii.edu/doc/latest/GMT_API.html)
Sorry, to be concrete, and repeating myself. I need to change n_rows of
http://gmt.soest.hawaii.edu/projects/gmt/repository/entry/branches/5.2.0/src/gmt_resources.h#L517
Can I do it?
BTW, I'm a developer of GMT (that's why I could create that backdoor
function that temporarily saved me on this), although the API was all
created by P. Wessel.
On Mon, Aug 10, 2015 at 2:53 PM J Luis jmf...@gmail.com wrote:
usually, there's is one big toplevel distinction driven by who will
free the memory: if C will be freeing the memory, then you must always
have
the C code also allocating the memory (hopefully the library author
provide
some sort of new / alloc / create function, although sometimes they rely
on
the user calling malloc).
Yes, the GMT_Create_Data (that ccals a C function with the same name)
does that.
https://github.com/joa-quim/GMT.jl/blob/master/src/gmt_main.jl#L852
https://github.com/joa-quim/GMT.jl/blob/master/src/gmt_main.jl#L852
Is that really impossible and I'm doomed to the path exemplified
with that C GMT_blind_change_struct() function?
ideally, C libraries that expect to manage a struct's memory also
provide accessor methods to read any of the structs important fields so
that you can always declare its type as Ptr{Void} in Julia. However, I
realize this is true of many C libraries.
The pointers are ok, my real problem are those 'scalars' like the n_rows
in
http://gmt.soest.hawaii.edu/projects/gmt/repository/entry/branches/5.2.0/src/gmt_resources.h#L517
I only know the number of rows after having read the data and than the
immut is already created
On Mon, Aug 10, 2015 at 9:58 AM J Luis jmf...@gmail.com wrote:
segunda-feira, 10 de Agosto de 2015 às 14:15:15 UTC+1, Isaiah escreveu:
https://github.com/JuliaLang/julia/pull/12113
So there is hope. Thanks
On Mon, Aug 10, 2015 at 8:11 AM, J Luis jmf...@gmail.com wrote:
Hi,
Let me try to explain my need to fight with immutables. I made
another round to my GMT wrapper (which is hopefully almost ready) and
struggled again with the immutables. The point is that GMT API structs
are
deeply nested and need to be mirrored with immutables (otherwise Julia
crashes). Namely this one
https://github.com/joa-quim/GMT.jl/blob/master/src/libgmt_h.jl#L1445
so I first went with the advised solution of creating and filling
the type is julia and replace the pointees of the immutable type, which
I
did here (see the commented code):
https://github.com/joa-quim/GMT.jl/blob/master/src/gmt_main.jl#L874
but the problem now is that *TS* is a memory owned by Julia and that
will be fatal when GMT's own memory cleaning functions try to free it
and
- Julia crashes.
So if use a pure Julia solution I've no choice but to let memory
leeks and other house keeping to be done. Not good at all.
For the time being I ended up with this ugly C trick of sending to
C the job of changing some members (non-pointers) that need absolutely
to
changed after the type has been created
http://gmt.soest.hawaii.edu/projects/gmt/repository/entry/branches/5.2.0/src/gmt_api.c#L7376
but besides being ugly I've no guarantee that I can convince my
partners to let that backdoor go into the GMT lib. I much lovelier
trick
would be if this chunk only of that backdoor function would work.
else if (!strncmp (keyword, API_POINTER_, 12U)) {/*
Blindly change a pointer to a scalar. Irritatingly Julia ignores this */
if (!strcmp(keyword[12], UINT64))
*(uint64_t *)ptr = *(uint64_t *)what;
}
but the problem is that I am not able to get from Julia the pointers
address that would make that work. I tried with
pointer(S0.n_rows)
and
pointer_from_objref(S0.n_rows)
and
Re: [julia-users] Really need to change immutables
right, you can't mutate it directly, but you can make a new copy with the
intended changes, then use unsafe_store to write those changes back into
the struct.
On Mon, Aug 10, 2015 at 3:56 PM J Luis jmfl...@gmail.com wrote:
... amd I am using unsafe_load()
S0 = unsafe_load(unsafe_load(TTABLE.segment,1),1)#
::GMT.GMT_TEXTSEGMENT
https://github.com/joa-quim/GMT.jl/blob/master/src/gmt_main.jl#L860
but if I add this line after
S0.n_rows = 2
ERROR: type GMT_TEXTSEGMENT is immutable
in GMTJL_Text_init at C:\progs_cygw\GMTdev\GMT.jl\src\gmt_main.jl:861
segunda-feira, 10 de Agosto de 2015 às 20:47:27 UTC+1, J Luis escreveu:
segunda-feira, 10 de Agosto de 2015 às 20:15:57 UTC+1, Jameson escreveu:
My suggestion would be to mirror only the parts of those structs that
are part of the public API. Then use unsafe_load / unsafe_store to read /
modify / write the intended changes. The API authors were nice to put the
public fields at the top so that you can do this (many libraries don't /
can't do this). Then the real object will always be the Ptr to the one
allocated by GMT, and you only have temporary copies of them alive at any
time inside Julia.
But GMT_TEXTSEGMENT is part of the GMT API (hm, have to check why it
doesn't show up in http://gmt.soest.hawaii.edu/doc/latest/GMT_API.html)
Sorry, to be concrete, and repeating myself. I need to change n_rows of
http://gmt.soest.hawaii.edu/projects/gmt/repository/entry/branches/5.2.0/src/gmt_resources.h#L517
Can I do it?
BTW, I'm a developer of GMT (that's why I could create that backdoor
function that temporarily saved me on this), although the API was all
created by P. Wessel.
On Mon, Aug 10, 2015 at 2:53 PM J Luis jmf...@gmail.com wrote:
usually, there's is one big toplevel distinction driven by who will
free the memory: if C will be freeing the memory, then you must always
have
the C code also allocating the memory (hopefully the library author
provide
some sort of new / alloc / create function, although sometimes they rely
on
the user calling malloc).
Yes, the GMT_Create_Data (that ccals a C function with the same name)
does that.
https://github.com/joa-quim/GMT.jl/blob/master/src/gmt_main.jl#L852
https://github.com/joa-quim/GMT.jl/blob/master/src/gmt_main.jl#L852
Is that really impossible and I'm doomed to the path exemplified
with that C GMT_blind_change_struct() function?
ideally, C libraries that expect to manage a struct's memory also
provide accessor methods to read any of the structs important fields so
that you can always declare its type as Ptr{Void} in Julia. However, I
realize this is true of many C libraries.
The pointers are ok, my real problem are those 'scalars' like the
n_rows in
http://gmt.soest.hawaii.edu/projects/gmt/repository/entry/branches/5.2.0/src/gmt_resources.h#L517
I only know the number of rows after having read the data and than the
immut is already created
On Mon, Aug 10, 2015 at 9:58 AM J Luis jmf...@gmail.com wrote:
segunda-feira, 10 de Agosto de 2015 às 14:15:15 UTC+1, Isaiah
escreveu:
https://github.com/JuliaLang/julia/pull/12113
So there is hope. Thanks
On Mon, Aug 10, 2015 at 8:11 AM, J Luis jmf...@gmail.com wrote:
Hi,
Let me try to explain my need to fight with immutables. I made
another round to my GMT wrapper (which is hopefully almost ready) and
struggled again with the immutables. The point is that GMT API structs
are
deeply nested and need to be mirrored with immutables (otherwise Julia
crashes). Namely this one
https://github.com/joa-quim/GMT.jl/blob/master/src/libgmt_h.jl#L1445
so I first went with the advised solution of creating and filling
the type is julia and replace the pointees of the immutable type,
which I
did here (see the commented code):
https://github.com/joa-quim/GMT.jl/blob/master/src/gmt_main.jl#L874
but the problem now is that *TS* is a memory owned by Julia and
that will be fatal when GMT's own memory cleaning functions try to
free it
and - Julia crashes.
So if use a pure Julia solution I've no choice but to let memory
leeks and other house keeping to be done. Not good at all.
For the time being I ended up with this ugly C trick of sending to
C the job of changing some members (non-pointers) that need absolutely
to
changed after the type has been created
http://gmt.soest.hawaii.edu/projects/gmt/repository/entry/branches/5.2.0/src/gmt_api.c#L7376
but besides being ugly I've no guarantee that I can convince my
partners to let that backdoor go into the GMT lib. I much lovelier
trick
would be if this chunk only of that backdoor function would work.
else if (!strncmp (keyword, API_POINTER_, 12U)) {/*
Blindly change a pointer to a scalar. Irritatingly Julia ignores this
*/
if (!strcmp(keyword[12], UINT64))
*(uint64_t *)ptr = *(uint64_t *)what;
}
but the problem is that I am not
Re: [julia-users] Re: Can Julia function be serialized and sent by network?
JLD doesn't support serializing functions but Julia itself does. On Mon, Aug 10, 2015 at 3:43 PM, Andrei Zh faithlessfri...@gmail.com wrote: I'm afraid it's not quite true, and I found simple way to show it. In the next code snippet I define function `f` and serialize it to a file: julia f(x) = x + 1 f (generic function with 1 method) julia f(5) 6 julia open(example.jld, w) do io serialize(io, f) end Then I close Julia REPL and in a new session try to load and use this function: julia f2 = open(example.jld) do io deserialize(io) end (anonymous function) julia f2(5) ERROR: function f not defined on process 1 in error at error.jl:21 in anonymous at serialize.jl:398 So deserialized function still refers to the old definition, which is not available in this new session. Is there any better way to serialize a function and run it on an unrelated Julia process? On Monday, August 10, 2015 at 2:33:11 PM UTC+3, Jeff Waller wrote: My question is: does Julia's serialization produce completely self-containing code that can be run on workers? In other words, is it possible to send serialized function over network to another host / Julia process and applied there without any additional information from the first process? I made some tests on a single machine, and when I defined function without `@everywhere`, worker failed with a message function myfunc not defined on process 1. With `@everywhere`, my code worked, but will it work on multiple hosts with essentially independent Julia processes? According to Jey here https://groups.google.com/forum/#!searchin/julia-users/jey/julia-users/bolLGcSCrs0/fGGVLgNhI2YJ, Base.serialize does what we want; it's contained in serialize.jl https://github.com/JuliaLang/julia/blob/master/base/serialize.jl
[julia-users] Re: julia 'stable' from git
You would likely need at least `make clean`, probably `make cleanall` as well when switching between release-0.3 and the forthcoming release-0.4 branch. Many of the dependency versions have changed, though not all. It will likely be more reliable to at least rebuild the dependencies that have changed versions, which `make cleanall` should usually do for you - it deletes the installed copies from ./usr which forces the right version to be re-installed, though it won't completely delete the built copies of the dependencies from ./deps. On Monday, August 10, 2015 at 10:14:31 AM UTC-7, Scott T wrote: Within the next month unless something unexpected happens, I'd say - you can follow discussion on the last few milestones here. https://github.com/JuliaLang/julia/milestones/0.4.0 Cheers, Scott On Monday, 10 August 2015 17:14:41 UTC+1, Federico Calboli wrote: Thanks! So something like git pull git checkout release-0.4 make should work... Any idea of when 0.4 will be out as stable (I know it is out as development)? Cheers F On Monday, 10 August 2015 16:50:12 UTC+3, Scott T wrote: In fact, I imagine there will be a release-0.4 branch instead of a tag, so you will probably be able to ignore the git fetch --tags and do something more like git checkout release-0.4. Anyway, my point being that quite a lot will change in v0.4 so it's best to run the upgrade yourself (and you can do it without deleting everything and starting again). There will definitely be instructions available! Scott On Monday, 10 August 2015 14:44:07 UTC+1, Scott T wrote: Hi Federico, I think that what you need is as simple as doing something like git fetch --tags git pull git checkout v0.4.0 make from within the git repo once version 0.4.0 is out. It's not quite magical, but you will probably want to push the button on the upgrade yourself instead of having it take you by surprise. Cheers, Scott On Monday, 10 August 2015 12:07:51 UTC+1, Federico Calboli wrote: Hi All, (assuming this is the right forum), according to the instructions here: https://github.com/JuliaLang/julia I can run the 'stable' release by cloning the git repo and then checking out the 0.3 release. Now, I presume the 0.4 release will eventually become 'stable', and so on and so forth. My lack of git-foo menas that I would then remove the whole /usr/local/julia directory and start from scratch with the new stable release. This is doable but I was wondering whether there is a way of tracking the stable release that will magically upgrade the whole thing to the next stable release when it is available. Cheers F
Re: [julia-users] Re: Can Julia function be serialized and sent by network?
On Monday, August 10, 2015 01:13:15 PM Tony Kelman wrote: Should probably use some different extension for that, .jls or something, to avoid confusion. Yes. That has been sufficiently confusing in the past, we even cover this here: https://github.com/JuliaLang/JLD.jl#saving-and-loading-variables-in-julia-data-format-jld --Tim On Monday, August 10, 2015 at 12:45:35 PM UTC-7, Stefan Karpinski wrote: JLD doesn't support serializing functions but Julia itself does. On Mon, Aug 10, 2015 at 3:43 PM, Andrei Zh faithle...@gmail.com javascript: wrote: I'm afraid it's not quite true, and I found simple way to show it. In the next code snippet I define function `f` and serialize it to a file: julia f(x) = x + 1 f (generic function with 1 method) julia f(5) 6 julia open(example.jld, w) do io serialize(io, f) end Then I close Julia REPL and in a new session try to load and use this function: julia f2 = open(example.jld) do io deserialize(io) end (anonymous function) julia f2(5) ERROR: function f not defined on process 1 in error at error.jl:21 in anonymous at serialize.jl:398 So deserialized function still refers to the old definition, which is not available in this new session. Is there any better way to serialize a function and run it on an unrelated Julia process? On Monday, August 10, 2015 at 2:33:11 PM UTC+3, Jeff Waller wrote: My question is: does Julia's serialization produce completely self-containing code that can be run on workers? In other words, is it possible to send serialized function over network to another host / Julia process and applied there without any additional information from the first process? I made some tests on a single machine, and when I defined function without `@everywhere`, worker failed with a message function myfunc not defined on process 1. With `@everywhere`, my code worked, but will it work on multiple hosts with essentially independent Julia processes? According to Jey here https://groups.google.com/forum/#!searchin/julia-users/jey/julia-users/ bolLGcSCrs0/fGGVLgNhI2YJ, Base.serialize does what we want; it's contained in serialize.jl https://github.com/JuliaLang/julia/blob/master/base/serialize.jl
[julia-users] Re: PSA: auto-precompile your modules
Sorry to follow up, but I've confirmed this is the case (it was also in the original announcement; sorry for overlooking it). In any case - during this testing phase, would it be possible to make __precompile__() work only for packages that have explicitly opted in, so that dependencies that cannot be precompiled don't suddenly start causing failure of the top-level package? I'd love to have precompilation for LightGraphs, but I don't really want to have to depend on the precompile-safeness of the dependencies to maintain a passing build status for LightGraphs. On Monday, August 10, 2015 at 2:03:54 PM UTC-7, Seth wrote: Hi, I'm running into a weird problem that I think is related to precompilation. It seems that __precompile__() will attempt to precompile all dependencies as well (that is, if you're using Foo, Foo will get precompiled). Is this true? In my case, it appears that the precompilation in LightGraphs causes a failure due to ERROR: LoadError: LoadError: UndefRefError: access to undefined reference in LightXML, which may not be precompile-safe. Thanks for any advice. On Thursday, August 6, 2015 at 9:39:23 PM UTC-7, Steven G. Johnson wrote: Many of you are aware that Julia 0.4 has some facilities for precompiling modules, but in the last couple of days they have become significantly more automated, which should make much faster load times accessible to all your users in 0.4. If your module (and any modules it imports) are safe to precompile, then just add: VERSION = v0.4.0-dev+6521 __precompile__() at the top of your module file (*before* module ... end), and it will be automatically precompiled (to a cached .ji file in ~/.julia/lib) the first time it is imported. Thereafter, if any of the dependencies (e.g. the included files or imported modules) are updated, it will automatically be recompiled the next time it is imported. See the Julia manual section on modules and precompilation http://docs.julialang.org/en/latest/manual/modules/#module-initialization-and-precompilationto find out how to make your module safe for precompiling. (If your module is *not* safe for precompiling yet and you don't have time to fix it, use __precompile__(false) to prevent your module from being accidentally precompiled, e.g. by being imported into another module that is precompiled.) --SGJ
Re: [julia-users] Embarrassingly parallel workload
What am I doing wrong in the following code? function foo(N; parallel=false) if parallel nprocs() CPU_CORES addprocs(CPU_CORES - nprocs()) end result = SharedArray(Float64, 9, N) @parallel for i=1:N sleep(1) result[:,i] = rand(3,3)[:] end result end If I call foo(60, parallel=true), result is all zeros. Expected behavior is a random matrix. -Júlio
[julia-users] Re: PSA: auto-precompile your modules
Hi, I'm running into a weird problem that I think is related to precompilation. It seems that __precompile__() will attempt to precompile all dependencies as well (that is, if you're using Foo, Foo will get precompiled). Is this true? In my case, it appears that the precompilation in LightGraphs causes a failure due to ERROR: LoadError: LoadError: UndefRefError: access to undefined reference in LightXML, which may not be precompile-safe. Thanks for any advice. On Thursday, August 6, 2015 at 9:39:23 PM UTC-7, Steven G. Johnson wrote: Many of you are aware that Julia 0.4 has some facilities for precompiling modules, but in the last couple of days they have become significantly more automated, which should make much faster load times accessible to all your users in 0.4. If your module (and any modules it imports) are safe to precompile, then just add: VERSION = v0.4.0-dev+6521 __precompile__() at the top of your module file (*before* module ... end), and it will be automatically precompiled (to a cached .ji file in ~/.julia/lib) the first time it is imported. Thereafter, if any of the dependencies (e.g. the included files or imported modules) are updated, it will automatically be recompiled the next time it is imported. See the Julia manual section on modules and precompilation http://docs.julialang.org/en/latest/manual/modules/#module-initialization-and-precompilationto find out how to make your module safe for precompiling. (If your module is *not* safe for precompiling yet and you don't have time to fix it, use __precompile__(false) to prevent your module from being accidentally precompiled, e.g. by being imported into another module that is precompiled.) --SGJ
[julia-users] Re: julia 'stable' from git
Great! On Monday, August 10, 2015 at 10:10:36 PM UTC+2, Tony Kelman wrote: You would likely need at least `make clean`, probably `make cleanall` as well when switching between release-0.3 and the forthcoming release-0.4 branch. Many of the dependency versions have changed, though not all. It will likely be more reliable to at least rebuild the dependencies that have changed versions, which `make cleanall` should usually do for you - it deletes the installed copies from ./usr which forces the right version to be re-installed, though it won't completely delete the built copies of the dependencies from ./deps. On Monday, August 10, 2015 at 10:14:31 AM UTC-7, Scott T wrote: Within the next month unless something unexpected happens, I'd say - you can follow discussion on the last few milestones here. https://github.com/JuliaLang/julia/milestones/0.4.0 Cheers, Scott On Monday, 10 August 2015 17:14:41 UTC+1, Federico Calboli wrote: Thanks! So something like git pull git checkout release-0.4 make should work... Any idea of when 0.4 will be out as stable (I know it is out as development)? Cheers F On Monday, 10 August 2015 16:50:12 UTC+3, Scott T wrote: In fact, I imagine there will be a release-0.4 branch instead of a tag, so you will probably be able to ignore the git fetch --tags and do something more like git checkout release-0.4. Anyway, my point being that quite a lot will change in v0.4 so it's best to run the upgrade yourself (and you can do it without deleting everything and starting again). There will definitely be instructions available! Scott On Monday, 10 August 2015 14:44:07 UTC+1, Scott T wrote: Hi Federico, I think that what you need is as simple as doing something like git fetch --tags git pull git checkout v0.4.0 make from within the git repo once version 0.4.0 is out. It's not quite magical, but you will probably want to push the button on the upgrade yourself instead of having it take you by surprise. Cheers, Scott On Monday, 10 August 2015 12:07:51 UTC+1, Federico Calboli wrote: Hi All, (assuming this is the right forum), according to the instructions here: https://github.com/JuliaLang/julia I can run the 'stable' release by cloning the git repo and then checking out the 0.3 release. Now, I presume the 0.4 release will eventually become 'stable', and so on and so forth. My lack of git-foo menas that I would then remove the whole /usr/local/julia directory and start from scratch with the new stable release. This is doable but I was wondering whether there is a way of tracking the stable release that will magically upgrade the whole thing to the next stable release when it is available. Cheers F
[julia-users] Re: PSA: auto-precompile your modules
Issue filed: https://github.com/JuliaLang/julia/issues/12545 On Monday, August 10, 2015 at 2:18:25 PM UTC-7, Seth wrote: Sorry to follow up, but I've confirmed this is the case (it was also in the original announcement; sorry for overlooking it). In any case - during this testing phase, would it be possible to make __precompile__() work only for packages that have explicitly opted in, so that dependencies that cannot be precompiled don't suddenly start causing failure of the top-level package? I'd love to have precompilation for LightGraphs, but I don't really want to have to depend on the precompile-safeness of the dependencies to maintain a passing build status for LightGraphs. On Monday, August 10, 2015 at 2:03:54 PM UTC-7, Seth wrote: Hi, I'm running into a weird problem that I think is related to precompilation. It seems that __precompile__() will attempt to precompile all dependencies as well (that is, if you're using Foo, Foo will get precompiled). Is this true? In my case, it appears that the precompilation in LightGraphs causes a failure due to ERROR: LoadError: LoadError: UndefRefError: access to undefined reference in LightXML, which may not be precompile-safe. Thanks for any advice. On Thursday, August 6, 2015 at 9:39:23 PM UTC-7, Steven G. Johnson wrote: Many of you are aware that Julia 0.4 has some facilities for precompiling modules, but in the last couple of days they have become significantly more automated, which should make much faster load times accessible to all your users in 0.4. If your module (and any modules it imports) are safe to precompile, then just add: VERSION = v0.4.0-dev+6521 __precompile__() at the top of your module file (*before* module ... end), and it will be automatically precompiled (to a cached .ji file in ~/.julia/lib) the first time it is imported. Thereafter, if any of the dependencies (e.g. the included files or imported modules) are updated, it will automatically be recompiled the next time it is imported. See the Julia manual section on modules and precompilation http://docs.julialang.org/en/latest/manual/modules/#module-initialization-and-precompilationto find out how to make your module safe for precompiling. (If your module is *not* safe for precompiling yet and you don't have time to fix it, use __precompile__(false) to prevent your module from being accidentally precompiled, e.g. by being imported into another module that is precompiled.) --SGJ
Re: [julia-users] Computer Vision Package
but I might still need a layer between Images.jl and my package because most of tracking algorithm uses only grayscale images, but still not sure about that. Based on my experience (re)implementing active appearance models, I'd say many tracking algorithms work even better with colored images than with grayscale ones. Anyway, you can always convert data to a format of your choice (e.g. 2D or 3D arrays), process in the algorithm and convert the result back to Image.
Re: [julia-users] Computer Vision Package
On Monday, August 10, 2015 01:49:08 PM Mehmet Cem Güntürkün wrote: However, for that I will need Scharr Operator. Do we have that operator in Images.jl? See http://timholy.github.io/Images.jl/function_reference.html and search for sobel (which is close). But if you want a different one, sure, that would be great to add. --Tim
[julia-users] use === for comparison to nothing
I saw this in recent commits. What's the purpose of it?
Re: [julia-users] Really need to change immutables
My suggestion would be to mirror only the parts of those structs that are
part of the public API. Then use unsafe_load / unsafe_store to read /
modify / write the intended changes. The API authors were nice to put the
public fields at the top so that you can do this (many libraries don't /
can't do this). Then the real object will always be the Ptr to the one
allocated by GMT, and you only have temporary copies of them alive at any
time inside Julia.
On Mon, Aug 10, 2015 at 2:53 PM J Luis jmfl...@gmail.com wrote:
usually, there's is one big toplevel distinction driven by who will free
the memory: if C will be freeing the memory, then you must always have the
C code also allocating the memory (hopefully the library author provide
some sort of new / alloc / create function, although sometimes they rely on
the user calling malloc).
Yes, the GMT_Create_Data (that ccals a C function with the same name) does
that.
https://github.com/joa-quim/GMT.jl/blob/master/src/gmt_main.jl#L852
https://github.com/joa-quim/GMT.jl/blob/master/src/gmt_main.jl#L852
Is that really impossible and I'm doomed to the path exemplified
with that C GMT_blind_change_struct() function?
ideally, C libraries that expect to manage a struct's memory also provide
accessor methods to read any of the structs important fields so that you
can always declare its type as Ptr{Void} in Julia. However, I realize this
is true of many C libraries.
The pointers are ok, my real problem are those 'scalars' like the n_rows in
http://gmt.soest.hawaii.edu/projects/gmt/repository/entry/branches/5.2.0/src/gmt_resources.h#L517
I only know the number of rows after having read the data and than the
immut is already created
On Mon, Aug 10, 2015 at 9:58 AM J Luis jmf...@gmail.com wrote:
segunda-feira, 10 de Agosto de 2015 às 14:15:15 UTC+1, Isaiah escreveu:
https://github.com/JuliaLang/julia/pull/12113
So there is hope. Thanks
On Mon, Aug 10, 2015 at 8:11 AM, J Luis jmf...@gmail.com wrote:
Hi,
Let me try to explain my need to fight with immutables. I made another
round to my GMT wrapper (which is hopefully almost ready) and struggled
again with the immutables. The point is that GMT API structs are deeply
nested and need to be mirrored with immutables (otherwise Julia crashes).
Namely this one
https://github.com/joa-quim/GMT.jl/blob/master/src/libgmt_h.jl#L1445
so I first went with the advised solution of creating and filling the
type is julia and replace the pointees of the immutable type, which I did
here (see the commented code):
https://github.com/joa-quim/GMT.jl/blob/master/src/gmt_main.jl#L874
but the problem now is that *TS* is a memory owned by Julia and that
will be fatal when GMT's own memory cleaning functions try to free it and
- Julia crashes.
So if use a pure Julia solution I've no choice but to let memory leeks
and other house keeping to be done. Not good at all.
For the time being I ended up with this ugly C trick of sending to C
the job of changing some members (non-pointers) that need absolutely to
changed after the type has been created
http://gmt.soest.hawaii.edu/projects/gmt/repository/entry/branches/5.2.0/src/gmt_api.c#L7376
but besides being ugly I've no guarantee that I can convince my
partners to let that backdoor go into the GMT lib. I much lovelier trick
would be if this chunk only of that backdoor function would work.
else if (!strncmp (keyword, API_POINTER_, 12U)) {/*
Blindly change a pointer to a scalar. Irritatingly Julia ignores this */
if (!strcmp(keyword[12], UINT64))
*(uint64_t *)ptr = *(uint64_t *)what;
}
but the problem is that I am not able to get from Julia the pointers
address that would make that work. I tried with
pointer(S0.n_rows)
and
pointer_from_objref(S0.n_rows)
and others, but nothing. It seams to work in the C side (I can see it
with the debugger when I set in that piece of code) but than nothing
changes. The C struct doesn't change.
Is there a way to get the C pointer address from Julia?
So, I guess my question is:
How do we do to change immutable members that are not pointers?
Is that really impossible and I'm doomed to the path exemplified
with that C GMT_blind_change_struct() function?
[julia-users] Re: Can Julia function be serialized and sent by network?
I'm afraid it's not quite true, and I found simple way to show it. In the next code snippet I define function `f` and serialize it to a file: julia f(x) = x + 1 f (generic function with 1 method) julia f(5) 6 julia open(example.jld, w) do io serialize(io, f) end Then I close Julia REPL and in a new session try to load and use this function: julia f2 = open(example.jld) do io deserialize(io) end (anonymous function) julia f2(5) ERROR: function f not defined on process 1 in error at error.jl:21 in anonymous at serialize.jl:398 So deserialized function still refers to the old definition, which is not available in this new session. Is there any better way to serialize a function and run it on an unrelated Julia process? On Monday, August 10, 2015 at 2:33:11 PM UTC+3, Jeff Waller wrote: My question is: does Julia's serialization produce completely self-containing code that can be run on workers? In other words, is it possible to send serialized function over network to another host / Julia process and applied there without any additional information from the first process? I made some tests on a single machine, and when I defined function without `@everywhere`, worker failed with a message function myfunc not defined on process 1. With `@everywhere`, my code worked, but will it work on multiple hosts with essentially independent Julia processes? According to Jey here https://groups.google.com/forum/#!searchin/julia-users/jey/julia-users/bolLGcSCrs0/fGGVLgNhI2YJ, Base.serialize does what we want; it's contained in serialize.jl https://github.com/JuliaLang/julia/blob/master/base/serialize.jl
[julia-users] Will be in Geneva later this week
Hello everyone, I'll be traveling to Geneva later this week (Thu/Fri). Not sure if there's a meet-up group around, but if you're in the area, I'd love to meet up while I'm there. Keno
Re: [julia-users] julia 'stable' from git
I was on 0.4 but daily updates from git just broke too often… I always did make clean as a norm because it saved time in the end… Having said that, since my GCC is 4.9.x, and I understand GCC has moved to 5.x.x I might just change GCC and thus reinstall from 0… Cheers F On 10 Aug 2015, at 23:10, Tony Kelman t...@kelman.net wrote: You would likely need at least `make clean`, probably `make cleanall` as well when switching between release-0.3 and the forthcoming release-0.4 branch. Many of the dependency versions have changed, though not all. It will likely be more reliable to at least rebuild the dependencies that have changed versions, which `make cleanall` should usually do for you - it deletes the installed copies from ./usr which forces the right version to be re-installed, though it won't completely delete the built copies of the dependencies from ./deps. On Monday, August 10, 2015 at 10:14:31 AM UTC-7, Scott T wrote: Within the next month unless something unexpected happens, I'd say - you can follow discussion on the last few milestones here. Cheers, Scott On Monday, 10 August 2015 17:14:41 UTC+1, Federico Calboli wrote: Thanks! So something like git pull git checkout release-0.4 make should work... Any idea of when 0.4 will be out as stable (I know it is out as development)? Cheers F On Monday, 10 August 2015 16:50:12 UTC+3, Scott T wrote: In fact, I imagine there will be a release-0.4 branch instead of a tag, so you will probably be able to ignore the git fetch --tags and do something more like git checkout release-0.4. Anyway, my point being that quite a lot will change in v0.4 so it's best to run the upgrade yourself (and you can do it without deleting everything and starting again). There will definitely be instructions available! Scott On Monday, 10 August 2015 14:44:07 UTC+1, Scott T wrote: Hi Federico, I think that what you need is as simple as doing something like git fetch --tags git pull git checkout v0.4.0 make from within the git repo once version 0.4.0 is out. It's not quite magical, but you will probably want to push the button on the upgrade yourself instead of having it take you by surprise. Cheers, Scott On Monday, 10 August 2015 12:07:51 UTC+1, Federico Calboli wrote: Hi All, (assuming this is the right forum), according to the instructions here: https://github.com/JuliaLang/julia I can run the 'stable' release by cloning the git repo and then checking out the 0.3 release. Now, I presume the 0.4 release will eventually become 'stable', and so on and so forth. My lack of git-foo menas that I would then remove the whole /usr/local/julia directory and start from scratch with the new stable release. This is doable but I was wondering whether there is a way of tracking the stable release that will magically upgrade the whole thing to the next stable release when it is available. Cheers F -- Federico Calboli f.calb...@gmail.com
Re: [julia-users] Re: Can Julia function be serialized and sent by network?
The above code wasn't using the HDF5-based JLD package/format, it was just using .jld as a file extension to store the results of serialize(). Should probably use some different extension for that, .jls or something, to avoid confusion. On Monday, August 10, 2015 at 12:45:35 PM UTC-7, Stefan Karpinski wrote: JLD doesn't support serializing functions but Julia itself does. On Mon, Aug 10, 2015 at 3:43 PM, Andrei Zh faithle...@gmail.com javascript: wrote: I'm afraid it's not quite true, and I found simple way to show it. In the next code snippet I define function `f` and serialize it to a file: julia f(x) = x + 1 f (generic function with 1 method) julia f(5) 6 julia open(example.jld, w) do io serialize(io, f) end Then I close Julia REPL and in a new session try to load and use this function: julia f2 = open(example.jld) do io deserialize(io) end (anonymous function) julia f2(5) ERROR: function f not defined on process 1 in error at error.jl:21 in anonymous at serialize.jl:398 So deserialized function still refers to the old definition, which is not available in this new session. Is there any better way to serialize a function and run it on an unrelated Julia process? On Monday, August 10, 2015 at 2:33:11 PM UTC+3, Jeff Waller wrote: My question is: does Julia's serialization produce completely self-containing code that can be run on workers? In other words, is it possible to send serialized function over network to another host / Julia process and applied there without any additional information from the first process? I made some tests on a single machine, and when I defined function without `@everywhere`, worker failed with a message function myfunc not defined on process 1. With `@everywhere`, my code worked, but will it work on multiple hosts with essentially independent Julia processes? According to Jey here https://groups.google.com/forum/#!searchin/julia-users/jey/julia-users/bolLGcSCrs0/fGGVLgNhI2YJ, Base.serialize does what we want; it's contained in serialize.jl https://github.com/JuliaLang/julia/blob/master/base/serialize.jl
[julia-users] Re: Help to eliminate Calendar.jl dependence
Hi Tom, I am writing you a timestamp function now. (a) is the most refined temporal resolution of your timestamps milliseconds (1/1_000 seconds)? (b) do you ever get multiple items that have identical timestamps? (If so, what is the most you have seen.) (c) do you want your timestamp to be an integer (best), a float (not a win), or a string? (If string, all digits or not.) These timestamps will be derived from UT not from localtime (I do the converting), to do otherwise invites various problems down the road. Are all time values given in your local timezone? On Wednesday, July 8, 2015 at 10:59:33 AM UTC-4, Tom Breloff wrote: I have some code which requires figuring out the number of seconds from the Epoch until midnight (local time) in order to quickly compute the local TimeOfDay. The reason is that I get passed a field which is seconds since Epoch, and I'd like to just subtract off the (cached) # seconds from Epoch--Midnight. Since I'm using a cached number, I don't care so much how long it takes to calculate. Right now I use both Dates and Calendar.jl, but I'm wondering if I can accomplish this without the dependency on Calendar.jl (which I currently use ONLY to get the hours offset between Eastern US and UTC). Is there a better way to write this function? function getHoursAdjustmentFromUTC(year::Integer, month::Integer, day::Integer) millisEST = *Calendar.ymd*(year, month, day, EST5EDT).millis millisUTC = *Calendar.ymd*(year, month, day, UTC).millis UInt64(round((millisEST - millisUTC) / (secondsInOneHour * millisInOneSecond))) end getEpochMillis() = UInt64(DateTime(1970,1,1).instant.periods.value) createUTCDateTimeFromSecondsSinceEpoch(secondsSinceEpoch::Integer) = DateTime(Dates.UTM(secondsSinceEpoch * millisInOneSecond + getEpochMillis())) # this is the function I care about... note that midnight refers to midnight local to Eastern US function calcSecondsEpochToMidnight(secondsSinceEpoch::Integer) dt = createUTCDateTimeFromSecondsSinceEpoch(secondsSinceEpoch) # get the hour adjustment using the Calendar module y = Dates.year(dt) m = Dates.month(dt) d = Dates.day(dt) hourAdjustment = getHoursAdjustmentFromUTC(y, m, d) millisMidnightUTC::UInt64 = DateTime(y, m, d).instant.periods.value millisMidnightEST::UInt64 = millisMidnightUTC + hourAdjustment * secondsInOneHour * millisInOneSecond return UInt64((millisMidnightEST - getEpochMillis()) / millisInOneSecond) end
Re: [julia-users] Really need to change immutables
segunda-feira, 10 de Agosto de 2015 às 20:15:57 UTC+1, Jameson escreveu:
My suggestion would be to mirror only the parts of those structs that are
part of the public API. Then use unsafe_load / unsafe_store to read /
modify / write the intended changes. The API authors were nice to put the
public fields at the top so that you can do this (many libraries don't /
can't do this). Then the real object will always be the Ptr to the one
allocated by GMT, and you only have temporary copies of them alive at any
time inside Julia.
But GMT_TEXTSEGMENT is part of the GMT API (hm, have to check why it
doesn't show up in http://gmt.soest.hawaii.edu/doc/latest/GMT_API.html)
Sorry, to be concrete, and repeating myself. I need to change n_rows of
http://gmt.soest.hawaii.edu/projects/gmt/repository/entry/branches/5.2.0/src/gmt_resources.h#L517
Can I do it?
BTW, I'm a developer of GMT (that's why I could create that backdoor
function that temporarily saved me on this), although the API was all
created by P. Wessel.
On Mon, Aug 10, 2015 at 2:53 PM J Luis jmf...@gmail.com javascript:
wrote:
usually, there's is one big toplevel distinction driven by who will free
the memory: if C will be freeing the memory, then you must always have the
C code also allocating the memory (hopefully the library author provide
some sort of new / alloc / create function, although sometimes they rely on
the user calling malloc).
Yes, the GMT_Create_Data (that ccals a C function with the same name)
does that.
https://github.com/joa-quim/GMT.jl/blob/master/src/gmt_main.jl#L852
https://github.com/joa-quim/GMT.jl/blob/master/src/gmt_main.jl#L852
Is that really impossible and I'm doomed to the path exemplified
with that C GMT_blind_change_struct() function?
ideally, C libraries that expect to manage a struct's memory also
provide accessor methods to read any of the structs important fields so
that you can always declare its type as Ptr{Void} in Julia. However, I
realize this is true of many C libraries.
The pointers are ok, my real problem are those 'scalars' like the n_rows
in
http://gmt.soest.hawaii.edu/projects/gmt/repository/entry/branches/5.2.0/src/gmt_resources.h#L517
I only know the number of rows after having read the data and than the
immut is already created
On Mon, Aug 10, 2015 at 9:58 AM J Luis jmf...@gmail.com wrote:
segunda-feira, 10 de Agosto de 2015 às 14:15:15 UTC+1, Isaiah escreveu:
https://github.com/JuliaLang/julia/pull/12113
So there is hope. Thanks
On Mon, Aug 10, 2015 at 8:11 AM, J Luis jmf...@gmail.com wrote:
Hi,
Let me try to explain my need to fight with immutables. I made
another round to my GMT wrapper (which is hopefully almost ready) and
struggled again with the immutables. The point is that GMT API structs
are
deeply nested and need to be mirrored with immutables (otherwise Julia
crashes). Namely this one
https://github.com/joa-quim/GMT.jl/blob/master/src/libgmt_h.jl#L1445
so I first went with the advised solution of creating and filling the
type is julia and replace the pointees of the immutable type, which I
did
here (see the commented code):
https://github.com/joa-quim/GMT.jl/blob/master/src/gmt_main.jl#L874
but the problem now is that *TS* is a memory owned by Julia and that
will be fatal when GMT's own memory cleaning functions try to free it
and
- Julia crashes.
So if use a pure Julia solution I've no choice but to let memory
leeks and other house keeping to be done. Not good at all.
For the time being I ended up with this ugly C trick of sending to C
the job of changing some members (non-pointers) that need absolutely to
changed after the type has been created
http://gmt.soest.hawaii.edu/projects/gmt/repository/entry/branches/5.2.0/src/gmt_api.c#L7376
but besides being ugly I've no guarantee that I can convince my
partners to let that backdoor go into the GMT lib. I much lovelier trick
would be if this chunk only of that backdoor function would work.
else if (!strncmp (keyword, API_POINTER_, 12U)) {/*
Blindly change a pointer to a scalar. Irritatingly Julia ignores this */
if (!strcmp(keyword[12], UINT64))
*(uint64_t *)ptr = *(uint64_t *)what;
}
but the problem is that I am not able to get from Julia the pointers
address that would make that work. I tried with
pointer(S0.n_rows)
and
pointer_from_objref(S0.n_rows)
and others, but nothing. It seams to work in the C side (I can see it
with the debugger when I set in that piece of code) but than nothing
changes. The C struct doesn't change.
Is there a way to get the C pointer address from Julia?
So, I guess my question is:
How do we do to change immutable members that are not pointers?
Is that really impossible and I'm doomed to the path exemplified
with that C GMT_blind_change_struct()
[julia-users] Re: 0.4 used to load fast
I commented on the commit, but I disagree with making worse backtraces the default. If you know what you're doing and want fast startup in exchange for worse backtraces, you can use --precompiled=yes (and we should probably document that more clearly), but for the sake of all of those users who don't know what they're doing, we can't make bad backtraces the default behavior. On Monday, August 10, 2015 at 10:49:51 AM UTC-7, Simon Danisch wrote: Ah okay, I suspected this. I wasn't convinced though. I compared the stack traces for precompiled=yes/no for the cases that really annoyed me and it seemed precompiled=no never gave better results. Guess I haven't hit the edge cases yet. But given these findings, I would greatly prefer precompiled=yes as the default, and if I'm unhappy with the stack traces I would want to switch to precompiled=no temporarily. I'll comment on the PR. Am Montag, 10. August 2015 16:17:17 UTC+2 schrieb J Luis: On Windows here. Some months ago the local build with MinGW used to start almost instantaneously. Now it takes almost 4 seconds again.
Re: [julia-users] Computer Vision Package
Hey everyone, thanks a lot for your answers and suggestions. That seems I have too many things to learn. *@Tim* Actually using Images.jl for input, output and core data structures is a good idea because I guess it is already handling many different image formats/extensions but I might still need a layer between Images.jl and my package because most of tracking algorithm uses only grayscale images, but still not sure about that. I guess I should start working on that ASAP to see how it's working. Also for filtering stuff, I am going to implement first: http://robots.stanford.edu/cs223b04/algo_tracking.pdf that is also what OpenCV has for calcOpticalFlowPyrLK, I guess that would be good start point. However, for that I will need Scharr Operator. Do we have that operator in Images.jl? Otherwise I can implement that operator on Images.jl. because If I am going to use Images.jl for filtering and pyramids, I guess all filtering logic should be there. Correct me if I'm wrong. *@Kevin *I didn't see VideoIO.jl, but it seems pretty cool. I will check that but as far as I can see that will solve video input problem. Thanks again. Mehmet -
Re: [julia-users] Re: Can Julia function be serialized and sent by network?
Yes, I incorrectly assumed `serialize` / `deserialize` use JLD format. But anyway, even when I saved the function into example.jls or even plain byte array (using IOBuffer and `takebuf_array`), nothing changed. Am I missing something obvious? On Monday, August 10, 2015 at 11:40:03 PM UTC+3, Tim Holy wrote: On Monday, August 10, 2015 01:13:15 PM Tony Kelman wrote: Should probably use some different extension for that, .jls or something, to avoid confusion. Yes. That has been sufficiently confusing in the past, we even cover this here: https://github.com/JuliaLang/JLD.jl#saving-and-loading-variables-in-julia-data-format-jld --Tim On Monday, August 10, 2015 at 12:45:35 PM UTC-7, Stefan Karpinski wrote: JLD doesn't support serializing functions but Julia itself does. On Mon, Aug 10, 2015 at 3:43 PM, Andrei Zh faithle...@gmail.com javascript: wrote: I'm afraid it's not quite true, and I found simple way to show it. In the next code snippet I define function `f` and serialize it to a file: julia f(x) = x + 1 f (generic function with 1 method) julia f(5) 6 julia open(example.jld, w) do io serialize(io, f) end Then I close Julia REPL and in a new session try to load and use this function: julia f2 = open(example.jld) do io deserialize(io) end (anonymous function) julia f2(5) ERROR: function f not defined on process 1 in error at error.jl:21 in anonymous at serialize.jl:398 So deserialized function still refers to the old definition, which is not available in this new session. Is there any better way to serialize a function and run it on an unrelated Julia process? On Monday, August 10, 2015 at 2:33:11 PM UTC+3, Jeff Waller wrote: My question is: does Julia's serialization produce completely self-containing code that can be run on workers? In other words, is it possible to send serialized function over network to another host / Julia process and applied there without any additional information from the first process? I made some tests on a single machine, and when I defined function without `@everywhere`, worker failed with a message function myfunc not defined on process 1. With `@everywhere`, my code worked, but will it work on multiple hosts with essentially independent Julia processes? According to Jey here https://groups.google.com/forum/#!searchin/julia-users/jey/julia-users/ bolLGcSCrs0/fGGVLgNhI2YJ, Base.serialize does what we want; it's contained in serialize.jl https://github.com/JuliaLang/julia/blob/master/base/serialize.jl
Re: [julia-users] Embarrassingly parallel workload
Something like this? @everywhere function fill(A::SharedArray) for idx in Base.localindexes(A) A[idx] = rand() end end function fill_array(m, n) A = SharedArray(Float64, (m, n)) @sync begin for p in procs(q) @async remotecall_wait(p, fill, A) end end A end fill_array(9, 60) On Tuesday, August 11, 2015 at 12:23:02 AM UTC+2, Júlio Hoffimann wrote: What am I doing wrong in the following code? function foo(N; parallel=false) if parallel nprocs() CPU_CORES addprocs(CPU_CORES - nprocs()) end result = SharedArray(Float64, 9, N) @parallel for i=1:N sleep(1) result[:,i] = rand(3,3)[:] end result end If I call foo(60, parallel=true), result is all zeros. Expected behavior is a random matrix. -Júlio
Re: [julia-users] use === for comparison to nothing
On Tuesday, August 11, 2015 at 2:32:01 AM UTC+2, Yichao Yu wrote: On Mon, Aug 10, 2015 at 7:52 PM, Sisyphuss zhengw...@gmail.com javascript: wrote: I saw this in recent commits. What's the purpose of it? https://github.com/JuliaLang/julia/pull/12440 Leave a comment at the commit or at least include a link to it when you post it elsewhere (like this list) would be better. It was truly a question. I didn't know I can click on the #12440 to see the answer. Anyway, thanks for the answer.
[julia-users] Collection of sparse matrices?
Hello everyone, I would like to construct an iterable collection of sparse matrices. It appears that I can do this with Array(SparseMatrixCSC, number_of_matrices). I was wondering if this would be the most sensible way of going about this problem? Does anyone see any drawback/pitfall associated with this method? Is there a better way of achieving the same result? Thanks!
Re: [julia-users] use === for comparison to nothing
On Mon, Aug 10, 2015 at 8:51 PM, Sisyphuss zhengwend...@gmail.com wrote: On Tuesday, August 11, 2015 at 2:32:01 AM UTC+2, Yichao Yu wrote: On Mon, Aug 10, 2015 at 7:52 PM, Sisyphuss zhengw...@gmail.com wrote: I saw this in recent commits. What's the purpose of it? https://github.com/JuliaLang/julia/pull/12440 Leave a comment at the commit or at least include a link to it when you post it elsewhere (like this list) would be better. It was truly a question. What I meant is that for people on the list who aren't closely following that commit, it will be hard for them to figure out which commit your are talking about. I wouldn't know what you are reference to if I didn't happen to be the one that brought it up... (OK, they can go search but that's still more confusing than if you provide a link to the commit) I didn't know I can click on the #12440 to see the answer. If the commit comes with a PR, there's usually some explaination there. (and github is doing a fairly good job at cross referencing...) Anyway, thanks for the answer.
[julia-users] Re: Collection of sparse matrices?
Hello, That looks like a very reasonable way to do it. On Tuesday, August 11, 2015 at 3:02:01 AM UTC+2, Arjun Narayanan wrote: Hello everyone, I would like to construct an iterable collection of sparse matrices. It appears that I can do this with Array(SparseMatrixCSC, number_of_matrices). I was wondering if this would be the most sensible way of going about this problem? Does anyone see any drawback/pitfall associated with this method? Is there a better way of achieving the same result? Thanks!
Re: [julia-users] use === for comparison to nothing
On Mon, Aug 10, 2015 at 7:52 PM, Sisyphuss zhengwend...@gmail.com wrote: I saw this in recent commits. What's the purpose of it? https://github.com/JuliaLang/julia/pull/12440 Leave a comment at the commit or at least include a link to it when you post it elsewhere (like this list) would be better.
[julia-users] Re: Collection of sparse matrices?
On Monday, August 10, 2015 at 9:02:01 PM UTC-4, Arjun Narayanan wrote: I would like to construct an iterable collection of sparse matrices. It appears that I can do this with Array(SparseMatrixCSC, number_of_matrices). That allocates an array of uninitialized entries and requires you to specify them afterwards. It would be slightly less error-prone to combine the two steps. If you know your arrays A,B,C already, you can just do [A, B, C]. If you don't know them, or are creating them in a loop, you can just do `matrices = SparseMatrixCSC[]` to create an empty array and then `push!(matrices, A)` one by one. (You can use the sizehint! function to slightly speed this up, but compared to the cost of creating sparse matrices I wouldn't worry about it.)
[julia-users] Re: Collection of sparse matrices?
Or you can use a comprehension: SparseMatrixCSC[ mysparsematrix(i) for i = 1:n ], where mysparsematrix(i) is a function that returns your i-th sparse matrix.
[julia-users] Re: Collection of sparse matrices?
Thanks for your replies, @StevenJohnson I would most likely need to do this in a loop, so push!() seems like a nice choice. Thanks! On Monday, 10 August 2015 18:02:01 UTC-7, Arjun Narayanan wrote: Hello everyone, I would like to construct an iterable collection of sparse matrices. It appears that I can do this with Array(SparseMatrixCSC, number_of_matrices). I was wondering if this would be the most sensible way of going about this problem? Does anyone see any drawback/pitfall associated with this method? Is there a better way of achieving the same result? Thanks!
Re: [julia-users] Computer Vision Package
On Sunday, August 09, 2015 05:26:33 PM Kevin Squire wrote: I'll second everything that Tim says, except that, for reading video, VideoIO.jl would probably be more appropriate. Definitely! It's an amazing package. --Tim
Re: [julia-users] reinterpret SubArray
On Sunday, August 09, 2015 06:45:07 PM Sebastian Good wrote: I found myself trying to reinterpret a subarray today, and it didn't work. There are obviously plenty of cases where this makes no sense, but it would be convenient where it did. Is this on someone's roadmap? Not mine :-). If you're interested in implementing this, check out the `iscontiguous` function in subarray.jl---that should basically handle the checking to see if this makes sense. Best, --Tim
[julia-users] can the REPL be safely set to use showcompact for arrays?
If it is safe and would not slow down everything, I would like to put something in .juliarc that would show Arrays in the REPL using showcompact rather than the default display().
[julia-users] Re: ANN: JuliaImages organization
Hi Tim, this is great! I have used the Image type a lot in the past months as it allows to carry image sizes with my 1D-5D arrays. Would be great if this could be split into a single package. I am sometimes not entirely sure if this type should be called Image because effectively it can also encode any other physical dimension (i.e. time). But I am not sure if this would be practical because currently the timedim has a special meaning. (By the way a big thank you for all the work on Images.jl!!!) Cheers, Tobi Am Montag, 10. August 2015 02:17:13 UTC+2 schrieb Tim Holy: Hi all, I'm in the process of splitting up Images.jl into smaller pieces, and for that purpose I've just created the JuliaImages organization: https://github.com/JuliaImages Right now it's blank :-), but aside from I/O pieces (most of which will go to https://github.com/JuliaIO) I expect to eventually transfer most of Images there. Let me know if you want to transfer your own repositories, and I'll help you get set up. Best, --Tim
[julia-users] julia 'stable' from git
Hi All, (assuming this is the right forum), according to the instructions here: https://github.com/JuliaLang/julia I can run the 'stable' release by cloning the git repo and then checking out the 0.3 release. Now, I presume the 0.4 release will eventually become 'stable', and so on and so forth. My lack of git-foo menas that I would then remove the whole /usr/local/julia directory and start from scratch with the new stable release. This is doable but I was wondering whether there is a way of tracking the stable release that will magically upgrade the whole thing to the next stable release when it is available. Cheers F
Re: [julia-users] Non-destructive assignment operator
The package https://github.com/mauro3/Parameters.jl has some of the functionality you seek. For instance, for some type PhysicalPara, construct a new instance from a existing instance `pp` with two fields modified: pp2 = PhysicalPara(pp; cw=.11e-7, rw=100.) On Sun, 2015-08-02 at 15:22, Cedric St-Jean cedric.stj...@gmail.com wrote: In my (generally functional) code, I frequently find myself asking for An object like X but with Y different, eg. that image but with the background set to transparent or that dataframe but with all inactive customers' profit set to 0. The usual idiom is: img2 = copy(img) img2[img2.==black] = transparent ... some_function(img2) I've written a macro that lets me write instead some_function(@assign(img[img.==black], transparent)) and it expands into the code above. This is very convenient, but it hits a snag with immutable types, as the assignment fails. eg. @assign(obj.field, 5) doesn't work. Thanks to Julia introspection, I can programmatically get the fields, change the one that is @assigned to, and pass them to the constructor, but that only works with the default constructor. This will fail immutable Foo a b Foo(x) = new(x,x) end f = Foo(1) @assign(f.b, 3) (arguably this @assign in particular violates Foo's invariant, but that's not the point) Is there any way to do this differently? In particular, can I access the default constructor for a type somehow? Cédric
[julia-users] Re: julia 'stable' from git
Hi Federico, I think that what you need is as simple as doing something like git fetch --tags git pull git checkout v0.4.0 make from within the git repo once version 0.4.0 is out. It's not quite magical, but you will probably want to push the button on the upgrade yourself instead of having it take you by surprise. Cheers, Scott On Monday, 10 August 2015 12:07:51 UTC+1, Federico Calboli wrote: Hi All, (assuming this is the right forum), according to the instructions here: https://github.com/JuliaLang/julia I can run the 'stable' release by cloning the git repo and then checking out the 0.3 release. Now, I presume the 0.4 release will eventually become 'stable', and so on and so forth. My lack of git-foo menas that I would then remove the whole /usr/local/julia directory and start from scratch with the new stable release. This is doable but I was wondering whether there is a way of tracking the stable release that will magically upgrade the whole thing to the next stable release when it is available. Cheers F
[julia-users] Re: julia 'stable' from git
I think `git checkout release-0.4 make` in same directory will do the work. But I'll make a new directory and build it from scratch. On Monday, August 10, 2015 at 1:07:51 PM UTC+2, Federico Calboli wrote: Hi All, (assuming this is the right forum), according to the instructions here: https://github.com/JuliaLang/julia I can run the 'stable' release by cloning the git repo and then checking out the 0.3 release. Now, I presume the 0.4 release will eventually become 'stable', and so on and so forth. My lack of git-foo menas that I would then remove the whole /usr/local/julia directory and start from scratch with the new stable release. This is doable but I was wondering whether there is a way of tracking the stable release that will magically upgrade the whole thing to the next stable release when it is available. Cheers F
Re: [julia-users] Really need to change immutables
https://github.com/JuliaLang/julia/pull/12113
On Mon, Aug 10, 2015 at 8:11 AM, J Luis jmfl...@gmail.com wrote:
Hi,
Let me try to explain my need to fight with immutables. I made another
round to my GMT wrapper (which is hopefully almost ready) and struggled
again with the immutables. The point is that GMT API structs are deeply
nested and need to be mirrored with immutables (otherwise Julia crashes).
Namely this one
https://github.com/joa-quim/GMT.jl/blob/master/src/libgmt_h.jl#L1445
so I first went with the advised solution of creating and filling the type
is julia and replace the pointees of the immutable type, which I did here
(see the commented code):
https://github.com/joa-quim/GMT.jl/blob/master/src/gmt_main.jl#L874
but the problem now is that *TS* is a memory owned by Julia and that will
be fatal when GMT's own memory cleaning functions try to free it and -
Julia crashes.
So if use a pure Julia solution I've no choice but to let memory leeks and
other house keeping to be done. Not good at all.
For the time being I ended up with this ugly C trick of sending to C the
job of changing some members (non-pointers) that need absolutely to changed
after the type has been created
http://gmt.soest.hawaii.edu/projects/gmt/repository/entry/branches/5.2.0/src/gmt_api.c#L7376
but besides being ugly I've no guarantee that I can convince my partners
to let that backdoor go into the GMT lib. I much lovelier trick would be if
this chunk only of that backdoor function would work.
else if (!strncmp (keyword, API_POINTER_, 12U)) {/* Blindly
change a pointer to a scalar. Irritatingly Julia ignores this */
if (!strcmp(keyword[12], UINT64))
*(uint64_t *)ptr = *(uint64_t *)what;
}
but the problem is that I am not able to get from Julia the pointers
address that would make that work. I tried with
pointer(S0.n_rows)
and
pointer_from_objref(S0.n_rows)
and others, but nothing. It seams to work in the C side (I can see it with
the debugger when I set in that piece of code) but than nothing changes.
The C struct doesn't change.
Is there a way to get the C pointer address from Julia?
So, I guess my question is:
How do we do to change immutable members that are not pointers?
Is that really impossible and I'm doomed to the path exemplified with
that C GMT_blind_change_struct() function?
[julia-users] Re: julia 'stable' from git
In fact, I imagine there will be a release-0.4 branch instead of a tag, so you will probably be able to ignore the git fetch --tags and do something more like git checkout release-0.4. Anyway, my point being that quite a lot will change in v0.4 so it's best to run the upgrade yourself (and you can do it without deleting everything and starting again). There will definitely be instructions available! Scott On Monday, 10 August 2015 14:44:07 UTC+1, Scott T wrote: Hi Federico, I think that what you need is as simple as doing something like git fetch --tags git pull git checkout v0.4.0 make from within the git repo once version 0.4.0 is out. It's not quite magical, but you will probably want to push the button on the upgrade yourself instead of having it take you by surprise. Cheers, Scott On Monday, 10 August 2015 12:07:51 UTC+1, Federico Calboli wrote: Hi All, (assuming this is the right forum), according to the instructions here: https://github.com/JuliaLang/julia I can run the 'stable' release by cloning the git repo and then checking out the 0.3 release. Now, I presume the 0.4 release will eventually become 'stable', and so on and so forth. My lack of git-foo menas that I would then remove the whole /usr/local/julia directory and start from scratch with the new stable release. This is doable but I was wondering whether there is a way of tracking the stable release that will magically upgrade the whole thing to the next stable release when it is available. Cheers F
[julia-users] Re: Can Julia function be serialized and sent by network?
My question is: does Julia's serialization produce completely self-containing code that can be run on workers? In other words, is it possible to send serialized function over network to another host / Julia process and applied there without any additional information from the first process? I made some tests on a single machine, and when I defined function without `@everywhere`, worker failed with a message function myfunc not defined on process 1. With `@everywhere`, my code worked, but will it work on multiple hosts with essentially independent Julia processes? According to Jey here https://groups.google.com/forum/#!searchin/julia-users/jey/julia-users/bolLGcSCrs0/fGGVLgNhI2YJ, Base.serialize does what we want; it's contained in serialize.jl https://github.com/JuliaLang/julia/blob/master/base/serialize.jl
[julia-users] Really need to change immutables
Hi,
Let me try to explain my need to fight with immutables. I made another
round to my GMT wrapper (which is hopefully almost ready) and struggled
again with the immutables. The point is that GMT API structs are deeply
nested and need to be mirrored with immutables (otherwise Julia crashes).
Namely this one
https://github.com/joa-quim/GMT.jl/blob/master/src/libgmt_h.jl#L1445
so I first went with the advised solution of creating and filling the type
is julia and replace the pointees of the immutable type, which I did here
(see the commented code):
https://github.com/joa-quim/GMT.jl/blob/master/src/gmt_main.jl#L874
but the problem now is that *TS* is a memory owned by Julia and that will
be fatal when GMT's own memory cleaning functions try to free it and -
Julia crashes.
So if use a pure Julia solution I've no choice but to let memory leeks and
other house keeping to be done. Not good at all.
For the time being I ended up with this ugly C trick of sending to C the
job of changing some members (non-pointers) that need absolutely to changed
after the type has been created
http://gmt.soest.hawaii.edu/projects/gmt/repository/entry/branches/5.2.0/src/gmt_api.c#L7376
but besides being ugly I've no guarantee that I can convince my partners to
let that backdoor go into the GMT lib. I much lovelier trick would be if
this chunk only of that backdoor function would work.
else if (!strncmp (keyword, API_POINTER_, 12U)) {/* Blindly
change a pointer to a scalar. Irritatingly Julia ignores this */
if (!strcmp(keyword[12], UINT64))
*(uint64_t *)ptr = *(uint64_t *)what;
}
but the problem is that I am not able to get from Julia the pointers
address that would make that work. I tried with
pointer(S0.n_rows)
and
pointer_from_objref(S0.n_rows)
and others, but nothing. It seams to work in the C side (I can see it with
the debugger when I set in that piece of code) but than nothing changes.
The C struct doesn't change.
Is there a way to get the C pointer address from Julia?
So, I guess my question is:
How do we do to change immutable members that are not pointers?
Is that really impossible and I'm doomed to the path exemplified with
that C GMT_blind_change_struct() function?
Re: [julia-users] Really need to change immutables
segunda-feira, 10 de Agosto de 2015 às 14:15:15 UTC+1, Isaiah escreveu:
https://github.com/JuliaLang/julia/pull/12113
So there is hope. Thanks
On Mon, Aug 10, 2015 at 8:11 AM, J Luis jmf...@gmail.com javascript:
wrote:
Hi,
Let me try to explain my need to fight with immutables. I made another
round to my GMT wrapper (which is hopefully almost ready) and struggled
again with the immutables. The point is that GMT API structs are deeply
nested and need to be mirrored with immutables (otherwise Julia crashes).
Namely this one
https://github.com/joa-quim/GMT.jl/blob/master/src/libgmt_h.jl#L1445
so I first went with the advised solution of creating and filling the
type is julia and replace the pointees of the immutable type, which I did
here (see the commented code):
https://github.com/joa-quim/GMT.jl/blob/master/src/gmt_main.jl#L874
but the problem now is that *TS* is a memory owned by Julia and that will
be fatal when GMT's own memory cleaning functions try to free it and -
Julia crashes.
So if use a pure Julia solution I've no choice but to let memory leeks
and other house keeping to be done. Not good at all.
For the time being I ended up with this ugly C trick of sending to C the
job of changing some members (non-pointers) that need absolutely to changed
after the type has been created
http://gmt.soest.hawaii.edu/projects/gmt/repository/entry/branches/5.2.0/src/gmt_api.c#L7376
but besides being ugly I've no guarantee that I can convince my partners
to let that backdoor go into the GMT lib. I much lovelier trick would be if
this chunk only of that backdoor function would work.
else if (!strncmp (keyword, API_POINTER_, 12U)) {/* Blindly
change a pointer to a scalar. Irritatingly Julia ignores this */
if (!strcmp(keyword[12], UINT64))
*(uint64_t *)ptr = *(uint64_t *)what;
}
but the problem is that I am not able to get from Julia the pointers
address that would make that work. I tried with
pointer(S0.n_rows)
and
pointer_from_objref(S0.n_rows)
and others, but nothing. It seams to work in the C side (I can see it
with the debugger when I set in that piece of code) but than nothing
changes. The C struct doesn't change.
Is there a way to get the C pointer address from Julia?
So, I guess my question is:
How do we do to change immutable members that are not pointers?
Is that really impossible and I'm doomed to the path exemplified with
that C GMT_blind_change_struct() function?
[julia-users] 0.4 used to load fast
On Windows here. Some months ago the local build with MinGW used to start almost instantaneously. Now it takes almost 4 seconds again.
