[julia-users] Where is documentation on { } notation for creating an Array(Any,1) ??
I am failing at finding where the { } notation is documented in the
manual. E.g.
{ 1, 2, 3}
creates an array of type Array{Any,1} of length 3.
Somehow both the TOC and the index are failing me...
I'm looking at the julia.pdf release 0.4.0-dev.
I did discover in my search that type Any seems to be missing from the
index.
Also, the notations in section 8.4 should also be in the index (e.g.
bracket notation for creating arrays is not in the index...)
Where is { } described ???
Thanks!
Cheers,
Ron
[julia-users] how does method dispatch treat varargs functions?
Perhaps this is a documentation bug, or perhaps I missed the explanation somehow. Suppose I define *(x::ASCIIString, y::Any) = x * string(y) so that * may be used for concatenation of any value to a string: the value of x is * x == the value of x is 2 (assuming x == 2) If I then list methods(*,(ASCIIString,Any)) I get * *(s::String...) at string.jl:76* * *(x::ASCIIString,y) at none:1 * So, my question is: how does Julia decide which function to call in the application of * in my new function body? Both definitions would seem to apply. Is the situation really ambiguous, and I'm just benefitting from some arbitrary convention, or is there really a rule that would make it clear that the varargs definition (i.e. the standard one) should take precedence? Thanks! Cheers, Ron
[julia-users] Re: string literals split to multilines?
Why not use concatenation: d = aaa * bbb * ccc ?? Cheers, Ron On Tuesday, January 6, 2015 2:15:13 AM UTC-8, Andreas Lobinger wrote: Hello colleagues, is there a counterpart for the string literal split to multiple lines like in python? d = '09ab\ eff1\ a2a4' Wishing a happy day, Andreas
[julia-users] Why does string('a') work but convert(ASCIIString,'a') does not?
Using Julia 0.3.4. The following seems somehow inconsistent. Is there
something about the philosophy of `convert` I am missing??
*julia convert(ASCIIString,'a')*
*ERROR: `convert` has no method matching convert(::Type{ASCIIString},
::Char)*
* in convert at base.jl:13*
*julia string('a')*
*a*
*julia typeof(ans)*
*ASCIIString (constructor with 2 methods)*
Cheers,
Ron
[julia-users] Re: Why does the following give an error: p = 1; 2p+1 ??
I now see the description of hex float notation in section 4.2 (Floating Point Numbers) of the manual (0.4.0-dev), which includes using p as binary exponent notation. I think that the error message could be improved, at a minimum. But there are problems with the current syntax that will cause others to get caught too, I fear, such as: * julia 3e+1* * 30.0* *julia 3e + 1* * 9.154845485377136* That is, the spaces around the + are significant here, as to whether e is treated as the math constant or the floating point exponent notation... I fear that numeric literal coefficients may introduce more hazards than its economy of notation justifies... ?? Cheers, Ron On Sunday, January 4, 2015 9:44:38 PM UTC-10, Ronald L. Rivest wrote: I'm using Julia 0.3.4 command line. Entering p = 1; 2p+1 gives an error: *julia p = 1; 2p+1* *ERROR: syntax: malformed expression* whereas using a different variable name doesn't give an error *julia x = 1; 2x+1* *3* There must be some aspect of Julia syntax I have missed?? Thanks for any light you can shed on this... Cheers, Ron Rivest
[julia-users] Why does the following give an error: p = 1; 2p+1 ??
I'm using Julia 0.3.4 command line. Entering p = 1; 2p+1 gives an error: *julia p = 1; 2p+1* *ERROR: syntax: malformed expression* whereas using a different variable name doesn't give an error *julia x = 1; 2x+1* *3* There must be some aspect of Julia syntax I have missed?? Thanks for any light you can shed on this... Cheers, Ron Rivest
[julia-users] Why doesn't Dict() accept an output from zip?
Since the Dict constructor accepts an array of pairs, I expected
it to also accept the output from zip. So
Dict(zip(abc,123))
should produce
[ 'a'='1', 'b'='2', 'c'='3' ]
but instead it breaks:
ERROR: `Dict{K,V}` has no method matching
Dict{K,V}(::Zip2{ASCIIString,ASCIIString})
The fix I could figure out seems to be to use collect
Dict(collect(zip(abc,123)))
but this feels rather inelegant...
Have I missed something, or perhaps this is a feature that should be added?
Cheers,
Ron
[julia-users] do I misunderstand zip, or is it not referentially transparent?
I find the following behavior of zip extremely confusing;
either I am misunderstanding something, or perhaps there
is a bug in the implementation of zip(?) [I would expect
the first and last expressions to give the same result!] :
julia [t for t in zip([2,3,4],[6,7,8])]
3-element Array{(Int64,Int64),1}:
(2,6)
(3,7)
(4,8)
julia x = [2,3,4]
3-element Array{Int64,1}:
2
3
4
julia y = [6,7,8]
3-element Array{Int64,1}:
6
7
8
julia [t for t in zip(x,y)]
1-element Array{(Any,Any),1}:
(4,7)
Can anyone explain this behavior??
Thanks!
Cheers,
Ron Rivest
[julia-users] Re: do I misunderstand zip, or is it not referentially transparent?
Versioninfo() gives:
*julia versioninfo()*
Julia Version 0.3.2
Commit 21d5433* (2014-10-21 20:18 UTC)
Platform Info:
System: Darwin (x86_64-apple-darwin13.3.0)
CPU: Intel(R) Core(TM) i5-4308U CPU @ 2.80GHz
WORD_SIZE: 64
BLAS: libopenblas (DYNAMIC_ARCH NO_AFFINITY Haswell)
LAPACK: libopenblas
LIBM: libopenlibm
LLVM: libLLVM-3.3
I see that I was a version behind; upgrading to 0.3.3 fixed the problem for
me, too!
Thanks for the speedy replies!
Cheers,
Ron
On Sunday, December 21, 2014 7:36:46 PM UTC-8, Ronald L. Rivest wrote:
I find the following behavior of zip extremely confusing;
either I am misunderstanding something, or perhaps there
is a bug in the implementation of zip(?) [I would expect
the first and last expressions to give the same result!] :
julia [t for t in zip([2,3,4],[6,7,8])]
3-element Array{(Int64,Int64),1}:
(2,6)
(3,7)
(4,8)
julia x = [2,3,4]
3-element Array{Int64,1}:
2
3
4
julia y = [6,7,8]
3-element Array{Int64,1}:
6
7
8
julia [t for t in zip(x,y)]
1-element Array{(Any,Any),1}:
(4,7)
Can anyone explain this behavior??
Thanks!
Cheers,
Ron Rivest
[julia-users] How should one think about cost of multiple dispatch?
Multiple dispatch is a unique and interesting feature of Julia... How is it implemented? What is the overhead incurred by multiple dispatch? (Esp. if there are a large number of methods, as for + ?). Cheers, Ron
[julia-users] Why doesn't @sprintf evaluate its format string?
I'm new to Julia, and got burned (aka wasted a fair amount of time) trying to sort out why @sprintf didn't work as I expected. julia @sprintf(%2d,29) 29 julia fmt = %2d %2d julia @sprintf(fmt,29) ERROR: @sprintf: first argument must be a format string julia @sprintf(%*2d,29) ERROR: @sprintf: first argument must be a format string I would expect that @sprintf would allow an arbitrary string expression as its format string. It obviously doesn't... There are many good reasons why one might want a format string expression instead of just a constant format string. For example, the same format may be needed in several places in the code, or you may want to compute the format string based on certain item widths or other alignment needs. At a minimum, this should (please!) be noted in the documentation. Better would be to have the extended functionality... (Or maybe it exists already -- have I missed something?) Thanks! Cheers, Ron Rivest
[julia-users] Re: Why doesn't @sprintf evaluate its format string?
Another reason for (perhaps) wanting an expression as a format string: Julia doesn't have a line continuation mechanism. How does one split a long format string into parts without having a string expression? The following doesn't work: @sprintf(This is the first part of a rather long format string, which I * would like to extend onto %d (or maybe more) lines., 5) Thanks. Cheers, Ron On Tuesday, December 2, 2014 8:40:56 PM UTC-5, Ronald L. Rivest wrote: I'm new to Julia, and got burned (aka wasted a fair amount of time) trying to sort out why @sprintf didn't work as I expected. julia @sprintf(%2d,29) 29 julia fmt = %2d %2d julia @sprintf(fmt,29) ERROR: @sprintf: first argument must be a format string julia @sprintf(%*2d,29) ERROR: @sprintf: first argument must be a format string I would expect that @sprintf would allow an arbitrary string expression as its format string. It obviously doesn't... There are many good reasons why one might want a format string expression instead of just a constant format string. For example, the same format may be needed in several places in the code, or you may want to compute the format string based on certain item widths or other alignment needs. At a minimum, this should (please!) be noted in the documentation. Better would be to have the extended functionality... (Or maybe it exists already -- have I missed something?) Thanks! Cheers, Ron Rivest
[julia-users] How to best convert array of Uint8's to long hex string
Suppose you have a length-n array x of Uint8's = [1,5,32,7,...] , and you want to convert this to a long string of hex digits (two per x[i]). The code y = reduce(string, [hex(xi,2) for xi in x]) == 01052007... will do the trick. (Or, you could use map_reduce or one of the fold operations.) But I would like this operation to run in linear time. Would any of the reduce, map_reduce, or fold operations run in linear time? With a strict abstract implementation, each intermediate result string would need to be created, and each such intermediate result string is immutable, so the running time would be Theta(n^2). Or is the compiler smart about this case? (A small experiment suggests not.) Perhaps the right approach is: x2 = [ hex(xi,2) for xi in x] y = string(x2) ?? Cheers, Ron
[julia-users] Security problem with unitialized memory
I am just learning Julia... I was quite shocked today to learn that Julia does *not* initialize allocated storage (e.g. to 0 or some default value). E.g. the code A = Array(Int64,5) println(A[1]) has unpredictable behavior, may disclose information from other modules, etc. This is really quite unacceptable in a modern programming language; it is as bad as not checking array reads for out-of-bounds indices. Google for uninitialized security to find numerous instances of security violations and unreliability problems caused by the use of uninitialized variables, and numerous security advisories warning of problems caused by the (perhaps inadvertent) use of uninitialized variables. You can't design a programming language today under the naive assumption that code in that language won't be used in highly critical applications or won't be under adversarial attack. You can't reasonably ask all programmers to properly initialize their allocated storage manually any more than you can ask them to test all indices before accessing an array manually; these are things that a high-level language should do for you. The default non-initialization of allocated storage is a mis-feature that should absolutely be fixed. There is no efficiency argument here in favor of uninitialized storage that can outweigh the security and reliability disadvantages... Cheers, Ron Rivest
Re: [julia-users] Security problem with unitialized memory
of bounds checking, however, this isn't actually a big concern since you will still either get a bounds error or a valid array value – not a meaningful one, of course, but still just a value. Writing programs that are secure against malicious attacks is a hard, unsolved problem. So is doing efficient, productive high-level numerical programming. Trying to solve both problems at the same time seems like a recipe for failing at both. On Nov 24, 2014, at 11:43 AM, David Smith david...@gmail.com javascript: wrote: Some ideas: Is there a way to return an error for accesses before at least one assignment in bits types? I.e. when the object is created uninitialized it is marked dirty and only after assignment of some user values can it be cleanly accessed? Can Julia provide a thin memory management layer that grabs memory from the OS first, zeroes it, and then gives it to the user upon initial allocation? After gc+reallocation it doesn't need to be zeroed again, unless the next allocation is larger than anything previous, at which time Julia grabs more memory, sanitizes it, and hands it off. On Monday, November 24, 2014 2:48:05 AM UTC-6, Mauro wrote: Pointer types will initialise to undef and any operation on them fails: julia a = Array(ASCIIString, 5); julia a[1] ERROR: access to undefined reference in getindex at array.jl:246 But you're right, for bits-types this is not an error an will just return whatever was there before. I think the reason this will stay that way is that Julia is a numerics oriented language. Thus you many wanna create a 1GB array of Float64 and then fill it with something as opposed to first fill it with zeros and then fill it with something. See: julia @time b = Array(Float64, 10^9); elapsed time: 0.029523638 seconds (800144 bytes allocated) julia @time c = zeros(Float64, 10^9); elapsed time: 0.835062841 seconds (800168 bytes allocated) You can argue that the time gain isn't worth the risk but I suspect that others may feel different. On Mon, 2014-11-24 at 09:28, Ronald L. Rivest rives...@gmail.com wrote: I am just learning Julia... I was quite shocked today to learn that Julia does *not* initialize allocated storage (e.g. to 0 or some default value). E.g. the code A = Array(Int64,5) println(A[1]) has unpredictable behavior, may disclose information from other modules, etc. This is really quite unacceptable in a modern programming language; it is as bad as not checking array reads for out-of-bounds indices. Google for uninitialized security to find numerous instances of security violations and unreliability problems caused by the use of uninitialized variables, and numerous security advisories warning of problems caused by the (perhaps inadvertent) use of uninitialized variables. You can't design a programming language today under the naive assumption that code in that language won't be used in highly critical applications or won't be under adversarial attack. You can't reasonably ask all programmers to properly initialize their allocated storage manually any more than you can ask them to test all indices before accessing an array manually; these are things that a high-level language should do for you. The default non-initialization of allocated storage is a mis-feature that should absolutely be fixed. There is no efficiency argument here in favor of uninitialized storage that can outweigh the security and reliability disadvantages... Cheers, Ron Rivest
[julia-users] UInt should be Uint in documentation
There is an apparent bug (or family of bugs) in the documentation julia.pdf. The document says unsigned integer is of type UInt, while the julia system only accepts Uint (lowercase i instead of uppercase I). This holds for UInt64/Uint64, etc... The documentation should be fixed... (or the implementation made to be in accordance with the documentation...) (Sorry if this is a repeat, but I searched and didn't find an early note of this...) Cheers, Ron
Re: [julia-users] Security problem with unitialized memory
The problem also exists for new() (e.g. when initializing a record/object). zeros() can apparently be used here instead. Cheers, Ron On Tuesday, November 25, 2014 12:29:07 AM UTC-5, Viral Shah wrote: Much has been already said on this topic. The Array(...) interface was kind of meant to be low-level for the user of scientific computing, only to be used when they know what they are doing. You get the raw uninitialized memory as fast as possible. The user-facing interface was always an array constructor - zeros(), ones(), rand(), etc. Some of this is because of our past experience coming from a matlab/R-like world. As Julia has become more popular, we have realized that those not coming from matlab/R end up using all the possible constructors. While this has raised a variety of issues, I'd like to say that this will not get sorted out satisfactorily before the 0.4 release. For a class that may be taught soon, the thing to do would be to use the zeros/ones/rand constructors to construct arrays, instead of Array(), which currently is more for a package developer. I understand that Array() is a much better name as Stefan points out, but zeros() is not too terrible - it at least clearly tells the user that they get zeroed out arrays. While we have other features that can lead to unsafe code (ccall, @inbounds), none of these are things one is likely to run into while learning the language. -viral On Tuesday, November 25, 2014 1:00:10 AM UTC+5:30, Ronald L. Rivest wrote: Regarding initialization: -- I'm toying with the idea of recommending Julia for an introductory programming class (rather than Python). -- For this purpose, the language should not have hazards that catch the unwary. -- Not initializing storage is definitely a hazard. With uninitialized storage, a program may run fine one day, and fail mysteriously the next, depending on the contents of memory. This is about predictability, reliability, dependability, and correctness. -- I would favor a solution like A = Array(Int64,n) -- fills with zeros A = Array(Int64,n,fill=1) -- to fill with ones A = Array(Int64,n,fill=None)-- for an uninitialized array so that the *default* is an initialized array, but the speed geeks can get what they want. Cheers, Ron On Monday, November 24, 2014 1:57:14 PM UTC-5, Stefan Karpinski wrote: If we can make allocating zeroed arrays faster that's great, but unless we can close the performance gap all the way and eliminate the need to allocated uninitialized arrays altogether, this proposal is just a rename – Unchecked.Array plays the exact same role as the current Array constructor. It's unclear that this would even address the original concern since it still *allows* uninitialized allocation of arrays. This rename would just force people who have used Array correctly in code that cares about being as efficient as possible even for very large arrays to change their code and use Unchecked.Array instead. On Nov 24, 2014, at 1:36 PM, Jameson Nash vtj...@gmail.com wrote: I think that Rivest’s question may be a good reason to rethink the initialization of structs and offer the explicit guarantee that all unassigned elements will be initialized to 0 (and not just the jl_value_t pointers). I would argue that the current behavior resulted more from a desire to avoid clearing the array twice (if the user is about to call fill, zeros, ones, +, etc.) than an intentional, casual exposure of uninitialized memory. A random array of integers is also a security concern if an attacker can extract some other information (with some probability) about the state of the program. Julia is not hardened by design, so you can’t safely run an unknown code fragment, but you still might have an unintended memory exposure in a client-facing app. While zero’ing memory doesn’t prevent the user from simply reusing a memory buffer in a security-unaware fashion (rather than consistently allocating a new one for each use), it’s not clear to me that the performance penalty would be all that noticeable for map Array(X) to zero(X), and only providing an internal constructor for grabbing uninitialized memory (perhaps Base.Unchecked.Array(X) from #8227) On Mon Nov 24 2014 at 12:57:22 PM Stefan Karpinski stefan.karpin...@gmail.com http://mailto:stefan.karpin...@gmail.com wrote: There are two rather different issues to consider: 1. Preventing problems due to inadvertent programmer errors. 2. Preventing malicious security attacks. When we initially made this choice, it wasn't clear if 1 would be a big issue but we decided to see how it played out. It hasn't been a problem in practice: once people grok that the Array(T, dims...) constructor gives uninitialized memory and that the standard
Re: [julia-users] Security problem with unitialized memory
Sorry; zeros() does not work here instead of new(). My mistake. Is there a safe alternative to new() that guarantees that all fields will have a definite fixed value? Cheers, Ron On Tuesday, November 25, 2014 1:05:40 AM UTC-5, Ronald L. Rivest wrote: The problem also exists for new() (e.g. when initializing a record/object). zeros() can apparently be used here instead. Cheers, Ron On Tuesday, November 25, 2014 12:29:07 AM UTC-5, Viral Shah wrote: Much has been already said on this topic. The Array(...) interface was kind of meant to be low-level for the user of scientific computing, only to be used when they know what they are doing. You get the raw uninitialized memory as fast as possible. The user-facing interface was always an array constructor - zeros(), ones(), rand(), etc. Some of this is because of our past experience coming from a matlab/R-like world. As Julia has become more popular, we have realized that those not coming from matlab/R end up using all the possible constructors. While this has raised a variety of issues, I'd like to say that this will not get sorted out satisfactorily before the 0.4 release. For a class that may be taught soon, the thing to do would be to use the zeros/ones/rand constructors to construct arrays, instead of Array(), which currently is more for a package developer. I understand that Array() is a much better name as Stefan points out, but zeros() is not too terrible - it at least clearly tells the user that they get zeroed out arrays. While we have other features that can lead to unsafe code (ccall, @inbounds), none of these are things one is likely to run into while learning the language. -viral On Tuesday, November 25, 2014 1:00:10 AM UTC+5:30, Ronald L. Rivest wrote: Regarding initialization: -- I'm toying with the idea of recommending Julia for an introductory programming class (rather than Python). -- For this purpose, the language should not have hazards that catch the unwary. -- Not initializing storage is definitely a hazard. With uninitialized storage, a program may run fine one day, and fail mysteriously the next, depending on the contents of memory. This is about predictability, reliability, dependability, and correctness. -- I would favor a solution like A = Array(Int64,n) -- fills with zeros A = Array(Int64,n,fill=1) -- to fill with ones A = Array(Int64,n,fill=None)-- for an uninitialized array so that the *default* is an initialized array, but the speed geeks can get what they want. Cheers, Ron On Monday, November 24, 2014 1:57:14 PM UTC-5, Stefan Karpinski wrote: If we can make allocating zeroed arrays faster that's great, but unless we can close the performance gap all the way and eliminate the need to allocated uninitialized arrays altogether, this proposal is just a rename – Unchecked.Array plays the exact same role as the current Array constructor. It's unclear that this would even address the original concern since it still *allows* uninitialized allocation of arrays. This rename would just force people who have used Array correctly in code that cares about being as efficient as possible even for very large arrays to change their code and use Unchecked.Array instead. On Nov 24, 2014, at 1:36 PM, Jameson Nash vtj...@gmail.com wrote: I think that Rivest’s question may be a good reason to rethink the initialization of structs and offer the explicit guarantee that all unassigned elements will be initialized to 0 (and not just the jl_value_t pointers). I would argue that the current behavior resulted more from a desire to avoid clearing the array twice (if the user is about to call fill, zeros, ones, +, etc.) than an intentional, casual exposure of uninitialized memory. A random array of integers is also a security concern if an attacker can extract some other information (with some probability) about the state of the program. Julia is not hardened by design, so you can’t safely run an unknown code fragment, but you still might have an unintended memory exposure in a client-facing app. While zero’ing memory doesn’t prevent the user from simply reusing a memory buffer in a security-unaware fashion (rather than consistently allocating a new one for each use), it’s not clear to me that the performance penalty would be all that noticeable for map Array(X) to zero(X), and only providing an internal constructor for grabbing uninitialized memory (perhaps Base.Unchecked.Array(X) from #8227) On Mon Nov 24 2014 at 12:57:22 PM Stefan Karpinski stefan.karpin...@gmail.com http://mailto:stefan.karpin...@gmail.com wrote: There are two rather different issues to consider: 1. Preventing problems due to inadvertent programmer errors. 2. Preventing malicious security attacks. When
Re: [julia-users] UInt should be Uint in documentation
You are correct; my apologies for this post. I was reading documentation for development version 0.4.0 but running 0.3.2. So, I guess UInt will be the new Uint as of 0.4.0... Cheers, Ron On Tuesday, November 25, 2014 1:09:29 AM UTC-5, Pontus Stenetorp wrote: On 25 November 2014 at 15:01, Ronald L. Rivest rives...@gmail.com javascript: wrote: There is an apparent bug (or family of bugs) in the documentation julia.pdf. The document says unsigned integer is of type UInt, while the julia system only accepts Uint (lowercase i instead of uppercase I). This holds for UInt64/Uint64, etc... The documentation should be fixed... (or the implementation made to be in accordance with the documentation...) Which version of Julia are you running? My guess is that you are referring to the development version documentation [1], when what you really want is the 0.3.x release documentation [2] julia0.4 -e 'UInt' julia0.3 -e 'UInt' ERROR: UInt not defined in process_options at ./client.jl:213 in _start at ./client.jl:354 in _start_3B_1729 at ~/git/julia0.3/usr/bin/../lib/julia/sys.so julia0.4 -e 'Uint' julia0.3 -e 'Uint' As seen above, there has been some breaking changes introduced in the development branch. Pontus [1]: https://media.readthedocs.org/pdf/julia/latest/julia.pdf [2]: https://media.readthedocs.org/pdf/julia/release-0.3/julia.pdf
[julia-users] Efficiency of julia compiler on a simple program (from a julia newbie)
I'm a julia newbie trying to get a sense of how efficient the Julia compiler is, for a very simple little program. I attach copies of foo.py and foo.jl, which compute the same thing. I then ran them (for the python version with both pypy and with python). The results: pypy: 2 seconds julia:30 seconds python:43 seconds I was surprised to see julia so badly beaten by pypy, and to see that the difference between julia and ordinary python was less than a factor of two... Is there an optimization trick or optimization switch that I missed for Julia?? Thanks! Cheers, Ron Rivest 11:36:12 temp $ date; pypy foo.py; date Fri Nov 21 23:36:22 EST 2014 0.726947459371 Done. Fri Nov 21 23:36:24 EST 2014 11:36:24 temp $ date; julia foo.jl; date Fri Nov 21 23:36:37 EST 2014 0.726947459370689 Done.Fri Nov 21 23:37:07 EST 2014 11:37:07 temp $ date; python foo.py; date Fri Nov 21 23:37:32 EST 2014 0.726947459371 Done. Fri Nov 21 23:38:15 EST 2014 n = 1 x = 0.234 for i in range(n): x = max(x*x,1.0-x*x) print(x) print(Done.) foo.jl Description: Binary data
