Re: [julia-users] Type-stable global variables?

2016-10-15 Thread Andrei Zh
Thanks, this is exactly what I was looking for. On Saturday, October 15, 2016 at 4:08:48 PM UTC+3, Yichao Yu wrote: > > > > On Sat, Oct 15, 2016 at 8:59 AM, Andrei Zh > wrote: > >> What is the most straightforward way to make a variable in the global >> scope t

[julia-users] Type-stable global variables?

2016-10-15 Thread Andrei Zh
What is the most straightforward way to make a variable in the global scope that can change it's value, but not its type? So far I use this: const GLOBAL_VAR = [MyType[]] # array with single element set_global_var(x::MyType) = GLOBAL_VAR[1] = x get_goval_var() = GLOBAL_VAR[1] This works fine

Re: [julia-users] GlobalRef(Module, func) vs. :(Module.func)

2016-08-11 Thread Andrei Zh
> > > `slotnames` of LambdaInfo should have what you want. > Do note that the name is not unique. > Do you mean that if's, loops, etc. may contain variables of the same name but with different slot?

Re: [julia-users] GlobalRef(Module, func) vs. :(Module.func)

2016-08-10 Thread Andrei Zh
roken under Julia 0.5, so I suppose there might be more stable way to do it. On Monday, August 8, 2016 at 10:39:34 PM UTC+3, Andrei Zh wrote: > > Thanks, this makes sense. Just for clarification: > > >> It strongly depend on what you want to do and whether you care about what

Re: [julia-users] GlobalRef(Module, func) vs. :(Module.func)

2016-08-08 Thread Andrei Zh
;> On Monday, August 8, 2016 at 1:55:51 AM UTC+3, Yichao Yu wrote: >>> >>> >>> >>> On Mon, Aug 8, 2016 at 3:57 AM, Andrei Zh wrote: >>> >>>> While parsing Julia expressions, I noticed that sometimes calls to >>>> >>>

Re: [julia-users] GlobalRef(Module, func) vs. :(Module.func)

2016-08-07 Thread Andrei Zh
> On Mon, Aug 8, 2016 at 3:57 AM, Andrei Zh > wrote: > >> While parsing Julia expressions, I noticed that sometimes calls to >> > > This shouldn't happen. > > >> global functions resolve to `GloablRef` and sometimes to >> > > and Globa

[julia-users] GlobalRef(Module, func) vs. :(Module.func)

2016-08-07 Thread Andrei Zh
While parsing Julia expressions, I noticed that sometimes calls to global functions resolve to `GloablRef` and sometimes to getfield(Module, func). Could somebody please clarify: 1. Why do we need both? 2. Is it safe to replace one by the other (assuming only modules and functions are involved

[julia-users] Re: ReverseDiffSource on v0.5

2016-08-06 Thread Andrei Zh
Just tried it on my Ubuntu 16.04. The last thing I could see was julia process eating all 16Gb or RAM in ~1 second and hanging on my laptop so that I had to do hard reboot. So this package seems to be a good tool for testing stability of Julia 0.5 release candidates :) On Thursday, August 4, 2

[julia-users] Re: Symbolic differentiation similar to TensorFlow / Theano

2016-07-09 Thread Andrei Zh
've made. Thanks again! On Saturday, July 9, 2016 at 3:02:55 AM UTC+3, Andrei Zh wrote: > > In Python, libraries like TensorFlow or Theano provide possibility to > perform automatic differentiation over their computational graphs. E.g. in > TensorFlow (example from SO > <htt

[julia-users] Symbolic differentiation similar to TensorFlow / Theano

2016-07-08 Thread Andrei Zh
In Python, libraries like TensorFlow or Theano provide possibility to perform automatic differentiation over their computational graphs. E.g. in TensorFlow (example from SO ): data = tf

Re: [julia-users] Performance of TCPServer

2016-06-16 Thread Andrei Zh
, June 15, 2016, Andrei Zh > > wrote: > >> Thanks, Tony. Moving to Julia 0.4.5 improved speed to ~0.025 seconds per >> megabyte (~39Mb / sec). This is still far behind awesome 2.5Gb for Julia >> 0.5, but I think it doesn't make sense to spend time on identifying

[julia-users] Re: Performance of TCPServer

2016-06-15 Thread Andrei Zh
https://github.com/JuliaLang/julia/milestones On Thursday, June 16, 2016 at 1:20:46 AM UTC+3, Tony Kelman wrote: > > Use 0.4.4 or 0.4.5. https://github.com/JuliaLang/julia/issues/14467 > > On Wednesday, June 15, 2016 at 1:46:34 PM UTC-7, Andrei Zh wrote: >> >> Seems like res

[julia-users] Re: Performance of TCPServer

2016-06-15 Thread Andrei Zh
allocations: 1.012 MB) 0.000400 seconds (114 allocations: 1.012 MB) Any suggestions on why the results are so different and how to fix TCP sockets for Julia 0.4? On Wednesday, June 15, 2016 at 5:10:12 PM UTC+3, Andrei Zh wrote: > > I've just tested performance of TCP server/client in Jul

[julia-users] Performance of TCPServer

2016-06-15 Thread Andrei Zh
I've just tested performance of TCP server/client in Julia REPL and find results pretty disappointing. The server is as simple as: server = listen(2000) while true sock = accept(server) begin while true @time readbytes(sock, 1024*1024) end end end It

[julia-users] Re: parallel computing in julia

2016-05-20 Thread Andrei Zh
In addition to what Fred said, `addprocs()` adds new workers, i.e. processes, which are not necessary bound to your CPU cores. On Friday, May 20, 2016 at 6:05:05 AM UTC+3, SHORE SHEN wrote: > > Hi > > Im doing a loop and need parallel computing. > > from the doc, it is said to "julia -p n" comma

Re: [julia-users] Re: Async file IO?

2016-05-19 Thread Andrei Zh
gt; >> >> Julia IO is mostly built on libuv, but file IO uses the internal ios >> library that is part of flisp. >> >> On Thu, May 19, 2016 at 5:43 PM, Yichao Yu > > wrote: >> >>> On Thu, May 19, 2016 at 5:24 PM, Andrei Zh >> > wrote: >

[julia-users] Re: Async file IO?

2016-05-19 Thread Andrei Zh
Based on answers to my own question , I believe it's safe to assume that `read()` on file will switch to another task during IO operation. On Tuesday, May 17, 2016 at 2:03:21 AM UTC+3, g wrote: > > Hello All, > > I think t

Re: [julia-users] Re: select/poll on TCPSocket in Julia?

2016-05-18 Thread Andrei Zh
oken up. > > On Tue, May 17, 2016 at 4:36 PM, Andrei Zh > wrote: > >> Hmm, I now think that I could misinterpret exception (EOFError, if I >> remember correctly) - it could be a server who closed connection by >> timeout, not Julia's `read()`. I will check

Re: [julia-users] Re: select/poll on TCPSocket in Julia?

2016-05-17 Thread Andrei Zh
ail > with timeout > > That sounds like a bug, although I'm not aware of any read code-path that > has a timeout. Can you post an example of when it fails? This should > absolutely be the correct way to handle IO. > > > > On Tue, May 17, 2016 at 3:09 AM Andr

[julia-users] Re: select/poll on TCPSocket in Julia?

2016-05-17 Thread Andrei Zh
implementation that would remove possible delay is welcome. On Tuesday, May 17, 2016 at 8:18:16 AM UTC+3, Jameson wrote: > > Launch each socket into it's own task (@async) and let Julia's > green-threading manage the poll/select for you internally. > > > On Friday, May 13,

[julia-users] Re: select/poll on TCPSocket in Julia?

2016-05-13 Thread Andrei Zh
. So if I manage to create a callback for incoming data, I will be able to borrow some ideas from (1) and come up with a `select` function that satisfies my needs. Does anybody have an example of using TCPSocket's callbacks/conditions? On Thursday, May 12, 2016 at 12:45:44 AM UTC+3,

[julia-users] Re: select/poll on TCPSocket in Julia?

2016-05-12 Thread Andrei Zh
et to a file descriptor? On Thursday, May 12, 2016 at 12:45:44 AM UTC+3, Andrei Zh wrote: > > Given several TCPSocket-s, how can I sequentially poll them and read the > first that becomes readable? > > I believe *nix's poll/select functions provide something similar for files

[julia-users] select/poll on TCPSocket in Julia?

2016-05-11 Thread Andrei Zh
Given several TCPSocket-s, how can I sequentially poll them and read the first that becomes readable? I believe *nix's poll/select functions provide something similar for files (and Julia has `poll_fd`), but what is the best way to approach this task for sockets?

[julia-users] Re: Write immutable with variable-length array field to a stream

2016-05-10 Thread Andrei Zh
y, May 7, 2016 at 2:31:59 PM UTC-4, Andrei Zh wrote: >> >> I work on implementing a binary protocol for a service. This protocol is >> based on messages - structs that support integer numbers, variable-length >> arrays and other structs, i.e. something we could implement in

Re: [julia-users] How fast is appending an element to an array?

2016-05-10 Thread Andrei Zh
Very detailed explanation, thank you! On Tuesday, May 10, 2016 at 11:35:43 PM UTC+3, Yichao Yu wrote: > > On Tue, May 10, 2016 at 4:24 PM, Andrei Zh > wrote: > > From performance perspective, how bad it is to use `push!(array, > element)` > > compared to pre-allo

[julia-users] How fast is appending an element to an array?

2016-05-10 Thread Andrei Zh
>From performance perspective, how bad it is to use `push!(array, element)` compared to pre-allocating array and setting elements in it? More generally, how extending arrays works in Julia? (code in array.c is quite readable, but som

[julia-users] Write immutable with variable-length array field to a stream

2016-05-07 Thread Andrei Zh
I work on implementing a binary protocol for a service. This protocol is based on messages - structs that support integer numbers, variable-length arrays and other structs, i.e. something we could implement in Julia as: immutable Message1 version::Int16 length::Int32 payload::Array{M

Re: [julia-users] What is the proper way to allocate and free C struct?

2016-04-17 Thread Andrei Zh
can manipulate original pointer and also have direct access to the `data` field. On Sunday, April 17, 2016 at 3:35:34 PM UTC+3, Yichao Yu wrote: > > On Sun, Apr 17, 2016 at 6:23 AM, Andrei Zh > wrote: > > A followup question. In the above example I'd like to add method &

Re: [julia-users] What is the proper way to allocate and free C struct?

2016-04-17 Thread Andrei Zh
April 17, 2016 at 3:49:51 AM UTC+3, Andrei Zh wrote: > > This is exactly the answer I hoped to see! Thanks a lot! > > On Sunday, April 17, 2016 at 2:46:15 AM UTC+3, Yichao Yu wrote: >> >> On Sat, Apr 16, 2016 at 6:55 PM, Andrei Zh wrote: >> > I have a question rega

Re: [julia-users] What is the proper way to allocate and free C struct?

2016-04-16 Thread Andrei Zh
This is exactly the answer I hoped to see! Thanks a lot! On Sunday, April 17, 2016 at 2:46:15 AM UTC+3, Yichao Yu wrote: > > On Sat, Apr 16, 2016 at 6:55 PM, Andrei Zh > wrote: > > I have a question regarding Struct Type correspondences section of the > > documentati

[julia-users] What is the proper way to allocate and free C struct?

2016-04-16 Thread Andrei Zh
I have a question regarding Struct Type correspondences section of the documentation. Let's say, in a C library we have a struct like this: typedef struct { int rows; int cols; doubl

[julia-users] [ANN] Spark.jl - Julia interface to Apache Spark

2016-04-14 Thread Andrei Zh
Spark.jl provides Julia bindings for Apache Spark - by far the most popular computational framework in Hadoop ecosystem. Find it at: https://github.com/dfdx/Spark.jl There's still *a lot* of work to do (Spark API is *huge*), but Spark.jl already supports: - map and reduce functions, as

Re: [julia-users] Re: CUDART and CURAND problem on running the same "do" loop twice

2015-12-08 Thread Andrei Zh
7.5 >> >> If somebody else could confirm the problem on curand in cuda 7.0, we >> could add that in CURAND.jl documentation. >> >> >> >> Em quarta-feira, 25 de novembro de 2015 19:20:48 UTC-2, Andrei Zh >> escreveu: >>> >>> @Joaquim W

Re: [julia-users] How to create array wrapper with constrains on type and number of dimensions

2015-11-30 Thread Andrei Zh
Hmm, seems like this gives an error when constructing a type: julia> ArrayWrapper(rand(3, 4)) ERROR: MethodError: `convert` has no method matching convert(::Type{ ArrayWrapper{T,N,AT<:AbstractArray{T,N}}}, ::Array{Float64,2}) This may have arisen from a call to the constructor ArrayWrapper{T,N,AT

[julia-users] How to create array wrapper with constrains on type and number of dimensions

2015-11-30 Thread Andrei Zh
I'm trying to do something like this (which doesn't compile in its current form): type ArrayWrapper{T,N,AT <: AbstractArray{T,N}} <: AbstractArray{T,N} arr::AT{T,N} end That is: - wrapper around any type AT inherited from AbstractArray{T,N} - wrapper should be itself parametrized by T

[julia-users] Re: CUDART and CURAND problem on running the same "do" loop twice

2015-11-24 Thread Andrei Zh
While investigating this issue we found that CUDArt and CURAND may behave differently on different platforms. Could somebody from user group with enabled CUDA try the following code in Julia REPL and report: * operating system * whether this code succeeded or failed; if failed, then what is t

Re: [julia-users] Re: CUDART and CURAND problem on running the same "do" loop twice

2015-11-10 Thread Andrei Zh
> But I can replicate the bug if I use CURAND. I'd suggest filing an issue > with > that package. Most likely it needs to (and, you need to) initialize > resources > according to these instructions: > https://github.com/JuliaGPU/CUDArt.jl#initializing-and-freeing-ptx-modules > > CURAND.jl i

[julia-users] Re: How to escape array of symbols in a macro?

2015-10-19 Thread Andrei Zh
(generic function with 1 method) > > julia> foo(5, 3.0) > ERROR: MethodError: `foo` has no method matching foo(::Int64, ::Float64) > Closest candidates are: > foo(::Any, ::Int64) > > julia> foo(5, 3) > hello > > > julia> @deffun bar(a::AbstractString,b::

[julia-users] Re: How to escape array of symbols in a macro?

2015-10-19 Thread Andrei Zh
@Cedric: > $(map(esc, args)...) > > works everywhere I've tried it _except_ in function definition argument > lists. > Ha, I tried a couple of scenarios and in most cases it really behaves as I want, but not in this specific case :) Even though I like macro hygiene by default, I should admit

[julia-users] How to escape array of symbols in a macro?

2015-10-19 Thread Andrei Zh
I'm trying to write a macro for generation of new functions given their name and arguments, e.g. given macro call: @deffun foo(a, b) it should generate function like: function foo(a, b) # do some stuff

[julia-users] Re: Julia ESS mode in Emacs

2015-10-12 Thread Andrei Zh
Just tried the same commit on my Linux Mint and everything works, so it may be something specific to your installation. If you type `M-x julia-` and press TAB, are there any autosuggestions shown at all? On Sunday, October 11, 2015 at 5:57:12 PM UTC+3, Alex wrote: > > Hello, > > I have instal

Re: [julia-users] Inner constructor in a type with type parameters

2015-10-12 Thread Andrei Zh
Ah, that was obvious. Example of why late night coding isn't very productive :) Thanks! On Monday, October 12, 2015 at 2:15:02 AM UTC+3, Yichao Yu wrote: > > On Sun, Oct 11, 2015 at 6:54 PM, Andrei Zh > wrote: > > I didn't know about such capability, thanks. But I

[julia-users] Re: is there a way to access help from within emacs shell?

2015-10-12 Thread Andrei Zh
If you are looking for docstrings: On Julia 0.3: julia> help(qr) INFO: Loading help data... Base.qr(A, [pivot=false,][thin=true]) -> Q, R, [p] Compute the (pivoted) QR factorization of "A" such that either "A = Q*R" or "A[:,p] = Q*R". Also see "qrfact". The default is to compute a thin

Re: [julia-users] Inner constructor in a type with type parameters

2015-10-11 Thread Andrei Zh
call{T}(::Type{T}, arg) at essentials.jl:56 call{T}(::Type{T}, args...) at essentials.jl:57 On Monday, October 12, 2015 at 1:06:14 AM UTC+3, Yichao Yu wrote: > > On Sun, Oct 11, 2015 at 5:57 PM, Andrei Zh > wrote: > > Let's consider 2 types with inner c

[julia-users] Inner constructor in a type with type parameters

2015-10-11 Thread Andrei Zh
Let's consider 2 types with inner constructors: type Foo x::Array{Int,1} Foo() = Foo(zeros(Int, 10)) end type Bar{T} x::Array{T,1} Bar() = Bar(zeros(T, 10)) end The only difference between them is that `Bar` has type parameter while `Foo` doesn't. I'd expect their inner con

Re: [julia-users] What does `Base.uncompressed_ast(f.code)` return?

2015-10-08 Thread Andrei Zh
` directly, and then > look at the form of that expanded expression (hint: it is a > LambdaStaticData). Also see how this is done in `base/serialize.jl` -- or, > ideally, just reuse that functionality. > > > On Wed, Oct 7, 2015 at 7:13 PM, Andrei Zh > wrote: > >> If

[julia-users] What does `Base.uncompressed_ast(f.code)` return?

2015-10-07 Thread Andrei Zh
If `f` is anonymous function: f = x -> x + 1 what is the format of AST returned by: Base.uncompressed_ast(f.code) ? I expected it to be pure AST of a lambda function, but it's quite different from quoting lambda directly: julia> dump(Base.uncompressed_ast(f.code)) Expr head: Symbol lamb

Re: [julia-users] Re: Implementing mapreduce parallel model (not general multi-threading) ? easy and enough ?

2015-10-06 Thread Andrei Zh
e only codes that really nail it are carefully handcrafted HPC codes. Could you please elaborate on this? I think I know Spark code quite well, but can't connect it to the notion of handcrafted HPC code. On Tue, Oct 6, 2015 at 12:57 PM, Andrei Zh > wrote: > >> Yet, calling Jul

Re: [julia-users] Re: Implementing mapreduce parallel model (not general multi-threading) ? easy and enough ?

2015-10-06 Thread Andrei Zh
owever, which was > arguably the major concern of the original MapReduce design. > > On Tue, Oct 6, 2015 at 10:24 AM, Andrei Zh > wrote: > >> Julia supports multiprocessing pretty well, including map-reduce-like >> jobs. E.g. in the next example I add 3 processes to a &q

[julia-users] Re: Implementing mapreduce parallel model (not general multi-threading) ? easy and enough ?

2015-10-06 Thread Andrei Zh
Julia supports multiprocessing pretty well, including map-reduce-like jobs. E.g. in the next example I add 3 processes to a "workgroup", distribute simulation between them and then reduce results via (+) operator: julia> addprocs(3) 3-element Array{Int64,1}: 2 3 4 julia> nheads = @parallel

[julia-users] Why Julia uses GitHub to maintain package index?

2015-09-30 Thread Andrei Zh
First of all, thanks to all developers of current packaging system in Julia - it has very convenient REPL-based interface, graceful treatment of binary dependencies and it never was that easy to make and submit changes to existing packages. So please don't consider this post as critique, but ra

[julia-users] Re: What's the reason of the Success of Python?

2015-09-30 Thread Andrei Zh
Coming back to the original question about success of Python, I'd stress these points: * it's *simple*. for both - programmers and non-programmers * it's a *system language* for most Unix-like operations systems * it has very strong *scientific stack* (read as: has a lot of mature scientific

Re: [julia-users] Re: What are the "strengths" of the Julia "ecosystem" (e.g. for bio and quants/finance)? This is not a very specific question (while sub-questions are), you've been warned..

2015-09-25 Thread Andrei Zh
d below should be now fixed by > https://github.com/JuliaWeb/HttpServer.jl/pull/59. > > On Fri, Sep 25, 2015 at 5:47 AM Páll Haraldsson > wrote: > >> On Thursday, September 24, 2015 at 11:07:56 PM UTC, Andrei Zh wrote: >>> >>> >>> >>>>

[julia-users] Re: What are the "strengths" of the Julia "ecosystem" (e.g. for bio and quants/finance)? This is not a very specific question (while sub-questions are), you've been warned..

2015-09-24 Thread Andrei Zh
> I find this statement highly surprising.. wander if you meant to reverse > this.. My quant friend who had worked for years in Python had trouble > parallelizing Python code (may be resolved now..). I'm not familiar with R, > but Python has the GIL and associated problems. I also thought Erl

[julia-users] Re: What are the "strengths" of the Julia "ecosystem" (e.g. for bio and quants/finance)? This is not a very specific question (while sub-questions are), you've been warned..

2015-09-24 Thread Andrei Zh
2015 at 2:35:57 AM UTC+3, Jonathan Malmaud wrote: > > The webstack has seen considerable improvement lately. Mux is the most > mature and supported webapp framework at this point. > > On Wednesday, September 23, 2015 at 4:58:01 PM UTC-4, Andrei Zh wrote: >> >> If you are

[julia-users] Re: What are the "strengths" of the Julia "ecosystem" (e.g. for bio and quants/finance)? This is not a very specific question (while sub-questions are), you've been warned..

2015-09-23 Thread Andrei Zh
If you are looking for a best in the class libraries, you probably won't find many. This is implied by a simple fact that most such libraries had already been created in other languages by the time Julia was born. However, if you want something comparable to such best libraries, then I would st

[julia-users] Re: What's your favourite editor?

2015-09-22 Thread Andrei Zh
s-use-auto-complete t) (setq ess-tab-complete-in-script t) On Tuesday, September 22, 2015 at 4:16:33 PM UTC+3, Andrei Zh wrote: > > > >> I'd be grateful to hear from other emacs users regarding your workflows >> for Julia development, e.g. if you want to write a J

[julia-users] Re: What's your favourite editor?

2015-09-22 Thread Andrei Zh
> > I'd be grateful to hear from other emacs users regarding your workflows > for Julia development, e.g. if you want to write a Julia package what emacs > packages, setup and workflow help you to be the most productive? > > I use ESS + autocomplete + few keybindings (I'll post exact .init.el l

Re: [julia-users] Emacs, ESS and julia-mode

2015-09-17 Thread Andrei Zh
ESS mode's integration with Julia's REPL is quite simple and can't handle special sequences. It's a pity that `help()` function has been removed from 0.4, I didn't see it before your post. There was a discussion regarding better REPL (network REPL?) recently, but AFAIK nobody is working on such

[julia-users] Re: [Concurrency] Julia with/vs. Python 3.5, Go, Erlang/Elixir

2015-09-15 Thread Andrei Zh
It would be more productive, if you could describe what exactly you expect from a good concurrent/parallel programming language. Julia already has built-in coroutines (Tasks) and multiprocessing, and will get shared-memory parallelism (multithreading) in observable future. The only thing in othe

[julia-users] Re: How to use Apache Spark from Julia

2015-09-05 Thread Andrei Zh
Spark's website provides pretty good documentation to get started. Basically, you just need to download and unpack Spark archive. To use it from Julia, you can try either Spock.jl, or Sparta.jl (Spark.jl isn't a wrapper, but instead an att

Re: [julia-users] IDE for Julia

2015-09-01 Thread Andrei Zh
@Cedric Unless you are interested specifically in notebooks, I'd suggest trying ESS mode for Emacs which has support for Julia including REPL. It looks something like this (though this video also uncovers sometimes annoying

Re: [julia-users] Re: What is the fastest way to perform 100k blocking IO operations in parallel?

2015-08-25 Thread Andrei Zh
t 2:44:30 AM UTC+3, Andrei Zh wrote: > > @Jameson: setting UV_THREADPOOL_SIZE to 32 seems to reduce DNS resolution > time twice (from 26 to 12 seconds on my latest tests), so thank you. > > However, DNS seems to be not the only root of the problem: I noticed that > with large n

Re: [julia-users] Re: What is the fastest way to perform 100k blocking IO operations in parallel?

2015-08-24 Thread Andrei Zh
ng a lot of parallel dns queries, you may want to try > increasing the number that can be run simultaneously but setting the > UV_THREADPOOL_SIZE environment variable before starting julia to something > larger (default is 4, max is 128). > > On Mon, Aug 24, 2015 at 9:17 AM Andrei

[julia-users] Re: What is the fastest way to perform 100k blocking IO operations in parallel?

2015-08-24 Thread Andrei Zh
in its > use for high-performance applications so don't hesitate to file an issue if > it gives you any performance problems. > > On Sunday, August 23, 2015 at 7:40:08 PM UTC-4, Andrei Zh wrote: >> >> Hi Steven, >> >> thanks for your answer! It turns out I m

[julia-users] Re: What is the fastest way to perform 100k blocking IO operations in parallel?

2015-08-24 Thread Andrei Zh
nterested in its > use for high-performance applications so don't hesitate to file an issue if > it gives you any performance problems. > > On Sunday, August 23, 2015 at 7:40:08 PM UTC-4, Andrei Zh wrote: >> >> Hi Steven, >> >> thanks for your answer! It tur

[julia-users] Re: What is the fastest way to perform 100k blocking IO operations in parallel?

2015-08-23 Thread Andrei Zh
Hi Steven, thanks for your answer! It turns out I misunderstood @async long time ago, assuming it also makes a remote call to other processes and thus introduces true multi-tasking. So now I need to rethink my approach before going further. Just to clarify: my goal is to perform as many requ

[julia-users] Re: What is the fastest way to perform 100k blocking IO operations in parallel?

2015-08-23 Thread Andrei Zh
More generally, is there anything like futures/callbacks or async IO in Julia? I couldn't find anything, but maybe I just don't know the right keywords.

[julia-users] Re: What is the fastest way to perform 100k blocking IO operations in parallel?

2015-08-22 Thread Andrei Zh
On Sunday, August 23, 2015 at 5:52:27 AM UTC+3, Andrei Zh wrote: > > I'm writing a kind of a web scanner that should retrieve and analyze about > 100k URLs as fast as possible. Of course, it will take time anyway, but I'm > looking for how to utilize my CPUs and network as

[julia-users] What is the fastest way to perform 100k blocking IO operations in parallel?

2015-08-22 Thread Andrei Zh
I'm writing a kind of a web scanner that should retrieve and analyze about 100k URLs as fast as possible. Of course, it will take time anyway, but I'm looking for how to utilize my CPUs and network as much as possible. My initial approach was to add all available processors, pack urls into tas

Re: [julia-users] Benchmarking Julia HttpServer

2015-08-19 Thread Andrei Zh
I've got similar results some time ago [1], but stopped investigating because of lack of time. If you could find out the reason why different tools result in different outcomes, it would really push Julia web stack to production level. [1]: https://github.com/JuliaWeb/HttpServer.jl/issues/48

Re: [julia-users] Re: Can Julia function be serialized and sent by network?

2015-08-14 Thread Andrei Zh
398 julia> On Friday, August 14, 2015 at 5:49:55 PM UTC+3, Jake Bolewski wrote: > > Andrei Zh > > I'm confused. Have you actually tried? > > julia> io = IOBuffer() > IOBuffer(data=UInt8[...], readable=true, writable=true, seekable=true, > append=false, s

Re: [julia-users] Re: Can Julia function be serialized and sent by network?

2015-08-14 Thread Andrei Zh
ly wrote: > > If you define the function with @everywhere, it will be defined on all > existing > workers. Likewise, `using MyPackage` loads the package on all workers. > > --Tim > > On Thursday, August 13, 2015 03:10:54 PM Andrei Zh wrote: > > Ok, after going t

Re: [julia-users] Re: Can Julia function be serialized and sent by network?

2015-08-13 Thread Andrei Zh
epresentation? On Monday, August 10, 2015 at 11:48:55 PM UTC+3, Andrei Zh wrote: > > Yes, I incorrectly assumed `serialize` / `deserialize` use JLD format. But > anyway, even when I saved the function into "example.jls" or even plain > byte array (using IOBuffer and `take

Re: [julia-users] Computer Vision Package

2015-08-10 Thread Andrei Zh
> but I might still need a layer between Images.jl and my package because > most of tracking algorithm uses only grayscale images, but still not sure > about that. > Based on my experience (re)implementing active appearance models, I'd say many tracking algorithms work even better with color

Re: [julia-users] Re: Can Julia function be serialized and sent by network?

2015-08-10 Thread Andrei Zh
ast, we even cover this > here: > > https://github.com/JuliaLang/JLD.jl#saving-and-loading-variables-in-julia-data-format-jld > > > --Tim > > > > > On Monday, August 10, 2015 at 12:45:35 PM UTC-7, Stefan Karpinski wrote: > > > JLD doesn't support serializ

[julia-users] Re: Can Julia function be serialized and sent by network?

2015-08-10 Thread Andrei Zh
I'm afraid it's not quite true, and I found simple way to show it. In the next code snippet I define function `f` and serialize it to a file: julia> f(x) = x + 1 f (generic function with 1 method) julia> f(5) 6 julia> open("example.jld", "w") do io serialize(io, f) end Then I close Julia REPL

[julia-users] Can Julia function be serialized and sent by network?

2015-08-09 Thread Andrei Zh
I'm working on a wrapper for Apache Spark in Julia. Essentially, workflow looks like this: 1. Julia driver creates instance of `JuliaRDD` class in JVM and passes serialized Julia function to it. 2. Spark core copies JuliaRDD to each machine in a cluster and runs its `.compute()` method. 3.

[julia-users] Re: ZMQ Publish, Subscribe

2015-07-15 Thread Andrei Zh
To start with zmq_socket.connect(zmq_tcp1) is not Julia syntax, correct syntax is connect(zmq_socket, zmq_tcp1) In other words, you don't invoke method of an object with 1 argument, but instead invoke a function with 2 arguments. On Wednesday, July 15, 2015 at 1:54:48 AM UTC+3, De

Re: [julia-users] Julia Summer of Code

2015-05-25 Thread Andrei Zh
@Jey, I'm willing to help, but I need some initial guidance. Please, check out issues tab in Spock.jl. On Monday, May 25, 2015 at 7:05:01 AM UTC+3, Jey Kottalam wrote: > > Viral -- Given the clear interest in it, I'll work toward fleshing it > out more fully. However, if anyone is interested in

Re: [julia-users] Re: High-performance metric collector in Julia

2015-05-22 Thread Andrei Zh
BTW, I've measured throughput of Redis on my machine, and it resulted in only 30K/sec, which is 10 times slower than ZMQ. Possibly, some hybrid solution of several alternative backends will work. On Saturday, May 23, 2015 at 2:04:02 AM UTC+3, Andrei Zh wrote: > > > It sti

Re: [julia-users] Re: High-performance metric collector in Julia

2015-05-22 Thread Andrei Zh
> It still helps a lot, because you can have many reporting programs, each > talking to different processes on the server, and those processes are able > to get the transactions done very quickly, with multiple write daemons and > journaling daemons doing the actual I/O from the shared buffer

Re: [julia-users] Re: High-performance metric collector in Julia

2015-05-20 Thread Andrei Zh
> > >> I was able to get 2M/sec without transactions, and 909K/sec with >>> transactions (so it's durable), on my laptop, using Caché... (from >>> InterSystems... I used to consult for them) >>> They do have a free single user database engine, Globals, that you might >>> be able to use... I don

[julia-users] Re: Using Julia program as computation backend

2015-05-19 Thread Andrei Zh
I would say the easiest way would be to create Julia wrapper around one of existing clients, e.g. use PyCall.jl and Python's pika library [1]. [1]: https://pika.readthedocs.org/en/0.9.14/ On Monday, May 18, 2015 at 5:34:12 PM UTC+3, Roman Kravchik wrote: > > I am building a web app for DSP tas

[julia-users] Re: High-performance metric collector in Julia

2015-05-19 Thread Andrei Zh
As I mentioned, StatsD is cool, but comes with a lot of dependencies. Also, as far as I know, StatsD doesn't provide storage, but instead pushes data further (e.g. to graphite). Thus, to use it for simple metric collection one will also need to create StatsD backend (in addition to front-end),

[julia-users] Re: High-performance metric collector in Julia

2015-05-19 Thread Andrei Zh
over it, and handle storage. > > One option, if you care to go down that route, is to use Redis as a store. > There are a couple of julia Redis drivers around. > > Regards > - > Avik > > On Tuesday, 19 May 2015 17:23:23 UTC+1, Scott Jones wrote: >> >>

[julia-users] Re: High-performance metric collector in Julia

2015-05-19 Thread Andrei Zh
@Scott, thanks for your comment, but I think we are talking about different scenarios and different speed levels :) ??? Not on any real RDBMS like MySQL, PostgreSQL... If you want reported message to be durable, you need either to replicate it to another server, or write it to disk. In-memor

[julia-users] Re: High-performance metric collector in Julia

2015-05-19 Thread Andrei Zh
Not quite, I'm more interested in systems for collecting metrics (think of reporting server or application health, for example) rather than storage with random reads and writes. It is possible to use databases for this, but it will be pretty inefficient. For example: 1. If we tale traditional

[julia-users] High-performance metric collector in Julia

2015-05-19 Thread Andrei Zh
I'm working on a simple load testing library. Basically, it provides macro `@runtimes` that takes number of parallel users, number of iterations and expression to execute, and repeats that expression specified number of times in different threads / tasks. In code, it looks something like this:

Re: [julia-users] Why `using` new module shadows old methods?

2015-04-27 Thread Andrei Zh
27, 2015 at 5:42 PM, Andrei Zh > wrote: > >> Let's say, we have 2 modules: >> >> module A >> export foo >> foo(n::Int) = println("A") >> end >> >> >> module B >> export foo >> foo(s::String) = println("B&q

[julia-users] Why `using` new module shadows old methods?

2015-04-27 Thread Andrei Zh
Let's say, we have 2 modules: module A export foo foo(n::Int) = println("A") end module B export foo foo(s::String) = println("B") end and then in REPL: julia> using A julia> using B !julia> methods(foo) # 1 method for generic function "foo":

[julia-users] Re: Julia and Spark

2015-04-16 Thread Andrei Zh
Julia bindings for Spark would provide much more than just RDD, they will give us access to multiple big data components for streaming, machine learning, SQL capabilities and much more. On Friday, April 17, 2015 at 12:54:32 AM UTC+3, wil...@gmail.com wrote: > > However, I wonder, how hard it wo

[julia-users] Re: poly2mask in Julia?

2015-01-31 Thread Andrei Zh
Andrew, thanks for the link! I spent quite some time to rewrite and optimize code for Julia, but it was worth it: for 100x100 matrix intersection-based algorithms gives almost 10 times speedup, and the larger the matrix, the higher is the difference. Here's my new code: function fillpoly!{T}(

[julia-users] Is `fetch(RemoteRef)` blocking a thread?

2015-01-29 Thread Andrei Zh
It's clear, that when we call `fetch` on an instance of `RemoteRef`, it blocks the task. But does it mean that the whole thread is blocked (and thus we experience thread context switching) or control is simply passed to another task? For context: I'm trying to understand different ways of asyn

Re: [julia-users] poly2mask in Julia?

2015-01-28 Thread Andrei Zh
y) = Float32[inpolygon(P, m,n) > for m in M, n in N] > > > > Am 28.01.2015 um 03:50 schrieb Steven G. Johnson >: > > > > > > > On Sunday, January 25, 2015 at 11:03:30 AM UTC-5, Andrei Zh wrote: > > In Matlab, `poly2mask` transforms polygon (give

[julia-users] Re: poly2mask in Julia?

2015-01-26 Thread Andrei Zh
@Andreas: do you refer to this thread: https://groups.google.com/forum/#!topic/julia-users/Kwy2Zh-i4ks Yes, it's an option. I was looking for something specifically designed for this purpose, but seems like Scikit Image and some others keep exactly the approach you've described [1], so I assu

[julia-users] Re: poly2mask in Julia?

2015-01-25 Thread Andrei Zh
@Max: thanks for suggestion, but unfortunately OpenCV.jl requires Julia 0.4-dev, which conflicts with some other dependencies. I still consider changing the stack and using OpenCV (not only of filling polygon, but for its general rich functionality), but for now I'd definitely prefer some simpl

[julia-users] poly2mask in Julia?

2015-01-25 Thread Andrei Zh
Is there something like Matlab's `poly2mask` function in any Julia package? In Matlab, `poly2mask` transforms polygon (given by arrays of `x` and `y` coordinates) into a binary mask, where all values inside polygon get value 1 and all others get value 0. I found Octave implementation [1] that

Re: [julia-users] Simple drawing over image?

2015-01-11 Thread Andrei Zh
Ah, my fault. I should have checked it before googling more sophisticated things. Thanks! On Monday, January 12, 2015 at 2:57:50 AM UTC+3, Tim Holy wrote: > > ImageView has an annotation framework; it's documented in the README. > > --Tim > > On Monday, January 12, 2015 02:32:12 AM Andrei wrot

[julia-users] Re: Test-scoped requirements

2015-01-05 Thread Andrei Zh
Thanks, works like a charm. On Sunday, January 4, 2015 10:39:27 PM UTC+3, Michael Hatherly wrote: > > You can put a REQUIRE file in the test directory that should do what you > want. > > — Mike > ​ > > > On Sunday, 4 January 2015 21:33:37 UTC+2, Andrei Zh wrote: &g

  1   2   >