Re: [julia-users] Slicing DataFrames resets indexes

Le dimanche 23 août 2015 à 21:20 -0700, Robert Smith a écrit : > Unfortunately, when it comes to displaying the columns in a front > -end, there are actually two real columns. For example, this is the > output using IJulia: > > In [18]: kc.get_iopub_msg(timeout=1) > Out[18]: > {'buffers': [], >

[julia-users] Re: What is the fastest way to perform 100k blocking IO operations in parallel?

As one of the maintainers of Requests.jl, I'm especially interested in its use for high-performance applications so don't hesitate to file an issue if it gives you any performance problems. On Sunday, August 23, 2015 at 7:40:08 PM UTC-4, Andrei Zh wrote: > > Hi Steven, > > thanks for your answe

[julia-users] Re: What is the fastest way to perform 100k blocking IO operations in parallel?

Jonathan, thanks for your support. So far I noticed that DNS gives pretty large delay. E.g. resolving IP addresses for 1000 URLs took 80 seconds in serial code and 26 seconds in muli-task code: Serial execution: julia> @time for url in urls begin Base.getadd

[julia-users] Re: What is the fastest way to perform 100k blocking IO operations in parallel?

Jonathan, thanks for your support. So far I noticed that DNS gives pretty large delay. E.g. resolving IP addresses for 1000 URLs took 80 seconds in serial code and 26 seconds in muli-task code: Serial execution: julia> @time for url in urls begin Base.getadd

Re: [julia-users] ode-sundials-events option as in deSolve (R)

Thanks for your reply. I used your code with some changes. using Sims, Winston function model() # note that the time scale is in hours D = Unknown(10.0, "D") A1 = Unknown("A1") rateConstant1 = 3.0 rateConstant2 = 5.0 Equation[ der(D) = -rateConstant1*D

[julia-users] Re: ANN: PLplot.jl

Hello, Thank you for your work. Could you give me an idea of the pros and cons of PLplot versus Matplotlib? Cheers, Daniel. On Sunday, 23 August 2015 06:34:38 UTC+2, wil...@gmail.com wrote: > > Hi all, > > I've started working on a Julia PLplot wrapper >

[julia-users] Re: ANN: PLplot.jl

On Sunday, 23 August 2015 15:53:37 UTC+2, Andreas Lobinger wrote: > > > Can you provide some background why and where's the problem with DataFrame > in which plotting package? > DataFrame is an external package. That seems like reason enough. Is there a strong reason to use DataFrame instead

Re: [julia-users] Re: What is the fastest way to perform 100k blocking IO operations in parallel?

Thanks for the report, Andrei - Would you mind filing this is an issue at https://github.com/JuliaWeb/Requests.jl? On Mon, Aug 24, 2015 at 9:17 AM, Andrei Zh wrote: > Jonathan, thanks for your support. So far I noticed that DNS gives pretty > large delay. E.g. resolving IP addresses for 1000 URL

[julia-users] Re: [ANN] SQLite.jl package update

Hi Jacob. I was wondering if you (or someone else) would mind answering a few questions? First, can you elaborate more on the method of converting a ResultSet to a DataFrame? Second, is there a way to list all of the tables in a database, something akin to dblistTables() in R? Thank you.

Re: [julia-users] ode-sundials-events option as in deSolve (R)

It's hard to tell from that. Rather than clog this mailing list with debugging emails, please file an issue at the Sims repo. Provide your Julia version in that, too. On Mon, Aug 24, 2015 at 9:27 AM, Martin J wrote: > Thanks for your reply. > > I used your code with some changes. > > us

[julia-users] Re: ANN: PLplot.jl

I had overlooked this: On Sunday, August 23, 2015 at 6:34:38 AM UTC+2, wil...@gmail.com wrote: > > > I currently wrapped routings which cover basic functionality and allow to > run most of examples. I've tried to make a Gadfly backend but unfortunately > the way PLplot works with colors makes it

[julia-users] Re: ANN: PLplot.jl

On Monday, August 24, 2015 at 3:48:03 PM UTC+2, Daniel Carrera wrote: > > > On Sunday, 23 August 2015 15:53:37 UTC+2, Andreas Lobinger wrote: >> >> >> Can you provide some background why and where's the problem with >> DataFrame in which plotting package? >> > > > DataFrame is an external packag

Re: [julia-users] Re: What is the fastest way to perform 100k blocking IO operations in parallel?

Name resolution delays are generally an issue with network latency. Trying to resolve 1000 uncached names will take a while on any system: seth@schroeder:~$ time host www.julialang.org www.julialang.org is an alias for julialang.github.io. julialang.github.io is an alias for github.map.fastly.net

[julia-users] Re: Speed Comparison "Julia vs R vs C++" a question.

Ugh... I tried to make this faster, but I can't. It looks like one of the culprits is the "@where" line. On my computer your program takes 13s, of which 5s are spent on the "@where" line. As a test, I re-wrote the program in plain Julia up to the "@where" line (which I replaced by native Julia)

Re: [julia-users] Re: What is the fastest way to perform 100k blocking IO operations in parallel?

Did you try the DNS servers from Google, e.g. 8.8.8.8 ? I never saw a reply that needs more than one second. (Well, in our university network.) Am Montag, 24. August 2015 16:25:06 UTC+2 schrieb Seth: > > Name resolution delays are generally an issue with network latency. Trying > to resolve 1000

[julia-users] Re: ANN: PLplot.jl

PLplot cannot use new colors, except predefined before initialization, during plotting. This makes usage of Compose (primitives drawing backend package for Gadfly) somewhat hard because information about primitive colors appear as as you parse plot structure. Moreover, PLplot has large collecti

[julia-users] Re: ANN: PLplot.jl

Essentially, both packages have very similar set of functions - various 2D and 3D plots, handling axes, views & text. But PLplot is one binary library compiled on many platforms, in opposite to matlibplot which is a big suite of python libraries (in Julia case add a PyCall & PyPlot wrappers). O

[julia-users] Re: [ANN] SQLite.jl package update

Hi Chris, see this comment (https://github.com/quinnj/SQLite.jl/issues/60#issuecomment-69058463) for how to convert to a DataFrame. As for dblistTables(), SQLite.jl exports a tables(db::SQLiteDB) method, does this do what you want? On Monday, 24 August 2015 15:13:33 UTC+1, Christopher Fisher wr

[julia-users] Re: ANN: PLplot.jl

Personally, I would like the following functionality: 1) `plot(X)`, where `X` is a high-dimension array, to plot each column of `X` on a single canvas (Matlab syntax); 2) `plot(X, c("red", "green", "blue"))` to plot each column with different color (the color vector is broadcast) (R syntax). O

Re: [julia-users] Re: ANN: PLplot.jl

Sisyphuss: This is the approach I have taken with Qwt.jl: `plot(rand(100,5))` will plot 5 lines, `plot(rand(100,2), colors=[:red,:blue])` will plot 2 lines (red/blue), 'subplot(rand(100,4))` will create a 2x2 grid of subplots, one for each column, etc. This interface has nothing to do with Qwt..

Re: [julia-users] Slicing DataFrames resets indexes

Well, the printed index is useful but leads to some confusion because it doesn't behave as a real index. It would be much better to have a real index. I believe there will be real indexes to increase speed, but Julia developers don't want to depend on them. Is that a correct assessment right no

Re: [julia-users] Defining composite types with fields that include each other

Thank you. Tim's solution on the issue page works well for me. On Saturday, August 22, 2015 at 8:43:44 PM UTC-4, Kevin Squire wrote: > > In particular, this comment has a work-around: > > https://github.com/JuliaLang/julia/issues/269#issuecomment-68421745 > > On Saturday, August 22, 2015, Stefan K

Re: [julia-users] Slicing DataFrames resets indexes

In my opinion, when you did `df[20:50,:]`, you constructed a new DataFrame, which had no longer anything to do with the original `df`. So you cannot expect it to know the original position in `df`. And when you print it, the row number (instead of index) is generated on the fly, just let you kno

Re: [julia-users] Re: ANN: PLplot.jl

I quite agree with Tom, various data interfaces are could be easily implemented it has nothing to do with underling plotting routines. @Tom In light of RFC: unify plotting packages under a common organization , it would be a good effort to descri

Re: [julia-users] Re: ANN: PLplot.jl

I like your approach and the idea of separating interface from library. This interface allows us to plot more easily, which is very handy when we are doing data exploration. Please go on and keep me updated ! On Monday, August 24, 2015 at 5:37:23 PM UTC+2, Tom Breloff wrote: > > Sisyphuss: T

Re: [julia-users] Re: What is the fastest way to perform 100k blocking IO operations in parallel?

Uni networks are typically high bandwidth and low latency and are optimally peered. My home network, unfortunately, exhibits none of those traits :) In any case, even a 70-millisecond resolution time results in 70 seconds of delay for 1000 uncached resolutions. On Monday, August 24, 2015 at 7:4

Re: [julia-users] Slicing DataFrames resets indexes

> > In my opinion, when you did `df[20:50,:]`, you constructed a new > DataFrame, which had no longer anything to do with the original `df`. So > you cannot expect it to know the original position in `df`. And when you > print it, the row number (instead of index) is generated on the fly, just

Re: [julia-users] Slicing DataFrames resets indexes

If the index is so important to you, you should have included an index (ID number, city name, date, etc) for it, instead of relying on the "row number" output on the fly. On Mon, Aug 24, 2015 at 6:28 PM, Robert Smith wrote: > In my opinion, when you did `df[20:50,:]`, you constructed a new >> Da

Re: [julia-users] Slicing DataFrames resets indexes

If the row index is intrinsically connected to and meaningful in relationship to the other columns, you should add it as a separate column. All arrays in Julia start indexing at 1, and data frames displays its "row" indices consistent with how it would index. Indexing arrays with Real numbers

Re: [julia-users] Slicing DataFrames resets indexes

By the way, `df[20:50,:]` is *not* a subset of `df`, but a *copy* of a subset of `df`. Since `df` doesn't have any column whose name is "row", how could you expect the "row" to be an index. On Monday, August 24, 2015 at 6:28:30 PM UTC+2, Robert Smith wrote: > > In my opinion, when you did `df[

Re: [julia-users] Slicing DataFrames resets indexes

Here we have a couple of different issues. The first one is that printing a fake index is a bit confusing because we can't do anything with that index and doesn't behave as an index. In my particular use case, I want to do something with the HTML representation of a DataFrame but the fake index

Re: [julia-users] Slicing DataFrames resets indexes

> > By the way, `df[20:50,:]` is *not* a subset of `df`, but a *copy* of a > subset of `df`. > Since `df` doesn't have any column whose name is "row", how could you > expect the "row" to be an index. > Because when you get the DataFrame back, there is a column called "Row" that looks like an i

Re: [julia-users] Re: What is the fastest way to perform 100k blocking IO operations in parallel?

If you are doing a lot of parallel dns queries, you may want to try increasing the number that can be run simultaneously but setting the UV_THREADPOOL_SIZE environment variable before starting julia to something larger (default is 4, max is 128). On Mon, Aug 24, 2015 at 9:17 AM Andrei Zh wrote:

[julia-users] Re: [ANN] SQLite.jl package update

Excellent. This does what I want. Thank you Sean! On Monday, August 24, 2015 at 11:23:39 AM UTC-4, Sean Marshallsay wrote: > > Hi Chris, see this comment ( > https://github.com/quinnj/SQLite.jl/issues/60#issuecomment-69058463) for > how to convert to a DataFrame. As for dblistTables(), SQLite.jl

[julia-users] Spped, assigning arrays

Hey. I'm new to Julia. I'm playing around with 0.4 on Windows. I'm interested in writing finite element code for research purposes. Consider the following code function get_gauss_points!(gp_xi::Matrix) gp_xi = [-1/sqrt(3) 1/sqrt(3) 1/sqrt(3) -1/sqrt(3); -1/sqrt(3) -1/sqrt(3)

[julia-users] Re: Spped, assigning arrays

You'll want to study http://julialang.org/blog/2013/09/fast-numeric/ to figure out how to make your code faster. On Monday, August 24, 2015 at 10:31:56 AM UTC-7, Mohamed Moussa wrote: > > Hey. I'm new to Julia. I'm playing around with 0.4 on Windows. I'm > interested in writing finite element co

[julia-users] Re: Spped, assigning arrays

Have you checked the results from `get_gauss_points!` ? I'm not even sure that is doing what you think. Instead of setting the values in the mutable array, it is setting the local variable (and creating a new matrix) every time. On Monday, August 24, 2015 at 7:31:56 PM UTC+2, Mohamed Moussa wr

Re: [julia-users] Spped, assigning arrays

The first version very likely doesn't do what you want, since after assigning to gp_xi, the array you allocate will immediately disappear. You could make the second version terser by using loops or broadcasted array assignment. On Mon, Aug 24, 2015 at 1:29 PM, Mohamed Moussa wrote: > Hey. I'm ne

[julia-users] Re: Spped, assigning arrays

You may want to modify your first function as something like `gp_xi = get_gauss_points!()`, so that you will at least get the correct result. Since it will only take 1.129231/1e7 seconds, It won't be the bottleneck of your program. On Monday, August 24, 2015 at 7:31:56 PM UTC+2, Mohamed Moussa

Re: [julia-users] Re: Speed Comparison "Julia vs R vs C++" a question.

Le lundi 24 août 2015 à 07:40 -0700, Daniel Carrera a écrit : > Ugh... I tried to make this faster, but I can't. > > It looks like one of the culprits is the "@where" line. On my > computer your program takes 13s, of which 5s are spent on the > "@where" line. As a test, I re-wrote the program in p

[julia-users] Re: Spped, assigning arrays

I thought this would work to assign in-place (but it does not) function get_gauss_points3!(gp_xi::Matrix) gp_xi[:] = [-1/sqrt(3) 1/sqrt(3) 1/sqrt(3) -1/sqrt(3); -1/sqrt(3) -1/sqrt(3) 1/sqrt(3) 1/sqrt(3)] end Is there any reason why this syntax shouldn't assign in-place? El l

[julia-users] Re: Spped, assigning arrays

Err, oops. Good point. On Monday, August 24, 2015 at 7:47:35 PM UTC+2, Scott Jones wrote: > > Have you checked the results from `get_gauss_points!` ? I'm not even sure > that is doing what you think. > Instead of setting the values in the mutable array, it is setting the > local variable (and

[julia-users] Re: Spped, assigning arrays

Correction (and sorry for the noise), function get_gauss_points3! function get_gauss_points3!(gp_xi::Matrix) gp_xi[:] = [-1/sqrt(3) 1/sqrt(3) 1/sqrt(3) -1/sqrt(3); -1/sqrt(3) -1/sqrt(3) 1/sqrt(3) 1/sqrt(3)] end DOES assign in-place. However, it's 50 times slowe

[julia-users] Re: Spped, assigning arrays

That works for me, but it still seems to be creating the temporary array. I suppose thats just what Julia does. On Monday, August 24, 2015 at 9:23:34 PM UTC+2, Mauricio Esteban Cuak wrote: > > I thought this would work to assign in-place (but it does not) > > function get_gauss_points3!(gp_xi::M

Re: [julia-users] Re: Spped, assigning arrays

That is what Julia does. It's also what most other systems do unless they have some very clever optimizations. On Mon, Aug 24, 2015 at 3:39 PM, Mohamed Moussa wrote: > That works for me, but it still seems to be creating the temporary array. > I suppose thats just what Julia does. > > On Monday,

[julia-users] Re: Spped, assigning arrays

How about saving integration points to custom type? type MyElement gauss_points :: Matrix end function MyElement() gauss_points = [-1/sqrt(3) 1/sqrt(3) 1/sqrt(3) -1/sqrt(3); -1/sqrt(3) -1/sqrt(3) 1/sqrt(3) 1/sqrt(3)] MyElement(gauss_points) end function get_ga

[julia-users] Re: Spped, assigning arrays

Are you running the function once to let Julia compile it before doing the timing? Looking at code_lowered() for each shows the assignment happens via one larger or eight smaller GenSym temporaries. I see the slower of the two running ~1.5x not 50x the faster. On Monday, August 24, 2015 at 3:3

[julia-users] Re: Spped, assigning arrays

Interesting. I'll be sure to have a look at JuliaFEM. On Monday, August 24, 2015 at 10:00:22 PM UTC+2, Jukka Aho wrote: > > How about saving integration points to custom type? > > type MyElement > gauss_points :: Matrix > end > > function MyElement() > gauss_points = [-1/sqrt(3) 1/sqrt(3

RE: [julia-users] "What makes a programming language successful?"

I think the current Expr object is ill-suited for anything like a gofix tool, because it is essentially not capable of lossless round-tripping a source code file into an Expr and then back into a file. Things like comments, formatting etc would all be lost. I’ve written up a very rough draft

[julia-users] Re: Performance between R, Julia and Cpp

As Kristoffer mentioned, I meant it was "unfair" to use a "slow" DataFrame, if straight Arrays are so much faster and available in both R and Julia. I'm not saying you shouldn't be able to use functionality, I'm saying you shouldn't be "forced" to use "slow libraries" :) (although for what I do,

Re: [julia-users] Re: What is the fastest way to perform 100k blocking IO operations in parallel?

@Jameson: setting UV_THREADPOOL_SIZE to 32 seems to reduce DNS resolution time twice (from 26 to 12 seconds on my latest tests), so thank you. However, DNS seems to be not the only root of the problem: I noticed that with large number of URLs first ones get result very quickly (300-800ms), but

[julia-users] build_executable.jl fails in OSX yosemite

Hi all I am trying to build an executable using the latest build_executable.jl in osx yosemite. This is my code: function main() println("Hi") println(1 + 1) end The log is given below. Let me know if I missing something. Any help is appreciated! Karthikeyans-MacBook-Pro:julia

[julia-users] Re: build_executable.jl fails in OSX yosemite

Not sure if this is related to: https://groups.google.com/forum/#!searchin/julia-users/build_executable.jl/julia-users/5Q4f4Kym3iM/ww25_zLQMmEJ (of which my test is based) https://groups.google.com/forum/#!searchin/julia-users/build_executable.jl/julia-users/mkePcWs5HgY/2vg_opFLMaEJ On Monday

[julia-users] Re: ANN: JuliaDiff -- differentiation tools in Julia

Probably the wrong place to post this, but I couldn't find a julia-diff list :) I'm having stability issues computing the derivative of functions I only know on (non-uniform) grids. For example, I have a grid of x (I can't choose this) and the associated values of y = f(x). I've tried a few di

Re: [julia-users] Slicing DataFrames resets indexes

There was some discussion about indexes on Github, but it didn't really get anywhere. I'm also not comfortable with that decision to not have indexes. It makes the dataframes asymmetric, whereas Pandas' and R's are "matrices with named ax

Re: [julia-users] Slicing DataFrames resets indexes

> > There was some discussion > > > about indexes on Github, but it didn't really get anywhere. I'm also not > comfortable with that decision t

[julia-users] Inheriting from Real works, from AbstractFloat does not?

julia> immutable SubtypeReal <: Real val::Float64 end julia> a=SubtypeReal(5.0) SubtypeReal(5.0) julia> immutable SubtypeAbsFloat <: AbstractFloat val::Float64 end julia> a=SubtypeAbsFloat(5.0) Error showing value of type SubtypeAbsFloat: ERROR: - not defined

[julia-users] Re: Inheriting from Real works, from AbstractFloat does not?

using v0.4 from nightly a few days ago On Monday, August 24, 2015 at 10:56:59 PM UTC-4, Jeffrey Sarnoff wrote: > > julia> immutable SubtypeReal <: Real >val::Float64 >end > > julia> a=SubtypeReal(5.0) > SubtypeReal(5.0) > > julia> immutable SubtypeAbsFloat <: AbstractFloat >

Re: [julia-users] Inheriting from Real works, from AbstractFloat does not?

When you subtype AbstractFloat, it's going to try to use the `grisu.jl` code to do the showing. The grisu code has all sorts of requirements to work, most of it semi-hard-coded for Float16, Float32, and Float64. Your best bet would probably be to define Base.show(io::IO, x::SubtypeAbsFloat) = show

Re: [julia-users] Inheriting from Real works, from AbstractFloat does not?

thanks On Monday, August 24, 2015 at 11:00:11 PM UTC-4, Jacob Quinn wrote: > > When you subtype AbstractFloat, it's going to try to use the `grisu.jl` > code to do the showing. The grisu code has all sorts of requirements to > work, most of it semi-hard-coded for Float16, Float32, and Float64. Y

Re: [julia-users] Slicing DataFrames resets indexes

I think that to the extent that they don't want a "real" index (and again, I also question that decision), printing the row number makes sense, since that's how you'll access the rows. If I have an array and I select half of its rows, the new array is still indexed 1:n, so they're following the