the DataFrame contains floats and i'd ultimately like to have an array of size nrow(data) with the sum of those 13 columns in it (the column combination changes with each iteration).
Is that enough detail? I've done the entire algorithm in c++ and at this point julia is a bit slower, but i have a feeling parts of my Julia code are just not efficient and therefore could be faster! On Sunday, March 2, 2014 12:03:48 PM UTC-5, John Myles White wrote: > > Hi Jason, > > Can you give a few more details about what objects are? What is data? What > is levels? > > In general, the performance problems with DataFrames are actually > performance issues with DataArrays not letting type inference work well. We > still haven’t agreed on the right solution, but this thread lays out the > relevant issues: https://github.com/JuliaStats/DataArrays.jl/issues/71 > > — John > > On Mar 2, 2014, at 9:01 AM, Jason Solack <jays...@gmail.com <javascript:>> > wrote: > > > Hello everyone, > > > > i am doing several millions of iterations over a dataframe and i need to > perform several computations over various combinations of columns. The > first of which is a simple sum of 13 column, this appears to be a slow > point of execution. > > > > right now i'm doing something like this: > > > > for c1 = levels[1], c2 = levels[2], c3 = levels[3], c4 = levels[4], > c5 = levels[5], c6 = levels[6], c7 = levels[7], c8 = levels[8], c9 = > levels[9], c10 = levels[10], c11 = levels[11], c12 = levels[12], c13 = > levels[13] > > > > s = data[levelStartColums[1]+(c1-1)] .+ > > data[levelStartColums[2]+(c2-1)] .+ > > data[levelStartColums[3]+(c3-1)] .+ > > data[levelStartColums[4]+(c4-1)] .+ > > data[levelStartColums[5]+(c5-1)] .+ > > data[levelStartColums[6]+(c6-1)] .+ > > data[levelStartColums[7]+(c7-1)] .+ > > data[levelStartColums[8]+(c8-1)] .+ > > data[levelStartColums[9]+(c9-1)] .+ > > data[levelStartColums[10]+(c10-1)] .+ > > data[levelStartColums[11]+(c11-1)] .+ > > data[levelStartColums[12]+(c12-1)] .+ > > data[levelStartColums[13]+(c13-1)] > > end > > > > does anyone have any tips on how to write this so it will execute > faster? > > > > Thank you in advance! > > > > Jason > >
