Big data = big fun! :-)
I enjoy those challenges because it (when successful) showcases man's
ability to bring the machine into his own service, instead of the other
way around. At least that's what I tell myself.
Phil Davis
On 9/12/13 6:46 AM, Geoff Canyon wrote:
On Thu, Sep 12, 2013 at 2:35 AM, FlexibleLearning.com <
[email protected]> wrote:
With multi-gigs of data, you will certainly have to chunk the data. And
access your progress bar judiciously!
Indeed. One of my co-workers was munging/aggregating using SQL queries --
not a good idea, and was still not done after an hour. Out of curiosity I
got him to send me his data before lunch, and my first pass yesterday
worked by processing all the (test) data into an array, and finished in
about 90 seconds. But that was just 100mb of data. Then I found out how big
the data set really is, and started re-writing. Right now I have something
that isn't chunked yet, but can be, and is much faster, but doesn't quite
work ;-)
Debugging in my future...
_______________________________________________
use-livecode mailing list
[email protected]
Please visit this url to subscribe, unsubscribe and manage your subscription
preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode
--
Phil Davis
_______________________________________________
use-livecode mailing list
[email protected]
Please visit this url to subscribe, unsubscribe and manage your subscription
preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode
