Hi, > I’m in the ‘big data’ practice at Accenture and am also a big fan of > Sagemath, using it almost daily. I’ve recently started wondering what > interest you and the community would have in integrating Sagemath a > bit with Hadoop or another map-reduce framework so that operations > written in Sage could be shipped off to a running cluster for some > distributed computation, along with being able to run Sage on much > larger datasets than would fit on a single host. > > There are some existing solutions which are clients to Hadoop’s map- > reduce framework, such as Hive, which provides a pretty standard SQL- > like front-end to a live cluster, that made me wonder if there’s not a > way to extend the usefulness of Sagemath in a similar way. > > I'd love to hear what interest there was in this idea.
I'm interested. But in the practical application I've in mind, the data isn't there it is generated on the fly by sage itself. The problem it to gather information (eg: the size) on huge sets from combinatorics. Those sets are generated by a choice tree. The exploration of the various branches can obviously be made in parallel. If you want more information on the problem you can have a look at the SearchForest [1]. I student made a prototype which can be found at [2]. Cheers, Florent [1] http://combinat.sagemath.org/doc/reference/sage/combinat/backtrack.html [2] http://combinat.sagemath.org/patches/file/931c5a1e58bd/SFparalell-fh.patch -- To post to this group, send an email to sage-devel@googlegroups.com To unsubscribe from this group, send an email to sage-devel+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/sage-devel URL: http://www.sagemath.org
