I can work with my institution and the NSF that we committ the time on the Beacon supercomputing cluster to Apache and the Drill project. Maybe 20 hours a month for 4-5 nodes.
I have discretionary hours that I can put in, and I can, with our HPC admins, create deploy scripts on few clustered machines (these are all very large boxes with 16 cores, 256 GB, 40gb IB interconnect, and with local 1 TB SSD each). There is also Medusa 10 PB filesystem attached but HDFS over local drives would probably be better. They are otherwise just a regular machines, and run regular JVMs on Linux. We can also get Rahul an access with a secure token to setup and run stress/performance/integration tests for Drill. I can actually help there as well. This can be automated to run tests and collect results. I think that the only requirement would be that the JICS team be named for commitment because both NSF/XSEDE and UT like to see the resources being officially used and acknowledged. They are there to support open and academic research; open source projects fit well. If this sounds OK with the project PMCs, I can start the process of allocation, accounts creation, setup. I would also, as a CDO, of JICS sign whatever standard papers with the Apache organization. With all this being said, let me know please if this is something we want to pursue. Thank you, Edmon On Tuesday, September 1, 2015, Jacques Nadeau <jacq...@dremio.com> wrote: > I spent a bunch of time looking at the Phi coprocessors and forgot to get > back to the thread. I'd love it if someone spent some time looking at > leveraging them (since Drill is frequently processor bound). Any takers? > > > > -- > Jacques Nadeau > CTO and Co-Founder, Dremio > > On Mon, Aug 31, 2015 at 10:24 PM, Parth Chandra <par...@apache.org > <javascript:;>> wrote: > > > Hi Edmon, > > Sorry no one seems to have got back to you on this. > > We are in the process of publishing a test suite for regression testing > > Drill and the cluster you have (even a few nodes ) would be a great > > resource for folks to run the test suite. Rahul, et al are working on > this > > and I would suggest watching out for Rahul's posts on the topic. > > > > Parth > > > > On Tue, Aug 25, 2015 at 9:55 PM, Edmon Begoli <ebeg...@gmail.com > <javascript:;>> wrote: > > > > > Hey folks, > > > > > > As we discussed today on a hangout, this is a machine that we have at > > > JICS/NICS > > > where I have Drill installed and where I could set up a test cluster > over > > > few nodes. > > > > > > > https://www.nics.tennessee.edu/computing-resources/beacon/configuration > > > > > > Note that each node is: > > > - 2x8-core Intel® Xeon® E5-2670 processors > > > - 256 GB of memory > > > - 4 Intel® Xeon Phi™ coprocessors 5110P with 8 GB of memory each > > > - 960 GB of SSD storage > > > > > > Would someone advise on what would be an interesting test setup? > > > > > > Thank you, > > > Edmon > > > > > >
