On the mini cluster as a maven artifact: The Beam KuduIO is in progress here [1], with Integration test (currently I use Docker) and I just refactored the code so I could mock a KuduService for unit tests. It is an ideal time to try your current work Mike/Grant, as I'd use a minicluster instead of mocking. Perhaps we should continue this on the dev@ list discussion I started a few weeks back [2]? Do you have a build for OS X by any chance?
> I'm not really sure there is a lot of overlap between creating a Docker > image and the kind of relocatable artifacts I'm trying to build, aside from > the actual compiling part. I've had to manipulate hosts files with fake entries to get the Docker images to work while the minicluster seemingly has a FakeDNS thing (?). That might just be me doing things wrong though. [1] https://github.com/timrobertson100/beam/tree/BEAM-2661-KuduIO/sdks/java/io/kudu [2] https://lists.apache.org/thread.html/ee697a022b72bbca2761b1af0581773d8fb708f701fc969bc259fc2d@%3Cdev.kudu.apache.org%3E On Wed, Jul 18, 2018 at 4:37 AM, Mike Percy <mpe...@apache.org> wrote: > On Tue, Jul 17, 2018 at 12:22 PM Grant Henke <ghe...@cloudera.com.invalid> > wrote: > > > I have started a document for blog post ideas/topics here: > > > > https://docs.google.com/document/d/12QFRIhNDMoOI1kOQBgch64xYJ9t6U > byVt1D3NaTl7lI/edit?usp=sharing > > > > Nice list, Grant. Actually I think that quarterly email would probably make > for a better blog post instead and I've added it as a suggestion on that > doc. > > On Tue, Jul 17, 2018 at 12:04 PM Mauricio Aristizabal <mauri...@impact.com > > > wrote: > > > I was disappointed that Strata SJ 2018 didn't have a single session on > > Kudu, there were no committers in attendance that I could tell, and it > > wasn't being highlighted at all in the Cloudera booth. Between Strata > and > > ScalaDays I must have enthusiastically mentioned the product to 15 people > > and none had heard of it. > > > > Hmm, that is disappointing, and a bit surprising. Perhaps everybody thought > everybody else was going to submit... actually I had intended to submit a > talk proposal to Strata this year but got busy and missed the deadline. :( > > I wonder if folks using Kudu would like to present on their use case? I'm > sure conference-goers would like to hear from more people using Kudu "in > anger" (hopefully not angrily). > > On Tue, Jul 17, 2018 at 2:59 PM Sailesh Mukil <sail...@cloudera.com.invalid > > > wrote: > > > A suggestion to add on to the easily downloadable pre-built packages, is > to > > have easily accessible/downloadable example test-data that's fairly > > representative of real world datasets (but it doesn't have to be too > > large). Additionally, we can write tutorials in kudu/examples/ that use > > this test data, to give new users a better feel for the system. > > > That sounds useful. Any ideas on where we could find such a data set? > > On Tue, Jul 17, 2018 at 11:59 AM Tim Robertson <timrobertson...@gmail.com> > wrote: > > > ++1 on the mini cluster > > Perhaps include a docker image build at the same time which presumably > > wouldn't be much effort? > > > > I'm not really sure there is a lot of overlap between creating a Docker > image and the kind of relocatable artifacts I'm trying to build, aside from > the actual compiling part. But I think it would be valuable for Docker > users to be able to easily pull down a Kudu image. > > > > l'll be happy to contribute on the Java / maven related parts to that. I > > will use this for the testing framework for the Apache Beam KuduIO and > will > > certainly help test / write a blog. > > > > I don't really know how to handle the Maven part where we unpack the > tarball and set it up somewhere so we can invoke it from the > KuduMiniCluster. Maybe it that would require writing a custom Maven plugin? > > I'd love to see a blog post about how to use Kudu with Beam! > > Mike >
