Hi Nacho,
Robert Metzger is our Bahir-Flink expert. I had sent him an email last week, but he may be out on vacation. I'm sure he will respond as soon as he gets a chance. Best regards, Christian Kadner ----- Forwarded by Christian R Kadner on 11/10/2017 03:46 PM ----- From: Christian R Kadner To: "Robert Metzger" <[email protected]> Date: 11/03/2017 02:08 PM Subject: Fw: New Flink connector Hi Robert, can you be the "somebody out there" ? :-) Thank you! Best regards, Christian Kadner ----- Forwarded by Christian R Kadner on 11/03/2017 02:06 PM ----- From: Nacho Garcia Fernandez <[email protected]> To: [email protected] Date: 11/03/2017 03:45 AM Subject: Re: New Flink connector Can somebody out there please reply my last question? Thanks in advance :D On 27 October 2017 at 14:23, Nacho Garcia Fernandez <[email protected] > wrote: > Hello all. > > I'm a little bit stuck with one issue that I hope you can help me with. > > I'm developing a flink-connector-kudu extension that allows to read from > Kudu and write to Flink and Kudu. This connector addresses the issue > [BAHIR-99] and is a full re-implemnetation of https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_apache_&d=DwIBaQ&c=jf_iaSHvJObTbx-siA1ZOg&r=LkTV6i-B9uor_5OdoaHPLFEnKf9LM710vW1s9F24nkY&m=7Ro-HONA-7XK9d-e8RrQzkGgf4f5jtwZ6iGHZ2UAec8&s=6Kw3Tfgk7WG3l9jq-JraMQu6im6XSJZjsk7CxHY8w84&e= > bahir-flink/pull/17. > > I'm struggle with testing: How is it supposed to be handled when the data > storage (kudu) do not provide an embedded driver? > > In the case of Kudu, it does not provide any embedded java-based driver > yet and I need a built Kudu to perform testing against it, otherwise I > cannot test (e2e) this connector with a "real" data storage. > > Because of that I see three main possibilities for this scenario: > > * Create a Mock for Kudu classes (KuduSession, KuduTable, KuduClient, etc). > > * Use MiniKuduCluster utility of Kudu to instantiate a local cluster: it > is not possible due to the fact that this needs a real build of Kudu in the > local machine. > > * Update travis.yml to install a Kudu server: it would fix the problem for > CI, but tests would fail locally. Moreover, bulding Kudu takes so long > (more than 20 minutes), which is not feasible for CI. > > * Ignore testing: not an option :) > > > In the case of Kudu, I saw that other connectors for other distributed > analytics platforms (i.e spark) are directly implemented in the Kudu repo ( > https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_apache_kudu_tree_master_java_kudu-2Dspark&d=DwIBaQ&c=jf_iaSHvJObTbx-siA1ZOg&r=LkTV6i-B9uor_5OdoaHPLFEnKf9LM710vW1s9F24nkY&m=7Ro-HONA-7XK9d-e8RrQzkGgf4f5jtwZ6iGHZ2UAec8&s=5bsVpEulsCi6iz3Cm2-KPs6tB5q-I8vxwHhtXamGfvA&e= ) instead of > using bahir-spark. I think this is good because when you execute the tests > you have a real build of Kudu to perform testing against it. > > What is the best place (kudu vs bahir) for this connector if we take into > consideration the abovementioned issues? > > If the answer is bahir-flink, how should I proceed with my tests? :) > > Thanks in advance. > >
