"Please also note that Kylin's metadata is persisted to HBase by default, you need have another implementation for ResourceStore."
I found almost 600 matches for string "hbase" in modules other then hbase-storage. Hbase seems tightly integrated into modules where it should not be. Even if it is used for metadata it should been pluggable and not directly called from other modules. I can't even run DebugTomcat.java because it relies on classes like AclHBaseStorage (part of server-base) and calls methods of Hbase storage directly. I think we should make Hbase dependency injection little bit more pluggable then what it is now. Or use embedded database like derby or rdbms like mysql which are not so costly to deploy like hbase. Thanks, Nirav On Mon, Jul 10, 2017 at 11:03 AM, Nirav Patel <npa...@xactlycorp.com> wrote: > No equivalent. Instead they came up with Drill - MaprDB connector for > performance and flexibility. So like earlier version of Kylin I should be > able to make drill calls but still do filtering and aggregation on cluster. > Only thing is we need to run separate drill-bit service under Yarn for > this to work. > > Thanks, > Nirav > > On Sun, Jul 9, 2017 at 5:06 PM, ShaoFeng Shi <shaofeng...@apache.org> > wrote: > >> Is there some coprocessor-equivalent mechanism in maprdb? If yes, you can >> try to implement the logic into it. Otherwise, the performance might be >> unacceptable when there is bunch of data in each region. >> >> I rememer in very early version, Kylin doesn't have coprocessor, all data >> be fetched to Kylin server and then do filtering and aggregation by >> Calcite. Later coprocessor was added and improved continiously. Now there >> is no option to disable it. You have to modify the code if don't want to >> trigger it. >> >> >> >> 2017-07-09 1:34 GMT+08:00 Nirav Patel <npa...@xactlycorp.com>: >> >> > >> > Thanks Shao for some pointers. I would like some more advice on how I >> can >> > replace maprdb as datasource. >> > >> > Currently, you can create cubes on maprdb using hbase-storage module as >> > maprdb does support hbase client/admin APIs. Only problem is while >> querying >> > it throws an error due to Lack of support of Co-processor. >> > >> > My initial goal is to get past coprocessor error and return query >> result. >> > Then find a better way to implement that strategy - may be a separate >> > maprdb-storage module. Then look into how to improve performance without >> > coprocessors. Any advice on these approaches ? >> > >> > Thanks, >> > Nirav >> > >> > On Sat, Jul 8, 2017 at 7:54 AM, ShaoFeng Shi <shaofeng...@apache.org> >> > wrote: >> > >> >> The core modules like core-cube, core-storage are totally independent >> of >> >> HBase; while some others like engine-mr, engine-spark has dependencies >> on >> >> HBase. If you want to replace it, you need implement new cubing engine >> as >> >> well. Please also note that Kylin's metadata is persisted to HBase by >> >> default, you need have another implementation for ResourceStore. >> >> >> >> In a short, the plug-in architecture works (we have verfied that), >> while >> >> changing the storage is a complex task which take some time to be >> function >> >> complete and performance stable. >> >> >> >> >> >> >> >> >> >> >> >> >> >> 2017-07-08 5:19 GMT+08:00 Nirav Patel <npa...@xactlycorp.com>: >> >> >> >>> Yes, MapR-DB doesn't support coprocessors. >> >>> >> >>> Here's the thing though - based on kylin plugin architecture it >> >>> shouldn't be a problem ideally. Aggregation as well as other DML/DDL >> >>> operation on datasources should be done transparently. i.e. using >> >>> kylin-hbase adapter written with calcite or something. It's upto the >> writer >> >>> of those adapters to implement aggregates however they want. i.e. >> either >> >>> using coprocessors, or in-memory on application server side or using >> spark. >> >>> http://kylin.apache.org/development/plugin_arch.html >> >>> >> >>> I think for mapr-db we can leverage mapr-drill with secondary indexes >> >>> for faster filtering and drill does parallel aggregation as well. >> Other >> >>> option is to use mapr-spark with mapr-db which can does the same. >> >>> >> >>> Do you know how tightly hbase is coupled with other modules of kylin >> >>> source other then hbase-storage. >> >>> >> >>> >> >>> >> >>> >> >>> >> >>> >> >>> On Thu, Jul 6, 2017 at 6:53 PM, ShaoFeng Shi <shaofeng...@apache.org> >> >>> wrote: >> >>> >> >>>> Hi Nirav, >> >>>> >> >>>> I googled that "HBase coprocessors are not present in MapR DB", is >> >>>> this true? You know Kylin relies on HBase coprocessor to do filering >> and >> >>>> aggregation in each region local; If coprocessor is not available, >> the >> >>>> performance will be a problem. >> >>>> >> >>>> 2017-07-07 1:29 GMT+08:00 Nirav Patel <npa...@xactlycorp.com>: >> >>>> >> >>>>> Hi, We are a mapr users. You will need to deploy separate Hbase >> >>>>> Cluster alongside your mapr Cluster or on top of it. You won't be >> able to >> >>>>> use Mapr-DB with Kylin. >> >>>>> I am looking into writing Kyling - MaprDB adapter. >> >>>>> >> >>>>> On Tue, Jul 4, 2017 at 8:23 AM, <jun....@nomura.com> wrote: >> >>>>> >> >>>>>> V5.2, >> >>>>>> >> >>>>>> I’m trying to figure out how to deploy Kylin on a MapR cluster, >> >>>>>> whether Kylin has to be installed on the same cluster of the MapR >> cluster >> >>>>>> >> >>>>>> >> >>>>>> >> >>>>>> *From:* Luke Han [mailto:luke...@gmail.com] >> >>>>>> *Sent:* Tuesday, July 04, 2017 10:04 PM >> >>>>>> *To:* user >> >>>>>> *Subject:* Re: Kylin for MapR >> >>>>>> >> >>>>>> >> >>>>>> >> >>>>>> Kylin support MapR, which MapR version you are using now? >> >>>>>> >> >>>>>> >> >>>>>> >> >>>>>> >> >>>>>> Best Regards! >> >>>>>> --------------------- >> >>>>>> >> >>>>>> Luke Han >> >>>>>> >> >>>>>> >> >>>>>> >> >>>>>> On Tue, Jul 4, 2017 at 4:19 PM, <jun....@nomura.com> wrote: >> >>>>>> >> >>>>>> Hi >> >>>>>> >> >>>>>> Does kylin support MapR version of the Hadoop? >> >>>>>> >> >>>>>> >> >>>>>> >> >>>>>> This e-mail (including any attachments) is private and >> confidential, >> >>>>>> may contain proprietary or privileged information and is intended >> for the >> >>>>>> named recipient(s) only. Unintended recipients are strictly >> prohibited from >> >>>>>> taking action on the basis of information in this e-mail and must >> contact >> >>>>>> the sender immediately, delete this e-mail (and all attachments) >> and >> >>>>>> destroy any hard copies. Nomura will not accept responsibility or >> liability >> >>>>>> for the accuracy or completeness of, or the presence of any virus >> or >> >>>>>> disabling code in, this e-mail. If verification is sought please >> request a >> >>>>>> hard copy. Any reference to the terms of executed transactions >> should be >> >>>>>> treated as preliminary only and subject to formal written >> confirmation by >> >>>>>> Nomura. Nomura reserves the right to retain, monitor and intercept >> e-mail >> >>>>>> communications through its networks (subject to and in accordance >> with >> >>>>>> applicable laws). No confidentiality or privilege is waived or >> lost by >> >>>>>> Nomura by any mistransmission of this e-mail. Any reference to >> "Nomura" is >> >>>>>> a reference to any entity in the Nomura Holdings, Inc. group. >> Please read >> >>>>>> our Electronic Communications Legal Notice which forms part of >> this e-mail: >> >>>>>> http://www.Nomura.com/email_disclaimer.htm >> >>>>>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__www. >> Nomura.com_email-5Fdisclaimer.htm&d=DwMFaQ&c=dCBwIlVXJsYZrY6 >> gpNt0LA&r=nFqGYx5SOTBIei-Hwk8uz_JfhmSu2UlelQfdmGX2k5I& >> m=s6jR2dvhFFTZIuHhCE8WU8EYNQ7M5u-9SggFAoyn5-M&s=iXZr3lmnuJ4X >> k2-qORnbYZQALda0jRLZ3kzGy816bco&e=> >> >>>>>> >> >>>>>> >> >>>>>> >> >>>>>> This e-mail (including any attachments) is private and >> confidential, >> >>>>>> may contain proprietary or privileged information and is intended >> for the >> >>>>>> named recipient(s) only. Unintended recipients are strictly >> prohibited from >> >>>>>> taking action on the basis of information in this e-mail and must >> contact >> >>>>>> the sender immediately, delete this e-mail (and all attachments) >> and >> >>>>>> destroy any hard copies. Nomura will not accept responsibility or >> liability >> >>>>>> for the accuracy or completeness of, or the presence of any virus >> or >> >>>>>> disabling code in, this e-mail. If verification is sought please >> request a >> >>>>>> hard copy. Any reference to the terms of executed transactions >> should be >> >>>>>> treated as preliminary only and subject to formal written >> confirmation by >> >>>>>> Nomura. Nomura reserves the right to retain, monitor and intercept >> e-mail >> >>>>>> communications through its networks (subject to and in accordance >> with >> >>>>>> applicable laws). No confidentiality or privilege is waived or >> lost by >> >>>>>> Nomura by any mistransmission of this e-mail. Any reference to >> "Nomura" is >> >>>>>> a reference to any entity in the Nomura Holdings, Inc. group. >> Please read >> >>>>>> our Electronic Communications Legal Notice which forms part of >> this e-mail: >> >>>>>> http://www.Nomura.com/email_disclaimer.htm >> >>>>>> >> >>>>> >> >>>>> >> >>>>> >> >>>>> >> >>>>> [image: What's New with Xactly] >> >>>>> <http://www.xactlycorp.com/email-click/> >> >>>>> >> >>>>> <https://www.nyse.com/quote/XNYS:XTLY> [image: LinkedIn] >> >>>>> <https://www.linkedin.com/company/xactly-corporation> [image: >> >>>>> Twitter] <https://twitter.com/Xactly> [image: Facebook] >> >>>>> <https://www.facebook.com/XactlyCorp> [image: YouTube] >> >>>>> <http://www.youtube.com/xactlycorporation> >> >>>> >> >>>> >> >>>> >> >>>> >> >>>> -- >> >>>> Best regards, >> >>>> >> >>>> Shaofeng Shi 史少锋 >> >>>> >> >>>> >> >>> >> >>> >> >>> >> >>> [image: What's New with Xactly] <http://www.xactlycorp.com/ema >> il-click/> >> >>> >> >>> <https://www.nyse.com/quote/XNYS:XTLY> [image: LinkedIn] >> >>> <https://www.linkedin.com/company/xactly-corporation> [image: >> Twitter] >> >>> <https://twitter.com/Xactly> [image: Facebook] >> >>> <https://www.facebook.com/XactlyCorp> [image: YouTube] >> >>> <http://www.youtube.com/xactlycorporation> >> >>> >> >> >> >> >> >> >> >> -- >> >> Best regards, >> >> >> >> Shaofeng Shi 史少锋 >> >> >> >> >> > >> > >> > >> > [image: What's New with Xactly] <http://www.xactlycorp.com/email-click/ >> > >> > >> > <https://www.nyse.com/quote/XNYS:XTLY> [image: LinkedIn] >> > <https://www.linkedin.com/company/xactly-corporation> [image: Twitter] >> > <https://twitter.com/Xactly> [image: Facebook] >> > <https://www.facebook.com/XactlyCorp> [image: YouTube] >> > <http://www.youtube.com/xactlycorporation> >> > >> >> >> >> -- >> Best regards, >> >> Shaofeng Shi 史少锋 >> > > -- [image: What's New with Xactly] <http://www.xactlycorp.com/email-click/> <https://www.nyse.com/quote/XNYS:XTLY> [image: LinkedIn] <https://www.linkedin.com/company/xactly-corporation> [image: Twitter] <https://twitter.com/Xactly> [image: Facebook] <https://www.facebook.com/XactlyCorp> [image: YouTube] <http://www.youtube.com/xactlycorporation>
