Thanks Shao for some pointers. I would like some more advice on how I can replace maprdb as datasource.
Currently, you can create cubes on maprdb using hbase-storage module as maprdb does support hbase client/admin APIs. Only problem is while querying it throws an error due to Lack of support of Co-processor. My initial goal is to get past coprocessor error and return query result. Then find a better way to implement that strategy - may be a separate maprdb-storage module. Then look into how to improve performance without coprocessors. Any advice on these approaches ? Thanks, Nirav On Sat, Jul 8, 2017 at 7:54 AM, ShaoFeng Shi <shaofeng...@apache.org> wrote: > The core modules like core-cube, core-storage are totally independent of > HBase; while some others like engine-mr, engine-spark has dependencies on > HBase. If you want to replace it, you need implement new cubing engine as > well. Please also note that Kylin's metadata is persisted to HBase by > default, you need have another implementation for ResourceStore. > > In a short, the plug-in architecture works (we have verfied that), while > changing the storage is a complex task which take some time to be function > complete and performance stable. > > > > > > > 2017-07-08 5:19 GMT+08:00 Nirav Patel <npa...@xactlycorp.com>: > >> Yes, MapR-DB doesn't support coprocessors. >> >> Here's the thing though - based on kylin plugin architecture it shouldn't >> be a problem ideally. Aggregation as well as other DML/DDL operation on >> datasources should be done transparently. i.e. using kylin-hbase adapter >> written with calcite or something. It's upto the writer of those adapters >> to implement aggregates however they want. i.e. either using coprocessors, >> or in-memory on application server side or using spark. >> http://kylin.apache.org/development/plugin_arch.html >> >> I think for mapr-db we can leverage mapr-drill with secondary indexes for >> faster filtering and drill does parallel aggregation as well. Other option >> is to use mapr-spark with mapr-db which can does the same. >> >> Do you know how tightly hbase is coupled with other modules of kylin >> source other then hbase-storage. >> >> >> >> >> >> >> On Thu, Jul 6, 2017 at 6:53 PM, ShaoFeng Shi <shaofeng...@apache.org> >> wrote: >> >>> Hi Nirav, >>> >>> I googled that "HBase coprocessors are not present in MapR DB", is this >>> true? You know Kylin relies on HBase coprocessor to do filering and >>> aggregation in each region local; If coprocessor is not available, the >>> performance will be a problem. >>> >>> 2017-07-07 1:29 GMT+08:00 Nirav Patel <npa...@xactlycorp.com>: >>> >>>> Hi, We are a mapr users. You will need to deploy separate Hbase Cluster >>>> alongside your mapr Cluster or on top of it. You won't be able to use >>>> Mapr-DB with Kylin. >>>> I am looking into writing Kyling - MaprDB adapter. >>>> >>>> On Tue, Jul 4, 2017 at 8:23 AM, <jun....@nomura.com> wrote: >>>> >>>>> V5.2, >>>>> >>>>> I’m trying to figure out how to deploy Kylin on a MapR cluster, >>>>> whether Kylin has to be installed on the same cluster of the MapR cluster >>>>> >>>>> >>>>> >>>>> *From:* Luke Han [mailto:luke...@gmail.com] >>>>> *Sent:* Tuesday, July 04, 2017 10:04 PM >>>>> *To:* user >>>>> *Subject:* Re: Kylin for MapR >>>>> >>>>> >>>>> >>>>> Kylin support MapR, which MapR version you are using now? >>>>> >>>>> >>>>> >>>>> >>>>> Best Regards! >>>>> --------------------- >>>>> >>>>> Luke Han >>>>> >>>>> >>>>> >>>>> On Tue, Jul 4, 2017 at 4:19 PM, <jun....@nomura.com> wrote: >>>>> >>>>> Hi >>>>> >>>>> Does kylin support MapR version of the Hadoop? >>>>> >>>>> >>>>> >>>>> This e-mail (including any attachments) is private and confidential, >>>>> may contain proprietary or privileged information and is intended for the >>>>> named recipient(s) only. Unintended recipients are strictly prohibited >>>>> from >>>>> taking action on the basis of information in this e-mail and must contact >>>>> the sender immediately, delete this e-mail (and all attachments) and >>>>> destroy any hard copies. Nomura will not accept responsibility or >>>>> liability >>>>> for the accuracy or completeness of, or the presence of any virus or >>>>> disabling code in, this e-mail. If verification is sought please request a >>>>> hard copy. Any reference to the terms of executed transactions should be >>>>> treated as preliminary only and subject to formal written confirmation by >>>>> Nomura. Nomura reserves the right to retain, monitor and intercept e-mail >>>>> communications through its networks (subject to and in accordance with >>>>> applicable laws). No confidentiality or privilege is waived or lost by >>>>> Nomura by any mistransmission of this e-mail. Any reference to "Nomura" is >>>>> a reference to any entity in the Nomura Holdings, Inc. group. Please read >>>>> our Electronic Communications Legal Notice which forms part of this >>>>> e-mail: >>>>> http://www.Nomura.com/email_disclaimer.htm >>>>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__www.Nomura.com_email-5Fdisclaimer.htm&d=DwMFaQ&c=dCBwIlVXJsYZrY6gpNt0LA&r=nFqGYx5SOTBIei-Hwk8uz_JfhmSu2UlelQfdmGX2k5I&m=s6jR2dvhFFTZIuHhCE8WU8EYNQ7M5u-9SggFAoyn5-M&s=iXZr3lmnuJ4Xk2-qORnbYZQALda0jRLZ3kzGy816bco&e=> >>>>> >>>>> >>>>> >>>>> This e-mail (including any attachments) is private and confidential, >>>>> may contain proprietary or privileged information and is intended for the >>>>> named recipient(s) only. Unintended recipients are strictly prohibited >>>>> from >>>>> taking action on the basis of information in this e-mail and must contact >>>>> the sender immediately, delete this e-mail (and all attachments) and >>>>> destroy any hard copies. Nomura will not accept responsibility or >>>>> liability >>>>> for the accuracy or completeness of, or the presence of any virus or >>>>> disabling code in, this e-mail. If verification is sought please request a >>>>> hard copy. Any reference to the terms of executed transactions should be >>>>> treated as preliminary only and subject to formal written confirmation by >>>>> Nomura. Nomura reserves the right to retain, monitor and intercept e-mail >>>>> communications through its networks (subject to and in accordance with >>>>> applicable laws). No confidentiality or privilege is waived or lost by >>>>> Nomura by any mistransmission of this e-mail. Any reference to "Nomura" is >>>>> a reference to any entity in the Nomura Holdings, Inc. group. Please read >>>>> our Electronic Communications Legal Notice which forms part of this >>>>> e-mail: >>>>> http://www.Nomura.com/email_disclaimer.htm >>>>> >>>> >>>> >>>> >>>> >>>> [image: What's New with Xactly] >>>> <http://www.xactlycorp.com/email-click/> >>>> >>>> <https://www.nyse.com/quote/XNYS:XTLY> [image: LinkedIn] >>>> <https://www.linkedin.com/company/xactly-corporation> [image: Twitter] >>>> <https://twitter.com/Xactly> [image: Facebook] >>>> <https://www.facebook.com/XactlyCorp> [image: YouTube] >>>> <http://www.youtube.com/xactlycorporation> >>> >>> >>> >>> >>> -- >>> Best regards, >>> >>> Shaofeng Shi 史少锋 >>> >>> >> >> >> >> [image: What's New with Xactly] <http://www.xactlycorp.com/email-click/> >> >> <https://www.nyse.com/quote/XNYS:XTLY> [image: LinkedIn] >> <https://www.linkedin.com/company/xactly-corporation> [image: Twitter] >> <https://twitter.com/Xactly> [image: Facebook] >> <https://www.facebook.com/XactlyCorp> [image: YouTube] >> <http://www.youtube.com/xactlycorporation> >> > > > > -- > Best regards, > > Shaofeng Shi 史少锋 > > -- [image: What's New with Xactly] <http://www.xactlycorp.com/email-click/> <https://www.nyse.com/quote/XNYS:XTLY> [image: LinkedIn] <https://www.linkedin.com/company/xactly-corporation> [image: Twitter] <https://twitter.com/Xactly> [image: Facebook] <https://www.facebook.com/XactlyCorp> [image: YouTube] <http://www.youtube.com/xactlycorporation>
