Re: Query Metadata

[Alberto Ramón]

HUE 1.12 and Kylin 1.6.0, works (no path is needed):
[image: Inline images 1]


But have an exception:
An error occurred while calling o.execute. : java.sql.SQLException:
Error while executing SQL "*SHOW DATABASES"*: java.sql.SQLException:
java.io.IOException: POST failed, error code 500 and response: {"url":"
http://172.17.0.2:7070/kylin/api/query","exception":"Not Supported SQL."}
at
org.apache.kylin.jdbc.shaded.org.apache.calcite.avatica.Helper.createException(Helper.java:56)
at
org.apache.kylin.jdbc.shaded.org.apache.calcite.avatica.Helper.createException(Helper.java:41)
at
org.apache.kylin.jdbc.shaded.org.apache.calcite.avatica.AvaticaStatement.executeInternal(AvaticaStatement.java:147)
at
org.apache.kylin.jdbc.shaded.org.apache.calcite.avatica.AvaticaStatement.execute(AvaticaStatement.java:199)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498) at
py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:231) at
py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:381) at
py4j.Gateway.invoke(Gateway.java:259) at
py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:133) at
py4j.commands.CallCommand.execute(CallCommand.java:79) at
py4j.GatewayConnection.run(GatewayConnection.java:209) at
java.lang.Thread.run(Thread.java:748) Caused by:
java.lang.RuntimeException: java.sql.SQLException: java.io.IOException:
POST failed, error code 500 and response: {"url":"
http://172.17.0.2:7070/kylin/api/query","exception":"Not Supported SQL."}

work? yes, but you must put the name cube in HUE config.
The connection is fixed to one cube

On 29 August 2016 at 18:10, Alberto Ramón  wrote:

> Hi   (This is ONLY a Idea - suggestion)
>
> I saw some problems integration between Kylin and HUE or Tableau, when try
> to discover metadata info, like: list of databases, list of tables and list
> of columns.
>
> The most clear example: HUE 4011
> , only works with
> * Show Databases;
> * Show Tables;
> * Show Columns from X.Y
>
> The result is the same:
>[image: Imágenes integradas 1]
> Other programs, tyr "select * from tb", to return list of columns  you
> know the result of this   ;)
>
> Other programs:
> * start all querires with a "use dbName; select . . ."
> * or try "from dbName.tbName"  (Nowadays there is a bug)
>
>
> Really: are small things... but complicate a lot the integration with
> others Apps
>
>
>
> A lot of thanks for all, Alb
>
>

Re: Kylin for MapR

[Nirav Patel]

Thanks Shao for some pointers. I would like some more advice on how I can
replace maprdb as datasource.

Currently, you can  create cubes on maprdb using hbase-storage module as
maprdb does support hbase client/admin APIs. Only problem is while querying
it throws an error due to Lack of support of Co-processor.

My initial goal is to get past coprocessor error and return query result.
Then find a better way to implement that strategy - may be a separate
maprdb-storage module. Then look into how to improve performance without
coprocessors. Any advice on these approaches ?

Thanks,
Nirav

On Sat, Jul 8, 2017 at 7:54 AM, ShaoFeng Shi  wrote:

> The core modules like core-cube, core-storage are totally independent of
> HBase; while some others like engine-mr, engine-spark has dependencies on
> HBase. If you want to replace it, you need implement new cubing engine as
> well. Please also note that Kylin's metadata is persisted to HBase by
> default, you need have another implementation for ResourceStore.
>
> In a short, the plug-in architecture works (we have verfied that), while
> changing the storage is a complex task which take some time to be function
> complete and performance stable.
>
>
>
>
>
>
> 2017-07-08 5:19 GMT+08:00 Nirav Patel :
>
>> Yes, MapR-DB doesn't support coprocessors.
>>
>> Here's the thing though - based on kylin plugin architecture it shouldn't
>> be a problem ideally. Aggregation as well as other DML/DDL operation on
>> datasources should be done transparently. i.e. using kylin-hbase adapter
>> written with calcite or something. It's upto the writer of those adapters
>> to implement aggregates however they want. i.e. either using coprocessors,
>> or in-memory on application server side or using spark.
>> http://kylin.apache.org/development/plugin_arch.html
>>
>> I think for mapr-db we can leverage mapr-drill with secondary indexes for
>> faster filtering and drill does parallel aggregation as well. Other option
>> is to use mapr-spark with mapr-db which can does the same.
>>
>> Do you know how tightly hbase is coupled with other modules of kylin
>> source other then hbase-storage.
>>
>>
>>
>>
>>
>>
>> On Thu, Jul 6, 2017 at 6:53 PM, ShaoFeng Shi 
>> wrote:
>>
>>> Hi Nirav,
>>>
>>> I googled that "HBase coprocessors are not present in MapR DB", is this
>>> true? You know Kylin relies on HBase coprocessor to do filering and
>>> aggregation in each region local; If coprocessor is not available, the
>>> performance will be a problem.
>>>
>>> 2017-07-07 1:29 GMT+08:00 Nirav Patel :
>>>
 Hi, We are a mapr users. You will need to deploy separate Hbase Cluster
 alongside your mapr Cluster or on top of it. You won't be able to use
 Mapr-DB with Kylin.
 I am looking into writing Kyling - MaprDB adapter.

 On Tue, Jul 4, 2017 at 8:23 AM,  wrote:

> V5.2,
>
> I’m trying to figure out how to deploy Kylin on a MapR cluster,
> whether Kylin has to be installed on the same cluster of the MapR cluster
>
>
>
> *From:* Luke Han [mailto:luke...@gmail.com]
> *Sent:* Tuesday, July 04, 2017 10:04 PM
> *To:* user
> *Subject:* Re: Kylin for MapR
>
>
>
> Kylin support MapR, which MapR version you are using now?
>
>
>
>
> Best Regards!
> -
>
> Luke Han
>
>
>
> On Tue, Jul 4, 2017 at 4:19 PM,  wrote:
>
> Hi
>
> Does kylin support MapR version of the Hadoop?
>
>
>
> This e-mail (including any attachments) is private and confidential,
> may contain proprietary or privileged information and is intended for the
> named recipient(s) only. Unintended recipients are strictly prohibited 
> from
> taking action on the basis of information in this e-mail and must contact
> the sender immediately, delete this e-mail (and all attachments) and
> destroy any hard copies. Nomura will not accept responsibility or 
> liability
> for the accuracy or completeness of, or the presence of any virus or
> disabling code in, this e-mail. If verification is sought please request a
> hard copy. Any reference to the terms of executed transactions should be
> treated as preliminary only and subject to formal written confirmation by
> Nomura. Nomura reserves the right to retain, monitor and intercept e-mail
> communications through its networks (subject to and in accordance with
> applicable laws). No confidentiality or privilege is waived or lost by
> Nomura by any mistransmission of this e-mail. Any reference to "Nomura" is
> a reference to any entity in the Nomura Holdings, Inc. group. Please read
> our Electronic Communications Legal Notice which forms part of this 
> e-mail:
> http://www.Nomura.com/email_disclaimer.htm
>