Re: DRILL-4199: Add Support for HBase 1.X - planning to merge
Hi all, Just to report my test progress. As the SQL always throw schema changes exception. I reformatted my SQL, now it will not throw exception. I changed SQL *from * select CONVERT_FROM(BYTE_SUBSTR(`ref0`.row_key,-8,8),'BIGINT_BE') as uid, convert_from(`ref0`.`v`.`v`,'UTF8') as v from hbase.`offers_nation_idx` as `nation` join hbase.offers_ref0 as `ref0` on BYTE_SUBSTR(`ref0`.row_key,-8,8) = nation.`v`.`v` where `nation`.row_key > '0br' and `nation`.row_key < '0bs' limit 10 *to * select t1.uid, t1.v from (select CONVERT_FROM(BYTE_SUBSTR(`ref0`.row_key,-8,8),'BIGINT_BE') as uid, convert_from(`ref0`.`v`.`v`,'UTF8') as v from hbase.offers_ref0 as `ref0` where `ref0`.row_key > '0' and `ref0`.row_key < '1' ) t1 join (select CONVERT_FROM(nation.`v`.`v`, 'BIGINT_BE') uid from hbase.`offers_nation_idx` as `nation` where `nation`.row_key > '0br' and `nation`.row_key < '0bs' ) t2 on t1.uid = t2.uid limit 10 Now, my work can going on. But I think the first SQL should working too, hope someone can take a look at it. 2016-06-23 10:59 GMT+08:00 qiang li : > Dear all, > > Today ,I retest it again step by step. This time I get exception in both > drillbit.log and drillbit.out, also I tested with released 1.6.0, its throw > Hash join does not support schema changes exception too. > > > I have reattached the error msg to 4734, Could some one help me solve this > issue, we are very need this function working. > > Thanks > > 2016-06-22 9:39 GMT+08:00 qiang li : > >> Thanks Aman. >> >> I think the second is not related to hbase too. Its >> happened occasionally. I wil give you more detail when its happen again. >> >> Hi Aditya, there is no error in drillbit.out, but have error in >> drillbit.log, I have attached to the DRILL-4734 >> <https://issues.apache.org/jira/browse/DRILL-4734> , please check it . >> >> >> >> 2016-06-22 4:31 GMT+08:00 Aman Sinha : >> >>> Qiang, >>> The last issue mentioned about failure to start drillbit seems unrelated >>> to HBase isn't it ? For now, I am going with the assumption that it is not >>> a regression, so does not impact the 1.7 release. Of course, if you find >>> out otherwise, we would need to address it. >>> Regarding the SchemaChangeException you reported, I have created a JIRA: >>> DRILL-4734. >>> >>> >>> -Aman >>> >>> On Tue, Jun 21, 2016 at 10:04 AM, Aditya >>> wrote: >>> >>>> There should be another stack trace in drillbit.out when this happens, >>>> could you please check that file. >>>> >>>> On Mon, Jun 20, 2016 at 8:14 PM, qiang li wrote: >>>> >>>> > Another issue is some time when I restart the node, the node can not >>>> be >>>> > startup. >>>> > >>>> > Here is the exception. >>>> > ache-drill-1.7.0/jars/drill-gis-1.7.0-SNAPSHOT.jar!/, >>>> > >>>> jar:file:/usr/lib/apache-drill-1.7.0/jars/drill-memory-base-1.7.0-SNAPSHOT.jar!/] >>>> > took 2800ms >>>> > 2016-06-20 19:10:18,313 [main] INFO >>>> o.a.d.e.s.s.PersistentStoreRegistry - >>>> > Using the configured PStoreProvider class: >>>> > >>>> 'org.apache.drill.exec.store.sys.store.provider.ZookeeperPersistentStoreProvider'. >>>> > 2016-06-20 19:10:19,221 [main] INFO >>>> o.apache.drill.exec.server.Drillbit - >>>> > Construction completed (1529 ms). >>>> > 2016-06-20 19:10:31,136 [main] WARN >>>> o.apache.drill.exec.server.Drillbit - >>>> > Failure on close() >>>> > java.lang.NullPointerException: null >>>> > at >>>> > org.apache.drill.exec.work.WorkManager.close(WorkManager.java:153) >>>> > ~[drill-java-exec-1.7.0-SNAPSHOT.jar:1.7.0-SNAPSHOT] >>>> > at >>>> > org.apache.drill.common.AutoCloseables.close(AutoCloseables.java:76) >>>> > ~[drill-common-1.7.0-SNAPSHOT.jar:1.7.0-SNAPSHOT] >>>> > at >>>> > org.apache.drill.common.AutoCloseables.close(AutoCloseables.java:64) >>>> > ~[drill-common-1.7.0-SNAPSHOT.jar:1.7.0-SNAPSHOT] >>>> > at >>>> org.apache.drill.exec.server.Drillbit.close(Drillbit.java:159) >>>> > [drill-java-exec-1.7.0-SNAPSHOT.jar:1.7.0-SNAPSHOT] >>>> > at >>>> org.apache.drill.exec.server.Drillbit.st
Re: DRILL-4199: Add Support for HBase 1.X - planning to merge
Dear all, Today ,I retest it again step by step. This time I get exception in both drillbit.log and drillbit.out, also I tested with released 1.6.0, its throw Hash join does not support schema changes exception too. I have reattached the error msg to 4734, Could some one help me solve this issue, we are very need this function working. Thanks 2016-06-22 9:39 GMT+08:00 qiang li : > Thanks Aman. > > I think the second is not related to hbase too. Its happened occasionally. > I wil give you more detail when its happen again. > > Hi Aditya, there is no error in drillbit.out, but have error in > drillbit.log, I have attached to the DRILL-4734 > <https://issues.apache.org/jira/browse/DRILL-4734> , please check it . > > > > 2016-06-22 4:31 GMT+08:00 Aman Sinha : > >> Qiang, >> The last issue mentioned about failure to start drillbit seems unrelated >> to HBase isn't it ? For now, I am going with the assumption that it is not >> a regression, so does not impact the 1.7 release. Of course, if you find >> out otherwise, we would need to address it. >> Regarding the SchemaChangeException you reported, I have created a JIRA: >> DRILL-4734. >> >> >> -Aman >> >> On Tue, Jun 21, 2016 at 10:04 AM, Aditya wrote: >> >>> There should be another stack trace in drillbit.out when this happens, >>> could you please check that file. >>> >>> On Mon, Jun 20, 2016 at 8:14 PM, qiang li wrote: >>> >>> > Another issue is some time when I restart the node, the node can not be >>> > startup. >>> > >>> > Here is the exception. >>> > ache-drill-1.7.0/jars/drill-gis-1.7.0-SNAPSHOT.jar!/, >>> > >>> jar:file:/usr/lib/apache-drill-1.7.0/jars/drill-memory-base-1.7.0-SNAPSHOT.jar!/] >>> > took 2800ms >>> > 2016-06-20 19:10:18,313 [main] INFO >>> o.a.d.e.s.s.PersistentStoreRegistry - >>> > Using the configured PStoreProvider class: >>> > >>> 'org.apache.drill.exec.store.sys.store.provider.ZookeeperPersistentStoreProvider'. >>> > 2016-06-20 19:10:19,221 [main] INFO >>> o.apache.drill.exec.server.Drillbit - >>> > Construction completed (1529 ms). >>> > 2016-06-20 19:10:31,136 [main] WARN >>> o.apache.drill.exec.server.Drillbit - >>> > Failure on close() >>> > java.lang.NullPointerException: null >>> > at >>> > org.apache.drill.exec.work.WorkManager.close(WorkManager.java:153) >>> > ~[drill-java-exec-1.7.0-SNAPSHOT.jar:1.7.0-SNAPSHOT] >>> > at >>> > org.apache.drill.common.AutoCloseables.close(AutoCloseables.java:76) >>> > ~[drill-common-1.7.0-SNAPSHOT.jar:1.7.0-SNAPSHOT] >>> > at >>> > org.apache.drill.common.AutoCloseables.close(AutoCloseables.java:64) >>> > ~[drill-common-1.7.0-SNAPSHOT.jar:1.7.0-SNAPSHOT] >>> > at >>> org.apache.drill.exec.server.Drillbit.close(Drillbit.java:159) >>> > [drill-java-exec-1.7.0-SNAPSHOT.jar:1.7.0-SNAPSHOT] >>> > at >>> org.apache.drill.exec.server.Drillbit.start(Drillbit.java:293) >>> > [drill-java-exec-1.7.0-SNAPSHOT.jar:1.7.0-SNAPSHOT] >>> > at >>> org.apache.drill.exec.server.Drillbit.start(Drillbit.java:271) >>> > [drill-java-exec-1.7.0-SNAPSHOT.jar:1.7.0-SNAPSHOT] >>> > at >>> org.apache.drill.exec.server.Drillbit.main(Drillbit.java:267) >>> > [drill-java-exec-1.7.0-SNAPSHOT.jar:1.7.0-SNAPSHOT] >>> > 2016-06-20 19:10:31,137 [main] INFO >>> o.apache.drill.exec.server.Drillbit - >>> > Shutdown completed (1914 ms). >>> > >>> > I did nothing and start it at next day, then it can startup. >>> > >>> > 2016-06-21 9:48 GMT+08:00 qiang li : >>> > >>> >> Hi Aman, >>> >> >>> >> I did not fully test with the old version. >>> >> >>> >> Cloud you please help me create the JIRA issue, I think my account >>> have >>> >> not the privilege, my account is griffinli and can not find the place >>> to >>> >> create new issue. Below is the explain detail for the same SQL in >>> different >>> >> nodes of cluster. >>> >> >>> >> >>> >> This is the correct plan which only have two nodes: >>> >> 0: jdbc:drill:zk=xxx:> explain plan for select >>> >> CONVERT_FROM(BYTE_SUBSTR(`ref0`.row_key,-8,8),'BIGINT_BE') as ui
Re: DRILL-4199: Add Support for HBase 1.X - planning to merge
Thanks Aman. I think the second is not related to hbase too. Its happened occasionally. I wil give you more detail when its happen again. Hi Aditya, there is no error in drillbit.out, but have error in drillbit.log, I have attached to the DRILL-4734 <https://issues.apache.org/jira/browse/DRILL-4734> , please check it . 2016-06-22 4:31 GMT+08:00 Aman Sinha : > Qiang, > The last issue mentioned about failure to start drillbit seems unrelated > to HBase isn't it ? For now, I am going with the assumption that it is not > a regression, so does not impact the 1.7 release. Of course, if you find > out otherwise, we would need to address it. > Regarding the SchemaChangeException you reported, I have created a JIRA: > DRILL-4734. > > > -Aman > > On Tue, Jun 21, 2016 at 10:04 AM, Aditya wrote: > >> There should be another stack trace in drillbit.out when this happens, >> could you please check that file. >> >> On Mon, Jun 20, 2016 at 8:14 PM, qiang li wrote: >> >> > Another issue is some time when I restart the node, the node can not be >> > startup. >> > >> > Here is the exception. >> > ache-drill-1.7.0/jars/drill-gis-1.7.0-SNAPSHOT.jar!/, >> > >> jar:file:/usr/lib/apache-drill-1.7.0/jars/drill-memory-base-1.7.0-SNAPSHOT.jar!/] >> > took 2800ms >> > 2016-06-20 19:10:18,313 [main] INFO >> o.a.d.e.s.s.PersistentStoreRegistry - >> > Using the configured PStoreProvider class: >> > >> 'org.apache.drill.exec.store.sys.store.provider.ZookeeperPersistentStoreProvider'. >> > 2016-06-20 19:10:19,221 [main] INFO >> o.apache.drill.exec.server.Drillbit - >> > Construction completed (1529 ms). >> > 2016-06-20 19:10:31,136 [main] WARN >> o.apache.drill.exec.server.Drillbit - >> > Failure on close() >> > java.lang.NullPointerException: null >> > at >> > org.apache.drill.exec.work.WorkManager.close(WorkManager.java:153) >> > ~[drill-java-exec-1.7.0-SNAPSHOT.jar:1.7.0-SNAPSHOT] >> > at >> > org.apache.drill.common.AutoCloseables.close(AutoCloseables.java:76) >> > ~[drill-common-1.7.0-SNAPSHOT.jar:1.7.0-SNAPSHOT] >> > at >> > org.apache.drill.common.AutoCloseables.close(AutoCloseables.java:64) >> > ~[drill-common-1.7.0-SNAPSHOT.jar:1.7.0-SNAPSHOT] >> > at >> org.apache.drill.exec.server.Drillbit.close(Drillbit.java:159) >> > [drill-java-exec-1.7.0-SNAPSHOT.jar:1.7.0-SNAPSHOT] >> > at >> org.apache.drill.exec.server.Drillbit.start(Drillbit.java:293) >> > [drill-java-exec-1.7.0-SNAPSHOT.jar:1.7.0-SNAPSHOT] >> > at >> org.apache.drill.exec.server.Drillbit.start(Drillbit.java:271) >> > [drill-java-exec-1.7.0-SNAPSHOT.jar:1.7.0-SNAPSHOT] >> > at org.apache.drill.exec.server.Drillbit.main(Drillbit.java:267) >> > [drill-java-exec-1.7.0-SNAPSHOT.jar:1.7.0-SNAPSHOT] >> > 2016-06-20 19:10:31,137 [main] INFO >> o.apache.drill.exec.server.Drillbit - >> > Shutdown completed (1914 ms). >> > >> > I did nothing and start it at next day, then it can startup. >> > >> > 2016-06-21 9:48 GMT+08:00 qiang li : >> > >> >> Hi Aman, >> >> >> >> I did not fully test with the old version. >> >> >> >> Cloud you please help me create the JIRA issue, I think my account >> have >> >> not the privilege, my account is griffinli and can not find the place >> to >> >> create new issue. Below is the explain detail for the same SQL in >> different >> >> nodes of cluster. >> >> >> >> >> >> This is the correct plan which only have two nodes: >> >> 0: jdbc:drill:zk=xxx:> explain plan for select >> >> CONVERT_FROM(BYTE_SUBSTR(`ref0`.row_key,-8,8),'BIGINT_BE') as uid, >> >> convert_from(`ref0`.`v`.`v`,'UTF8') as v from >> hbase.`offers_nation_idx` as >> >> `nation` join hbase.offers_ref0 as `ref0` on >> >> BYTE_SUBSTR(`ref0`.row_key,-8,8) = nation.`v`.`v` where >> `nation`.row_key >> >> > '0br' and `nation`.row_key < '0bs' limit 10; >> >> +--+--+ >> >> | text | json | >> >> +--+--+ >> >> | 00-00Screen >> >> 00-01 Project(uid=[$0], v=[$1]) >> >> 00-02SelectionVectorRemover >> >> 00-03 Limit(fetch=[10]) >> >> 00-04UnionExchange >> >> 01-01 SelectionVectorRemover >
Re: DRILL-4199: Add Support for HBase 1.X - planning to merge
Another issue is some time when I restart the node, the node can not be startup. Here is the exception. ache-drill-1.7.0/jars/drill-gis-1.7.0-SNAPSHOT.jar!/, jar:file:/usr/lib/apache-drill-1.7.0/jars/drill-memory-base-1.7.0-SNAPSHOT.jar!/] took 2800ms 2016-06-20 19:10:18,313 [main] INFO o.a.d.e.s.s.PersistentStoreRegistry - Using the configured PStoreProvider class: 'org.apache.drill.exec.store.sys.store.provider.ZookeeperPersistentStoreProvider'. 2016-06-20 19:10:19,221 [main] INFO o.apache.drill.exec.server.Drillbit - Construction completed (1529 ms). 2016-06-20 19:10:31,136 [main] WARN o.apache.drill.exec.server.Drillbit - Failure on close() java.lang.NullPointerException: null at org.apache.drill.exec.work.WorkManager.close(WorkManager.java:153) ~[drill-java-exec-1.7.0-SNAPSHOT.jar:1.7.0-SNAPSHOT] at org.apache.drill.common.AutoCloseables.close(AutoCloseables.java:76) ~[drill-common-1.7.0-SNAPSHOT.jar:1.7.0-SNAPSHOT] at org.apache.drill.common.AutoCloseables.close(AutoCloseables.java:64) ~[drill-common-1.7.0-SNAPSHOT.jar:1.7.0-SNAPSHOT] at org.apache.drill.exec.server.Drillbit.close(Drillbit.java:159) [drill-java-exec-1.7.0-SNAPSHOT.jar:1.7.0-SNAPSHOT] at org.apache.drill.exec.server.Drillbit.start(Drillbit.java:293) [drill-java-exec-1.7.0-SNAPSHOT.jar:1.7.0-SNAPSHOT] at org.apache.drill.exec.server.Drillbit.start(Drillbit.java:271) [drill-java-exec-1.7.0-SNAPSHOT.jar:1.7.0-SNAPSHOT] at org.apache.drill.exec.server.Drillbit.main(Drillbit.java:267) [drill-java-exec-1.7.0-SNAPSHOT.jar:1.7.0-SNAPSHOT] 2016-06-20 19:10:31,137 [main] INFO o.apache.drill.exec.server.Drillbit - Shutdown completed (1914 ms). I did nothing and start it at next day, then it can startup. 2016-06-21 9:48 GMT+08:00 qiang li : > Hi Aman, > > I did not fully test with the old version. > > Cloud you please help me create the JIRA issue, I think my account have > not the privilege, my account is griffinli and can not find the place to > create new issue. Below is the explain detail for the same SQL in different > nodes of cluster. > > > This is the correct plan which only have two nodes: > 0: jdbc:drill:zk=xxx:> explain plan for select > CONVERT_FROM(BYTE_SUBSTR(`ref0`.row_key,-8,8),'BIGINT_BE') as uid, > convert_from(`ref0`.`v`.`v`,'UTF8') as v from hbase.`offers_nation_idx` as > `nation` join hbase.offers_ref0 as `ref0` on > BYTE_SUBSTR(`ref0`.row_key,-8,8) = nation.`v`.`v` where `nation`.row_key > > '0br' and `nation`.row_key < '0bs' limit 10; > +--+--+ > | text | json | > +--+--+ > | 00-00Screen > 00-01 Project(uid=[$0], v=[$1]) > 00-02SelectionVectorRemover > 00-03 Limit(fetch=[10]) > 00-04UnionExchange > 01-01 SelectionVectorRemover > 01-02Limit(fetch=[10]) > 01-03 Project(uid=[CONVERT_FROMBIGINT_BE(BYTE_SUBSTR($3, > -8, 8))], v=[CONVERT_FROMUTF8(ITEM($4, 'v'))]) > 01-04Project(row_key=[$3], v=[$4], ITEM=[$5], > row_key0=[$0], v0=[$1], $f2=[$2]) > 01-05 HashJoin(condition=[=($2, $5)], > joinType=[inner]) > 01-07Project(row_key=[$0], v=[$1], > $f2=[BYTE_SUBSTR($0, -8, 8)]) > 01-09 Scan(groupscan=[HBaseGroupScan > [HBaseScanSpec=HBaseScanSpec [tableName=offers_ref0, startRow=null, > stopRow=null, filter=null], columns=[`*`]]]) > 01-06Project(row_key0=[$0], v0=[$1], ITEM=[$2]) > 01-08 *BroadcastExchange* > 02-01Project(row_key=[$0], v=[$1], > ITEM=[ITEM($1, 'v')]) > 02-02 Scan(groupscan=[HBaseGroupScan > [HBaseScanSpec=HBaseScanSpec [tableName=offers_nation_idx, > startRow=0br\x00, stopRow=0bs, filter=FilterList AND (2/2): [RowFilter > (GREATER, 0br), RowFilter (LESS, 0bs)]], columns=[`row_key`, `v`, > `v`.`v`]]]) > > > This is the plan that fails which have more than 5 nodes: > 0: jdbc:drill:zk=xxx:> explain plan for select > CONVERT_FROM(BYTE_SUBSTR(`ref0`.row_key,-8,8),'BIGINT_BE') as uid, > convert_from(`ref0`.`v`.`v`,'UTF8') as v from hbase.`offers_nation_idx` as > `nation` join hbase.offers_ref0 as `ref0` on > BYTE_SUBSTR(`ref0`.row_key,-8,8) = nation.`v`.`v` where `nation`.row_key > > '0br' and `nation`.row_key < '0bs' limit 10; > +--+--+ > | text | json | > +--+--+ > | 00-00Screen > 00-01 Project(uid=[$0], v=[$1]) > 00-02SelectionVectorRemover > 00-03 Limit(fetch=[10]) > 00-04UnionExchange > 01-01 SelectionVectorRemover > 01-02Limit(fet
Re: DRILL-4199: Add Support for HBase 1.X - planning to merge
f the hash join may be getting empty input > batches and I am guessing that has something to do with the > SchemaChangeException. Someone would need to debug once you create the > JIRA with relevant details. > > -Aman > > On Mon, Jun 20, 2016 at 5:13 AM, qiang li wrote: > > > Thanks Aditya. > > > > By the way, I found another issue. > > > > Let say I have two tables. > > > > offers_ref0 : rowkey salt(1byte)+long uid(8 byte ) , family: v, > qualifier: > > v(string) > > offers_nation_idx: rowkey salt(1byte) + string, family:v, qualifier: > v(long > > 8 byte) > > > > there is the SQL: > > > > select CONVERT_FROM(BYTE_SUBSTR(`ref0`.row_key,-8,8),'BIGINT_BE') as uid, > > convert_from(`ref0`.`v`.`v`,'UTF8') as v from hbase.`offers_nation_idx` > as > > `nation` join hbase.offers_ref0 as `ref0` on > > CONVERT_FROM(BYTE_SUBSTR(`ref0`.row_key,-8,8),'BIGINT_BE') = > > CONVERT_FROM(nation.`v`.`v`,'BIGINT_BE') where `nation`.row_key > '0br' > > and `nation`.row_key < '0bs' limit 10 > > > > When I execute the query with single node or less than 5 nodes, its > working > > good. But when I execute it in cluster which have about 14 nodes, its > throw > > a exception: > > > > First time will throw this exception: > > *Caused by: java.sql.SQLException: SYSTEM ERROR: SchemaChangeException: > > Hash join does not support schema changes* > > > > Then if I query again, it will always throw below exception: > > *Query Failed: An Error Occurred* > > *org.apache.drill.common.exceptions.UserRemoteException: SYSTEM ERROR: > > IllegalStateException: Failure while reading vector. Expected vector > class > > of org.apache.drill.exec.vector.NullableIntVector but was holding vector > > class org.apache.drill.exec.vector.complex.MapVector, field= > > v(MAP:REQUIRED)[v(VARBINARY:OPTIONAL)[$bits$(UINT1:REQUIRED), > > v(VARBINARY:OPTIONAL)[$offsets$(UINT4:REQUIRED)]]] Fragment 12:4 [Error > Id: > > 06c6eae4-0822-4714-b0bf-a6e04ebfec79 on xxx:31010]* > > > > Its very strange, and I do not know how to solve it. > > I tried add node to the cluster one by one, it will reproduce when I > added > > 5 nodes. Can anyone help me solve this issue? > > > > > > > > > > 2016-06-17 4:39 GMT+08:00 Aditya : > > > > > https://issues.apache.org/jira/browse/DRILL-4727 > > > > > > On Thu, Jun 16, 2016 at 11:39 AM, Aman Sinha > > wrote: > > > > > >> Qiang/Aditya can you create a JIRA for this and mark it for 1.7. > > thanks. > > >> > > >> On Thu, Jun 16, 2016 at 11:25 AM, Aditya > > wrote: > > >> > > >> > Thanks for reporting, I'm looking into it and will post a patch > soon. > > >> > > > >> > On Wed, Jun 15, 2016 at 7:27 PM, qiang li > > wrote: > > >> > > > >> > > Hi Aditya, > > >> > > > > >> > > I tested the latest version and got this exception and the > drillbit > > >> fail > > >> > > to startup . > > >> > > > > >> > > Exception in thread "main" java.lang.NoSuchMethodError: > > >> > > io.netty.util.UniqueName.(Ljava/lang/String;)V > > >> > > at > > >> io.netty.channel.ChannelOption.(ChannelOption.java:136) > > >> > > at > > >> io.netty.channel.ChannelOption.valueOf(ChannelOption.java:99) > > >> > > at > > >> io.netty.channel.ChannelOption.(ChannelOption.java:42) > > >> > > at > > >> > > org.apache.drill.exec.rpc.BasicServer.(BasicServer.java:63) > > >> > > at > > >> > > > org.apache.drill.exec.rpc.user.UserServer.(UserServer.java:74) > > >> > > at > > >> > > > > >> > > org.apache.drill.exec.service.ServiceEngine.(ServiceEngine.java:78) > > >> > > at > > >> > org.apache.drill.exec.server.Drillbit.(Drillbit.java:108) > > >> > > at > > >> org.apache.drill.exec.server.Drillbit.start(Drillbit.java:285) > > >> > > at > > >> org.apache.drill.exec.server.Drillbit.start(Drillbit.java:271) > > >> > > at > > >> org.apache.drill.exec.server.Drillbit.main(Drillbit.java:267) > > >> > > > > >> > > It will working if I remove > > jars/3rdparty/netty-all-4.0.23.Final.jar, > > >> the > > >> > > drill can startup. I think there have some package dependency > > version > > >> > > issue, do you think so ? > > >> > > > > >> > > > > >> > > > > >> > > 2016-06-15 8:14 GMT+08:00 Aditya : > > >> > > > > >> > >> HBase 1.x support has been merged and is available in latest > > >> > >> 1.7.0-SNAPSHOT > > >> > >> builds. > > >> > >> > > >> > >> On Wed, Jun 1, 2016 at 1:23 PM, Aditya > > >> wrote: > > >> > >> > > >> > >> > Thanks Jacques for promptly reviewing my long series of > patches! > > >> > >> > > > >> > >> > I'm planning to merge the HBase 1.x support some time in next > 48 > > >> > hours. > > >> > >> > > > >> > >> > If anyone else is interested and willing, please review the > > latest > > >> > patch > > >> > >> > here[1]. > > >> > >> > > > >> > >> > aditya... > > >> > >> > > > >> > >> > [1] https://github.com/apache/drill/pull/443/files > > >> > >> > > > >> > >> > > >> > > > > >> > > > > >> > > > >> > > > > > > > > >
Re: DRILL-4199: Add Support for HBase 1.X - planning to merge
Thanks Aditya. By the way, I found another issue. Let say I have two tables. offers_ref0 : rowkey salt(1byte)+long uid(8 byte ) , family: v, qualifier: v(string) offers_nation_idx: rowkey salt(1byte) + string, family:v, qualifier: v(long 8 byte) there is the SQL: select CONVERT_FROM(BYTE_SUBSTR(`ref0`.row_key,-8,8),'BIGINT_BE') as uid, convert_from(`ref0`.`v`.`v`,'UTF8') as v from hbase.`offers_nation_idx` as `nation` join hbase.offers_ref0 as `ref0` on CONVERT_FROM(BYTE_SUBSTR(`ref0`.row_key,-8,8),'BIGINT_BE') = CONVERT_FROM(nation.`v`.`v`,'BIGINT_BE') where `nation`.row_key > '0br' and `nation`.row_key < '0bs' limit 10 When I execute the query with single node or less than 5 nodes, its working good. But when I execute it in cluster which have about 14 nodes, its throw a exception: First time will throw this exception: *Caused by: java.sql.SQLException: SYSTEM ERROR: SchemaChangeException: Hash join does not support schema changes* Then if I query again, it will always throw below exception: *Query Failed: An Error Occurred* *org.apache.drill.common.exceptions.UserRemoteException: SYSTEM ERROR: IllegalStateException: Failure while reading vector. Expected vector class of org.apache.drill.exec.vector.NullableIntVector but was holding vector class org.apache.drill.exec.vector.complex.MapVector, field= v(MAP:REQUIRED)[v(VARBINARY:OPTIONAL)[$bits$(UINT1:REQUIRED), v(VARBINARY:OPTIONAL)[$offsets$(UINT4:REQUIRED)]]] Fragment 12:4 [Error Id: 06c6eae4-0822-4714-b0bf-a6e04ebfec79 on xxx:31010]* Its very strange, and I do not know how to solve it. I tried add node to the cluster one by one, it will reproduce when I added 5 nodes. Can anyone help me solve this issue? 2016-06-17 4:39 GMT+08:00 Aditya : > https://issues.apache.org/jira/browse/DRILL-4727 > > On Thu, Jun 16, 2016 at 11:39 AM, Aman Sinha wrote: > >> Qiang/Aditya can you create a JIRA for this and mark it for 1.7. thanks. >> >> On Thu, Jun 16, 2016 at 11:25 AM, Aditya wrote: >> >> > Thanks for reporting, I'm looking into it and will post a patch soon. >> > >> > On Wed, Jun 15, 2016 at 7:27 PM, qiang li wrote: >> > >> > > Hi Aditya, >> > > >> > > I tested the latest version and got this exception and the drillbit >> fail >> > > to startup . >> > > >> > > Exception in thread "main" java.lang.NoSuchMethodError: >> > > io.netty.util.UniqueName.(Ljava/lang/String;)V >> > > at >> io.netty.channel.ChannelOption.(ChannelOption.java:136) >> > > at >> io.netty.channel.ChannelOption.valueOf(ChannelOption.java:99) >> > > at >> io.netty.channel.ChannelOption.(ChannelOption.java:42) >> > > at >> > > org.apache.drill.exec.rpc.BasicServer.(BasicServer.java:63) >> > > at >> > > org.apache.drill.exec.rpc.user.UserServer.(UserServer.java:74) >> > > at >> > > >> org.apache.drill.exec.service.ServiceEngine.(ServiceEngine.java:78) >> > > at >> > org.apache.drill.exec.server.Drillbit.(Drillbit.java:108) >> > > at >> org.apache.drill.exec.server.Drillbit.start(Drillbit.java:285) >> > > at >> org.apache.drill.exec.server.Drillbit.start(Drillbit.java:271) >> > > at >> org.apache.drill.exec.server.Drillbit.main(Drillbit.java:267) >> > > >> > > It will working if I remove jars/3rdparty/netty-all-4.0.23.Final.jar, >> the >> > > drill can startup. I think there have some package dependency version >> > > issue, do you think so ? >> > > >> > > >> > > >> > > 2016-06-15 8:14 GMT+08:00 Aditya : >> > > >> > >> HBase 1.x support has been merged and is available in latest >> > >> 1.7.0-SNAPSHOT >> > >> builds. >> > >> >> > >> On Wed, Jun 1, 2016 at 1:23 PM, Aditya >> wrote: >> > >> >> > >> > Thanks Jacques for promptly reviewing my long series of patches! >> > >> > >> > >> > I'm planning to merge the HBase 1.x support some time in next 48 >> > hours. >> > >> > >> > >> > If anyone else is interested and willing, please review the latest >> > patch >> > >> > here[1]. >> > >> > >> > >> > aditya... >> > >> > >> > >> > [1] https://github.com/apache/drill/pull/443/files >> > >> > >> > >> >> > > >> > > >> > >> > >
Re: DRILL-4199: Add Support for HBase 1.X - planning to merge
Also when I query hbase, its throw exception : 2016-06-16 10:29:51,788 [289df0e3-05dc-f882-8c51-20dd0f98235b:frag:0:0] ERROR o.a.drill.exec.compile.MergeAdapter - Failure while merging classes. java.lang.RuntimeException: Error at instruction 370: Expected an object reference, but found . doEval(II)V at org.objectweb.asm.util.CheckMethodAdapter$1.visitEnd(CheckMethodAdapter.java:463) ~[asm-debug-all-5.0.3.jar:5.0.3] But the query is success. Does this exception will affect the query process, for example will it affect the query speed or data integrity? 2016-06-16 10:27 GMT+08:00 qiang li : > Hi Aditya, > > I tested the latest version and got this exception and the drillbit fail > to startup . > > Exception in thread "main" java.lang.NoSuchMethodError: > io.netty.util.UniqueName.(Ljava/lang/String;)V > at io.netty.channel.ChannelOption.(ChannelOption.java:136) > at io.netty.channel.ChannelOption.valueOf(ChannelOption.java:99) > at io.netty.channel.ChannelOption.(ChannelOption.java:42) > at > org.apache.drill.exec.rpc.BasicServer.(BasicServer.java:63) > at > org.apache.drill.exec.rpc.user.UserServer.(UserServer.java:74) > at > org.apache.drill.exec.service.ServiceEngine.(ServiceEngine.java:78) > at org.apache.drill.exec.server.Drillbit.(Drillbit.java:108) > at org.apache.drill.exec.server.Drillbit.start(Drillbit.java:285) > at org.apache.drill.exec.server.Drillbit.start(Drillbit.java:271) > at org.apache.drill.exec.server.Drillbit.main(Drillbit.java:267) > > It will working if I remove jars/3rdparty/netty-all-4.0.23.Final.jar, the > drill can startup. I think there have some package dependency version > issue, do you think so ? > > > > 2016-06-15 8:14 GMT+08:00 Aditya : > >> HBase 1.x support has been merged and is available in latest >> 1.7.0-SNAPSHOT >> builds. >> >> On Wed, Jun 1, 2016 at 1:23 PM, Aditya wrote: >> >> > Thanks Jacques for promptly reviewing my long series of patches! >> > >> > I'm planning to merge the HBase 1.x support some time in next 48 hours. >> > >> > If anyone else is interested and willing, please review the latest patch >> > here[1]. >> > >> > aditya... >> > >> > [1] https://github.com/apache/drill/pull/443/files >> > >> > >
Re: DRILL-4199: Add Support for HBase 1.X - planning to merge
Hi Aditya, I tested the latest version and got this exception and the drillbit fail to startup . Exception in thread "main" java.lang.NoSuchMethodError: io.netty.util.UniqueName.(Ljava/lang/String;)V at io.netty.channel.ChannelOption.(ChannelOption.java:136) at io.netty.channel.ChannelOption.valueOf(ChannelOption.java:99) at io.netty.channel.ChannelOption.(ChannelOption.java:42) at org.apache.drill.exec.rpc.BasicServer.(BasicServer.java:63) at org.apache.drill.exec.rpc.user.UserServer.(UserServer.java:74) at org.apache.drill.exec.service.ServiceEngine.(ServiceEngine.java:78) at org.apache.drill.exec.server.Drillbit.(Drillbit.java:108) at org.apache.drill.exec.server.Drillbit.start(Drillbit.java:285) at org.apache.drill.exec.server.Drillbit.start(Drillbit.java:271) at org.apache.drill.exec.server.Drillbit.main(Drillbit.java:267) It will working if I remove jars/3rdparty/netty-all-4.0.23.Final.jar, the drill can startup. I think there have some package dependency version issue, do you think so ? 2016-06-15 8:14 GMT+08:00 Aditya : > HBase 1.x support has been merged and is available in latest 1.7.0-SNAPSHOT > builds. > > On Wed, Jun 1, 2016 at 1:23 PM, Aditya wrote: > > > Thanks Jacques for promptly reviewing my long series of patches! > > > > I'm planning to merge the HBase 1.x support some time in next 48 hours. > > > > If anyone else is interested and willing, please review the latest patch > > here[1]. > > > > aditya... > > > > [1] https://github.com/apache/drill/pull/443/files > > >
drill hbase logical plan with group by
Dear, Please help me, recently I met a issue when I test drill with hbase. I tested two sqls, one return the wrong result and the other return correct result. the wrong sql: select CONVERT_FROM(a.`v`.`e0`, 'UTF8') as k, count(a.`v`.`e0`) p from hbase.browser_action2 a where a.row_key >'0' group by a.`v`.`e0`; 03-04 Scan(groupscan=[HBaseGroupScan [HBaseScanSpec=HBaseScanSpec [tableName=browser_action2, startRow=0\x00, stopRow=, filter=null], *columns=[`*`]*]]) the correct sql : select CONVERT_FROM(a.`v`.`e0`, 'UTF8') as k, count(a.`v`.`e0`) p from hbase.browser_action2 a group by a.`v`.`e0` can return the right result. 03-04 Scan(groupscan=[HBaseGroupScan [HBaseScanSpec=HBaseScanSpec [tableName=browser_action2, startRow=null, stopRow=null, filter=null], *columns=[`v`.`e0`]*]]) As you can see, the difference of the plan is the columns, the plan is not what I want when I have where clause. Can anyone help me as I do not know how to submit issue for drill at issues.apache.org ?