Re: select * clause still case all regionserver crash

[hongbin ma]

Hi, are you "qstar" from
https://issues.apache.org/jira/browse/KYLIN-1936?focusedCommentId=15727422&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15727422
? Just want to make sure whether it's a individual case or common case


On Thu, Dec 8, 2016 at 10:00 AM, alaleiwang  wrote:

> about v1.5.4,"select * from table limit N" clause seems not to crash
> regionserver,but meantime no result return for the clause
>
> --
> View this message in context: http://apache-kylin.74782.x6.
> nabble.com/select-clause-still-cause-all-regionserver-
> crash-tp6474p6535.html
> Sent from the Apache Kylin mailing list archive at Nabble.com.
>



-- 
Regards,

*Bin Mahone | 马洪宾*

Re: select * clause still case all regionserver crash

[alaleiwang]

about v1.5.4,"select * from table limit N" clause seems not to crash
regionserver,but meantime no result return for the clause

--
View this message in context: 
http://apache-kylin.74782.x6.nabble.com/select-clause-still-cause-all-regionserver-crash-tp6474p6535.html
Sent from the Apache Kylin mailing list archive at Nabble.com.

Re: select * clause still case all regionserver crash

[alaleiwang]

i know it not recommend...

as i said above,some user are not technical;they just click from tableau or
write some test query clause,
while not understanding simple operation will do serious thing,we cannot 
avoid such cases at all

so i still insist on doing something form kylin for  guarantee of such
unrecommened "select *" clause with or without "limit",to make regionserver
running more stable on production enviroment

--
View this message in context: 
http://apache-kylin.74782.x6.nabble.com/select-clause-still-cause-all-regionserver-crash-tp6474p6494.html
Sent from the Apache Kylin mailing list archive at Nabble.com.

Re: select * clause still case all regionserver crash

[roger shi]

It’s not recommend to select * without limit. Query server cannot handle too 
many records got from storage.

On 05/12/2016, 10:25 AM, "alaleiwang"  wrote:

thanks a lot,i will  check KYLIN-1936 and try v1.5.4 to find if it will 
solve
our problem by use "select * from table limit N"

i am still afraid the case the user will use "select *" clause even without
"limit",how can we deal with this one?


--
View this message in context: 
http://apache-kylin.74782.x6.nabble.com/select-clause-still-cause-all-regionserver-crash-tp6474p6489.html
Sent from the Apache Kylin mailing list archive at Nabble.com.

Re: select * clause still case all regionserver crash

[roger shi]

According to hongbin’s comments in JIRA KYLIN-1936, it fixed limit push down 
issue when dealing with multiple segments.

On 04/12/2016, 7:01 PM, "Alberto Ramón"  wrote:

about KYLIN-1936: what make this change ?
   (keep in mind groups by? keep in mind differents regions ? what is the
new limit ?)


Thanks

2016-12-04 9:39 GMT+01:00 roger shi :

> The limit push down issue is completely resolved in JIRA KYLIN-1936, which
> is applied in 1.5.4. So please try kylin 1.5.4.
>
> On 02/12/2016, 5:51 PM, "ShaoFeng Shi"  wrote:
>
> I remember hongbin has further optimization on the limit push down
> after
> 1.5.3; @hongbin, can you confirm this?
>
> 2016-12-02 17:02 GMT+08:00 alaleiwang :
>
> > i can see each regionserver scan over 2000+
> >
> > the query "select *"  is called from tableau used by non-technical
> > analyst,so we can't force them to bypass such clause  or to do
> something
> > from tableau ...
> >
> > how can "limit" work for "select *" without "where"?
> >
> >
> >
> >
> > --
> > View this message in context: http://apache-kylin.74782.x6.
> > nabble.com/select-clause-still-cause-all-regionserver-
> > crash-tp6474p6481.html
> > Sent from the Apache Kylin mailing list archive at Nabble.com.
> >
>
>
>
> --
> Best regards,
>
> Shaofeng Shi 史少锋
>
>
>

Re: select * clause still case all regionserver crash

[35925138]

sorry, it's my mistake , I click the "load hive table from tree" button , it's 
always empty, 
but when I click "load hive table" button and input the table name ,it's ok, 
and when I click the "load hive table from tree" button again,It's ok too








-- Original --
From:  "alaleiwang";;
Date:  Mon, Dec 5, 2016 10:25 AM
To:  "dev"; 

Subject:  Re: select * clause still case all regionserver crash



thanks a lot,i will  check KYLIN-1936 and try v1.5.4 to find if it will solve
our problem by use "select * from table limit N"

i am still afraid the case the user will use "select *" clause even without
"limit",how can we deal with this one?


--
View this message in context: 
http://apache-kylin.74782.x6.nabble.com/select-clause-still-cause-all-regionserver-crash-tp6474p6489.html
Sent from the Apache Kylin mailing list archive at Nabble.com.

Re: select * clause still case all regionserver crash

[alaleiwang]

thanks a lot,i will  check KYLIN-1936 and try v1.5.4 to find if it will solve
our problem by use "select * from table limit N"

i am still afraid the case the user will use "select *" clause even without
"limit",how can we deal with this one?


--
View this message in context: 
http://apache-kylin.74782.x6.nabble.com/select-clause-still-cause-all-regionserver-crash-tp6474p6489.html
Sent from the Apache Kylin mailing list archive at Nabble.com.

Re: select * clause still case all regionserver crash

[Alberto Ramón]

about KYLIN-1936: what make this change ?
   (keep in mind groups by? keep in mind differents regions ? what is the
new limit ?)


Thanks

2016-12-04 9:39 GMT+01:00 roger shi :

> The limit push down issue is completely resolved in JIRA KYLIN-1936, which
> is applied in 1.5.4. So please try kylin 1.5.4.
>
> On 02/12/2016, 5:51 PM, "ShaoFeng Shi"  wrote:
>
> I remember hongbin has further optimization on the limit push down
> after
> 1.5.3; @hongbin, can you confirm this?
>
> 2016-12-02 17:02 GMT+08:00 alaleiwang :
>
> > i can see each regionserver scan over 2000+
> >
> > the query "select *"  is called from tableau used by non-technical
> > analyst,so we can't force them to bypass such clause  or to do
> something
> > from tableau ...
> >
> > how can "limit" work for "select *" without "where"?
> >
> >
> >
> >
> > --
> > View this message in context: http://apache-kylin.74782.x6.
> > nabble.com/select-clause-still-cause-all-regionserver-
> > crash-tp6474p6481.html
> > Sent from the Apache Kylin mailing list archive at Nabble.com.
> >
>
>
>
> --
> Best regards,
>
> Shaofeng Shi 史少锋
>
>
>

Re: select * clause still case all regionserver crash

[roger shi]

The limit push down issue is completely resolved in JIRA KYLIN-1936, which is 
applied in 1.5.4. So please try kylin 1.5.4.

On 02/12/2016, 5:51 PM, "ShaoFeng Shi"  wrote:

I remember hongbin has further optimization on the limit push down after
1.5.3; @hongbin, can you confirm this?

2016-12-02 17:02 GMT+08:00 alaleiwang :

> i can see each regionserver scan over 2000+
>
> the query "select *"  is called from tableau used by non-technical
> analyst,so we can't force them to bypass such clause  or to do something
> from tableau ...
>
> how can "limit" work for "select *" without "where"?
>
>
>
>
> --
> View this message in context: http://apache-kylin.74782.x6.
> nabble.com/select-clause-still-cause-all-regionserver-
> crash-tp6474p6481.html
> Sent from the Apache Kylin mailing list archive at Nabble.com.
>



-- 
Best regards,

Shaofeng Shi 史少锋

Re: select * clause still case all regionserver crash

[Alberto Ramón]

:)  We have some similiar problem with non-IT
"Why do you want a result table of 1 Millon of rows ¿? "

I don't Know any solution.
The Q is: this is a problem/bug/issue of Kylin ? I think NO, the target of
this is protect to coprocesor (is mandatory)

Select * is not addecuate for Kylin
Impala, Hive, Cassandra, presto, ... Is the best solution for this: View
raw data



2016-12-02 10:02 GMT+01:00 alaleiwang :

> i can see each regionserver scan over 2000+
>
> the query "select *"  is called from tableau used by non-technical
> analyst,so we can't force them to bypass such clause  or to do something
> from tableau ...
>
> how can "limit" work for "select *" without "where"?
>
>
>
>
> --
> View this message in context: http://apache-kylin.74782.x6.
> nabble.com/select-clause-still-cause-all-regionserver-
> crash-tp6474p6481.html
> Sent from the Apache Kylin mailing list archive at Nabble.com.
>

Re: select * clause still case all regionserver crash

[ShaoFeng Shi]

I remember hongbin has further optimization on the limit push down after
1.5.3; @hongbin, can you confirm this?

2016-12-02 17:02 GMT+08:00 alaleiwang :

> i can see each regionserver scan over 2000+
>
> the query "select *"  is called from tableau used by non-technical
> analyst,so we can't force them to bypass such clause  or to do something
> from tableau ...
>
> how can "limit" work for "select *" without "where"?
>
>
>
>
> --
> View this message in context: http://apache-kylin.74782.x6.
> nabble.com/select-clause-still-cause-all-regionserver-
> crash-tp6474p6481.html
> Sent from the Apache Kylin mailing list archive at Nabble.com.
>



-- 
Best regards,

Shaofeng Shi 史少锋

Re: select * clause still case all regionserver crash

[alaleiwang]

i can see each regionserver scan over 2000+

the query "select *"  is called from tableau used by non-technical
analyst,so we can't force them to bypass such clause  or to do something
from tableau ...

how can "limit" work for "select *" without "where"?




--
View this message in context: 
http://apache-kylin.74782.x6.nabble.com/select-clause-still-cause-all-regionserver-crash-tp6474p6481.html
Sent from the Apache Kylin mailing list archive at Nabble.com.

Re: select * clause still case all regionserver crash

[Alberto Ramón]

About select * from   :Kylin 1792
 v1.5.3

Can you check if you see this error: "*Scan row count exceeded threshold:
100, please add filter condition to narrow down backend scan range,
like where clause*"

With Queries like this:

SELECT * FROM TB

There are some limits:

- Read more 1M rows per RS

- Read more 100MB per RD

- Total result of Kylin 3.0 GB (more or less)

- Time limit to solve querie . (I don't remember if it was 10 seconds)


*As resume:* Kylin, can't read Billon of rows, and compute its. Because is
a OLAP Cube and we espect solve queries very very fast (This is the reason
of existence of a Precalculates Cubes)


You must add where condition to Select column from TB Where ...  to make a
predicate pushdown on queries




2016-12-02 3:23 GMT+01:00 alaleiwang :

> KYLIN-1787  say fixed in v1.5.3,but it still happened from v1.5.1 to v1.5.3
>
> --
> View this message in context: http://apache-kylin.74782.x6.
> nabble.com/select-clause-still-cause-all-regionserver-
> crash-tp6474p6476.html
> Sent from the Apache Kylin mailing list archive at Nabble.com.
>

Re: select * clause still case all regionserver crash

[alaleiwang]

KYLIN-1787  say fixed in v1.5.3,but it still happened from v1.5.1 to v1.5.3

--
View this message in context: 
http://apache-kylin.74782.x6.nabble.com/select-clause-still-cause-all-regionserver-crash-tp6474p6476.html
Sent from the Apache Kylin mailing list archive at Nabble.com.

Re: select * clause still case all regionserver crash

[Alberto Ramón]

Sounds like:
KYLIN-1787 <https://issues.apache.org/jira/browse/KYLIN-1787>

MailList
<http://mail-archives.apache.org/mod_mbox/kylin-user/201608.mbox/%3CCALjEW7M_YYi7Xs55OqPdxS6pzNvD0%2BamN2AX3hetnF0%3D9uFnow%40mail.gmail.com%3E>
MailList
<http://apache-kylin.74782.x6.nabble.com/Exceed-scan-threshold-at-1001-td6101.html>


2016-12-01 14:04 GMT+01:00 alaleiwang :

> hi:
>i ask the question at
> ：http://apache-kylin.74782.x6.nabble.com/some-question-
> about-setMaxResultSize-for-scanner-CubeHBaseScanRPC-
> CubeHBaseEndpointRPC-td4983.html#a5007
>
>https://issues.apache.org/jira/browse/KYLIN-1787 say it solved the
> problem
>
>but i am still now suffering from the same thing,all regionserver(180+)
> crashed from time to time
> by the clause "select * from table",and limit does not help
>
>the relate cube size is about 3.88TB,and my regionserver memory sumed up
> to 2.04T
>
>this happend from kylin 1.5.1 and kylin 1.5.3,i don't test on kylin
> 1.5.4
>
>more to be added:
>select * from tablename where aa=bb  limit 10 do work,and will not crash
> regionserver
>
>
>
> --
> View this message in context: http://apache-kylin.74782.x6.
> nabble.com/select-clause-still-case-all-regionserver-crash-tp6474.html
> Sent from the Apache Kylin mailing list archive at Nabble.com.
>

select * clause still case all regionserver crash

[alaleiwang]

hi:
   i ask the question at
：http://apache-kylin.74782.x6.nabble.com/some-question-about-setMaxResultSize-for-scanner-CubeHBaseScanRPC-CubeHBaseEndpointRPC-td4983.html#a5007
   
   https://issues.apache.org/jira/browse/KYLIN-1787 say it solved the
problem

   but i am still now suffering from the same thing,all regionserver(180+)
crashed from time to time 
by the clause "select * from table",and limit does not help

   the relate cube size is about 3.88TB,and my regionserver memory sumed up
to 2.04T

   this happend from kylin 1.5.1 and kylin 1.5.3,i don't test on kylin 1.5.4
   
   more to be added:
   select * from tablename where aa=bb  limit 10 do work,and will not crash
regionserver
   


--
View this message in context: 
http://apache-kylin.74782.x6.nabble.com/select-clause-still-case-all-regionserver-crash-tp6474.html
Sent from the Apache Kylin mailing list archive at Nabble.com.