Re: [DISCUSSION] Apache CarbonData 1.3.0 scope

[Liang Chen]

Hi yuhai

I have same comment as Jacky,please provide more info about this
requirement.
It would be better if you could create a new topic to detailedly discuss
this requirement.

Regards
Liang

Jacky Li wrote
> Hi Cenyuhai,
> 
> Can you further describe your requirement? Currently carbon supports read
> by hive already
> 
> Regards,
> Jacky
> 
>> 在 2017年10月10日，下午8:51，cenyuhai11 <

> cenyuhai11@

> > 写道：
>> 
>> Hi, Liang:
>>Can I add one feature:
>>6) Hive integration (bring carbondata to data warehouse)
>> 
>> 
>> --
>> Sent from:
>> http://apache-carbondata-dev-mailing-list-archive.1130556.n5.nabble.com/





--
Sent from: 
http://apache-carbondata-dev-mailing-list-archive.1130556.n5.nabble.com/

Re: [DISCUSSION] Apache CarbonData 1.3.0 scope

[Jacky Li]

Hi Cenyuhai,

Can you further describe your requirement? Currently carbon supports read by 
hive already

Regards,
Jacky

> 在 2017年10月10日，下午8:51，cenyuhai11  写道：
> 
> Hi, Liang:
>Can I add one feature:
>6) Hive integration (bring carbondata to data warehouse)
> 
> 
> --
> Sent from: 
> http://apache-carbondata-dev-mailing-list-archive.1130556.n5.nabble.com/

Re: [DISCUSSION] support user specified segment reading for query

[Rahul Kumar]

 Thanks @Raghunandan for updating the use case.

As i have analyzed UnresolvedHint is supported with Spark-2.2. So we can
support Query-Hint once spark2.2 integration is completed. Jira(
CARBONDATA-1546 ) is
already raised for QueryHint.

*@dev* What if we keep both SET query and Query HINT implementation for
segment reading?


For now as we have some use cases for session level and thread level. We
can implement SET query as part of CARBONDATA-1398
.


@Lione *'show segments' *is already implemented . It may cause
incompatiblity to existing user of carbondata if we change.I suggest we can
make '*For Table*' optional part in separate jira.


  Thanks and Regards

*   Rahul Kumar *



On Mon, Oct 9, 2017 at 8:21 AM, Lu Cao  wrote:

> Hi Rahul,
> I agree query hint is better. The syntax from Ravi is the same with Oracle,
> which is more popular and acceptable I think...
> Show Segments statement could remove 'For table', just 'show segments
> [dbname].'. It's simple and same style with show partition
> statement...
>
> Thanks,
> Lionel Cao
>
> On Mon, Oct 9, 2017 at 8:54 AM, Raghunandan S <
> carbondatacontributi...@gmail.com> wrote:
>
> > Query hint is a better option.
> > Some down side in using this approach:
> > User will have to modify all his query to accommodate this.
> > This won't be session scope but rather query scope
> > On Mon, 9 Oct 2017 at 2:36 AM, jarray  wrote:
> >
> > > same opinion with @Ravindra，use query hint is better approach ，the
> syntax
> > > is easy to use
> > >
> > >
> > >
> > >
> > >
> > >
> > > On 10/05/2017 17:10, Rahul Kumar wrote:
> > > @Ravindra Thanks. Using query hint may be better approach.
> > >
> > > I suggest following syntax :
> > >
> > > *select * from t1 [in SEGMENTS(1,3,5)]; *
> > >
> > >  Thanks and Regards
> > >
> > > *   Rahul Kumar *
> > >
> > >
> > >
> > > On Thu, Oct 5, 2017 at 1:04 PM, Ravindra Pesala  >
> > > wrote:
> > >
> > > > Hi,
> > > >
> > > > Instead of using SET command to use for segments why don't you use
> > QUERY
> > > > HINT . Using query hint we can mention the segments inside the query
> > > itself
> > > > as a hint.
> > > >
> > > > For example  SELECT /*+SEGMENTS(1,3,5) */ from t1.
> > > >
> > > > By using the above custom hint we can query from selected segments
> > only,
> > > > This concept is supported in Spark also and this concept will be
> > helpful
> > > in
> > > > our any future optimizations
> > > >
> > > > Regards,
> > > > Ravindra.
> > > >
> > > > On 5 October 2017 at 12:22, Rahul Kumar 
> > wrote:
> > > >
> > > > > @Jacky please find the reply of your doubts as follow :
> > > > >
> > > > >
> > > > > *1.If user uses following command in two different beeline
> > session,
> > > > > will there be problem due to multithreading?   SET
> > > > > carbon.input.segments.default.*
> > > > >
> > > > > *carbontable=1,3,5;   select * from carbontable;   SET
> > > > > carbon.input.segments.default.**carbontable=*;*
> > > > >
> > > > > *Ans: *In case of multithreading ,yes there will be problem.
> > > > >
> > > > >   So threadSet() can be use to set the same property in
> > multithread
> > > > > mode.
> > > > > *  Folowing syntax can be used to set segment ids for
> multithread
> > > > mode*
> > > > > :
> > > > >Syntax : CarbonSession.threadSet(“carbon.input.segments.<
> > > > > databese_name>.”,” > > > > of segment ids>”)
> > > > >e.g =>*future{*
> > > > >
> > > > > * CarbonSession.threadSet(“**carbon.input.segments.
> > > > > default.carbontable”,”1,3,5”)*
> > > > >
> > > > > * sparkSession.sql(“select * from carbontable”).show*
> > > > >
> > > > > * CarbonSession.threadSet(“carbon.input.segments.
> > > > > default.carbontable”,”*”)*
> > > > >
> > > > > * }*
> > > > >
> > > > > *Above will override the property at thread level. So property will
> > be
> > > > set
> > > > > for each thread .*
> > > > >
> > > > >
> > > > > *2.   The RESET command is not clear, why this is needed? It seems
> > SET
> > > > > carbon.input.segments.default.**carbontable=* is enough, right?
> and
> > > what
> > > > > parameter it has?*
> > > > >
> > > > > *Ans:* RESET command doesn't take any parameter. RESET is already
> > > > > implemented behavior which resets all the properties to their
> default
> > > > > value.So simillarly RESET query will set the above property also to
> > its
> > > > > default value.
> > > > >
> > > > >   Thanks and Regards
> > > > >
> > > > > *   Rahul Kumar *
> > > > >
> > > > >
> > > > >
> > > > > On Wed, Oct 4, 2017 at 7:21 PM, Jacky Li 
> wrote:
> > > > >
> > > > > > I have 2 doubts:
> > > > > > 1. If user uses following command in two different beeline
> session,
> > > > will
> > > > > > there be problem due to multithreading?
> > > > > > SET carbon.input.segments.default.carbontable=1,3,5;
> > > > > > select * from carbontable;
> > > > > > SET carbon.input.s

Re: [DISCUSSION] Support only spark 2 in carbon 1.3.0

[Bhavya Aggarwal]

+1

Regards
Bhavya

On Tue, Oct 10, 2017 at 7:15 PM, jarray  wrote:

> +1
>
>
>
>
>
>
> On 10/10/2017 14:13, gururajshetty wrote:
> + 1
>
> Regards,
> Gururaj
>
>
>
>
> --
> Sent from: http://apache-carbondata-dev-mailing-list-archive.1130556.
> n5.nabble.com/
>

Re: [DISCUSSION] Support only spark 2 in carbon 1.3.0

[jarray]

+1






On 10/10/2017 14:13, gururajshetty wrote:
+ 1

Regards,
Gururaj




--
Sent from: 
http://apache-carbondata-dev-mailing-list-archive.1130556.n5.nabble.com/

Re: [DISCUSSION] Apache CarbonData 1.3.0 scope

[cenyuhai11]

Hi, Liang:
Can I add one feature:
6) Hive integration (bring carbondata to data warehouse)



--
Sent from: 
http://apache-carbondata-dev-mailing-list-archive.1130556.n5.nabble.com/