Re: Want to contribute

[phalodi]

Thanks Jean, Yes its project :)



--
View this message in context: 
http://apache-carbondata-mailing-list-archive.1130556.n5.nabble.com/Want-to-contribute-tp7614p7680.html
Sent from the Apache CarbonData Mailing List archive mailing list archive at 
Nabble.com.

Re: Want to contribute

[phalodi]

Thanks vishal, so i am interested in data querying part but yeah i also want
to look how the data will be compressed while loading.



--
View this message in context: 
http://apache-carbondata-mailing-list-archive.1130556.n5.nabble.com/Want-to-contribute-tp7614p7679.html
Sent from the Apache CarbonData Mailing List archive mailing list archive at 
Nabble.com.

is NOT NULL and NOT NULL filters on measures which is in dictionary column

[phalodi]

Hello in bug CARBONDATA-688 the NOT NULL and is NOT NULL filter is not giving
correct result so the case is when we use the measure in dictionary include
and run query with null filters then it will not give expected results. In
IncludeFilterExecuterImpl.java will give us the blank biset in
isScanRequired so for Null filters where is will create the BlockInfo.
Because right now the filter expressions are generated correct in
FilterUtils.java



--
View this message in context: 
http://apache-carbondata-mailing-list-archive.1130556.n5.nabble.com/is-NOT-NULL-and-NOT-NULL-filters-on-measures-which-is-in-dictionary-column-tp7678.html
Sent from the Apache CarbonData Mailing List archive mailing list archive at 
Nabble.com.

Re: question about the order between original values and its encoded values

[Ravindra Pesala]

Hi,
 Yes, it works because we are sorting the column values before assigning
dictionary values to it. So it can work only if you have loaded the data
only once( it means there is no incremental load). If you do incremental
load and some more dictionary values are added to store then there is no
guarantee that you get sorted result on encoded data.

Regards,
Ravindra.

On 16 February 2017 at 15:46, Ma Yun 马云  wrote:

> Hi dev team,
>
> One question about the dictionary encode,
> As you know, the original values of a dimension column will be encoded as
> integer and stored to carbon file ordered by the encoded values.
> I have done some test of order by dimension query in my local machine. I
> changed some code to use the encoded values to sort first, then decode to
> original values.
> The query results are correct. It seems the encoded values has the same
> order of the original values.
> But in the carbondata it always decode to original value first, then
> order by the  original values.
>
> Could you help to tell me which scenarios has the different order between
> the original values and the encoded values?
> BTW is there any document to explain the dictionary encode algorithm?
>
> Thanks
>
> Ma, yun
> 邮件免责申明- 该电子邮件中的信息是保密的，除收件人外任何人无权访问此电子邮件。
> 如果您不是收件人，公开、复制、分发或基于此封邮件的任何行动，都是禁止的，并可能是违法的。该邮件包含的任何意见与建议均应遵循上汽集团关于信息传递与保密
> 的制度或规定。除经上汽集团信函以正式书面方式确认外，任何相关的内容或信息不得作为正式依据。 Email Disclaimer- The
> information in this email is confidential and may be legally privileged. It
> is intended solely for the addressee. Access to this email by anyone else
> is unauthorized. If you are not the intended recipient, any disclosure,
> copying, distribution or any action taken or omitted to be taken in
> reliance on it, is prohibited and may be unlawful. Any opinions or advice
> contained in this email are subject to the terms and conditions expressed
> in the governing SAICMOTOR client engagement letter and should not be
> relied upon unless they are confirmed in writing on SAICMOTOR's letterhead.
>



-- 
Thanks & Regards,
Ravi

question about the order between original values and its encoded values

[Ma Yun 马云]

Hi dev team,

One question about the dictionary encode,
As you know, the original values of a dimension column will be encoded as 
integer and stored to carbon file ordered by the encoded values.
I have done some test of order by dimension query in my local machine. I 
changed some code to use the encoded values to sort first, then decode to 
original values.
The query results are correct. It seems the encoded values has the same order 
of the original values.
But in the carbondata it always decode to original value first, then  order by 
the  original values.

Could you help to tell me which scenarios has the different order between the 
original values and the encoded values?
BTW is there any document to explain the dictionary encode algorithm?

Thanks

Ma, yun
邮件免责申明- 该电子邮件中的信息是保密的，除收件人外任何人无权访问此电子邮件。 
如果您不是收件人，公开、复制、分发或基于此封邮件的任何行动，都是禁止的，并可能是违法的。该邮件包含的任何意见与建议均应遵循上汽集团关于信息传递与保密的制度或规定。除经上汽集团信函以正式书面方式确认外，任何相关的内容或信息不得作为正式依据。
 Email Disclaimer- The information in this email is confidential and may be 
legally privileged. It is intended solely for the addressee. Access to this 
email by anyone else is unauthorized. If you are not the intended recipient, 
any disclosure, copying, distribution or any action taken or omitted to be 
taken in reliance on it, is prohibited and may be unlawful. Any opinions or 
advice contained in this email are subject to the terms and conditions 
expressed in the governing SAICMOTOR client engagement letter and should not be 
relied upon unless they are confirmed in writing on SAICMOTOR's letterhead.

[jira] [Created] (CARBONDATA-710) Add content to Faqs and Troubleshooting

[Pallavi Singh (JIRA)]

Pallavi Singh created CARBONDATA-710:


 Summary: Add content to Faqs and Troubleshooting
 Key: CARBONDATA-710
 URL: https://issues.apache.org/jira/browse/CARBONDATA-710
 Project: CarbonData
  Issue Type: Improvement
  Components: docs
Reporter: Pallavi Singh
Assignee: Pallavi Singh






--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[GitHub] incubator-carbondata-site pull request #16: Resolved Image not displaying is...

[PallaviSingh1992]

Github user PallaviSingh1992 closed the pull request at:

https://github.com/apache/incubator-carbondata-site/pull/16


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-carbondata-site issue #16: Resolved Image not displaying issue in ...

[PallaviSingh1992]

Github user PallaviSingh1992 commented on the issue:

https://github.com/apache/incubator-carbondata-site/pull/16
  
Updated the changes in new PR


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-carbondata-site pull request #17: Updated the meetup page,links, d...

[PallaviSingh1992]

GitHub user PallaviSingh1992 opened a pull request:

https://github.com/apache/incubator-carbondata-site/pull/17

Updated the meetup page,links, ddl file for bucketing and fixed the image 
issues and added sitemap.html



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/PallaviSingh1992/incubator-carbondata-site 
feature/MinorIssuesFixed

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/incubator-carbondata-site/pull/17.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #17


commit 9c16863666b5664a27e6e901ed77ba5a988335b7
Author: PallaviSingh1992 
Date:   2017-02-16T05:55:36Z

updated the meetup page,links,image issues and ddl file for bucketing query

added sitemap.html page in the website




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Re: whether carbondata can be used in hive on spark?

[Ravindra Pesala]

Hi,

We have so far integrated only to the Spark, not yet integrated to Hive. So
carbondata cannot be used in Hive on Spark at this moment.

Regards,
Ravindra.

On 16 February 2017 at 14:35, wangzheng <18031...@qq.com> wrote:

> we use cdh5.7, it remove the thriftserver of spark, so sparksql is not
> suitable for us.
>
>
>
> --
> View this message in context: http://apache-carbondata-
> mailing-list-archive.1130556.n5.nabble.com/whether-
> carbondata-can-be-used-in-hive-on-spark-tp7661.html
> Sent from the Apache CarbonData Mailing List archive mailing list archive
> at Nabble.com.
>



-- 
Thanks & Regards,
Ravi

Re: A question about sort in carbon

[Cao Lu 曹鲁]

Yes, I also think so. Thanks for the confirmation! :)


On 2/16/17, 5:24 PM, "Kumar Vishal"  wrote:

>Hi Lionel,
>In case of order by it will convert the dictionary value to actual value
>then it will sort the data as dictionary values can not give data in sorted
>order because  for example vishal=2 and for Lionel=3 but in query Lionel
>will come first then vishal, so carbon will sort based on actual value.
>
>*File name:*
>org.apache.spark.sql.CarbonDictionaryDecoder.scala
>
>
>
>On Thu, Feb 16, 2017 at 4:44 PM, Lu Cao  wrote:
>
>> Hi dev team,
>> I have a question about the sort in carbon data.
>> When we have following query:
>> select country, area, name, salary from table_a order by country;
>> It seems carbon will decode the country column from dictionary value to
>> original value first, and then sort by original value.
>>
>> My question : Is the dictionary value order always the same with original
>> value order?
>> Or if we sort the dictionary value first and then decode to original value,
>> would that be correct operation?
>>
>> BTW: where can I see the algorithm of Dictionary encode(class name or file
>> name)?
>>
>> Thanks,
>> Lionel
>>
邮件免责申明- 该电子邮件中的信息是保密的，除收件人外任何人无权访问此电子邮件。 
如果您不是收件人，公开、复制、分发或基于此封邮件的任何行动，都是禁止的，并可能是违法的。该邮件包含的任何意见与建议均应遵循上汽集团关于信息传递与保密的制度或规定。除经上汽集团信函以正式书面方式确认外，任何相关的内容或信息不得作为正式依据。
 Email Disclaimer- The information in this email is confidential and may be 
legally privileged. It is intended solely for the addressee. Access to this 
email by anyone else is unauthorized. If you are not the intended recipient, 
any disclosure, copying, distribution or any action taken or omitted to be 
taken in reliance on it, is prohibited and may be unlawful. Any opinions or 
advice contained in this email are subject to the terms and conditions 
expressed in the governing SAICMOTOR client engagement letter and should not be 
relied upon unless they are confirmed in writing on SAICMOTOR's letterhead.

Re: A question about sort in carbon

[Kumar Vishal]

Hi Lionel,
In case of order by it will convert the dictionary value to actual value
then it will sort the data as dictionary values can not give data in sorted
order because  for example vishal=2 and for Lionel=3 but in query Lionel
will come first then vishal, so carbon will sort based on actual value.

*File name:*
org.apache.spark.sql.CarbonDictionaryDecoder.scala



On Thu, Feb 16, 2017 at 4:44 PM, Lu Cao  wrote:

> Hi dev team,
> I have a question about the sort in carbon data.
> When we have following query:
> select country, area, name, salary from table_a order by country;
> It seems carbon will decode the country column from dictionary value to
> original value first, and then sort by original value.
>
> My question : Is the dictionary value order always the same with original
> value order?
> Or if we sort the dictionary value first and then decode to original value,
> would that be correct operation?
>
> BTW: where can I see the algorithm of Dictionary encode(class name or file
> name)?
>
> Thanks,
> Lionel
>

whether carbondata can be used in hive on spark?

[wangzheng]

we use cdh5.7, it remove the thriftserver of spark, so sparksql is not
suitable for us.



--
View this message in context: 
http://apache-carbondata-mailing-list-archive.1130556.n5.nabble.com/whether-carbondata-can-be-used-in-hive-on-spark-tp7661.html
Sent from the Apache CarbonData Mailing List archive mailing list archive at 
Nabble.com.

A question about sort in carbon

[Lu Cao]

Hi dev team,
I have a question about the sort in carbon data.
When we have following query:
select country, area, name, salary from table_a order by country;
It seems carbon will decode the country column from dictionary value to
original value first, and then sort by original value.

My question : Is the dictionary value order always the same with original
value order?
Or if we sort the dictionary value first and then decode to original value,
would that be correct operation?

BTW: where can I see the algorithm of Dictionary encode(class name or file
name)?

Thanks,
Lionel

Re: 回复： data lost when loading data from csv file to carbon table

[Ravindra Pesala]

Hi QiangCai,

PR594 fix does not solve the data lost issue, it fixes the data mismatch in
some cases.

Regards,
Ravindra.

On 16 February 2017 at 09:35, QiangCai  wrote:

> Maybe you can check PR594, it will fix a bug which will impact the result
> of
> loading.
>
>
>
> --
> View this message in context: http://apache-carbondata-
> mailing-list-archive.1130556.n5.nabble.com/data-lost-when-
> loading-data-from-csv-file-to-carbon-table-tp7554p7639.html
> Sent from the Apache CarbonData Mailing List archive mailing list archive
> at Nabble.com.
>



-- 
Thanks & Regards,
Ravi