of specific improvements may give good performance in
>some specific type of queries but these will give long term complications
>in maintainability.
>
>
>Regards,
>Ravindra.
>
>On 30 March 2017 at 08:23, 马云 <simafengyun1...@163.com> wrote:
>
>> Hi Ravindran,
>
ter sort performance as spark
>try to do sorting inside memory.
>
>I can see we can do some optimizations in integration layer itself with out
>pushing down any logic to carbon like if the order by column is first
>column then we can just get limit values with out sorting any data.
>
&
Maybe I am wrong.
At 2017-03-29 02:21:58, "Ravindra Pesala" <ravi.pes...@gmail.com> wrote:
Hi Jarck Ma,
It is great to try optimizing Carbondata.
I think this solution comes up with many limitations. What if the order by
column is not the first column? It needs to scan all bl
Hi Carbon Dev,
I create table according to the below SQL
cc.sql("""
CREATE TABLE IF NOT EXISTS t3
(ID Int,
date Timestamp,
country String,
name String,
phonetype String,
serialname String,
salary Int,
name1 String,
name2 String,
name3 String,
name4
Hi Carbon Dev,
currently I have done optimization for ordering by 1 dimension.
my local performance test as below. Please give your suggestion.
my optimization solution for order by 1 dimension as below
mainly filter some unnecessary blocklets and leverage the dimension's order
stored
please ignore the email.
my mistake, the mail is not finished.
I will sent a new mail later
At 2017-03-28 13:24:34, "马云" <simafengyun1...@163.com> wrote:
>Hi Dev,
>
>
>currently I have done an optimization for order by 1 dimension.
>performance tes
Hi Dev,
currently I have done an optimization for order by 1 dimension.
performance test as below
my optimization solution for order by 1 dimension as below
mainly leverage the dimension's order stored feature in each blocklet
step1. change logical plan and push down the order by and limit
Hi DEV,
I create table according to the below SQL
cc.sql("""
CREATE TABLE IF NOT EXISTS t3
(ID Int,
date Timestamp,
country String,
name String,
phonetype String,
serialname String,
salary Int,
name1 String,
name2 String,
name3 String,
name4 String,
please ignore my issue.
I change JDK from 1.8 to 1.7 and add the below, it runs successfully now.
-Xmx3550m -Xms3550m -XX:MaxPermSize=512m
At 2017-03-08 17:20:58, "马云" <simafengyun1...@163.com> wrote:
Hi dev,
today I start setup carbon data 1.0 in my local ec
Hi dev,
today I start setup carbon data 1.0 in my local eclipse
I use "-X -DskipTests -Pspark-1.6 -Dspark.version=1.6.2 clean package" to do
maven build in eclipse successfully.
but when I run the CarbonExample in eclipse, it shows the below issue(refer to
the below log).
Even I configure
nverted index
>or not. If user has selected No inverted index while creating a column this
>code is fine, if user has not selected then data will be sorted so we can
>add binary search which will improve the performance.
>
>Please raise a Jira for this improvement
>
>-Regard
Hi Dev,
I used carbondata version 0.2 in my local machine, and found that the "between
and" filter query is very slow.
the root caused is by the below code in IncludeFilterExecuterImpl.java. It
takes about 20s in my test.
The code's time complexity is O(n*m). I think it needs to optimized,
Hi dev team,
One question about the dictionary encode,
As you know, the original values of a dimension column will be encoded as
integer and stored to carbon file ordered by the encoded values.
I have done some test of order by dimension query in my local machine. I
changed some code to use the
13 matches
Mail list logo
