[Disscussion] Support High concurrency queries for hot data

[haomarch]

carbondata meets the performance requirements of point queries and complex
analysis in the same time. Before carbondata, users can only use hbase to
meet the requirement of point query, while using MPP or HiveonORC to run
complex analysis. With carbondata, they can use carbondata instead of hbase
and hiveonorc, which brings cost reduction.
But because of the low concurrency of carbondata, This benefit can only be
achieved on the low－concurrency scenario, like maintenance，safety check. For
the online scenario and high-concurrency offline scenario, carbondata can't
do anything to help, which makes carbondata can't be applied to the more 
valuable production env.

There is a suggestion:
We shall support high concurrency key-based queries(maybe 1000~1 tps)
for the hot data(Recent one
month data). and low concurrency queries for the cold data(data of one month
ago).
Some design like HBase's RegionServer shall be used for reference;



--
Sent from: 
http://apache-carbondata-dev-mailing-list-archive.1130556.n5.nabble.com/

Re: [Dissussion] Support FLOAT datatype in the CDC Flow

[David CaiQiang]

please check another topic:
http://apache-carbondata-dev-mailing-list-archive.1130556.n5.nabble.com/Discussion-Float-and-Double-compatibility-issue-with-external-segments-to-Carbon-td93870.html.

if this is an issue,  you can create an issue in carbondata jira.




-
Best Regards
David Cai
--
Sent from: 
http://apache-carbondata-dev-mailing-list-archive.1130556.n5.nabble.com/

[Dissussion] Support FLOAT datatype in the CDC Flow

[haomarch]

We don't support FLOAT datatype in the CDC Flow. This is a big issue.


 



--
Sent from: 
http://apache-carbondata-dev-mailing-list-archive.1130556.n5.nabble.com/

Re: [Discussion]Float and Double compatibility issue with external segments to Carbon

[Jacky Li]

Hi,

Yes, I think we should correct it.
In the schema, it should be float for float type. In internal store, it is 
usinng adaptive encoding, so I think it is ok anyway.

Regards,
Jacky

> 2020年5月8日 上午10:23，David CaiQiang  写道：
> 
> It is a historical legacy issue and easy to reuse the solution of the double
> data type.
> 
> Suggest implementing the float data type independently.
> 
> 
> 
> 
> -
> Best Regards
> David Cai
> --
> Sent from: 
> http://apache-carbondata-dev-mailing-list-archive.1130556.n5.nabble.com/
>