[GitHub] incubator-carbondata issue #650: [WIP] add intergation with presto

[ffpeng90]

Github user ffpeng90 commented on the issue:

https://github.com/apache/incubator-carbondata/pull/650
  
as your wish


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-carbondata issue #650: [WIP] add intergation with presto

[CarbonDataQA]

Github user CarbonDataQA commented on the issue:

https://github.com/apache/incubator-carbondata/pull/650
  
Build Success with Spark 1.6.2, Please check CI 
http://136.243.101.176:8080/job/ApacheCarbonPRBuilder/1290/



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-carbondata issue #650: [WIP] add intergation with presto

[chenliang613]

Github user chenliang613 commented on the issue:

https://github.com/apache/incubator-carbondata/pull/650
  
@ffpeng90  please update the PR title also.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-carbondata issue #650: [WIP] add intergation with presto

[ffpeng90]

Github user ffpeng90 commented on the issue:

https://github.com/apache/incubator-carbondata/pull/650
  
I'm focusing on two things.
1. let user can debug presto-carbondata in his IDE.
2. use new presto API to support lazy decode.
They will be ok soon.






At 2017-03-15 10:52:01, "å½­"  wrote:

Hi:
1. This version only suppport DML,  
All tables for test are created by spark-sql(DML part), 
and i submit queries to presto to get results.
I only tested the "Select" Case , like where, group , sum , join.


2.  I use APIs like createQueryPlan, resolveFilter from class 
"CarbonInputFormatUtil". 
   To read carbon formatted table, i make the read process into several 
steps:
   a). load table metadata 
   b). get splits from table (pushing down filtering to filter 
datablocks of one segment @CarbonTableReader.getInputSplits2 )
   c). parse records ( pushing down column projection and filtering 
into QueryModel  @CarbondataRecordSetProvider.getRecordSet ) 


3. As i described  in partC "parse records", I use QueryModel to get  
decoded records.
   For lazy decoding,  I will keep on exploring a better solution.  
Maybe we can get inspiration from module presto-orc, presto-parquet.
  
 
  
   






At 2017-03-15 09:11:19, "Jacky Li"  wrote:


Thanks for working on this. Can you describe what feature is added in term 
of:

What SQL syntax is supported? DDL ?
I think it uses CarbonInputFormat to read, so are you pushing down column 
projection and filtering by setting the configuration in CarbonInputFormat?
Is there any SQL optimization integration with Presto's optimizer? like 
leveraging carbon's global dictionary to do lazy decode?

—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub, or mute the thread.





 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-carbondata issue #650: [WIP] add intergation with presto

[ffpeng90]

Github user ffpeng90 commented on the issue:

https://github.com/apache/incubator-carbondata/pull/650
  
Hi:
1. This version only suppport DML,  
All tables for test are created by spark-sql(DML part), 
and i submit queries to presto to get results.
I only tested the "Select" Case , like where, group , sum , join.


2.  I use APIs like createQueryPlan, resolveFilter from class 
"CarbonInputFormatUtil". 
   To read carbon formatted table, i make the read process into several 
steps:
   a). load table metadata 
   b). get splits from table (pushing down filtering to filter 
datablocks of one segment @CarbonTableReader.getInputSplits2 )
   c). parse records ( pushing down column projection and filtering 
into QueryModel  @CarbondataRecordSetProvider.getRecordSet ) 


3. As i described  in partC "parse records", I use QueryModel to get  
decoded records.
   For lazy decoding,  I will keep on exploring a better solution.  
Maybe we can get inspiration from module presto-orc, presto-parquet.
  
 
  
   






At 2017-03-15 09:11:19, "Jacky Li"  wrote:


Thanks for working on this. Can you describe what feature is added in term 
of:

What SQL syntax is supported? DDL ?
I think it uses CarbonInputFormat to read, so are you pushing down column 
projection and filtering by setting the configuration in CarbonInputFormat?
Is there any SQL optimization integration with Presto's optimizer? like 
leveraging carbon's global dictionary to do lazy decode?

—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub, or mute the thread.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-carbondata issue #650: [WIP] add intergation with presto

[jackylk]

Github user jackylk commented on the issue:

https://github.com/apache/incubator-carbondata/pull/650
  
And I think it is easier to review and can be merged sooner if you could 
break this PR down into smaller one. Just provide the very basic functionality 
in the first round of the integration. You can add more functionality in 
subsequent PR.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-carbondata issue #650: [WIP] add intergation with presto

[jackylk]

Github user jackylk commented on the issue:

https://github.com/apache/incubator-carbondata/pull/650
  
Thanks for working on this. Can you describe what feature is added in term 
of:
1. What SQL syntax is supported? DDL ?
2. I think it uses CarbonInputFormat to read, so are you pushing down 
column projection and filtering by setting the configuration in 
CarbonInputFormat?
3. Is there any SQL optimization integration with Presto's optimizer? like 
leveraging carbon's global dictionary to do lazy decode?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-carbondata issue #650: [WIP] add intergation with presto

[CarbonDataQA]

Github user CarbonDataQA commented on the issue:

https://github.com/apache/incubator-carbondata/pull/650
  
Build Failed  with Spark 1.6.2, Please check CI 
http://136.243.101.176:8080/job/ApacheCarbonPRBuilder/1100/



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-carbondata issue #650: [WIP] add intergation with presto

[chenliang613]

Github user chenliang613 commented on the issue:

https://github.com/apache/incubator-carbondata/pull/650
  
add to whitelist


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---