[jira] [Commented] (HAWQ-786) Framework to support pluggable formats and file systems
[ https://issues.apache.org/jira/browse/HAWQ-786?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16344191#comment-16344191 ] Yi Jin commented on HAWQ-786: - As its demo and feature test will not be delivered in version 2.3.0.0, this issue is extended to next version to complete in the future. > Framework to support pluggable formats and file systems > --- > > Key: HAWQ-786 > URL: https://issues.apache.org/jira/browse/HAWQ-786 > Project: Apache HAWQ > Issue Type: Wish > Components: Storage >Reporter: Lei Chang >Assignee: Chiyang Wan >Priority: Major > Fix For: 2.4.0.0-incubating > > Attachments: HAWQ Pluggable Storage Framework.pdf, > ORCdesign-v0.1-2016-06-17.pdf > > > In current HAWQ, two native formats are supported: AO and parquet. Now we > want to support ORC. A framework to support naive c/c++ pluggable format is > needed to support ORC more easily. And it can also be potentially used for > fast external data access. > And there are a lot of requests for supporting S3, Ceph and other file > systems, and this is closely related to pluggable formats, so this JIRA is > proposing a framework to support both. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HAWQ-786) Framework to support pluggable formats and file systems
[ https://issues.apache.org/jira/browse/HAWQ-786?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16249054#comment-16249054 ] Ruilong Huo commented on HAWQ-786: -- Assigning to Chiyang so that he can work on this feature. > Framework to support pluggable formats and file systems > --- > > Key: HAWQ-786 > URL: https://issues.apache.org/jira/browse/HAWQ-786 > Project: Apache HAWQ > Issue Type: Wish > Components: Storage >Reporter: Lei Chang >Assignee: chiyang wan > Fix For: 2.3.0.0-incubating > > Attachments: ORCdesign-v0.1-2016-06-17.pdf > > > In current HAWQ, two native formats are supported: AO and parquet. Now we > want to support ORC. A framework to support naive c/c++ pluggable format is > needed to support ORC more easily. And it can also be potentially used for > fast external data access. > And there are a lot of requests for supporting S3, Ceph and other file > systems, and this is closely related to pluggable formats, so this JIRA is > proposing a framework to support both. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HAWQ-786) Framework to support pluggable formats and file systems
[ https://issues.apache.org/jira/browse/HAWQ-786?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16246939#comment-16246939 ] Ruilong Huo commented on HAWQ-786: -- [~chiyang1], sounds a good idea! > Framework to support pluggable formats and file systems > --- > > Key: HAWQ-786 > URL: https://issues.apache.org/jira/browse/HAWQ-786 > Project: Apache HAWQ > Issue Type: Wish > Components: Storage >Reporter: Lei Chang >Assignee: Ruilong Huo > Fix For: 2.3.0.0-incubating > > Attachments: ORCdesign-v0.1-2016-06-17.pdf > > > In current HAWQ, two native formats are supported: AO and parquet. Now we > want to support ORC. A framework to support naive c/c++ pluggable format is > needed to support ORC more easily. And it can also be potentially used for > fast external data access. > And there are a lot of requests for supporting S3, Ceph and other file > systems, and this is closely related to pluggable formats, so this JIRA is > proposing a framework to support both. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HAWQ-786) Framework to support pluggable formats and file systems
[ https://issues.apache.org/jira/browse/HAWQ-786?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16246934#comment-16246934 ] chiyang wan commented on HAWQ-786: -- Hello, I would like to take this feature development. > Framework to support pluggable formats and file systems > --- > > Key: HAWQ-786 > URL: https://issues.apache.org/jira/browse/HAWQ-786 > Project: Apache HAWQ > Issue Type: Wish > Components: Storage >Reporter: Lei Chang >Assignee: Ruilong Huo > Fix For: 2.3.0.0-incubating > > Attachments: ORCdesign-v0.1-2016-06-17.pdf > > > In current HAWQ, two native formats are supported: AO and parquet. Now we > want to support ORC. A framework to support naive c/c++ pluggable format is > needed to support ORC more easily. And it can also be potentially used for > fast external data access. > And there are a lot of requests for supporting S3, Ceph and other file > systems, and this is closely related to pluggable formats, so this JIRA is > proposing a framework to support both. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HAWQ-786) Framework to support pluggable formats and file systems
[ https://issues.apache.org/jira/browse/HAWQ-786?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16152365#comment-16152365 ] Yi Jin commented on HAWQ-786: - Now let's assign Ruilong to complete this feature. Thanks. > Framework to support pluggable formats and file systems > --- > > Key: HAWQ-786 > URL: https://issues.apache.org/jira/browse/HAWQ-786 > Project: Apache HAWQ > Issue Type: Wish > Components: Storage >Reporter: Lei Chang >Assignee: Ruilong Huo > Fix For: 2.3.0.0-incubating > > Attachments: ORCdesign-v0.1-2016-06-17.pdf > > > In current HAWQ, two native formats are supported: AO and parquet. Now we > want to support ORC. A framework to support naive c/c++ pluggable format is > needed to support ORC more easily. And it can also be potentially used for > fast external data access. > And there are a lot of requests for supporting S3, Ceph and other file > systems, and this is closely related to pluggable formats, so this JIRA is > proposing a framework to support both. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HAWQ-786) Framework to support pluggable formats and file systems
[ https://issues.apache.org/jira/browse/HAWQ-786?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16103037#comment-16103037 ] Lei Chang commented on HAWQ-786: [~rlei] This feature has been implemented in Oushu commercial hawq version. currently we are refactoring the code and making it work on oss hawq. After it is ready, we will update this JIRA. > Framework to support pluggable formats and file systems > --- > > Key: HAWQ-786 > URL: https://issues.apache.org/jira/browse/HAWQ-786 > Project: Apache HAWQ > Issue Type: Wish > Components: Storage >Reporter: Lei Chang >Assignee: Lei Chang > Fix For: backlog > > Attachments: ORCdesign-v0.1-2016-06-17.pdf > > > In current HAWQ, two native formats are supported: AO and parquet. Now we > want to support ORC. A framework to support naive c/c++ pluggable format is > needed to support ORC more easily. And it can also be potentially used for > fast external data access. > And there are a lot of requests for supporting S3, Ceph and other file > systems, and this is closely related to pluggable formats, so this JIRA is > proposing a framework to support both. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HAWQ-786) Framework to support pluggable formats and file systems
[ https://issues.apache.org/jira/browse/HAWQ-786?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16102879#comment-16102879 ] Radar Lei commented on HAWQ-786: Hi [~lei_chang], glad to see this feature is scheduled in next release. We are starting to do some spike on Parquet 2.0 support, which is related with the pluggable storage feature your are working on. So we want to know more details about the interfaces between pluggable service and the parquet file format, this can help a lot to our spike. As the design doc in this jira is for ORC only and might be out of date, would you update the design doc to include pluggable storage? Thanks a lot. > Framework to support pluggable formats and file systems > --- > > Key: HAWQ-786 > URL: https://issues.apache.org/jira/browse/HAWQ-786 > Project: Apache HAWQ > Issue Type: Wish > Components: Storage >Reporter: Lei Chang >Assignee: Lei Chang > Fix For: backlog > > Attachments: ORCdesign-v0.1-2016-06-17.pdf > > > In current HAWQ, two native formats are supported: AO and parquet. Now we > want to support ORC. A framework to support naive c/c++ pluggable format is > needed to support ORC more easily. And it can also be potentially used for > fast external data access. > And there are a lot of requests for supporting S3, Ceph and other file > systems, and this is closely related to pluggable formats, so this JIRA is > proposing a framework to support both. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HAWQ-786) Framework to support pluggable formats and file systems
[ https://issues.apache.org/jira/browse/HAWQ-786?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16065952#comment-16065952 ] Lei Chang commented on HAWQ-786: This feature has been not active for some time. A lot of hawq users are asking this feature. currently, we are working on this feature. I am assigning this issue to me. Hope that we can release this feature in next release. > Framework to support pluggable formats and file systems > --- > > Key: HAWQ-786 > URL: https://issues.apache.org/jira/browse/HAWQ-786 > Project: Apache HAWQ > Issue Type: Wish > Components: Storage >Reporter: Lei Chang >Assignee: hongwu > Fix For: backlog > > Attachments: ORCdesign-v0.1-2016-06-17.pdf > > > In current HAWQ, two native formats are supported: AO and parquet. Now we > want to support ORC. A framework to support naive c/c++ pluggable format is > needed to support ORC more easily. And it can also be potentially used for > fast external data access. > And there are a lot of requests for supporting S3, Ceph and other file > systems, and this is closely related to pluggable formats, so this JIRA is > proposing a framework to support both. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
