[jira] [Commented] (FLUME-1906) Ability to disable WAL for put operation in HBaseSink
[ https://issues.apache.org/jira/browse/FLUME-1906?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13573264#comment-13573264 ] Hari Shreedharan commented on FLUME-1906: - Doing this would void Flume's delivery guarantee no? (not exactly Flume's guarantee, but still the data would not be persisted to disk). > Ability to disable WAL for put operation in HBaseSink > - > > Key: FLUME-1906 > URL: https://issues.apache.org/jira/browse/FLUME-1906 > Project: Flume > Issue Type: Improvement > Components: Sinks+Sources >Affects Versions: v1.3.1 >Reporter: Mubarak Seyed > > HBase supports setWriteToWAL(boolean) in Put to write edits to WAL or not. > User can specify the disable option in HBase sink configurations. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
Re: Flume schedule
Hi Aline, Currently there is no discussion around the timing for Flume 1.4.0. Could you share your motivation behind asking for the release schedule? Regards, Arvind Prabhakar On Mon, Feb 4, 2013 at 9:10 AM, Aline Guedes wrote: > Hello, > > Is there a schedule for Flume available somewhere? I am interested in the > planned release date for Flume 1.4.0 (in case there is a planned date), but > I can't find it anywhere. > > Thanks! > Aline > >
[jira] [Created] (FLUME-1906) Ability to disable WAL for put operation in HBaseSink
Mubarak Seyed created FLUME-1906: Summary: Ability to disable WAL for put operation in HBaseSink Key: FLUME-1906 URL: https://issues.apache.org/jira/browse/FLUME-1906 Project: Flume Issue Type: Improvement Components: Sinks+Sources Affects Versions: v1.3.1 Reporter: Mubarak Seyed HBase supports setWriteToWAL(boolean) in Put to write edits to WAL or not. User can specify the disable option in HBase sink configurations. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (FLUME-1905) Implement atomic update operation (checkAndPut) in HBaseSink
Mubarak Seyed created FLUME-1905: Summary: Implement atomic update operation (checkAndPut) in HBaseSink Key: FLUME-1905 URL: https://issues.apache.org/jira/browse/FLUME-1905 Project: Flume Issue Type: Improvement Components: Sinks+Sources Affects Versions: v1.3.1 Reporter: Mubarak Seyed Flume can be used as a data freeway to stream data across datacenters and data update/insert can happen conditionally (checking some qualifier value). HBase client supports checkAndPut. Event header can have an expected qualifier:value (row/column_family can be obtained from row) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (FLUME-1898) Implement Thrift Source
[ https://issues.apache.org/jira/browse/FLUME-1898?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hari Shreedharan updated FLUME-1898: Attachment: FLUME-1898.patch Thrift Source - patch from rb > Implement Thrift Source > --- > > Key: FLUME-1898 > URL: https://issues.apache.org/jira/browse/FLUME-1898 > Project: Flume > Issue Type: Sub-task >Reporter: Hari Shreedharan >Assignee: Hari Shreedharan > Attachments: FLUME-1898.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
Review Request: FLUME-1898. Thrift Source
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/9352/ --- Review request for Flume. Description --- Thrift Source. This addresses bug FLUME-1898. https://issues.apache.org/jira/browse/FLUME-1898 Diffs - flume-ng-core/pom.xml 7db4f23 flume-ng-core/src/main/java/org/apache/flume/source/ThriftSource.java PRE-CREATION flume-ng-core/src/test/java/org/apache/flume/source/TestThriftSource.java PRE-CREATION Diff: https://reviews.apache.org/r/9352/diff/ Testing --- Added several unit tests. Thanks, Hari Shreedharan
[jira] [Updated] (FLUME-1896) Implement Thrift RpcClient
[ https://issues.apache.org/jira/browse/FLUME-1896?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hari Shreedharan updated FLUME-1896: Attachment: FLUME-1896-3.patch Updating the Rpc client to use TFastFramedTransport and TCompactProtocol > Implement Thrift RpcClient > -- > > Key: FLUME-1896 > URL: https://issues.apache.org/jira/browse/FLUME-1896 > Project: Flume > Issue Type: Sub-task >Reporter: Hari Shreedharan >Assignee: Hari Shreedharan > Attachments: FLUME-1896-1.patch, FLUME-1896-2.patch, > FLUME-1896-3.patch, FLUME-1896.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
Re: Review Request: FLUME-1896. Thrift Rpc Client.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/9284/ --- (Updated Feb. 7, 2013, 3:31 a.m.) Review request for Flume. Changes --- Use TFastFramedTransport and TCompactProtocol. Description --- Added thrift rpc client. Detailed description posted on jira. This addresses bug FLUME-1896. https://issues.apache.org/jira/browse/FLUME-1896 Diffs (updated) - flume-ng-legacy-sources/flume-thrift-source/pom.xml b9667cd flume-ng-legacy-sources/flume-thrift-source/src/main/java/com/cloudera/flume/handlers/thrift/EventStatus.java 327107a flume-ng-legacy-sources/flume-thrift-source/src/main/java/com/cloudera/flume/handlers/thrift/Priority.java d2495d2 flume-ng-legacy-sources/flume-thrift-source/src/main/java/com/cloudera/flume/handlers/thrift/ThriftFlumeEvent.java 2bb6cfd flume-ng-legacy-sources/flume-thrift-source/src/main/java/com/cloudera/flume/handlers/thrift/ThriftFlumeEventServer.java 0f2ad2d flume-ng-legacy-sources/flume-thrift-source/src/main/thrift/aslv2 PRE-CREATION flume-ng-sdk/pom.xml ab066d5 flume-ng-sdk/src/main/java/org/apache/flume/api/HostInfo.java 7388a45 flume-ng-sdk/src/main/java/org/apache/flume/api/RpcClientConfigurationConstants.java ab4c3de flume-ng-sdk/src/main/java/org/apache/flume/api/RpcClientFactory.java 3c93921 flume-ng-sdk/src/main/java/org/apache/flume/api/ThriftRpcClient.java PRE-CREATION flume-ng-sdk/src/main/java/org/apache/flume/thrift/Status.java PRE-CREATION flume-ng-sdk/src/main/java/org/apache/flume/thrift/ThriftFlumeEvent.java PRE-CREATION flume-ng-sdk/src/main/java/org/apache/flume/thrift/ThriftSourceProtocol.java PRE-CREATION flume-ng-sdk/src/main/thrift/aslv2 PRE-CREATION flume-ng-sdk/src/main/thrift/flume.thrift PRE-CREATION flume-ng-sdk/src/test/java/org/apache/flume/api/TestThriftRpcClient.java PRE-CREATION flume-ng-sdk/src/test/java/org/apache/flume/api/ThriftTestingSource.java PRE-CREATION flume-ng-sources/flume-scribe-source/pom.xml 588207d flume-ng-sources/flume-scribe-source/src/main/java/org/apache/flume/source/scribe/LogEntry.java 401a7e4 flume-ng-sources/flume-scribe-source/src/main/java/org/apache/flume/source/scribe/ResultCode.java 6bfa84c flume-ng-sources/flume-scribe-source/src/main/java/org/apache/flume/source/scribe/Scribe.java ccde51b flume-ng-sources/flume-scribe-source/src/main/java/org/apache/flume/source/scribe/ScribeSource.java 6190b11 flume-ng-sources/flume-scribe-source/src/main/thrift/aslv2 PRE-CREATION flume-ng-sources/flume-scribe-source/src/main/thrift/scribe-source.thrift PRE-CREATION pom.xml faa67f9 Diff: https://reviews.apache.org/r/9284/diff/ Testing --- Added unit tests for new code. Thanks, Hari Shreedharan
[jira] [Created] (FLUME-1904) Add 'message' parser in ElasticSearchLogStashEventSerializer.java
Frank Yao created FLUME-1904: Summary: Add 'message' parser in ElasticSearchLogStashEventSerializer.java Key: FLUME-1904 URL: https://issues.apache.org/jira/browse/FLUME-1904 Project: Flume Issue Type: New Feature Components: Sinks+Sources Affects Versions: v1.3.0 Reporter: Frank Yao Priority: Trivial in 'appendBody', before add "@message" field, I think it should check the headers if there is "message" headers just like below. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (FLUME-1903) Add factory methods for Thrift Rpc Client
[ https://issues.apache.org/jira/browse/FLUME-1903?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hari Shreedharan resolved FLUME-1903. - Resolution: Fixed Fixed in FLUME-1896 > Add factory methods for Thrift Rpc Client > - > > Key: FLUME-1903 > URL: https://issues.apache.org/jira/browse/FLUME-1903 > Project: Flume > Issue Type: Sub-task >Reporter: Hari Shreedharan >Assignee: Hari Shreedharan > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (FLUME-1896) Implement Thrift RpcClient
[ https://issues.apache.org/jira/browse/FLUME-1896?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hari Shreedharan updated FLUME-1896: Attachment: FLUME-1896-2.patch Added factory methods as it was interfering with testing the Thrift Source > Implement Thrift RpcClient > -- > > Key: FLUME-1896 > URL: https://issues.apache.org/jira/browse/FLUME-1896 > Project: Flume > Issue Type: Sub-task >Reporter: Hari Shreedharan >Assignee: Hari Shreedharan > Attachments: FLUME-1896-1.patch, FLUME-1896-2.patch, FLUME-1896.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
Re: Review Request: FLUME-1896. Thrift Rpc Client.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/9284/ --- (Updated Feb. 7, 2013, 2:01 a.m.) Review request for Flume. Changes --- Added Factory methods, removed an old file which was Thrift generated. Description --- Added thrift rpc client. Detailed description posted on jira. This addresses bug FLUME-1896. https://issues.apache.org/jira/browse/FLUME-1896 Diffs (updated) - flume-ng-legacy-sources/flume-thrift-source/pom.xml b9667cd flume-ng-legacy-sources/flume-thrift-source/src/main/java/com/cloudera/flume/handlers/thrift/EventStatus.java 327107a flume-ng-legacy-sources/flume-thrift-source/src/main/java/com/cloudera/flume/handlers/thrift/Priority.java d2495d2 flume-ng-legacy-sources/flume-thrift-source/src/main/java/com/cloudera/flume/handlers/thrift/ThriftFlumeEvent.java 2bb6cfd flume-ng-legacy-sources/flume-thrift-source/src/main/java/com/cloudera/flume/handlers/thrift/ThriftFlumeEventServer.java 0f2ad2d flume-ng-legacy-sources/flume-thrift-source/src/main/thrift/aslv2 PRE-CREATION flume-ng-sdk/pom.xml ab066d5 flume-ng-sdk/src/main/java/org/apache/flume/api/HostInfo.java 7388a45 flume-ng-sdk/src/main/java/org/apache/flume/api/RpcClientConfigurationConstants.java ab4c3de flume-ng-sdk/src/main/java/org/apache/flume/api/RpcClientFactory.java 3c93921 flume-ng-sdk/src/main/java/org/apache/flume/api/ThriftRpcClient.java PRE-CREATION flume-ng-sdk/src/main/java/org/apache/flume/thrift/Status.java PRE-CREATION flume-ng-sdk/src/main/java/org/apache/flume/thrift/ThriftFlumeEvent.java PRE-CREATION flume-ng-sdk/src/main/java/org/apache/flume/thrift/ThriftSourceProtocol.java PRE-CREATION flume-ng-sdk/src/main/thrift/aslv2 PRE-CREATION flume-ng-sdk/src/main/thrift/flume.thrift PRE-CREATION flume-ng-sdk/src/test/java/org/apache/flume/api/TestThriftRpcClient.java PRE-CREATION flume-ng-sdk/src/test/java/org/apache/flume/api/ThriftTestingSource.java PRE-CREATION flume-ng-sources/flume-scribe-source/pom.xml 588207d flume-ng-sources/flume-scribe-source/src/main/java/org/apache/flume/source/scribe/LogEntry.java 401a7e4 flume-ng-sources/flume-scribe-source/src/main/java/org/apache/flume/source/scribe/ResultCode.java 6bfa84c flume-ng-sources/flume-scribe-source/src/main/java/org/apache/flume/source/scribe/Scribe.java ccde51b flume-ng-sources/flume-scribe-source/src/main/java/org/apache/flume/source/scribe/ScribeSource.java 6190b11 flume-ng-sources/flume-scribe-source/src/main/thrift/aslv2 PRE-CREATION flume-ng-sources/flume-scribe-source/src/main/thrift/scribe-source.thrift PRE-CREATION pom.xml faa67f9 Diff: https://reviews.apache.org/r/9284/diff/ Testing --- Added unit tests for new code. Thanks, Hari Shreedharan
[jira] [Commented] (FLUME-1896) Implement Thrift RpcClient
[ https://issues.apache.org/jira/browse/FLUME-1896?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13573074#comment-13573074 ] Hari Shreedharan commented on FLUME-1896: - I filed FLUME-1903 to add methods in RpcClientFactory for thrift rpc client. > Implement Thrift RpcClient > -- > > Key: FLUME-1896 > URL: https://issues.apache.org/jira/browse/FLUME-1896 > Project: Flume > Issue Type: Sub-task >Reporter: Hari Shreedharan >Assignee: Hari Shreedharan > Attachments: FLUME-1896-1.patch, FLUME-1896.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (FLUME-1903) Add factory methods for Thrift Rpc Client
Hari Shreedharan created FLUME-1903: --- Summary: Add factory methods for Thrift Rpc Client Key: FLUME-1903 URL: https://issues.apache.org/jira/browse/FLUME-1903 Project: Flume Issue Type: Sub-task Reporter: Hari Shreedharan Assignee: Hari Shreedharan -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (FLUME-1896) Implement Thrift RpcClient
[ https://issues.apache.org/jira/browse/FLUME-1896?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13572837#comment-13572837 ] Hari Shreedharan commented on FLUME-1896: - I have verified that Thrift 0.6.1 is wire compatible with the Thrift legacy source after upgrading to 0.9 by writing an app that uses thrift 0.6.1 to write flume og style events to the thrift legacy source running thrift 0.9.0 - so we can go ahead with this upgrade. > Implement Thrift RpcClient > -- > > Key: FLUME-1896 > URL: https://issues.apache.org/jira/browse/FLUME-1896 > Project: Flume > Issue Type: Sub-task >Reporter: Hari Shreedharan >Assignee: Hari Shreedharan > Attachments: FLUME-1896-1.patch, FLUME-1896.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
