[jira] [Commented] (FLUME-3216) When I rename the file, it causes flume to read the data repeatedly.
[ https://issues.apache.org/jira/browse/FLUME-3216?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16349776#comment-16349776 ] ASF GitHub Bot commented on FLUME-3216: --- GitHub user highfei2011 opened a pull request: https://github.com/apache/flume/pull/196 FLUME-3216.Avoid reading content repeatedly. When I use Taildir source, whenever I rename a file, it causes flume to read the data repeatedly.This resulted in duplication of uploaded data. You can merge this pull request into a Git repository by running: $ git pull https://github.com/highfei2011/flume trunk Alternatively you can review and apply these changes as the patch at: https://github.com/apache/flume/pull/196.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #196 commit db7996ca0e287aa78d68e460ee9287b731f0e5f1 Author: yangjifei Date: 2018-02-02T04:32:40Z FLUME-3216.Avoid reading content repeatedly. When I use Taildir source, whenever I rename a file, it causes flume to read the data repeatedly.This resulted in duplication of uploaded data. commit 574db14d91cdc10c6333ccfdf20c543d64303abc Author: yangjifei Date: 2018-02-02T04:38:02Z FLUME-3216.Avoid reading content repeatedly. When I use Taildir source, whenever I rename a file, it causes flume to read the data repeatedly.This resulted in duplication of uploaded data. > When I rename the file, it causes flume to read the data repeatedly. > > > Key: FLUME-3216 > URL: https://issues.apache.org/jira/browse/FLUME-3216 > Project: Flume > Issue Type: Improvement > Components: Sinks+Sources >Affects Versions: 1.8.0 > Environment: centos-6.8 >Reporter: jifei_yang >Priority: Critical > Fix For: 1.7.0, 1.8.0 > > > When I use Taildir source, whenever I rename a file, it causes flume to read > the data repeatedly.This resulted in duplication of uploaded data. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-unsubscr...@flume.apache.org For additional commands, e-mail: issues-h...@flume.apache.org
[jira] [Created] (FLUME-3216) When I rename the file, it causes flume to read the data repeatedly.
jifei_yang created FLUME-3216: - Summary: When I rename the file, it causes flume to read the data repeatedly. Key: FLUME-3216 URL: https://issues.apache.org/jira/browse/FLUME-3216 Project: Flume Issue Type: Improvement Components: Sinks+Sources Affects Versions: 1.8.0 Environment: centos-6.8 Reporter: jifei_yang Fix For: 1.8.0, 1.7.0 When I use Taildir source, whenever I rename a file, it causes flume to read the data repeatedly.This resulted in duplication of uploaded data. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-unsubscr...@flume.apache.org For additional commands, e-mail: issues-h...@flume.apache.org
[jira] [Commented] (FLUME-3021) flume Elasticsearch 5.0 not support
[ https://issues.apache.org/jira/browse/FLUME-3021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16348607#comment-16348607 ] Helmut Wahrman commented on FLUME-3021: --- Sorry, i have no idea, what you want to tell me with your last post. You have data in your ES index with the Patch. So everything is ok. Now you have another problem with not seeing any logs, but this is NOT related to the patch at all. > flume Elasticsearch 5.0 not support > --- > > Key: FLUME-3021 > URL: https://issues.apache.org/jira/browse/FLUME-3021 > Project: Flume > Issue Type: New Feature > Components: Sinks+Sources >Affects Versions: 1.7.0 >Reporter: tycho_yang >Assignee: Yonghao Zou >Priority: Major > Fix For: 1.9.0 > > Attachments: FLUME-3021-0.patch, FLUME-3021-1.patch, > FLUME-3021-Enable-ElasticSearch-6.1.x-support.patch, > FLUME-3021-Enable-ElasticSearch-6.1.x-support.patch, > image-2018-01-29-13-53-59-412.png, image-2018-01-31-12-00-06-854.png, > image-2018-01-31-12-00-50-050.png, image-2018-01-31-12-01-55-481.png, > image-2018-02-01-17-42-37-816.png, image-2018-02-01-17-44-24-082.png, > image-2018-02-01-17-45-54-668.png, image-2018-02-01-18-13-05-753.png, > log4j2.xml, screenshot-1.png, screenshot-2.png, screenshot-3.png > > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-unsubscr...@flume.apache.org For additional commands, e-mail: issues-h...@flume.apache.org
[jira] [Commented] (FLUME-3021) flume Elasticsearch 5.0 not support
[ https://issues.apache.org/jira/browse/FLUME-3021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16348343#comment-16348343 ] tycho_yang commented on FLUME-3021: --- [^log4j2.xml] > flume Elasticsearch 5.0 not support > --- > > Key: FLUME-3021 > URL: https://issues.apache.org/jira/browse/FLUME-3021 > Project: Flume > Issue Type: New Feature > Components: Sinks+Sources >Affects Versions: 1.7.0 >Reporter: tycho_yang >Assignee: Yonghao Zou >Priority: Major > Fix For: 1.9.0 > > Attachments: FLUME-3021-0.patch, FLUME-3021-1.patch, > FLUME-3021-Enable-ElasticSearch-6.1.x-support.patch, > FLUME-3021-Enable-ElasticSearch-6.1.x-support.patch, > image-2018-01-29-13-53-59-412.png, image-2018-01-31-12-00-06-854.png, > image-2018-01-31-12-00-50-050.png, image-2018-01-31-12-01-55-481.png, > image-2018-02-01-17-42-37-816.png, image-2018-02-01-17-44-24-082.png, > image-2018-02-01-17-45-54-668.png, image-2018-02-01-18-13-05-753.png, > log4j2.xml, screenshot-1.png, screenshot-2.png, screenshot-3.png > > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-unsubscr...@flume.apache.org For additional commands, e-mail: issues-h...@flume.apache.org
[jira] [Updated] (FLUME-3021) flume Elasticsearch 5.0 not support
[ https://issues.apache.org/jira/browse/FLUME-3021?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] tycho_yang updated FLUME-3021: -- Attachment: log4j2.xml > flume Elasticsearch 5.0 not support > --- > > Key: FLUME-3021 > URL: https://issues.apache.org/jira/browse/FLUME-3021 > Project: Flume > Issue Type: New Feature > Components: Sinks+Sources >Affects Versions: 1.7.0 >Reporter: tycho_yang >Assignee: Yonghao Zou >Priority: Major > Fix For: 1.9.0 > > Attachments: FLUME-3021-0.patch, FLUME-3021-1.patch, > FLUME-3021-Enable-ElasticSearch-6.1.x-support.patch, > FLUME-3021-Enable-ElasticSearch-6.1.x-support.patch, > image-2018-01-29-13-53-59-412.png, image-2018-01-31-12-00-06-854.png, > image-2018-01-31-12-00-50-050.png, image-2018-01-31-12-01-55-481.png, > image-2018-02-01-17-42-37-816.png, image-2018-02-01-17-44-24-082.png, > image-2018-02-01-17-45-54-668.png, image-2018-02-01-18-13-05-753.png, > log4j2.xml, screenshot-1.png, screenshot-2.png, screenshot-3.png > > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-unsubscr...@flume.apache.org For additional commands, e-mail: issues-h...@flume.apache.org
[jira] [Commented] (FLUME-3021) flume Elasticsearch 5.0 not support
[ https://issues.apache.org/jira/browse/FLUME-3021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16348337#comment-16348337 ] tycho_yang commented on FLUME-3021: --- Normal operation, look at the pressure test !image-2018-02-01-18-13-05-753.png! > flume Elasticsearch 5.0 not support > --- > > Key: FLUME-3021 > URL: https://issues.apache.org/jira/browse/FLUME-3021 > Project: Flume > Issue Type: New Feature > Components: Sinks+Sources >Affects Versions: 1.7.0 >Reporter: tycho_yang >Assignee: Yonghao Zou >Priority: Major > Fix For: 1.9.0 > > Attachments: FLUME-3021-0.patch, FLUME-3021-1.patch, > FLUME-3021-Enable-ElasticSearch-6.1.x-support.patch, > FLUME-3021-Enable-ElasticSearch-6.1.x-support.patch, > image-2018-01-29-13-53-59-412.png, image-2018-01-31-12-00-06-854.png, > image-2018-01-31-12-00-50-050.png, image-2018-01-31-12-01-55-481.png, > image-2018-02-01-17-42-37-816.png, image-2018-02-01-17-44-24-082.png, > image-2018-02-01-17-45-54-668.png, image-2018-02-01-18-13-05-753.png, > screenshot-1.png, screenshot-2.png, screenshot-3.png > > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-unsubscr...@flume.apache.org For additional commands, e-mail: issues-h...@flume.apache.org
[jira] [Updated] (FLUME-3021) flume Elasticsearch 5.0 not support
[ https://issues.apache.org/jira/browse/FLUME-3021?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] tycho_yang updated FLUME-3021: -- Attachment: image-2018-02-01-18-13-05-753.png > flume Elasticsearch 5.0 not support > --- > > Key: FLUME-3021 > URL: https://issues.apache.org/jira/browse/FLUME-3021 > Project: Flume > Issue Type: New Feature > Components: Sinks+Sources >Affects Versions: 1.7.0 >Reporter: tycho_yang >Assignee: Yonghao Zou >Priority: Major > Fix For: 1.9.0 > > Attachments: FLUME-3021-0.patch, FLUME-3021-1.patch, > FLUME-3021-Enable-ElasticSearch-6.1.x-support.patch, > FLUME-3021-Enable-ElasticSearch-6.1.x-support.patch, > image-2018-01-29-13-53-59-412.png, image-2018-01-31-12-00-06-854.png, > image-2018-01-31-12-00-50-050.png, image-2018-01-31-12-01-55-481.png, > image-2018-02-01-17-42-37-816.png, image-2018-02-01-17-44-24-082.png, > image-2018-02-01-17-45-54-668.png, image-2018-02-01-18-13-05-753.png, > screenshot-1.png, screenshot-2.png, screenshot-3.png > > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-unsubscr...@flume.apache.org For additional commands, e-mail: issues-h...@flume.apache.org
[jira] [Commented] (FLUME-3021) flume Elasticsearch 5.0 not support
[ https://issues.apache.org/jira/browse/FLUME-3021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16348312#comment-16348312 ] Helmut Wahrman commented on FLUME-3021: --- The patch is working ok, since you now longer don't get any errors. You should have a log4j2.xml file in your flume conf folder. Sorry, but setting a log4j config file is out of scope for this patch. Please use search engines to find out how to configure log4j. mine is like this: ./logs > flume Elasticsearch 5.0 not support > --- > > Key: FLUME-3021 > URL: https://issues.apache.org/jira/browse/FLUME-3021 > Project: Flume > Issue Type: New Feature > Components: Sinks+Sources >Affects Versions: 1.7.0 >Reporter: tycho_yang >Assignee: Yonghao Zou >Priority: Major > Fix For: 1.9.0 > > Attachments: FLUME-3021-0.patch, FLUME-3021-1.patch, > FLUME-3021-Enable-ElasticSearch-6.1.x-support.patch, > FLUME-3021-Enable-ElasticSearch-6.1.x-support.patch, > image-2018-01-29-13-53-59-412.png, image-2018-01-31-12-00-06-854.png, > image-2018-01-31-12-00-50-050.png, image-2018-01-31-12-01-55-481.png, > image-2018-02-01-17-42-37-816.png, image-2018-02-01-17-44-24-082.png, > image-2018-02-01-17-45-54-668.png, screenshot-1.png, screenshot-2.png, > screenshot-3.png > > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-unsubscr...@flume.apache.org For additional commands, e-mail: issues-h...@flume.apache.org
[jira] [Comment Edited] (FLUME-3021) flume Elasticsearch 5.0 not support
[ https://issues.apache.org/jira/browse/FLUME-3021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16348299#comment-16348299 ] tycho_yang edited comment on FLUME-3021 at 2/1/18 9:45 AM: --- !image-2018-02-01-17-42-37-816.png! !image-2018-02-01-17-44-24-082.png! The console has an error message, but the data can be passed up was (Author: ytc301): !image-2018-02-01-17-42-37-816.png! > flume Elasticsearch 5.0 not support > --- > > Key: FLUME-3021 > URL: https://issues.apache.org/jira/browse/FLUME-3021 > Project: Flume > Issue Type: New Feature > Components: Sinks+Sources >Affects Versions: 1.7.0 >Reporter: tycho_yang >Assignee: Yonghao Zou >Priority: Major > Fix For: 1.9.0 > > Attachments: FLUME-3021-0.patch, FLUME-3021-1.patch, > FLUME-3021-Enable-ElasticSearch-6.1.x-support.patch, > FLUME-3021-Enable-ElasticSearch-6.1.x-support.patch, > image-2018-01-29-13-53-59-412.png, image-2018-01-31-12-00-06-854.png, > image-2018-01-31-12-00-50-050.png, image-2018-01-31-12-01-55-481.png, > image-2018-02-01-17-42-37-816.png, image-2018-02-01-17-44-24-082.png, > image-2018-02-01-17-45-54-668.png, screenshot-1.png, screenshot-2.png, > screenshot-3.png > > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-unsubscr...@flume.apache.org For additional commands, e-mail: issues-h...@flume.apache.org
[jira] [Commented] (FLUME-3021) flume Elasticsearch 5.0 not support
[ https://issues.apache.org/jira/browse/FLUME-3021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16348305#comment-16348305 ] tycho_yang commented on FLUME-3021: --- !image-2018-02-01-17-45-54-668.png! > flume Elasticsearch 5.0 not support > --- > > Key: FLUME-3021 > URL: https://issues.apache.org/jira/browse/FLUME-3021 > Project: Flume > Issue Type: New Feature > Components: Sinks+Sources >Affects Versions: 1.7.0 >Reporter: tycho_yang >Assignee: Yonghao Zou >Priority: Major > Fix For: 1.9.0 > > Attachments: FLUME-3021-0.patch, FLUME-3021-1.patch, > FLUME-3021-Enable-ElasticSearch-6.1.x-support.patch, > FLUME-3021-Enable-ElasticSearch-6.1.x-support.patch, > image-2018-01-29-13-53-59-412.png, image-2018-01-31-12-00-06-854.png, > image-2018-01-31-12-00-50-050.png, image-2018-01-31-12-01-55-481.png, > image-2018-02-01-17-42-37-816.png, image-2018-02-01-17-44-24-082.png, > image-2018-02-01-17-45-54-668.png, screenshot-1.png, screenshot-2.png, > screenshot-3.png > > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-unsubscr...@flume.apache.org For additional commands, e-mail: issues-h...@flume.apache.org
[jira] [Updated] (FLUME-3021) flume Elasticsearch 5.0 not support
[ https://issues.apache.org/jira/browse/FLUME-3021?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] tycho_yang updated FLUME-3021: -- Attachment: image-2018-02-01-17-45-54-668.png > flume Elasticsearch 5.0 not support > --- > > Key: FLUME-3021 > URL: https://issues.apache.org/jira/browse/FLUME-3021 > Project: Flume > Issue Type: New Feature > Components: Sinks+Sources >Affects Versions: 1.7.0 >Reporter: tycho_yang >Assignee: Yonghao Zou >Priority: Major > Fix For: 1.9.0 > > Attachments: FLUME-3021-0.patch, FLUME-3021-1.patch, > FLUME-3021-Enable-ElasticSearch-6.1.x-support.patch, > FLUME-3021-Enable-ElasticSearch-6.1.x-support.patch, > image-2018-01-29-13-53-59-412.png, image-2018-01-31-12-00-06-854.png, > image-2018-01-31-12-00-50-050.png, image-2018-01-31-12-01-55-481.png, > image-2018-02-01-17-42-37-816.png, image-2018-02-01-17-44-24-082.png, > image-2018-02-01-17-45-54-668.png, screenshot-1.png, screenshot-2.png, > screenshot-3.png > > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-unsubscr...@flume.apache.org For additional commands, e-mail: issues-h...@flume.apache.org
[jira] [Updated] (FLUME-3021) flume Elasticsearch 5.0 not support
[ https://issues.apache.org/jira/browse/FLUME-3021?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] tycho_yang updated FLUME-3021: -- Attachment: image-2018-02-01-17-42-37-816.png > flume Elasticsearch 5.0 not support > --- > > Key: FLUME-3021 > URL: https://issues.apache.org/jira/browse/FLUME-3021 > Project: Flume > Issue Type: New Feature > Components: Sinks+Sources >Affects Versions: 1.7.0 >Reporter: tycho_yang >Assignee: Yonghao Zou >Priority: Major > Fix For: 1.9.0 > > Attachments: FLUME-3021-0.patch, FLUME-3021-1.patch, > FLUME-3021-Enable-ElasticSearch-6.1.x-support.patch, > FLUME-3021-Enable-ElasticSearch-6.1.x-support.patch, > image-2018-01-29-13-53-59-412.png, image-2018-01-31-12-00-06-854.png, > image-2018-01-31-12-00-50-050.png, image-2018-01-31-12-01-55-481.png, > image-2018-02-01-17-42-37-816.png, screenshot-1.png, screenshot-2.png, > screenshot-3.png > > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-unsubscr...@flume.apache.org For additional commands, e-mail: issues-h...@flume.apache.org
[jira] [Commented] (FLUME-3021) flume Elasticsearch 5.0 not support
[ https://issues.apache.org/jira/browse/FLUME-3021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16348299#comment-16348299 ] tycho_yang commented on FLUME-3021: --- !image-2018-02-01-17-42-37-816.png! > flume Elasticsearch 5.0 not support > --- > > Key: FLUME-3021 > URL: https://issues.apache.org/jira/browse/FLUME-3021 > Project: Flume > Issue Type: New Feature > Components: Sinks+Sources >Affects Versions: 1.7.0 >Reporter: tycho_yang >Assignee: Yonghao Zou >Priority: Major > Fix For: 1.9.0 > > Attachments: FLUME-3021-0.patch, FLUME-3021-1.patch, > FLUME-3021-Enable-ElasticSearch-6.1.x-support.patch, > FLUME-3021-Enable-ElasticSearch-6.1.x-support.patch, > image-2018-01-29-13-53-59-412.png, image-2018-01-31-12-00-06-854.png, > image-2018-01-31-12-00-50-050.png, image-2018-01-31-12-01-55-481.png, > image-2018-02-01-17-42-37-816.png, screenshot-1.png, screenshot-2.png, > screenshot-3.png > > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-unsubscr...@flume.apache.org For additional commands, e-mail: issues-h...@flume.apache.org
[jira] [Commented] (FLUME-3021) flume Elasticsearch 5.0 not support
[ https://issues.apache.org/jira/browse/FLUME-3021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16348184#comment-16348184 ] tycho_yang commented on FLUME-3021: --- jar package : elasticsearch-6.1.2.jar transport-6.1.2.jar transport-netty4-client-6.1.2.jar spatial4j-0.6.jar elasticsearch-cli-6.1.2.jar jts-1.13.jar reindex-client-6.1.2.jar elasticsearch-rest-client-6.1.2.jar lang-mustache-client-6.1.2.jar percolator-client-6.1.2.jar parent-join-client-6.1.2.jar lang-mustache-client-6.1.2.jar netty-transport-4.1.13.Final.jar netty-resolver-4.1.13.Final.jar netty-handler-4.1.13.Final.jar netty-common-4.1.13.Final.jar netty-codec-http-4.1.13.Final.jar netty-codec-4.1.13.Final.jar netty-buffer-4.1.13.Final.jar jackson-dataformat-yaml-2.8.10.jar jackson-dataformat-smile-2.8.10.jar jackson-dataformat-cbor-2.8.10.jar lucene-suggest-7.1.0.jar lucene-spatial-extras-7.1.0.jar lucene-spatial-7.1.0.jar lucene-spatial3d-7.1.0.jar lucene-sandbox-7.1.0.jar lucene-queryparser-7.1.0.jar lucene-queries-7.1.0.jar lucene-misc-7.1.0.jar lucene-memory-7.1.0.jar lucene-join-7.1.0.jar lucene-highlighter-7.1.0.jar lucene-grouping-7.1.0.jar lucene-core-7.1.0.jar lucene-backward-codecs-7.1.0.jar lucene-analyzers-common-7.1.0.jar t-digest-3.0.jar hppc-0.7.1.jar log4j-core-2.7.jar log4j-api-2.7.jar log4j-1.2.17.jar > flume Elasticsearch 5.0 not support > --- > > Key: FLUME-3021 > URL: https://issues.apache.org/jira/browse/FLUME-3021 > Project: Flume > Issue Type: New Feature > Components: Sinks+Sources >Affects Versions: 1.7.0 >Reporter: tycho_yang >Assignee: Yonghao Zou >Priority: Major > Fix For: 1.9.0 > > Attachments: FLUME-3021-0.patch, FLUME-3021-1.patch, > FLUME-3021-Enable-ElasticSearch-6.1.x-support.patch, > FLUME-3021-Enable-ElasticSearch-6.1.x-support.patch, > image-2018-01-29-13-53-59-412.png, image-2018-01-31-12-00-06-854.png, > image-2018-01-31-12-00-50-050.png, image-2018-01-31-12-01-55-481.png, > screenshot-1.png, screenshot-2.png, screenshot-3.png > > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-unsubscr...@flume.apache.org For additional commands, e-mail: issues-h...@flume.apache.org
