[jira] [Commented] (FLUME-2624) Improve Hive Sink performance

[ https://issues.apache.org/jira/browse/FLUME-2624?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14392082#comment-14392082 ] Hudson commented on FLUME-2624: --- SUCCESS: Integrated in flume-trunk #724 (See [https://buil

[jira] [Commented] (FLUME-2624) Improve Hive Sink performance

[ https://issues.apache.org/jira/browse/FLUME-2624?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14392078#comment-14392078 ] Hudson commented on FLUME-2624: --- FAILURE: Integrated in Flume-trunk-hbase-98 #81 (See [http

Build failed in Jenkins: Flume-trunk-hbase-98 #81

See Changes: [roshan] FLUME-2624 : Improve Hive Sink performance -- [...truncated 5890 lines...] Compression is 0.0% Took 0.16 sec [JENKINS] Archiving

[jira] [Commented] (FLUME-2624) Improve Hive Sink performance

[ https://issues.apache.org/jira/browse/FLUME-2624?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14392051#comment-14392051 ] Deepesh Khandelwal commented on FLUME-2624: --- Thanks [~roshan_naik] for the revie

[jira] [Commented] (FLUME-2624) Improve Hive Sink performance

[ https://issues.apache.org/jira/browse/FLUME-2624?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14392048#comment-14392048 ] Roshan Naik commented on FLUME-2624: FYI.. as part of this commit I also deleted the e

[jira] [Comment Edited] (FLUME-2624) Improve Hive Sink performance

[ https://issues.apache.org/jira/browse/FLUME-2624?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14392014#comment-14392014 ] Roshan Naik edited comment on FLUME-2624 at 4/2/15 2:37 AM: Th

[jira] [Commented] (FLUME-2624) Improve Hive Sink performance

[ https://issues.apache.org/jira/browse/FLUME-2624?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14392013#comment-14392013 ] ASF subversion and git services commented on FLUME-2624: Commit 78

[jira] [Commented] (FLUME-2624) Improve Hive Sink performance

[ https://issues.apache.org/jira/browse/FLUME-2624?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14392005#comment-14392005 ] Roshan Naik commented on FLUME-2624: +1 > Improve Hive Sink performance > ---

Re: Sinks without timeout and ELB Spillover - hangs

Is it possible to share the method and class name? It would be somewhere in client package I guess. I shall do the rest, else would need to re-invent the wheel :) On Wed, Apr 1, 2015 at 3:30 AM, Otis Gospodnetic wrote: > Hi, > > Unfortunately don't have a patch, though I think this is just 1 meth

Re: File channel and FIFO ordering

None which get packaged with Flume. Thanks, Hari On Wed, Apr 1, 2015 at 5:24 PM, Roshan Naik wrote: > I can't think of any sinks which can do that. Do you know of any such > sinks ? HDFS sink (I don't think) ? Hbase sink ? > -roshan > On 4/1/15 5:17 PM, "Hari Shreedharan" wrote: >>If a sin

Re: File channel and FIFO ordering

I can't think of any sinks which can do that. Do you know of any such sinks ? HDFS sink (I don't think) ? Hbase sink ? -roshan On 4/1/15 5:17 PM, "Hari Shreedharan" wrote: >If a sink continuously rolls back, but it is single threaded - only one >txn at a time, then the ordering is still guara

Re: File channel and FIFO ordering

If a sink continuously rolls back, but it is single threaded - only one txn at a time, then the ordering is still guaranteed. But if the sink keeps multiple txns open, then the ordering is gone. Thanks, Hari On Wed, Apr 1, 2015 at 5:10 PM, Roshan Naik wrote: > Specifically in the case of a

File channel and FIFO ordering

Specifically in the case of a single source and sink attached to the channel... I know that in Flume we do not explicitly guarantee FIFO order for any of the channels although in practice the channels do seem to exhibit FIFO. Specifically wrt to file channel... Not sure if under normal operatio

Re: Is 1.6 a good time to switch to hadoop2 ?

Seems like hbase 1.0.0 has now dropped hadoop1 support. -roshan On 4/1/15 4:19 PM, "Jarek Jarcec Cecho" wrote: >Sqoop2 method is very simple - we have different profiles for Hadoop 1 >and 2 and simply use classifiers to mark the appropriate hadoop >"profile". No need to generate different pom.x

Re: Is 1.6 a good time to switch to hadoop2 ?

Sqoop2 method is very simple - we have different profiles for Hadoop 1 and 2 and simply use classifiers to mark the appropriate hadoop “profile”. No need to generate different pom.xml file. Check it out here: https://github.com/apache/sqoop/blob/branch-1.99.4/execution/mapreduce/pom.xml#L65 Jar

[jira] [Commented] (FLUME-2631) End to End authentication in Flume

[ https://issues.apache.org/jira/browse/FLUME-2631?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14391627#comment-14391627 ] Johny Rufus commented on FLUME-2631: [~roshan_naik], The documentation is part of the

Re: Is 1.6 a good time to switch to hadoop2 ?

We should choose which ever is the easiest to do I guess (Sqoop method or HBase method). Thanks, Hari On Wed, Apr 1, 2015 at 1:33 PM, Roshan Naik wrote: > FWIWŠ From what I recollect, Hbase did this when they started supporting > hadoop1&2. Not sure if its still the same method they use.. >

[jira] [Created] (FLUME-2654) Memory channel error message related to byteCapacity needs refining

Johny Rufus created FLUME-2654: -- Summary: Memory channel error message related to byteCapacity needs refining Key: FLUME-2654 URL: https://issues.apache.org/jira/browse/FLUME-2654 Project: Flume

Re: Is 1.6 a good time to switch to hadoop2 ?

FWIWŠ From what I recollect, Hbase did this when they started supporting hadoop1&2. Not sure if its still the same method they use.. - their default pom had the artifacts named for hadoop1 binaries - they had a shell script to produce a modified pom ..with -hadoop2 suffix for the artifact names -

Re: Is 1.6 a good time to switch to hadoop2 ?

I know sqoop2 does it. Maybe Jarcec can help? Thanks, Hari On Wed, Apr 1, 2015 at 1:13 PM, Roshan Naik wrote: > To push artifacts for both hadoop2 and hadoop1 will need to name the > hadoop2 and hadoop1 artifacts differently. Tricky to do that with one pom > I think. > -roshan > On 4/1/15 1:

Re: Is 1.6 a good time to switch to hadoop2 ?

To push artifacts for both hadoop2 and hadoop1 will need to name the hadoop2 and hadoop1 artifacts differently. Tricky to do that with one pom I think. -roshan On 4/1/15 1:05 PM, "Hari Shreedharan" wrote: >I am a +1 for doing this, since this is the last JDK 6 release. Maybe we >should push ar

Re: Is 1.6 a good time to switch to hadoop2 ?

I am a +1 for doing this, since this is the last JDK 6 release. Maybe we should push artifacts for both Hadoop-1 and hbase-98 profiles this time and switch out to Hadoop 2 exclusively from 1.7? Thanks, Hari On Wed, Apr 1, 2015 at 1:04 PM, Roshan Naik wrote: > Don't recall if this has been

Is 1.6 a good time to switch to hadoop2 ?

Don't recall if this has been discussed before. Its been sometime since hadoop2 has been out. Sooner or later Flume will switch to hadoop2 based builds (hbase98 profile?) as the default. Not sure if 1.6 is the time or worth waiting longer. -roshan

[jira] [Commented] (FLUME-2649) Elasticsearch sink doesn't handle JSON fields correctly

[ https://issues.apache.org/jira/browse/FLUME-2649?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14390928#comment-14390928 ] Edward Sargisson commented on FLUME-2649: - Great - thanks! Could you please put th

[jira] [Updated] (FLUME-2649) Elasticsearch sink doesn't handle JSON fields correctly

[ https://issues.apache.org/jira/browse/FLUME-2649?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Benjamin Fiorini updated FLUME-2649: Attachment: FLUME-2649-2.patch And this patch provides better tests. > Elasticsearch sink d

[jira] [Updated] (FLUME-2649) Elasticsearch sink doesn't handle JSON fields correctly

[ https://issues.apache.org/jira/browse/FLUME-2649?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Benjamin Fiorini updated FLUME-2649: Attachment: FLUME-2649-1.patch This patch passes all the tests. > Elasticsearch sink doesn'

[jira] [Updated] (FLUME-2649) Elasticsearch sink doesn't handle JSON fields correctly

[ https://issues.apache.org/jira/browse/FLUME-2649?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Benjamin Fiorini updated FLUME-2649: Attachment: FLUME-2649-0.patch This is exactly what was suggested 2 years ago in FLUME-2126.

[GitHub] flume pull request: fix: netcat source tcp connection is not close...

GitHub user ericx10ng opened a pull request: https://github.com/apache/flume/pull/16 fix: netcat source tcp connection is not closed when client using tcp short connection, netcat source tcp connections were not closed. flume had lots of tcp close_wait until process exited! You ca