[jira] [Resolved] (FLUME-189) Enable output bucketing for the text() sink
[ https://issues.apache.org/jira/browse/FLUME-189?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Paliwal resolved FLUME-189. -- Resolution: Won't Fix Fix Version/s: v0.9.5 Won't fix. 0.X branch not maintained anymore > Enable output bucketing for the text() sink > --- > > Key: FLUME-189 > URL: https://issues.apache.org/jira/browse/FLUME-189 > Project: Flume > Issue Type: New Feature > Components: Sinks+Sources >Affects Versions: v0.9.4 >Reporter: Disabled imported user >Priority: Minor > Fix For: v0.9.5 > > > Output bucketing would be useful for the text() sink, but currently is only > an option when writing to hdfs. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (FLUME-193) excessive error logging for auto*Chains when using multiple masters
[ https://issues.apache.org/jira/browse/FLUME-193?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Paliwal resolved FLUME-193. -- Resolution: Won't Fix Won't fix. 0.X branch not maintained anymore > excessive error logging for auto*Chains when using multiple masters > --- > > Key: FLUME-193 > URL: https://issues.apache.org/jira/browse/FLUME-193 > Project: Flume > Issue Type: Bug >Affects Versions: v0.9.0 > Environment: RHE5 >Reporter: Disabled imported user >Priority: Minor > Fix For: v0.9.5 > > > Jon mentioned that auto-chaining is not fully functional for 0.9.0 when using > multiple masters, and the errors generated by generates massive logs. Would > be helpful if logging was either backed off or throttled in this case. > cheers, > -James -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (FLUME-174) Create standard unit tests for decorators
[ https://issues.apache.org/jira/browse/FLUME-174?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Paliwal resolved FLUME-174. -- Resolution: Won't Fix Won't fix. 0.X branch not maintained anymore > Create standard unit tests for decorators > - > > Key: FLUME-174 > URL: https://issues.apache.org/jira/browse/FLUME-174 > Project: Flume > Issue Type: New Feature > Components: Sinks+Sources, Test >Affects Versions: v0.9.4 >Reporter: Jonathan Hsieh > Fix For: v0.9.5 > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (FLUME-175) Add a dummy autoCollectorSink for system testing
[ https://issues.apache.org/jira/browse/FLUME-175?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Paliwal resolved FLUME-175. -- Resolution: Won't Fix Fix Version/s: v0.9.5 Won't fix. 0.X branch not maintained anymore > Add a dummy autoCollectorSink for system testing > - > > Key: FLUME-175 > URL: https://issues.apache.org/jira/browse/FLUME-175 > Project: Flume > Issue Type: New Feature > Components: Easy, Master, Sinks+Sources >Affects Versions: v0.9.4 >Reporter: Jonathan Hsieh > Fix For: v0.9.5 > > > Currently, it is difficult to test failure cases of the autoCollectorSource. > Creating a autoCollectorSource that doesn't actually start an rpc server > would be a great way to test communications failures when using the auto* > modes. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (FLUME-187) Remove non-ascii space characters in debian/control
[ https://issues.apache.org/jira/browse/FLUME-187?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Paliwal resolved FLUME-187. -- Resolution: Won't Fix Fix Version/s: v0.9.5 Won't fix. 0.X branch not maintained anymore > Remove non-ascii space characters in debian/control > --- > > Key: FLUME-187 > URL: https://issues.apache.org/jira/browse/FLUME-187 > Project: Flume > Issue Type: Bug > Components: Build >Affects Versions: v0.9.0 > Environment: Ubuntu 10.04 >Reporter: Disabled imported user >Priority: Minor > Fix For: v0.9.5 > > > I grabbed the debian/control files to build my own Debian packages, and one > of our packaging tools, repoman (http://github.com/synack/repoman) checks > that all characters in debian/control are ASCII (they should be according to > Debian). There are some non-ascii space characters in the debian/control > provided by Cloudera. > Here's the patch, it's pretty simple: > http://github.com/simplegeo/flume/commit/44b9b69a1b43d22047fc25687699098282f172a8 > -Ben -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (FLUME-185) Consider how to globally align timestamps of events
[ https://issues.apache.org/jira/browse/FLUME-185?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Paliwal resolved FLUME-185. -- Resolution: Won't Fix Fix Version/s: v0.9.5 Won't fix. 0.X branch not maintained anymore > Consider how to globally align timestamps of events > --- > > Key: FLUME-185 > URL: https://issues.apache.org/jira/browse/FLUME-185 > Project: Flume > Issue Type: New Feature >Affects Versions: v0.9.4 >Reporter: Disabled imported user > Fix For: v0.9.5 > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (FLUME-166) flume-node in simple syslog | collectorSink configuration using a lot of CPU
[ https://issues.apache.org/jira/browse/FLUME-166?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Paliwal resolved FLUME-166. -- Resolution: Won't Fix Won't fix. 0.X branch not maintained anymore > flume-node in simple syslog | collectorSink configuration using a lot of CPU > > > Key: FLUME-166 > URL: https://issues.apache.org/jira/browse/FLUME-166 > Project: Flume > Issue Type: Bug > Components: Node >Affects Versions: v0.9.0 > Environment: Ubuntu 10.04. 12GB RAM, 4X Intel L5320@1.86GHz. >Reporter: Disabled imported user > Fix For: v0.9.5 > > > I have 1 flume-node and it is configured as follows, > syslogTcp( 5140 ) | collectorSink( > "hdfs://hadoop-1.i:9000/flume/logs/%Y/%m/%d/%H", "web-", 30) > It is showing 200-250% CPU use in top. I'm sending about 100MB/minute of > syslogs toward it (from maybe 15 sources total). > jvisualvm info to come. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (FLUME-173) Create standard unit tests for Sinks
[ https://issues.apache.org/jira/browse/FLUME-173?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Paliwal resolved FLUME-173. -- Resolution: Won't Fix Won't fix. 0.X branch not maintained anymore > Create standard unit tests for Sinks > > > Key: FLUME-173 > URL: https://issues.apache.org/jira/browse/FLUME-173 > Project: Flume > Issue Type: New Feature > Components: Sinks+Sources >Affects Versions: v0.9.4 >Reporter: Jonathan Hsieh > Fix For: v0.9.5 > > > Create standard tests for sinks -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (FLUME-157) Allow masterless deployments
[ https://issues.apache.org/jira/browse/FLUME-157?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Paliwal resolved FLUME-157. -- Resolution: Won't Fix Fix Version/s: v0.9.5 Won't fix. 0.X branch not maintained anymore > Allow masterless deployments > > > Key: FLUME-157 > URL: https://issues.apache.org/jira/browse/FLUME-157 > Project: Flume > Issue Type: New Feature > Components: Master >Affects Versions: v0.9.4 >Reporter: Disabled imported user > Fix For: v0.9.5 > > > Some folks have mentioned that Flume sometimes feels "heavyweight" because it > requires a master. I'm not sure if it's a good idea, but I wanted to > centralize the discussion about whether allowing users to deploy Flume > without a master is a reasonable thing to do. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (FLUME-156) Build Flume node in an alternative language
[ https://issues.apache.org/jira/browse/FLUME-156?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Paliwal resolved FLUME-156. -- Resolution: Won't Fix Fix Version/s: v0.9.5 Won't fix. 0.X branch not maintained anymore > Build Flume node in an alternative language > --- > > Key: FLUME-156 > URL: https://issues.apache.org/jira/browse/FLUME-156 > Project: Flume > Issue Type: New Feature > Components: Node >Affects Versions: v0.9.4 >Reporter: Disabled imported user > Fix For: v0.9.5 > > > To demonstrate the value of the move to Avro, it would be great to implement > a simple Flume node in another language. Python seems like the most likely > candidate, since the Avro RPC implementation is most complete of the non-Java > Avro implementations. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (FLUME-150) restart of node in multi-master setup creates LOST node when reregisters with new master
[ https://issues.apache.org/jira/browse/FLUME-150?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Paliwal resolved FLUME-150. -- Resolution: Won't Fix Fix Version/s: v0.9.5 Won't fix. 0.X branch not maintained anymore > restart of node in multi-master setup creates LOST node when reregisters with > new master > > > Key: FLUME-150 > URL: https://issues.apache.org/jira/browse/FLUME-150 > Project: Flume > Issue Type: Bug > Components: Master >Affects Versions: v0.9.0 > Environment: linux >Reporter: Disabled imported user > Fix For: v0.9.5 > > > I have a 3 node flume-master setup that is backed by an external 3 node > zookeeper cluster. > flume-node registers on flume-master node A. If the flume-node process is > restarted/box rebooted/whatever, when it comes back up and it registers on > flume-master node B, node A shows a LOST node even though the node is > registered on node B. > Screenshots I'll attach: > 1) beforeRestart.png shows 'cdh3collector3 registered on master (server id:1) > 2) afterRestartS0.png shows 'cdh3collector3' registered on new master (server > id:0) > 3) deadNodeS1.png shows the old registration for 'cdh3collector3' as not seen > in a long time on old master (server id:1). Eventually this changes to 'LOST' > It seems that the old registration should disappear if the node reappears on > a different master node. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (FLUME-146) Strip agent and collector to a core set of features and complete transition to plugins architecture
[ https://issues.apache.org/jira/browse/FLUME-146?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Paliwal resolved FLUME-146. -- Resolution: Won't Fix Fix Version/s: v0.9.5 Won't fix. 0.X branch not maintained anymore > Strip agent and collector to a core set of features and complete transition > to plugins architecture > --- > > Key: FLUME-146 > URL: https://issues.apache.org/jira/browse/FLUME-146 > Project: Flume > Issue Type: Epic > Components: Node >Affects Versions: v0.9.4 >Reporter: Disabled imported user > Fix For: v0.9.5 > > > The plugins architecture was designed halfway through implementing Flume; it > would be nice to move most sinks/sources/decorators to work as plugins so > that Flume core can be nice and svelte. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (FLUME-145) Decorator to cryptographically hash an attribute of event.
[ https://issues.apache.org/jira/browse/FLUME-145?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Paliwal resolved FLUME-145. -- Resolution: Won't Fix Fix Version/s: v0.9.5 Won't fix. 0.X branch not maintained anymore > Decorator to cryptographically hash an attribute of event. > -- > > Key: FLUME-145 > URL: https://issues.apache.org/jira/browse/FLUME-145 > Project: Flume > Issue Type: New Feature > Components: Sinks+Sources >Affects Versions: v0.9.4 >Reporter: Jonathan Hsieh > Fix For: v0.9.5 > > > At a customer visit, we saw records with a single field cryptographically > hashed to obscure sensitive information. Something like this would probably > be useful in flume. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (FLUME-142) add syslog source that can listen on a unix domain socket
[ https://issues.apache.org/jira/browse/FLUME-142?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Paliwal resolved FLUME-142. -- Resolution: Won't Fix Fix Version/s: v0.9.5 Won't fix. 0.X branch not maintained anymore > add syslog source that can listen on a unix domain socket > - > > Key: FLUME-142 > URL: https://issues.apache.org/jira/browse/FLUME-142 > Project: Flume > Issue Type: New Feature > Components: Sinks+Sources >Affects Versions: v0.9.4 >Reporter: Patrick Hunt > Fix For: v0.9.5 > > > Today we have syslog sources that can listen on a network socket, we should > have the ability to listen on a unix domain socket as well - similar to > syslogd support. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (FLUME-123) HDFS Dir Source
[ https://issues.apache.org/jira/browse/FLUME-123?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Paliwal resolved FLUME-123. -- Resolution: Won't Fix Fix Version/s: v0.9.5 Won't fix. 0.X branch not maintained anymore > HDFS Dir Source > --- > > Key: FLUME-123 > URL: https://issues.apache.org/jira/browse/FLUME-123 > Project: Flume > Issue Type: New Feature > Components: Easy, Sinks+Sources >Affects Versions: v0.9.4 >Reporter: Jonathan Hsieh > Fix For: v0.9.5 > > > We should have an HDFS source. This will allow for events to be replayed > once written to hdfs, allows for consolidation passes, multistage flume > pipelines (store to smaller staging cluster, then flume it to a large > repository). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (FLUME-118) Alphabetize source, sink, and decorator catalog
[ https://issues.apache.org/jira/browse/FLUME-118?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Paliwal resolved FLUME-118. -- Resolution: Won't Fix Won't fix. 0.X branch not maintained anymore > Alphabetize source, sink, and decorator catalog > --- > > Key: FLUME-118 > URL: https://issues.apache.org/jira/browse/FLUME-118 > Project: Flume > Issue Type: Improvement > Components: Docs, Easy >Affects Versions: v0.9.4 >Reporter: Disabled imported user > Fix For: v0.9.5 > > > Whenever I go to search for a source/sink/decorator, I have to use a search > tool. I recognize that some sources/sinks/decorators make sense to introduce > logically before others, but the catalog at the end of all available > sources/sinks/decorators should be alphabetized to use as an index. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (FLUME-112) Think of ways to reuse transformation logic in source/sink/decorator as a Map or Reduce task in Hadoop MapReduce
[ https://issues.apache.org/jira/browse/FLUME-112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Paliwal resolved FLUME-112. -- Resolution: Won't Fix Fix Version/s: v0.9.5 Won't fix. 0.X branch not maintained anymore > Think of ways to reuse transformation logic in source/sink/decorator as a Map > or Reduce task in Hadoop MapReduce > > > Key: FLUME-112 > URL: https://issues.apache.org/jira/browse/FLUME-112 > Project: Flume > Issue Type: New Feature > Components: Sinks+Sources >Affects Versions: v0.9.4 >Reporter: Disabled imported user > Fix For: v0.9.5 > > > It has come up on the mailing list that while Flume is great for formatting > new data that's coming into HDFS, a lot of time there is old data sitting > around that you'd like to make magically match the format that the new data > is getting written in. It's not a pressing feature, but opening a ticket now > to see if we can come up with some clever ways to bundle up Flume logic and > reuse it in a Map or Reduce task. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (FLUME-109) Node doesn't auto restart after HD full recovery
[ https://issues.apache.org/jira/browse/FLUME-109?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Paliwal resolved FLUME-109. -- Resolution: Won't Fix Fix Version/s: v0.9.5 Won't fix. 0.X branch not maintained anymore > Node doesn't auto restart after HD full recovery > > > Key: FLUME-109 > URL: https://issues.apache.org/jira/browse/FLUME-109 > Project: Flume > Issue Type: Bug > Components: Node >Affects Versions: v0.9.4 >Reporter: Nicholas Verbeck > Fix For: v0.9.5 > > > If you have a node thats HD get full and you recover from this state. The > node seems to hang and must be restarted before it will start sending data to > the collectors. Ideally the node should detect the HD being full and monitor > it tell it gets room again and auto start back up. It may also be worth while > to have a config that is used for a HD available space threshold. That causes > the processes to wait tell space becomes available. > The issue that caused me to find this problem is that it also appears the > write ahead logic seems to not regulate itself on how much data it writes to > the "sent" folder before waiting for it to actually be sent and received. So > a 25MB file will be expanded to ~2.5GB of seq (6582) files in the "sent" > folder. This is assuming that the sent folder is being used as a buffer for > the write ahead logic before actually sending the data. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (FLUME-96) Document how to run Flume with a non-ZooKeeper master
[ https://issues.apache.org/jira/browse/FLUME-96?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Paliwal resolved FLUME-96. - Resolution: Won't Fix Won't fix. 0.X branch not maintained anymore > Document how to run Flume with a non-ZooKeeper master > - > > Key: FLUME-96 > URL: https://issues.apache.org/jira/browse/FLUME-96 > Project: Flume > Issue Type: New Feature > Components: Docs >Affects Versions: v0.9.4 >Reporter: Disabled imported user > Fix For: v0.9.5 > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (FLUME-93) _ in event field names produce problems when using Attributes object
[ https://issues.apache.org/jira/browse/FLUME-93?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Paliwal resolved FLUME-93. - Resolution: Won't Fix Fix Version/s: v0.9.5 Won't fix. 0.X branch not maintained anymore > _ in event field names produce problems when using Attributes object > > > Key: FLUME-93 > URL: https://issues.apache.org/jira/browse/FLUME-93 > Project: Flume > Issue Type: Bug >Affects Versions: v0.9.4 >Reporter: Nicholas Verbeck > Fix For: v0.9.5 > > > If you are using the Attributes object to set/get Event fields, and a _ is > used in the fields name. When looking at the Event the field name will be > missing the _ and the value will be that of what appears to be the oct value > of the string. This produces problems in the Event string replace and the > storage of the actual event in avrojson. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (FLUME-91) Membase source
[ https://issues.apache.org/jira/browse/FLUME-91?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Paliwal resolved FLUME-91. - Resolution: Won't Fix Fix Version/s: v0.9.5 Won't fix. 0.X branch not maintained anymore > Membase source > -- > > Key: FLUME-91 > URL: https://issues.apache.org/jira/browse/FLUME-91 > Project: Flume > Issue Type: New Feature > Components: Sinks+Sources >Affects Versions: v0.9.4 >Reporter: Disabled imported user > Fix For: v0.9.5 > > > Membase exposes a "Tap" interface into the mutation log. We could send each > mutation into Flume for monitoring, audit logging, replication, or other > purposes. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (FLUME-80) IllegalArgumentException in Master
[ https://issues.apache.org/jira/browse/FLUME-80?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Paliwal resolved FLUME-80. - Resolution: Won't Fix Won't fix. 0.X branch not maintained anymore > IllegalArgumentException in Master > -- > > Key: FLUME-80 > URL: https://issues.apache.org/jira/browse/FLUME-80 > Project: Flume > Issue Type: Bug > Components: Master >Affects Versions: v0.9.4 >Reporter: Disabled imported user > Fix For: v0.9.5 > > > I had one node lock up on a pseudo-distributed install of Flume (using the > console as source and sync). So I tried running another node and then changed > my sink and got this error when trying to open the master: > null > java.lang.IllegalArgumentException > at > com.google.common.base.Preconditions.checkArgument(Preconditions.java:75) > at > com.cloudera.flume.master.failover.FailoverConfigurationManager.translateSink(FailoverConfigurationManager.java:103) > at > com.cloudera.flume.master.TranslatingConfigurationManager.setConfig(TranslatingConfigurationManager.java:105) > at > com.cloudera.flume.master.flows.FlowConfigManager.setConfig(FlowConfigManager.java:262) > at > com.cloudera.flume.master.TranslatingConfigurationManager.setConfig(TranslatingConfigurationManager.java:98) > at com.cloudera.flume.master.ConfigCommand$1.exec(ConfigCommand.java:87) > at > com.cloudera.flume.master.CommandManager.exec(CommandManager.java:255) > at > com.cloudera.flume.master.CommandManager.handleCommand(CommandManager.java:194) > at > com.cloudera.flume.master.CommandManager$ExecThread.run(CommandManager.java:225) > 10/07/27 12:39:46 WARN /: /flumemaster.jsp: > java.lang.NullPointerException > at java.lang.String.compareTo(String.java:1167) > at java.lang.String.compareTo(String.java:92) > at java.util.TreeMap.put(TreeMap.java:545) > at java.util.AbstractMap.putAll(AbstractMap.java:256) > at java.util.TreeMap.putAll(TreeMap.java:304) > at java.util.TreeMap.(TreeMap.java:162) > at > com.cloudera.flume.master.TranslatingConfigurationManager.getReport(TranslatingConfigurationManager.java:154) > at > com.cloudera.flume.master.FlumeMaster.reportHtml(FlumeMaster.java:327) > at org.apache.jsp.flumemaster_jsp._jspService(flumemaster_jsp.java:68) > at org.apache.jasper.runtime.HttpJspBase.service(HttpJspBase.java:94) > at javax.servlet.http.HttpServlet.service(HttpServlet.java:802) > at > org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:427) > at > org.mortbay.jetty.servlet.WebApplicationHandler.dispatch(WebApplicationHandler.java:475) > at > org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:567) > at org.mortbay.http.HttpContext.handle(HttpContext.java:1565) > at > org.mortbay.jetty.servlet.WebApplicationContext.handle(WebApplicationContext.java:635) > at org.mortbay.http.HttpContext.handle(HttpContext.java:1517) > at org.mortbay.http.HttpServer.service(HttpServer.java:954) > at org.mortbay.http.HttpConnection.service(HttpConnection.java:814) > at org.mortbay.http.HttpConnection.handleNext(HttpConnection.java:981) > at org.mortbay.http.HttpConnection.handle(HttpConnection.java:831) > at > org.mortbay.http.SocketListener.handleConnection(SocketListener.java:244) > at org.mortbay.util.ThreadedServer.handle(ThreadedServer.java:357) > at org.mortbay.util.ThreadPool$PoolThread.run(ThreadPool.java:534) > 10/07/27 12:40:12 WARN /: /flumemaster.jsp: > java.lang.NullPointerException > at java.lang.String.compareTo(String.java:1167) > at java.lang.String.compareTo(String.java:92) > at java.util.TreeMap.put(TreeMap.java:545) > at java.util.AbstractMap.putAll(AbstractMap.java:256) > at java.util.TreeMap.putAll(TreeMap.java:304) > at java.util.TreeMap.(TreeMap.java:162) > at > com.cloudera.flume.master.TranslatingConfigurationManager.getReport(TranslatingConfigurationManager.java:154) > at > com.cloudera.flume.master.FlumeMaster.reportHtml(FlumeMaster.java:327) > at org.apache.jsp.flumemaster_jsp._jspService(flumemaster_jsp.java:68) > at org.apache.jasper.runtime.HttpJspBase.service(HttpJspBase.java:94) > at javax.servlet.http.HttpServlet.service(HttpServlet.java:802) > at > org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:427) > at > org.mortbay.jetty.servlet.WebApplicationHandler.dispatch(WebApplicationHandler.java:475) > at > org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:567) > at org.mortbay.http.HttpContext.handle(HttpContext.java:1565) > at > org.mortbay.jetty.servlet.WebApplicationContext.handle(WebApplicationConte
[jira] [Resolved] (FLUME-79) Custom Text Formated Output
[
https://issues.apache.org/jira/browse/FLUME-79?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ashish Paliwal resolved FLUME-79.
-
Resolution: Won't Fix
Fix Version/s: v0.9.5
Won't fix. 0.X branch not maintained anymore
> Custom Text Formated Output
> ---
>
> Key: FLUME-79
> URL: https://issues.apache.org/jira/browse/FLUME-79
> Project: Flume
> Issue Type: New Feature
> Components: Sinks+Sources
>Affects Versions: v0.9.0
>Reporter: Nicholas Verbeck
> Fix For: v0.9.5
>
>
> Create a new output format to allow for a custom text based output of log
> info. Example would be tab or comma delimited output.
> This would result in a new option for the flume.collector.output.format
> property of formatedtext. And an optional config property of
> flume.collector.output.format.str where its value is a formated string where
> a text replace will be preformed upon. If not set the default will be
> "%{host}\t%{body}"
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (FLUME-62) add a logging (log4j) decorator/sink
[
https://issues.apache.org/jira/browse/FLUME-62?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ashish Paliwal resolved FLUME-62.
-
Resolution: Won't Fix
Fix Version/s: v0.9.5
Won't fix. 0.X branch not maintained anymore
> add a logging (log4j) decorator/sink
>
>
> Key: FLUME-62
> URL: https://issues.apache.org/jira/browse/FLUME-62
> Project: Flume
> Issue Type: New Feature
> Components: Sinks+Sources
>Affects Versions: v0.9.4
>Reporter: Patrick Hunt
> Fix For: v0.9.5
>
>
> Would be great to have a logging sink (maybe it should be a decorator?)
> available so that users could more easily debug their flows, esp when getting
> started or if problems arise.
> For now one could do:
> [ normalSink, console]
> [ normalsink, {probSampler(0.1) => console} ]
> (10%chance of writting event to console)
> or use a text sink instead, maybe with a format decorator.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (FLUME-59) Membase sink
[ https://issues.apache.org/jira/browse/FLUME-59?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Paliwal resolved FLUME-59. - Resolution: Won't Fix Fix Version/s: v0.9.5 Won't fix. 0.X branch not maintained anymore > Membase sink > > > Key: FLUME-59 > URL: https://issues.apache.org/jira/browse/FLUME-59 > Project: Flume > Issue Type: New Feature > Components: Sinks+Sources >Affects Versions: v0.9.4 >Reporter: Disabled imported user > Fix For: v0.9.5 > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (FLUME-49) flume's time escaping should deal with daylight savings / standard time issues.
[ https://issues.apache.org/jira/browse/FLUME-49?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Paliwal resolved FLUME-49. - Resolution: Won't Fix Fix Version/s: v0.9.5 Won't fix. 0.X branch not maintained anymore > flume's time escaping should deal with daylight savings / standard time > issues. > --- > > Key: FLUME-49 > URL: https://issues.apache.org/jira/browse/FLUME-49 > Project: Flume > Issue Type: Improvement > Components: Node, Sinks+Sources >Affects Versions: v0.9.0 >Reporter: Jonathan Hsieh >Assignee: Alexander Alten-Lorenz >Priority: Minor > Fix For: v0.9.5 > > > Inspired by a talk about oozie. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (FLUME-40) Use @Categories in JUnit tests
[ https://issues.apache.org/jira/browse/FLUME-40?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Paliwal resolved FLUME-40. - Resolution: Won't Fix Fix Version/s: v0.9.5 Won't fix. 0.X branch not maintained anymore > Use @Categories in JUnit tests > -- > > Key: FLUME-40 > URL: https://issues.apache.org/jira/browse/FLUME-40 > Project: Flume > Issue Type: Improvement > Components: Test >Affects Versions: v0.9.4 >Reporter: Jonathan Hsieh >Priority: Minor > Fix For: v0.9.5 > > > This is a new JUnit 4.8 feature that can be used to group slow tests, and > tests for different subsystems, or other categories like the torture or perf > tests. > Look at these examples: > http://saffgreenbar.blogspot.com/2009/11/heres-test-code-snippet-public.html > http://weblogs.java.net/blog/johnsmart/archive/2010/04/25/grouping-tests-using-junit-categories-0 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (FLUME-39) Support for collector rebalancing
[ https://issues.apache.org/jira/browse/FLUME-39?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Paliwal resolved FLUME-39. - Resolution: Won't Fix Fix Version/s: v0.9.5 Won't fix. 0.X branch not maintained anymore > Support for collector rebalancing > - > > Key: FLUME-39 > URL: https://issues.apache.org/jira/browse/FLUME-39 > Project: Flume > Issue Type: New Feature > Components: Master >Affects Versions: v0.9.0 >Reporter: Disabled imported user >Priority: Minor > Fix For: v0.9.5 > > > Currently, mappings between nodes and collectors change when nodes become > disconnected, or when new nodes join the cluster (though through consistent > hashing, most existing connections do not wind up getting remapped, > preventing the system from thrashing about too much). It would be interesting > to allow for custom FailoverChainManager implementations that can adjust > mappings based on custom load-based policies, as this would allow one to deal > with situations when the collectors are not equivalent (due to bad or busy > machines, unexpectedly skewed load, etc). The load could be retrieved from > stats in ReportManager. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (FLUME-35) Documentation of counters / ganglia integration
[ https://issues.apache.org/jira/browse/FLUME-35?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Paliwal resolved FLUME-35. - Resolution: Won't Fix Fix Version/s: v0.9.5 Won't fix. 0.X branch not maintained anymore > Documentation of counters / ganglia integration > --- > > Key: FLUME-35 > URL: https://issues.apache.org/jira/browse/FLUME-35 > Project: Flume > Issue Type: Improvement > Components: Docs >Affects Versions: v0.9.4 >Reporter: Disabled imported user > Fix For: v0.9.5 > > > There is currently no documentation of the counters Flume exposes, and how it > can be integrated with Ganglia. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (FLUME-23) agent*[sink|chain] should be translated instead of using 'AgentSink' and 'AgentFailOverSink'
[ https://issues.apache.org/jira/browse/FLUME-23?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Paliwal updated FLUME-23: Fix Version/s: (was: v1.6.0) v0.9.5 > agent*[sink|chain] should be translated instead of using 'AgentSink' and > 'AgentFailOverSink' > > > Key: FLUME-23 > URL: https://issues.apache.org/jira/browse/FLUME-23 > Project: Flume > Issue Type: Improvement > Components: Master, Node >Affects Versions: v0.9.0 >Reporter: Jonathan Hsieh > Fix For: v0.9.5 > > > Currently, 'agentSink' and 'agentFaiiOverSink' are implemented as opaque > sinks. They should instead use the translation mechanism to allow for easier > debugging and to offer easier explanation of how reliability mechanisms work. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (FLUME-32) Configuration verification mode
[ https://issues.apache.org/jira/browse/FLUME-32?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Paliwal resolved FLUME-32. - Resolution: Won't Fix Fix Version/s: v0.9.5 Won't fix. 0.X branch not maintained anymore > Configuration verification mode > --- > > Key: FLUME-32 > URL: https://issues.apache.org/jira/browse/FLUME-32 > Project: Flume > Issue Type: Improvement > Components: Master >Affects Versions: v0.9.4 >Reporter: Disabled imported user >Assignee: Patrick Hunt > Fix For: v0.9.5 > > > There is currently no way to sanity-check configuration commands prior to > actually issuing them. > Maybe some sort of a dry-run mode? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (FLUME-16) Deprecate com.cloudera.flume.core.Attributes
[ https://issues.apache.org/jira/browse/FLUME-16?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Paliwal updated FLUME-16: Fix Version/s: v0.9.5 > Deprecate com.cloudera.flume.core.Attributes > > > Key: FLUME-16 > URL: https://issues.apache.org/jira/browse/FLUME-16 > Project: Flume > Issue Type: Improvement > Components: Master, Node, Sinks+Sources >Affects Versions: v0.9.4 >Reporter: Jonathan Hsieh >Assignee: Alexander Alten-Lorenz > Fix For: v0.9.5 > > > This old API is being replaced with a get/put*Metric API found in ReportEvent -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (FLUME-5) Add HTTP POST Sink
[ https://issues.apache.org/jira/browse/FLUME-5?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Paliwal updated FLUME-5: --- Fix Version/s: (was: v1.6.0) v0.9.5 > Add HTTP POST Sink > -- > > Key: FLUME-5 > URL: https://issues.apache.org/jira/browse/FLUME-5 > Project: Flume > Issue Type: New Feature > Components: Sinks+Sources >Affects Versions: v0.9.4 >Reporter: Disabled imported user >Assignee: Alexander Alten-Lorenz > Fix For: v0.9.5 > > > For web dashboards, it's useful to have Flume just fire an HTTP POST with > event data to a waiting URL. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (FLUME-23) agent*[sink|chain] should be translated instead of using 'AgentSink' and 'AgentFailOverSink'
[ https://issues.apache.org/jira/browse/FLUME-23?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Paliwal resolved FLUME-23. - Resolution: Won't Fix Fix Version/s: v1.6.0 Won't fix. 0.X branch not maintained anymore > agent*[sink|chain] should be translated instead of using 'AgentSink' and > 'AgentFailOverSink' > > > Key: FLUME-23 > URL: https://issues.apache.org/jira/browse/FLUME-23 > Project: Flume > Issue Type: Improvement > Components: Master, Node >Affects Versions: v0.9.0 >Reporter: Jonathan Hsieh > Fix For: v1.6.0 > > > Currently, 'agentSink' and 'agentFaiiOverSink' are implemented as opaque > sinks. They should instead use the translation mechanism to allow for easier > debugging and to offer easier explanation of how reliability mechanisms work. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (FLUME-16) Deprecate com.cloudera.flume.core.Attributes
[ https://issues.apache.org/jira/browse/FLUME-16?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Paliwal updated FLUME-16: Fix Version/s: (was: v1.6.0) > Deprecate com.cloudera.flume.core.Attributes > > > Key: FLUME-16 > URL: https://issues.apache.org/jira/browse/FLUME-16 > Project: Flume > Issue Type: Improvement > Components: Master, Node, Sinks+Sources >Affects Versions: v0.9.4 >Reporter: Jonathan Hsieh >Assignee: Alexander Alten-Lorenz > > This old API is being replaced with a get/put*Metric API found in ReportEvent -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (FLUME-16) Deprecate com.cloudera.flume.core.Attributes
[ https://issues.apache.org/jira/browse/FLUME-16?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Paliwal resolved FLUME-16. - Resolution: Won't Fix Fix Version/s: v1.6.0 Won't fix. 0.X branch not maintained anymore > Deprecate com.cloudera.flume.core.Attributes > > > Key: FLUME-16 > URL: https://issues.apache.org/jira/browse/FLUME-16 > Project: Flume > Issue Type: Improvement > Components: Master, Node, Sinks+Sources >Affects Versions: v0.9.4 >Reporter: Jonathan Hsieh >Assignee: Alexander Alten-Lorenz > Fix For: v1.6.0 > > > This old API is being replaced with a get/put*Metric API found in ReportEvent -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (FLUME-5) Add HTTP POST Sink
[ https://issues.apache.org/jira/browse/FLUME-5?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Paliwal resolved FLUME-5. Resolution: Won't Fix Fix Version/s: v1.6.0 Won't fix. 0.X branch not maintained anymore > Add HTTP POST Sink > -- > > Key: FLUME-5 > URL: https://issues.apache.org/jira/browse/FLUME-5 > Project: Flume > Issue Type: New Feature > Components: Sinks+Sources >Affects Versions: v0.9.4 >Reporter: Disabled imported user >Assignee: Alexander Alten-Lorenz > Fix For: v1.6.0 > > > For web dashboards, it's useful to have Flume just fire an HTTP POST with > event data to a waiting URL. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
Re: Flume 0.X JIRA's
There are around 351 JIRA's on pre 1.X release. I shall look at them and mark them as won't fix selectively. If there are features that can be moved to 1.X series, shall post them on ML. Apologies in advance for flooding the Inbox. On Tue, Nov 4, 2014 at 2:28 PM, Ashish wrote: > @Otis - Neither am I :) > > Lets wait for a day or so, if no one objects I shall update the status in JIRA > > On Tue, Nov 4, 2014 at 10:39 AM, Hari Shreedharan > wrote: >> +1. >> >> >> >> Thanks, >> Hari >> >> On Mon, Nov 3, 2014 at 1:45 AM, Ashish wrote: >> >>> Folks, >>> Whats the plan for pre-FlumeNG jira's? To me the branch seems dead. >>> IMHO, we can mark the ticket's as Won't Fix and clean up JIRA. >>> wdyt? >>> thanks >>> ashish > > > > -- > thanks > ashish > > Blog: http://www.ashishpaliwal.com/blog > My Photo Galleries: http://www.pbase.com/ashishpaliwal -- thanks ashish Blog: http://www.ashishpaliwal.com/blog My Photo Galleries: http://www.pbase.com/ashishpaliwal
Re: [ANNOUNCE] New Flume PMC Member - Roshan Naik
Congrats Roshan! On Tue, Nov 4, 2014 at 2:12 PM, Arvind Prabhakar wrote: > On behalf of Apache Flume PMC, it is my pleasure to announce that Roshan > Naik has been elected to the Flume Project Management Committee. Roshan has > been active with the project for many years and has been a committer on the > project since September of 2013. > > Please join me in congratulating Roshan and welcoming him to the Flume PMC. > > Regards, > Arvind Prabhakar >
[jira] [Commented] (FLUME-1335) Make Flume running on Windows
[ https://issues.apache.org/jira/browse/FLUME-1335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14197260#comment-14197260 ] Roshan Naik commented on FLUME-1335: linking remaining windows jiras > Make Flume running on Windows > - > > Key: FLUME-1335 > URL: https://issues.apache.org/jira/browse/FLUME-1335 > Project: Flume > Issue Type: Improvement > Components: Configuration, Sinks+Sources, Windows >Affects Versions: v1.2.0 >Reporter: Alexander Alten-Lorenz >Assignee: Alexander Alten-Lorenz > > Make Flume running on Windows, targeted versions Windows Server 2008 / > Windows 7 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (FLUME-2536) Windows : Fix various unit test failures
[ https://issues.apache.org/jira/browse/FLUME-2536?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Roshan Naik updated FLUME-2536: --- Description: This is umbrella jira for fixing various windows test failures only. Remaining windows jiras are listed under FLUME-1335 which master Windows umbrella jira. was:This is umbrella jira for fixing various windows test failures. > Windows : Fix various unit test failures > > > Key: FLUME-2536 > URL: https://issues.apache.org/jira/browse/FLUME-2536 > Project: Flume > Issue Type: Bug > Components: Test, Windows >Affects Versions: v1.5.0.1 >Reporter: Roshan Naik >Assignee: Roshan Naik > Labels: windows > > This is umbrella jira for fixing various windows test failures only. > Remaining windows jiras are listed under FLUME-1335 which master Windows > umbrella jira. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (FLUME-2508) Windows : LineDeserializer causes incorrect behavior in SpoolDir source on Windows
[ https://issues.apache.org/jira/browse/FLUME-2508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Roshan Naik updated FLUME-2508: --- Labels: windows (was: Windows) > Windows : LineDeserializer causes incorrect behavior in SpoolDir source on > Windows > -- > > Key: FLUME-2508 > URL: https://issues.apache.org/jira/browse/FLUME-2508 > Project: Flume > Issue Type: Bug > Components: Sinks+Sources, Windows >Affects Versions: v1.5.0.1 >Reporter: Roshan Naik >Assignee: Roshan Naik > Labels: windows > Attachments: FLUME-2508.patch > > > LineDeserializer does not handle CRLF correctly when reading files -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (FLUME-2149) Windows: Temporary files generated by File Channel unit tests are not cleaned up properly
[ https://issues.apache.org/jira/browse/FLUME-2149?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Roshan Naik updated FLUME-2149: --- Issue Type: Sub-task (was: Bug) Parent: FLUME-2536 > Windows: Temporary files generated by File Channel unit tests are not cleaned > up properly > - > > Key: FLUME-2149 > URL: https://issues.apache.org/jira/browse/FLUME-2149 > Project: Flume > Issue Type: Sub-task > Components: Windows >Affects Versions: v1.4.0 >Reporter: Roshan Naik >Assignee: Roshan Naik > Labels: windows > > TestFileChannelBase.teardown() does not appear to be cleaning up properly on > Windows -- This message was sent by Atlassian JIRA (v6.3.4#6332)
Re: [ANNOUNCE] New Flume PMC Member - Roshan Naik
Congratulations Roshan! Jarcec > On Nov 4, 2014, at 2:12 PM, Arvind Prabhakar wrote: > > On behalf of Apache Flume PMC, it is my pleasure to announce that Roshan Naik > has been elected to the Flume Project Management Committee. Roshan has been > active with the project for many years and has been a committer on the > project since September of 2013. > > Please join me in congratulating Roshan and welcoming him to the Flume PMC. > > Regards, > Arvind Prabhakar
Re: [ANNOUNCE] New Flume PMC Member - Roshan Naik
Congrats :) On Wed, Nov 5, 2014 at 3:42 AM, Arvind Prabhakar wrote: > On behalf of Apache Flume PMC, it is my pleasure to announce that Roshan > Naik has been elected to the Flume Project Management Committee. Roshan has > been active with the project for many years and has been a committer on the > project since September of 2013. > > Please join me in congratulating Roshan and welcoming him to the Flume PMC. > > Regards, > Arvind Prabhakar -- thanks ashish Blog: http://www.ashishpaliwal.com/blog My Photo Galleries: http://www.pbase.com/ashishpaliwal
[jira] [Commented] (FLUME-2534) Windows : TestFlumeEventQueue,TestCheckpoint & TestCheckpointRebuilder failures
[
https://issues.apache.org/jira/browse/FLUME-2534?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14197203#comment-14197203
]
Roshan Naik commented on FLUME-2534:
[~hshreedharan] FLUME-2536 is the umbrella jira.
> Windows : TestFlumeEventQueue,TestCheckpoint & TestCheckpointRebuilder
> failures
> ---
>
> Key: FLUME-2534
> URL: https://issues.apache.org/jira/browse/FLUME-2534
> Project: Flume
> Issue Type: Sub-task
> Components: File Channel, Test
>Affects Versions: v1.5.0.1
>Reporter: Roshan Naik
>Assignee: Roshan Naik
> Labels: windows
> Attachments: FLUME-2534.patch
>
>
> lots of failures .. need to ensure queue is closed properly before creating a
> new one. Otherwise files are not closed before deletion
> {code}
> org.apache.flume.channel.file.TestFlumeEventQueue.addHeadRemove2[0] 29 ms
> 94
> org.apache.flume.channel.file.TestFlumeEventQueue.testWrappingCorrectly[0]
> 18 ms 94
> org.apache.flume.channel.file.TestFlumeEventQueue.testInflightPuts[0]
> 13 ms 94
> org.apache.flume.channel.file.TestFlumeEventQueue.testInflightTakes[0]
> 30 ms 94
> org.apache.flume.channel.file.TestFlumeEventQueue.testCorruptInflightPuts[0]
> 11 ms 94
>
> org.apache.flume.channel.file.TestFlumeEventQueue.testCorruptInflightTakes[0]
>29 ms 94
> org.apache.flume.channel.file.TestFlumeEventQueue.addTail1[0]34 ms
> 94
> org.apache.flume.channel.file.TestFlumeEventQueue.addTail2[0]17 ms
> 94
> org.apache.flume.channel.file.TestFlumeEventQueue.addTailLarge[0]27 ms
> 94
> org.apache.flume.channel.file.TestFlumeEventQueue.addHead1[0]10 ms
> 94
> org.apache.flume.channel.file.TestFlumeEventQueue.addHead2[0]15 ms
> 94
> org.apache.flume.channel.file.TestFlumeEventQueue.addHeadLarge[0]26 ms
> 94
>
> org.apache.flume.channel.file.TestFlumeEventQueue.testUnknownPointerDoesNotCauseSearch[0]
> 15 ms 94
>
> org.apache.flume.channel.file.TestFlumeEventQueue.testRemoveAfterReplayComplete[0]
>12 ms 94
> org.apache.flume.channel.file.TestFlumeEventQueue.addHeadRemove2[1] 32 ms
> 94
> org.apache.flume.channel.file.TestFlumeEventQueue.testWrappingCorrectly[1]
> 0.11 sec94
> org.apache.flume.channel.file.TestFlumeEventQueue.testInflightPuts[1]
> 46 ms 94
> org.apache.flume.channel.file.TestFlumeEventQueue.testInflightTakes[1]
> 22 ms 94
> org.apache.flume.channel.file.TestFlumeEventQueue.testCorruptInflightPuts[1]
> 44 ms 94
>
> org.apache.flume.channel.file.TestFlumeEventQueue.testCorruptInflightTakes[1]
>0.21 sec94
> org.apache.flume.channel.file.TestFlumeEventQueue.addTail1[1]36 ms
> 94
> org.apache.flume.channel.file.TestFlumeEventQueue.addTail2[1]36 ms
> 94
> org.apache.flume.channel.file.TestFlumeEventQueue.addTailLarge[1]48 ms
> 94
> org.apache.flume.channel.file.TestFlumeEventQueue.addHead1[1]46 ms
> 94
> org.apache.flume.channel.file.TestFlumeEventQueue.addHead2[1]24 ms
> 94
> org.apache.flume.channel.file.TestFlumeEventQueue.addHeadLarge[1]44 ms
> 94
>
> org.apache.flume.channel.file.TestFlumeEventQueue.testUnknownPointerDoesNotCauseSearch[1]
> 28 ms 94
>
> org.apache.flume.channel.file.TestFlumeEventQueue.testRemoveAfterReplayComplete[1]
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (FLUME-2536) Windows : Fix various unit test failures
[ https://issues.apache.org/jira/browse/FLUME-2536?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Roshan Naik updated FLUME-2536: --- Description: This is umbrella jira for fixing various windows test failures. > Windows : Fix various unit test failures > > > Key: FLUME-2536 > URL: https://issues.apache.org/jira/browse/FLUME-2536 > Project: Flume > Issue Type: Bug > Components: Test, Windows >Affects Versions: v1.5.0.1 >Reporter: Roshan Naik >Assignee: Roshan Naik > Labels: windows > > This is umbrella jira for fixing various windows test failures. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Issue Comment Deleted] (FLUME-2536) Windows : Fix various unit test failures
[ https://issues.apache.org/jira/browse/FLUME-2536?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Roshan Naik updated FLUME-2536: --- Comment: was deleted (was: This is umbrella jira for fixing various windows test failures.) > Windows : Fix various unit test failures > > > Key: FLUME-2536 > URL: https://issues.apache.org/jira/browse/FLUME-2536 > Project: Flume > Issue Type: Bug > Components: Test, Windows >Affects Versions: v1.5.0.1 >Reporter: Roshan Naik >Assignee: Roshan Naik > Labels: windows > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (FLUME-2137) Fix StagedInstall.java to invoke the correct startup script on Windows
[ https://issues.apache.org/jira/browse/FLUME-2137?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Roshan Naik updated FLUME-2137: --- Issue Type: Sub-task (was: Bug) Parent: FLUME-2536 > Fix StagedInstall.java to invoke the correct startup script on Windows > -- > > Key: FLUME-2137 > URL: https://issues.apache.org/jira/browse/FLUME-2137 > Project: Flume > Issue Type: Sub-task > Components: Test, Windows >Affects Versions: v1.4.0 >Reporter: Roshan Naik >Assignee: Roshan Naik > Labels: windows > Fix For: v1.4.1, v1.6.0 > > Attachments: FLUME-2137.patch > > > StagedInstall fails during Integration test run on Windows .. as it invokes > the unix startup script. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (FLUME-2219) Windows: Flume tests need to know location of Hadoop native libraries (hadoop.dll)
[ https://issues.apache.org/jira/browse/FLUME-2219?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Roshan Naik updated FLUME-2219: --- Issue Type: Sub-task (was: Bug) Parent: FLUME-2536 > Windows: Flume tests need to know location of Hadoop native libraries > (hadoop.dll) > -- > > Key: FLUME-2219 > URL: https://issues.apache.org/jira/browse/FLUME-2219 > Project: Flume > Issue Type: Sub-task >Affects Versions: v1.4.0 >Reporter: Roshan Naik >Assignee: Roshan Naik > Labels: hadoop, hdfs, sink, windows > Attachments: FLUME-2219.patch > > > Hadoop uses some native libraries on Windows which are need for HDFS sink > unit tests at runtime. > Modify pom.xml so that surefire plugin is able to setup java.library.path to > include hadoop native libraries. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (FLUME-2146) Windows: Tmp file creation in TestBodyTextEventSerializer.java needs fixing
[ https://issues.apache.org/jira/browse/FLUME-2146?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Roshan Naik updated FLUME-2146: --- Issue Type: Sub-task (was: Bug) Parent: FLUME-2536 > Windows: Tmp file creation in TestBodyTextEventSerializer.java needs fixing > --- > > Key: FLUME-2146 > URL: https://issues.apache.org/jira/browse/FLUME-2146 > Project: Flume > Issue Type: Sub-task > Components: Windows >Reporter: Roshan Naik >Assignee: Roshan Naik > Labels: windows > Attachments: FLUME-2146.patch > > > avoid using hard coded path + filename for temp file -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (FLUME-2218) Windows: TestFileChannelIntegrityTool tests failing due to inability to delete checkpoint during tearDown
[ https://issues.apache.org/jira/browse/FLUME-2218?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Roshan Naik updated FLUME-2218: --- Issue Type: Sub-task (was: Bug) Parent: FLUME-2536 > Windows: TestFileChannelIntegrityTool tests failing due to inability to > delete checkpoint during tearDown > -- > > Key: FLUME-2218 > URL: https://issues.apache.org/jira/browse/FLUME-2218 > Project: Flume > Issue Type: Sub-task > Components: File Channel, Test, Windows >Affects Versions: v1.4.0 >Reporter: Roshan Naik >Assignee: Roshan Naik > Labels: test, windows > Attachments: FLUME-2218.patch > > > As checkpoints are memory mapped and java does not support unmapping, windows > does not allow deletion of files in use. > *Error Message* > Unable to delete file: > C:\Users\jenkins\AppData\Local\Temp\1382481767920-0\checkpoint\checkpoint_1382481770668 > *Stacktrace* > java.io.IOException: Unable to delete file: > C:\Users\jenkins\AppData\Local\Temp\1382481767920-0\checkpoint\checkpoint_1382481770668 > at org.apache.commons.io.FileUtils.forceDelete(FileUtils.java:1919) > at org.apache.commons.io.FileUtils.cleanDirectory(FileUtils.java:1399) > at org.apache.commons.io.FileUtils.deleteDirectory(FileUtils.java:1331) > at > org.apache.flume.tools.TestFileChannelIntegrityTool.tearDown(TestFileChannelIntegrityTool.java:89) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) > at java.lang.reflect.Method.invoke(Method.java:597) > at > org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45) > at > org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15) > at > org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42) > at > org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:36) > at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:263) > at > org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:68) > at > org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:47) > at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231) > at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:60) > at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:229) > at org.junit.runners.ParentRunner.access$000(ParentRunner.java:50) > at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:222) > at > org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28) > at > org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:30) > at org.junit.runners.ParentRunner.run(ParentRunner.java:300) > at > org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:252) > at > org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:141) > at > org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:112) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) > at java.lang.reflect.Method.invoke(Method.java:597) > at > org.apache.maven.surefire.util.ReflectionUtils.invokeMethodWithArray(ReflectionUtils.java:189) > at > org.apache.maven.surefire.booter.ProviderFactory$ProviderProxy.invoke(ProviderFactory.java:165) > at > org.apache.maven.surefire.booter.ProviderFactory.invokeProvider(ProviderFactory.java:85) > at > org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:115) > at > org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:75) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (FLUME-2150) Windows: TestFileChannelEncryption needs to be a bit more conservative when examining the exception message
[
https://issues.apache.org/jira/browse/FLUME-2150?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Roshan Naik updated FLUME-2150:
---
Issue Type: Sub-task (was: Bug)
Parent: FLUME-2536
> Windows: TestFileChannelEncryption needs to be a bit more conservative when
> examining the exception message
>
>
> Key: FLUME-2150
> URL: https://issues.apache.org/jira/browse/FLUME-2150
> Project: Flume
> Issue Type: Sub-task
> Components: Windows
>Affects Versions: v1.4.0
>Reporter: Roshan Naik
>Assignee: Roshan Naik
> Labels: windows
> Attachments: FLUME-2150.patch
>
>
> Current assertion is :
> Assert.assertEquals("java.io.FileNotFoundException: /path/does/not/exist " +
> ...)
> On Windows the message is a bit different The tail end part /path/.. is not
> same.
> Suffices to check the prefix to be platform independent:
> Assert.assertTrue(ex.getMessage().startsWith("java.io.FileNotFoundException"));
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (FLUME-2532) Windows : TestReliableSpoolingFileEventReader - need to close filereader
[ https://issues.apache.org/jira/browse/FLUME-2532?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Roshan Naik updated FLUME-2532: --- Issue Type: Sub-task (was: Bug) Parent: FLUME-2536 > Windows : TestReliableSpoolingFileEventReader - need to close filereader > > > Key: FLUME-2532 > URL: https://issues.apache.org/jira/browse/FLUME-2532 > Project: Flume > Issue Type: Sub-task >Affects Versions: v1.5.0.1 >Reporter: Roshan Naik >Assignee: Roshan Naik > Labels: windows > Attachments: FLUME-2532.patch > > > Need to close file reader so that the files can be deleted. windows does not > allow deletion of files that are open. > Also need to skip testConsumeFileYoungest which is very flaky on windows as > the timestamps of the files it created are not as expected. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (FLUME-2136) Windows - Fix intermitent test failure in TestMonitoredCounterGroup.java
[ https://issues.apache.org/jira/browse/FLUME-2136?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Roshan Naik updated FLUME-2136: --- Issue Type: Sub-task (was: Bug) Parent: FLUME-2536 > Windows - Fix intermitent test failure in TestMonitoredCounterGroup.java > > > Key: FLUME-2136 > URL: https://issues.apache.org/jira/browse/FLUME-2136 > Project: Flume > Issue Type: Sub-task > Components: Test, Windows >Affects Versions: v1.4.0 >Reporter: Roshan Naik >Assignee: Roshan Naik > Labels: windows > Fix For: v1.4.1, v1.6.0 > > Attachments: FLUME-2136.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (FLUME-2534) Windows : TestFlumeEventQueue,TestCheckpoint & TestCheckpointRebuilder failures
[
https://issues.apache.org/jira/browse/FLUME-2534?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Roshan Naik updated FLUME-2534:
---
Issue Type: Sub-task (was: Bug)
Parent: FLUME-2536
> Windows : TestFlumeEventQueue,TestCheckpoint & TestCheckpointRebuilder
> failures
> ---
>
> Key: FLUME-2534
> URL: https://issues.apache.org/jira/browse/FLUME-2534
> Project: Flume
> Issue Type: Sub-task
> Components: File Channel, Test
>Affects Versions: v1.5.0.1
>Reporter: Roshan Naik
>Assignee: Roshan Naik
> Labels: windows
> Attachments: FLUME-2534.patch
>
>
> lots of failures .. need to ensure queue is closed properly before creating a
> new one. Otherwise files are not closed before deletion
> {code}
> org.apache.flume.channel.file.TestFlumeEventQueue.addHeadRemove2[0] 29 ms
> 94
> org.apache.flume.channel.file.TestFlumeEventQueue.testWrappingCorrectly[0]
> 18 ms 94
> org.apache.flume.channel.file.TestFlumeEventQueue.testInflightPuts[0]
> 13 ms 94
> org.apache.flume.channel.file.TestFlumeEventQueue.testInflightTakes[0]
> 30 ms 94
> org.apache.flume.channel.file.TestFlumeEventQueue.testCorruptInflightPuts[0]
> 11 ms 94
>
> org.apache.flume.channel.file.TestFlumeEventQueue.testCorruptInflightTakes[0]
>29 ms 94
> org.apache.flume.channel.file.TestFlumeEventQueue.addTail1[0]34 ms
> 94
> org.apache.flume.channel.file.TestFlumeEventQueue.addTail2[0]17 ms
> 94
> org.apache.flume.channel.file.TestFlumeEventQueue.addTailLarge[0]27 ms
> 94
> org.apache.flume.channel.file.TestFlumeEventQueue.addHead1[0]10 ms
> 94
> org.apache.flume.channel.file.TestFlumeEventQueue.addHead2[0]15 ms
> 94
> org.apache.flume.channel.file.TestFlumeEventQueue.addHeadLarge[0]26 ms
> 94
>
> org.apache.flume.channel.file.TestFlumeEventQueue.testUnknownPointerDoesNotCauseSearch[0]
> 15 ms 94
>
> org.apache.flume.channel.file.TestFlumeEventQueue.testRemoveAfterReplayComplete[0]
>12 ms 94
> org.apache.flume.channel.file.TestFlumeEventQueue.addHeadRemove2[1] 32 ms
> 94
> org.apache.flume.channel.file.TestFlumeEventQueue.testWrappingCorrectly[1]
> 0.11 sec94
> org.apache.flume.channel.file.TestFlumeEventQueue.testInflightPuts[1]
> 46 ms 94
> org.apache.flume.channel.file.TestFlumeEventQueue.testInflightTakes[1]
> 22 ms 94
> org.apache.flume.channel.file.TestFlumeEventQueue.testCorruptInflightPuts[1]
> 44 ms 94
>
> org.apache.flume.channel.file.TestFlumeEventQueue.testCorruptInflightTakes[1]
>0.21 sec94
> org.apache.flume.channel.file.TestFlumeEventQueue.addTail1[1]36 ms
> 94
> org.apache.flume.channel.file.TestFlumeEventQueue.addTail2[1]36 ms
> 94
> org.apache.flume.channel.file.TestFlumeEventQueue.addTailLarge[1]48 ms
> 94
> org.apache.flume.channel.file.TestFlumeEventQueue.addHead1[1]46 ms
> 94
> org.apache.flume.channel.file.TestFlumeEventQueue.addHead2[1]24 ms
> 94
> org.apache.flume.channel.file.TestFlumeEventQueue.addHeadLarge[1]44 ms
> 94
>
> org.apache.flume.channel.file.TestFlumeEventQueue.testUnknownPointerDoesNotCauseSearch[1]
> 28 ms 94
>
> org.apache.flume.channel.file.TestFlumeEventQueue.testRemoveAfterReplayComplete[1]
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (FLUME-2058) TestFlumeEventQueue in FileChannel fails on Windows
[ https://issues.apache.org/jira/browse/FLUME-2058?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Roshan Naik updated FLUME-2058: --- Issue Type: Sub-task (was: Bug) Parent: FLUME-2536 > TestFlumeEventQueue in FileChannel fails on Windows > > > Key: FLUME-2058 > URL: https://issues.apache.org/jira/browse/FLUME-2058 > Project: Flume > Issue Type: Sub-task > Components: File Channel, Test, Windows >Affects Versions: v1.3.1 >Reporter: Roshan Naik >Assignee: Roshan Naik > Labels: windows > Attachments: FLUME-2058.patch > > > Windows does not allow deletion of files that are in use. Java does not yet > allow explicitly unmapping memory mapped files so until these files are > unmapped by the GC, we cannot delete them explicitly. > Checkpoint files are being reused across tests. Deletion of checkpoint files > fails in these tests as the file memory mapped. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (FLUME-2058) TestFlumeEventQueue in FileChannel fails on Windows
[ https://issues.apache.org/jira/browse/FLUME-2058?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Roshan Naik updated FLUME-2058: --- Labels: windows (was: ) > TestFlumeEventQueue in FileChannel fails on Windows > > > Key: FLUME-2058 > URL: https://issues.apache.org/jira/browse/FLUME-2058 > Project: Flume > Issue Type: Bug > Components: File Channel, Test, Windows >Affects Versions: v1.3.1 >Reporter: Roshan Naik >Assignee: Roshan Naik > Labels: windows > Attachments: FLUME-2058.patch > > > Windows does not allow deletion of files that are in use. Java does not yet > allow explicitly unmapping memory mapped files so until these files are > unmapped by the GC, we cannot delete them explicitly. > Checkpoint files are being reused across tests. Deletion of checkpoint files > fails in these tests as the file memory mapped. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (FLUME-2359) TestFileChannelIntegrityTool throws exception on class teardown on Windows
[
https://issues.apache.org/jira/browse/FLUME-2359?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Roshan Naik updated FLUME-2359:
---
Issue Type: Sub-task (was: Bug)
Parent: FLUME-2536
> TestFileChannelIntegrityTool throws exception on class teardown on Windows
> --
>
> Key: FLUME-2359
> URL: https://issues.apache.org/jira/browse/FLUME-2359
> Project: Flume
> Issue Type: Sub-task
> Components: File Channel, Test
>Affects Versions: v1.4.0
>Reporter: Roshan Naik
>Assignee: Roshan Naik
> Labels: windows
> Attachments: FLUME-2359.patch
>
>
> Exception is due to inability to delete checkpoint directory. Since there are
> memory mapped files there, this directory cannot be deleted from within the
> VM.
> {code}
> java.io.IOException: Unable to delete file:
> D:\tmp\flume-baikal-GA-1.4.0\1397096136128-0\chkpt\checkpoint_1397096136398
> at org.apache.commons.io.FileUtils.forceDelete(FileUtils.java:1919)
> at org.apache.commons.io.FileUtils.cleanDirectory(FileUtils.java:1399)
> at org.apache.commons.io.FileUtils.deleteDirectory(FileUtils.java:1331)
> at
> org.apache.flume.tools.TestFileChannelIntegrityTool.tearDownClass(TestFileChannelIntegrityTool.java:98)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:601)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42)
> at
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:36)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:300)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:252)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:141)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:112)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:601)
> at
> org.apache.maven.surefire.util.ReflectionUtils.invokeMethodWithArray(ReflectionUtils.java:189)
> at
> org.apache.maven.surefire.booter.ProviderFactory$ProviderProxy.invoke(ProviderFactory.java:165)
> at
> org.apache.maven.surefire.booter.ProviderFactory.invokeProvider(ProviderFactory.java:85)
> at
> org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:115)
> at
> org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:75)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (FLUME-2145) TestCheckpointRebuilder.testFastReplay fails on Windows due to checkpoint file being memory mapped
[ https://issues.apache.org/jira/browse/FLUME-2145?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Roshan Naik updated FLUME-2145: --- Issue Type: Sub-task (was: Bug) Parent: FLUME-2536 > TestCheckpointRebuilder.testFastReplay fails on Windows due to checkpoint > file being memory mapped > --- > > Key: FLUME-2145 > URL: https://issues.apache.org/jira/browse/FLUME-2145 > Project: Flume > Issue Type: Sub-task > Components: Test, Windows >Affects Versions: v1.4.0 >Reporter: Roshan Naik >Assignee: Roshan Naik > Labels: windows > Fix For: v1.6.0 > > Attachments: FLUME-2145.patch, FLUME-2145.v2.patch > > > Memory mapped files are not released by JVM causing failure when attempting > to clean the up. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (FLUME-2536) Windows : Fix various unit test failures
[ https://issues.apache.org/jira/browse/FLUME-2536?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14197180#comment-14197180 ] Roshan Naik commented on FLUME-2536: This is umbrella jira for fixing various windows test failures. > Windows : Fix various unit test failures > > > Key: FLUME-2536 > URL: https://issues.apache.org/jira/browse/FLUME-2536 > Project: Flume > Issue Type: Bug > Components: Test, Windows >Affects Versions: v1.5.0.1 >Reporter: Roshan Naik >Assignee: Roshan Naik > Labels: windows > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (FLUME-2536) Windows : Fix various unit test failures
Roshan Naik created FLUME-2536: -- Summary: Windows : Fix various unit test failures Key: FLUME-2536 URL: https://issues.apache.org/jira/browse/FLUME-2536 Project: Flume Issue Type: Bug Components: Test, Windows Affects Versions: v1.5.0.1 Reporter: Roshan Naik Assignee: Roshan Naik -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (FLUME-2135) Add zip to the build distribution for Windows support
[ https://issues.apache.org/jira/browse/FLUME-2135?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Roshan Naik updated FLUME-2135: --- Labels: windows (was: ) > Add zip to the build distribution for Windows support > - > > Key: FLUME-2135 > URL: https://issues.apache.org/jira/browse/FLUME-2135 > Project: Flume > Issue Type: Bug > Components: Build, Windows >Affects Versions: v1.4.0 >Reporter: Roshan Naik >Assignee: Roshan Naik > Labels: windows > Fix For: v1.4.1, v1.6.0 > > Attachments: FLUME-2135.patch > > > create distro in zip format -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (FLUME-2068) File Channel issue - recovering from BadCheckpoint exception on Windows
[ https://issues.apache.org/jira/browse/FLUME-2068?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Roshan Naik updated FLUME-2068: --- Labels: windows (was: ) > File Channel issue - recovering from BadCheckpoint exception on Windows > --- > > Key: FLUME-2068 > URL: https://issues.apache.org/jira/browse/FLUME-2068 > Project: Flume > Issue Type: Bug > Components: Channel, File Channel, Windows >Affects Versions: v1.3.1 >Reporter: Roshan Naik >Assignee: Roshan Naik > Labels: windows > Attachments: FLUME-1.3.1-2068.patch, FLUME-2068v2.patch > > > In EventQueueBackingStoreFileV3 constructor, if it detects that the > checkpoint and meta files have differing logWriteOrderIds, it throws a > BadCheckpointException. Controls goes back to the exception handler in > Log.replay() which attempts to delete all the files in checkpoint directory > and start fresh. The same file names are reused when starting fresh. > Unfortunately this does not work on Windows since the deletion of the > checkpoint file in the checkpointDir fails. The failure is due to the fact > that the checkpoint file is memory mapped. Unless it is unmapped the deletion > will not succeed... and unfortunately Java does not have unmap support. > Windows does not permit deletion (or renaming) of files in use. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (FLUME-2532) Windows : TestReliableSpoolingFileEventReader - need to close filereader
[ https://issues.apache.org/jira/browse/FLUME-2532?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Roshan Naik updated FLUME-2532: --- Labels: windows (was: Windows) > Windows : TestReliableSpoolingFileEventReader - need to close filereader > > > Key: FLUME-2532 > URL: https://issues.apache.org/jira/browse/FLUME-2532 > Project: Flume > Issue Type: Bug >Affects Versions: v1.5.0.1 >Reporter: Roshan Naik >Assignee: Roshan Naik > Labels: windows > Attachments: FLUME-2532.patch > > > Need to close file reader so that the files can be deleted. windows does not > allow deletion of files that are open. > Also need to skip testConsumeFileYoungest which is very flaky on windows as > the timestamps of the files it created are not as expected. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (FLUME-2534) Windows : TestFlumeEventQueue,TestCheckpoint & TestCheckpointRebuilder failures
[
https://issues.apache.org/jira/browse/FLUME-2534?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Roshan Naik updated FLUME-2534:
---
Labels: windows (was: Windows windows)
> Windows : TestFlumeEventQueue,TestCheckpoint & TestCheckpointRebuilder
> failures
> ---
>
> Key: FLUME-2534
> URL: https://issues.apache.org/jira/browse/FLUME-2534
> Project: Flume
> Issue Type: Bug
> Components: File Channel, Test
>Affects Versions: v1.5.0.1
>Reporter: Roshan Naik
>Assignee: Roshan Naik
> Labels: windows
> Attachments: FLUME-2534.patch
>
>
> lots of failures .. need to ensure queue is closed properly before creating a
> new one. Otherwise files are not closed before deletion
> {code}
> org.apache.flume.channel.file.TestFlumeEventQueue.addHeadRemove2[0] 29 ms
> 94
> org.apache.flume.channel.file.TestFlumeEventQueue.testWrappingCorrectly[0]
> 18 ms 94
> org.apache.flume.channel.file.TestFlumeEventQueue.testInflightPuts[0]
> 13 ms 94
> org.apache.flume.channel.file.TestFlumeEventQueue.testInflightTakes[0]
> 30 ms 94
> org.apache.flume.channel.file.TestFlumeEventQueue.testCorruptInflightPuts[0]
> 11 ms 94
>
> org.apache.flume.channel.file.TestFlumeEventQueue.testCorruptInflightTakes[0]
>29 ms 94
> org.apache.flume.channel.file.TestFlumeEventQueue.addTail1[0]34 ms
> 94
> org.apache.flume.channel.file.TestFlumeEventQueue.addTail2[0]17 ms
> 94
> org.apache.flume.channel.file.TestFlumeEventQueue.addTailLarge[0]27 ms
> 94
> org.apache.flume.channel.file.TestFlumeEventQueue.addHead1[0]10 ms
> 94
> org.apache.flume.channel.file.TestFlumeEventQueue.addHead2[0]15 ms
> 94
> org.apache.flume.channel.file.TestFlumeEventQueue.addHeadLarge[0]26 ms
> 94
>
> org.apache.flume.channel.file.TestFlumeEventQueue.testUnknownPointerDoesNotCauseSearch[0]
> 15 ms 94
>
> org.apache.flume.channel.file.TestFlumeEventQueue.testRemoveAfterReplayComplete[0]
>12 ms 94
> org.apache.flume.channel.file.TestFlumeEventQueue.addHeadRemove2[1] 32 ms
> 94
> org.apache.flume.channel.file.TestFlumeEventQueue.testWrappingCorrectly[1]
> 0.11 sec94
> org.apache.flume.channel.file.TestFlumeEventQueue.testInflightPuts[1]
> 46 ms 94
> org.apache.flume.channel.file.TestFlumeEventQueue.testInflightTakes[1]
> 22 ms 94
> org.apache.flume.channel.file.TestFlumeEventQueue.testCorruptInflightPuts[1]
> 44 ms 94
>
> org.apache.flume.channel.file.TestFlumeEventQueue.testCorruptInflightTakes[1]
>0.21 sec94
> org.apache.flume.channel.file.TestFlumeEventQueue.addTail1[1]36 ms
> 94
> org.apache.flume.channel.file.TestFlumeEventQueue.addTail2[1]36 ms
> 94
> org.apache.flume.channel.file.TestFlumeEventQueue.addTailLarge[1]48 ms
> 94
> org.apache.flume.channel.file.TestFlumeEventQueue.addHead1[1]46 ms
> 94
> org.apache.flume.channel.file.TestFlumeEventQueue.addHead2[1]24 ms
> 94
> org.apache.flume.channel.file.TestFlumeEventQueue.addHeadLarge[1]44 ms
> 94
>
> org.apache.flume.channel.file.TestFlumeEventQueue.testUnknownPointerDoesNotCauseSearch[1]
> 28 ms 94
>
> org.apache.flume.channel.file.TestFlumeEventQueue.testRemoveAfterReplayComplete[1]
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (FLUME-2146) Windows: Tmp file creation in TestBodyTextEventSerializer.java needs fixing
[ https://issues.apache.org/jira/browse/FLUME-2146?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Roshan Naik updated FLUME-2146: --- Labels: windows (was: Windows) > Windows: Tmp file creation in TestBodyTextEventSerializer.java needs fixing > --- > > Key: FLUME-2146 > URL: https://issues.apache.org/jira/browse/FLUME-2146 > Project: Flume > Issue Type: Bug > Components: Windows >Reporter: Roshan Naik >Assignee: Roshan Naik > Labels: windows > Attachments: FLUME-2146.patch > > > avoid using hard coded path + filename for temp file -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (FLUME-2534) Windows : TestFlumeEventQueue,TestCheckpoint & TestCheckpointRebuilder failures
[
https://issues.apache.org/jira/browse/FLUME-2534?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Roshan Naik updated FLUME-2534:
---
Labels: Windows windows (was: Windows)
> Windows : TestFlumeEventQueue,TestCheckpoint & TestCheckpointRebuilder
> failures
> ---
>
> Key: FLUME-2534
> URL: https://issues.apache.org/jira/browse/FLUME-2534
> Project: Flume
> Issue Type: Bug
> Components: File Channel, Test
>Affects Versions: v1.5.0.1
>Reporter: Roshan Naik
>Assignee: Roshan Naik
> Labels: Windows, windows
> Attachments: FLUME-2534.patch
>
>
> lots of failures .. need to ensure queue is closed properly before creating a
> new one. Otherwise files are not closed before deletion
> {code}
> org.apache.flume.channel.file.TestFlumeEventQueue.addHeadRemove2[0] 29 ms
> 94
> org.apache.flume.channel.file.TestFlumeEventQueue.testWrappingCorrectly[0]
> 18 ms 94
> org.apache.flume.channel.file.TestFlumeEventQueue.testInflightPuts[0]
> 13 ms 94
> org.apache.flume.channel.file.TestFlumeEventQueue.testInflightTakes[0]
> 30 ms 94
> org.apache.flume.channel.file.TestFlumeEventQueue.testCorruptInflightPuts[0]
> 11 ms 94
>
> org.apache.flume.channel.file.TestFlumeEventQueue.testCorruptInflightTakes[0]
>29 ms 94
> org.apache.flume.channel.file.TestFlumeEventQueue.addTail1[0]34 ms
> 94
> org.apache.flume.channel.file.TestFlumeEventQueue.addTail2[0]17 ms
> 94
> org.apache.flume.channel.file.TestFlumeEventQueue.addTailLarge[0]27 ms
> 94
> org.apache.flume.channel.file.TestFlumeEventQueue.addHead1[0]10 ms
> 94
> org.apache.flume.channel.file.TestFlumeEventQueue.addHead2[0]15 ms
> 94
> org.apache.flume.channel.file.TestFlumeEventQueue.addHeadLarge[0]26 ms
> 94
>
> org.apache.flume.channel.file.TestFlumeEventQueue.testUnknownPointerDoesNotCauseSearch[0]
> 15 ms 94
>
> org.apache.flume.channel.file.TestFlumeEventQueue.testRemoveAfterReplayComplete[0]
>12 ms 94
> org.apache.flume.channel.file.TestFlumeEventQueue.addHeadRemove2[1] 32 ms
> 94
> org.apache.flume.channel.file.TestFlumeEventQueue.testWrappingCorrectly[1]
> 0.11 sec94
> org.apache.flume.channel.file.TestFlumeEventQueue.testInflightPuts[1]
> 46 ms 94
> org.apache.flume.channel.file.TestFlumeEventQueue.testInflightTakes[1]
> 22 ms 94
> org.apache.flume.channel.file.TestFlumeEventQueue.testCorruptInflightPuts[1]
> 44 ms 94
>
> org.apache.flume.channel.file.TestFlumeEventQueue.testCorruptInflightTakes[1]
>0.21 sec94
> org.apache.flume.channel.file.TestFlumeEventQueue.addTail1[1]36 ms
> 94
> org.apache.flume.channel.file.TestFlumeEventQueue.addTail2[1]36 ms
> 94
> org.apache.flume.channel.file.TestFlumeEventQueue.addTailLarge[1]48 ms
> 94
> org.apache.flume.channel.file.TestFlumeEventQueue.addHead1[1]46 ms
> 94
> org.apache.flume.channel.file.TestFlumeEventQueue.addHead2[1]24 ms
> 94
> org.apache.flume.channel.file.TestFlumeEventQueue.addHeadLarge[1]44 ms
> 94
>
> org.apache.flume.channel.file.TestFlumeEventQueue.testUnknownPointerDoesNotCauseSearch[1]
> 28 ms 94
>
> org.apache.flume.channel.file.TestFlumeEventQueue.testRemoveAfterReplayComplete[1]
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (FLUME-2532) Windows : TestReliableSpoolingFileEventReader - need to close filereader
[ https://issues.apache.org/jira/browse/FLUME-2532?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Roshan Naik updated FLUME-2532: --- Labels: Windows (was: ) > Windows : TestReliableSpoolingFileEventReader - need to close filereader > > > Key: FLUME-2532 > URL: https://issues.apache.org/jira/browse/FLUME-2532 > Project: Flume > Issue Type: Bug >Affects Versions: v1.5.0.1 >Reporter: Roshan Naik >Assignee: Roshan Naik > Labels: Windows > Attachments: FLUME-2532.patch > > > Need to close file reader so that the files can be deleted. windows does not > allow deletion of files that are open. > Also need to skip testConsumeFileYoungest which is very flaky on windows as > the timestamps of the files it created are not as expected. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (FLUME-2150) Windows: TestFileChannelEncryption needs to be a bit more conservative when examining the exception message
[
https://issues.apache.org/jira/browse/FLUME-2150?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Roshan Naik updated FLUME-2150:
---
Labels: windows (was: )
> Windows: TestFileChannelEncryption needs to be a bit more conservative when
> examining the exception message
>
>
> Key: FLUME-2150
> URL: https://issues.apache.org/jira/browse/FLUME-2150
> Project: Flume
> Issue Type: Bug
> Components: Windows
>Affects Versions: v1.4.0
>Reporter: Roshan Naik
>Assignee: Roshan Naik
> Labels: windows
> Attachments: FLUME-2150.patch
>
>
> Current assertion is :
> Assert.assertEquals("java.io.FileNotFoundException: /path/does/not/exist " +
> ...)
> On Windows the message is a bit different The tail end part /path/.. is not
> same.
> Suffices to check the prefix to be platform independent:
> Assert.assertTrue(ex.getMessage().startsWith("java.io.FileNotFoundException"));
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (FLUME-2534) Windows : TestFlumeEventQueue,TestCheckpoint & TestCheckpointRebuilder failures
[
https://issues.apache.org/jira/browse/FLUME-2534?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14197166#comment-14197166
]
Roshan Naik commented on FLUME-2534:
sure. thanks.
> Windows : TestFlumeEventQueue,TestCheckpoint & TestCheckpointRebuilder
> failures
> ---
>
> Key: FLUME-2534
> URL: https://issues.apache.org/jira/browse/FLUME-2534
> Project: Flume
> Issue Type: Bug
> Components: File Channel, Test
>Affects Versions: v1.5.0.1
>Reporter: Roshan Naik
>Assignee: Roshan Naik
> Labels: Windows
> Attachments: FLUME-2534.patch
>
>
> lots of failures .. need to ensure queue is closed properly before creating a
> new one. Otherwise files are not closed before deletion
> {code}
> org.apache.flume.channel.file.TestFlumeEventQueue.addHeadRemove2[0] 29 ms
> 94
> org.apache.flume.channel.file.TestFlumeEventQueue.testWrappingCorrectly[0]
> 18 ms 94
> org.apache.flume.channel.file.TestFlumeEventQueue.testInflightPuts[0]
> 13 ms 94
> org.apache.flume.channel.file.TestFlumeEventQueue.testInflightTakes[0]
> 30 ms 94
> org.apache.flume.channel.file.TestFlumeEventQueue.testCorruptInflightPuts[0]
> 11 ms 94
>
> org.apache.flume.channel.file.TestFlumeEventQueue.testCorruptInflightTakes[0]
>29 ms 94
> org.apache.flume.channel.file.TestFlumeEventQueue.addTail1[0]34 ms
> 94
> org.apache.flume.channel.file.TestFlumeEventQueue.addTail2[0]17 ms
> 94
> org.apache.flume.channel.file.TestFlumeEventQueue.addTailLarge[0]27 ms
> 94
> org.apache.flume.channel.file.TestFlumeEventQueue.addHead1[0]10 ms
> 94
> org.apache.flume.channel.file.TestFlumeEventQueue.addHead2[0]15 ms
> 94
> org.apache.flume.channel.file.TestFlumeEventQueue.addHeadLarge[0]26 ms
> 94
>
> org.apache.flume.channel.file.TestFlumeEventQueue.testUnknownPointerDoesNotCauseSearch[0]
> 15 ms 94
>
> org.apache.flume.channel.file.TestFlumeEventQueue.testRemoveAfterReplayComplete[0]
>12 ms 94
> org.apache.flume.channel.file.TestFlumeEventQueue.addHeadRemove2[1] 32 ms
> 94
> org.apache.flume.channel.file.TestFlumeEventQueue.testWrappingCorrectly[1]
> 0.11 sec94
> org.apache.flume.channel.file.TestFlumeEventQueue.testInflightPuts[1]
> 46 ms 94
> org.apache.flume.channel.file.TestFlumeEventQueue.testInflightTakes[1]
> 22 ms 94
> org.apache.flume.channel.file.TestFlumeEventQueue.testCorruptInflightPuts[1]
> 44 ms 94
>
> org.apache.flume.channel.file.TestFlumeEventQueue.testCorruptInflightTakes[1]
>0.21 sec94
> org.apache.flume.channel.file.TestFlumeEventQueue.addTail1[1]36 ms
> 94
> org.apache.flume.channel.file.TestFlumeEventQueue.addTail2[1]36 ms
> 94
> org.apache.flume.channel.file.TestFlumeEventQueue.addTailLarge[1]48 ms
> 94
> org.apache.flume.channel.file.TestFlumeEventQueue.addHead1[1]46 ms
> 94
> org.apache.flume.channel.file.TestFlumeEventQueue.addHead2[1]24 ms
> 94
> org.apache.flume.channel.file.TestFlumeEventQueue.addHeadLarge[1]44 ms
> 94
>
> org.apache.flume.channel.file.TestFlumeEventQueue.testUnknownPointerDoesNotCauseSearch[1]
> 28 ms 94
>
> org.apache.flume.channel.file.TestFlumeEventQueue.testRemoveAfterReplayComplete[1]
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (FLUME-2534) Windows : TestFlumeEventQueue,TestCheckpoint & TestCheckpointRebuilder failures
[
https://issues.apache.org/jira/browse/FLUME-2534?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14197097#comment-14197097
]
Hari Shreedharan commented on FLUME-2534:
-
Hey Roshan,
Do you mind creating an umbrella jira for the windows issues and making all
these sub-tasks. That way, it can be tracked more easily. I will review these
over the next several days.
> Windows : TestFlumeEventQueue,TestCheckpoint & TestCheckpointRebuilder
> failures
> ---
>
> Key: FLUME-2534
> URL: https://issues.apache.org/jira/browse/FLUME-2534
> Project: Flume
> Issue Type: Bug
> Components: File Channel, Test
>Affects Versions: v1.5.0.1
>Reporter: Roshan Naik
>Assignee: Roshan Naik
> Labels: Windows
> Attachments: FLUME-2534.patch
>
>
> lots of failures .. need to ensure queue is closed properly before creating a
> new one. Otherwise files are not closed before deletion
> {code}
> org.apache.flume.channel.file.TestFlumeEventQueue.addHeadRemove2[0] 29 ms
> 94
> org.apache.flume.channel.file.TestFlumeEventQueue.testWrappingCorrectly[0]
> 18 ms 94
> org.apache.flume.channel.file.TestFlumeEventQueue.testInflightPuts[0]
> 13 ms 94
> org.apache.flume.channel.file.TestFlumeEventQueue.testInflightTakes[0]
> 30 ms 94
> org.apache.flume.channel.file.TestFlumeEventQueue.testCorruptInflightPuts[0]
> 11 ms 94
>
> org.apache.flume.channel.file.TestFlumeEventQueue.testCorruptInflightTakes[0]
>29 ms 94
> org.apache.flume.channel.file.TestFlumeEventQueue.addTail1[0]34 ms
> 94
> org.apache.flume.channel.file.TestFlumeEventQueue.addTail2[0]17 ms
> 94
> org.apache.flume.channel.file.TestFlumeEventQueue.addTailLarge[0]27 ms
> 94
> org.apache.flume.channel.file.TestFlumeEventQueue.addHead1[0]10 ms
> 94
> org.apache.flume.channel.file.TestFlumeEventQueue.addHead2[0]15 ms
> 94
> org.apache.flume.channel.file.TestFlumeEventQueue.addHeadLarge[0]26 ms
> 94
>
> org.apache.flume.channel.file.TestFlumeEventQueue.testUnknownPointerDoesNotCauseSearch[0]
> 15 ms 94
>
> org.apache.flume.channel.file.TestFlumeEventQueue.testRemoveAfterReplayComplete[0]
>12 ms 94
> org.apache.flume.channel.file.TestFlumeEventQueue.addHeadRemove2[1] 32 ms
> 94
> org.apache.flume.channel.file.TestFlumeEventQueue.testWrappingCorrectly[1]
> 0.11 sec94
> org.apache.flume.channel.file.TestFlumeEventQueue.testInflightPuts[1]
> 46 ms 94
> org.apache.flume.channel.file.TestFlumeEventQueue.testInflightTakes[1]
> 22 ms 94
> org.apache.flume.channel.file.TestFlumeEventQueue.testCorruptInflightPuts[1]
> 44 ms 94
>
> org.apache.flume.channel.file.TestFlumeEventQueue.testCorruptInflightTakes[1]
>0.21 sec94
> org.apache.flume.channel.file.TestFlumeEventQueue.addTail1[1]36 ms
> 94
> org.apache.flume.channel.file.TestFlumeEventQueue.addTail2[1]36 ms
> 94
> org.apache.flume.channel.file.TestFlumeEventQueue.addTailLarge[1]48 ms
> 94
> org.apache.flume.channel.file.TestFlumeEventQueue.addHead1[1]46 ms
> 94
> org.apache.flume.channel.file.TestFlumeEventQueue.addHead2[1]24 ms
> 94
> org.apache.flume.channel.file.TestFlumeEventQueue.addHeadLarge[1]44 ms
> 94
>
> org.apache.flume.channel.file.TestFlumeEventQueue.testUnknownPointerDoesNotCauseSearch[1]
> 28 ms 94
>
> org.apache.flume.channel.file.TestFlumeEventQueue.testRemoveAfterReplayComplete[1]
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (FLUME-2534) Windows : TestFlumeEventQueue,TestCheckpoint & TestCheckpointRebuilder failures
[
https://issues.apache.org/jira/browse/FLUME-2534?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Roshan Naik updated FLUME-2534:
---
Attachment: FLUME-2534.patch
uploading patch
> Windows : TestFlumeEventQueue,TestCheckpoint & TestCheckpointRebuilder
> failures
> ---
>
> Key: FLUME-2534
> URL: https://issues.apache.org/jira/browse/FLUME-2534
> Project: Flume
> Issue Type: Bug
> Components: File Channel, Test
>Affects Versions: v1.5.0.1
>Reporter: Roshan Naik
>Assignee: Roshan Naik
> Labels: Windows
> Attachments: FLUME-2534.patch
>
>
> lots of failures .. need to ensure queue is closed properly before creating a
> new one. Otherwise files are not closed before deletion
> {code}
> org.apache.flume.channel.file.TestFlumeEventQueue.addHeadRemove2[0] 29 ms
> 94
> org.apache.flume.channel.file.TestFlumeEventQueue.testWrappingCorrectly[0]
> 18 ms 94
> org.apache.flume.channel.file.TestFlumeEventQueue.testInflightPuts[0]
> 13 ms 94
> org.apache.flume.channel.file.TestFlumeEventQueue.testInflightTakes[0]
> 30 ms 94
> org.apache.flume.channel.file.TestFlumeEventQueue.testCorruptInflightPuts[0]
> 11 ms 94
>
> org.apache.flume.channel.file.TestFlumeEventQueue.testCorruptInflightTakes[0]
>29 ms 94
> org.apache.flume.channel.file.TestFlumeEventQueue.addTail1[0]34 ms
> 94
> org.apache.flume.channel.file.TestFlumeEventQueue.addTail2[0]17 ms
> 94
> org.apache.flume.channel.file.TestFlumeEventQueue.addTailLarge[0]27 ms
> 94
> org.apache.flume.channel.file.TestFlumeEventQueue.addHead1[0]10 ms
> 94
> org.apache.flume.channel.file.TestFlumeEventQueue.addHead2[0]15 ms
> 94
> org.apache.flume.channel.file.TestFlumeEventQueue.addHeadLarge[0]26 ms
> 94
>
> org.apache.flume.channel.file.TestFlumeEventQueue.testUnknownPointerDoesNotCauseSearch[0]
> 15 ms 94
>
> org.apache.flume.channel.file.TestFlumeEventQueue.testRemoveAfterReplayComplete[0]
>12 ms 94
> org.apache.flume.channel.file.TestFlumeEventQueue.addHeadRemove2[1] 32 ms
> 94
> org.apache.flume.channel.file.TestFlumeEventQueue.testWrappingCorrectly[1]
> 0.11 sec94
> org.apache.flume.channel.file.TestFlumeEventQueue.testInflightPuts[1]
> 46 ms 94
> org.apache.flume.channel.file.TestFlumeEventQueue.testInflightTakes[1]
> 22 ms 94
> org.apache.flume.channel.file.TestFlumeEventQueue.testCorruptInflightPuts[1]
> 44 ms 94
>
> org.apache.flume.channel.file.TestFlumeEventQueue.testCorruptInflightTakes[1]
>0.21 sec94
> org.apache.flume.channel.file.TestFlumeEventQueue.addTail1[1]36 ms
> 94
> org.apache.flume.channel.file.TestFlumeEventQueue.addTail2[1]36 ms
> 94
> org.apache.flume.channel.file.TestFlumeEventQueue.addTailLarge[1]48 ms
> 94
> org.apache.flume.channel.file.TestFlumeEventQueue.addHead1[1]46 ms
> 94
> org.apache.flume.channel.file.TestFlumeEventQueue.addHead2[1]24 ms
> 94
> org.apache.flume.channel.file.TestFlumeEventQueue.addHeadLarge[1]44 ms
> 94
>
> org.apache.flume.channel.file.TestFlumeEventQueue.testUnknownPointerDoesNotCauseSearch[1]
> 28 ms 94
>
> org.apache.flume.channel.file.TestFlumeEventQueue.testRemoveAfterReplayComplete[1]
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (FLUME-2534) Windows : TestFlumeEventQueue,TestCheckpoint & TestCheckpointRebuilder failures
[
https://issues.apache.org/jira/browse/FLUME-2534?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Roshan Naik updated FLUME-2534:
---
Summary: Windows : TestFlumeEventQueue,TestCheckpoint &
TestCheckpointRebuilder failures (was: Windows : TestFlumeEventQueue has many
failures)
> Windows : TestFlumeEventQueue,TestCheckpoint & TestCheckpointRebuilder
> failures
> ---
>
> Key: FLUME-2534
> URL: https://issues.apache.org/jira/browse/FLUME-2534
> Project: Flume
> Issue Type: Bug
> Components: File Channel, Test
>Affects Versions: v1.5.0.1
>Reporter: Roshan Naik
>Assignee: Roshan Naik
> Labels: Windows
>
> lots of failures .. need to ensure queue is closed properly before creating a
> new one. Otherwise files are not closed before deletion
> {code}
> org.apache.flume.channel.file.TestFlumeEventQueue.addHeadRemove2[0] 29 ms
> 94
> org.apache.flume.channel.file.TestFlumeEventQueue.testWrappingCorrectly[0]
> 18 ms 94
> org.apache.flume.channel.file.TestFlumeEventQueue.testInflightPuts[0]
> 13 ms 94
> org.apache.flume.channel.file.TestFlumeEventQueue.testInflightTakes[0]
> 30 ms 94
> org.apache.flume.channel.file.TestFlumeEventQueue.testCorruptInflightPuts[0]
> 11 ms 94
>
> org.apache.flume.channel.file.TestFlumeEventQueue.testCorruptInflightTakes[0]
>29 ms 94
> org.apache.flume.channel.file.TestFlumeEventQueue.addTail1[0]34 ms
> 94
> org.apache.flume.channel.file.TestFlumeEventQueue.addTail2[0]17 ms
> 94
> org.apache.flume.channel.file.TestFlumeEventQueue.addTailLarge[0]27 ms
> 94
> org.apache.flume.channel.file.TestFlumeEventQueue.addHead1[0]10 ms
> 94
> org.apache.flume.channel.file.TestFlumeEventQueue.addHead2[0]15 ms
> 94
> org.apache.flume.channel.file.TestFlumeEventQueue.addHeadLarge[0]26 ms
> 94
>
> org.apache.flume.channel.file.TestFlumeEventQueue.testUnknownPointerDoesNotCauseSearch[0]
> 15 ms 94
>
> org.apache.flume.channel.file.TestFlumeEventQueue.testRemoveAfterReplayComplete[0]
>12 ms 94
> org.apache.flume.channel.file.TestFlumeEventQueue.addHeadRemove2[1] 32 ms
> 94
> org.apache.flume.channel.file.TestFlumeEventQueue.testWrappingCorrectly[1]
> 0.11 sec94
> org.apache.flume.channel.file.TestFlumeEventQueue.testInflightPuts[1]
> 46 ms 94
> org.apache.flume.channel.file.TestFlumeEventQueue.testInflightTakes[1]
> 22 ms 94
> org.apache.flume.channel.file.TestFlumeEventQueue.testCorruptInflightPuts[1]
> 44 ms 94
>
> org.apache.flume.channel.file.TestFlumeEventQueue.testCorruptInflightTakes[1]
>0.21 sec94
> org.apache.flume.channel.file.TestFlumeEventQueue.addTail1[1]36 ms
> 94
> org.apache.flume.channel.file.TestFlumeEventQueue.addTail2[1]36 ms
> 94
> org.apache.flume.channel.file.TestFlumeEventQueue.addTailLarge[1]48 ms
> 94
> org.apache.flume.channel.file.TestFlumeEventQueue.addHead1[1]46 ms
> 94
> org.apache.flume.channel.file.TestFlumeEventQueue.addHead2[1]24 ms
> 94
> org.apache.flume.channel.file.TestFlumeEventQueue.addHeadLarge[1]44 ms
> 94
>
> org.apache.flume.channel.file.TestFlumeEventQueue.testUnknownPointerDoesNotCauseSearch[1]
> 28 ms 94
>
> org.apache.flume.channel.file.TestFlumeEventQueue.testRemoveAfterReplayComplete[1]
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
Re: [ANNOUNCE] New Flume PMC Member - Roshan Naik
Congrats Roshan On Tue, Nov 4, 2014 at 2:31 PM, Hari Shreedharan wrote: > Congrats Roshan! > > > Thanks, > Hari > > On Tue, Nov 4, 2014 at 2:12 PM, Arvind Prabhakar > wrote: > > > On behalf of Apache Flume PMC, it is my pleasure to announce that Roshan > > Naik has been elected to the Flume Project Management Committee. Roshan > has > > been active with the project for many years and has been a committer on > the > > project since September of 2013. > > Please join me in congratulating Roshan and welcoming him to the Flume > PMC. > > Regards, > > Arvind Prabhakar >
Re: [ANNOUNCE] New Flume PMC Member - Roshan Naik
Congrats Roshan! Thanks, Hari On Tue, Nov 4, 2014 at 2:12 PM, Arvind Prabhakar wrote: > On behalf of Apache Flume PMC, it is my pleasure to announce that Roshan > Naik has been elected to the Flume Project Management Committee. Roshan has > been active with the project for many years and has been a committer on the > project since September of 2013. > Please join me in congratulating Roshan and welcoming him to the Flume PMC. > Regards, > Arvind Prabhakar
[jira] [Commented] (FLUME-2521) Ability to manually recover data from corrupted file channel
[ https://issues.apache.org/jira/browse/FLUME-2521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14196976#comment-14196976 ] Bijith Kumar commented on FLUME-2521: - What I am referring to is extracting log events manually from a channel. A simple way to extract out the event lines from the channel file would be good enough. > Ability to manually recover data from corrupted file channel > > > Key: FLUME-2521 > URL: https://issues.apache.org/jira/browse/FLUME-2521 > Project: Flume > Issue Type: New Feature > Components: Channel, File Channel >Reporter: Bijith Kumar > > Though a file channel could be corrupted for a number of reasons, currently > there is no way to recover the events from flume files. E.g. > https://issues.apache.org/jira/browse/FLUME-2519 > In the event of such a failure, considerable amount of data could be stuck in > the channel files. It is critical to have at least a manual recovery option > so that the data in file channel could be extracted out. > A tool or even some instructions to do this manually would be a great thing. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[ANNOUNCE] New Flume PMC Member - Roshan Naik
On behalf of Apache Flume PMC, it is my pleasure to announce that Roshan Naik has been elected to the Flume Project Management Committee. Roshan has been active with the project for many years and has been a committer on the project since September of 2013. Please join me in congratulating Roshan and welcoming him to the Flume PMC. Regards, Arvind Prabhakar
[jira] [Updated] (FLUME-2525) flume should handle a zero byte .flumespool-main.meta file for the spooldir source
[ https://issues.apache.org/jira/browse/FLUME-2525?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Johny Rufus updated FLUME-2525: --- Attachment: FLUME-2525-1.patch Thanks Hari, the initial idea was to decode the IOException to find the root cause, but since we cant do that accurately, following your suggestion. Attaching a new patch > flume should handle a zero byte .flumespool-main.meta file for the spooldir > source > -- > > Key: FLUME-2525 > URL: https://issues.apache.org/jira/browse/FLUME-2525 > Project: Flume > Issue Type: Bug > Components: Sinks+Sources >Affects Versions: v1.5.0.1 >Reporter: Patrick Dvorak > Fix For: v1.6.0 > > Attachments: FLUME-2525-1.patch, FLUME-2525.patch > > > When a zero byte .flumespool-main.meta file exists in the trackerDir (usually > do to the partition filling up), flume will throw the following ambiguous > error message when trying to read in new spool files: > 2014-10-19 18:28:31,333 ERROR > org.apache.flume.client.avro.ReliableSpoolingFileEventReader: Exception > opening file: /home/spooldir/input.log > java.io.IOException: Not a data file. > at org.apache.avro.file.DataFileStream.initialize(DataFileStream.java:102) > at org.apache.avro.file.DataFileReader.(DataFileReader.java:97) > at org.apache.avro.file.DataFileWriter.appendTo(DataFileWriter.java:160) > at org.apache.avro.file.DataFileWriter.appendTo(DataFileWriter.java:149) > at > org.apache.flume.serialization.DurablePositionTracker.(DurablePositionTracker.java:141) > > at > org.apache.flume.serialization.DurablePositionTracker.getInstance(DurablePositionTracker.java:76) > > at > org.apache.flume.client.avro.ReliableSpoolingFileEventReader.getNextFile(ReliableSpoolingFileEventReader.java:420) > > at > org.apache.flume.client.avro.ReliableSpoolingFileEventReader.readEvents(ReliableSpoolingFileEventReader.java:215) > > at > org.apache.flume.source.SpoolDirectorySource$SpoolDirectoryRunnable.run(SpoolDirectorySource.java:182) > > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441) > at java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:317) > at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150) > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:98) > > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:180) > > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:204) > > at > java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) > > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) > > at java.lang.Thread.run(Thread.java:662) > Restarts of the flume agent do not resolve the issue. Only when the zero > byte file is removed, will flume properly start processing files from the > spooldir again. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (FLUME-2535) DEFAULT_BUF_SIZE hits special unicode character on the last byte truncates the event stream
Asim Zafir created FLUME-2535: - Summary: DEFAULT_BUF_SIZE hits special unicode character on the last byte truncates the event stream Key: FLUME-2535 URL: https://issues.apache.org/jira/browse/FLUME-2535 Project: Flume Issue Type: Bug Components: Sinks+Sources Affects Versions: v1.4.0 Reporter: Asim Zafir Priority: Critical we have come to acknowledge a special situation and that is if the DEFAULT_BUF_SIZE mentioned on line 54 of org/apache/flume/serialization/ResettableFileInputStream.java happens to hit special unicode character (Latin Small Letter E with Circumflex Unicode number: U+00EA HTML-code: ê) anything includring and beyond this on the entire stream event will be truncated. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
Re: Flume 0.X JIRA's
@Otis - Neither am I :) Lets wait for a day or so, if no one objects I shall update the status in JIRA On Tue, Nov 4, 2014 at 10:39 AM, Hari Shreedharan wrote: > +1. > > > > Thanks, > Hari > > On Mon, Nov 3, 2014 at 1:45 AM, Ashish wrote: > >> Folks, >> Whats the plan for pre-FlumeNG jira's? To me the branch seems dead. >> IMHO, we can mark the ticket's as Won't Fix and clean up JIRA. >> wdyt? >> thanks >> ashish -- thanks ashish Blog: http://www.ashishpaliwal.com/blog My Photo Galleries: http://www.pbase.com/ashishpaliwal
