[
https://issues.apache.org/jira/browse/METRON-936?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16013109#comment-16013109
]
ASF GitHub Bot commented on METRON-936:
---------------------------------------
Github user mmiklavc commented on a diff in the pull request:
https://github.com/apache/metron/pull/585#discussion_r116857304
--- Diff:
metron-platform/metron-pcap-backend/src/main/java/org/apache/metron/spout/pcap/PartitionHDFSWriter.java
---
@@ -183,14 +219,14 @@ private void turnoverIfNecessary(long ts, boolean
force) throws IOException {
}
}
- writer = SequenceFile.createWriter(new Configuration()
+ writer = SequenceFile.createWriter(this.fsConfig
, SequenceFile.Writer.keyClass(LongWritable.class)
, SequenceFile.Writer.valueClass(BytesWritable.class)
, SequenceFile.Writer.stream(outputStream)
,
SequenceFile.Writer.compression(SequenceFile.CompressionType.NONE)
);
//reset state
- LOG.info("Turning over and writing to " + path);
+ LOG.info(String.format("Turning over and writing to %s: [duration=%s
NS, force=%s, initial=%s, overDuration=%s, tooManyPackets=%s]", path, duration,
force, initial, overDuration, tooManyPackets));
--- End diff --
I ended up removing Log4j from all classes in pcap-backend. Submitting
shortly.
> Fixes to pcap for performance and testing
> -----------------------------------------
>
> Key: METRON-936
> URL: https://issues.apache.org/jira/browse/METRON-936
> Project: Metron
> Issue Type: Bug
> Reporter: Michael Miklavcic
> Assignee: Michael Miklavcic
>
> Default in pcap.properties for kafka.pcap.start=END, and it's also not used
> in the Flux file. topology.workers should also be provided via the properties
> file.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
