Maybe Hadoop Local FileSystem returns different things from ListFiles() on
different platforms?
That would be something to check?
Sorry that is all I got right now
On February 25, 2017 at 22:57:49, Otto Fowler (ottobackwa...@gmail.com)
wrote:
There are also some if Log.isDebugEnabled() outputs, so maybe try changing
the logging level, maybe running just this test?
On February 25, 2017 at 22:39:02, Otto Fowler (ottobackwa...@gmail.com)
wrote:
There are multiple “tests” within the test, with different parameters. If
you look at where this is breaking, it is at
{
//make sure I get them all.
Iterable<byte[]> results =
job.query(new Path(outDir.getAbsolutePath())
, new Path(queryDir.getAbsolutePath())
, getTimestamp(0, pcapEntries)
, getTimestamp(pcapEntries.size()-1, pcapEntries) + 1
, 10
, new EnumMap<>(Constants.Fields.class)
, new Configuration()
, FileSystem.get(new Configuration())
, new FixedPcapFilter.Configurator()
);
assertInOrder(results);
Assert.assertEquals(Iterables.size(results), pcapEntries.size());
Which is the 7th test job run against the data. I am not familiar with
this test or code, but
that has to be significant.
Maybe you should enable and print out the information of the results - and
we can see a pattern there?
On February 25, 2017 at 22:19:00, Kyle Richardson (kylerichards...@gmail.com)
wrote:
mvn integration-test
Although I have also tried...
mvn clean install && mvn integration-test
mvn clean package && mvn integration-test
mvn install && mvn surefire-test@unit-tests && mvn
surefire-test@integration-tests
-Kyle
On Feb 25, 2017, at 8:34 PM, Otto Fowler <ottobackwa...@gmail.com> wrote:
What command are you using to build?
On February 25, 2017 at 17:40:20, Kyle Richardson (kylerichards...@gmail.com)
wrote:
Tried with Oracle JDK and got the same result. I went as far as trying to
run it through the debugger but am not that familiar with this part of the
code. The timestamps of the packets are definitely not coming back in the
expected order, but I'm not sure why. Could it be related to something
filesystem specific?
Apologies if I'm just being dense but I'd really like to understand why
this consistently fails on some platforms and not others.
-Kyle
> On Feb 25, 2017, at 9:07 AM, Kyle Richardson <kylerichards...@gmail.com>
wrote:
>
> Ok, I've tried this so many times I may be going crazy, so thought I'd
ask the community for a sanity check.
>
> I'm trying to verify RC5 and I keep running into the same integration
test failures but only on my Fedora (24 and 25) and CentOS 7 systems. It
passes fine on my Macbook.
>
> It always fails on the PcapTopologyIntegrationTest (test results pasted
below). Anyone have any ideas? I'm using the exact same version of maven in
all cases (v3.3.9). The only difference I can think of is the Fedora/CentOS
systems are using OpenJDK whereas the Macbook is running Sun/Oracle JDK.
>
> -------------------------------------------------------
> T E S T S
> -------------------------------------------------------
> Running org.apache.metron.pcap.integration.PcapTopologyIntegrationTest
> Formatting using clusterid: testClusterID
> Formatting using clusterid: testClusterID
> Sent pcap data: 20
> Wrote 20 to kafka
> Tests run: 2, Failures: 2, Errors: 0, Skipped: 0, Time elapsed: 42.011
sec <<< FAILURE! - in
org.apache.metron.pcap.integration.PcapTopologyIntegrationTest
>
testTimestampInPacket(org.apache.metron.pcap.integration.PcapTopologyIntegrationTest)
Time elapsed: 26.968 sec <<< FAILURE!
> java.lang.AssertionError
> at org.junit.Assert.fail(Assert.java:86)
> at org.junit.Assert.assertTrue(Assert.java:41)
> at org.junit.Assert.assertTrue(Assert.java:52)
> at
org.apache.metron.pcap.integration.PcapTopologyIntegrationTest.assertInOrder(PcapTopologyIntegrationTest.java:537)
> at
org.apache.metron.pcap.integration.PcapTopologyIntegrationTest.testTopology(PcapTopologyIntegrationTest.java:383)
> at
org.apache.metron.pcap.integration.PcapTopologyIntegrationTest.testTimestampInPacket(PcapTopologyIntegrationTest.java:135)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
> at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
> at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
> at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
> at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
> at
org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:283)
> at
org.apache.maven.surefire.junit4.JUnit4Provider.executeWithRerun(JUnit4Provider.java:173)
> at
org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:153)
> at
org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:128)
> at
org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:203)
> at
org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:155)
> at
org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:103)
>
>
testTimestampInKey(org.apache.metron.pcap.integration.PcapTopologyIntegrationTest)
Time elapsed: 15.038 sec <<< FAILURE!
> java.lang.AssertionError
> at org.junit.Assert.fail(Assert.java:86)
> at org.junit.Assert.assertTrue(Assert.java:41)
> at org.junit.Assert.assertTrue(Assert.java:52)
> at
org.apache.metron.pcap.integration.PcapTopologyIntegrationTest.assertInOrder(PcapTopologyIntegrationTest.java:537)
> at
org.apache.metron.pcap.integration.PcapTopologyIntegrationTest.testTopology(PcapTopologyIntegrationTest.java:383)
> at
org.apache.metron.pcap.integration.PcapTopologyIntegrationTest.testTimestampInKey(PcapTopologyIntegrationTest.java:152)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
> at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
> at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
> at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
> at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
> at
org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:283)
> at
org.apache.maven.surefire.junit4.JUnit4Provider.executeWithRerun(JUnit4Provider.java:173)
> at
org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:153)
> at
org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:128)
> at
org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:203)
> at
org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:155)
> at
org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:103)
>
>
> Results :
>
> Failed tests:
>
PcapTopologyIntegrationTest.testTimestampInKey:152->testTopology:383->assertInOrder:537
null
>
PcapTopologyIntegrationTest.testTimestampInPacket:135->testTopology:383->assertInOrder:537
null
>
>
>
> Tests run: 2, Failures: 2, Errors: 0, Skipped: 0
>
> [ERROR] Failed to execute goal
org.apache.maven.plugins:maven-surefire-plugin:2.18:test
(integration-tests) on project metron-pcap-backend: There are test failures.
> [ERROR]
> [ERROR] Please refer to
/home/kyle/projects/metron-fork/metron-platform/metron-pcap-backend/target/surefire-reports
for the individual test results.
> [ERROR] -> [Help 1]
> [ERROR]
> [ERROR] To see the full stack trace of the errors, re-run Maven with the
-e switch.
> [ERROR] Re-run Maven using the -X switch to enable full debug logging.
> [ERROR]
> [ERROR] For more information about the errors and possible solutions,
please read the following articles:
> [ERROR] [Help 1]
http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
> [ERROR]
> [ERROR] After correcting the problems, you can resume the build with the
command
> [ERROR] mvn <goals> -rf :metron-pcap-backend
>
>
