GitHub user mmiklavc reopened a pull request:
https://github.com/apache/metron/pull/1155
METRON-1731: PCAP - Escape colons in output dir names
## Contributor Comments
https://issues.apache.org/jira/browse/METRON-1731
This is an improvement on handling existing PCAP output directory writing.
The PcapJob is extremely permissive in values it accepts for filtering packets,
however the fixed/query filter parameters are used for constructing the output
directory of the MapReduce job. When the query/field value contains a colon
":", this causes the hdfs Path object to throw a URIException.
**Includes:**
- token replace colons with underscores in output dir name
- consolidate redundant unit test classes for Fixed and Query PCAP filters.
- create an output directory formatter class to enable testing in isolation
from the rest of pcapjob
- consolidates the char escape convention used by fixed and query filter
dir naming. Currently handle spaces, periods, apostrophes, and colons.
**Testing**
This doesn't technically require test data to validate, since it's an error
that occurs prior to the job starting, though you're certainly welcome to. Run
the following from the CLI and verify no exception is thrown.
```
$METRON_HOME/bin/pcap_query.sh fixed -bp /apps/metron/pcap/input -df
"yyyy-MM-dd-HH-mm" -st 2018-07-26-00-00 -et 2018-07-26-00-10 -rpf 500 -p
"Protocol: ICMP (1)"
```
## Pull Request Checklist
### For all changes:
- [x] Is there a JIRA ticket associated with this PR? If not one needs to
be created at [Metron
Jira](https://issues.apache.org/jira/browse/METRON/?selectedTab=com.atlassian.jira.jira-projects-plugin:summary-panel).
- [x] Does your PR title start with METRON-XXXX where XXXX is the JIRA
number you are trying to resolve? Pay particular attention to the hyphen "-"
character.
- [x] Has your PR been rebased against the latest commit within the target
branch (typically master)?
### For code changes:
- [x] Have you included steps to reproduce the behavior or problem that is
being changed or addressed?
- [x] Have you included steps or a guide to how the change may be verified
and tested manually?
- [x] Have you ensured that the full suite of tests and checks have been
executed in the root metron folder via:
```
mvn -q clean integration-test install &&
dev-utilities/build-utils/verify_licenses.sh
```
- [x] Have you written or updated unit tests and or integration tests to
verify your changes?
- [x] Have you verified the basic functionality of the build by building
and running locally with Vagrant full-dev environment or the equivalent?
### For documentation related changes:
n/a
#### Note:
Please ensure that once the PR is submitted, you check travis-ci for build
issues and submit an update to your PR as soon as possible.
It is also recommended that [travis-ci](https://travis-ci.org) is set up
for your personal repository such that your branches are built there before
submitting a pull request.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/mmiklavc/metron output-dir-formatter
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/metron/pull/1155.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #1155
----
commit 16ffcc16bef169d9082151b9b2fa381512bc813a
Author: Michael Miklavcic <michael.miklavcic@...>
Date: 2018-08-08T20:56:10Z
Extract output directory formatter, add tests, consolidate redundant pcap
filter unit test classes.
commit 256a21dfe92f95e466b0a9488427331f176ab5d8
Author: Michael Miklavcic <michael.miklavcic@...>
Date: 2018-08-09T15:34:47Z
Merge branch 'feature/METRON-1554-pcap-query-panel' into
output-dir-formatter
----
---
