[jira] [Commented] (METRON-1728) Handle null values in config in Pcap backend more gracefully

2018-08-09 Thread ASF GitHub Bot (JIRA) 


[ 
https://issues.apache.org/jira/browse/METRON-1728?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16574999#comment-16574999
 ] 

ASF GitHub Bot commented on METRON-1728:


Github user mmiklavc commented on the issue:

https://github.com/apache/metron/pull/1151
  
Closing as merged


> Handle null values in config in Pcap backend more gracefully
> 
>
> Key: METRON-1728
> URL: https://issues.apache.org/jira/browse/METRON-1728
> Project: Metron
>  Issue Type: Sub-task
>Reporter: Michael Miklavcic
>Assignee: Michael Miklavcic
>Priority: Major
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (METRON-1728) Handle null values in config in Pcap backend more gracefully

2018-08-09 Thread ASF GitHub Bot (JIRA) 


[ 
https://issues.apache.org/jira/browse/METRON-1728?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16575000#comment-16575000
 ] 

ASF GitHub Bot commented on METRON-1728:


Github user mmiklavc closed the pull request at:

https://github.com/apache/metron/pull/1151


> Handle null values in config in Pcap backend more gracefully
> 
>
> Key: METRON-1728
> URL: https://issues.apache.org/jira/browse/METRON-1728
> Project: Metron
>  Issue Type: Sub-task
>Reporter: Michael Miklavcic
>Assignee: Michael Miklavcic
>Priority: Major
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (METRON-1728) Handle null values in config in Pcap backend more gracefully

2018-08-09 Thread ASF GitHub Bot (JIRA) 


[ 
https://issues.apache.org/jira/browse/METRON-1728?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16574829#comment-16574829
 ] 

ASF GitHub Bot commented on METRON-1728:


Github user justinleet commented on the issue:

https://github.com/apache/metron/pull/1151
  
+1 by inspection. Thanks for the improvement!


> Handle null values in config in Pcap backend more gracefully
> 
>
> Key: METRON-1728
> URL: https://issues.apache.org/jira/browse/METRON-1728
> Project: Metron
>  Issue Type: Sub-task
>Reporter: Michael Miklavcic
>Assignee: Michael Miklavcic
>Priority: Major
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (METRON-1728) Handle null values in config in Pcap backend more gracefully

2018-08-08 Thread ASF GitHub Bot (JIRA) 


[ 
https://issues.apache.org/jira/browse/METRON-1728?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16573711#comment-16573711
 ] 

ASF GitHub Bot commented on METRON-1728:


Github user mmiklavc commented on a diff in the pull request:

https://github.com/apache/metron/pull/1151#discussion_r208697541
  
--- Diff: 
metron-platform/metron-pcap/src/main/java/org/apache/metron/pcap/mr/PcapJob.java
 ---
@@ -216,20 +218,22 @@ public void setCompleteCheckInterval(long interval) {
 Configuration hadoopConf = PcapOptions.HADOOP_CONF.get(configuration, 
Configuration.class);
 FileSystem fileSystem = PcapOptions.FILESYSTEM.get(configuration, 
FileSystem.class);
 Path basePath = PcapOptions.BASE_PATH.getTransformed(configuration, 
Path.class);
-Path baseInterimResultPath = 
PcapOptions.BASE_INTERIM_RESULT_PATH.getTransformed(configuration, Path.class);
+Path baseInterimResultPath = PcapOptions.BASE_INTERIM_RESULT_PATH
+.getTransformedOrDefault(configuration, Path.class,
+new Path(PcapGlobalDefaults.BASE_INTERIM_RESULT_PATH_DEFAULT));
 long startTime;
 if (configuration.containsKey(PcapOptions.START_TIME_NS.getKey())) {
-  startTime = PcapOptions.START_TIME_NS.get(configuration, Long.class);
+  startTime = PcapOptions.START_TIME_NS.getOrDefault(configuration, 
Long.class, 0L);
 } else {
-  startTime = PcapOptions.START_TIME_MS.get(configuration, Long.class) 
* 100;
+  startTime = PcapOptions.START_TIME_MS.getOrDefault(configuration, 
Long.class, 0L) * 100;
 }
 long endTime;
 if (configuration.containsKey(PcapOptions.END_TIME_NS.getKey())) {
-  endTime = PcapOptions.END_TIME_NS.get(configuration, Long.class);
+  endTime = PcapOptions.END_TIME_NS.getOrDefault(configuration, 
Long.class, System.nanoTime());
--- End diff --

@justinleet good catch! Thanks.


> Handle null values in config in Pcap backend more gracefully
> 
>
> Key: METRON-1728
> URL: https://issues.apache.org/jira/browse/METRON-1728
> Project: Metron
>  Issue Type: Sub-task
>Reporter: Michael Miklavcic
>Assignee: Michael Miklavcic
>Priority: Major
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (METRON-1728) Handle null values in config in Pcap backend more gracefully

2018-08-08 Thread ASF GitHub Bot (JIRA) 


[ 
https://issues.apache.org/jira/browse/METRON-1728?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16573607#comment-16573607
 ] 

ASF GitHub Bot commented on METRON-1728:


Github user justinleet commented on a diff in the pull request:

https://github.com/apache/metron/pull/1151#discussion_r208679310
  
--- Diff: 
metron-platform/metron-pcap/src/main/java/org/apache/metron/pcap/mr/PcapJob.java
 ---
@@ -216,20 +218,22 @@ public void setCompleteCheckInterval(long interval) {
 Configuration hadoopConf = PcapOptions.HADOOP_CONF.get(configuration, 
Configuration.class);
 FileSystem fileSystem = PcapOptions.FILESYSTEM.get(configuration, 
FileSystem.class);
 Path basePath = PcapOptions.BASE_PATH.getTransformed(configuration, 
Path.class);
-Path baseInterimResultPath = 
PcapOptions.BASE_INTERIM_RESULT_PATH.getTransformed(configuration, Path.class);
+Path baseInterimResultPath = PcapOptions.BASE_INTERIM_RESULT_PATH
+.getTransformedOrDefault(configuration, Path.class,
+new Path(PcapGlobalDefaults.BASE_INTERIM_RESULT_PATH_DEFAULT));
 long startTime;
 if (configuration.containsKey(PcapOptions.START_TIME_NS.getKey())) {
-  startTime = PcapOptions.START_TIME_NS.get(configuration, Long.class);
+  startTime = PcapOptions.START_TIME_NS.getOrDefault(configuration, 
Long.class, 0L);
 } else {
-  startTime = PcapOptions.START_TIME_MS.get(configuration, Long.class) 
* 100;
+  startTime = PcapOptions.START_TIME_MS.getOrDefault(configuration, 
Long.class, 0L) * 100;
 }
 long endTime;
 if (configuration.containsKey(PcapOptions.END_TIME_NS.getKey())) {
-  endTime = PcapOptions.END_TIME_NS.get(configuration, Long.class);
+  endTime = PcapOptions.END_TIME_NS.getOrDefault(configuration, 
Long.class, System.nanoTime());
--- End diff --

Nanotime isn't actually current time in nanos or anything objectively 
meaningful (and can vary by jvm or be negative or whatever).  Check out 
https://docs.oracle.com/javase/7/docs/api/java/lang/System.html#nanoTime()

Particularly,
> This method can only be used to measure elapsed time and is not related 
to any other notion of system or wall-clock time. The value returned represents 
nanoseconds since some fixed but arbitrary origin time (perhaps in the future, 
so values may be negative). 

I'd just make it System.currentTimeMillis() and adjust, since I don't think 
there's a clean way to get a real nanos timestamp in Java (at least until 9, 
maybe)


> Handle null values in config in Pcap backend more gracefully
> 
>
> Key: METRON-1728
> URL: https://issues.apache.org/jira/browse/METRON-1728
> Project: Metron
>  Issue Type: Sub-task
>Reporter: Michael Miklavcic
>Assignee: Michael Miklavcic
>Priority: Major
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (METRON-1728) Handle null values in config in Pcap backend more gracefully

2018-08-07 Thread ASF GitHub Bot (JIRA) 


[ 
https://issues.apache.org/jira/browse/METRON-1728?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16572241#comment-16572241
 ] 

ASF GitHub Bot commented on METRON-1728:


GitHub user mmiklavc opened a pull request:

https://github.com/apache/metron/pull/1151

METRON-1728: Handle null values in config in Pcap backend more gracefully

## Contributor Comments

https://issues.apache.org/jira/browse/METRON-1728

I modified the ConfigOptions interface to provide the ability to handle 
default values similar to the Map interface. ie getOrDefault. For PcapJob, I 
only improved/changed the values we expect to be user-supplied, ie start time, 
end time, etc., so we don't throw NPE's if a value gets lost in translation. I 
think we should definitely still add some validation to users via both REST and 
the UI similar to what is done in PcapCLI. 

**Note:** This PR is an improvement to existing functionality, not a 
regression fix.

## Pull Request Checklist

### For all changes:
- [x] Is there a JIRA ticket associated with this PR? If not one needs to 
be created at [Metron 
Jira](https://issues.apache.org/jira/browse/METRON/?selectedTab=com.atlassian.jira.jira-projects-plugin:summary-panel).
- [x] Does your PR title start with METRON- where  is the JIRA 
number you are trying to resolve? Pay particular attention to the hyphen "-" 
character.
- [x] Has your PR been rebased against the latest commit within the target 
branch (typically master)?


### For code changes:
- [x] Have you included steps to reproduce the behavior or problem that is 
being changed or addressed?
- [x] Have you included steps or a guide to how the change may be verified 
and tested manually?
- [x] Have you ensured that the full suite of tests and checks have been 
executed in the root metron folder via:
  ```
  mvn -q clean integration-test install && 
dev-utilities/build-utils/verify_licenses.sh 
  ```

- [x] Have you written or updated unit tests and or integration tests to 
verify your changes?
- [x] Have you verified the basic functionality of the build by building 
and running locally with Vagrant full-dev environment or the equivalent?

### For documentation related changes:
n/a

 Note:
Please ensure that once the PR is submitted, you check travis-ci for build 
issues and submit an update to your PR as soon as possible.
It is also recommended that [travis-ci](https://travis-ci.org) is set up 
for your personal repository such that your branches are built there before 
submitting a pull request.


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/mmiklavc/metron pcap-config-handle-nulls

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/metron/pull/1151.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1151


commit 1fee540969b2f192e58805cdca1ce0a666759af6
Author: Michael Miklavcic 
Date:   2018-08-07T19:29:41Z

Handle null values in config backend more gracefully




> Handle null values in config in Pcap backend more gracefully
> 
>
> Key: METRON-1728
> URL: https://issues.apache.org/jira/browse/METRON-1728
> Project: Metron
>  Issue Type: Sub-task
>Reporter: Michael Miklavcic
>Assignee: Michael Miklavcic
>Priority: Major
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)