GitHub user cestella opened a pull request:
https://github.com/apache/metron/pull/882
METRON-1380: Create a typosquatting use-case
## Contributor Comments
This is a documented use-case on how to use the following JIRAs (PRs) to
detect typosquatting in-stream using bloom filters:
* METRON-1379 (#880)
* METRON-1377 (#878 )
* METRON-1378 (#879 )
The code here is a merger of the PRs above to allow reviewers to test the
entire feature together. The manual testing plan is to execute the
typosquatting use-case
[instructions](https://github.com/cestella/incubator-metron/tree/typosquat_merge/use-cases/typosquat_detection).
## Pull Request Checklist
Thank you for submitting a contribution to Apache Metron.
Please refer to our [Development
Guidelines](https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=61332235)
for the complete guide to follow for contributions.
Please refer also to our [Build Verification
Guidelines](https://cwiki.apache.org/confluence/display/METRON/Verifying+Builds?show-miniview)
for complete smoke testing guides.
In order to streamline the review of the contribution we ask you follow
these guidelines and ask you to double check the following:
### For all changes:
- [x] Is there a JIRA ticket associated with this PR? If not one needs to
be created at [Metron
Jira](https://issues.apache.org/jira/browse/METRON/?selectedTab=com.atlassian.jira.jira-projects-plugin:summary-panel).
- [x] Does your PR title start with METRON-XXXX where XXXX is the JIRA
number you are trying to resolve? Pay particular attention to the hyphen "-"
character.
- [x] Has your PR been rebased against the latest commit within the target
branch (typically master)?
### For code changes:
- [x] Have you included steps to reproduce the behavior or problem that is
being changed or addressed?
- [x] Have you included steps or a guide to how the change may be verified
and tested manually?
- [x] Have you ensured that the full suite of tests and checks have been
executed in the root metron folder via:
```
mvn -q clean integration-test install && build_utils/verify_licenses.sh
```
- [x] Have you written or updated unit tests and or integration tests to
verify your changes?
- [x] If adding new dependencies to the code, are these dependencies
licensed in a way that is compatible for inclusion under [ASF
2.0](http://www.apache.org/legal/resolved.html#category-a)?
- [x] Have you verified the basic functionality of the build by building
and running locally with Vagrant full-dev environment or the equivalent?
### For documentation related changes:
- [x] Have you ensured that format looks appropriate for the output in
which it is rendered by building and verifying the site-book? If not then run
the following commands and the verify changes via
`site-book/target/site/index.html`:
```
cd site-book
mvn site
```
#### Note:
Please ensure that once the PR is submitted, you check travis-ci for build
issues and submit an update to your PR as soon as possible.
It is also recommended that [travis-ci](https://travis-ci.org) is set up
for your personal repository such that your branches are built there before
submitting a pull request.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/cestella/incubator-metron typosquat_merge
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/metron/pull/882.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #882
----
commit a95014ed1e145f9133dd95dcbfbf7e9212401fef
Author: cstella <cestella@...>
Date: 2017-12-19T22:26:03Z
METRON-1377: Stellar function to generate typosquatted domains (similar to
dnstwist)
commit 9c492c4540534fa72550aff330ce6c588f640965
Author: cstella <cestella@...>
Date: 2017-12-21T15:17:18Z
flatfile summarizer initial commit.
commit 71e63b2604ad94c51423762582e547184169d8a2
Author: cstella <cestella@...>
Date: 2017-12-21T15:20:48Z
Don't want to generate original domain as it's not a typosquatted domain
commit 42af879d5fc1623fd9b24dd24af687292d9bcc73
Author: cstella <cestella@...>
Date: 2017-12-21T16:20:10Z
Fixed homoglyph bug with ACE domains
commit 7ee3ab14b81b0cb3fd899cf082050b7e3fade63e
Author: cstella <cestella@...>
Date: 2017-12-21T17:04:58Z
Persistent bug..
commit 15681143e86913a692777770d0a89e1c877e3d99
Author: cstella <cestella@...>
Date: 2017-12-21T18:50:58Z
typo
commit 0d1e7b304b926bae65a2d6b4c63dec565542ad7e
Author: cstella <cestella@...>
Date: 2017-12-21T18:51:50Z
Weirdness with international domains.
commit 935d4d2933e7156219722e54cec5dfce228fdbcc
Author: cstella <cestella@...>
Date: 2017-12-21T21:17:23Z
Updating tests and docs.
commit afe91c341608468e2637db4a02f9428ebe19353a
Author: cstella <cestella@...>
Date: 2017-12-21T21:18:20Z
more docs.
commit d955e26cf4e7776642e83b23deb305fd5a238cc2
Author: cstella <cestella@...>
Date: 2017-12-21T21:46:30Z
Renamed test.
commit 5328931504bdd069e827ba55cc6015ce9b3031f8
Author: cstella <cestella@...>
Date: 2017-12-21T23:23:32Z
METRON-1379: Add an OBJECT_GET stellar function
commit 8fbaa25a675d709e2a371f25ab0cf8e318653211
Author: cstella <cestella@...>
Date: 2017-12-21T23:31:19Z
Merge branch 'typosquat' of github.com:cestella/incubator-metron into
typosquat_merge
commit 6ef76fefa289e95314a19e97cf607983907d7cbe
Author: cstella <cestella@...>
Date: 2017-12-21T23:31:36Z
Merge branch 'object_get' of github.com:cestella/incubator-metron into
typosquat_merge
commit 27a0c965369dfb9475649f27d620ad20f652c37f
Author: cstella <cestella@...>
Date: 2017-12-21T23:31:55Z
Merge branch 'flatfile_object_gen' of github.com:cestella/incubator-metron
into typosquat_merge
commit 1c20be8ddcf2afad8a0772645b611ddd52c2cdf4
Author: cstella <cestella@...>
Date: 2017-12-22T01:45:26Z
Ensuring original domain doesn't make it into a typosquatted domain
commit 9712b77de8df30f6d5117335580badc28e11593d
Author: cstella <cestella@...>
Date: 2017-12-22T01:45:41Z
Merge branch 'typosquat' into typosquat_merge
commit 3c028304aaf8844c2dc83a7114c06d0670d896ee
Author: cstella <cestella@...>
Date: 2017-12-22T12:19:50Z
adding base document
commit ac3c612cd6fd7140a14fac9692000f04b65ecc83
Author: cstella <cestella@...>
Date: 2017-12-22T12:23:04Z
Adding a ToString writer.
commit 9909924da2133815c4e5907097f7369cb1005b92
Author: cstella <cestella@...>
Date: 2017-12-22T12:23:26Z
Merge branch 'flatfile_object_gen' into typosquat_merge
commit 34cdb55f6c43049151c5b5242a73a09119de31ef
Author: cstella <cestella@...>
Date: 2017-12-22T15:10:15Z
Renamed to console writer
commit bd3d0f66520beeabf7f94ff949bcafa3becd2dfc
Author: cstella <cestella@...>
Date: 2017-12-22T15:10:28Z
Merge branch 'flatfile_object_gen' into typosquat_merge
commit b3e4408ab98d69866774bae452e9cc47efc4fbdd
Author: cstella <cestella@...>
Date: 2017-12-22T15:14:43Z
newline issue.
commit 261ff35b2d598ffb44caec156651ba695cd77fe2
Author: cstella <cestella@...>
Date: 2017-12-22T15:14:56Z
Merge branch 'flatfile_object_gen' into typosquat_merge
commit e7d416e61a3a79b63bccf0edf6e301fce9b0a4f7
Author: cstella <cestella@...>
Date: 2017-12-22T17:41:25Z
Added readme to accompany typosquat use-case
commit a272da183fcd5031a8a55f00dd9143a2069881f1
Author: cstella <cestella@...>
Date: 2017-12-22T17:47:59Z
Updating OBJECT_GET documentation to be better.
commit 22dbd63a2f831a13d37b11e3d41a936087dfcc7b
Author: cstella <cestella@...>
Date: 2017-12-22T17:48:20Z
Merge branch 'object_get' into typosquat_merge
----
---
