http://git-wip-us.apache.org/repos/asf/metron/blob/a97e575f/site/current-book/metron-deployment/development/index.html ---------------------------------------------------------------------- diff --git a/site/current-book/metron-deployment/development/index.html b/site/current-book/metron-deployment/development/index.html index ea99fdf..9c6f49e 100644 --- a/site/current-book/metron-deployment/development/index.html +++ b/site/current-book/metron-deployment/development/index.html @@ -1,13 +1,13 @@ <!DOCTYPE html> <!-- - | Generated by Apache Maven Doxia Site Renderer 1.8 from src/site/markdown/metron-deployment/development/index.md at 2018-06-07 + | Generated by Apache Maven Doxia Site Renderer 1.8 from src/site/markdown/metron-deployment/development/index.md at 2018-09-12 | Rendered using Apache Maven Fluido Skin 1.7 --> <html xmlns="http://www.w3.org/1999/xhtml"; xml:lang="en" lang="en"> <head> <meta charset="UTF-8" /> <meta name="viewport" content="width=device-width, initial-scale=1.0" /> - <meta name="Date-Revision-yyyymmdd" content="20180607" /> + <meta name="Date-Revision-yyyymmdd" content="20180912" /> <meta http-equiv="Content-Language" content="en" /> <title>Metron – Metron Development Environments</title> <link rel="stylesheet" href="../../css/apache-maven-fluido-1.7.min.css" /> @@ -32,8 +32,8 @@ <li class=""><a href="http://metron.apache.org/"; class="externalLink" title="Metron">Metron</a><span class="divider">/</span></li> <li class=""><a href="../../index.html" title="Documentation">Documentation</a><span class="divider">/</span></li> <li class="active ">Metron Development Environments</li> - <li id="publishDate" class="pull-right"><span class="divider">|</span> Last Published: 2018-06-07</li> - <li id="projectVersion" class="pull-right">Version: 0.5.0</li> + <li id="publishDate" class="pull-right"><span class="divider">|</span> Last Published: 2018-09-12</li> + <li id="projectVersion" class="pull-right">Version: 0.6.0</li> </ul> </div> <div class="row-fluid">
http://git-wip-us.apache.org/repos/asf/metron/blob/a97e575f/site/current-book/metron-deployment/development/ubuntu14/index.html ---------------------------------------------------------------------- diff --git a/site/current-book/metron-deployment/development/ubuntu14/index.html b/site/current-book/metron-deployment/development/ubuntu14/index.html index 16b2d81..fb15a5e 100644 --- a/site/current-book/metron-deployment/development/ubuntu14/index.html +++ b/site/current-book/metron-deployment/development/ubuntu14/index.html @@ -1,13 +1,13 @@ <!DOCTYPE html> <!-- - | Generated by Apache Maven Doxia Site Renderer 1.8 from src/site/markdown/metron-deployment/development/ubuntu14/index.md at 2018-06-07 + | Generated by Apache Maven Doxia Site Renderer 1.8 from src/site/markdown/metron-deployment/development/ubuntu14/index.md at 2018-09-12 | Rendered using Apache Maven Fluido Skin 1.7 --> <html xmlns="http://www.w3.org/1999/xhtml"; xml:lang="en" lang="en"> <head> <meta charset="UTF-8" /> <meta name="viewport" content="width=device-width, initial-scale=1.0" /> - <meta name="Date-Revision-yyyymmdd" content="20180607" /> + <meta name="Date-Revision-yyyymmdd" content="20180912" /> <meta http-equiv="Content-Language" content="en" /> <title>Metron – Metron on Ubuntu 14</title> <link rel="stylesheet" href="../../../css/apache-maven-fluido-1.7.min.css" /> @@ -32,8 +32,8 @@ <li class=""><a href="http://metron.apache.org/"; class="externalLink" title="Metron">Metron</a><span class="divider">/</span></li> <li class=""><a href="../../../index.html" title="Documentation">Documentation</a><span class="divider">/</span></li> <li class="active ">Metron on Ubuntu 14</li> - <li id="publishDate" class="pull-right"><span class="divider">|</span> Last Published: 2018-06-07</li> - <li id="projectVersion" class="pull-right">Version: 0.5.0</li> + <li id="publishDate" class="pull-right"><span class="divider">|</span> Last Published: 2018-09-12</li> + <li id="projectVersion" class="pull-right">Version: 0.6.0</li> </ul> </div> <div class="row-fluid"> @@ -190,8 +190,8 @@ vagrant up <p>Navigate to the following resources to explore your newly minted Apache Metron environment.</p> <ul> -<li><a class="externalLink" href="http://node1:4201";>Metron Alerts</a></li> -<li><a class="externalLink" href="http://node1:8080";>Ambari</a></li> +<li><a class="externalLink" href="http://node1:4201";>Metron Alerts</a> credentials: user/password</li> +<li><a class="externalLink" href="http://node1:8080";>Ambari</a> credentials: admin/admin</li> </ul> <p>Connecting to the host through SSH is as simple as running the following command.</p> http://git-wip-us.apache.org/repos/asf/metron/blob/a97e575f/site/current-book/metron-deployment/index.html ---------------------------------------------------------------------- diff --git a/site/current-book/metron-deployment/index.html b/site/current-book/metron-deployment/index.html index a3ffcbd..ef7ddb1 100644 --- a/site/current-book/metron-deployment/index.html +++ b/site/current-book/metron-deployment/index.html @@ -1,13 +1,13 @@ <!DOCTYPE html> <!-- - | Generated by Apache Maven Doxia Site Renderer 1.8 from src/site/markdown/metron-deployment/index.md at 2018-06-07 + | Generated by Apache Maven Doxia Site Renderer 1.8 from src/site/markdown/metron-deployment/index.md at 2018-09-12 | Rendered using Apache Maven Fluido Skin 1.7 --> <html xmlns="http://www.w3.org/1999/xhtml"; xml:lang="en" lang="en"> <head> <meta charset="UTF-8" /> <meta name="viewport" content="width=device-width, initial-scale=1.0" /> - <meta name="Date-Revision-yyyymmdd" content="20180607" /> + <meta name="Date-Revision-yyyymmdd" content="20180912" /> <meta http-equiv="Content-Language" content="en" /> <title>Metron – </title> <link rel="stylesheet" href="../css/apache-maven-fluido-1.7.min.css" /> @@ -32,8 +32,8 @@ <li class=""><a href="http://metron.apache.org/"; class="externalLink" title="Metron">Metron</a><span class="divider">/</span></li> <li class=""><a href="../index.html" title="Documentation">Documentation</a><span class="divider">/</span></li> <li class="active "></li> - <li id="publishDate" class="pull-right"><span class="divider">|</span> Last Published: 2018-06-07</li> - <li id="projectVersion" class="pull-right">Version: 0.5.0</li> + <li id="publishDate" class="pull-right"><span class="divider">|</span> Last Published: 2018-09-12</li> + <li id="projectVersion" class="pull-right">Version: 0.6.0</li> </ul> </div> <div class="row-fluid"> @@ -171,7 +171,10 @@ limitations under the License. <p>Running Metron within the resource constraints of a single VM is incredibly challenging. Failing to respect this warning, will cause various services to fail mysteriously as the system runs into memory and processing limits.</p></div> <div class="section"> <h4><a name="How.3F"></a>How?</h4> -<p>To deploy Metron in a VM running on your computer, follow the instructions at <a href="development/centos6/index.html">development/centos6</a>.</p></div></div></div> +<p>To deploy Metron in a VM running on your computer, follow the instructions at <a href="development/centos6/index.html">development/centos6</a>.</p></div> +<div class="section"> +<h4><a name="How_do_I_address_services_crashing_when_running_Metron_on_a_single_VM.3F"></a>How do I address services crashing when running Metron on a single VM?</h4> +<p>We recommend looking at Ambari and shutting down any services you may not be using. For example, we recommend turning off Metron Profiler, as this commonly causes REST services to crash when running on a single VM.</p></div></div></div> <div class="section"> <h2><a name="How_do_I_build_RPM_packages.3F"></a>How do I build RPM packages?</h2> <p>This provides RPM packages that allow you to install Metron on an RPM-based operating system like CentOS.</p> http://git-wip-us.apache.org/repos/asf/metron/blob/a97e575f/site/current-book/metron-deployment/other-examples/index.html ---------------------------------------------------------------------- diff --git a/site/current-book/metron-deployment/other-examples/index.html b/site/current-book/metron-deployment/other-examples/index.html index 3a89cb0..41ee390 100644 --- a/site/current-book/metron-deployment/other-examples/index.html +++ b/site/current-book/metron-deployment/other-examples/index.html @@ -1,13 +1,13 @@ <!DOCTYPE html> <!-- - | Generated by Apache Maven Doxia Site Renderer 1.8 from src/site/markdown/metron-deployment/other-examples/index.md at 2018-06-07 + | Generated by Apache Maven Doxia Site Renderer 1.8 from src/site/markdown/metron-deployment/other-examples/index.md at 2018-09-12 | Rendered using Apache Maven Fluido Skin 1.7 --> <html xmlns="http://www.w3.org/1999/xhtml"; xml:lang="en" lang="en"> <head> <meta charset="UTF-8" /> <meta name="viewport" content="width=device-width, initial-scale=1.0" /> - <meta name="Date-Revision-yyyymmdd" content="20180607" /> + <meta name="Date-Revision-yyyymmdd" content="20180912" /> <meta http-equiv="Content-Language" content="en" /> <title>Metron – Other Example Deployments</title> <link rel="stylesheet" href="../../css/apache-maven-fluido-1.7.min.css" /> @@ -32,8 +32,8 @@ <li class=""><a href="http://metron.apache.org/"; class="externalLink" title="Metron">Metron</a><span class="divider">/</span></li> <li class=""><a href="../../index.html" title="Documentation">Documentation</a><span class="divider">/</span></li> <li class="active ">Other Example Deployments</li> - <li id="publishDate" class="pull-right"><span class="divider">|</span> Last Published: 2018-06-07</li> - <li id="projectVersion" class="pull-right">Version: 0.5.0</li> + <li id="publishDate" class="pull-right"><span class="divider">|</span> Last Published: 2018-09-12</li> + <li id="projectVersion" class="pull-right">Version: 0.6.0</li> </ul> </div> <div class="row-fluid"> http://git-wip-us.apache.org/repos/asf/metron/blob/a97e575f/site/current-book/metron-deployment/other-examples/manual-install/Manual_Install_CentOS6.html ---------------------------------------------------------------------- diff --git a/site/current-book/metron-deployment/other-examples/manual-install/Manual_Install_CentOS6.html b/site/current-book/metron-deployment/other-examples/manual-install/Manual_Install_CentOS6.html index 5cf9775..f3d8dff 100644 --- a/site/current-book/metron-deployment/other-examples/manual-install/Manual_Install_CentOS6.html +++ b/site/current-book/metron-deployment/other-examples/manual-install/Manual_Install_CentOS6.html @@ -1,13 +1,13 @@ <!DOCTYPE html> <!-- - | Generated by Apache Maven Doxia Site Renderer 1.8 from src/site/markdown/metron-deployment/other-examples/manual-install/Manual_Install_CentOS6.md at 2018-06-07 + | Generated by Apache Maven Doxia Site Renderer 1.8 from src/site/markdown/metron-deployment/other-examples/manual-install/Manual_Install_CentOS6.md at 2018-09-12 | Rendered using Apache Maven Fluido Skin 1.7 --> <html xmlns="http://www.w3.org/1999/xhtml"; xml:lang="en" lang="en"> <head> <meta charset="UTF-8" /> <meta name="viewport" content="width=device-width, initial-scale=1.0" /> - <meta name="Date-Revision-yyyymmdd" content="20180607" /> + <meta name="Date-Revision-yyyymmdd" content="20180912" /> <meta http-equiv="Content-Language" content="en" /> <title>Metron – </title> <link rel="stylesheet" href="../../../css/apache-maven-fluido-1.7.min.css" /> @@ -32,8 +32,8 @@ <li class=""><a href="http://metron.apache.org/"; class="externalLink" title="Metron">Metron</a><span class="divider">/</span></li> <li class=""><a href="../../../index.html" title="Documentation">Documentation</a><span class="divider">/</span></li> <li class="active "></li> - <li id="publishDate" class="pull-right"><span class="divider">|</span> Last Published: 2018-06-07</li> - <li id="projectVersion" class="pull-right">Version: 0.5.0</li> + <li id="publishDate" class="pull-right"><span class="divider">|</span> Last Published: 2018-09-12</li> + <li id="projectVersion" class="pull-right">Version: 0.6.0</li> </ul> </div> <div class="row-fluid"> http://git-wip-us.apache.org/repos/asf/metron/blob/a97e575f/site/current-book/metron-deployment/packaging/ambari/elasticsearch-mpack/index.html ---------------------------------------------------------------------- diff --git a/site/current-book/metron-deployment/packaging/ambari/elasticsearch-mpack/index.html b/site/current-book/metron-deployment/packaging/ambari/elasticsearch-mpack/index.html index 499e89b..8a59770 100644 --- a/site/current-book/metron-deployment/packaging/ambari/elasticsearch-mpack/index.html +++ b/site/current-book/metron-deployment/packaging/ambari/elasticsearch-mpack/index.html @@ -1,13 +1,13 @@ <!DOCTYPE html> <!-- - | Generated by Apache Maven Doxia Site Renderer 1.8 from src/site/markdown/metron-deployment/packaging/ambari/elasticsearch-mpack/index.md at 2018-06-07 + | Generated by Apache Maven Doxia Site Renderer 1.8 from src/site/markdown/metron-deployment/packaging/ambari/elasticsearch-mpack/index.md at 2018-09-12 | Rendered using Apache Maven Fluido Skin 1.7 --> <html xmlns="http://www.w3.org/1999/xhtml"; xml:lang="en" lang="en"> <head> <meta charset="UTF-8" /> <meta name="viewport" content="width=device-width, initial-scale=1.0" /> - <meta name="Date-Revision-yyyymmdd" content="20180607" /> + <meta name="Date-Revision-yyyymmdd" content="20180912" /> <meta http-equiv="Content-Language" content="en" /> <title>Metron – </title> <link rel="stylesheet" href="../../../../css/apache-maven-fluido-1.7.min.css" /> @@ -32,8 +32,8 @@ <li class=""><a href="http://metron.apache.org/"; class="externalLink" title="Metron">Metron</a><span class="divider">/</span></li> <li class=""><a href="../../../../index.html" title="Documentation">Documentation</a><span class="divider">/</span></li> <li class="active "></li> - <li id="publishDate" class="pull-right"><span class="divider">|</span> Last Published: 2018-06-07</li> - <li id="projectVersion" class="pull-right">Version: 0.5.0</li> + <li id="publishDate" class="pull-right"><span class="divider">|</span> Last Published: 2018-09-12</li> + <li id="projectVersion" class="pull-right">Version: 0.6.0</li> </ul> </div> <div class="row-fluid"> http://git-wip-us.apache.org/repos/asf/metron/blob/a97e575f/site/current-book/metron-deployment/packaging/ambari/index.html ---------------------------------------------------------------------- diff --git a/site/current-book/metron-deployment/packaging/ambari/index.html b/site/current-book/metron-deployment/packaging/ambari/index.html index bac5758..7acd537 100644 --- a/site/current-book/metron-deployment/packaging/ambari/index.html +++ b/site/current-book/metron-deployment/packaging/ambari/index.html @@ -1,13 +1,13 @@ <!DOCTYPE html> <!-- - | Generated by Apache Maven Doxia Site Renderer 1.8 from src/site/markdown/metron-deployment/packaging/ambari/index.md at 2018-06-07 + | Generated by Apache Maven Doxia Site Renderer 1.8 from src/site/markdown/metron-deployment/packaging/ambari/index.md at 2018-09-12 | Rendered using Apache Maven Fluido Skin 1.7 --> <html xmlns="http://www.w3.org/1999/xhtml"; xml:lang="en" lang="en"> <head> <meta charset="UTF-8" /> <meta name="viewport" content="width=device-width, initial-scale=1.0" /> - <meta name="Date-Revision-yyyymmdd" content="20180607" /> + <meta name="Date-Revision-yyyymmdd" content="20180912" /> <meta http-equiv="Content-Language" content="en" /> <title>Metron – Ambari Management Pack Development</title> <link rel="stylesheet" href="../../../css/apache-maven-fluido-1.7.min.css" /> @@ -32,8 +32,8 @@ <li class=""><a href="http://metron.apache.org/"; class="externalLink" title="Metron">Metron</a><span class="divider">/</span></li> <li class=""><a href="../../../index.html" title="Documentation">Documentation</a><span class="divider">/</span></li> <li class="active ">Ambari Management Pack Development</li> - <li id="publishDate" class="pull-right"><span class="divider">|</span> Last Published: 2018-06-07</li> - <li id="projectVersion" class="pull-right">Version: 0.5.0</li> + <li id="publishDate" class="pull-right"><span class="divider">|</span> Last Published: 2018-09-12</li> + <li id="projectVersion" class="pull-right">Version: 0.6.0</li> </ul> </div> <div class="row-fluid"> http://git-wip-us.apache.org/repos/asf/metron/blob/a97e575f/site/current-book/metron-deployment/packaging/ambari/metron-mpack/index.html ---------------------------------------------------------------------- diff --git a/site/current-book/metron-deployment/packaging/ambari/metron-mpack/index.html b/site/current-book/metron-deployment/packaging/ambari/metron-mpack/index.html index dd97717..ddf0663 100644 --- a/site/current-book/metron-deployment/packaging/ambari/metron-mpack/index.html +++ b/site/current-book/metron-deployment/packaging/ambari/metron-mpack/index.html @@ -1,13 +1,13 @@ <!DOCTYPE html> <!-- - | Generated by Apache Maven Doxia Site Renderer 1.8 from src/site/markdown/metron-deployment/packaging/ambari/metron-mpack/index.md at 2018-06-07 + | Generated by Apache Maven Doxia Site Renderer 1.8 from src/site/markdown/metron-deployment/packaging/ambari/metron-mpack/index.md at 2018-09-12 | Rendered using Apache Maven Fluido Skin 1.7 --> <html xmlns="http://www.w3.org/1999/xhtml"; xml:lang="en" lang="en"> <head> <meta charset="UTF-8" /> <meta name="viewport" content="width=device-width, initial-scale=1.0" /> - <meta name="Date-Revision-yyyymmdd" content="20180607" /> + <meta name="Date-Revision-yyyymmdd" content="20180912" /> <meta http-equiv="Content-Language" content="en" /> <title>Metron – </title> <link rel="stylesheet" href="../../../../css/apache-maven-fluido-1.7.min.css" /> @@ -32,8 +32,8 @@ <li class=""><a href="http://metron.apache.org/"; class="externalLink" title="Metron">Metron</a><span class="divider">/</span></li> <li class=""><a href="../../../../index.html" title="Documentation">Documentation</a><span class="divider">/</span></li> <li class="active "></li> - <li id="publishDate" class="pull-right"><span class="divider">|</span> Last Published: 2018-06-07</li> - <li id="projectVersion" class="pull-right">Version: 0.5.0</li> + <li id="publishDate" class="pull-right"><span class="divider">|</span> Last Published: 2018-09-12</li> + <li id="projectVersion" class="pull-right">Version: 0.6.0</li> </ul> </div> <div class="row-fluid"> http://git-wip-us.apache.org/repos/asf/metron/blob/a97e575f/site/current-book/metron-deployment/packaging/docker/ansible-docker/index.html ---------------------------------------------------------------------- diff --git a/site/current-book/metron-deployment/packaging/docker/ansible-docker/index.html b/site/current-book/metron-deployment/packaging/docker/ansible-docker/index.html index 8145c8f..4c85d3f 100644 --- a/site/current-book/metron-deployment/packaging/docker/ansible-docker/index.html +++ b/site/current-book/metron-deployment/packaging/docker/ansible-docker/index.html @@ -1,13 +1,13 @@ <!DOCTYPE html> <!-- - | Generated by Apache Maven Doxia Site Renderer 1.8 from src/site/markdown/metron-deployment/packaging/docker/ansible-docker/index.md at 2018-06-07 + | Generated by Apache Maven Doxia Site Renderer 1.8 from src/site/markdown/metron-deployment/packaging/docker/ansible-docker/index.md at 2018-09-12 | Rendered using Apache Maven Fluido Skin 1.7 --> <html xmlns="http://www.w3.org/1999/xhtml"; xml:lang="en" lang="en"> <head> <meta charset="UTF-8" /> <meta name="viewport" content="width=device-width, initial-scale=1.0" /> - <meta name="Date-Revision-yyyymmdd" content="20180607" /> + <meta name="Date-Revision-yyyymmdd" content="20180912" /> <meta http-equiv="Content-Language" content="en" /> <title>Metron – </title> <link rel="stylesheet" href="../../../../css/apache-maven-fluido-1.7.min.css" /> @@ -32,8 +32,8 @@ <li class=""><a href="http://metron.apache.org/"; class="externalLink" title="Metron">Metron</a><span class="divider">/</span></li> <li class=""><a href="../../../../index.html" title="Documentation">Documentation</a><span class="divider">/</span></li> <li class="active "></li> - <li id="publishDate" class="pull-right"><span class="divider">|</span> Last Published: 2018-06-07</li> - <li id="projectVersion" class="pull-right">Version: 0.5.0</li> + <li id="publishDate" class="pull-right"><span class="divider">|</span> Last Published: 2018-09-12</li> + <li id="projectVersion" class="pull-right">Version: 0.6.0</li> </ul> </div> <div class="row-fluid"> http://git-wip-us.apache.org/repos/asf/metron/blob/a97e575f/site/current-book/metron-deployment/packaging/docker/deb-docker/index.html ---------------------------------------------------------------------- diff --git a/site/current-book/metron-deployment/packaging/docker/deb-docker/index.html b/site/current-book/metron-deployment/packaging/docker/deb-docker/index.html index a83a363..8dd1e3f 100644 --- a/site/current-book/metron-deployment/packaging/docker/deb-docker/index.html +++ b/site/current-book/metron-deployment/packaging/docker/deb-docker/index.html @@ -1,13 +1,13 @@ <!DOCTYPE html> <!-- - | Generated by Apache Maven Doxia Site Renderer 1.8 from src/site/markdown/metron-deployment/packaging/docker/deb-docker/index.md at 2018-06-07 + | Generated by Apache Maven Doxia Site Renderer 1.8 from src/site/markdown/metron-deployment/packaging/docker/deb-docker/index.md at 2018-09-12 | Rendered using Apache Maven Fluido Skin 1.7 --> <html xmlns="http://www.w3.org/1999/xhtml"; xml:lang="en" lang="en"> <head> <meta charset="UTF-8" /> <meta name="viewport" content="width=device-width, initial-scale=1.0" /> - <meta name="Date-Revision-yyyymmdd" content="20180607" /> + <meta name="Date-Revision-yyyymmdd" content="20180912" /> <meta http-equiv="Content-Language" content="en" /> <title>Metron – </title> <link rel="stylesheet" href="../../../../css/apache-maven-fluido-1.7.min.css" /> @@ -32,8 +32,8 @@ <li class=""><a href="http://metron.apache.org/"; class="externalLink" title="Metron">Metron</a><span class="divider">/</span></li> <li class=""><a href="../../../../index.html" title="Documentation">Documentation</a><span class="divider">/</span></li> <li class="active "></li> - <li id="publishDate" class="pull-right"><span class="divider">|</span> Last Published: 2018-06-07</li> - <li id="projectVersion" class="pull-right">Version: 0.5.0</li> + <li id="publishDate" class="pull-right"><span class="divider">|</span> Last Published: 2018-09-12</li> + <li id="projectVersion" class="pull-right">Version: 0.6.0</li> </ul> </div> <div class="row-fluid"> http://git-wip-us.apache.org/repos/asf/metron/blob/a97e575f/site/current-book/metron-deployment/packaging/docker/rpm-docker/index.html ---------------------------------------------------------------------- diff --git a/site/current-book/metron-deployment/packaging/docker/rpm-docker/index.html b/site/current-book/metron-deployment/packaging/docker/rpm-docker/index.html index d684c0d..4d0cee3 100644 --- a/site/current-book/metron-deployment/packaging/docker/rpm-docker/index.html +++ b/site/current-book/metron-deployment/packaging/docker/rpm-docker/index.html @@ -1,13 +1,13 @@ <!DOCTYPE html> <!-- - | Generated by Apache Maven Doxia Site Renderer 1.8 from src/site/markdown/metron-deployment/packaging/docker/rpm-docker/index.md at 2018-06-07 + | Generated by Apache Maven Doxia Site Renderer 1.8 from src/site/markdown/metron-deployment/packaging/docker/rpm-docker/index.md at 2018-09-12 | Rendered using Apache Maven Fluido Skin 1.7 --> <html xmlns="http://www.w3.org/1999/xhtml"; xml:lang="en" lang="en"> <head> <meta charset="UTF-8" /> <meta name="viewport" content="width=device-width, initial-scale=1.0" /> - <meta name="Date-Revision-yyyymmdd" content="20180607" /> + <meta name="Date-Revision-yyyymmdd" content="20180912" /> <meta http-equiv="Content-Language" content="en" /> <title>Metron – </title> <link rel="stylesheet" href="../../../../css/apache-maven-fluido-1.7.min.css" /> @@ -32,8 +32,8 @@ <li class=""><a href="http://metron.apache.org/"; class="externalLink" title="Metron">Metron</a><span class="divider">/</span></li> <li class=""><a href="../../../../index.html" title="Documentation">Documentation</a><span class="divider">/</span></li> <li class="active "></li> - <li id="publishDate" class="pull-right"><span class="divider">|</span> Last Published: 2018-06-07</li> - <li id="projectVersion" class="pull-right">Version: 0.5.0</li> + <li id="publishDate" class="pull-right"><span class="divider">|</span> Last Published: 2018-09-12</li> + <li id="projectVersion" class="pull-right">Version: 0.6.0</li> </ul> </div> <div class="row-fluid"> http://git-wip-us.apache.org/repos/asf/metron/blob/a97e575f/site/current-book/metron-deployment/packaging/packer-build/index.html ---------------------------------------------------------------------- diff --git a/site/current-book/metron-deployment/packaging/packer-build/index.html b/site/current-book/metron-deployment/packaging/packer-build/index.html index fb4b71b..63a85b0 100644 --- a/site/current-book/metron-deployment/packaging/packer-build/index.html +++ b/site/current-book/metron-deployment/packaging/packer-build/index.html @@ -1,13 +1,13 @@ <!DOCTYPE html> <!-- - | Generated by Apache Maven Doxia Site Renderer 1.8 from src/site/markdown/metron-deployment/packaging/packer-build/index.md at 2018-06-07 + | Generated by Apache Maven Doxia Site Renderer 1.8 from src/site/markdown/metron-deployment/packaging/packer-build/index.md at 2018-09-12 | Rendered using Apache Maven Fluido Skin 1.7 --> <html xmlns="http://www.w3.org/1999/xhtml"; xml:lang="en" lang="en"> <head> <meta charset="UTF-8" /> <meta name="viewport" content="width=device-width, initial-scale=1.0" /> - <meta name="Date-Revision-yyyymmdd" content="20180607" /> + <meta name="Date-Revision-yyyymmdd" content="20180912" /> <meta http-equiv="Content-Language" content="en" /> <title>Metron – Build Metron Images</title> <link rel="stylesheet" href="../../../css/apache-maven-fluido-1.7.min.css" /> @@ -32,8 +32,8 @@ <li class=""><a href="http://metron.apache.org/"; class="externalLink" title="Metron">Metron</a><span class="divider">/</span></li> <li class=""><a href="../../../index.html" title="Documentation">Documentation</a><span class="divider">/</span></li> <li class="active ">Build Metron Images</li> - <li id="publishDate" class="pull-right"><span class="divider">|</span> Last Published: 2018-06-07</li> - <li id="projectVersion" class="pull-right">Version: 0.5.0</li> + <li id="publishDate" class="pull-right"><span class="divider">|</span> Last Published: 2018-09-12</li> + <li id="projectVersion" class="pull-right">Version: 0.6.0</li> </ul> </div> <div class="row-fluid"> http://git-wip-us.apache.org/repos/asf/metron/blob/a97e575f/site/current-book/metron-interface/metron-alerts/index.html ---------------------------------------------------------------------- diff --git a/site/current-book/metron-interface/metron-alerts/index.html b/site/current-book/metron-interface/metron-alerts/index.html index 9459bd1..f4c3d04 100644 --- a/site/current-book/metron-interface/metron-alerts/index.html +++ b/site/current-book/metron-interface/metron-alerts/index.html @@ -1,13 +1,13 @@ <!DOCTYPE html> <!-- - | Generated by Apache Maven Doxia Site Renderer 1.8 from src/site/markdown/metron-interface/metron-alerts/index.md at 2018-06-07 + | Generated by Apache Maven Doxia Site Renderer 1.8 from src/site/markdown/metron-interface/metron-alerts/index.md at 2018-09-12 | Rendered using Apache Maven Fluido Skin 1.7 --> <html xmlns="http://www.w3.org/1999/xhtml"; xml:lang="en" lang="en"> <head> <meta charset="UTF-8" /> <meta name="viewport" content="width=device-width, initial-scale=1.0" /> - <meta name="Date-Revision-yyyymmdd" content="20180607" /> + <meta name="Date-Revision-yyyymmdd" content="20180912" /> <meta http-equiv="Content-Language" content="en" /> <title>Metron – </title> <link rel="stylesheet" href="../../css/apache-maven-fluido-1.7.min.css" /> @@ -32,8 +32,8 @@ <li class=""><a href="http://metron.apache.org/"; class="externalLink" title="Metron">Metron</a><span class="divider">/</span></li> <li class=""><a href="../../index.html" title="Documentation">Documentation</a><span class="divider">/</span></li> <li class="active "></li> - <li id="publishDate" class="pull-right"><span class="divider">|</span> Last Published: 2018-06-07</li> - <li id="projectVersion" class="pull-right">Version: 0.5.0</li> + <li id="publishDate" class="pull-right"><span class="divider">|</span> Last Published: 2018-09-12</li> + <li id="projectVersion" class="pull-right">Version: 0.6.0</li> </ul> </div> <div class="row-fluid"> @@ -118,7 +118,8 @@ limitations under the License. <h2><a name="Prerequisites"></a>Prerequisites</h2> <ul> -<li>The Metron REST application should be up and running and Elasticsearch should have some alerts populated by Metron topologies</li> +<li>The Metron REST application should be up and running</li> +<li>Elasticsearch or Solr should have some alerts populated by Metron topologies, depending on which real-time store is enabled</li> <li>The Management UI should be installed (which includes <a class="externalLink" href="https://expressjs.com/";>Express</a>)</li> <li>The alerts can be populated using Full Dev or any other setup</li> <li>UI is developed using angular4 and uses angular-cli</li> @@ -202,7 +203,10 @@ rest: <h2><a name="Global_Configuration_Properties"></a>Global Configuration Properties</h2> <div class="section"> <h3><a name="source.type.field"></a><tt>source.type.field</tt></h3> -<p>The source type format used. Defaults to <tt>source:type</tt>.</p></div></div> +<p>The source type field name used in the real-time store. Defaults to <tt>source:type</tt>.</p></div> +<div class="section"> +<h3><a name="threat.triage.score.field"></a><tt>threat.triage.score.field</tt></h3> +<p>The threat triage score field name used in the real-time store. Defaults to <tt>threat:triage:score</tt>.</p></div></div> <div class="section"> <h2><a name="Usage"></a>Usage</h2> <p>After configuration is complete, the Management UI can be managed as a service:</p> @@ -237,12 +241,34 @@ npm install <p><b>NOTE</b>: <i>In the development mode ui by default connects to REST at <a class="externalLink" href="http://node1:8082";>http://node1:8082</a> for fetching data. If you wish to change it you can change the REST url at metron/metron-interface/metron-alerts/proxy.conf.json</i></p></div> <div class="section"> <h2><a name="E2E_Tests"></a>E2E Tests</h2> -<p>An expressjs server is available for mocking the elastic search api.</p> +<div class="section"> +<h3><a name="Caveats"></a>Caveats</h3> <ol style="list-style-type: decimal"> <li> -<p>Run e2e webserver :</p> +<p>E2E tests uses data from full-dev wherever applicable. The tests assume rest-api’s are available @<a class="externalLink" href="http://node1:8082";>http://node1:8082</a>. It is recommended to shutdown all other Metron services while running the E2E tests including Parsers, Enrichment, Indexing and the Profiler.</p> +</li> +<li> + +<p>E2E tests are run on headless chrome. To see the chrome browser in action, remove the ‘–headless’ parameter of chromeOptions in metron/metron-interface/metron-alerts/protractor.conf.js file</p> +</li> +<li> + +<p>E2E tests delete all the data in HBase table ‘metron_update’ and Elastic search index ‘meta_alerts_index’ for testing against its test data</p> +</li> +<li> + +<p>E2E tests use <a class="externalLink" href="https://github.com/NickTomlin/protractor-flake";>protractor-flake</a> to re-run flaky tests.</p> +</li> +</ol></div> +<div class="section"> +<h3><a name="Steps_to_run"></a>Steps to run</h3> +<ol style="list-style-type: decimal"> + +<li> + +<p>An Express.js server is available for accessing the rest api. Run the e2e webserver:</p> <div> <div> @@ -252,7 +278,7 @@ sh ./scripts/start-server-for-e2e.sh </li> <li> -<p>run e2e test using the following command</p> +<p>Run e2e tests using the following command:</p> <div> <div> @@ -260,12 +286,8 @@ sh ./scripts/start-server-for-e2e.sh npm run e2e </pre></div></div> </li> -<li> - -<p>E2E tests uses data from full-dev wherever applicable. The tests assume rest-api’s are available @<a class="externalLink" href="http://node1:8082";>http://node1:8082</a></p> -</li> </ol> -<p><b>NOTE</b>: <i>e2e tests covers all the general workflows and we will extend them as we need</i></p></div> +<p><b>NOTE</b>: <i>e2e tests cover all the general workflows and we will extend them as we need</i></p></div></div> </div> </div> </div> http://git-wip-us.apache.org/repos/asf/metron/blob/a97e575f/site/current-book/metron-interface/metron-config/index.html ---------------------------------------------------------------------- diff --git a/site/current-book/metron-interface/metron-config/index.html b/site/current-book/metron-interface/metron-config/index.html index 0c4c073..c46f751 100644 --- a/site/current-book/metron-interface/metron-config/index.html +++ b/site/current-book/metron-interface/metron-config/index.html @@ -1,13 +1,13 @@ <!DOCTYPE html> <!-- - | Generated by Apache Maven Doxia Site Renderer 1.8 from src/site/markdown/metron-interface/metron-config/index.md at 2018-06-07 + | Generated by Apache Maven Doxia Site Renderer 1.8 from src/site/markdown/metron-interface/metron-config/index.md at 2018-09-12 | Rendered using Apache Maven Fluido Skin 1.7 --> <html xmlns="http://www.w3.org/1999/xhtml"; xml:lang="en" lang="en"> <head> <meta charset="UTF-8" /> <meta name="viewport" content="width=device-width, initial-scale=1.0" /> - <meta name="Date-Revision-yyyymmdd" content="20180607" /> + <meta name="Date-Revision-yyyymmdd" content="20180912" /> <meta http-equiv="Content-Language" content="en" /> <title>Metron – Metron Management UI</title> <link rel="stylesheet" href="../../css/apache-maven-fluido-1.7.min.css" /> @@ -32,8 +32,8 @@ <li class=""><a href="http://metron.apache.org/"; class="externalLink" title="Metron">Metron</a><span class="divider">/</span></li> <li class=""><a href="../../index.html" title="Documentation">Documentation</a><span class="divider">/</span></li> <li class="active ">Metron Management UI</li> - <li id="publishDate" class="pull-right"><span class="divider">|</span> Last Published: 2018-06-07</li> - <li id="projectVersion" class="pull-right">Version: 0.5.0</li> + <li id="publishDate" class="pull-right"><span class="divider">|</span> Last Published: 2018-09-12</li> + <li id="projectVersion" class="pull-right">Version: 0.6.0</li> </ul> </div> <div class="row-fluid"> http://git-wip-us.apache.org/repos/asf/metron/blob/a97e575f/site/current-book/metron-interface/metron-rest/index.html ---------------------------------------------------------------------- diff --git a/site/current-book/metron-interface/metron-rest/index.html b/site/current-book/metron-interface/metron-rest/index.html index eb378b3..b0f91d7 100644 --- a/site/current-book/metron-interface/metron-rest/index.html +++ b/site/current-book/metron-interface/metron-rest/index.html @@ -1,13 +1,13 @@ <!DOCTYPE html> <!-- - | Generated by Apache Maven Doxia Site Renderer 1.8 from src/site/markdown/metron-interface/metron-rest/index.md at 2018-06-07 + | Generated by Apache Maven Doxia Site Renderer 1.8 from src/site/markdown/metron-interface/metron-rest/index.md at 2018-09-12 | Rendered using Apache Maven Fluido Skin 1.7 --> <html xmlns="http://www.w3.org/1999/xhtml"; xml:lang="en" lang="en"> <head> <meta charset="UTF-8" /> <meta name="viewport" content="width=device-width, initial-scale=1.0" /> - <meta name="Date-Revision-yyyymmdd" content="20180607" /> + <meta name="Date-Revision-yyyymmdd" content="20180912" /> <meta http-equiv="Content-Language" content="en" /> <title>Metron – Metron REST</title> <link rel="stylesheet" href="../../css/apache-maven-fluido-1.7.min.css" /> @@ -32,8 +32,8 @@ <li class=""><a href="http://metron.apache.org/"; class="externalLink" title="Metron">Metron</a><span class="divider">/</span></li> <li class=""><a href="../../index.html" title="Documentation">Documentation</a><span class="divider">/</span></li> <li class="active ">Metron REST</li> - <li id="publishDate" class="pull-right"><span class="divider">|</span> Last Published: 2018-06-07</li> - <li id="projectVersion" class="pull-right">Version: 0.5.0</li> + <li id="publishDate" class="pull-right"><span class="divider">|</span> Last Published: 2018-09-12</li> + <li id="projectVersion" class="pull-right">Version: 0.6.0</li> </ul> </div> <div class="row-fluid"> @@ -97,6 +97,7 @@ limitations under the License. <ul> <li>A running Metron cluster</li> +<li>A running real-time store, either Elasticsearch or Solr depending on which one is enabled</li> <li>Java 8 installed</li> <li>Storm CLI and Metron topology scripts (start_parser_topology.sh, start_enrichment_topology.sh, start_elasticsearch_topology.sh) installed</li> <li>A relational database</li> @@ -429,6 +430,23 @@ METRON_SERVICE_KEYTAB="/etc/security/keytabs/metron.keytab" </pre></div></div> </div> <div class="section"> +<h2><a name="Pcap_Query"></a>Pcap Query</h2> +<p>The REST application exposes endpoints for querying Pcap data. For more information about filtering options see <a href="../../metron-platform/metron-pcap-backend/index.html#Query_Filter_Utility">Query Filter Utility</a>.</p> +<p>There is an endpoint available that will return Pcap data in <a class="externalLink" href="https://wiki.wireshark.org/PDML";>PDML</a> format. <a class="externalLink" href="https://www.wireshark.org/";>Wireshark</a> must be installed for this feature to work. Installing wireshark in CentOS can be done with <tt>yum -y install wireshark</tt>.</p> +<p>The REST application uses a Java Process object to call out to the <tt>pcap_to_pdml.sh</tt> script. This script is installed at <tt>$METRON_HOME/bin/pcap_to_pdml.sh</tt> by default. Out of the box it is a simple wrapper around the tshark command to transform raw pcap data to PDML. However it can be extended to do additional processing as long as the expected input/output is maintained. REST will supply the script with raw pcap data through standard in and expects PDML data serialized as XML.</p> +<p>Pcap query jobs can be configured for submission to a YARN queue. This setting is exposed as the Spring property <tt>pcap.yarn.queue</tt>. If configured, the REST application will set the <tt>mapreduce.job.queuename</tt> Hadoop property to that value. It is highly recommended that a dedicated YARN queue be created and configured for Pcap queries to prevent a job from consuming too many cluster resources. More information about setting up YARN queues can be found <a class="externalLink" href="https://hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-site/CapacityScheduler.html#Setting_up_queues";>here</a>.</p> +<p>Pcap query results are stored in HDFS. The location of query results when run through the REST app is determined by a couple factors. The root of Pcap query results defaults to <tt>/apps/metron/pcap/output</tt> but can be changed with the Spring property <tt>pcap.final.output.path</tt>. Assuming the default Pcap query output directory, the path to a result page will follow this pattern:</p> + +<div> +<div> +<pre class="source">/apps/metron/pcap/output/{username}/MAP_REDUCE/{job id}/page-{page number}.pcap +</pre></div></div> + +<p>Over time Pcap query results will accumulate in HDFS. Currently these results are not cleaned up automatically so cluster administrators should be aware of this and monitor them. It is highly recommended that a process be put in place to periodically delete files and directories under the Pcap query results root.</p> +<p>Users should also be mindful of date ranges used in queries so they don’t produce result sets that are too large. Currently there are no limits enforced on date ranges.</p> +<p>Queries can also be configured on a global level for setting the number of results per page via a Spring property <tt>pcap.page.size</tt>. By default, this value is set to 10 pcaps per page, but you may choose to set this value higher based on observing frequenetly-run query result sizes. This setting works in conjunction with the property for setting finalizer threadpool size when optimizing query performance.</p> +<p>Pcap query jobs have a finalization routine that writes their results out to HDFS in pages. Depending on the size of your pcaps, the number or results typically returned, page sizing (described above), and available CPU cores for running your REST application, your performance can be improved by adjusting the number of files that can be written to HDFS in parallel. To this end, there is a threadpool used for this finalization step that can be configured to use a specified number of threads. This setting is exposed as the Spring property <tt>pcap.finalizer.threadpool.size</tt>. A default value of “1” is used if not specified by the user. Generally speaking, you should see a performance gain when this value is set to anything higher than 1. A sizeable increase in performance can be achieved, especially for larger numbers of files of smaller size, by increasing the number of threads. It should be noted that this property is parsed as a String to allow for more complex parallelism values. In addition to normal integer values, you can specify a multiple of the number of cores. If it’s a string and ends with “C”, then strip the C and treat it as an integral multiple of the number of cores. If it’s a string and does not end with a C, then treat it as a number in string form.</p></div> +<div class="section"> <h2><a name="API"></a>API</h2> <p>Request and Response objects are JSON formatted. The JSON schemas are available in the Swagger UI.</p> <table border="0" class="table table-striped"> @@ -439,7 +457,7 @@ METRON_SERVICE_KEYTAB="/etc/security/keytabs/metron.keytab" </thead><tbody> <tr class="b"> -<td> <a href="#get-apiv1alertsuiescalate"> <tt>POST /api/v1/alerts/ui/escalate</tt></a></td></tr> +<td> <a href="#POST_apiv1alertsuiescalate"> <tt>POST /api/v1/alerts/ui/escalate</tt></a></td></tr> <tr class="a"> <td> <a href="#GET_apiv1alertsuisettings"> <tt>GET /api/v1/alerts/ui/settings</tt></a></td></tr> <tr class="b"> @@ -491,11 +509,27 @@ METRON_SERVICE_KEYTAB="/etc/security/keytabs/metron.keytab" <tr class="a"> <td> <a href="#GET_apiv1metaalertupdatestatusguidstatus"> <tt>GET /api/v1/metaalert/update/status/{guid}/{status}</tt></a></td></tr> <tr class="b"> +<td> <a href="#POST_apiv1pcapfixed"> <tt>POST /api/v1/pcap/fixed</tt></a></td></tr> +<tr class="a"> +<td> <a href="#POST_apiv1pcapquery"> <tt>POST /api/v1/pcap/query</tt></a></td></tr> +<tr class="b"> +<td> <a href="#GET_apiv1pcap"> <tt>GET /api/v1/pcap</tt></a></td></tr> +<tr class="a"> +<td> <a href="#GET_apiv1pcapjobId"> <tt>GET /api/v1/pcap/{jobId}</tt></a></td></tr> +<tr class="b"> +<td> <a href="#GET_apiv1pcapjobIdpdml"> <tt>GET /api/v1/pcap/{jobId}/pdml</tt></a></td></tr> +<tr class="a"> +<td> <a href="#GET_apiv1pcapjobIdraw"> <tt>GET /api/v1/pcap/{jobId}/raw</tt></a></td></tr> +<tr class="b"> +<td> <a href="#DELETE_apiv1pcapkilljobId"> <tt>DELETE /api/v1/pcap/kill/{jobId}</tt></a></td></tr> +<tr class="a"> +<td> <a href="#GET_apiv1pcapjobIdconfig"> <tt>GET /api/v1/pcap/{jobId}/config</tt></a></td></tr> +<tr class="b"> <td> <a href="#GET_apiv1searchsearch"> <tt>GET /api/v1/search/search</tt></a></td></tr> <tr class="a"> -<td> <a href="#get-apiv1searchsearch"> <tt>POST /api/v1/search/search</tt></a></td></tr> +<td> <a href="#POST_apiv1searchsearch"> <tt>POST /api/v1/search/search</tt></a></td></tr> <tr class="b"> -<td> <a href="#get-apiv1searchgroup"> <tt>POST /api/v1/search/group</tt></a></td></tr> +<td> <a href="#POST_apiv1searchgroup"> <tt>POST /api/v1/search/group</tt></a></td></tr> <tr class="a"> <td> <a href="#GET_apiv1searchfindOne"> <tt>GET /api/v1/search/findOne</tt></a></td></tr> <tr class="b"> @@ -593,7 +627,7 @@ METRON_SERVICE_KEYTAB="/etc/security/keytabs/metron.keytab" <tr class="b"> <td> <a href="#PATCH_apiv1updatepatch"> <tt>PATCH /api/v1/update/patch</tt></a></td></tr> <tr class="a"> -<td> <a href="#patch-apiv1updatereplace"> <tt>PUT /api/v1/update/replace</tt></a></td></tr> +<td> <a href="#PUT_apiv1updatereplace"> <tt>PUT /api/v1/update/replace</tt></a></td></tr> <tr class="b"> <td> <a href="#GET_apiv1user"> <tt>GET /api/v1/user</tt></a></td></tr> </tbody> @@ -1051,6 +1085,156 @@ METRON_SERVICE_KEYTAB="/etc/security/keytabs/metron.keytab" </li> </ul></div> <div class="section"> +<h3><a name="POST_.2Fapi.2Fv1.2Fpcap.2Ffixed"></a><tt>POST /api/v1/pcap/fixed</tt></h3> +<ul> + +<li>Description: Executes a Fixed Filter Pcap Query.</li> +<li>Input: +<ul> + +<li>fixedPcapRequest - A Fixed Pcap Request which includes fixed filter fields like ip source address and protocol</li> +</ul> +</li> +<li>Returns: +<ul> + +<li>200 - Returns a job status with job ID.</li> +</ul> +</li> +</ul></div> +<div class="section"> +<h3><a name="POST_.2Fapi.2Fv1.2Fpcap.2Fquery"></a><tt>POST /api/v1/pcap/query</tt></h3> +<ul> + +<li>Description: Executes a Query Filter Pcap Query.</li> +<li>Input: +<ul> + +<li>queryPcapRequest - A Query Pcap Request which includes Stellar query field</li> +</ul> +</li> +<li>Returns: +<ul> + +<li>200 - Returns a job status with job ID.</li> +</ul> +</li> +</ul></div> +<div class="section"> +<h3><a name="GET_.2Fapi.2Fv1.2Fpcap"></a><tt>GET /api/v1/pcap</tt></h3> +<ul> + +<li>Description: Gets a list of job statuses for Pcap query jobs that match the requested state.</li> +<li>Input: +<ul> + +<li>state - Job state</li> +</ul> +</li> +<li>Returns: +<ul> + +<li>200 - Returns a list of job statuses for jobs that match the requested state.</li> +</ul> +</li> +</ul></div> +<div class="section"> +<h3><a name="GET_.2Fapi.2Fv1.2Fpcap.2F.7BjobId.7D"></a><tt>GET /api/v1/pcap/{jobId}</tt></h3> +<ul> + +<li>Description: Gets job status for Pcap query job.</li> +<li>Input: +<ul> + +<li>jobId - Job ID of submitted job</li> +</ul> +</li> +<li>Returns: +<ul> + +<li>200 - Returns a job status for the Job ID.</li> +<li>404 - Job is missing.</li> +</ul> +</li> +</ul></div> +<div class="section"> +<h3><a name="GET_.2Fapi.2Fv1.2Fpcap.2F.7BjobId.7D.2Fpdml"></a><tt>GET /api/v1/pcap/{jobId}/pdml</tt></h3> +<ul> + +<li>Description: Gets Pcap Results for a page in PDML format.</li> +<li>Input: +<ul> + +<li>jobId - Job ID of submitted job</li> +<li>page - Page number</li> +</ul> +</li> +<li>Returns: +<ul> + +<li>200 - Returns PDML in json format.</li> +<li>404 - Job or page is missing.</li> +</ul> +</li> +</ul></div> +<div class="section"> +<h3><a name="GET_.2Fapi.2Fv1.2Fpcap.2F.7BjobId.7D.2Fraw"></a><tt>GET /api/v1/pcap/{jobId}/raw</tt></h3> +<ul> + +<li>Description: Download Pcap Results for a page.</li> +<li>Input: +<ul> + +<li>jobId - Job ID of submitted job</li> +<li>page - Page number</li> +</ul> +</li> +<li>Returns: +<ul> + +<li>200 - Returns Pcap as a file download.</li> +<li>404 - Job or page is missing.</li> +</ul> +</li> +</ul></div> +<div class="section"> +<h3><a name="DELETE_.2Fapi.2Fv1.2Fpcap.2Fkill.2F.7BjobId.7D"></a><tt>DELETE /api/v1/pcap/kill/{jobId}</tt></h3> +<ul> + +<li>Description: Kills running job.</li> +<li>Input: +<ul> + +<li>jobId - Job ID of submitted job</li> +</ul> +</li> +<li>Returns: +<ul> + +<li>200 - Kills passed job.</li> +</ul> +</li> +</ul></div> +<div class="section"> +<h3><a name="GET_.2Fapi.2Fv1.2Fpcap.2F.7BjobId.7D.2Fconfig"></a><tt>GET /api/v1/pcap/{jobId}/config</tt></h3> +<ul> + +<li>Description: Gets job configuration for Pcap query job.</li> +<li>Input: +<ul> + +<li>jobId - Job ID of submitted job</li> +</ul> +</li> +<li>Returns: +<ul> + +<li>200 - Returns a map of job properties for the Job ID.</li> +<li>404 - Job is missing.</li> +</ul> +</li> +</ul></div> +<div class="section"> <h3><a name="POST_.2Fapi.2Fv1.2Fsearch.2Fsearch"></a><tt>POST /api/v1/search/search</tt></h3> <ul> @@ -1865,8 +2049,8 @@ METRON_SERVICE_KEYTAB="/etc/security/keytabs/metron.keytab" <li>Returns: <ul> -<li>200 - nothing</li> -<li>404 - document not found</li> +<li>200 - Nothing</li> +<li>404 - Document not found</li> </ul> </li> </ul></div> http://git-wip-us.apache.org/repos/asf/metron/blob/a97e575f/site/current-book/metron-platform/Performance-tuning-guide.html ---------------------------------------------------------------------- diff --git a/site/current-book/metron-platform/Performance-tuning-guide.html b/site/current-book/metron-platform/Performance-tuning-guide.html index 00d2907..b9134de 100644 --- a/site/current-book/metron-platform/Performance-tuning-guide.html +++ b/site/current-book/metron-platform/Performance-tuning-guide.html @@ -1,13 +1,13 @@ <!DOCTYPE html> <!-- - | Generated by Apache Maven Doxia Site Renderer 1.8 from src/site/markdown/metron-platform/Performance-tuning-guide.md at 2018-06-07 + | Generated by Apache Maven Doxia Site Renderer 1.8 from src/site/markdown/metron-platform/Performance-tuning-guide.md at 2018-09-12 | Rendered using Apache Maven Fluido Skin 1.7 --> <html xmlns="http://www.w3.org/1999/xhtml"; xml:lang="en" lang="en"> <head> <meta charset="UTF-8" /> <meta name="viewport" content="width=device-width, initial-scale=1.0" /> - <meta name="Date-Revision-yyyymmdd" content="20180607" /> + <meta name="Date-Revision-yyyymmdd" content="20180912" /> <meta http-equiv="Content-Language" content="en" /> <title>Metron – Metron Performance Tuning Guide</title> <link rel="stylesheet" href="../css/apache-maven-fluido-1.7.min.css" /> @@ -32,8 +32,8 @@ <li class=""><a href="http://metron.apache.org/"; class="externalLink" title="Metron">Metron</a><span class="divider">/</span></li> <li class=""><a href="../index.html" title="Documentation">Documentation</a><span class="divider">/</span></li> <li class="active ">Metron Performance Tuning Guide</li> - <li id="publishDate" class="pull-right"><span class="divider">|</span> Last Published: 2018-06-07</li> - <li id="projectVersion" class="pull-right">Version: 0.5.0</li> + <li id="publishDate" class="pull-right"><span class="divider">|</span> Last Published: 2018-09-12</li> + <li id="projectVersion" class="pull-right">Version: 0.6.0</li> </ul> </div> <div class="row-fluid"> @@ -55,15 +55,16 @@ <li><a href="../metron-platform/index.html" title="Platform"><span class="icon-chevron-down"></span>Platform</a> <ul class="nav nav-list"> <li class="active"><a href="#"><span class="none"></span>Performance-tuning-guide</a></li> - <li><a href="../metron-platform/metron-api/index.html" title="Api"><span class="none"></span>Api</a></li> <li><a href="../metron-platform/metron-common/index.html" title="Common"><span class="none"></span>Common</a></li> <li><a href="../metron-platform/metron-data-management/index.html" title="Data-management"><span class="none"></span>Data-management</a></li> <li><a href="../metron-platform/metron-elasticsearch/index.html" title="Elasticsearch"><span class="none"></span>Elasticsearch</a></li> <li><a href="../metron-platform/metron-enrichment/index.html" title="Enrichment"><span class="icon-chevron-right"></span>Enrichment</a></li> <li><a href="../metron-platform/metron-indexing/index.html" title="Indexing"><span class="none"></span>Indexing</a></li> + <li><a href="../metron-platform/metron-job/index.html" title="Job"><span class="none"></span>Job</a></li> <li><a href="../metron-platform/metron-management/index.html" title="Management"><span class="none"></span>Management</a></li> <li><a href="../metron-platform/metron-parsers/index.html" title="Parsers"><span class="icon-chevron-right"></span>Parsers</a></li> <li><a href="../metron-platform/metron-pcap-backend/index.html" title="Pcap-backend"><span class="none"></span>Pcap-backend</a></li> + <li><a href="../metron-platform/metron-solr/index.html" title="Solr"><span class="none"></span>Solr</a></li> <li><a href="../metron-platform/metron-writer/index.html" title="Writer"><span class="none"></span>Writer</a></li> </ul> </li> @@ -781,7 +782,7 @@ enrichments enrichments 43 29754331 297 <div> <div> -<pre class="source">/usr/metron/0.5.0/bin/start_parser_topology.sh \ +<pre class="source">/usr/metron/0.6.0/bin/start_parser_topology.sh \ -e ~metron/.storm/storm-bro.config \ -esc ~/.storm/spout-bro.config \ -k $BROKERLIST \ @@ -966,7 +967,7 @@ export KAFKA_HOME=$HDP_HOME/kafka-broker export STORM_UI=http://node1:8744 export ELASTIC=http://node1:9200 export ZOOKEEPER=node1:2181 -export METRON_VERSION=0.5.0 +export METRON_VERSION=0.6.0 export METRON_HOME=/usr/metron/${METRON_VERSION} </pre></div></div> http://git-wip-us.apache.org/repos/asf/metron/blob/a97e575f/site/current-book/metron-platform/index.html ---------------------------------------------------------------------- diff --git a/site/current-book/metron-platform/index.html b/site/current-book/metron-platform/index.html index 7819640..9fa8d70 100644 --- a/site/current-book/metron-platform/index.html +++ b/site/current-book/metron-platform/index.html @@ -1,13 +1,13 @@ <!DOCTYPE html> <!-- - | Generated by Apache Maven Doxia Site Renderer 1.8 from src/site/markdown/metron-platform/index.md at 2018-06-07 + | Generated by Apache Maven Doxia Site Renderer 1.8 from src/site/markdown/metron-platform/index.md at 2018-09-12 | Rendered using Apache Maven Fluido Skin 1.7 --> <html xmlns="http://www.w3.org/1999/xhtml"; xml:lang="en" lang="en"> <head> <meta charset="UTF-8" /> <meta name="viewport" content="width=device-width, initial-scale=1.0" /> - <meta name="Date-Revision-yyyymmdd" content="20180607" /> + <meta name="Date-Revision-yyyymmdd" content="20180912" /> <meta http-equiv="Content-Language" content="en" /> <title>Metron – Current Build</title> <link rel="stylesheet" href="../css/apache-maven-fluido-1.7.min.css" /> @@ -32,8 +32,8 @@ <li class=""><a href="http://metron.apache.org/"; class="externalLink" title="Metron">Metron</a><span class="divider">/</span></li> <li class=""><a href="../index.html" title="Documentation">Documentation</a><span class="divider">/</span></li> <li class="active ">Current Build</li> - <li id="publishDate" class="pull-right"><span class="divider">|</span> Last Published: 2018-06-07</li> - <li id="projectVersion" class="pull-right">Version: 0.5.0</li> + <li id="publishDate" class="pull-right"><span class="divider">|</span> Last Published: 2018-09-12</li> + <li id="projectVersion" class="pull-right">Version: 0.6.0</li> </ul> </div> <div class="row-fluid"> @@ -55,15 +55,16 @@ <li class="active"><a href="#"><span class="icon-chevron-down"></span>Platform</a> <ul class="nav nav-list"> <li><a href="../metron-platform/Performance-tuning-guide.html" title="Performance-tuning-guide"><span class="none"></span>Performance-tuning-guide</a></li> - <li><a href="../metron-platform/metron-api/index.html" title="Api"><span class="none"></span>Api</a></li> <li><a href="../metron-platform/metron-common/index.html" title="Common"><span class="none"></span>Common</a></li> <li><a href="../metron-platform/metron-data-management/index.html" title="Data-management"><span class="none"></span>Data-management</a></li> <li><a href="../metron-platform/metron-elasticsearch/index.html" title="Elasticsearch"><span class="none"></span>Elasticsearch</a></li> <li><a href="../metron-platform/metron-enrichment/index.html" title="Enrichment"><span class="icon-chevron-right"></span>Enrichment</a></li> <li><a href="../metron-platform/metron-indexing/index.html" title="Indexing"><span class="none"></span>Indexing</a></li> + <li><a href="../metron-platform/metron-job/index.html" title="Job"><span class="none"></span>Job</a></li> <li><a href="../metron-platform/metron-management/index.html" title="Management"><span class="none"></span>Management</a></li> <li><a href="../metron-platform/metron-parsers/index.html" title="Parsers"><span class="icon-chevron-right"></span>Parsers</a></li> <li><a href="../metron-platform/metron-pcap-backend/index.html" title="Pcap-backend"><span class="none"></span>Pcap-backend</a></li> + <li><a href="../metron-platform/metron-solr/index.html" title="Solr"><span class="none"></span>Solr</a></li> <li><a href="../metron-platform/metron-writer/index.html" title="Writer"><span class="none"></span>Writer</a></li> </ul> </li> @@ -104,7 +105,7 @@ limitations under the License. --> <h1>Current Build</h1> <p><a name="Current_Build"></a></p> -<p>The latest build of metron-platform is 0.5.0.</p> +<p>The latest build of metron-platform is 0.6.0.</p> <p>We are still in the process of merging/porting additional features from our production code base into this open source release. This release will be followed by a number of additional beta releases until the port is complete. We will also work on getting additional documentation and user/developer guides to the community as soon as we can. At this time we offer no support for the beta software, but will try to respond to requests as promptly as we can.</p> <p><a name="metron-platform"></a></p> <h1>metron-platform</h1> http://git-wip-us.apache.org/repos/asf/metron/blob/a97e575f/site/current-book/metron-platform/metron-api/index.html ---------------------------------------------------------------------- diff --git a/site/current-book/metron-platform/metron-api/index.html b/site/current-book/metron-platform/metron-api/index.html deleted file mode 100644 index ddcaec5..0000000 --- a/site/current-book/metron-platform/metron-api/index.html +++ /dev/null @@ -1,161 +0,0 @@ -<!DOCTYPE html> -<!-- - | Generated by Apache Maven Doxia Site Renderer 1.8 from src/site/markdown/metron-platform/metron-api/index.md at 2018-06-07 - | Rendered using Apache Maven Fluido Skin 1.7 ---> -<html xmlns="http://www.w3.org/1999/xhtml"; xml:lang="en" lang="en"> - <head> - <meta charset="UTF-8" /> - <meta name="viewport" content="width=device-width, initial-scale=1.0" /> - <meta name="Date-Revision-yyyymmdd" content="20180607" /> - <meta http-equiv="Content-Language" content="en" /> - <title>Metron – Metron PCAP Service</title> - <link rel="stylesheet" href="../../css/apache-maven-fluido-1.7.min.css" /> - <link rel="stylesheet" href="../../css/site.css" /> - <link rel="stylesheet" href="../../css/print.css" media="print" /> - <script type="text/javascript" src="../../js/apache-maven-fluido-1.7.min.js"></script> -<script type="text/javascript"> - $( document ).ready( function() { $( '.carousel' ).carousel( { interval: 3500 } ) } ); - </script> - </head> - <body class="topBarDisabled"> - <div class="container-fluid"> - <div id="banner"> - <div class="pull-left"><a href="http://metron.apache.org/"; id="bannerLeft"><img src="../../images/metron-logo.png" alt="Apache Metron" width="148px" height="48px"/></a></div> - <div class="pull-right"></div> - <div class="clear"><hr/></div> - </div> - - <div id="breadcrumbs"> - <ul class="breadcrumb"> - <li class=""><a href="http://www.apache.org"; class="externalLink" title="Apache">Apache</a><span class="divider">/</span></li> - <li class=""><a href="http://metron.apache.org/"; class="externalLink" title="Metron">Metron</a><span class="divider">/</span></li> - <li class=""><a href="../../index.html" title="Documentation">Documentation</a><span class="divider">/</span></li> - <li class="active ">Metron PCAP Service</li> - <li id="publishDate" class="pull-right"><span class="divider">|</span> Last Published: 2018-06-07</li> - <li id="projectVersion" class="pull-right">Version: 0.5.0</li> - </ul> - </div> - <div class="row-fluid"> - <div id="leftColumn" class="span2"> - <div class="well sidebar-nav"> - <ul class="nav nav-list"> - <li class="nav-header">User Documentation</li> - <li><a href="../../index.html" title="Metron"><span class="icon-chevron-down"></span>Metron</a> - <ul class="nav nav-list"> - <li><a href="../../CONTRIBUTING.html" title="CONTRIBUTING"><span class="none"></span>CONTRIBUTING</a></li> - <li><a href="../../Upgrading.html" title="Upgrading"><span class="none"></span>Upgrading</a></li> - <li><a href="../../metron-analytics/index.html" title="Analytics"><span class="icon-chevron-right"></span>Analytics</a></li> - <li><a href="../../metron-contrib/metron-docker/index.html" title="Docker"><span class="none"></span>Docker</a></li> - <li><a href="../../metron-contrib/metron-performance/index.html" title="Performance"><span class="none"></span>Performance</a></li> - <li><a href="../../metron-deployment/index.html" title="Deployment"><span class="icon-chevron-right"></span>Deployment</a></li> - <li><a href="../../metron-interface/metron-alerts/index.html" title="Alerts"><span class="none"></span>Alerts</a></li> - <li><a href="../../metron-interface/metron-config/index.html" title="Config"><span class="none"></span>Config</a></li> - <li><a href="../../metron-interface/metron-rest/index.html" title="Rest"><span class="none"></span>Rest</a></li> - <li><a href="../../metron-platform/index.html" title="Platform"><span class="icon-chevron-down"></span>Platform</a> - <ul class="nav nav-list"> - <li><a href="../../metron-platform/Performance-tuning-guide.html" title="Performance-tuning-guide"><span class="none"></span>Performance-tuning-guide</a></li> - <li class="active"><a href="#"><span class="none"></span>Api</a></li> - <li><a href="../../metron-platform/metron-common/index.html" title="Common"><span class="none"></span>Common</a></li> - <li><a href="../../metron-platform/metron-data-management/index.html" title="Data-management"><span class="none"></span>Data-management</a></li> - <li><a href="../../metron-platform/metron-elasticsearch/index.html" title="Elasticsearch"><span class="none"></span>Elasticsearch</a></li> - <li><a href="../../metron-platform/metron-enrichment/index.html" title="Enrichment"><span class="icon-chevron-right"></span>Enrichment</a></li> - <li><a href="../../metron-platform/metron-indexing/index.html" title="Indexing"><span class="none"></span>Indexing</a></li> - <li><a href="../../metron-platform/metron-management/index.html" title="Management"><span class="none"></span>Management</a></li> - <li><a href="../../metron-platform/metron-parsers/index.html" title="Parsers"><span class="icon-chevron-right"></span>Parsers</a></li> - <li><a href="../../metron-platform/metron-pcap-backend/index.html" title="Pcap-backend"><span class="none"></span>Pcap-backend</a></li> - <li><a href="../../metron-platform/metron-writer/index.html" title="Writer"><span class="none"></span>Writer</a></li> - </ul> -</li> - <li><a href="../../metron-sensors/index.html" title="Sensors"><span class="icon-chevron-right"></span>Sensors</a></li> - <li><a href="../../metron-stellar/stellar-3rd-party-example/index.html" title="Stellar-3rd-party-example"><span class="none"></span>Stellar-3rd-party-example</a></li> - <li><a href="../../metron-stellar/stellar-common/index.html" title="Stellar-common"><span class="icon-chevron-right"></span>Stellar-common</a></li> - <li><a href="../../metron-stellar/stellar-zeppelin/index.html" title="Stellar-zeppelin"><span class="none"></span>Stellar-zeppelin</a></li> - <li><a href="../../use-cases/index.html" title="Use-cases"><span class="icon-chevron-right"></span>Use-cases</a></li> - </ul> -</li> -</ul> - <hr /> - <div id="poweredBy"> - <div class="clear"></div> - <div class="clear"></div> - <div class="clear"></div> - <div class="clear"></div> -<a href="http://maven.apache.org/"; title="Built by Maven" class="poweredBy"><img class="builtBy" alt="Built by Maven" src="../../images/logos/maven-feather.png" /></a> - </div> - </div> - </div> - <div id="bodyColumn" class="span10" > -<!-- -Licensed to the Apache Software Foundation (ASF) under one -or more contributor license agreements. See the NOTICE file -distributed with this work for additional information -regarding copyright ownership. The ASF licenses this file -to you under the Apache License, Version 2.0 (the -"License"); you may not use this file except in compliance -with the License. You may obtain a copy of the License at - -http://www.apache.org/licenses/LICENSE-2.0 - -Unless required by applicable law or agreed to in writing, software -distributed under the License is distributed on an "AS IS" BASIS, -WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -See the License for the specific language governing permissions and -limitations under the License. ---> -<h1>Metron PCAP Service</h1> -<p><a name="Metron_PCAP_Service"></a></p> -<p>The purpose of the Metron PCAP service is to provide a middle tier to negotiate retrieving packet capture data which flows into Metron. This packet data is of a form which <tt>libpcap</tt> based tools can read.</p> -<div class="section"> -<h2><a name="Starting_the_Service"></a>Starting the Service</h2> -<p>You can start the service either via the init.d script installed, <tt>/etc/init.d/pcapservice</tt> or directly via the <tt>yarn jar</tt> command: <tt>yarn jar $METRON_HOME/lib/metron-api-$METRON_VERSION.jar org.apache.metron.pcapservice.rest.PcapService -port $SERVICE_PORT -query_hdfs_path $QUERY_PATH -pcap_hdfs_path $PCAP_PATH</tt></p> -<p>where</p> -<ul> - -<li><tt>METRON_HOME</tt> is the location of the metron installation</li> -<li><tt>METRON_VERSION</tt> is the version of the metron installation</li> -<li><tt>SERVICE_PORT</tt> is the port to bind the REST service to.</li> -<li><tt>QUERY_PATH</tt> is the temporary location to store query results. They are deleted after the service reads them.</li> -<li><tt>PCAP_PATH</tt> is the path to the packet data on HDFS</li> -</ul></div> -<div class="section"> -<h2><a name="The_.2FpcapGetter.2FgetPcapsByIdentifiers_endpoint"></a>The <tt>/pcapGetter/getPcapsByIdentifiers</tt> endpoint</h2> -<p>This endpoint takes the following query parameters and returns the subset of packets matching this query:</p> -<ul> - -<li><tt>srcIp</tt> : The source IP to match on</li> -<li><tt>srcPort</tt> : The source port to match on</li> -<li><tt>dstIp</tt> : The destination IP to match on</li> -<li><tt>dstPort</tt> : The destination port to match on</li> -<li><tt>startTime</tt> : The start time in milliseconds</li> -<li><tt>endTime</tt> : The end time in milliseconds</li> -<li><tt>numReducers</tt> : Specify the number of reducers to use when executing the mapreduce job</li> -<li><tt>includeReverseTraffic</tt> : Indicates if filter should check swapped src/dest addresses and IPs</li> -</ul></div> -<div class="section"> -<h2><a name="The_.2FpcapGetter.2FgetPcapsByQuery_endpoint"></a>The <tt>/pcapGetter/getPcapsByQuery</tt> endpoint</h2> -<p>This endpoint takes the following query parameters and returns the subset of packets matching this query. This endpoint exposes Stellar querying capabilities:</p> -<ul> - -<li><tt>query</tt> : The Stellar query to execute</li> -<li><tt>startTime</tt> : The start time in milliseconds</li> -<li><tt>endTime</tt> : The end time in milliseconds</li> -<li><tt>numReducers</tt> : Specify the number of reducers to use when executing the mapreduce job</li> -</ul> -<p>Example: <tt>curl -XGET "http://node1:8081/pcapGetter/getPcapsByQuery?query=ip_src_addr+==+'192.168.66.121'+and+ip_src_port+==+'60500'&startTime=1476936000000"</tt></p> -<p>All of these parameters are optional. In the case of a missing parameter, it is treated as a wildcard.</p> -<p>Unlike the CLI tool, there is no paging mechanism. The REST API will stream back data as a single file.</p></div> - </div> - </div> - </div> - <hr/> - <footer> - <div class="container-fluid"> - <div class="row-fluid"> -é 2015-2016 The Apache Software Foundation. Apache Metron, Metron, Apache, the Apache feather logo, - and the Apache Metron project logo are trademarks of The Apache Software Foundation. - </div> - </div> - </footer> - </body> -</html> http://git-wip-us.apache.org/repos/asf/metron/blob/a97e575f/site/current-book/metron-platform/metron-common/index.html ---------------------------------------------------------------------- diff --git a/site/current-book/metron-platform/metron-common/index.html b/site/current-book/metron-platform/metron-common/index.html index 8d0dc0e..caa5a3b 100644 --- a/site/current-book/metron-platform/metron-common/index.html +++ b/site/current-book/metron-platform/metron-common/index.html @@ -1,13 +1,13 @@ <!DOCTYPE html> <!-- - | Generated by Apache Maven Doxia Site Renderer 1.8 from src/site/markdown/metron-platform/metron-common/index.md at 2018-06-07 + | Generated by Apache Maven Doxia Site Renderer 1.8 from src/site/markdown/metron-platform/metron-common/index.md at 2018-09-12 | Rendered using Apache Maven Fluido Skin 1.7 --> <html xmlns="http://www.w3.org/1999/xhtml"; xml:lang="en" lang="en"> <head> <meta charset="UTF-8" /> <meta name="viewport" content="width=device-width, initial-scale=1.0" /> - <meta name="Date-Revision-yyyymmdd" content="20180607" /> + <meta name="Date-Revision-yyyymmdd" content="20180912" /> <meta http-equiv="Content-Language" content="en" /> <title>Metron – Contents</title> <link rel="stylesheet" href="../../css/apache-maven-fluido-1.7.min.css" /> @@ -32,8 +32,8 @@ <li class=""><a href="http://metron.apache.org/"; class="externalLink" title="Metron">Metron</a><span class="divider">/</span></li> <li class=""><a href="../../index.html" title="Documentation">Documentation</a><span class="divider">/</span></li> <li class="active ">Contents</li> - <li id="publishDate" class="pull-right"><span class="divider">|</span> Last Published: 2018-06-07</li> - <li id="projectVersion" class="pull-right">Version: 0.5.0</li> + <li id="publishDate" class="pull-right"><span class="divider">|</span> Last Published: 2018-09-12</li> + <li id="projectVersion" class="pull-right">Version: 0.6.0</li> </ul> </div> <div class="row-fluid"> @@ -55,15 +55,16 @@ <li><a href="../../metron-platform/index.html" title="Platform"><span class="icon-chevron-down"></span>Platform</a> <ul class="nav nav-list"> <li><a href="../../metron-platform/Performance-tuning-guide.html" title="Performance-tuning-guide"><span class="none"></span>Performance-tuning-guide</a></li> - <li><a href="../../metron-platform/metron-api/index.html" title="Api"><span class="none"></span>Api</a></li> <li class="active"><a href="#"><span class="none"></span>Common</a></li> <li><a href="../../metron-platform/metron-data-management/index.html" title="Data-management"><span class="none"></span>Data-management</a></li> <li><a href="../../metron-platform/metron-elasticsearch/index.html" title="Elasticsearch"><span class="none"></span>Elasticsearch</a></li> <li><a href="../../metron-platform/metron-enrichment/index.html" title="Enrichment"><span class="icon-chevron-right"></span>Enrichment</a></li> <li><a href="../../metron-platform/metron-indexing/index.html" title="Indexing"><span class="none"></span>Indexing</a></li> + <li><a href="../../metron-platform/metron-job/index.html" title="Job"><span class="none"></span>Job</a></li> <li><a href="../../metron-platform/metron-management/index.html" title="Management"><span class="none"></span>Management</a></li> <li><a href="../../metron-platform/metron-parsers/index.html" title="Parsers"><span class="icon-chevron-right"></span>Parsers</a></li> <li><a href="../../metron-platform/metron-pcap-backend/index.html" title="Pcap-backend"><span class="none"></span>Pcap-backend</a></li> + <li><a href="../../metron-platform/metron-solr/index.html" title="Solr"><span class="none"></span>Solr</a></li> <li><a href="../../metron-platform/metron-writer/index.html" title="Writer"><span class="none"></span>Writer</a></li> </ul> </li> @@ -231,6 +232,16 @@ limitations under the License. <td> String </td> <td> <tt>profiler_period_units</tt> </td></tr> <tr class="a"> +<td> <a href="../../metron-analytics/metron-profiler/index.html#profiler.writer.batchSize"><tt>profiler.writer.batchSize</tt></a> </td> +<td> Profiler </td> +<td> Integer </td> +<td> N/A </td></tr> +<tr class="b"> +<td> <a href="../../metron-analytics/metron-profiler/index.html#profiler.writer.batchTimeout"><tt>profiler.writer.batchTimeout</tt></a> </td> +<td> Profiler </td> +<td> Integer </td> +<td> N/A </td></tr> +<tr class="a"> <td> <a href="../metron-indexing/index.html#update.hbase.table"><tt>update.hbase.table</tt></a> </td> <td> REST/Indexing </td> <td> String </td> @@ -246,10 +257,30 @@ limitations under the License. <td> String </td> <td> <tt>geo_hdfs_file</tt> </td></tr> <tr class="b"> +<td> <a href="../metron-enrichment/index.html#enrichment.writer.batchSize"><tt>enrichment.writer.batchSize</tt></a> </td> +<td> Enrichment </td> +<td> Integer </td> +<td> N/A </td></tr> +<tr class="a"> +<td> <a href="../metron-enrichment/index.html#enrichment.writer.batchTimeout"><tt>enrichment.writer.batchTimeout</tt></a> </td> +<td> Enrichment </td> +<td> Integer </td> +<td> N/A </td></tr> +<tr class="b"> +<td> <a href="../metron-enrichment/index.html#geo.hdfs.file"><tt>geo.hdfs.file</tt></a> </td> +<td> Enrichment </td> +<td> String </td> +<td> <tt>geo_hdfs_file</tt> </td></tr> +<tr class="a"> <td> <a href="../../metron-interface/metron-alerts/index.html#source.type.field"><tt>source.type.field</tt></a> </td> <td> UI </td> <td> String </td> -<td> N/A </td></tr> +<td> <tt>source_type_field</tt> </td></tr> +<tr class="b"> +<td> <a href="../../metron-interface/metron-alerts/index.html#threat.triage.score.field"><tt>threat.triage.score.field</tt></a> </td> +<td> UI </td> +<td> String </td> +<td> <tt>threat_triage_score_field</tt> </td></tr> </tbody> </table> <div class="section"> http://git-wip-us.apache.org/repos/asf/metron/blob/a97e575f/site/current-book/metron-platform/metron-data-management/index.html ---------------------------------------------------------------------- diff --git a/site/current-book/metron-platform/metron-data-management/index.html b/site/current-book/metron-platform/metron-data-management/index.html index dea600c..610c5c6 100644 --- a/site/current-book/metron-platform/metron-data-management/index.html +++ b/site/current-book/metron-platform/metron-data-management/index.html @@ -1,13 +1,13 @@ <!DOCTYPE html> <!-- - | Generated by Apache Maven Doxia Site Renderer 1.8 from src/site/markdown/metron-platform/metron-data-management/index.md at 2018-06-07 + | Generated by Apache Maven Doxia Site Renderer 1.8 from src/site/markdown/metron-platform/metron-data-management/index.md at 2018-09-12 | Rendered using Apache Maven Fluido Skin 1.7 --> <html xmlns="http://www.w3.org/1999/xhtml"; xml:lang="en" lang="en"> <head> <meta charset="UTF-8" /> <meta name="viewport" content="width=device-width, initial-scale=1.0" /> - <meta name="Date-Revision-yyyymmdd" content="20180607" /> + <meta name="Date-Revision-yyyymmdd" content="20180912" /> <meta http-equiv="Content-Language" content="en" /> <title>Metron – Resource Data Management</title> <link rel="stylesheet" href="../../css/apache-maven-fluido-1.7.min.css" /> @@ -32,8 +32,8 @@ <li class=""><a href="http://metron.apache.org/"; class="externalLink" title="Metron">Metron</a><span class="divider">/</span></li> <li class=""><a href="../../index.html" title="Documentation">Documentation</a><span class="divider">/</span></li> <li class="active ">Resource Data Management</li> - <li id="publishDate" class="pull-right"><span class="divider">|</span> Last Published: 2018-06-07</li> - <li id="projectVersion" class="pull-right">Version: 0.5.0</li> + <li id="publishDate" class="pull-right"><span class="divider">|</span> Last Published: 2018-09-12</li> + <li id="projectVersion" class="pull-right">Version: 0.6.0</li> </ul> </div> <div class="row-fluid"> @@ -55,15 +55,16 @@ <li><a href="../../metron-platform/index.html" title="Platform"><span class="icon-chevron-down"></span>Platform</a> <ul class="nav nav-list"> <li><a href="../../metron-platform/Performance-tuning-guide.html" title="Performance-tuning-guide"><span class="none"></span>Performance-tuning-guide</a></li> - <li><a href="../../metron-platform/metron-api/index.html" title="Api"><span class="none"></span>Api</a></li> <li><a href="../../metron-platform/metron-common/index.html" title="Common"><span class="none"></span>Common</a></li> <li class="active"><a href="#"><span class="none"></span>Data-management</a></li> <li><a href="../../metron-platform/metron-elasticsearch/index.html" title="Elasticsearch"><span class="none"></span>Elasticsearch</a></li> <li><a href="../../metron-platform/metron-enrichment/index.html" title="Enrichment"><span class="icon-chevron-right"></span>Enrichment</a></li> <li><a href="../../metron-platform/metron-indexing/index.html" title="Indexing"><span class="none"></span>Indexing</a></li> + <li><a href="../../metron-platform/metron-job/index.html" title="Job"><span class="none"></span>Job</a></li> <li><a href="../../metron-platform/metron-management/index.html" title="Management"><span class="none"></span>Management</a></li> <li><a href="../../metron-platform/metron-parsers/index.html" title="Parsers"><span class="icon-chevron-right"></span>Parsers</a></li> <li><a href="../../metron-platform/metron-pcap-backend/index.html" title="Pcap-backend"><span class="none"></span>Pcap-backend</a></li> + <li><a href="../../metron-platform/metron-solr/index.html" title="Solr"><span class="none"></span>Solr</a></li> <li><a href="../../metron-platform/metron-writer/index.html" title="Writer"><span class="none"></span>Writer</a></li> </ul> </li> http://git-wip-us.apache.org/repos/asf/metron/blob/a97e575f/site/current-book/metron-platform/metron-elasticsearch/index.html ---------------------------------------------------------------------- diff --git a/site/current-book/metron-platform/metron-elasticsearch/index.html b/site/current-book/metron-platform/metron-elasticsearch/index.html index cc360b1..bf3c630 100644 --- a/site/current-book/metron-platform/metron-elasticsearch/index.html +++ b/site/current-book/metron-platform/metron-elasticsearch/index.html @@ -1,13 +1,13 @@ <!DOCTYPE html> <!-- - | Generated by Apache Maven Doxia Site Renderer 1.8 from src/site/markdown/metron-platform/metron-elasticsearch/index.md at 2018-06-07 + | Generated by Apache Maven Doxia Site Renderer 1.8 from src/site/markdown/metron-platform/metron-elasticsearch/index.md at 2018-09-12 | Rendered using Apache Maven Fluido Skin 1.7 --> <html xmlns="http://www.w3.org/1999/xhtml"; xml:lang="en" lang="en"> <head> <meta charset="UTF-8" /> <meta name="viewport" content="width=device-width, initial-scale=1.0" /> - <meta name="Date-Revision-yyyymmdd" content="20180607" /> + <meta name="Date-Revision-yyyymmdd" content="20180912" /> <meta http-equiv="Content-Language" content="en" /> <title>Metron – Elasticsearch in Metron</title> <link rel="stylesheet" href="../../css/apache-maven-fluido-1.7.min.css" /> @@ -32,8 +32,8 @@ <li class=""><a href="http://metron.apache.org/"; class="externalLink" title="Metron">Metron</a><span class="divider">/</span></li> <li class=""><a href="../../index.html" title="Documentation">Documentation</a><span class="divider">/</span></li> <li class="active ">Elasticsearch in Metron</li> - <li id="publishDate" class="pull-right"><span class="divider">|</span> Last Published: 2018-06-07</li> - <li id="projectVersion" class="pull-right">Version: 0.5.0</li> + <li id="publishDate" class="pull-right"><span class="divider">|</span> Last Published: 2018-09-12</li> + <li id="projectVersion" class="pull-right">Version: 0.6.0</li> </ul> </div> <div class="row-fluid"> @@ -55,15 +55,16 @@ <li><a href="../../metron-platform/index.html" title="Platform"><span class="icon-chevron-down"></span>Platform</a> <ul class="nav nav-list"> <li><a href="../../metron-platform/Performance-tuning-guide.html" title="Performance-tuning-guide"><span class="none"></span>Performance-tuning-guide</a></li> - <li><a href="../../metron-platform/metron-api/index.html" title="Api"><span class="none"></span>Api</a></li> <li><a href="../../metron-platform/metron-common/index.html" title="Common"><span class="none"></span>Common</a></li> <li><a href="../../metron-platform/metron-data-management/index.html" title="Data-management"><span class="none"></span>Data-management</a></li> <li class="active"><a href="#"><span class="none"></span>Elasticsearch</a></li> <li><a href="../../metron-platform/metron-enrichment/index.html" title="Enrichment"><span class="icon-chevron-right"></span>Enrichment</a></li> <li><a href="../../metron-platform/metron-indexing/index.html" title="Indexing"><span class="none"></span>Indexing</a></li> + <li><a href="../../metron-platform/metron-job/index.html" title="Job"><span class="none"></span>Job</a></li> <li><a href="../../metron-platform/metron-management/index.html" title="Management"><span class="none"></span>Management</a></li> <li><a href="../../metron-platform/metron-parsers/index.html" title="Parsers"><span class="icon-chevron-right"></span>Parsers</a></li> <li><a href="../../metron-platform/metron-pcap-backend/index.html" title="Pcap-backend"><span class="none"></span>Pcap-backend</a></li> + <li><a href="../../metron-platform/metron-solr/index.html" title="Solr"><span class="none"></span>Solr</a></li> <li><a href="../../metron-platform/metron-writer/index.html" title="Writer"><span class="none"></span>Writer</a></li> </ul> </li> @@ -405,13 +406,13 @@ limitations under the License. </ul></div></div></div></div> <div class="section"> <h2><a name="Using_Metron_with_Elasticsearch_5.6.2"></a>Using Metron with Elasticsearch 5.6.2</h2> -<p>There is a requirement that all sensors templates have a nested alert field defined. This field is a dummy field. See <a class="externalLink" href="https://www.elastic.co/guide/en/elasticsearch/reference/current/search-request-sort.html#_ignoring_unmapped_fields";>Ignoring Unmapped Fields</a> for more information</p> +<p>There is a requirement that all sensors templates have a nested <tt>metron_alert</tt> field defined. This field is a dummy field. See <a class="externalLink" href="https://www.elastic.co/guide/en/elasticsearch/reference/current/search-request-sort.html#_ignoring_unmapped_fields";>Ignoring Unmapped Fields</a> for more information</p> <p>Without this field, an error will be thrown during ALL searches (including from UIs, resulting in no alerts being found for any sensor). This error will be found in the REST service’s logs.</p> <p>Exception seen:</p> <div> <div> -<pre class="source">QueryParsingException[[nested] failed to find nested object under path [alert]]; +<pre class="source">QueryParsingException[[nested] failed to find nested object under path [metron_alert]]; </pre></div></div> <p>There are two steps to resolve this issue. First is to update the Elasticsearch template for each sensor, so any new indices have the field. This requires retrieving the template, removing an extraneous JSON field so we can put it back later, and adding our new field.</p> @@ -424,7 +425,7 @@ export SENSOR="bro" curl -XGET "http://${ELASTICSEARCH}:9200/_template/${SENSOR}_index*?pretty=true" -o "${SENSOR}.template" sed -i '' '2d;$d' ./${SENSOR}.template sed -i '' '/"properties" : {/ a\ -"alert": { "type": "nested"},' ${SENSOR}.template +"metron_alert": { "type": "nested"},' ${SENSOR}.template </pre></div></div> <p>To manually verify this, you can optionally pretty print it again with:</p> @@ -448,7 +449,7 @@ sed -i '' '/"properties" : {/ a\ <pre class="source">curl -XPUT "http://${ELASTICSEARCH}:9200/${SENSOR}_index*/_mapping/${SENSOR}_doc" -d ' { "properties" : { - "alert" : { + "metron_alert" : { "type" : "nested" } }
