http://git-wip-us.apache.org/repos/asf/madlib-site/blob/e283664c/docs/v1.14/group__grp__pmml.html ---------------------------------------------------------------------- diff --git a/docs/v1.14/group__grp__pmml.html b/docs/v1.14/group__grp__pmml.html new file mode 100644 index 0000000..83261c7 --- /dev/null +++ b/docs/v1.14/group__grp__pmml.html @@ -0,0 +1,332 @@ +<!-- HTML header for doxygen 1.8.4--> +<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd";> +<html xmlns="http://www.w3.org/1999/xhtml";> +<head> +<meta http-equiv="Content-Type" content="text/xhtml;charset=UTF-8"/> +<meta http-equiv="X-UA-Compatible" content="IE=9"/> +<meta name="generator" content="Doxygen 1.8.13"/> +<meta name="keywords" content="madlib,postgres,greenplum,machine learning,data mining,deep learning,ensemble methods,data science,market basket analysis,affinity analysis,pca,lda,regression,elastic net,huber white,proportional hazards,k-means,latent dirichlet allocation,bayes,support vector machines,svm"/> +<title>MADlib: PMML Export</title> +<link href="tabs.css" rel="stylesheet" type="text/css"/> +<script type="text/javascript" src="jquery.js"></script> +<script type="text/javascript" src="dynsections.js"></script> +<link href="navtree.css" rel="stylesheet" type="text/css"/> +<script type="text/javascript" src="resize.js"></script> +<script type="text/javascript" src="navtreedata.js"></script> +<script type="text/javascript" src="navtree.js"></script> +<script type="text/javascript"> + $(document).ready(initResizable); +</script> +<link href="search/search.css" rel="stylesheet" type="text/css"/> +<script type="text/javascript" src="search/searchdata.js"></script> +<script type="text/javascript" src="search/search.js"></script> +<script type="text/javascript"> + $(document).ready(function() { init_search(); }); +</script> +<script type="text/x-mathjax-config"> + MathJax.Hub.Config({ + extensions: ["tex2jax.js", "TeX/AMSmath.js", "TeX/AMSsymbols.js"], + jax: ["input/TeX","output/HTML-CSS"], +}); +</script><script type="text/javascript" src="http://cdn.mathjax.org/mathjax/latest/MathJax.js";></script> +<!-- hack in the navigation tree --> +<script type="text/javascript" src="eigen_navtree_hacks.js"></script> +<link href="doxygen.css" rel="stylesheet" type="text/css" /> +<link href="madlib_extra.css" rel="stylesheet" type="text/css"/> +<!-- google analytics --> +<script> + (function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]||function(){ + (i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o), + m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m) + })(window,document,'script','//www.google-analytics.com/analytics.js','ga'); + ga('create', 'UA-45382226-1', 'madlib.apache.org'); + ga('send', 'pageview'); +</script> +</head> +<body> +<div id="top"><!-- do not remove this div, it is closed by doxygen! --> +<div id="titlearea"> +<table cellspacing="0" cellpadding="0"> + <tbody> + <tr style="height: 56px;"> + <td id="projectlogo"><a href="http://madlib.apache.org";><img alt="Logo" src="madlib.png" height="50" style="padding-left:0.5em;" border="0"/ ></a></td> + <td style="padding-left: 0.5em;"> + <div id="projectname"> + <span id="projectnumber">1.14</span> + </div> + <div id="projectbrief">User Documentation for Apache MADlib</div> + </td> + <td> <div id="MSearchBox" class="MSearchBoxInactive"> + <span class="left"> + <img id="MSearchSelect" src="search/mag_sel.png" + onmouseover="return searchBox.OnSearchSelectShow()" + onmouseout="return searchBox.OnSearchSelectHide()" + alt=""/> + <input type="text" id="MSearchField" value="Search" accesskey="S" + onfocus="searchBox.OnSearchFieldFocus(true)" + onblur="searchBox.OnSearchFieldFocus(false)" + onkeyup="searchBox.OnSearchFieldChange(event)"/> + </span><span class="right"> + <a id="MSearchClose" href="javascript:searchBox.CloseResultsWindow()"><img id="MSearchCloseImg" border="0" src="search/close.png" alt=""/></a> + </span> + </div> +</td> + </tr> + </tbody> +</table> +</div> +<!-- end header part --> +<!-- Generated by Doxygen 1.8.13 --> +<script type="text/javascript"> +var searchBox = new SearchBox("searchBox", "search",false,'Search'); +</script> +</div><!-- top --> +<div id="side-nav" class="ui-resizable side-nav-resizable"> + <div id="nav-tree"> + <div id="nav-tree-contents"> + <div id="nav-sync" class="sync"></div> + </div> + </div> + <div id="splitbar" style="-moz-user-select:none;" + class="ui-resizable-handle"> + </div> +</div> +<script type="text/javascript"> +$(document).ready(function(){initNavTree('group__grp__pmml.html','');}); +</script> +<div id="doc-content"> +<!-- window showing the filter options --> +<div id="MSearchSelectWindow" + onmouseover="return searchBox.OnSearchSelectShow()" + onmouseout="return searchBox.OnSearchSelectHide()" + onkeydown="return searchBox.OnSearchSelectKey(event)"> +</div> + +<!-- iframe showing the search results (closed by default) --> +<div id="MSearchResultsWindow"> +<iframe src="javascript:void(0)" frameborder="0" + name="MSearchResults" id="MSearchResults"> +</iframe> +</div> + +<div class="header"> + <div class="headertitle"> +<div class="title">PMML Export<div class="ingroups"><a class="el" href="group__grp__utility__functions.html">Utility Functions</a></div></div> </div> +</div><!--header--> +<div class="contents"> +<div class="toc"><b>Contents</b><ul> +<li class="level1"> +<a href="#function">PMML Export Function</a> </li> +<li class="level1"> +<a href="#examples">Examples</a> </li> +<li class="level1"> +<a href="#background">Background</a> </li> +<li class="level1"> +<a href="#related">Related Topics</a> </li> +</ul> +</div><p><a class="anchor" id="function"></a></p><dl class="section user"><dt>PMML Export Function</dt><dd>The PMML export function in MADlib has the following syntax: <pre class="syntax"> +pmml ( model_table, + name_spec + ) +</pre> <b>Arguments</b> <dl class="arglist"> +<dt>model_table </dt> +<dd><p class="startdd">VARCHAR. The name of the table containing the model.</p> +<p class="enddd"></p> +</dd> +<dt>name_spec (optional) </dt> +<dd>VARCHAR or VARCHAR[]. Names to be used in the Data Dictionary of the PMML. See <a class="el" href="table__to__pmml_8sql__in.html#a9635b6989d9f972497b6b4164b77aa0a" title="Given the model constructed from a data mining algorithm, this function converts the model into PMML ...">pmml()</a> for detailed explanation. </dd> +</dl> +</dd></dl> +<p><b>Output</b> XML. The output of this function is a standard PMML document, some examples of which are covered in the next section. </p> +<dl class="section note"><dt>Note</dt><dd>In PostgreSQL, users may be required to install their database with XML support in order to use this function.</dd></dl> +<p>Usually the user wants to export the resulting PMML contents into a PMML file so that external softwares can use it. The following method can be used (Note: the user needs to use unaligned table output mode for psql with '-A' flag. And inside psql client, both '\t' and '\o' should be used):</p> +<pre class="example"> +> # under bash +> psql -A my_database +# -- in psql now +# \t +# \o test.pmml -- export to a file +# select madlib.pmml('tree_out'); +# \o +# \t +</pre><p><a class="anchor" id="examples"></a></p><dl class="section user"><dt>Examples</dt><dd><ol type="1"> +<li>Create the training data table. <pre class="example"> +CREATE TABLE patients( id integer NOT NULL, + second_attack integer, + treatment integer, + trait_anxiety integer); +INSERT INTO patients(id, second_attack, treatment, trait_anxiety) VALUES +( 1, 1, 1, 70), +( 3, 1, 1, 50), +( 5, 1, 0, 40), +( 7, 1, 0, 75), +( 9, 1, 0, 70), +(11, 0, 1, 65), +(13, 0, 1, 45), +(15, 0, 1, 40), +(17, 0, 0, 55), +(19, 0, 0, 50), +( 2, 1, 1, 80), +( 4, 1, 0, 60), +( 6, 1, 0, 65), +( 8, 1, 0, 80), +(10, 1, 0, 60), +(12, 0, 1, 50), +(14, 0, 1, 35), +(16, 0, 1, 50), +(18, 0, 0, 45), +(20, 0, 0, 60); +</pre></li> +<li>Train a regression model using <a class="el" href="logistic_8sql__in.html#a74210a7ef513dfcbdfdd9f3b37bfe428" title="Compute logistic-regression coefficients and diagnostic statistics. ">logregr_train()</a>. <pre class="example"> +SELECT madlib.logregr_train( + 'patients', + 'patients_logregr', + 'second_attack', + 'ARRAY[1, treatment, trait_anxiety]'); +</pre></li> +<li>View the PMML export for this model. <pre class="example"> +SELECT madlib.pmml('patients_logregr'); +</pre> Result: <pre class="result"> +<?xml version="1.0" standalone="yes"?> +<PMML version="4.1" xmlns="http://www.dmg.org/pmml-v4-1.html"> + <Header copyright="redacted for this example"> + <Extension extender="MADlib" name="user" value="gpadmin"> + <Application name="MADlib" version="1.7"> + <Timestamp> + 2014-06-13 17:30:14.527899 PDT + </Timestamp> + </Header> + <DataDictionary numberoffields="4"> + <DataField datatype="boolean" name="second_attack_pmml_prediction" optype="categorical"> + <DataField datatype="double" name="1" optype="continuous"> + <DataField datatype="double" name="treatment" optype="continuous"> + <DataField datatype="double" name="trait_anxiety" optype="continuous"> + </DataDictionary> + <RegressionModel functionname="classification" normalizationmethod="softmax"> + <MiningSchema> + <MiningField name="second_attack_pmml_prediction" usagetype="predicted"> + <MiningField name="1"> + <MiningField name="treatment"> + <MiningField name="trait_anxiety"> + </MiningSchema> + <RegressionTable intercept="0.0" targetcategory="True"> + <NumericPredictor coefficient="-6.36346994178" name="1"> + <NumericPredictor coefficient="-1.02410605239" name="treatment"> + <NumericPredictor coefficient="0.119044916669" name="trait_anxiety"> + </RegressionTable> + <RegressionTable intercept="0.0" targetcategory="False"> + </RegressionModel> +</PMML> +</pre></li> +</ol> +</dd></dl> +<p>Alternatively, the above can also be invoked as below if custom names are needed for fields in the Data Dictionary: </p><pre class="example"> +SELECT madlib.pmml('patients_logregr', + 'out_attack~1+in_trait_anxiety+in_treatment'); +</pre><p><b>Note:</b> If the second argument of 'pmml' function is not specified, a default suffix "_pmml_prediction" will be automatically append to the column name to be predicted. This can help avoid name conflicts.</p> +<p>The following example demonstrates grouping columns in the model table for the same dataset as the previous example.</p> +<ol type="1"> +<li>Train a different regression model with 'treatment' as the grouping column. <pre class="example"> +SELECT madlib.logregr_train( + 'patients', + 'patients_logregr_grouping', + 'second_attack', + 'ARRAY[1, trait_anxiety]', + 'treatment'); +</pre></li> +<li>View the PMML export for this model. <pre class="example"> +SELECT madlib.pmml('patients_logregr_grouping', + ARRAY['second_attack','1','in_trait_anxiety']); +</pre> Result: <pre class="result"> +<?xml version="1.0" standalone="yes"?> + <PMML version="4.1" xmlns="http://www.dmg.org/pmml-v4-1.html"> + <Header copyright="redacted for this example"> + <Extension extender="MADlib" name="user" value="gpadmin"> + <Application name="MADlib" version="1.7"> + <Timestamp> + 2014-06-13 17:37:55.786307 PDT + </Timestamp> + </Header> + <DataDictionary numberoffields="4"> + <DataField datatype="boolean" name="second_attack" optype="categorical"> + <DataField datatype="double" name="1" optype="continuous"> + <DataField datatype="double" name="in_trait_anxiety" optype="continuous"> + <DataField datatype="string" name="treatment" optype="categorical"> + </DataDictionary> + <MiningModel functionname="classification"> + <MiningSchema> + <MiningField name="second_attack" usagetype="predicted"> + <MiningField name="1"> + <MiningField name="in_trait_anxiety"> + <MiningField name="treatment"> + </MiningSchema> + <Segmentation multiplemodelmethod="selectFirst"> + <Segment> + <SimplePredicate field="treatment" operator="equal" value="1"> + <RegressionModel functionname="classification" normalizationmethod="softmax"> + <MiningSchema> + <MiningField name="second_attack" usagetype="predicted"> + <MiningField name="1"> + <MiningField name="in_trait_anxiety"> + </MiningSchema> + <RegressionTable intercept="0.0" targetcategory="True"> + <NumericPredictor coefficient="-8.02068430057" name="1"> + <NumericPredictor coefficient="0.130090428526" name="in_trait_anxiety"> + </RegressionTable> + <RegressionTable intercept="0.0" targetcategory="False"> + </RegressionModel> + </Segment> + <Segment> + <SimplePredicate field="treatment" operator="equal" value="0"> + <RegressionModel functionname="classification" normalizationmethod="softmax"> + <MiningSchema> + <MiningField name="second_attack" usagetype="predicted"> + <MiningField name="1"> + <MiningField name="in_trait_anxiety"> + </MiningSchema> + <RegressionTable intercept="0.0" targetcategory="True"> + <NumericPredictor coefficient="-5.75043192191" name="1"> + <NumericPredictor coefficient="0.108282446319" name="in_trait_anxiety"> + </RegressionTable> + <RegressionTable intercept="0.0" targetcategory="False"> + </RegressionModel> + </Segment> + </Segmentation> + </MiningModel> + </PMML> +</pre></li> +</ol> +<p><b>Note:</b> MADlib currently supports PMML export for Linear Regression, Logistic Regression, Generalized Linear Regression Model, Multinomial Logistic Regression, Ordinal Linear Regression, Decision Tree and Random Forests.</p> +<p>In Ordinal Regression, the signs of feature coefficients will be different in PMML export and in the default output model table from ordinal(). This is due to the difference of model settings.</p> +<p><a class="anchor" id="background"></a></p><dl class="section user"><dt>Background</dt><dd>The Predictive Model Markup Language (PMML) is an XML-based file format that provides a way for applications to describe and exchange models produced by data mining and machine learning algorithms. A PMML file comprises the following components:<ul> +<li>Header: Contains general information of the model, such as copyright information and model description.</li> +<li>Data Dictionary: Contains definitions of fields used in the model.</li> +<li>Data Transformations: Contains transformations for mapping user data into a form that can be used by the mining model.</li> +<li>Model: Contains definitions of the data mining model, which includes attributes such as the model name, function name, and algorithm name.</li> +<li>Mining Schema: Contains specific information for the fields used in the model, which includes the name and usage type.</li> +<li>Targets: Allows for post-processing of the predicted value.</li> +<li>Output: Allows for naming of output fields expected from the model.</li> +</ul> +</dd></dl> +<p>MADlib follows the PMML v4.1 standard. For more details about PMML, see <a href="http://www.dmg.org/v4-1/GeneralStructure.html";>http://www.dmg.org/v4-1/GeneralStructure.html</a>.</p> +<p><a class="anchor" id="related"></a></p><dl class="section user"><dt>Related Topics</dt><dd></dd></dl> +<p>File <a class="el" href="table__to__pmml_8sql__in.html">table_to_pmml.sql_in</a> documenting the PMML export functions.</p> +<p><a class="el" href="group__grp__linreg.html">Linear Regression</a></p> +<p><a class="el" href="group__grp__logreg.html">Logistic Regression</a></p> +<p><a class="el" href="group__grp__glm.html">Generalized Linear Models</a></p> +<p><a class="el" href="group__grp__ordinal.html">Ordinal Regression</a></p> +<p><a class="el" href="group__grp__multinom.html">Multinomial Regression</a></p> +<p><a class="el" href="group__grp__decision__tree.html">Decision Tree</a></p> +<p><a class="el" href="group__grp__random__forest.html">Random Forest</a> </p> +</div><!-- contents --> +</div><!-- doc-content --> +<!-- start footer part --> +<div id="nav-path" class="navpath"><!-- id is needed for treeview function! --> + <ul> + <li class="footer">Generated on Wed May 2 2018 13:00:12 for MADlib by + <a href="http://www.doxygen.org/index.html";> + <img class="footer" src="doxygen.png" alt="doxygen"/></a> 1.8.13 </li> + </ul> +</div> +</body> +</html>
http://git-wip-us.apache.org/repos/asf/madlib-site/blob/e283664c/docs/v1.14/group__grp__pred.html ---------------------------------------------------------------------- diff --git a/docs/v1.14/group__grp__pred.html b/docs/v1.14/group__grp__pred.html new file mode 100644 index 0000000..142b3c9 --- /dev/null +++ b/docs/v1.14/group__grp__pred.html @@ -0,0 +1,365 @@ +<!-- HTML header for doxygen 1.8.4--> +<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd";> +<html xmlns="http://www.w3.org/1999/xhtml";> +<head> +<meta http-equiv="Content-Type" content="text/xhtml;charset=UTF-8"/> +<meta http-equiv="X-UA-Compatible" content="IE=9"/> +<meta name="generator" content="Doxygen 1.8.13"/> +<meta name="keywords" content="madlib,postgres,greenplum,machine learning,data mining,deep learning,ensemble methods,data science,market basket analysis,affinity analysis,pca,lda,regression,elastic net,huber white,proportional hazards,k-means,latent dirichlet allocation,bayes,support vector machines,svm"/> +<title>MADlib: Prediction Metrics</title> +<link href="tabs.css" rel="stylesheet" type="text/css"/> +<script type="text/javascript" src="jquery.js"></script> +<script type="text/javascript" src="dynsections.js"></script> +<link href="navtree.css" rel="stylesheet" type="text/css"/> +<script type="text/javascript" src="resize.js"></script> +<script type="text/javascript" src="navtreedata.js"></script> +<script type="text/javascript" src="navtree.js"></script> +<script type="text/javascript"> + $(document).ready(initResizable); +</script> +<link href="search/search.css" rel="stylesheet" type="text/css"/> +<script type="text/javascript" src="search/searchdata.js"></script> +<script type="text/javascript" src="search/search.js"></script> +<script type="text/javascript"> + $(document).ready(function() { init_search(); }); +</script> +<script type="text/x-mathjax-config"> + MathJax.Hub.Config({ + extensions: ["tex2jax.js", "TeX/AMSmath.js", "TeX/AMSsymbols.js"], + jax: ["input/TeX","output/HTML-CSS"], +}); +</script><script type="text/javascript" src="http://cdn.mathjax.org/mathjax/latest/MathJax.js";></script> +<!-- hack in the navigation tree --> +<script type="text/javascript" src="eigen_navtree_hacks.js"></script> +<link href="doxygen.css" rel="stylesheet" type="text/css" /> +<link href="madlib_extra.css" rel="stylesheet" type="text/css"/> +<!-- google analytics --> +<script> + (function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]||function(){ + (i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o), + m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m) + })(window,document,'script','//www.google-analytics.com/analytics.js','ga'); + ga('create', 'UA-45382226-1', 'madlib.apache.org'); + ga('send', 'pageview'); +</script> +</head> +<body> +<div id="top"><!-- do not remove this div, it is closed by doxygen! --> +<div id="titlearea"> +<table cellspacing="0" cellpadding="0"> + <tbody> + <tr style="height: 56px;"> + <td id="projectlogo"><a href="http://madlib.apache.org";><img alt="Logo" src="madlib.png" height="50" style="padding-left:0.5em;" border="0"/ ></a></td> + <td style="padding-left: 0.5em;"> + <div id="projectname"> + <span id="projectnumber">1.14</span> + </div> + <div id="projectbrief">User Documentation for Apache MADlib</div> + </td> + <td> <div id="MSearchBox" class="MSearchBoxInactive"> + <span class="left"> + <img id="MSearchSelect" src="search/mag_sel.png" + onmouseover="return searchBox.OnSearchSelectShow()" + onmouseout="return searchBox.OnSearchSelectHide()" + alt=""/> + <input type="text" id="MSearchField" value="Search" accesskey="S" + onfocus="searchBox.OnSearchFieldFocus(true)" + onblur="searchBox.OnSearchFieldFocus(false)" + onkeyup="searchBox.OnSearchFieldChange(event)"/> + </span><span class="right"> + <a id="MSearchClose" href="javascript:searchBox.CloseResultsWindow()"><img id="MSearchCloseImg" border="0" src="search/close.png" alt=""/></a> + </span> + </div> +</td> + </tr> + </tbody> +</table> +</div> +<!-- end header part --> +<!-- Generated by Doxygen 1.8.13 --> +<script type="text/javascript"> +var searchBox = new SearchBox("searchBox", "search",false,'Search'); +</script> +</div><!-- top --> +<div id="side-nav" class="ui-resizable side-nav-resizable"> + <div id="nav-tree"> + <div id="nav-tree-contents"> + <div id="nav-sync" class="sync"></div> + </div> + </div> + <div id="splitbar" style="-moz-user-select:none;" + class="ui-resizable-handle"> + </div> +</div> +<script type="text/javascript"> +$(document).ready(function(){initNavTree('group__grp__pred.html','');}); +</script> +<div id="doc-content"> +<!-- window showing the filter options --> +<div id="MSearchSelectWindow" + onmouseover="return searchBox.OnSearchSelectShow()" + onmouseout="return searchBox.OnSearchSelectHide()" + onkeydown="return searchBox.OnSearchSelectKey(event)"> +</div> + +<!-- iframe showing the search results (closed by default) --> +<div id="MSearchResultsWindow"> +<iframe src="javascript:void(0)" frameborder="0" + name="MSearchResults" id="MSearchResults"> +</iframe> +</div> + +<div class="header"> + <div class="headertitle"> +<div class="title">Prediction Metrics<div class="ingroups"><a class="el" href="group__grp__mdl.html">Model Selection</a></div></div> </div> +</div><!--header--> +<div class="contents"> +<div class="toc"><b>Contents</b> <ul> +<li> +<a href="#list">List of Prediction Metric Functions</a> </li> +<li> +<a href="#specs">Function Specific Details</a> </li> +<li> +<a href="#examples">Examples</a> </li> +<li> +<a href="#literature">Literature</a> </li> +<li> +<a href="#related">Related Topics</a> </li> +</ul> +</div><p>This module provides a set of metrics to evaluate the quality of predictions of a model. A typical function will take a set of "prediction" and "observation" values and use them to calculate the desired metric, unless noted otherwise. Grouping is supported for all functions (except confusion matrix).</p> +<p><a class="anchor" id="list"></a></p><dl class="section user"><dt>Prediction Metrics Functions</dt><dd><table class="output"> +<tr> +<th>mean_abs_error(table_in, table_out, prediction_col, observed_col, grouping_cols)</th><td>Mean absolute error </td></tr> +<tr> +<th>mean_abs_perc_error(table_in, table_out, prediction_col, observed_col, grouping_cols)</th><td>Mean absolute percentage error </td></tr> +<tr> +<th>mean_perc_error(table_in, table_out, prediction_col, observed_col, grouping_cols)</th><td>Mean percentage error </td></tr> +<tr> +<th>mean_squared_error(table_in, table_out, prediction_col, observed_col, grouping_cols)</th><td>Mean squared error </td></tr> +<tr> +<th>r2_score(table_in, table_out, prediction_col, observed_col, grouping_cols)</th><td>R-squared </td></tr> +<tr> +<th>adjusted_r2_score(table_in, table_out, prediction_col, observed_col, num_predictors, training_size, grouping_cols)</th><td>Adjusted R-squared </td></tr> +<tr> +<th>binary_classifier(table_in, table_out, prediction_col, observed_col, grouping_cols)</th><td>Collection of prediction metrics related to binary classification </td></tr> +<tr> +<th>area_under_roc(table_in, table_out, prediction_col, observed_col, grouping_cols)</th><td>Area under the ROC curve (in binary classification) </td></tr> +<tr> +<th>confusion_matrix(table_in, table_out, prediction_col, observed_col, grouping_cols)</th><td>Confusion matrix for a multi-class classifier </td></tr> +</table> +</dd></dl> +<p><b>Arguments</b> </p><dl class="arglist"> +<dt>table_in </dt> +<dd>TEXT. Name of the input table. </dd> +<dt>table_out </dt> +<dd>TEXT. Name of the output table. For consistency, a table is created for all metric outputs even when grouping is not used, which may mean there is only a single value in the output table in some cases. </dd> +<dt>prediction_col </dt> +<dd>TEXT. Name of the column of predicted values from input table. </dd> +<dt>observed_col </dt> +<dd>TEXT. Name of the column of observed values from input table. </dd> +<dt>num_predictors (for adjusted R-squared score only) </dt> +<dd>INTEGER. The number of parameters in the predicting model, not counting the constant term. </dd> +<dt>training_size (for adjusted R-squared score only) </dt> +<dd>INTEGER. The number of rows used for training, excluding any NULL rows. </dd> +<dt>grouping_cols (optional) </dt> +<dd>TEXT, default: NULL. Name of the column of grouping values from input table. </dd> +</dl> +<p><a class="anchor" id="specs"></a></p><dl class="section user"><dt>Function Specific Details</dt><dd></dd></dl> +<p><b>R-squared Score</b></p> +<p>This function returns the coefficient of determination (R2) between the predicted and observed values. An R2 of 1 indicates that the regression line perfectly fits the data, while an R2 of 0 indicates that the line does not fit the data at all. Negative values of R2 may occur when fitting non-linear functions to data. Please refer to reference <a href="#r2">[1]</a> for more details.</p> +<p><b>Adjusted R-squared Score</b></p> +<p>This function returns the adjusted R2 score in addition to the R-squared score described above. Adjusted R2 score is used to counter the problem of the R2 automatically increasing when extra explanatory variables are added to the model. It takes two additional parameters describing the degrees of freedom of the model (num_predictors) and the size of the training set over which it was developed (training_size):</p> +<ul> +<li>num_predictors: Indicates the number of parameters the model has other than the constant term. For example, if it is set to '3' the model may take the following form as an example: 7 + 5x + 39y + 0.91z.</li> +<li>training_size: Indicates the number of rows in the training set (excluding any NULL rows).</li> +</ul> +<p>Neither of these arguments can be deduced from the predicted values and the test data alone which is why they are explicit inputs. Please refer to reference <a href="#r2">[1]</a> for more details.</p> +<p><a class="anchor" id="bc"></a><b>Binary Classification</b></p> +<p>This function returns an output table with a number of metrics commonly used in binary classification.</p> +<p>The definitions of the various metrics are as follows:</p> +<ul> +<li>\(\textit{tp}\) is the count of correctly-classified positives.</li> +<li>\(\textit{tn}\) is the count of correctly-classified negatives.</li> +<li>\(\textit{fp}\) is the count of misclassified negatives.</li> +<li>\(\textit{fn}\) is the count of misclassified positives.</li> +<li>\(\textit{tpr}=\textit{tp}/(\textit{tp}+\textit{fn})\).</li> +<li>\(\textit{tnr}=\textit{tn}/(\textit{fp}+\textit{tn})\).</li> +<li>\(\textit{ppv}=\textit{tp}/(\textit{tp}+\textit{fp})\).</li> +<li>\(\textit{npv}=\textit{tn}/(\textit{tn}+\textit{fn})\).</li> +<li>\(\textit{fpr}=\textit{fp}/(\textit{fp}+\textit{tn})\).</li> +<li>\(\textit{fdr}=1-\textit{ppv}\).</li> +<li>\(\textit{fnr}=\textit{fn}/(\textit{fn}+\textit{tp})\).</li> +<li>\(\textit{acc}=(\textit{tp}+\textit{tn})/(\textit{tp}+\textit{tn}+\textit{fp} +\textit{fn})\).</li> +<li>\(\textit{f1}=2*\textit{tp}/(2*\textit{tp}+\textit{fp}+\textit{fn})\).</li> +</ul> +<p><b>Area Under ROC Curve</b></p> +<p>This function returns the area under the Receiver Operating Characteristic curve for binary classification (the AUC). The ROC curve is the curve relating the classifier's TPR and FPR metrics. (See <a href="#bc">Binary Classification</a> above for a definition of these metrics). Please refer to reference <a href="#aoc">[2]</a> for more details. Note that the binary classification function can be used to obtain the data (TPR and FPR values) required for drawing the ROC curve.</p> +<dl class="section note"><dt>Note</dt><dd>For 'binary_classifier' and 'area_under_roc' functions:<ul> +<li>The 'observed_col' column is assumed to be a numeric column with two values: 0 and 1, or a Boolean column. For the purposes of the metric calculation, 0 is considered to be negative and 1 to be positive.</li> +<li>The 'pred_col' column is expected to contain numeric values corresponding to likelihood/probability. A larger value corresponds to greater certainty that the observed value will be '1', and a lower value corresponds to a greater certainty that it will be '0'.</li> +</ul> +</dd></dl> +<p><b>Confusion Matrix</b></p> +<p>This function returns the confusion matrix of a multi-class classification. Each column of the matrix represents the instances in a predicted class while each row represents the instances in an actual class. This allows more detailed analysis than mere proportion of correct guesses (accuracy). Please refer to the reference <a href="#cm">[3]</a> for more details. Please note that grouping is not supported for the confusion matrix.</p> +<p><a class="anchor" id="examples"></a></p><dl class="section user"><dt>Examples</dt><dd></dd></dl> +<ol type="1"> +<li>Create the sample data: <pre class="example"> +DROP TABLE IF EXISTS test_set; +CREATE TABLE test_set( + pred FLOAT8, + obs FLOAT8 + ); +INSERT INTO test_set VALUES + (37.5,53.1), (12.3,34.2), (74.2,65.4), (91.1,82.1); +</pre></li> +<li>Run the Mean Absolute Error function: <pre class="example"> +DROP TABLE IF EXISTS table_out; +SELECT madlib.mean_abs_error( 'test_set', 'table_out', 'pred', 'obs'); +SELECT * FROM table_out; +</pre> Result <pre class="result"> + mean_abs_error + ---------------- + 13.825 +</pre></li> +<li>Run the Mean Absolute Percentage Error function: <pre class="example"> +DROP TABLE IF EXISTS table_out; +SELECT madlib.mean_abs_perc_error( 'test_set', 'table_out', 'pred', 'obs'); +SELECT * FROM table_out; +</pre> Result <pre class="result"> + mean_abs_perc_error + --------------------- + 0.294578793636013 +</pre></li> +<li>Run the Mean Percentage Error function: <pre class="example"> +DROP TABLE IF EXISTS table_out; +SELECT madlib.mean_perc_error( 'test_set', 'table_out', 'pred', 'obs'); +SELECT * FROM table_out; +</pre> Result <pre class="result"> + mean_perc_error + ------------------- + -0.17248930032771 +</pre></li> +<li>Run the Mean Squared Error function: <pre class="example"> +DROP TABLE IF EXISTS table_out; +SELECT madlib.mean_squared_error( 'test_set', 'table_out', 'pred', 'obs'); +SELECT * FROM table_out; +</pre> Result <pre class="result"> + mean_squared_error + -------------------- + 220.3525 +</pre></li> +<li>Run the R2 Score function: <pre class="example"> +DROP TABLE IF EXISTS table_out; +SELECT madlib.r2_score( 'test_set', 'table_out', 'pred', 'obs'); +SELECT * FROM table_out; +</pre> Result <pre class="result"> + r2_score + ------------------------ + 0.27992908844337695865 +</pre></li> +<li>Run the Adjusted R2 Score function: <pre class="example"> +DROP TABLE IF EXISTS table_out; +SELECT madlib.adjusted_r2_score( 'test_set', 'table_out', 'pred', 'obs', 3, 100); +SELECT * FROM table_out; +</pre> Result <pre class="result"> + r2_score | adjusted_r2_score + --------------------+------------------ + 0.279929088443375 | 0.257426872457231 +</pre></li> +<li>Create the sample data for binary classifier metrics: <pre class="example"> +DROP TABLE IF EXISTS test_set; +CREATE TABLE test_set AS + SELECT ((a*8)::integer)/8.0 pred, + ((a*0.5+random()*0.5)>0.5) obs + FROM (select random() as a from generate_series(1,100)) x; +</pre></li> +<li>Run the Binary Classifier metrics function: <pre class="example"> +DROP TABLE IF EXISTS table_out; +SELECT madlib.binary_classifier( 'test_set', 'table_out', 'pred', 'obs'); +</pre></li> +<li>View the True Positive Rate and the False Positive Rate: <pre class="example"> +SELECT threshold, tpr, fpr FROM table_out ORDER BY threshold; +</pre> Result (your results for this and other functions below will look different due to the presence of the random function in sample data generator): <pre class="result"> + threshold | tpr | fpr +------------------------+------------------------+------------------------ + 0.00000000000000000000 | 1.00000000000000000000 | 1.00000000000000000000 + 0.12500000000000000000 | 1.00000000000000000000 | 0.94915254237288135593 + 0.25000000000000000000 | 0.92682926829268292683 | 0.64406779661016949153 + 0.37500000000000000000 | 0.80487804878048780488 | 0.47457627118644067797 + 0.50000000000000000000 | 0.70731707317073170732 | 0.35593220338983050847 + 0.62500000000000000000 | 0.63414634146341463415 | 0.25423728813559322034 + 0.75000000000000000000 | 0.48780487804878048780 | 0.06779661016949152542 + 0.87500000000000000000 | 0.29268292682926829268 | 0.03389830508474576271 + 1.00000000000000000000 | 0.12195121951219512195 | 0.00000000000000000000 +</pre></li> +<li>View all metrics at a given threshold value: <pre class="example"> +-- Set extended display on for easier reading of output +\x on +SELECT * FROM table_out WHERE threshold=0.5; +</pre> Result <pre class="result"> +-[ RECORD 1 ]--------------------- +threshold | 0.50000000000000000000 +tp | 29 +fp | 21 +fn | 12 +tn | 38 +tpr | 0.70731707317073170732 +tnr | 0.64406779661016949153 +ppv | 0.58000000000000000000 +npv | 0.76000000000000000000 +fpr | 0.35593220338983050847 +fdr | 0.42000000000000000000 +fnr | 0.29268292682926829268 +acc | 0.67000000000000000000 +f1 | 0.63736263736263736264 +</pre></li> +<li>Run the Area Under ROC curve function: <pre class="example"> +DROP TABLE IF EXISTS table_out; +SELECT madlib.area_under_roc( 'test_set', 'table_out', 'pred', 'obs'); +SELECT * FROM table_out; +</pre> Result <pre class="result"> + area_under_roc + --------------------------------------------- +0.77428689541132699462698842496899545266640 +</pre></li> +<li>Create the sample data for confusion matrix. <pre class="example"> +DROP TABLE IF EXISTS test_set; +CREATE TABLE test_set AS + SELECT (x+y)%5+1 AS pred, + (x*y)%5 AS obs + FROM generate_series(1,5) x, + generate_series(1,5) y; +</pre></li> +<li>Run the confusion matrix function: <pre class="example"> +DROP TABLE IF EXISTS table_out; +SELECT madlib.confusion_matrix( 'test_set', 'table_out', 'pred', 'obs'); +SELECT * FROM table_out ORDER BY class; +</pre> Result <pre class="result"> + class | confusion_arr +-------+--------------- + 0 | {0,1,2,2,2,2} + 1 | {0,2,0,1,1,0} + 2 | {0,0,0,2,2,0} + 3 | {0,0,2,0,0,2} + 4 | {0,2,1,0,0,1} + 5 | {0,0,0,0,0,0} +</pre></li> +</ol> +<p><a class="anchor" id="literature"></a></p><dl class="section user"><dt>Literature</dt><dd></dd></dl> +<p><a class="anchor" id="r2"></a> [1] <a href="https://en.wikipedia.org/wiki/Coefficient_of_determination";>https://en.wikipedia.org/wiki/Coefficient_of_determination</a></p> +<p><a class="anchor" id="aoc"></a> [2] <a href="https://en.wikipedia.org/wiki/Receiver_operating_characteristic";>https://en.wikipedia.org/wiki/Receiver_operating_characteristic</a></p> +<p><a class="anchor" id="cm"></a> [3] <a href="https://en.wikipedia.org/wiki/Confusion_matrix";>https://en.wikipedia.org/wiki/Confusion_matrix</a></p> +<p><a class="anchor" id="related"></a></p><dl class="section user"><dt>Related Topics</dt><dd></dd></dl> +<p>File <a class="el" href="pred__metrics_8sql__in.html" title="A collection of summary statistics to gauge model accuracy based on predicted values vs...">pred_metrics.sql_in</a> for list of functions and usage. </p> +</div><!-- contents --> +</div><!-- doc-content --> +<!-- start footer part --> +<div id="nav-path" class="navpath"><!-- id is needed for treeview function! --> + <ul> + <li class="footer">Generated on Wed May 2 2018 13:00:11 for MADlib by + <a href="http://www.doxygen.org/index.html";> + <img class="footer" src="doxygen.png" alt="doxygen"/></a> 1.8.13 </li> + </ul> +</div> +</body> +</html> http://git-wip-us.apache.org/repos/asf/madlib-site/blob/e283664c/docs/v1.14/group__grp__prob.html ---------------------------------------------------------------------- diff --git a/docs/v1.14/group__grp__prob.html b/docs/v1.14/group__grp__prob.html new file mode 100644 index 0000000..32cfd92 --- /dev/null +++ b/docs/v1.14/group__grp__prob.html @@ -0,0 +1,164 @@ +<!-- HTML header for doxygen 1.8.4--> +<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd";> +<html xmlns="http://www.w3.org/1999/xhtml";> +<head> +<meta http-equiv="Content-Type" content="text/xhtml;charset=UTF-8"/> +<meta http-equiv="X-UA-Compatible" content="IE=9"/> +<meta name="generator" content="Doxygen 1.8.13"/> +<meta name="keywords" content="madlib,postgres,greenplum,machine learning,data mining,deep learning,ensemble methods,data science,market basket analysis,affinity analysis,pca,lda,regression,elastic net,huber white,proportional hazards,k-means,latent dirichlet allocation,bayes,support vector machines,svm"/> +<title>MADlib: Probability Functions</title> +<link href="tabs.css" rel="stylesheet" type="text/css"/> +<script type="text/javascript" src="jquery.js"></script> +<script type="text/javascript" src="dynsections.js"></script> +<link href="navtree.css" rel="stylesheet" type="text/css"/> +<script type="text/javascript" src="resize.js"></script> +<script type="text/javascript" src="navtreedata.js"></script> +<script type="text/javascript" src="navtree.js"></script> +<script type="text/javascript"> + $(document).ready(initResizable); +</script> +<link href="search/search.css" rel="stylesheet" type="text/css"/> +<script type="text/javascript" src="search/searchdata.js"></script> +<script type="text/javascript" src="search/search.js"></script> +<script type="text/javascript"> + $(document).ready(function() { init_search(); }); +</script> +<script type="text/x-mathjax-config"> + MathJax.Hub.Config({ + extensions: ["tex2jax.js", "TeX/AMSmath.js", "TeX/AMSsymbols.js"], + jax: ["input/TeX","output/HTML-CSS"], +}); +</script><script type="text/javascript" src="http://cdn.mathjax.org/mathjax/latest/MathJax.js";></script> +<!-- hack in the navigation tree --> +<script type="text/javascript" src="eigen_navtree_hacks.js"></script> +<link href="doxygen.css" rel="stylesheet" type="text/css" /> +<link href="madlib_extra.css" rel="stylesheet" type="text/css"/> +<!-- google analytics --> +<script> + (function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]||function(){ + (i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o), + m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m) + })(window,document,'script','//www.google-analytics.com/analytics.js','ga'); + ga('create', 'UA-45382226-1', 'madlib.apache.org'); + ga('send', 'pageview'); +</script> +</head> +<body> +<div id="top"><!-- do not remove this div, it is closed by doxygen! --> +<div id="titlearea"> +<table cellspacing="0" cellpadding="0"> + <tbody> + <tr style="height: 56px;"> + <td id="projectlogo"><a href="http://madlib.apache.org";><img alt="Logo" src="madlib.png" height="50" style="padding-left:0.5em;" border="0"/ ></a></td> + <td style="padding-left: 0.5em;"> + <div id="projectname"> + <span id="projectnumber">1.14</span> + </div> + <div id="projectbrief">User Documentation for Apache MADlib</div> + </td> + <td> <div id="MSearchBox" class="MSearchBoxInactive"> + <span class="left"> + <img id="MSearchSelect" src="search/mag_sel.png" + onmouseover="return searchBox.OnSearchSelectShow()" + onmouseout="return searchBox.OnSearchSelectHide()" + alt=""/> + <input type="text" id="MSearchField" value="Search" accesskey="S" + onfocus="searchBox.OnSearchFieldFocus(true)" + onblur="searchBox.OnSearchFieldFocus(false)" + onkeyup="searchBox.OnSearchFieldChange(event)"/> + </span><span class="right"> + <a id="MSearchClose" href="javascript:searchBox.CloseResultsWindow()"><img id="MSearchCloseImg" border="0" src="search/close.png" alt=""/></a> + </span> + </div> +</td> + </tr> + </tbody> +</table> +</div> +<!-- end header part --> +<!-- Generated by Doxygen 1.8.13 --> +<script type="text/javascript"> +var searchBox = new SearchBox("searchBox", "search",false,'Search'); +</script> +</div><!-- top --> +<div id="side-nav" class="ui-resizable side-nav-resizable"> + <div id="nav-tree"> + <div id="nav-tree-contents"> + <div id="nav-sync" class="sync"></div> + </div> + </div> + <div id="splitbar" style="-moz-user-select:none;" + class="ui-resizable-handle"> + </div> +</div> +<script type="text/javascript"> +$(document).ready(function(){initNavTree('group__grp__prob.html','');}); +</script> +<div id="doc-content"> +<!-- window showing the filter options --> +<div id="MSearchSelectWindow" + onmouseover="return searchBox.OnSearchSelectShow()" + onmouseout="return searchBox.OnSearchSelectHide()" + onkeydown="return searchBox.OnSearchSelectKey(event)"> +</div> + +<!-- iframe showing the search results (closed by default) --> +<div id="MSearchResultsWindow"> +<iframe src="javascript:void(0)" frameborder="0" + name="MSearchResults" id="MSearchResults"> +</iframe> +</div> + +<div class="header"> + <div class="headertitle"> +<div class="title">Probability Functions<div class="ingroups"><a class="el" href="group__grp__stats.html">Statistics</a></div></div> </div> +</div><!--header--> +<div class="contents"> +<div class="toc"><b>Contents</b> <ul> +<li> +<a href="#syntax">Function Syntax</a> </li> +<li> +<a href="#examples">Examples</a> </li> +<li> +<a href="#literature">Literature</a> </li> +<li> +<a href="#related">Related Topics</a> </li> +</ul> +</div><p>The Probability Functions module provides cumulative distribution, density/mass, and quantile functions for a wide range of probability distributions.</p> +<p>Unless otherwise documented, all of these functions are wrappers around functionality provided by the boost C++ library [1, â<a href="http://www.boost.org/doc/libs/1_49_0/libs/math/doc/sf_and_dist/html/math_toolkit/dist.html";>Statistical Distributions and Functions</a>â].</p> +<p>For convenience, all cumulative distribution and density/mass functions (CDFs and PDF/PMFs in short) are defined over the range of all floating-point numbers including infinity. Inputs that are <code>NULL</code> or <code>NaN</code> (not a number) will always produce a <code>NULL</code> or <code>NaN</code> result, respectively. Inputs that are plus or minus infinity will return the respective limits.</p> +<p>A quantile function for a probability distrution with CDF \( F \) takes a probability argument \( p \in [0,1] \) and returns the value \( x \) so that \( F(x) = p \), provided such an \( x \) exists and it is unique. If it does not, the result will be \( \sup \{ x \in D \mid F(x) \leq p \} \) (interpreted as 0 if the supremum is over an empty set) if \( p < 0.5 \), and \( \inf \{ x \in D \mid F(x) \geq p \} \) if \( p \geq 0.5 \). Here \( D \) denotes the domain of the distribution, which is the set of reals \( \mathbb R \) for continuous and the set of nonnegative integers \( \mathbb N_0 \) for discrete distributions.</p> +<p>Intuitively, the formulas in the previous paragraph deal with the following special cases. The 0-quantile will always be the âleft endâ of the support, and the 1-quantile will be the âright endâ of the support of the distribution. For discrete distributions, most values of \( p \in [0,1] \) do not admit an \( x \) with \( F(x) = p \). Instead, there is an \( x \in \mathbb N_0 \) so that \( F(x) < p < F(x + 1) \). The above formulas mean that the value returned as \( p \)-quantile is \( x \) if \( p < 0.5 \), and it is \( x + 1 \) if \( p \geq 0.5 \). (As a special case, in order to ensure that quantiles are always within the support, the \( p \)-quantile will be 0 if \( p < F(0) \)).</p> +<p>The rationale for choosing this behavior is that \(p\)-quantiles for \( p < 0.5 \) are typically requested when interested in the value \( x \) such that with confidence level <b>at least</b> \( 1 - p \) a random variable will be \( > x \) (or equivalently, with probability <b>at most</b> \( p \), it will be \( \leq x \)). Likewise, \(p\)-quantiles for \( p \geq 0.5 \) are typically requested when interested in the value \( x \) such that with confidence level <b>at least</b> \( p \) a random variable will be \( \leq x \). See also [1, â<a href="http://www.boost.org/doc/libs/1_46_1/libs/math/doc/sf_and_dist/html/math_toolkit/policy/pol_tutorial/understand_dis_quant.html";>Understanding Quantiles of Discrete Distributions</a>â].</p> +<p><a class="anchor" id="syntax"></a></p><dl class="section user"><dt>Function Syntax</dt><dd></dd></dl> +<p>Cumulative distribution functions:</p> +<pre class="syntax"><em>distribution</em>_cdf(<em>random variate</em>[, <em>parameter1</em> [, <em>parameter2</em> [, <em>parameter3</em>] ] ])</pre><p>Probability density/mass functions: </p><pre class="syntax"><em>distribution</em>_{pdf|pmf}(<em>random variate</em>[, <em>parameter1</em> [, <em>parameter2</em> [, <em>parameter3</em>] ] ])</pre><p>Quantile functions: </p><pre class="syntax"><em>distribution</em>_quantile(<em>probability</em>[, <em>parameter1</em> [, <em>parameter2</em> [, <em>parameter3</em>] ] ])</pre><p>For concrete function signatures, see <a class="el" href="prob_8sql__in.html">prob.sql_in</a>.</p> +<p><a class="anchor" id="examples"></a></p><dl class="section user"><dt>Examples</dt><dd></dd></dl> +<pre class="example"> +SELECT madlib.normal_cdf(0); +</pre><p> Result: </p><pre class="result"> + normal_cdf + ----------- + 0.5 +</pre> <pre class="example"> +SELECT madlib.normal_quantile(0.5, 0, 1); +</pre><p> Result: </p><pre class="result"> + normal_quantile + ---------------- + 0 +(1 row) +</pre><p><a class="anchor" id="literature"></a></p><dl class="section user"><dt>Literature</dt><dd></dd></dl> +<p>[1] John Maddock, Paul A. Bristow, Hubert Holin, Xiaogang Zhang, Bruno Lalande, Johan RÃ¥de, Gautam Sewani and Thijs van den Berg: <em>Boost Math Toolkit</em>, Version 1.49, available at: <a href="http://www.boost.org/doc/libs/1_49_0/libs/math/doc/sf_and_dist/html/index.html";>http://www.boost.org/doc/libs/1_49_0/libs/math/doc/sf_and_dist/html/index.html</a></p> +<dl class="section user"><dt>Related Topics</dt><dd><a class="anchor" id="related"></a>File <a class="el" href="prob_8sql__in.html" title="SQL functions for evaluating probability functions. ">prob.sql_in</a> documenting the SQL functions. </dd></dl> +</div><!-- contents --> +</div><!-- doc-content --> +<!-- start footer part --> +<div id="nav-path" class="navpath"><!-- id is needed for treeview function! --> + <ul> + <li class="footer">Generated on Wed May 2 2018 13:00:11 for MADlib by + <a href="http://www.doxygen.org/index.html";> + <img class="footer" src="doxygen.png" alt="doxygen"/></a> 1.8.13 </li> + </ul> +</div> +</body> +</html>
