(madlib) 02/02: Analyze to pick optimal plan for orca
This is an automated email from the ASF dual-hosted git repository. okislal pushed a commit to branch madlib2-master in repository https://gitbox.apache.org/repos/asf/madlib.git commit b195300080853098b38609548ef99a10168ad382 Author: Ekta Khanna AuthorDate: Thu May 9 07:39:57 2024 -0700 Analyze to pick optimal plan for orca --- .../postgres/modules/deep_learning/test/madlib_keras_predict.sql_in | 1 + 1 file changed, 1 insertion(+) diff --git a/src/ports/postgres/modules/deep_learning/test/madlib_keras_predict.sql_in b/src/ports/postgres/modules/deep_learning/test/madlib_keras_predict.sql_in index 8f2b18e6..ca868dea 100644 --- a/src/ports/postgres/modules/deep_learning/test/madlib_keras_predict.sql_in +++ b/src/ports/postgres/modules/deep_learning/test/madlib_keras_predict.sql_in @@ -146,6 +146,7 @@ WHERE y = 0; INSERT INTO cifar_10_sample(id, x, y, imgpath) SELECT 5, x, 5, '0/img5.jpg' FROM cifar_10_sample WHERE y = 1; +ANALYZE cifar_10_sample; DROP TABLE IF EXISTS cifar_10_sample_int_batched; DROP TABLE IF EXISTS cifar_10_sample_int_batched_summary;
(madlib) 01/02: DL: Use legacy optimizers
This is an automated email from the ASF dual-hosted git repository. okislal pushed a commit to branch madlib2-master in repository https://gitbox.apache.org/repos/asf/madlib.git commit ea6b535f45d36981552e2dba2b2a0e9b3bf68c6c Author: Orhan Kislal AuthorDate: Thu May 2 20:16:08 2024 -0400 DL: Use legacy optimizers With the tf 2.12 version decay parameter got replaced. Now the user has to create a LearningRateSchedule object. While this should be possible to do inside the parameters, we use the legacy optimizers to keep the existing notations possible as well. --- src/ports/postgres/modules/deep_learning/madlib_keras.py_in | 2 +- src/ports/postgres/modules/deep_learning/madlib_keras_predict.py_in | 2 +- src/ports/postgres/modules/deep_learning/madlib_keras_wrapper.py_in | 4 ++-- 3 files changed, 4 insertions(+), 4 deletions(-) diff --git a/src/ports/postgres/modules/deep_learning/madlib_keras.py_in b/src/ports/postgres/modules/deep_learning/madlib_keras.py_in index 2bd98c4f..0f68165c 100644 --- a/src/ports/postgres/modules/deep_learning/madlib_keras.py_in +++ b/src/ports/postgres/modules/deep_learning/madlib_keras.py_in @@ -47,7 +47,7 @@ import utilities.debug as DEBUG from tensorflow.compat.v1.keras import backend as K from tensorflow.keras.layers import * from tensorflow.keras.models import * -from tensorflow.keras.optimizers import * +from tensorflow.keras.optimizers.legacy import * from tensorflow.keras.regularizers import * DEBUG.timings_enabled = False diff --git a/src/ports/postgres/modules/deep_learning/madlib_keras_predict.py_in b/src/ports/postgres/modules/deep_learning/madlib_keras_predict.py_in index 99bde223..4dcc174b 100644 --- a/src/ports/postgres/modules/deep_learning/madlib_keras_predict.py_in +++ b/src/ports/postgres/modules/deep_learning/madlib_keras_predict.py_in @@ -37,7 +37,7 @@ from deep_learning.madlib_keras_wrapper import * from tensorflow.keras.layers import * from tensorflow.keras.models import * -from tensorflow.keras.optimizers import * +from tensorflow.keras.optimizers.legacy import * import utilities.debug as DEBUG from utilities.debug import plpy_prepare diff --git a/src/ports/postgres/modules/deep_learning/madlib_keras_wrapper.py_in b/src/ports/postgres/modules/deep_learning/madlib_keras_wrapper.py_in index ecfe4c4d..3d343318 100644 --- a/src/ports/postgres/modules/deep_learning/madlib_keras_wrapper.py_in +++ b/src/ports/postgres/modules/deep_learning/madlib_keras_wrapper.py_in @@ -36,10 +36,10 @@ from utilities.utilities import is_superuser import tensorflow as tf from tensorflow.compat.v1.keras import backend as K -from tensorflow.keras.optimizers import * +from tensorflow.keras.optimizers.legacy import * from tensorflow.keras.callbacks import TensorBoard -import tensorflow.keras.optimizers as opt +import tensorflow.keras.optimizers.legacy as opt import tensorflow.keras.losses as losses import tensorflow.keras.metrics as metrics
(madlib) branch madlib2-master updated (9e32dd98 -> b1953000)
This is an automated email from the ASF dual-hosted git repository. okislal pushed a change to branch madlib2-master in repository https://gitbox.apache.org/repos/asf/madlib.git from 9e32dd98 PMML: Update user docs new ea6b535f DL: Use legacy optimizers new b1953000 Analyze to pick optimal plan for orca The 2 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: src/ports/postgres/modules/deep_learning/madlib_keras.py_in | 2 +- src/ports/postgres/modules/deep_learning/madlib_keras_predict.py_in | 2 +- src/ports/postgres/modules/deep_learning/madlib_keras_wrapper.py_in | 4 ++-- .../postgres/modules/deep_learning/test/madlib_keras_predict.sql_in | 1 + 4 files changed, 5 insertions(+), 4 deletions(-)
[madlib] annotated tag rel/v2.1.0 updated (3c568245 -> 585b1bb0)
This is an automated email from the ASF dual-hosted git repository. okislal pushed a change to annotated tag rel/v2.1.0 in repository https://gitbox.apache.org/repos/asf/madlib.git *** WARNING: tag rel/v2.1.0 was modified! *** from 3c568245 (commit) to 585b1bb0 (tag) tagging 3c568245cc55242bed549a66f47defdf1653dbca (commit) replaces rc/2.1.0-rc1 by Orhan Kislal on Fri Sep 8 22:13:05 2023 +0300 - Log - Apache MADlib 2.1.0 -BEGIN PGP SIGNATURE- iQEzBAABCAAdFiEEGocCIKIuO+T6G3jEyiWcTeuzJCIFAmT7ckEACgkQyiWcTeuz JCJhnwf/XPYDBkfl6DSKzkO1mIAUritSK4UUFVAu0XFwm6dTBJbU65UrilBuAF77 cnG4D+z+HWcyPu1JdupPc2VEwlvqjpU9jufHk+gooQZLQ/S37F0A834DYs8WXpSl qZ+uWqxqaDi4M3bNobZr8ObcvGqOej8RLcS7jWkwzjhhKxHUZm77c3Obu2Pc0uwd vdtkK6gRypCTf0eellBQK4rBfCVg6j9MQDUiM/A0tS2gxj/Jjn9bCrfGpMdoGet/ MY6CD/xVxeCu3f7E6n8g6RzwymoTpaEy+Edr7xhU1nv9ob7/+66QwHkq1r0fKiAA p91FwRQuEWvv3TXhnisvklBf6iToRQ== =WqUn -END PGP SIGNATURE- --- No new revisions were added by this update. Summary of changes:
[madlib-site] branch asf-site updated: Update old documentation dropdown
This is an automated email from the ASF dual-hosted git repository.
okislal pushed a commit to branch asf-site
in repository https://gitbox.apache.org/repos/asf/madlib-site.git
The following commit(s) were added to refs/heads/asf-site by this push:
new 930ccfe Update old documentation dropdown
930ccfe is described below
commit 930ccfe985bd09d65befeaf1ae5b9dd554fd0e85
Author: Orhan Kislal
AuthorDate: Fri Sep 8 22:10:42 2023 +0300
Update old documentation dropdown
---
documentation.html | 1 +
1 file changed, 1 insertion(+)
diff --git a/documentation.html b/documentation.html
index 19b82d3..d655faf 100644
--- a/documentation.html
+++ b/documentation.html
@@ -47,6 +47,7 @@ jQuery(document).ready(function() {
The primary documentation reference material providing
detailed information on the functions and algorithms within MADlib as well as
background theory and references into the literature.
Older Documentation
+MADlib v2.1.0
MADlib v1.21.0
MADlib v1.20.0
MADlib v1.19.0
[madlib-site] branch asf-site updated: Add v2.1.0 related docs and announcements
This is an automated email from the ASF dual-hosted git repository.
okislal pushed a commit to branch asf-site
in repository https://gitbox.apache.org/repos/asf/madlib-site.git
The following commit(s) were added to refs/heads/asf-site by this push:
new 97386f3 Add v2.1.0 related docs and announcements
97386f3 is described below
commit 97386f33d7d8ccf715decb024b6da4dbda6a7c5d
Author: Orhan Kislal
AuthorDate: Fri Sep 8 22:08:23 2023 +0300
Add v2.1.0 related docs and announcements
---
docs/latest| 2 +-
docs/{rc => v2.1.0}/apsp_8sql__in.html | 0
docs/{rc => v2.1.0}/arima_8sql__in.html| 0
docs/{rc => v2.1.0}/array__ops_8sql__in.html | 0
docs/{rc => v2.1.0}/assoc__rules_8sql__in.html | 0
docs/{rc => v2.1.0}/balance__sample_8sql__in.html | 0
docs/{rc => v2.1.0}/bayes_8sql__in.html| 0
docs/{rc => v2.1.0}/bc_s.png | Bin
docs/{rc => v2.1.0}/bdwn.png | Bin
docs/{rc => v2.1.0}/bfs_8sql__in.html | 0
docs/{rc => v2.1.0}/closed.png | Bin
.../clustered__variance_8sql__in.html | 0
.../clustered__variance__coxph_8sql__in.html | 0
docs/{rc => v2.1.0}/cols2vec_8sql__in.html | 0
.../conjugate__gradient_8sql__in.html | 0
docs/{rc => v2.1.0}/correlation_8sql__in.html | 0
.../cox__prop__hazards_8sql__in.html | 0
.../create__indicators_8sql__in.html | 0
docs/{rc => v2.1.0}/crf_8sql__in.html | 0
.../{rc => v2.1.0}/crf__data__loader_8sql__in.html | 0
.../{rc => v2.1.0}/crf__feature__gen_8sql__in.html | 0
.../{rc => v2.1.0}/cross__validation_8sql__in.html | 0
docs/{rc => v2.1.0}/dbscan_8sql__in.html | 0
docs/{rc => v2.1.0}/decision__tree_8sql__in.html | 0
.../dense__linear__systems_8sql__in.html | 0
.../dir_097d0c47873b413f914e0be0fb8e0c70.html | 0
.../dir_16b90e9a6c080d7fc2617cfecd7f223b.html | 0
.../dir_1960c8c5a7f1e7616d29e5a0c180bcf2.html | 0
.../dir_1c3de670490e30b2adf792227ef51e32.html | 0
.../dir_2e2767a73e2d710192c730fd426a8c90.html | 0
.../dir_3749b33f41e0a82f4dfb20be7f91dfa3.html | 0
.../dir_3ee26ceeadb429f807bd6cd822eca1b9.html | 0
.../dir_42e4eb27424bda9fbbfa95509de09bad.html | 0
.../dir_49426dfa1e413b9e4a461cc826e399ac.html | 0
.../dir_4f54709f5fc5d0f64da91428555e2469.html | 0
.../dir_505cd743a8a717435eca324f49291a46.html | 0
.../dir_5341dbb6d7f0a427749c6136276fa506.html | 0
.../dir_5cd13365c66443c54a8f3b4c82c03aee.html | 0
.../dir_5ff04c1d4b766f0a4dd68c6f19ed2092.html | 0
.../dir_60056874394c01e6c892c7492b3fbe27.html | 0
.../dir_68267d1309a1af8e8297ef4c3efbcdba.html | 0
.../dir_6ff79b0655deb26abf8f86290b84a97c.html | 0
.../dir_704eb8350b43e1ca74c0f90ed1ba450e.html | 0
.../dir_70e9abe17564e15776adcab8550c7667.html | 0
.../dir_73ccba3aa44ce35463f879b4ebbd3f46.html | 0
.../dir_7513a8b3b7336e99c30a333cfb144104.html | 0
.../dir_7592ceb856cf018f9811e11407c61fda.html | 0
.../dir_834e55cf992733df24fbe86b49356157.html | 0
.../dir_87d2e9694998410c1b0408bb775833a4.html | 0
.../dir_8b8525adb4e4d2ae68fa3d70bbeff8da.html | 0
.../dir_8cdb0d06bc3d9fdbe366fbcc82abc3e3.html | 0
.../dir_8d53e1c0026e7e34b4cd68f8b91426d6.html | 0
.../dir_93c42bb4df0f3e1302223b6dfd48c81e.html | 0
.../dir_9db0190ed89ab407304aec250d1ef64f.html | 0
.../dir_9e42ee0a0235722f482630aa6cc99334.html | 0
.../dir_9f5e6edf0db58b0627c46b41d27f.html | 0
.../dir_a4a48839224ef8488facbffa8a397967.html | 0
.../dir_ac52a4b89b7b1b1591f2952b5cbd041a.html | 0
.../dir_b0d8a62e5a4314de84ce58e7cd249850.html | 0
.../dir_b10c028a6a58d6e0840bc150baba59b9.html | 0
.../dir_b19a2d10787fafb5edd4ef1da0d2cacf.html | 0
.../dir_c2d3987992538edb8f634d5f4e0fae80.html | 0
.../dir_c8a9890f716ab6621b63d44c36a168eb.html | 0
.../dir_d06d449079888c21a85aa00b7664e813.html | 0
.../dir_d662e943e8adcb86abe4b822d2348c9e.html | 0
.../dir_d84ff4a1f4eeae069cebd336452cd47c.html | 0
.../dir_dc596537ad427a4d866006d1a3e1fe29.html | 0
.../dir_df86748cb94fb6c2fa09e991cce090c0.html | 0
.../dir_e2e6d80cc57daae810c80cbdae687e33.html | 0
.../dir_e502304d8bc7f1ef2456a474c6d92acd.html | 0
.../dir_e6c126a997181663ba81c11cbf416bb1.html | 0
.../dir_efbcf68973d247bbf15f9eecae7f24e3.html | 0
.../dir_fe647384fff178c2cff5ce75fb4044e5.html | 0
docs/{rc => v2.1.0}/distribution_8sql__in.html | 0
docs/{rc => v2.1.0}/doc.png| Bin
docs/{rc => v2.1.0}/dot_inline_dotgraph_1.svg | 0
docs/{rc =>
svn commit: r63872 - /dev/madlib/2.1.0.RC1/
Author: okislal Date: Fri Sep 8 18:27:13 2023 New Revision: 63872 Log: Removing 2.1.0 RC releases Removed: dev/madlib/2.1.0.RC1/
svn commit: r63871 - /dev/madlib/2.1.0.RC2/ /release/madlib/2.1.0/
Author: okislal Date: Fri Sep 8 18:25:12 2023 New Revision: 63871 Log: Move 2.1.0.RC2 artifacts to release Added: release/madlib/2.1.0/ - copied from r63870, dev/madlib/2.1.0.RC2/ Removed: dev/madlib/2.1.0.RC2/
[madlib] annotated tag rc/2.1.0-rc2 updated (3c568245 -> 1f529806)
This is an automated email from the ASF dual-hosted git repository. okislal pushed a change to annotated tag rc/2.1.0-rc2 in repository https://gitbox.apache.org/repos/asf/madlib.git *** WARNING: tag rc/2.1.0-rc2 was modified! *** from 3c568245 (commit) to 1f529806 (tag) tagging 3c568245cc55242bed549a66f47defdf1653dbca (commit) replaces rc/2.1.0-rc1 by Orhan Kislal on Tue Sep 5 20:10:50 2023 +0300 - Log - Apache MADlib 2.1.0 RC2 -BEGIN PGP SIGNATURE- iQEzBAABCAAdFiEEGocCIKIuO+T6G3jEyiWcTeuzJCIFAmT3YRoACgkQyiWcTeuz JCKDuggAwRp4XwxjkyKBRFXfyH11ni5UTI58fZVIPekUfvYn2vl6oNjEwz1wMBTC SEkMxuwyPTky0lJqm0boE5/Vp9XMTOZVZO83hls4dB/zyQbj+TzW1Uzs+lC4LFxX jtKX3KoKUdiQAm0Ml0ppzOCkQr4z62MAwSNgqo3dU+B6HBVU9sViVQ/gJHSa/RoQ 3GQYMmKQa9K+PwaZwJccGIvFtOxCAp/LkoUOFPw+z4OF0GEPfr7TVrX6Y9icKE2v /YbxkPeMVaOwfoFWDvppNhHfeiIPxddJ8puy833s1Nj7CpNPJB+gBsx+g0WAuKpe 7Cwl3BRt4S4VmEtlRWLSmAUcsB4CBQ== =CQV2 -END PGP SIGNATURE- --- No new revisions were added by this update. Summary of changes:
svn commit: r63800 - in /dev/madlib/2.1.0.RC2: apache-madlib-2.1.0-src.tar.gz apache-madlib-2.1.0-src.tar.gz.asc apache-madlib-2.1.0-src.tar.gz.sha512
Author: okislal Date: Tue Sep 5 17:06:34 2023 New Revision: 63800 Log: updating 2.1.0 RC2 candidate release artifacts Modified: dev/madlib/2.1.0.RC2/apache-madlib-2.1.0-src.tar.gz dev/madlib/2.1.0.RC2/apache-madlib-2.1.0-src.tar.gz.asc dev/madlib/2.1.0.RC2/apache-madlib-2.1.0-src.tar.gz.sha512 Modified: dev/madlib/2.1.0.RC2/apache-madlib-2.1.0-src.tar.gz == Binary files - no diff available. Modified: dev/madlib/2.1.0.RC2/apache-madlib-2.1.0-src.tar.gz.asc == --- dev/madlib/2.1.0.RC2/apache-madlib-2.1.0-src.tar.gz.asc (original) +++ dev/madlib/2.1.0.RC2/apache-madlib-2.1.0-src.tar.gz.asc Tue Sep 5 17:06:34 2023 @@ -1,11 +1,11 @@ -BEGIN PGP SIGNATURE- -iQEzBAABCAAdFiEEGocCIKIuO+T6G3jEyiWcTeuzJCIFAmT3XjYACgkQyiWcTeuz -JCLCcQf7BU1QaiGrO6Lf3fId080F3DfG3UxL2GjXk2sB6E26cFSeuNVc9gtbFMSJ -+gkaHwlSHpD1rbz9RcWy3yeDOgqVdUnqwPWD11pJpqJvKS2bcQQ2GoDgP18Wc+D9 -I9O+xqb5ZtZdT4MtAWDCloA5FhWsqqbmpb89K8Ny+4u0UFmsM0Jp6S97oWoaJL71 -jWcGpDwpvAgBFK473dBnnQK54p205CAWF66MwgKiOEbCYithaRRTQnVLsD5gb7P2 -AbBLnNDGw/s+ilnuN7EEdcpAoPTMjx8tFpxb4SlmDaPKOgqT+fq2mDDG+CRADhpp -l3nM8W/Qg4CK5sTonW4oBQFbgXiiJA== -=ZCF2 +iQEzBAABCAAdFiEEGocCIKIuO+T6G3jEyiWcTeuzJCIFAmT3YAAACgkQyiWcTeuz +JCKJugf/US1XpUraArh5XMQYVsFpNmoUSiS4MhxuC++1dNzT4TrLNODovPZ8/BLL +ctYvHtvFQP83nVTOLydpd90Pt/QyrNCzbVHO3Ccft0LpE4C/jBQYlX7EgAcF6cqr +50wRWzM4UhOzM0YPA8pnMu+gbv+gZs6ruBhRDDvMZmQEl8v6pb4xjPVbMo3VaWOX +sl4AItMOlTI0SBma755dUq+pxgHAFPVxL+gEEDCJvxzZd1Qa4Q+lrws/Ga2+qZ0I +S4pIfvbpYJc1Emh6pTxv13X35Y8109nQfbj5ezxL5azVaSHMcQoebnSJJVg4CnKJ +xTE16R13DmwJwtOcSIsQexLYA5uHRQ== +=4HCf -END PGP SIGNATURE- Modified: dev/madlib/2.1.0.RC2/apache-madlib-2.1.0-src.tar.gz.sha512 == --- dev/madlib/2.1.0.RC2/apache-madlib-2.1.0-src.tar.gz.sha512 (original) +++ dev/madlib/2.1.0.RC2/apache-madlib-2.1.0-src.tar.gz.sha512 Tue Sep 5 17:06:34 2023 @@ -1 +1 @@ -711b83fd5657d01e77bffd867dbee0447f19af09324a25039678443d02d3bab313a0be9f16d237085448f1e0a8b4d91247fc832637a20c1249b8cee21a7f742e apache-madlib-2.1.0-src.tar.gz +a538a0822fbe00ad422590340800cbde80a0bfa3794c58a5ecdd5079445d039d9694bc990b2704102e147bb8ed221af8aff3dde60cdaa1fa58c7e16d9e09f71d apache-madlib-2.1.0-src.tar.gz
[madlib] branch madlib2-master updated: Update release_notes with RC2 changes
This is an automated email from the ASF dual-hosted git repository. okislal pushed a commit to branch madlib2-master in repository https://gitbox.apache.org/repos/asf/madlib.git The following commit(s) were added to refs/heads/madlib2-master by this push: new 3c568245 Update release_notes with RC2 changes 3c568245 is described below commit 3c568245cc55242bed549a66f47defdf1653dbca Author: Orhan Kislal AuthorDate: Tue Sep 5 20:04:54 2023 +0300 Update release_notes with RC2 changes --- RELEASE_NOTES | 4 1 file changed, 4 insertions(+) diff --git a/RELEASE_NOTES b/RELEASE_NOTES index 8ec4be69..aaa91996 100644 --- a/RELEASE_NOTES +++ b/RELEASE_NOTES @@ -20,6 +20,10 @@ Bug fixes: - DL: Remove SERIAL from load_keras_model - Build: Add ubuntu flag for PyXB installation - Build: Add the actual path of $libdir to dynamic_library_path +- Build: Remove PyXB as a packaged dependency and replace it with +external pyxb-x dependency. +- Build: Use PG15 in Jenkins CI +- CRF: Fix anyarray -> anycompatiblearray change for PG14 —- MADlib v2.0.0:
svn commit: r63799 - in /dev/madlib/2.1.0.RC2: ./ apache-madlib-2.1.0-src.tar.gz apache-madlib-2.1.0-src.tar.gz.asc apache-madlib-2.1.0-src.tar.gz.sha512
Author: okislal Date: Tue Sep 5 17:00:01 2023 New Revision: 63799 Log: adding 2.1.0 RC2 candidate release artifacts Added: dev/madlib/2.1.0.RC2/ dev/madlib/2.1.0.RC2/apache-madlib-2.1.0-src.tar.gz (with props) dev/madlib/2.1.0.RC2/apache-madlib-2.1.0-src.tar.gz.asc dev/madlib/2.1.0.RC2/apache-madlib-2.1.0-src.tar.gz.sha512 Added: dev/madlib/2.1.0.RC2/apache-madlib-2.1.0-src.tar.gz == Binary file - no diff available. Propchange: dev/madlib/2.1.0.RC2/apache-madlib-2.1.0-src.tar.gz -- svn:mime-type = application/octet-stream Added: dev/madlib/2.1.0.RC2/apache-madlib-2.1.0-src.tar.gz.asc == --- dev/madlib/2.1.0.RC2/apache-madlib-2.1.0-src.tar.gz.asc (added) +++ dev/madlib/2.1.0.RC2/apache-madlib-2.1.0-src.tar.gz.asc Tue Sep 5 17:00:01 2023 @@ -0,0 +1,11 @@ +-BEGIN PGP SIGNATURE- + +iQEzBAABCAAdFiEEGocCIKIuO+T6G3jEyiWcTeuzJCIFAmT3XjYACgkQyiWcTeuz +JCLCcQf7BU1QaiGrO6Lf3fId080F3DfG3UxL2GjXk2sB6E26cFSeuNVc9gtbFMSJ ++gkaHwlSHpD1rbz9RcWy3yeDOgqVdUnqwPWD11pJpqJvKS2bcQQ2GoDgP18Wc+D9 +I9O+xqb5ZtZdT4MtAWDCloA5FhWsqqbmpb89K8Ny+4u0UFmsM0Jp6S97oWoaJL71 +jWcGpDwpvAgBFK473dBnnQK54p205CAWF66MwgKiOEbCYithaRRTQnVLsD5gb7P2 +AbBLnNDGw/s+ilnuN7EEdcpAoPTMjx8tFpxb4SlmDaPKOgqT+fq2mDDG+CRADhpp +l3nM8W/Qg4CK5sTonW4oBQFbgXiiJA== +=ZCF2 +-END PGP SIGNATURE- Added: dev/madlib/2.1.0.RC2/apache-madlib-2.1.0-src.tar.gz.sha512 == --- dev/madlib/2.1.0.RC2/apache-madlib-2.1.0-src.tar.gz.sha512 (added) +++ dev/madlib/2.1.0.RC2/apache-madlib-2.1.0-src.tar.gz.sha512 Tue Sep 5 17:00:01 2023 @@ -0,0 +1 @@ +711b83fd5657d01e77bffd867dbee0447f19af09324a25039678443d02d3bab313a0be9f16d237085448f1e0a8b4d91247fc832637a20c1249b8cee21a7f742e apache-madlib-2.1.0-src.tar.gz
[madlib] 05/05: Remove PyXB mentions from code, docs and licenses
This is an automated email from the ASF dual-hosted git repository. okislal pushed a commit to branch madlib2-master in repository https://gitbox.apache.org/repos/asf/madlib.git commit 70c548c102b1e5d9013294284affcde00a6412fb Author: Orhan Kislal AuthorDate: Tue Sep 5 10:19:59 2023 +0300 Remove PyXB mentions from code, docs and licenses --- README.md| 3 +- ReadMe_Build.txt | 6 - deploy/PGXN/ReadMe.txt | 1 - licenses/third_party/PyXB_v1.2.6.txt | 268 --- src/CMakeLists.txt | 60 src/patch/CMakeLists.txt | 2 - src/patch/PyXB.sh| 223 - 7 files changed, 1 insertion(+), 562 deletions(-) diff --git a/README.md b/README.md index a48a3b7c..1d988665 100644 --- a/README.md +++ b/README.md @@ -127,8 +127,7 @@ MADlib incorporates software from the following third-party components. Bundled Downloaded at build time (or supplied as build dependencies): 6. [`Boost 1.61.0 (or newer)`](http://www.boost.org/) "provides peer-reviewed portable C++ source libraries" -7. [`PyXB 1.2.6`](http://pyxb.sourceforge.net/) "Python library for XML Schema Bindings" -8. [`Eigen 3.2.2`](http://eigen.tuxfamily.org/index.php?title=Main_Page) "C++ template library for linear algebra" +7. [`Eigen 3.2.2`](http://eigen.tuxfamily.org/index.php?title=Main_Page) "C++ template library for linear algebra" Licensing == diff --git a/ReadMe_Build.txt b/ReadMe_Build.txt index 583bd400..592d3a41 100644 --- a/ReadMe_Build.txt +++ b/ReadMe_Build.txt @@ -167,12 +167,6 @@ root directory) for more options, after having run `cmake` the first time. with `-DEIGEN_TAR_SOURCE=/path/to/eigen_x.tar.gz`, in which case this tarball is used. -- `PYXB_TAR_SOURCE` (default: *empty*) - -PyXB is downloaded automatically, unless you call `./configure` -with `-DPYXB_TAR_SOURCE=/path/to/pyxb_x.tar.gz`, in which case -this tarball is used. - - `CREATE_RPM_FOR_UBUNTU` (default: *empty*) By default, we create a .deb madlib installer on Ubuntu. If this diff --git a/deploy/PGXN/ReadMe.txt b/deploy/PGXN/ReadMe.txt index 3b1db376..78462bf4 100644 --- a/deploy/PGXN/ReadMe.txt +++ b/deploy/PGXN/ReadMe.txt @@ -39,7 +39,6 @@ Downloaded at build time: http://www.boost.org/ - Eigen 3.2 "is a C++ template library for linear algebra" http://eigen.tuxfamily.org/index.php?title=Main_Page -- PyXB 1.2.4 "Python library for XML Schema Bindings" License information regarding MADlib and included third-party libraries can be found inside the 'licenses' directory. diff --git a/licenses/third_party/PyXB_v1.2.6.txt b/licenses/third_party/PyXB_v1.2.6.txt deleted file mode 100644 index 5c732c94.. --- a/licenses/third_party/PyXB_v1.2.6.txt +++ /dev/null @@ -1,268 +0,0 @@ -From the PyXB licensing page -(http://pyxb.sourceforge.net/legal.html) - - - Apache License - Version 2.0, January 2004 -http://www.apache.org/licenses/ - - TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION - - 1. Definitions. - - "License" shall mean the terms and conditions for use, - reproduction, - and distribution as defined by Sections 1 through 9 of this - document. - - "Licensor" shall mean the copyright owner or entity authorized - by - the copyright owner that is granting the License. - - "Legal Entity" shall mean the union of the acting entity and all - other entities that control, are controlled by, or are under - common - control with that entity. For the purposes of this definition, - "control" means (i) the power, direct or indirect, to cause the - direction or management of such entity, whether by contract or - otherwise, or (ii) ownership of fifty percent (50%) or more of - the - outstanding shares, or (iii) beneficial ownership of such - entity. - - "You" (or "Your") shall mean an individual or Legal Entity - exercising permissions granted by this License. - - "Source" form shall mean the preferred form for making - modifications, - including but not limited to software source code, documentation - source, and configuration files. - - "Object" form shall mean any form resulting from mechanical - transformation or translation of a Source form, including but - not limited to compiled object code, generated documentation, - and conversions to other media types. - - "Work" shall mean the work of authorship, whether in Source or - Object form, made available under the License, as indicated by a - copyright notice that is included in or attached to the work
[madlib] 04/05: Fix anyarray -> anycompatiblearray change for PG14
This is an automated email from the ASF dual-hosted git repository.
okislal pushed a commit to branch madlib2-master
in repository https://gitbox.apache.org/repos/asf/madlib.git
commit 395a8192386c3a641187441bfdd0e05f
Author: Orhan Kislal
AuthorDate: Sat Sep 2 09:02:58 2023 +0300
Fix anyarray -> anycompatiblearray change for PG14
---
src/madpack/madpack.py| 5 +++--
src/ports/postgres/modules/crf/crf.sql_in | 4 ++--
2 files changed, 5 insertions(+), 4 deletions(-)
diff --git a/src/madpack/madpack.py b/src/madpack/madpack.py
index 87df1c06..fc4c6537 100755
--- a/src/madpack/madpack.py
+++ b/src/madpack/madpack.py
@@ -188,7 +188,8 @@ def _run_m4_and_append(schema, maddir_mod_py, module,
sqlfile,
'-DMODULE_NAME=' + module,
'-I' + maddir_madpack,
sqlfile]
-if (((portid == 'postgres') & (dbver == '15'))):
+if ( (portid == 'postgres') &
+ (is_rev_gte(get_rev_num(dbver), get_rev_num('14.0'))) ):
m4args = ['m4',
'-P',
'-DMADLIB_SCHEMA=' + schema,
@@ -197,7 +198,7 @@ def _run_m4_and_append(schema, maddir_mod_py, module,
sqlfile,
'-DMODULE_PATHNAME=' + maddir_lib,
'-DMADLIB_LIBRARY_PATH=' + madlib_library_path,
'-DMODULE_NAME=' + module,
- '-DIS_PG_15=TRUE',
+ '-DUSE_COMPATIBLE_ARRAY=TRUE',
'-I' + maddir_madpack,
sqlfile]
diff --git a/src/ports/postgres/modules/crf/crf.sql_in
b/src/ports/postgres/modules/crf/crf.sql_in
index 302de8a4..9354fdac 100644
--- a/src/ports/postgres/modules/crf/crf.sql_in
+++ b/src/ports/postgres/modules/crf/crf.sql_in
@@ -719,13 +719,13 @@ CREATE AGGREGATE MADLIB_SCHEMA.lincrf_lbfgs_step(
INITCOND='{0,0,0,0,0,0,0,0,0,0, 0,0,0,0,0,0,0,0,0,0, 0,0,0,0,0,0,0,0,0,0,
0,0,0,0,0,0,0,0,0,0, 0,0,0,0,0,0,0,0,0,0, 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0}'
);
-m4_ifdef(`IS_PG_15',
+m4_ifdef(`USE_COMPATIBLE_ARRAY',
`DROP AGGREGATE IF EXISTS MADLIB_SCHEMA.array_union(anycompatiblearray)
CASCADE;',
`DROP AGGREGATE IF EXISTS MADLIB_SCHEMA.array_union(anyarray) CASCADE;')
CREATE m4_ifdef(`__POSTGRESQL__', `',
m4_ifdef(`__HAS_ORDERED_AGGREGATES__', `ORDERED')) AGGREGATE
-m4_ifdef(`IS_PG_15',
+m4_ifdef(`USE_COMPATIBLE_ARRAY',
`MADLIB_SCHEMA.array_union(anycompatiblearray) (SFUNC = array_cat, STYPE =
anycompatiblearray);',
`MADLIB_SCHEMA.array_union(anyarray) (SFUNC = array_cat, STYPE = anyarray);'
)
[madlib] 02/05: DL: Update get_state_to_return to always return bytea
This is an automated email from the ASF dual-hosted git repository. okislal pushed a commit to branch madlib2-master in repository https://gitbox.apache.org/repos/asf/madlib.git commit 40dfa0e3810b2f8d070941ba8053cc7b7886f169 Author: Orhan Kislal AuthorDate: Fri Sep 1 20:34:48 2023 +0300 DL: Update get_state_to_return to always return bytea get_state_to_return returned bytea in some cases and a single float in others. The float value is not actually used so it did not break the process. However, this still causes python instability and errors in some cases. This commit fixes the issue by converting the float value to bytea just like the rest of the options. --- src/ports/postgres/modules/deep_learning/madlib_keras.py_in | 2 +- .../modules/deep_learning/test/unit_tests/test_madlib_keras.py_in | 4 ++-- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/src/ports/postgres/modules/deep_learning/madlib_keras.py_in b/src/ports/postgres/modules/deep_learning/madlib_keras.py_in index 5a99f129..2bd98c4f 100644 --- a/src/ports/postgres/modules/deep_learning/madlib_keras.py_in +++ b/src/ports/postgres/modules/deep_learning/madlib_keras.py_in @@ -800,7 +800,7 @@ def get_state_to_return(segment_model, is_last_row, is_multiple_model, agg_image new_state = serialize_state_with_nd_weights( agg_image_count, updated_model_weights) else: -new_state = float(agg_image_count) +new_state = np.float32(agg_image_count).tostring() return new_state diff --git a/src/ports/postgres/modules/deep_learning/test/unit_tests/test_madlib_keras.py_in b/src/ports/postgres/modules/deep_learning/test/unit_tests/test_madlib_keras.py_in index ab8dc383..3da8549d 100644 --- a/src/ports/postgres/modules/deep_learning/test/unit_tests/test_madlib_keras.py_in +++ b/src/ports/postgres/modules/deep_learning/test/unit_tests/test_madlib_keras.py_in @@ -127,7 +127,7 @@ class MadlibKerasFitEvalTransitionTestCase(unittest.TestCase): image_count = kwargs['GD']['agg_image_count'] self.assertEqual(ending_image_count, image_count) -image_count = new_state +image_count = np.fromstring(new_state,dtype=np.float32)[0] self.assertEqual(ending_image_count, image_count) def _test_fit_transition_multiple_model_no_cache_first_buffer_pass(self, **kwargs): @@ -179,7 +179,7 @@ class MadlibKerasFitEvalTransitionTestCase(unittest.TestCase): self.total_images_per_seg, self.accessible_gpus_for_seg, self.dummy_prev_weights, **kwargs) -image_count = new_state +image_count = np.fromstring(new_state,dtype=np.float32)[0] self.assertEqual(ending_image_count, image_count) def _test_fit_transition_multiple_model_no_cache_middle_buffer_pass(self,
[madlib] 03/05: Build: Remove PyXB from cmake and use pyxb-x
This is an automated email from the ASF dual-hosted git repository.
okislal pushed a commit to branch madlib2-master
in repository https://gitbox.apache.org/repos/asf/madlib.git
commit b734a8f672bef782d5415419934ae6adcb11d148
Author: Orhan Kislal
AuthorDate: Fri Sep 1 20:39:20 2023 +0300
Build: Remove PyXB from cmake and use pyxb-x
PyXB is abandoned and broken for py3.10+. pyxb-x is a patched version of
pyxb that still has backwards compatibility. Instead of trying to pack
pyxb with the MADlib, we just rely on yum like many other libraries.
---
src/CMakeLists.txt | 84 -
tool/docker/base/Dockerfile_postgres_15_Jenkins | 2 +-
tool/jenkins/jenkins_build.sh | 4 +-
3 files changed, 45 insertions(+), 45 deletions(-)
diff --git a/src/CMakeLists.txt b/src/CMakeLists.txt
index 9b02df8d..247f6899 100644
--- a/src/CMakeLists.txt
+++ b/src/CMakeLists.txt
@@ -80,20 +80,20 @@ endif (NOT EIGEN_TAR_SOURCE)
# specify an alternative path to the PyXB tarball:
# -DPYXB_TAR_SOURCE=/path/to/PyXB-x.x.x.tar.gz
-set(PYXB_VERSION "1.2.6")
-set(PYXB_TAR_MD5 4303573fc7094ce4664c5b71cd4bdb48)
+# set(PYXB_VERSION "1.2.6")
+# set(PYXB_TAR_MD5 5689c94a3c47419213b8872753a73c07)
-set(PYXB_TAR "PyXB-${PYXB_VERSION}.tar.gz")
-set(PYXB_URL "${SOURCEFORGE_BASE_URL}/pyxb/files/${PYXB_TAR}")
+# set(PYXB_TAR "PyXB-${PYXB_VERSION}.tar.gz")
+# set(PYXB_URL
"https://github.com/renalreg/PyXB-X/archive/refs/tags/v1.2.6.1.tar.gz;)
-if(NOT PYXB_TAR_SOURCE)
-find_file(PYXB_TAR_SOURCE ${PYXB_TAR}
-PATHS ${MAD_THIRD_PARTY}/downloads)
-endif(NOT PYXB_TAR_SOURCE)
+# if(NOT PYXB_TAR_SOURCE)
+# find_file(PYXB_TAR_SOURCE ${PYXB_TAR}
+# PATHS ${MAD_THIRD_PARTY}/downloads)
+# endif(NOT PYXB_TAR_SOURCE)
-if(NOT PYXB_TAR_SOURCE)
-set(PYXB_TAR_SOURCE ${PYXB_URL})
-endif (NOT PYXB_TAR_SOURCE)
+# if(NOT PYXB_TAR_SOURCE)
+# set(PYXB_TAR_SOURCE ${PYXB_URL})
+# endif (NOT PYXB_TAR_SOURCE)
# -- Local definitions (filenames, paths, etc.)
@@ -147,37 +147,37 @@ find_package(PythonInterp REQUIRED)
set(BUILD_PYTHON_LIBDIR "${CMAKE_BINARY_DIR}/src/lib/python")
-if(IS_UBUNTU)
-ExternalProject_Add(EP_pyxb
-PREFIX ${MAD_THIRD_PARTY}
-DOWNLOAD_DIR ${MAD_THIRD_PARTY}/downloads
-URL ${PYXB_TAR_SOURCE}
-URL_MD5 ${PYXB_TAR_MD5}
-PATCH_COMMAND "${CMAKE_CURRENT_SOURCE_DIR}/patch/PyXB.sh"
-CONFIGURE_COMMAND ""
-SOURCE_DIR ${MAD_THIRD_PARTY}/src/EP_pyxb
-BUILD_COMMAND ""
-BUILD_IN_SOURCE 1
-
-INSTALL_COMMAND pip3 install .
-)
-endif(IS_UBUNTU)
-
-if((NOT IS_UBUNTU))
-ExternalProject_Add(EP_pyxb
-PREFIX ${MAD_THIRD_PARTY}
-DOWNLOAD_DIR ${MAD_THIRD_PARTY}/downloads
-URL ${PYXB_TAR_SOURCE}
-URL_MD5 ${PYXB_TAR_MD5}
-PATCH_COMMAND "${CMAKE_CURRENT_SOURCE_DIR}/patch/PyXB.sh"
-CONFIGURE_COMMAND ""
-SOURCE_DIR ${MAD_THIRD_PARTY}/src/EP_pyxb
-BUILD_COMMAND ""
-BUILD_IN_SOURCE 1
-
-INSTALL_COMMAND ${PYTHON_EXECUTABLE} setup.py install --install-lib
${BUILD_PYTHON_LIBDIR} > /dev/null
-)
-endif((NOT IS_UBUNTU))
+# if(IS_UBUNTU)
+# ExternalProject_Add(EP_pyxb
+# PREFIX ${MAD_THIRD_PARTY}
+# DOWNLOAD_DIR ${MAD_THIRD_PARTY}/downloads
+# URL ${PYXB_TAR_SOURCE}
+# URL_MD5 ${PYXB_TAR_MD5}
+# # PATCH_COMMAND "${CMAKE_CURRENT_SOURCE_DIR}/patch/PyXB.sh"
+# CONFIGURE_COMMAND ""
+# SOURCE_DIR ${MAD_THIRD_PARTY}/src/EP_pyxb
+# BUILD_COMMAND ""
+# BUILD_IN_SOURCE 1
+
+# INSTALL_COMMAND pip3 install .
+# )
+# endif(IS_UBUNTU)
+
+# if((NOT IS_UBUNTU))
+# ExternalProject_Add(EP_pyxb
+# PREFIX ${MAD_THIRD_PARTY}
+# DOWNLOAD_DIR ${MAD_THIRD_PARTY}/downloads
+# URL ${PYXB_TAR_SOURCE}
+# URL_MD5 ${PYXB_TAR_MD5}
+# # PATCH_COMMAND "${CMAKE_CURRENT_SOURCE_DIR}/patch/PyXB.sh"
+# CONFIGURE_COMMAND ""
+# SOURCE_DIR ${MAD_THIRD_PARTY}/src/EP_pyxb
+# BUILD_COMMAND ""
+# BUILD_IN_SOURCE 1
+
+# INSTALL_COMMAND ${PYTHON_EXECUTABLE} setup.py install --install-lib
${BUILD_PYTHON_LIBDIR} > /dev/null
+# )
+# endif((NOT IS_UBUNTU))
install(DIRECTORY ${BUILD_PYTHON_LIBDIR}
DESTINATION lib
diff --git a/tool/docker/base/Dockerfile_postgres_15_Jenkins
b/tool/docker/base/Dockerfile_postgres_15_Jenkins
index 37798616..760a6787 100644
--- a/tool/docker/base/Dockerfile_postgres_15_Jenkins
+++ b/tool/docker/base/Dockerfile_postgres_15_Jenkins
@@ -55,7 +55,7 @@ RUN apt-get update && apt-get install -y \
postgresql-client-15 \
libpq-dev
-RUN python3 -m pip install dill rtree xgboost mock pandas numpy scikit-learn
+RUN python3 -m pip install dill rtree xgboost mock pandas numpy scikit-learn
pyxb-x
## To build an image from this docker file,
[madlib] 01/05: Build: Use PG15 in jenkins build
This is an automated email from the ASF dual-hosted git repository. okislal pushed a commit to branch madlib2-master in repository https://gitbox.apache.org/repos/asf/madlib.git commit 845bcb5c8e6f0a71b609e50905683bd4d84ed70c Author: Orhan Kislal AuthorDate: Thu Aug 31 13:09:12 2023 +0300 Build: Use PG15 in jenkins build --- tool/jenkins/jenkins_build.sh | 59 ++- 1 file changed, 30 insertions(+), 29 deletions(-) diff --git a/tool/jenkins/jenkins_build.sh b/tool/jenkins/jenkins_build.sh index 72fb66ea..1a77b697 100755 --- a/tool/jenkins/jenkins_build.sh +++ b/tool/jenkins/jenkins_build.sh @@ -20,7 +20,7 @@ workdir=`pwd` user_name=`whoami` -# FIXME: Jenkins build is not working, even though PG13 compiles on local Mac. +# FIXME: Jenkins build is not working, even though PG15 compiles on local Mac. # Need to find an efficient way to debug since it has a server crash. echo "==" @@ -30,8 +30,8 @@ echo "--" echo "ls -la" ls -la echo "---" -echo "rm -rf build" -rm -rf build +echo "rm -rf build_jenkins" +rm -rf build_jenkins echo "---" echo "rm -rf logs" rm -rf logs @@ -46,14 +46,14 @@ docker rm madlib echo "Creating docker container" # Pull down the base docker images -echo "docker pull madlib/postgres_13:jenkins" -docker pull madlib/postgres_13:jenkins +echo "docker pull madlib/postgres_15:jenkins" +docker pull madlib/postgres_15:jenkins # Launch docker container with volume mounted from workdir echo "---" cat < /etc/security/limits.d/postgres-limits.conf ' | tee $workdir/logs/madlib_compile.log +docker exec madlib bash -c 'cp /madlib/tool/pg_hba.conf.postgres /etc/postgresql/15/main/pg_hba.conf; echo "* soft nproc unlimited" > /etc/security/limits.d/postgres-limits.conf ' | tee $workdir/logs/madlib_compile.log EOF -docker exec madlib bash -c 'cp /madlib/tool/pg_hba.conf.postgres /var/lib/postgresql/data/pg_hba.conf; echo "* soft nproc unlimited" > /etc/security/limits.d/postgres-limits.conf ' | tee $workdir/logs/madlib_compile.log +docker exec madlib bash -c 'cp /madlib/tool/pg_hba.conf.postgres /etc/postgresql/15/main/pg_hba.conf; echo "* soft nproc unlimited" > /etc/security/limits.d/postgres-limits.conf ' | tee $workdir/logs/madlib_compile.log cat <
[madlib] branch madlib2-master updated (6d621d52 -> 70c548c1)
This is an automated email from the ASF dual-hosted git repository. okislal pushed a change to branch madlib2-master in repository https://gitbox.apache.org/repos/asf/madlib.git from 6d621d52 Build: Add post- scripts for gppkg and gp7 version checks in madpack new 845bcb5c Build: Use PG15 in jenkins build new 40dfa0e3 DL: Update get_state_to_return to always return bytea new b734a8f6 Build: Remove PyXB from cmake and use pyxb-x new 395a8192 Fix anyarray -> anycompatiblearray change for PG14 new 70c548c1 Remove PyXB mentions from code, docs and licenses The 5 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: README.md | 3 +- ReadMe_Build.txt | 6 - deploy/PGXN/ReadMe.txt | 1 - licenses/third_party/PyXB_v1.2.6.txt | 268 - src/CMakeLists.txt | 60 - src/madpack/madpack.py | 5 +- src/patch/CMakeLists.txt | 2 - src/patch/PyXB.sh | 223 - src/ports/postgres/modules/crf/crf.sql_in | 4 +- .../modules/deep_learning/madlib_keras.py_in | 2 +- .../test/unit_tests/test_madlib_keras.py_in| 4 +- tool/docker/base/Dockerfile_postgres_15_Jenkins| 2 +- tool/jenkins/jenkins_build.sh | 63 ++--- 13 files changed, 42 insertions(+), 601 deletions(-) delete mode 100644 licenses/third_party/PyXB_v1.2.6.txt delete mode 100644 src/patch/CMakeLists.txt delete mode 100755 src/patch/PyXB.sh
svn commit: r63662 - in /dev/madlib: 2.1.0.RC1/ 2.1.0.RC1/apache-madlib-2.1.0-src.tar.gz 2.1.0.RC1/apache-madlib-2.1.0-src.tar.gz.asc 2.1.0.RC1/apache-madlib-2.1.0-src.tar.gz.sha512 KEYS
Author: okislal Date: Mon Aug 28 08:25:08 2023 New Revision: 63662 Log: adding 2.1.0 RC1 candidate release artifacts Added: dev/madlib/2.1.0.RC1/ dev/madlib/2.1.0.RC1/apache-madlib-2.1.0-src.tar.gz (with props) dev/madlib/2.1.0.RC1/apache-madlib-2.1.0-src.tar.gz.asc dev/madlib/2.1.0.RC1/apache-madlib-2.1.0-src.tar.gz.sha512 Modified: dev/madlib/KEYS Added: dev/madlib/2.1.0.RC1/apache-madlib-2.1.0-src.tar.gz == Binary file - no diff available. Propchange: dev/madlib/2.1.0.RC1/apache-madlib-2.1.0-src.tar.gz -- svn:mime-type = application/octet-stream Added: dev/madlib/2.1.0.RC1/apache-madlib-2.1.0-src.tar.gz.asc == --- dev/madlib/2.1.0.RC1/apache-madlib-2.1.0-src.tar.gz.asc (added) +++ dev/madlib/2.1.0.RC1/apache-madlib-2.1.0-src.tar.gz.asc Mon Aug 28 08:25:08 2023 @@ -0,0 +1,16 @@ +-BEGIN PGP SIGNATURE- + +iQIzBAABCAAdFiEEmb6cxeyylUySbAlYB2I3tq1KLboFAmTsVAcACgkQB2I3tq1K +LbrdRxAArfTe0oGOXSOWxro1GGTSp8ojUGwvQulm2H6PBUpZkL6EYAPAlRmHShN3 +5/CCBonr3ec/4tHIsByZYE/znvt1Y0bEPdbBlQxrgIYk7mPJURqBpRPYCNN25x/I +EbMShbNUp95iRb+Hh8jIy6OSA/+QY5RXnOYB3FOpidpqEot0A/XUeTOI4ns0qLAE +RNSZmO+4X9pfmDX3czoGVLdIg+GQcOtLmt5fbY5K2Rr6qPfePyTpr7Tr6vdvlL/F +0QO5+zvKex4X4jwg7zVrOqFNbUn6ojArHR2CBFG6bffLJKSb6d+PG1/2dT5B+B0G +2tmvWSywjdM3LwK9FC/02iJSsycu+ugFAWAeOZByRHdhI5uyEZS7JaEY3+wAFUMo +mOh5Eg7aFSrrxEEVsCl3CSKxu9IDyJoYwduUadste9CPVhJAM8y+wnn4JeKsDp4/ +wtJlMH47Zdb04t3/jy+vNmSlDnSbA3OICLGwS82BQzwbFcEDE3KWHbIcbin6wcrC +9Nz1kKDuqJmb99W+RQVuSAdhvGva8zUamYQjsEMTSfvpCwALnE2uIF//fhFn+AgB +S++3vf7Y5mMJbQaKFeQOLurne0c9yi6SNha4xACsqqVgDaU/nKb08FShzKuaOr9S +IvzQq7dOv3K6FsWdfCyITqnSJeFoqtBi4Q7zUAyAip7UK2mGkOs= +=RMMU +-END PGP SIGNATURE- Added: dev/madlib/2.1.0.RC1/apache-madlib-2.1.0-src.tar.gz.sha512 == --- dev/madlib/2.1.0.RC1/apache-madlib-2.1.0-src.tar.gz.sha512 (added) +++ dev/madlib/2.1.0.RC1/apache-madlib-2.1.0-src.tar.gz.sha512 Mon Aug 28 08:25:08 2023 @@ -0,0 +1 @@ +3695464bbe14e0f9e872e18660cbc47b535e2589f729c832f59735e5ef9b6571d33602645b5f9de34c56c844136b5fd22d4d957a25061cbf63be3ed31034fe2d apache-madlib-2.1.0-src.tar.gz Modified: dev/madlib/KEYS == --- dev/madlib/KEYS (original) +++ dev/madlib/KEYS Mon Aug 28 08:25:08 2023 @@ -916,3 +916,62 @@ D19xsEK94oAgFzRt8z+1i5AC3r5JqX+Hu/VYJcPA vxKk5jNhKFms9tPYXh8bUypynx4H2BjsY6bF =M01/ -END PGP PUBLIC KEY BLOCK- +pub rsa4096 2023-08-28 [SC] + 99BE9CC5ECB2954C926C0958076237B6AD4A2DBA +uid [ultimate] Orhan Kislal (Signing Key 2023) +sig 3076237B6AD4A2DBA 2023-08-28 [self-signature] +sub rsa4096 2023-08-28 [E] +sig 076237B6AD4A2DBA 2023-08-28 [self-signature] + +-BEGIN PGP PUBLIC KEY BLOCK- + +mQINBGTsUxsBEADHb/bhm13h7jv94fssKvm93n6v4aubXzxkkrxjCQ8ZQ5pf6YrZ +lF33pR/VC3l4K3YGd/J3pqAFIYf0E5/ZujWRmE/sw1yuRBjj4CE1+/cfRLiHsQK6 +I2MBUHqelIYQZU23dZM8QYd8imCccjtQ90tUz/E21FMgVdxvrB+hQMc2yFdfBe4Z +dVHFg5YMhUsh0ae07yFSXW/6McFJss7bUaLf5SAOXtwrlCbrwnpLZZP60wKAYju5 ++cZtvH0c7aHZHXO0CYnC8CXBAKU+bj096x8rA4DS18bqLE8mRePbltRQezOY7/hq +dfgFx0kJv4MnjgaUhmqMmPcr/hVZbNxxQwp+zFb5DmmELPqw9mhrSVNY4GvBZKoQ +hcd+2dsiWEPVju53FZxET2Ep4QZK8rH0beMaxTomivydV0K0zQtRa4ra1tEfsBUo +XdF0gQKoWCzKVUSgRSSJbXAEWE5l7DysM0tP3L2jIO7J0cC4UaWydpMtVnuseWPQ +sVjmS8Rf3uWQZhiGtyCMPui898iM6btUn0Gc+7ZUr5hD43JO+DmUkha2G+tPIdQm +xgYtvB4uMzfPJ0WiOE64FPcwQvFwVLtZyFKStmm3osamJ+BjiPJikcifSSDLNdtM +jrSA3arT3EnTVlnG9qwqVOjw0nOSsz68Wmmfmb0QG+tFQwl/gKf1gVFgmQARAQAB +tDRPcmhhbiBLaXNsYWwgKFNpZ25pbmcgS2V5IDIwMjMpIDxva2lzbGFsQHZtd2Fy +ZS5jb20+iQJRBBMBCAA7FiEEmb6cxeyylUySbAlYB2I3tq1KLboFAmTsUxsCGwMF +CwkIBwICIgIGFQoJCAsCBBYCAwECHgcCF4AACgkQB2I3tq1KLbpiag//XPORkZ5a +fl9uNaNbmiTYTBBGcHGnkqNisR2EDFqu1bTrXNMxfbX3aAjdxEmuqHciqP7SWh9e +bBs0k4BcKkMIvnyoGtxFmWpuKlachGYMuIJihw4cbyqsa7Hr1JuNdYS/ucJMBWl4 +smT+qOE9Lx1wFKRir4kNfmwjMEzcwvfwp9QOw2TA0crCLTz7m48fRz+DOQZqqEjR +59aOIVIxukvZGejLNNxUAoL9gZPveDqmc1VGydIci/VLmXmJEpFWKnms3WoH/8ij +ZnVjdG6bOe/vBDYt9WyQDo4cSWrC8fr5tdkBTyHKKO4MPwIm5pnSsGpHf0INx/nb +TpQGnlwmyKyaHl4o0sQXo8R8DXES9CjnvJmTYd4GizEZ4SJDStXR688pICFoFEuc +45qd78EHzK4qo2ZeLhoznYmuYlMMixdVtJoBQdkuBSWJx9eYFeFG8R9JWsA2I5QQ +wdyht5fqdtgPsFIywjaO98DkPIQdjbwyqScHevU2d1PFJg1+xnslqXoSHM7aNiyA +ZN9l/huQJCSY8rXXI5l0hU+G7o+SejMpJAK+HtvlAvgecgGiFg+XjRDb56pK7EXf +frBt4+D445bZePpI3CrUjfREUT0ZnpJPMjv5il1NYBw64hV4v1mJUaQmL5JRVUt/ +gFfQo7ms7vyIYSWz0m6Q+IgUwHqpcEIq/gO5Ag0EZOxTGwEQALFGS418vEzdONoA +o+svivY+UQ6s8zs/wKPMu01yF+/g6TQWzlv6B8F9bxTZEiWMYjMnCDkiAGAMyuKK +qMRcEz4+Ff5q0kXwW6Kmb799Rgq8KC/CePTjeOHIaxi1v2Ev37AFDTUZPbNewg31 +bC2Xtr4bApfDhn1alX3UKyuR3p60fBW+N6eoFZJFPcjz0wPyJCDNQ/0iHdOmbLY4 +A/iYv6IC2m8klLCgaHuJK31yenTRl4QMtr+Jg5Plpv46MMEeQpL4ESYoYBudtugV +Jpiu8UCQREDM+Cji2h/91Tf3tSY/EcXczvyCwgeBRe5o
[madlib] annotated tag rc/2.1.0-rc1 updated (6d621d52 -> 8da13a83)
This is an automated email from the ASF dual-hosted git repository. okislal pushed a change to annotated tag rc/2.1.0-rc1 in repository https://gitbox.apache.org/repos/asf/madlib.git *** WARNING: tag rc/2.1.0-rc1 was modified! *** from 6d621d52 (commit) to 8da13a83 (tag) tagging 6d621d5291b5381d1cb048739e8328cc5c527dea (commit) replaces rel/v2.0.0 by Orhan Kislal on Mon Aug 28 08:56:11 2023 +0300 - Log - Apache MADlib 2.1.0 RC1 -BEGIN PGP SIGNATURE- iQEzBAABCAAdFiEEGocCIKIuO+T6G3jEyiWcTeuzJCIFAmTsNvwACgkQyiWcTeuz JCLP3ggA1ZBjcrzNn5U5ZYBtjf22MnS1ttF6I4ZRQM9eA7+jT/Kw3ULL2JrHrCuQ BGiab5dO/KcOPCsTLZLc0cm5XB8+PCBbYa0Rk0fdkcnmVWLxJvUCq1cWRQCgv08f HNLxa+I9DfMRsxpn+LF0kFz3cqXdr6p7av5fv+2XHs18yY5vordbNLhAa/z9CBmW wG3CgDivSQo7IlNUm0g5M0EQWUQOx0el4ibjn2UzzBXY+CQ9zH1DqpjYezXcpUDa QsvKENAsEZRdXk3eJlW2PDGiZeq9L/j2QL5X3cXx909cNqIx/0QGlN2XK7x52jj2 53TbRvKgmvNPem1WdoYVswog8WGjmQ== =nVpQ -END PGP SIGNATURE- --- No new revisions were added by this update. Summary of changes:
[madlib] branch madlib2-master updated (c0e6e6d8 -> 6d621d52)
This is an automated email from the ASF dual-hosted git repository.
okislal pushed a change to branch madlib2-master
in repository https://gitbox.apache.org/repos/asf/madlib.git
from c0e6e6d8 Update version number to 2.1.0 and add release notes
new 279d25e7 Add 2.0.0 to 2.1.0 changelist
new 6d621d52 Build: Add post- scripts for gppkg and gp7 version checks in
madpack
The 2 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
deploy/gppkg/gppkg_spec_v2.yml.in| 16 +++-
...st_1.18.0_1.19.0.yaml => changelist_2.0.0_2.1.0.yaml} | 2 +-
src/madpack/madpack.py | 11 ++-
src/madpack/upgrade_util.py | 13 +
4 files changed, 31 insertions(+), 11 deletions(-)
copy src/madpack/{changelist_1.18.0_1.19.0.yaml =>
changelist_2.0.0_2.1.0.yaml} (97%)
[madlib] 02/02: Build: Add post- scripts for gppkg and gp7 version checks in madpack
This is an automated email from the ASF dual-hosted git repository.
okislal pushed a commit to branch madlib2-master
in repository https://gitbox.apache.org/repos/asf/madlib.git
commit 6d621d5291b5381d1cb048739e8328cc5c527dea
Author: Orhan Kislal
AuthorDate: Thu Aug 24 23:33:21 2023 +0300
Build: Add post- scripts for gppkg and gp7 version checks in madpack
---
deploy/gppkg/gppkg_spec_v2.yml.in | 16 +++-
src/madpack/madpack.py| 11 ++-
src/madpack/upgrade_util.py | 13 +
3 files changed, 30 insertions(+), 10 deletions(-)
diff --git a/deploy/gppkg/gppkg_spec_v2.yml.in
b/deploy/gppkg/gppkg_spec_v2.yml.in
index 268d131c..25fd84a6 100644
--- a/deploy/gppkg/gppkg_spec_v2.yml.in
+++ b/deploy/gppkg/gppkg_spec_v2.yml.in
@@ -40,7 +40,21 @@ PostInstall:
find $GPHOME/madlib/bin -depth -type l -exec rm {} \; 2>/dev/null;
find $GPHOME/madlib/doc -depth -type l -exec rm {} \; 2>/dev/null;
find $GPHOME/madlib/Current -depth -type l -exec rm {} \; 2>/dev/null;
-
ln -nsf $GPHOME/madlib/Versions/@MADLIB_VERSION_STRING@
$GPHOME/madlib/Current;
ln -nsf $GPHOME/madlib/Current/bin $GPHOME/madlib/bin;
ln -nsf $GPHOME/madlib/Current/doc $GPHOME/madlib/doc;
+PostUpgrade:
+ All:
+# Remove existing soft links
+find $GPHOME/madlib/bin -depth -type l -exec rm {} \; 2>/dev/null;
+find $GPHOME/madlib/doc -depth -type l -exec rm {} \; 2>/dev/null;
+find $GPHOME/madlib/Current -depth -type l -exec rm {} \; 2>/dev/null;
+ln -nsf $GPHOME/madlib/Versions/@MADLIB_VERSION_STRING@
$GPHOME/madlib/Current;
+ln -nsf $GPHOME/madlib/Current/bin $GPHOME/madlib/bin;
+ln -nsf $GPHOME/madlib/Current/doc $GPHOME/madlib/doc;
+PostUninstall:
+ All:
+# Remove existing soft links
+find $GPHOME/madlib/bin -depth -type l -exec rm {} \; 2>/dev/null;
+find $GPHOME/madlib/doc -depth -type l -exec rm {} \; 2>/dev/null;
+find $GPHOME/madlib/Current -depth -type l -exec rm {} \; 2>/dev/null;
diff --git a/src/madpack/madpack.py b/src/madpack/madpack.py
index 92836a12..87df1c06 100755
--- a/src/madpack/madpack.py
+++ b/src/madpack/madpack.py
@@ -1238,16 +1238,17 @@ def create_install_madlib_sqlfile(args, madpack_cmd):
def get_madlib_function_drop_str(schema):
-if portid == 'greenplum':
+if ((portid == 'greenplum' and is_rev_gte(get_rev_num(dbver),
get_rev_num('7.0'))) or
+(portid == 'postgres')):
case_str = """
-CASE
- WHEN p.proisagg THEN 'aggregate'
+CASE p.prokind
+ WHEN 'a' THEN 'aggregate'
ELSE 'function'
"""
else:
case_str = """
-CASE p.prokind
- WHEN 'a' THEN 'aggregate'
+CASE
+ WHEN p.proisagg THEN 'aggregate'
ELSE 'function'
"""
madlib_functions = _internal_run_query("""
diff --git a/src/madpack/upgrade_util.py b/src/madpack/upgrade_util.py
index f9bbb7b9..f7e5cd09 100644
--- a/src/madpack/upgrade_util.py
+++ b/src/madpack/upgrade_util.py
@@ -498,9 +498,12 @@ class ViewDependency(UpgradeBase):
"""
@brief Detect direct view dependencies on MADlib UDFs/UDAs
"""
-proisagg_wrapper = "p.proisagg"
-if self._portid == 'postgres' and self._dbver > 11:
+if ((self._portid == 'greenplum' and
is_rev_gte(get_rev_num(self._dbver), get_rev_num('7.0'))) or
+(self._portid == 'postgres')):
proisagg_wrapper = "p.prokind = 'a'"
+else:
+proisagg_wrapper = "p.proisagg"
+
rows = self._run_sql("""
SELECT
view, nsp.nspname AS schema, procname, procoid, proisagg
@@ -1029,9 +1032,11 @@ class ScriptCleaner(UpgradeBase):
"""
# See _get_function_info for explanations.
-proisagg_wrapper = "p.proisagg = true"
-if self._portid == 'postgres' and self._dbver > 11:
+if ((self._portid == 'greenplum' and
is_rev_gte(get_rev_num(self._dbver), get_rev_num('7.0'))) or
+(self._portid == 'postgres')):
proisagg_wrapper = "p.prokind = 'a'"
+else:
+proisagg_wrapper = "p.proisagg"
rows = self._run_sql("""
SELECT
[madlib] 01/02: Add 2.0.0 to 2.1.0 changelist
This is an automated email from the ASF dual-hosted git repository. okislal pushed a commit to branch madlib2-master in repository https://gitbox.apache.org/repos/asf/madlib.git commit 279d25e7261ef0462c0ad681d3efe9caf76a8ee7 Author: Orhan Kislal AuthorDate: Thu Aug 24 22:53:05 2023 +0300 Add 2.0.0 to 2.1.0 changelist --- src/madpack/changelist_2.0.0_2.1.0.yaml | 54 + 1 file changed, 54 insertions(+) diff --git a/src/madpack/changelist_2.0.0_2.1.0.yaml b/src/madpack/changelist_2.0.0_2.1.0.yaml new file mode 100644 index ..907b4789 --- /dev/null +++ b/src/madpack/changelist_2.0.0_2.1.0.yaml @@ -0,0 +1,54 @@ +# -- +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. +# -- + +# Changelist for MADlib version rel/v2.0.0 to 2.1.0 + +# This file contains all changes that were introduced in a new version of +# MADlib. This changelist is used by the upgrade script to detect what objects +# should be upgraded (while retaining all other objects from the previous version) + +# New modules (actually .sql_in files) added in upgrade version +# For these files the sql_in code is retained as is with the functions in the +# file installed on the upgrade version. All other files (that don't have +# updates), are cleaned up to remove object replacements +new module: + +# Changes in the types (UDT) including removal and modification +udt: + +# List of the UDF changes that affect the user externally. This includes change +# in function name, return type, argument order or types, or removal of +# the function. In each case, the original function is as good as removed and a +# new function is created. In such cases, we should abort the upgrade if there +# are user views dependent on this function, since the original function will +# not be present in the upgraded version. +udf: + +# Changes to aggregates (UDA) including removal and modification +# Overloaded functions should be mentioned separately +uda: + +# List of the UDC, UDO and UDOC changes. +udc: + +# Changes in the operators (UDO) +udo: + +# Changes in the operator classes (UDOC) +udoc:
[madlib] branch madlib2-master updated: Update version number to 2.1.0 and add release notes
This is an automated email from the ASF dual-hosted git repository.
okislal pushed a commit to branch madlib2-master
in repository https://gitbox.apache.org/repos/asf/madlib.git
The following commit(s) were added to refs/heads/madlib2-master by this push:
new c0e6e6d8 Update version number to 2.1.0 and add release notes
c0e6e6d8 is described below
commit c0e6e6d834187b2bb4a25386a5877acbe8843620
Author: Orhan Kislal
AuthorDate: Wed Aug 23 21:38:08 2023 +0300
Update version number to 2.1.0 and add release notes
---
RELEASE_NOTES| 12
deploy/DEB/postinst | 2 +-
deploy/postflight.sh | 2 +-
pom.xml | 2 +-
src/config/Version.yml | 2 +-
src/ports/postgres/modules/deep_learning/madlib_keras.sql_in | 8
.../modules/deep_learning/madlib_keras_automl.sql_in | 4 ++--
.../deep_learning/madlib_keras_fit_multiple_model.sql_in | 6 +++---
src/ports/postgres/modules/svm/svm.sql_in| 2 +-
9 files changed, 26 insertions(+), 14 deletions(-)
diff --git a/RELEASE_NOTES b/RELEASE_NOTES
index 7a26d45d..8ec4be69 100644
--- a/RELEASE_NOTES
+++ b/RELEASE_NOTES
@@ -9,6 +9,18 @@ commit history located at
https://github.com/apache/madlib/commits/master.
Current list of bugs and issues can be found at
https://issues.apache.org/jira/browse/MADLIB.
+—-
+MADlib v2.1.0:
+
+Release Date: 2023-Aug-XX
+
+Bug fixes:
+- Build: Fix PG 15 support
+- Assoc_rules: Fix SERIAL cache issue
+- DL: Remove SERIAL from load_keras_model
+- Build: Add ubuntu flag for PyXB installation
+- Build: Add the actual path of $libdir to dynamic_library_path
+
—-
MADlib v2.0.0:
diff --git a/deploy/DEB/postinst b/deploy/DEB/postinst
index 974cc7d2..a00cd197 100755
--- a/deploy/DEB/postinst
+++ b/deploy/DEB/postinst
@@ -22,7 +22,7 @@
# Source debconf library.
. /usr/share/debconf/confmodule
-MADLIB_VERSION="2.1.0-dev"
+MADLIB_VERSION="2.1.0"
MADLIB_INSTALL_PATH="InstallPathNotFound"
# Fetching configuration from debconf
diff --git a/deploy/postflight.sh b/deploy/postflight.sh
index da2df7fd..2b698c0f 100755
--- a/deploy/postflight.sh
+++ b/deploy/postflight.sh
@@ -2,7 +2,7 @@
# $0 - Script Path, $1 - Package Path, $2 - Target Location, and $3 - Target
Volume
-MADLIB_VERSION=2.1.0-dev
+MADLIB_VERSION=2.1.0
# Remove existing soft links
find $2/usr/local/madlib/bin -depth -type l -exec rm {} \; 2>/dev/null
diff --git a/pom.xml b/pom.xml
index ed41f2c8..f70c25d5 100644
--- a/pom.xml
+++ b/pom.xml
@@ -22,7 +22,7 @@
org.apache.madlib
madlib
- 2.1.0-dev
+ 2.1.0
pom
diff --git a/src/config/Version.yml b/src/config/Version.yml
index be32da7e..7396d02b 100644
--- a/src/config/Version.yml
+++ b/src/config/Version.yml
@@ -1 +1 @@
-version: 2.1.0-dev
+version: 2.1.0
diff --git a/src/ports/postgres/modules/deep_learning/madlib_keras.sql_in
b/src/ports/postgres/modules/deep_learning/madlib_keras.sql_in
index 67cf2722..10805847 100644
--- a/src/ports/postgres/modules/deep_learning/madlib_keras.sql_in
+++ b/src/ports/postgres/modules/deep_learning/madlib_keras.sql_in
@@ -1143,7 +1143,7 @@ model_size| 0.7900390625
start_training_time | 2021-02-01 15:58:43.760568
end_training_time | 2021-02-01 15:58:44.470054
metrics_elapsed_time | {0.709463119506836}
-madlib_version| 2.1.0-dev
+madlib_version| 2.1.0
num_classes | {3}
dependent_vartype | {"character varying"}
normalizing_const | 1
@@ -1457,7 +1457,7 @@ model_size| 0.7900390625
start_training_time | 2021-01-29 14:41:16.943861
end_training_time | 2021-01-29 14:41:19.478149
metrics_elapsed_time |
{2.3377411365509,2.42358803749084,2.49885511398315,2.53427410125732}
-madlib_version| 2.1.0-dev
+madlib_version| 2.1.0
num_classes | {3}
dependent_vartype | {"character varying"}
normalizing_const | 1
@@ -1627,7 +1627,7 @@ model_size| 0.7900390625
start_training_time | 2021-01-29 14:42:28.780276
end_training_time | 2021-01-29 14:42:31.177561
metrics_elapsed_time |
{2.24628114700317,2.28473520278931,2.32178020477295,2.35844302177429,2.39726710319519}
-madlib_version| 2.1.0-dev
+madlib_version| 2.1.0
num_classes | {3}
dependent_vartype | {"character varying"}
normalizing_const | 1
@@ -1742,7 +1742,7 @@ model_size| 0.7900390625
start_training_time | 2021-01-29 14:44:51
[madlib] branch madlib2-master updated: Madpack: Add the actual path of $libdir
This is an automated email from the ASF dual-hosted git repository.
okislal pushed a commit to branch madlib2-master
in repository https://gitbox.apache.org/repos/asf/madlib.git
The following commit(s) were added to refs/heads/madlib2-master by this push:
new 66f3fe9e Madpack: Add the actual path of $libdir
66f3fe9e is described below
commit 66f3fe9e5f55fb91be72832d71b5f7813fd1103f
Author: Orhan Kislal
AuthorDate: Tue Aug 22 16:58:29 2023 +0300
Madpack: Add the actual path of $libdir
MADlib has to set dynamic_library_path but gpconfig do not work with the
default value $libdir. We get the full libdir path and use it instead of
the variable.
---
src/madpack/madpack.py | 9 -
1 file changed, 8 insertions(+), 1 deletion(-)
diff --git a/src/madpack/madpack.py b/src/madpack/madpack.py
index 7c91911b..92836a12 100755
--- a/src/madpack/madpack.py
+++ b/src/madpack/madpack.py
@@ -1332,11 +1332,18 @@ def set_dynamic_library_path_in_database(dbver_split,
madlib_library_path):
global dynamic_library_path
dynamic_library_path = _internal_run_query("SHOW dynamic_library_path",
True)[0]['dynamic_library_path']
-# GP7 gpconfig messes up $libdir so we remove it for now
+# GP7 gpconfig messes up $libdir so we put the actual path
paths = dynamic_library_path.split(":")
if madlib_library_path not in paths:
if '$libdir' in paths:
paths.remove('$libdir')
+libdir = subprocess.check_output(['pg_config','--libdir'])
+if ((portid == 'greenplum' and is_rev_gte(dbver_split,
get_rev_num('7.0'))) or
+(portid == 'postgres' and is_rev_gte(dbver_split,
get_rev_num('13.0':
+libdir = libdir.decode()
+
+libdir = libdir.strip()+'/postgresql'
+paths.append(libdir)
paths.append(madlib_library_path)
dynamic_library_path = ':'.join(paths)
[madlib] 02/02: Build: Add Ubuntu flag for PyXB installation
This is an automated email from the ASF dual-hosted git repository.
okislal pushed a commit to branch madlib2-master
in repository https://gitbox.apache.org/repos/asf/madlib.git
commit 96dd1d5b705c7d51f91cb5a34a4df44cdc8c3960
Author: Orhan Kislal
AuthorDate: Fri Aug 18 15:34:03 2023 +0300
Build: Add Ubuntu flag for PyXB installation
PyXB uses distutils which is being phased out. The ubuntu images fail to
install it the old way so this commit adds a flag to identify the OS as
Ubuntu during cmake and use pip install as needed.
---
src/CMakeLists.txt | 20
1 file changed, 20 insertions(+)
diff --git a/src/CMakeLists.txt b/src/CMakeLists.txt
index fc4be6ba..9b02df8d 100644
--- a/src/CMakeLists.txt
+++ b/src/CMakeLists.txt
@@ -146,6 +146,8 @@ find_package(PythonInterp REQUIRED)
set(BUILD_PYTHON_LIBDIR "${CMAKE_BINARY_DIR}/src/lib/python")
+
+if(IS_UBUNTU)
ExternalProject_Add(EP_pyxb
PREFIX ${MAD_THIRD_PARTY}
DOWNLOAD_DIR ${MAD_THIRD_PARTY}/downloads
@@ -156,8 +158,26 @@ ExternalProject_Add(EP_pyxb
SOURCE_DIR ${MAD_THIRD_PARTY}/src/EP_pyxb
BUILD_COMMAND ""
BUILD_IN_SOURCE 1
+
+INSTALL_COMMAND pip3 install .
+)
+endif(IS_UBUNTU)
+
+if((NOT IS_UBUNTU))
+ExternalProject_Add(EP_pyxb
+PREFIX ${MAD_THIRD_PARTY}
+DOWNLOAD_DIR ${MAD_THIRD_PARTY}/downloads
+URL ${PYXB_TAR_SOURCE}
+URL_MD5 ${PYXB_TAR_MD5}
+PATCH_COMMAND "${CMAKE_CURRENT_SOURCE_DIR}/patch/PyXB.sh"
+CONFIGURE_COMMAND ""
+SOURCE_DIR ${MAD_THIRD_PARTY}/src/EP_pyxb
+BUILD_COMMAND ""
+BUILD_IN_SOURCE 1
+
INSTALL_COMMAND ${PYTHON_EXECUTABLE} setup.py install --install-lib
${BUILD_PYTHON_LIBDIR} > /dev/null
)
+endif((NOT IS_UBUNTU))
install(DIRECTORY ${BUILD_PYTHON_LIBDIR}
DESTINATION lib
[madlib] 01/02: Various: Reduce SERIAL usage
This is an automated email from the ASF dual-hosted git repository.
okislal pushed a commit to branch madlib2-master
in repository https://gitbox.apache.org/repos/asf/madlib.git
commit 7a16084c03343e1f7ee8aa22708ac94d8b52cd68
Author: Orhan Kislal
AuthorDate: Wed Jul 12 08:56:01 2023 -0400
Various: Reduce SERIAL usage
Since SERIAL columns do not guarantee a sequence of numbers with no
gaps, this commit removes its usage from load_keras_model.
The issue was unearthed by a change in the GPDB code setting the cache
for sequences to 20 and causing the numbers to jump unexpectedly.
A similar issue is found in assoc_rules. There we wanted to avoid using
row_number so we set the cache to 1 to get the previous behavior.
---
.../postgres/modules/assoc_rules/assoc_rules.py_in | 1 +
.../deep_learning/keras_model_arch_table.py_in | 12 --
.../test/madlib_keras_iris.setup.sql_in| 47 +-
.../test/madlib_keras_transfer_learning.sql_in | 30 +-
4 files changed, 48 insertions(+), 42 deletions(-)
diff --git a/src/ports/postgres/modules/assoc_rules/assoc_rules.py_in
b/src/ports/postgres/modules/assoc_rules/assoc_rules.py_in
index 1d7d8314..ada27c71 100644
--- a/src/ports/postgres/modules/assoc_rules/assoc_rules.py_in
+++ b/src/ports/postgres/modules/assoc_rules/assoc_rules.py_in
@@ -322,6 +322,7 @@ def assoc_rules(madlib_schema, support, confidence, tid_col,
)
m4_ifdef(`__POSTGRESQL__', `', `DISTRIBUTED BY (id)')
""".format(**locals()));
+plpy.execute("ALTER SEQUENCE {assoc_loop_aux}_id_seq CACHE
1".format(**locals()));
if verbose :
plpy.info("{0} Frequent itemsets found in this iteration".format(
diff --git
a/src/ports/postgres/modules/deep_learning/keras_model_arch_table.py_in
b/src/ports/postgres/modules/deep_learning/keras_model_arch_table.py_in
index 5fc6c35a..286f68bf 100644
--- a/src/ports/postgres/modules/deep_learning/keras_model_arch_table.py_in
+++ b/src/ports/postgres/modules/deep_learning/keras_model_arch_table.py_in
@@ -56,7 +56,7 @@ class ModelArchSchema:
"""
col_names = ('model_id', 'model_arch', 'model_weights', 'name',
'description',
'__internal_madlib_id__')
-col_types = ('SERIAL PRIMARY KEY', 'JSON', 'bytea', 'TEXT', 'TEXT', 'TEXT')
+col_types = ('INTEGER', 'JSON', 'bytea', 'TEXT', 'TEXT', 'TEXT')
(MODEL_ID, MODEL_ARCH, MODEL_WEIGHTS, NAME, DESCRIPTION,
__INTERNAL_MADLIB_ID__) = col_names
@@ -75,6 +75,7 @@ def load_keras_model(keras_model_arch_table, model_arch,
model_weights,
plpy.execute(sql, 0)
plpy.info("Keras Model Arch: Created new keras model architecture
table {0}." \
.format(model_arch_table))
+last_id = 0
else:
missing_cols = columns_missing_from_table(model_arch_table,
ModelArchSchema.col_names)
@@ -82,12 +83,15 @@ def load_keras_model(keras_model_arch_table, model_arch,
model_weights,
plpy.error("Keras Model Arch: Invalid keras model architecture
table {0},"
" missing columns: {1}".format(model_arch_table,
missing_cols))
+last_id = plpy.execute("SELECT max({0}) FROM {1}".format(
+ModelArchSchema.MODEL_ID, model_arch_table))[0]['max']
+last_id = last_id if last_id is not None else 0
unique_str = unique_string(prefix_has_temp=False)
insert_query = plpy.prepare("INSERT INTO {model_arch_table} "
-"VALUES(DEFAULT, $1, $2, $3, $4,
$5);".format(**locals()),
-ModelArchSchema.col_types[1:])
-insert_res = plpy.execute(insert_query,[model_arch, model_weights, name,
description,
+"VALUES($1, $2, $3, $4, $5,
$6);".format(**locals()),
+ModelArchSchema.col_types)
+insert_res = plpy.execute(insert_query, [last_id+1, model_arch,
model_weights, name, description,
unique_str], 0)
select_query = """SELECT {model_id_col}, {model_arch_col} FROM
{model_arch_table}
diff --git
a/src/ports/postgres/modules/deep_learning/test/madlib_keras_iris.setup.sql_in
b/src/ports/postgres/modules/deep_learning/test/madlib_keras_iris.setup.sql_in
index 7f68268d..e87930d1 100644
---
a/src/ports/postgres/modules/deep_learning/test/madlib_keras_iris.setup.sql_in
+++
b/src/ports/postgres/modules/deep_learning/test/madlib_keras_iris.setup.sql_in
@@ -23,7 +23,7 @@
DROP TABLE IF EXISTS iris_data;
CREATE TABLE iris_data(
-id serial,
+id integer,
attributes numeric[],
class_text varchar
);
@@ -281,6 +281,43 @@ $$
$$
);
+SELECT load_keras_model('iris_model_arch',
[madlib] branch madlib2-master updated (f91813be -> 96dd1d5b)
This is an automated email from the ASF dual-hosted git repository. okislal pushed a change to branch madlib2-master in repository https://gitbox.apache.org/repos/asf/madlib.git from f91813be Build: Add support for PG15 new 7a16084c Various: Reduce SERIAL usage new 96dd1d5b Build: Add Ubuntu flag for PyXB installation The 2 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: src/CMakeLists.txt | 20 + .../postgres/modules/assoc_rules/assoc_rules.py_in | 1 + .../deep_learning/keras_model_arch_table.py_in | 12 -- .../test/madlib_keras_iris.setup.sql_in| 47 +- .../test/madlib_keras_transfer_learning.sql_in | 30 +- 5 files changed, 68 insertions(+), 42 deletions(-)
[madlib] branch madlib2-master updated: Build: Add support for PG15
This is an automated email from the ASF dual-hosted git repository.
okislal pushed a commit to branch madlib2-master
in repository https://gitbox.apache.org/repos/asf/madlib.git
The following commit(s) were added to refs/heads/madlib2-master by this push:
new f91813be Build: Add support for PG15
f91813be is described below
commit f91813bedb275bb042968a5e38d4293bbad6b325
Author: Orhan Kislal
AuthorDate: Thu Jun 15 21:02:08 2023 -0400
Build: Add support for PG15
---
methods/array_ops/src/pg_gp/array_ops.c| 4
methods/sketch/src/pg_gp/sketch_support.c | 10 --
src/madpack/madpack.py | 12
src/ports/postgres/dbconnector/UDF_impl.hpp| 4
.../modules/bayes/test/gaussian_naive_bayes.sql_in | 2 +-
src/ports/postgres/modules/crf/crf.sql_in | 14 --
.../modules/deep_learning/madlib_keras_gpu_info.py_in | 8 +++-
src/ports/postgres/modules/graph/wcc.sql_in| 4 ++--
src/ports/postgres/modules/sample/balance_sample.py_in | 4 ++--
.../postgres/modules/utilities/encode_categorical.py_in| 2 +-
10 files changed, 49 insertions(+), 15 deletions(-)
diff --git a/methods/array_ops/src/pg_gp/array_ops.c
b/methods/array_ops/src/pg_gp/array_ops.c
index a842a605..574a9571 100644
--- a/methods/array_ops/src/pg_gp/array_ops.c
+++ b/methods/array_ops/src/pg_gp/array_ops.c
@@ -8,7 +8,11 @@
#include "utils/numeric.h"
#include "utils/builtins.h"
#include "utils/memutils.h"
+#if GP_VERSION_NUM >= 7 || PG_VERSION_NUM >= 13
+#include "utils/fmgrprotos.h"
+#else
#include "utils/int8.h"
+#endif
#include "utils/datum.h"
#include "utils/lsyscache.h"
#include "utils/typcache.h"
diff --git a/methods/sketch/src/pg_gp/sketch_support.c
b/methods/sketch/src/pg_gp/sketch_support.c
index 1cdeb904..f8baf6a6 100644
--- a/methods/sketch/src/pg_gp/sketch_support.c
+++ b/methods/sketch/src/pg_gp/sketch_support.c
@@ -306,6 +306,7 @@ bytea *sketch_md5_bytea(Datum dat, Oid typOid)
bool byval = get_typbyval(typOid);
int len = ExtractDatumLen(dat, get_typlen(typOid), byval, -1);
void *datp = DatumExtractPointer(dat, byval);
+
/*
* it's very common to be hashing 0 for countmin sketches. Rather than
* hard-code it here, we cache on first lookup. In future a bigger cache
here
@@ -318,9 +319,14 @@ bytea *sketch_md5_bytea(Datum dat, Oid typOid)
if (byval && len == sizeof(int64) && *(int64 *)datp == 0 && zero_cached) {
return md5_of_0;
}
-else
+else{
+#if defined(GP_VERSION_NUM) || PG_VERSION_NUM < 15
pg_md5_hash(datp, len, outbuf);
-
+#else
+const char *errstr = NULL;
+pg_md5_hash(datp, len, outbuf, );
+#endif
+}
hex_to_bytes(outbuf, (uint8 *)VARDATA(out), MD5_HASHLEN*2);
SET_VARSIZE(out, MD5_HASHLEN+VARHDRSZ);
if (byval && len == sizeof(int64) && *(int64 *)datp == 0 && !zero_cached) {
diff --git a/src/madpack/madpack.py b/src/madpack/madpack.py
index 6524a3de..7c91911b 100755
--- a/src/madpack/madpack.py
+++ b/src/madpack/madpack.py
@@ -188,6 +188,18 @@ def _run_m4_and_append(schema, maddir_mod_py, module,
sqlfile,
'-DMODULE_NAME=' + module,
'-I' + maddir_madpack,
sqlfile]
+if (((portid == 'postgres') & (dbver == '15'))):
+m4args = ['m4',
+ '-P',
+ '-DMADLIB_SCHEMA=' + schema,
+ '-DPLPYTHON_LIBDIR=' + maddir_mod_py,
+ '-DEXT_PYTHON_LIBDIR=' + maddir_ext_py,
+ '-DMODULE_PATHNAME=' + maddir_lib,
+ '-DMADLIB_LIBRARY_PATH=' + madlib_library_path,
+ '-DMODULE_NAME=' + module,
+ '-DIS_PG_15=TRUE',
+ '-I' + maddir_madpack,
+ sqlfile]
info_(this, "> ... parsing: " + " ".join(m4args), verbose)
output_filehandle.flush()
diff --git a/src/ports/postgres/dbconnector/UDF_impl.hpp
b/src/ports/postgres/dbconnector/UDF_impl.hpp
index cb377083..6674a346 100644
--- a/src/ports/postgres/dbconnector/UDF_impl.hpp
+++ b/src/ports/postgres/dbconnector/UDF_impl.hpp
@@ -13,6 +13,9 @@ namespace dbconnector {
namespace postgres {
+#pragma GCC diagnostic push
+#pragma GCC diagnostic ignored "-Wstringop-truncation"
+
#define MADLIB_HANDLE_STANDARD_EXCEPTION(err) \
sqlerrcode = err; \
strncpy(msg, exc.what(), sizeof(msg));
@@ -215,6 +218,7 @@ UDF::call(FunctionCallInfo fcinfo) {
#undef MADLIB_HANDLE_STANDARD_EXCEPTION
+#pragma GCC diagnostic pop
} // namespace postgres
} // namespace dbconnector
diff --git a/src/ports/postgres/modules/bayes/test/gaussian_naive_bayes.sql_i
[madlib] branch madlib2-master updated: Update version number to 2.1.0-dev
This is an automated email from the ASF dual-hosted git repository.
okislal pushed a commit to branch madlib2-master
in repository https://gitbox.apache.org/repos/asf/madlib.git
The following commit(s) were added to refs/heads/madlib2-master by this push:
new 8e5c222b Update version number to 2.1.0-dev
8e5c222b is described below
commit 8e5c222b91056ff5989ab71dbccf046a772339bf
Author: Orhan Kislal
AuthorDate: Wed Jun 28 14:01:02 2023 -0400
Update version number to 2.1.0-dev
---
RELEASE_NOTES | 2 +-
deploy/DEB/postinst | 2 +-
deploy/postflight.sh | 2 +-
pom.xml | 2 +-
src/config/Version.yml| 2 +-
src/ports/postgres/modules/deep_learning/madlib_keras.sql_in | 8
.../postgres/modules/deep_learning/madlib_keras_automl.sql_in | 4 ++--
.../modules/deep_learning/madlib_keras_fit_multiple_model.sql_in | 6 +++---
src/ports/postgres/modules/svm/svm.sql_in | 2 +-
9 files changed, 15 insertions(+), 15 deletions(-)
diff --git a/RELEASE_NOTES b/RELEASE_NOTES
index 7d1c67da..7a26d45d 100644
--- a/RELEASE_NOTES
+++ b/RELEASE_NOTES
@@ -12,7 +12,7 @@ Current list of bugs and issues can be found at
https://issues.apache.org/jira/b
—-
MADlib v2.0.0:
-Release Date: 2023-June-XX
+Release Date: 2023-June-23
NOTE: Upgrading from MADlib 1.X to MADlib 2.X is not supported.
diff --git a/deploy/DEB/postinst b/deploy/DEB/postinst
index 70fd58e9..974cc7d2 100755
--- a/deploy/DEB/postinst
+++ b/deploy/DEB/postinst
@@ -22,7 +22,7 @@
# Source debconf library.
. /usr/share/debconf/confmodule
-MADLIB_VERSION="2.0.0"
+MADLIB_VERSION="2.1.0-dev"
MADLIB_INSTALL_PATH="InstallPathNotFound"
# Fetching configuration from debconf
diff --git a/deploy/postflight.sh b/deploy/postflight.sh
index f2d6f7da..da2df7fd 100755
--- a/deploy/postflight.sh
+++ b/deploy/postflight.sh
@@ -2,7 +2,7 @@
# $0 - Script Path, $1 - Package Path, $2 - Target Location, and $3 - Target
Volume
-MADLIB_VERSION=2.0.0
+MADLIB_VERSION=2.1.0-dev
# Remove existing soft links
find $2/usr/local/madlib/bin -depth -type l -exec rm {} \; 2>/dev/null
diff --git a/pom.xml b/pom.xml
index 7605f011..ed41f2c8 100644
--- a/pom.xml
+++ b/pom.xml
@@ -22,7 +22,7 @@
org.apache.madlib
madlib
- 2.0.0
+ 2.1.0-dev
pom
diff --git a/src/config/Version.yml b/src/config/Version.yml
index 70effe80..be32da7e 100644
--- a/src/config/Version.yml
+++ b/src/config/Version.yml
@@ -1 +1 @@
-version: 2.0.0
+version: 2.1.0-dev
diff --git a/src/ports/postgres/modules/deep_learning/madlib_keras.sql_in
b/src/ports/postgres/modules/deep_learning/madlib_keras.sql_in
index e80f8d24..67cf2722 100644
--- a/src/ports/postgres/modules/deep_learning/madlib_keras.sql_in
+++ b/src/ports/postgres/modules/deep_learning/madlib_keras.sql_in
@@ -1143,7 +1143,7 @@ model_size| 0.7900390625
start_training_time | 2021-02-01 15:58:43.760568
end_training_time | 2021-02-01 15:58:44.470054
metrics_elapsed_time | {0.709463119506836}
-madlib_version| 2.0.0
+madlib_version| 2.1.0-dev
num_classes | {3}
dependent_vartype | {"character varying"}
normalizing_const | 1
@@ -1457,7 +1457,7 @@ model_size| 0.7900390625
start_training_time | 2021-01-29 14:41:16.943861
end_training_time | 2021-01-29 14:41:19.478149
metrics_elapsed_time |
{2.3377411365509,2.42358803749084,2.49885511398315,2.53427410125732}
-madlib_version| 2.0.0
+madlib_version| 2.1.0-dev
num_classes | {3}
dependent_vartype | {"character varying"}
normalizing_const | 1
@@ -1627,7 +1627,7 @@ model_size| 0.7900390625
start_training_time | 2021-01-29 14:42:28.780276
end_training_time | 2021-01-29 14:42:31.177561
metrics_elapsed_time |
{2.24628114700317,2.28473520278931,2.32178020477295,2.35844302177429,2.39726710319519}
-madlib_version| 2.0.0
+madlib_version| 2.1.0-dev
num_classes | {3}
dependent_vartype | {"character varying"}
normalizing_const | 1
@@ -1742,7 +1742,7 @@ model_size| 0.7900390625
start_training_time | 2021-01-29 14:44:51.176983
end_training_time | 2021-01-29 14:44:53.666457
metrics_elapsed_time | {2.48945999145508}
-madlib_version| 2.0.0
+madlib_version| 2.1.0-dev
num_classes | {3}
dependent_vartype | {"character varying"}
normalizing_const | 1
diff --git
a/src/ports/postgres/modules/deep_learni
[madlib] annotated tag rel/v2.0.0 updated (92739b20 -> a79064ac)
This is an automated email from the ASF dual-hosted git repository. okislal pushed a change to annotated tag rel/v2.0.0 in repository https://gitbox.apache.org/repos/asf/madlib.git *** WARNING: tag rel/v2.0.0 was modified! *** from 92739b20 (commit) to a79064ac (tag) tagging 92739b20167808c017c6ac93b857b309a6b80624 (commit) replaces rel/v1.21.0 by Orhan Kislal on Fri Jun 23 17:57:32 2023 -0400 - Log - Apache MADlib 2.0.0 -BEGIN PGP SIGNATURE- iQEzBAABCAAdFiEEGocCIKIuO+T6G3jEyiWcTeuzJCIFAmSWFU0ACgkQyiWcTeuz JCLiegf+IXTEoMK0OQqUyqNSocn5Hnqn8bsWOLOrgc8cVoS4dwni3PkyIRp02fYY puKVL6omCUDGigK2KU7GxKWEwKkTQU1gNAuC5Z0BG36lSdU7VU7iPl7BPyLmpcMt cEq0Ln6vkXXbn0D7yea5yPBvV6C9TAQBmpCTF25hpFHpjQRTJibwL/DUJd+kJp9M zOTVmmbkCu554CMPVbrMbfQQXMzruXiQqx33C34IKS+jBLC3sowhHSq/tY4CxAhU hVEINPTpAuZzJArBGWPSCY7liIi+QzjSP2tuJG0yfo7Sa7PxXSC5Ny0dbX1fArTL Bkt7S1ZzuSfB02xGDwA19becvKMYgA== =OV9G -END PGP SIGNATURE- --- No new revisions were added by this update. Summary of changes:
[madlib] branch madlib2-master updated: Update version number and add release notes
This is an automated email from the ASF dual-hosted git repository.
okislal pushed a commit to branch madlib2-master
in repository https://gitbox.apache.org/repos/asf/madlib.git
The following commit(s) were added to refs/heads/madlib2-master by this push:
new 92739b20 Update version number and add release notes
92739b20 is described below
commit 92739b20167808c017c6ac93b857b309a6b80624
Author: Orhan Kislal
AuthorDate: Fri Jun 9 10:40:03 2023 -0400
Update version number and add release notes
---
RELEASE_NOTES | 18 +-
deploy/DEB/postinst| 2 +-
deploy/postflight.sh | 2 +-
pom.xml| 2 +-
src/config/Version.yml | 2 +-
.../postgres/modules/deep_learning/madlib_keras.sql_in | 8
.../modules/deep_learning/madlib_keras_automl.sql_in | 4 ++--
.../madlib_keras_fit_multiple_model.sql_in | 6 +++---
src/ports/postgres/modules/svm/svm.sql_in | 2 +-
9 files changed, 31 insertions(+), 15 deletions(-)
diff --git a/RELEASE_NOTES b/RELEASE_NOTES
index 50f0f94e..7d1c67da 100644
--- a/RELEASE_NOTES
+++ b/RELEASE_NOTES
@@ -9,10 +9,26 @@ commit history located at
https://github.com/apache/madlib/commits/master.
Current list of bugs and issues can be found at
https://issues.apache.org/jira/browse/MADLIB.
+—-
+MADlib v2.0.0:
+
+Release Date: 2023-June-XX
+
+NOTE: Upgrading from MADlib 1.X to MADlib 2.X is not supported.
+
+New features:
+- Build: Add support for python3
+- Build: Add support for GP7 Beta, GP6 python3 extension, Postgres 13/14/15
+
+Improvements:
+- XGBoost: Add support for version 1.7.5
+- DL: Add support for tensorflow 2.10.1 and keras 2.10.0
+- DBScan: Add support for rtree 1.0.1
+
—-
MADlib v1.21.0:
-Release Date: 2023-March-XX
+Release Date: 2023-March-01
New features:
- Graph: Add warm start for weakly connected components
diff --git a/deploy/DEB/postinst b/deploy/DEB/postinst
index 8ffce914..70fd58e9 100755
--- a/deploy/DEB/postinst
+++ b/deploy/DEB/postinst
@@ -22,7 +22,7 @@
# Source debconf library.
. /usr/share/debconf/confmodule
-MADLIB_VERSION="2.0.0-dev"
+MADLIB_VERSION="2.0.0"
MADLIB_INSTALL_PATH="InstallPathNotFound"
# Fetching configuration from debconf
diff --git a/deploy/postflight.sh b/deploy/postflight.sh
index dea1fd40..f2d6f7da 100755
--- a/deploy/postflight.sh
+++ b/deploy/postflight.sh
@@ -2,7 +2,7 @@
# $0 - Script Path, $1 - Package Path, $2 - Target Location, and $3 - Target
Volume
-MADLIB_VERSION=2.0.0-dev
+MADLIB_VERSION=2.0.0
# Remove existing soft links
find $2/usr/local/madlib/bin -depth -type l -exec rm {} \; 2>/dev/null
diff --git a/pom.xml b/pom.xml
index dc24966e..7605f011 100644
--- a/pom.xml
+++ b/pom.xml
@@ -22,7 +22,7 @@
org.apache.madlib
madlib
- 2.0.0-dev
+ 2.0.0
pom
diff --git a/src/config/Version.yml b/src/config/Version.yml
index 9d51a0ae..70effe80 100644
--- a/src/config/Version.yml
+++ b/src/config/Version.yml
@@ -1 +1 @@
-version: 2.0.0-dev
+version: 2.0.0
diff --git a/src/ports/postgres/modules/deep_learning/madlib_keras.sql_in
b/src/ports/postgres/modules/deep_learning/madlib_keras.sql_in
index 57f96616..e80f8d24 100644
--- a/src/ports/postgres/modules/deep_learning/madlib_keras.sql_in
+++ b/src/ports/postgres/modules/deep_learning/madlib_keras.sql_in
@@ -1143,7 +1143,7 @@ model_size| 0.7900390625
start_training_time | 2021-02-01 15:58:43.760568
end_training_time | 2021-02-01 15:58:44.470054
metrics_elapsed_time | {0.709463119506836}
-madlib_version| 2.0.0-dev
+madlib_version| 2.0.0
num_classes | {3}
dependent_vartype | {"character varying"}
normalizing_const | 1
@@ -1457,7 +1457,7 @@ model_size| 0.7900390625
start_training_time | 2021-01-29 14:41:16.943861
end_training_time | 2021-01-29 14:41:19.478149
metrics_elapsed_time |
{2.3377411365509,2.42358803749084,2.49885511398315,2.53427410125732}
-madlib_version| 2.0.0-dev
+madlib_version| 2.0.0
num_classes | {3}
dependent_vartype | {"character varying"}
normalizing_const | 1
@@ -1627,7 +1627,7 @@ model_size| 0.7900390625
start_training_time | 2021-01-29 14:42:28.780276
end_training_time | 2021-01-29 14:42:31.177561
metrics_elapsed_time |
{2.24628114700317,2.28473520278931,2.32178020477295,2.35844302177429,2.39726710319519}
-madlib_version| 2.0.0-dev
+madlib_version| 2.0.0
num_classes | {3}
dependent_varty
[madlib] branch madlib2-master updated (769188e9 -> 70fac35b)
This is an automated email from the ASF dual-hosted git repository.
okislal pushed a change to branch madlib2-master
in repository https://gitbox.apache.org/repos/asf/madlib.git
from 769188e9 Build: Add PG13 on Ubuntu 20 support for Jenkins
new 2e5a7ba0 Build: Add gppkg v2 support for GP7
new 70fac35b Bayes: Disable ORCA before accessing views
The 2 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
.../{gppkg_spec.yml.in => gppkg_spec_v2.yml.in}| 29 +-
src/ports/greenplum/cmake/FindGreenplum_6.cmake| 61 ++
src/ports/greenplum/cmake/GreenplumUtils.cmake | 13 -
src/ports/postgres/modules/bayes/test/bayes.sql_in | 16 +-
.../modules/bayes/test/gaussian_naive_bayes.sql_in | 10
5 files changed, 125 insertions(+), 4 deletions(-)
copy deploy/gppkg/{gppkg_spec.yml.in => gppkg_spec_v2.yml.in} (50%)
[madlib] 01/02: Build: Add gppkg v2 support for GP7
This is an automated email from the ASF dual-hosted git repository.
okislal pushed a commit to branch madlib2-master
in repository https://gitbox.apache.org/repos/asf/madlib.git
commit 2e5a7ba01a03a27f5a8bbe274642e58c0ac194b7
Author: Orhan Kislal
AuthorDate: Tue May 16 18:09:58 2023 -0400
Build: Add gppkg v2 support for GP7
This commit updates the packaging process to use the new gppkg_v2 for
GP7. Currently we unpack the existing RPM to create it but at some point
we should overhaul the system to actually collect the files and don't
even create the intermediate RPM.
---
deploy/gppkg/gppkg_spec_v2.yml.in | 46 +++
src/ports/greenplum/cmake/FindGreenplum_6.cmake | 61 +
src/ports/greenplum/cmake/GreenplumUtils.cmake | 13 +-
3 files changed, 119 insertions(+), 1 deletion(-)
diff --git a/deploy/gppkg/gppkg_spec_v2.yml.in
b/deploy/gppkg/gppkg_spec_v2.yml.in
new file mode 100644
index ..268d131c
--- /dev/null
+++ b/deploy/gppkg/gppkg_spec_v2.yml.in
@@ -0,0 +1,46 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+
+# http://www.apache.org/licenses/LICENSE-2.0
+
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+Format: V2
+Pkgname: madlib
+Architecture: @CPACK_RPM_PACKAGE_ARCHITECTURE@
+Version:
@MADLIB_VERSION_STRING_NO_HYPHEN@@GPPKG_VER@-@GPDB_VARIANT_SHORT@@GPDB_VERSION_LC@
+OS: rhel@RH_MAJOR_VERSION@
+GPDBVersion: @GPDB_VERSION_LC@
+Description: Madlib is an open source library which provides scalable
in-database analytics. It provides data-parallel implementations of
mathematical, statistical and machine learning methods for structured and
unstructured data.
+PostInstall:
+ Coordinator: "echo 'Please run the following command to deploy MADlib';
+ echo 'usage: madpack install [-s schema_name] -p @PORT_NAME@ -c
user@host:port/database';
+ echo 'Example:';
+ echo ' $ $GPHOME/madlib/bin/madpack install -s madlib -p
@PORT_NAME@ -c gpadmin@mdw:5432/testdb';
+ echo ' This will install MADlib objects into a @GPDB_VARIANT@
database named \"testdb\"';
+ echo ' running on server \"mdw\" on port 5432. Installer will
try to login as \"gpadmin\"';
+ echo ' and will prompt for password. The target schema will
be \"madlib\".';
+ echo ' To upgrade to a new version of MADlib from version
v@UPGRADE_SUPPORT@ or later, use option \"upgrade\",';
+ echo ' instead of \"install\" ';
+ echo 'For additional options run:';
+ echo '$ madpack --help';
+ echo 'Release notes and additional documentation can be found at
http://madlib.apache.org';"
+ All:
+# Remove existing soft links
+find $GPHOME/madlib/bin -depth -type l -exec rm {} \; 2>/dev/null;
+find $GPHOME/madlib/doc -depth -type l -exec rm {} \; 2>/dev/null;
+find $GPHOME/madlib/Current -depth -type l -exec rm {} \; 2>/dev/null;
+
+ln -nsf $GPHOME/madlib/Versions/@MADLIB_VERSION_STRING@
$GPHOME/madlib/Current;
+ln -nsf $GPHOME/madlib/Current/bin $GPHOME/madlib/bin;
+ln -nsf $GPHOME/madlib/Current/doc $GPHOME/madlib/doc;
diff --git a/src/ports/greenplum/cmake/FindGreenplum_6.cmake
b/src/ports/greenplum/cmake/FindGreenplum_6.cmake
index c5ea28f6..19ff7be7 100644
--- a/src/ports/greenplum/cmake/FindGreenplum_6.cmake
+++ b/src/ports/greenplum/cmake/FindGreenplum_6.cmake
@@ -17,3 +17,64 @@
set(_FIND_PACKAGE_FILE "${CMAKE_CURRENT_LIST_FILE}")
include("${CMAKE_CURRENT_LIST_DIR}/FindGreenplum.cmake")
+
+function(add_gppkg GPDB_VERSION GPDB_VARIANT GPDB_VARIANT_SHORT
UPGRADE_SUPPORT)
+string(TOLOWER ${GPDB_VERSION} GPDB_VERSION_LC)
+string(REPLACE "." "_" VERSION_ "${GPDB_VERSION}")
+
+# Get information about the rhel version
+rh_version(RH_VERSION)
+string(REGEX MATCH "([0-9])" RH_MAJOR_VERSION "${RH_VERSION}")
+
+file(WRITE
"${CMAKE_BINARY_DIR}/deploy/gppkg/${GPDB_VARIANT}_${VERSION_}_gppkg.cmake" "
+file(MAKE_DIRECTORY
+\"\${CMAKE_CURRENT_BINARY_DIR}/${GPDB_VERSION}/BUILD\"
+\"\${CMAKE_CURRENT_BINARY_DIR}
[madlib] 02/02: Bayes: Disable ORCA before accessing views
This is an automated email from the ASF dual-hosted git repository. okislal pushed a commit to branch madlib2-master in repository https://gitbox.apache.org/repos/asf/madlib.git commit 70fac35b0158b35017ae93f8abafda62bbb3c224 Author: Orhan Kislal AuthorDate: Fri May 19 09:08:33 2023 -0400 Bayes: Disable ORCA before accessing views Starting with GP7 Beta3, accesing bayes created views started crashing the db with ORCA. This commit disabled the optimizer with a FIXME comment to investigate in the future. --- src/ports/postgres/modules/bayes/test/bayes.sql_in | 16 ++-- .../modules/bayes/test/gaussian_naive_bayes.sql_in | 10 ++ 2 files changed, 24 insertions(+), 2 deletions(-) diff --git a/src/ports/postgres/modules/bayes/test/bayes.sql_in b/src/ports/postgres/modules/bayes/test/bayes.sql_in index 88604a6b..e04852a7 100644 --- a/src/ports/postgres/modules/bayes/test/bayes.sql_in +++ b/src/ports/postgres/modules/bayes/test/bayes.sql_in @@ -392,5 +392,17 @@ $$ language plpgsql; SELECT install_test_1(); SELECT install_test_2(); SELECT install_test_3(); -SELECT install_test_4(); -SELECT install_test_5(); + +-- FIXME: ORCA started to fail with GP7 beta3 while accessing bayes created views in these tests +m4_include(`SQLCommon.m4') +m4_changequote(`') +m4_ifdef(, , ) + + SELECT install_test_4(); + SELECT install_test_5(); + +m4_ifdef(, , ) diff --git a/src/ports/postgres/modules/bayes/test/gaussian_naive_bayes.sql_in b/src/ports/postgres/modules/bayes/test/gaussian_naive_bayes.sql_in index ba506ca0..26c874f7 100644 --- a/src/ports/postgres/modules/bayes/test/gaussian_naive_bayes.sql_in +++ b/src/ports/postgres/modules/bayes/test/gaussian_naive_bayes.sql_in @@ -216,6 +216,13 @@ SELECT create_nb_probs_view( 'iris_probs' ); +-- FIXME: ORCA started to fail with GP7 beta3 while accessing bayes created views in these tests +m4_include(`SQLCommon.m4') +m4_changequote(`') +m4_ifdef(, , + SELECT assert(count(*) = 10, 'Gaussian Naive Bayes produces wrong classes!') FROM iris_clasif, iris_test WHERE key = id @@ -229,3 +236,6 @@ FROM iris_probs WHERE key = 87 AND class = 2; +m4_ifdef(, , )
[madlib] branch madlib2-master updated: Build: Add PG13 on Ubuntu 20 support for Jenkins
This is an automated email from the ASF dual-hosted git repository. okislal pushed a commit to branch madlib2-master in repository https://gitbox.apache.org/repos/asf/madlib.git The following commit(s) were added to refs/heads/madlib2-master by this push: new 769188e9 Build: Add PG13 on Ubuntu 20 support for Jenkins 769188e9 is described below commit 769188e927e9e5564b4aa9bfd5e77bcf36dda16f Author: Orhan Kislal AuthorDate: Wed May 10 14:41:48 2023 -0400 Build: Add PG13 on Ubuntu 20 support for Jenkins This commit fixes the Jenkins pipeline for PG13. It disables some tests (with FIXME tag) since they had docker specific memory issues. PMML is also fixed for py3. Finally, it adds PG14 and PG15 support for future convenience. --- src/madpack/create_changelist.py | 2 +- src/madpack/madpack.py | 2 +- src/madpack/upgrade_util.py| 5 +- src/madpack/utilities.py | 2 +- src/ports/postgres/14/CMakeLists.txt | 22 src/ports/postgres/15/CMakeLists.txt | 22 src/ports/postgres/cmake/FindPostgreSQL_14.cmake | 21 src/ports/postgres/cmake/FindPostgreSQL_15.cmake | 21 src/ports/postgres/modules/pmml/pmml_builder.py_in | 13 ++--- tool/docker/base/Dockerfile_postgres_11_Jenkins| 13 ++--- ...s_13_Jenkins => Dockerfile_postgres_12_Jenkins} | 16 +++--- tool/docker/base/Dockerfile_postgres_13_Jenkins| 51 -- tool/docker/base/Dockerfile_postgres_14_Jenkins| 61 ++ tool/docker/base/Dockerfile_postgres_15_Jenkins| 61 ++ tool/jenkins/jenkins_build.sh | 35 + tool/pg_hba.conf.postgres | 30 +++ 16 files changed, 331 insertions(+), 46 deletions(-) diff --git a/src/madpack/create_changelist.py b/src/madpack/create_changelist.py index d5a54920..6b12ac22 100644 --- a/src/madpack/create_changelist.py +++ b/src/madpack/create_changelist.py @@ -1,4 +1,4 @@ -#!/usr/bin/python +#!/usr/bin/python3 # -- # Licensed to the Apache Software Foundation (ASF) under one # or more contributor license agreements. See the NOTICE file diff --git a/src/madpack/madpack.py b/src/madpack/madpack.py index eb0619de..6524a3de 100755 --- a/src/madpack/madpack.py +++ b/src/madpack/madpack.py @@ -1,4 +1,4 @@ -#!/usr/bin/env python +#!/usr/bin/env python3 # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # Main Madpack installation executable. # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # diff --git a/src/madpack/upgrade_util.py b/src/madpack/upgrade_util.py index f00d174c..f9bbb7b9 100644 --- a/src/madpack/upgrade_util.py +++ b/src/madpack/upgrade_util.py @@ -1,5 +1,8 @@ from collections import defaultdict -from collections import Iterable +try: +from collections.abc import Iterable +except ImportError: +from collections import Iterable import glob import os import re diff --git a/src/madpack/utilities.py b/src/madpack/utilities.py index 6e9be610..053dd112 100644 --- a/src/madpack/utilities.py +++ b/src/madpack/utilities.py @@ -1,4 +1,4 @@ -#!/usr/bin/env python# +#!/usr/bin/env python3 # # Licensed to the Apache Software Foundation (ASF) under one # or more contributor license agreements. See the NOTICE file diff --git a/src/ports/postgres/14/CMakeLists.txt b/src/ports/postgres/14/CMakeLists.txt new file mode 100644 index ..2d44a6f3 --- /dev/null +++ b/src/ports/postgres/14/CMakeLists.txt @@ -0,0 +1,22 @@ +# -- +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. +# -- + +add_current_postgresql_version() +add_extension_support() + diff --git a/src/ports/postgres/15/CMakeLists.txt b/src/ports/postgres/15/CMakeLists.txt new file mode 100644 index ..2d44a6f3 --- /dev/null +++ b/src/ports/postgres/15/CMake
[madlib-site] branch asf-site updated (4995b51 -> 0b8d0f0)
This is an automated email from the ASF dual-hosted git repository. okislal pushed a change to branch asf-site in repository https://gitbox.apache.org/repos/asf/madlib-site.git from 4995b51 Remove html5lightbox.js external resource references. add 644a4c8 WCC: Update docs with warm start example new 0b8d0f0 Merge pull request #27 from apache/update-wcc-docs The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: docs/v1.21.0/group__grp__wcc.html | 35 ++- 1 file changed, 34 insertions(+), 1 deletion(-)
[madlib-site] 01/01: Merge pull request #27 from apache/update-wcc-docs
This is an automated email from the ASF dual-hosted git repository. okislal pushed a commit to branch asf-site in repository https://gitbox.apache.org/repos/asf/madlib-site.git commit 0b8d0f085916a7e4e2cdcc01247e02dcd46de2f6 Merge: 4995b51 644a4c8 Author: Orhan Kislal AuthorDate: Wed Apr 19 18:20:06 2023 -0400 Merge pull request #27 from apache/update-wcc-docs WCC: Update docs with warm start example docs/v1.21.0/group__grp__wcc.html | 35 ++- 1 file changed, 34 insertions(+), 1 deletion(-)
[madlib] branch master updated: WCC: Update docs with warm start example
This is an automated email from the ASF dual-hosted git repository. okislal pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/madlib.git The following commit(s) were added to refs/heads/master by this push: new d9a4ee58 WCC: Update docs with warm start example d9a4ee58 is described below commit d9a4ee58ad40597ec346c442e80bc8e6c95bffe2 Author: Orhan Kislal AuthorDate: Wed Apr 19 17:18:21 2023 -0400 WCC: Update docs with warm start example --- src/ports/postgres/modules/graph/wcc.sql_in | 36 + 1 file changed, 36 insertions(+) diff --git a/src/ports/postgres/modules/graph/wcc.sql_in b/src/ports/postgres/modules/graph/wcc.sql_in index 594b74a4..699d348b 100644 --- a/src/ports/postgres/modules/graph/wcc.sql_in +++ b/src/ports/postgres/modules/graph/wcc.sql_in @@ -53,6 +53,8 @@ ignoring the direction of edges. In case of an undirected graph, a weakly connected component is also a strongly connected component. This module also includes a number of helper functions that operate on the WCC output. +@note MADlib graph functions might create a large number of subtransactions with very large graphs. Since the cache for subtransaction metadata is limited, running them in conjunction with other processes might degrade the performance of the whole database. The warm start feature of WCC can be used to limit the number of subtransactions in a given session to ensure that the performance is preserved. While the ideal iteration limit is dependent on the graph as well as the rest of the syst [...] + @anchor wcc @par Weakly Connected Components @@ -598,6 +600,40 @@ SELECT * FROM wcc_multicol_out ORDER BY user_id_major, user_id_minor, component_ (14 rows) +-# Use iteration limit and warm start in a bash loop to avoid subtx limitations + +#!/bin/bash +psql madlib -c "DROP TABLE IF EXISTS wcc_out, wcc_out_summary, wcc_out_message;" +\# Run wcc for 2 iterations +psql madlib -c "SELECT madlib.weakly_connected_components( +'vertex', +'node_id', +'edge', +'src=conn_src,dest=conn_dest', +'wcc_out', +'user_id', +2);" # Number of iterations +\# Check if wcc is done +mynode=$(psql madlib -tc "SELECT nodes_to_update FROM wcc_out_summary") +echo $mynode +\# While there are remaining nodes to update +while [ $mynode -ne 0 ] +do +\# Run WCC with warm start to continue building on the previous output tables +psql madlib -c "SELECT madlib.weakly_connected_components( +'vertex', +'node_id', +'edge', +'src=conn_src,dest=conn_dest', +'wcc_out', +'user_id', +2, +True);" # Warm start +mynode=$(psql madlib -tc "SELECT nodes_to_update FROM wcc_out_summary") +echo $mynode +done + + @anchor notes @par Notes
[madlib-site] branch update-wcc-docs updated: WCC: Update docs with warm start example
This is an automated email from the ASF dual-hosted git repository.
okislal pushed a commit to branch update-wcc-docs
in repository https://gitbox.apache.org/repos/asf/madlib-site.git
The following commit(s) were added to refs/heads/update-wcc-docs by this push:
new 644a4c8 WCC: Update docs with warm start example
644a4c8 is described below
commit 644a4c8d19cb5d6318de0fd7c6824da66f6fd01b
Author: Orhan Kislal
AuthorDate: Wed Apr 19 17:22:18 2023 -0400
WCC: Update docs with warm start example
---
docs/v1.21.0/group__grp__wcc.html | 35 ++-
1 file changed, 34 insertions(+), 1 deletion(-)
diff --git a/docs/v1.21.0/group__grp__wcc.html
b/docs/v1.21.0/group__grp__wcc.html
index d4fc0ac..5ff8021 100644
--- a/docs/v1.21.0/group__grp__wcc.html
+++ b/docs/v1.21.0/group__grp__wcc.html
@@ -133,6 +133,7 @@
$(document).ready(function(){initNavTree('group__grp__wcc.html','');});
Notes
Given a directed graph, a weakly connected component (WCC) is a
subgraph of the original graph where all vertices are connected to each other
by some path, ignoring the direction of edges. In case of an undirected graph,
a weakly connected component is also a strongly connected component. This
module also includes a number of helper functions that operate on the WCC
output.
+NoteMADlib graph functions might create
a large number of subtransactions with very large graphs. Since the cache for
subtransaction metadata is limited, running them in conjunction with other
processes might degrade the performance of the whole database. The warm start
feature of WCC can be used to limit the number of subtransactions in a given
session to ensure that the performance is preserved. While the ideal iteration
limit is dependent on the g [...]
Weakly
Connected Components
weakly_connected_components( vertex_table,
vertex_id,
@@ -544,6 +545,38 @@ SELECT * FROM wcc_multicol_out ORDER BY user_id_major,
user_id_minor, component_
{6,6} |3 | 2 | 2
(14 rows)
+Use iteration limit and warm start in a bash loop to avoid subtx
limitations
+#!/bin/bash
+psql madlib -c "DROP TABLE IF EXISTS wcc_out, wcc_out_summary,
wcc_out_message;"
+# Run wcc for 2 iterations
+psql madlib -c "SELECT madlib.weakly_connected_components(
+'vertex',
+'node_id',
+'edge',
+'src=conn_src,dest=conn_dest',
+'wcc_out',
+'user_id',
+2);" # Number of iterations
+# Check if wcc is done
+mynode=$(psql madlib -tc "SELECT nodes_to_update FROM wcc_out_summary")
+echo $mynode
+# While there are remaining nodes to update
+while [ $mynode -ne 0 ]
+do
+# Run WCC with warm start to continue building on the previous output tables
+psql madlib -c "SELECT madlib.weakly_connected_components(
+'vertex',
+'node_id',
+'edge',
+'src=conn_src,dest=conn_dest',
+'wcc_out',
+'user_id',
+2,
+True);" # Warm start
+mynode=$(psql madlib -tc "SELECT nodes_to_update FROM wcc_out_summary")
+echo $mynode
+done
+
Notes
@@ -554,7 +587,7 @@ SELECT * FROM wcc_multicol_out ORDER BY user_id_major,
user_id_minor, component_
-Generated on Thu Feb 23 2023 19:26:40 for MADlib by
+Generated on Wed Apr 19 2023 21:17:52 for MADlib by
http://www.doxygen.org/index.html;>
1.8.13
[madlib-site] branch update-wcc-docs created (now 4995b51)
This is an automated email from the ASF dual-hosted git repository. okislal pushed a change to branch update-wcc-docs in repository https://gitbox.apache.org/repos/asf/madlib-site.git at 4995b51 Remove html5lightbox.js external resource references. No new revisions were added by this update.
[madlib] 06/08: Add Postgres 13 with python3 support
This is an automated email from the ASF dual-hosted git repository.
okislal pushed a commit to branch madlib2-master
in repository https://gitbox.apache.org/repos/asf/madlib.git
commit 8ce0eef32979cd8a6d3b1137aad9095b56b0c134
Author: Orhan Kislal
AuthorDate: Fri Mar 17 19:14:19 2023 +0300
Add Postgres 13 with python3 support
---
methods/array_ops/src/pg_gp/test/array_ops.sql_in | 3 ++-
methods/svec_util/src/pg_gp/sql/svec_test.sql_in| 3 ++-
src/madpack/madpack.py | 9 +++--
src/madpack/upgrade_util.py | 3 ++-
src/ports/postgres/CMakeLists.txt | 9 -
src/ports/postgres/cmake/FindPostgreSQL_13.cmake| 21 +
.../postgres/dbconnector/SystemInformation_impl.hpp | 4 ++--
src/ports/postgres/dbconnector/TypeTraits_impl.hpp | 10 ++
.../postgres/modules/mxgboost/madlib_xgboost.py_in | 2 +-
.../recursive_partitioning/decision_tree.py_in | 2 +-
.../test/decision_tree.sql_in | 14 --
.../test/random_forest.sql_in | 6 --
.../modules/utilities/create_indicators.py_in | 2 +-
.../modules/utilities/in_mem_group_control.py_in| 2 +-
.../postgres/modules/utilities/utilities.py_in | 2 +-
.../postgres/modules/utilities/validate_args.py_in | 2 +-
16 files changed, 60 insertions(+), 34 deletions(-)
diff --git a/methods/array_ops/src/pg_gp/test/array_ops.sql_in
b/methods/array_ops/src/pg_gp/test/array_ops.sql_in
index 511564f7..419b22f4 100644
--- a/methods/array_ops/src/pg_gp/test/array_ops.sql_in
+++ b/methods/array_ops/src/pg_gp/test/array_ops.sql_in
@@ -18,13 +18,14 @@ SELECT
'{1,2,3}'::float8[] AS an,
'{4,5,7}'::float8[] AS b;
+-- use MADLIB_SCHEMA.normalize to avoid conflicts with PG13 catalog normalize
SELECT array_dot(
array_mult(
array_add(an,b),
array_sub(an,b)),
array_mult(
array_div(an,b),
-normalize(an))) AS result1
+MADLIB_SCHEMA.normalize(an))) AS result1
FROM data;
SELECT array_max(b) FROM data;
diff --git a/methods/svec_util/src/pg_gp/sql/svec_test.sql_in
b/methods/svec_util/src/pg_gp/sql/svec_test.sql_in
index e8e6f804..9301ea1e 100644
--- a/methods/svec_util/src/pg_gp/sql/svec_test.sql_in
+++ b/methods/svec_util/src/pg_gp/sql/svec_test.sql_in
@@ -169,7 +169,8 @@ select angle(result1, result2) from svec_svec;
select tanimoto_distance(result1, result2) from svec_svec;
-- Calculate normalized vectors
-select normalize(result) from corpus_proj;
+-- use MADLIB_SCHEMA.normalize to avoid conflicts with PG13 catalog normalize
+select MADLIB_SCHEMA.normalize(result) from corpus_proj;
-- Test the pivot operator
create table pivot_test(a float8);
diff --git a/src/madpack/madpack.py b/src/madpack/madpack.py
index f6743483..b2b6dd3e 100755
--- a/src/madpack/madpack.py
+++ b/src/madpack/madpack.py
@@ -27,6 +27,11 @@ from utilities import run_query
# Required Python version
py_min_ver = [2, 6]
+# raw_input isn't defined in Python3.x, whereas input wasn't behaving like
raw_input in Python 2.x
+# this should make both input and raw_input work in Python 2.x/3.x like the
raw_input from Python 2.x
+try: input = raw_input
+except NameError: raw_input = input
+
# Find MADlib root directory. This file is installed to
# $MADLIB_ROOT/madpack/madpack.py, so to get $MADLIB_ROOT we need to go
# two levels up in the directory hierarchy. We use (a) os.path.realpath and
@@ -,9 +1116,9 @@ def _append_uninstall_madlib_sqlfile(schema,
db_madlib_ver, is_schema_in_db,
ao['column'] + ' : ' + ao['type'], True)
info_(this,
"***",
True)
info_(this, "Would you like to continue? [Y/N]", True)
-go = input('>>> ').upper()
+go = raw_input('>>> ').upper()
while (go not in ('Y', 'N', 'YES', 'NO')):
-go = input('Yes or No >>> ').upper()
+go = raw_input('Yes or No >>> ').upper()
# 2) Do the uninstall/drop
if go in ('N', 'NO'):
diff --git a/src/madpack/upgrade_util.py b/src/madpack/upgrade_util.py
index 4f971f5d..f00d174c 100644
--- a/src/madpack/upgrade_util.py
+++ b/src/madpack/upgrade_util.py
@@ -1,4 +1,5 @@
-from collections import defaultdict, Iterable
+from collections import defaultdict
+from collections import Iterable
import glob
import os
import re
diff --git a/src/ports/postgres/CMakeLists.txt
b/src/ports/postgres/CMakeLists.txt
index 22941b4c..5f7f966d 100644
--- a/src/ports/postgres/CMakeLists.txt
+++ b/src/ports/postgres/CMakeLists.txt
@@ -207,11 +207,10 @@ function(add_${PORT_LC}_library IN_PORT_VERSION)
# END Legacy Code
-# TODO py3
-#configure_file("${PORT_SOURCE_DIR}/madpack/SQLCommon.m4_in"
-#"${CMAKE_CURRENT_BINARY_DIR}/madpack/SQLCommon.m4"
-#@
[madlib] 05/08: Remove redundant cmake files
This is an automated email from the ASF dual-hosted git repository. okislal pushed a commit to branch madlib2-master in repository https://gitbox.apache.org/repos/asf/madlib.git commit 7e6d0563fd83a74cfba31b5d878fa1dd2265e57b Author: Orhan Kislal AuthorDate: Fri Mar 17 19:11:51 2023 +0300 Remove redundant cmake files --- cmake/FindPackageHandleStandardArgs.cmake | 606 -- cmake/FindPackageMessage.cmake| 49 --- cmake/FindPostgreSQL_13.cmake | 318 cmake/SelectLibraryConfigurations.cmake | 81 4 files changed, 1054 deletions(-) diff --git a/cmake/FindPackageHandleStandardArgs.cmake b/cmake/FindPackageHandleStandardArgs.cmake deleted file mode 100644 index 54129d64.. --- a/cmake/FindPackageHandleStandardArgs.cmake +++ /dev/null @@ -1,606 +0,0 @@ -# Distributed under the OSI-approved BSD 3-Clause License. See accompanying -# file Copyright.txt or https://cmake.org/licensing for details. - -#[===[.rst: -FindPackageHandleStandardArgs -- - -This module provides functions intended to be used in :ref:`Find Modules` -implementing :command:`find_package()` calls. - -.. command:: find_package_handle_standard_args - - This command handles the ``REQUIRED``, ``QUIET`` and version-related - arguments of :command:`find_package`. It also sets the - ``_FOUND`` variable. The package is considered found if all - variables listed contain valid results, e.g. valid filepaths. - - There are two signatures: - - .. code-block:: cmake - -find_package_handle_standard_args( - (DEFAULT_MSG|) - ... - ) - -find_package_handle_standard_args( - [FOUND_VAR ] - [REQUIRED_VARS ...] - [VERSION_VAR ] - [HANDLE_VERSION_RANGE] - [HANDLE_COMPONENTS] - [CONFIG_MODE] - [NAME_MISMATCHED] - [REASON_FAILURE_MESSAGE ] - [FAIL_MESSAGE ] - ) - - The ``_FOUND`` variable will be set to ``TRUE`` if all - the variables ``...`` are valid and any optional - constraints are satisfied, and ``FALSE`` otherwise. A success or - failure message may be displayed based on the results and on - whether the ``REQUIRED`` and/or ``QUIET`` option was given to - the :command:`find_package` call. - - The options are: - - ``(DEFAULT_MSG|)`` -In the simple signature this specifies the failure message. -Use ``DEFAULT_MSG`` to ask for a default message to be computed -(recommended). Not valid in the full signature. - - ``FOUND_VAR `` -.. deprecated:: 3.3 - -Specifies either ``_FOUND`` or -``_FOUND`` as the result variable. This exists only -for compatibility with older versions of CMake and is now ignored. -Result variables of both names are always set for compatibility. - - ``REQUIRED_VARS ...`` -Specify the variables which are required for this package. -These may be named in the generated failure message asking the -user to set the missing variable values. Therefore these should -typically be cache entries such as ``FOO_LIBRARY`` and not output -variables like ``FOO_LIBRARIES``. - -.. versionchanged:: 3.18 - If ``HANDLE_COMPONENTS`` is specified, this option can be omitted. - - ``VERSION_VAR `` -Specify the name of a variable that holds the version of the package -that has been found. This version will be checked against the -(potentially) specified required version given to the -:command:`find_package` call, including its ``EXACT`` option. -The default messages include information about the required -version and the version which has been actually found, both -if the version is ok or not. - - ``HANDLE_VERSION_RANGE`` -.. versionadded:: 3.19 - -Enable handling of a version range, if one is specified. Without this -option, a developer warning will be displayed if a version range is -specified. - - ``HANDLE_COMPONENTS`` -Enable handling of package components. In this case, the command -will report which components have been found and which are missing, -and the ``_FOUND`` variable will be set to ``FALSE`` -if any of the required components (i.e. not the ones listed after -the ``OPTIONAL_COMPONENTS`` option of :command:`find_package`) are -missing. - - ``CONFIG_MODE`` -Specify that the calling find module is a wrapper around a -call to ``find_package( NO_MODULE)``. This implies -a ``VERSION_VAR`` value of ``_VERSION``. The command -will automatically check whether the package configuration file -was found. - - ``REASON_FAILURE_MESSAGE `` -.. versionadded:: 3.16 - -Specify a custom message of the reason for the failure which will be -appended to the default generated message. - - ``FAIL_MESSAGE `` -Specify a custom failure message instead of using the default -generated message. Not recommended. - - ``NAME_MISMATCHED
[madlib] 08/08: Update jenkins to use PG13
This is an automated email from the ASF dual-hosted git repository. okislal pushed a commit to branch madlib2-master in repository https://gitbox.apache.org/repos/asf/madlib.git commit 03fba10fc0d115afcf25c00e1b97ef747676396a Author: Orhan Kislal AuthorDate: Wed Mar 29 23:01:58 2023 +0300 Update jenkins to use PG13 The current jenkins build is failing because of a server crash. Added a FIXME in the build script to look at this problem and find an efficient way to debug in a different JIRA. --- src/bin/madpack | 2 +- tool/docker/base/Dockerfile_postgres_13_Jenkins | 40 + tool/jenkins/jenkins_build.sh | 21 +++-- 3 files changed, 53 insertions(+), 10 deletions(-) diff --git a/src/bin/madpack b/src/bin/madpack index 0507fbc2..66fea155 100755 --- a/src/bin/madpack +++ b/src/bin/madpack @@ -13,7 +13,7 @@ # 2. Pass all arguments to ../madpack/madpack.py PYTHON_PREFIX="python" -PYTHON_VERSIONS="2.7 2.6" +PYTHON_VERSIONS="2.7 2.6 3" # create absolute path to madpack.py pushd `dirname $0` > /dev/null diff --git a/tool/docker/base/Dockerfile_postgres_13_Jenkins b/tool/docker/base/Dockerfile_postgres_13_Jenkins new file mode 100644 index ..59ecd695 --- /dev/null +++ b/tool/docker/base/Dockerfile_postgres_13_Jenkins @@ -0,0 +1,40 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. + +FROM postgres:13 + +### Get postgres specific add-ons +RUN apt-get update && apt-get install -y wget \ + build-essential \ + postgresql-server-dev-13 \ + postgresql-plpython3-13 \ + openssl \ + libssl-dev \ + libboost-all-dev \ + m4 \ + rpm \ + python3-pip \ + python3-dev \ + build-essential \ + cmake \ + libspatialindex-dev + +RUN python3 -m pip install tensorflow dill rtree xgboost + +## To build an image from this docker file, from madlib folder, run: +# docker build -t madlib/postgres_13:jenkins -f tool/docker/base/Dockerfile_postgres_13_Jenkins . diff --git a/tool/jenkins/jenkins_build.sh b/tool/jenkins/jenkins_build.sh index 904a102c..f1df043f 100755 --- a/tool/jenkins/jenkins_build.sh +++ b/tool/jenkins/jenkins_build.sh @@ -20,6 +20,9 @@ workdir=`pwd` user_name=`whoami` +# FIXME: Jenkins build is not working, even though PG13 compiles on local Mac. +# Need to find an efficient way to debug since it has a server crash. + echo "==" echo "Build user: $user_name" echo "Work directory: $workdir" @@ -43,14 +46,14 @@ docker rm madlib echo "Creating docker container" # Pull down the base docker images -echo "docker pull madlib/postgres_11:jenkins" -docker pull madlib/postgres_11:jenkins +echo "docker pull madlib/postgres_13:jenkins" +docker pull madlib/postgres_13:jenkins # Launch docker container with volume mounted from workdir echo "---" cat <
[madlib] 07/08: Address review comments
This is an automated email from the ASF dual-hosted git repository.
okislal pushed a commit to branch madlib2-master
in repository https://gitbox.apache.org/repos/asf/madlib.git
commit 533d5e73f8444009026c9b06908edc69c51af189
Author: Orhan Kislal
AuthorDate: Wed Mar 29 19:12:38 2023 +0300
Address review comments
---
src/madpack/configyml.py | 1 -
src/madpack/madpack.py| 9 -
src/ports/postgres/madpack/SQLCommon.m4_in| 15 ++-
.../modules/dbscan/test/unit_tests/test_dbscan.py_in | 13 +++--
.../modules/deep_learning/madlib_keras_predict.py_in | 2 +-
src/ports/postgres/modules/glm/multinom.py_in | 1 -
src/ports/postgres/modules/linalg/matrix_ops.py_in| 4 +++-
.../recursive_partitioning/test/decision_tree.sql_in | 2 +-
src/ports/postgres/modules/utilities/debug.py_in | 3 +++
9 files changed, 33 insertions(+), 17 deletions(-)
diff --git a/src/madpack/configyml.py b/src/madpack/configyml.py
index 91143ebd..1251b068 100644
--- a/src/madpack/configyml.py
+++ b/src/madpack/configyml.py
@@ -42,7 +42,6 @@ def get_version(configdir):
print("configyml : ERROR : missing or malformed Version.yml")
exit(2)
-# XXX
conf = convert(conf)
try:
conf['version']
diff --git a/src/madpack/madpack.py b/src/madpack/madpack.py
index b2b6dd3e..eb0619de 100755
--- a/src/madpack/madpack.py
+++ b/src/madpack/madpack.py
@@ -25,7 +25,13 @@ from utilities import remove_comments_from_sql
from utilities import run_query
# Required Python version
-py_min_ver = [2, 6]
+py_min_ver = [2, 7]
+
+if list(sys.version_info[:2]) < py_min_ver:
+print("ERROR: python version too old ({0}). You need {1} or greater.".
+ format('.'.join(map(str, sys.version_info[:3])),
+ '.'.join(map(str, py_min_ver
+exit(1)
# raw_input isn't defined in Python3.x, whereas input wasn't behaving like
raw_input in Python 2.x
# this should make both input and raw_input work in Python 2.x/3.x like the
raw_input from Python 2.x
@@ -399,6 +405,7 @@ def _plpy_check(py_min_ver):
error_(this, """Cannot create language plpython3u. Please check if
you
have configured and installed portid (your platform) with
`--with-python` option. Stopping installation...""", False)
+raise Exception
# Check PL/Python version
_internal_run_query("DROP FUNCTION IF EXISTS plpy_version_for_madlib();",
False)
diff --git a/src/ports/postgres/madpack/SQLCommon.m4_in
b/src/ports/postgres/madpack/SQLCommon.m4_in
index 81427b9e..9521d3d2 100644
--- a/src/ports/postgres/madpack/SQLCommon.m4_in
+++ b/src/ports/postgres/madpack/SQLCommon.m4_in
@@ -36,7 +36,20 @@ m4_define(,
[madlib] 04/08: Various fixes: Add gpdb7 specific checks
This is an automated email from the ASF dual-hosted git repository.
okislal pushed a commit to branch madlib2-master
in repository https://gitbox.apache.org/repos/asf/madlib.git
commit 149363a17ce9f3675d617b6abeb861542309d2ea
Author: Orhan Kislal
AuthorDate: Mon Nov 21 16:43:44 2022 +0300
Various fixes: Add gpdb7 specific checks
---
cmake/TestIfNoUTF8BOM.py | 2 +-
deploy/gppkg/madlib.spec.in| 1 +
deploy/madlib.spec.in | 1 +
methods/kmeans/src/pg_gp/kmeans.c | 5 +++
src/madpack/argparse.py| 23 --
src/madpack/madpack.py | 36 +++---
src/madpack/utilities.py | 2 --
src/ports/greenplum/dbconnector/dbconnector.hpp| 3 ++
.../dbconnector/SystemInformation_impl.hpp | 11 +++
src/ports/postgres/dbconnector/dbconnector.hpp | 3 --
src/ports/postgres/modules/dbscan/dbscan.py_in | 2 ++
.../deep_learning/input_data_preprocessor.py_in| 5 +--
.../modules/deep_learning/madlib_keras.py_in | 3 ++
.../madlib_keras_custom_function.py_in | 1 -
.../test/unit_tests/test_madlib_keras.py_in| 8 ++---
src/ports/postgres/modules/graph/wcc.py_in | 4 ---
src/ports/postgres/modules/graph/wcc.sql_in| 4 +--
.../postgres/modules/mxgboost/madlib_xgboost.py_in | 9 +++---
18 files changed, 49 insertions(+), 74 deletions(-)
diff --git a/cmake/TestIfNoUTF8BOM.py b/cmake/TestIfNoUTF8BOM.py
index ce9b7b02..1a2824f7 100755
--- a/cmake/TestIfNoUTF8BOM.py
+++ b/cmake/TestIfNoUTF8BOM.py
@@ -1,4 +1,4 @@
-#!/usr/bin/env python3
+#!/usr/bin/env python
import sys
def detectBOM(inFileName):
diff --git a/deploy/gppkg/madlib.spec.in b/deploy/gppkg/madlib.spec.in
index 78fdb908..b71bf98a 100644
--- a/deploy/gppkg/madlib.spec.in
+++ b/deploy/gppkg/madlib.spec.in
@@ -1,3 +1,4 @@
+%define _build_id_links none
%define _topdir @CMAKE_CURRENT_BINARY_DIR@/@GPDB_VERSION@
%define __os_install_post %{nil}
%define _rpmfilename @MADLIB_GPPKG_RPM_FILE_NAME@
diff --git a/deploy/madlib.spec.in b/deploy/madlib.spec.in
index 8be7c823..a92ebd5f 100644
--- a/deploy/madlib.spec.in
+++ b/deploy/madlib.spec.in
@@ -1,4 +1,5 @@
# -*- rpm-spec -*-
+%define _build_id_links none
%define _rpmdir @CPACK_RPM_DIRECTORY@
%define _rpmfilename @CPACK_RPM_FILE_NAME@
%define _unpackaged_files_terminate_build 0
diff --git a/methods/kmeans/src/pg_gp/kmeans.c
b/methods/kmeans/src/pg_gp/kmeans.c
index d74f65b9..a9d0cc0d 100644
--- a/methods/kmeans/src/pg_gp/kmeans.c
+++ b/methods/kmeans/src/pg_gp/kmeans.c
@@ -86,7 +86,12 @@ compute_metric(PGFunction inMetricFn, MemoryContext
inMemContext, Datum inVec1,
* The 50k bound here is arbitrary, and motivated by ResetExprContext()
* in execUtils.c
*/
+
+#if GP_VERSION_NUM >= 7
+if(inMemContext->mem_allocated > 5)
+#else
if(inMemContext->allBytesAlloc - inMemContext->allBytesFreed > 5)
+#endif
MemoryContextReset(inMemContext);
#else
/* PostgreSQL does not have the allBytesAlloc and allBytesFreed fields */
diff --git a/src/madpack/argparse.py b/src/madpack/argparse.py
index f006fd48..3a812c2a 100644
--- a/src/madpack/argparse.py
+++ b/src/madpack/argparse.py
@@ -90,29 +90,6 @@ import textwrap as _textwrap
from gettext import gettext as _
-try:
-set
-except NameError:
-# for python < 2.4 compatibility (sets module is there since 2.3):
-from sets import Set as set
-
-try:
-str
-except NameError:
-str = str
-
-try:
-sorted
-except NameError:
-# for python < 2.4 compatibility:
-def sorted(iterable, reverse=False):
-result = list(iterable)
-result.sort()
-if reverse:
-result.reverse()
-return result
-
-
def _callable(obj):
return hasattr(obj, '__call__') or hasattr(obj, '__bases__')
diff --git a/src/madpack/madpack.py b/src/madpack/madpack.py
index c662ae89..f6743483 100755
--- a/src/madpack/madpack.py
+++ b/src/madpack/madpack.py
@@ -1,4 +1,4 @@
-#!/usr/bin/env python3
+#!/usr/bin/env python
# # # # # # # # # # # # # # # # # # # # # # # # # # # # # #
# Main Madpack installation executable.
# # # # # # # # # # # # # # # # # # # # # # # # # # # # # #
@@ -27,13 +27,6 @@ from utilities import run_query
# Required Python version
py_min_ver = [2, 6]
-# XXX py3 Check python version
-#if sys.version_info[:2] < py_min_ver:
-#print(("ERROR: python version too old ({0}). You need {1} or greater.".
-# format('.'.join(map(str, sys.version_info[:3])),
-# '.'.join(map(str, py_min_ver)
-#exit(1)
-
# Find MADlib root directory. This file is installed to
# $MADLIB_ROOT/madpack/madpack.py, so to get $MADLIB_ROOT we need to go
# two levels up in the directory hierarchy. We use (a) os.path.realpath and
@@ -55,7 +48,6
[madlib] branch madlib2-master updated (f06638ee -> 03fba10f)
This is an automated email from the ASF dual-hosted git repository.
okislal pushed a change to branch madlib2-master
in repository https://gitbox.apache.org/repos/asf/madlib.git
from f06638ee Update version number to 2.0.0-dev
new 3eaae997 update: fix madlib support py3.
new 76fe872b Add python3 support
new 5442a644 Add python3 support for GPDB6
new 149363a1 Various fixes: Add gpdb7 specific checks
new 7e6d0563 Remove redundant cmake files
new 8ce0eef3 Add Postgres 13 with python3 support
new 533d5e73 Address review comments
new 03fba10f Update jenkins to use PG13
The 8 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
cmake/TestIfNoUTF8BOM.py |7 +-
deploy/gppkg/madlib.spec.in|1 +
deploy/madlib.spec.in |1 +
.../hello_world/iterative/simple_logistic.py_in|2 +-
.../hello_world/iterative/simple_logistic.sql_in |4 +-
methods/array_ops/src/pg_gp/array_ops.sql_in | 42 +-
methods/array_ops/src/pg_gp/test/array_ops.sql_in |3 +-
methods/kmeans/src/pg_gp/kmeans.c |5 +
methods/sketch/src/pg_gp/countmin.py_in| 19 +-
methods/sketch/src/pg_gp/sketch.sql_in | 12 +-
methods/sketch/src/pg_gp/sql/cm.sql_in |2 +-
methods/stemmer/src/pg_gp/porter_stemmer.sql_in|2 +-
methods/svec_util/src/pg_gp/sql/svec_test.sql_in |3 +-
methods/svec_util/src/pg_gp/svec_util.sql_in |4 +-
src/bin/madpack|3 +-
src/madpack/argparse.py| 27 +-
src/madpack/configyml.py | 58 +-
src/madpack/create_changelist.py | 14 +-
src/madpack/diff_udf.sql |4 +-
src/madpack/diff_udo.sql |6 +-
src/madpack/diff_udoc.sql |6 +-
src/madpack/diff_udt.sql |6 +-
src/madpack/madpack.py | 53 +-
src/madpack/sort-module.py |6 +-
src/madpack/upgrade_util.py| 37 +-
src/madpack/utilities.py | 21 +-
src/madpack/yaml/LICENSE | 19 -
src/madpack/yaml/__init__.py | 290 -
src/madpack/yaml/composer.py | 118 -
src/madpack/yaml/constructor.py| 675 --
src/madpack/yaml/cyaml.py | 85 -
src/madpack/yaml/dumper.py | 62 -
src/madpack/yaml/emitter.py| 1163 ---
src/madpack/yaml/error.py | 75 -
src/madpack/yaml/events.py | 86 -
src/madpack/yaml/loader.py | 40 -
src/madpack/yaml/nodes.py | 49 -
src/madpack/yaml/parser.py | 586 --
src/madpack/yaml/reader.py | 225 -
src/madpack/yaml/representer.py| 488 --
src/madpack/yaml/resolver.py | 223 -
src/madpack/yaml/scanner.py| 1456
src/madpack/yaml/serializer.py | 111 -
src/madpack/yaml/tokens.py | 104 -
src/ports/greenplum/{5 => 7}/CMakeLists.txt|0
...FindGreenplum_5.cmake => FindGreenplum_7.cmake} |0
src/ports/greenplum/dbconnector/dbconnector.hpp|3 +
src/ports/postgres/{12 => 13}/CMakeLists.txt |0
src/ports/postgres/CMakeLists.txt | 13 +-
...PostgreSQL_10.cmake => FindPostgreSQL_13.cmake} |0
.../dbconnector/SystemInformation_impl.hpp | 11 +
src/ports/postgres/dbconnector/TypeTraits_impl.hpp | 10 +-
src/ports/postgres/dbconnector/dbconnector.hpp |3 -
src/ports/postgres/extension/madlib.control_in |4 +-
src/ports/postgres/madpack/SQLCommon.m4_in | 37 +-
.../modules/assoc_rules/assoc_rules.sql_in | 12 +-
src/ports/postgres/modules/bayes/bayes.py_in | 40 +-
src/ports/postgres/modules/bayes/bayes.sql_in | 20 +-
src/ports/postgres/modules/convex/lmf.sql_in |2 +-
src/ports/postgres/modules/convex/mlp.sql_in | 12 +-
src/ports/postgres/modules/convex/mlp_igd.py_in|9 +-
.../modules/convex/test/unit_tests/plpy_mock.py_in |2 +-
src/ports/postgres/modules/crf/crf.sql_in |4 +-
.../postgres/modules/crf/crf_data_loader.sql_in| 10 +-
.../postgres/modules/crf/crf_feature_gen.py_in |4 +-
.../postgres/modules/crf/crf_feature_gen.sql_in|4 +-
.../modules/crf/test/crf_
[madlib] branch master updated: Update version numbers to 1.22.0-dev
This is an automated email from the ASF dual-hosted git repository.
okislal pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/madlib.git
The following commit(s) were added to refs/heads/master by this push:
new a2a9db18 Update version numbers to 1.22.0-dev
a2a9db18 is described below
commit a2a9db18bd9cc93fee259b47f7989b606ce45bba
Author: Orhan Kislal
AuthorDate: Thu Mar 2 16:35:04 2023 +0300
Update version numbers to 1.22.0-dev
---
RELEASE_NOTES | 2 +-
deploy/DEB/postinst | 2 +-
deploy/postflight.sh | 2 +-
pom.xml | 2 +-
src/config/Version.yml| 2 +-
src/ports/postgres/modules/deep_learning/madlib_keras.sql_in | 8
.../postgres/modules/deep_learning/madlib_keras_automl.sql_in | 4 ++--
.../modules/deep_learning/madlib_keras_fit_multiple_model.sql_in | 6 +++---
src/ports/postgres/modules/svm/svm.sql_in | 2 +-
9 files changed, 15 insertions(+), 15 deletions(-)
diff --git a/RELEASE_NOTES b/RELEASE_NOTES
index 50f0f94e..0734a963 100644
--- a/RELEASE_NOTES
+++ b/RELEASE_NOTES
@@ -12,7 +12,7 @@ Current list of bugs and issues can be found at
https://issues.apache.org/jira/b
—-
MADlib v1.21.0:
-Release Date: 2023-March-XX
+Release Date: 2023-March-01
New features:
- Graph: Add warm start for weakly connected components
diff --git a/deploy/DEB/postinst b/deploy/DEB/postinst
index 8569abe5..586adb8d 100755
--- a/deploy/DEB/postinst
+++ b/deploy/DEB/postinst
@@ -22,7 +22,7 @@
# Source debconf library.
. /usr/share/debconf/confmodule
-MADLIB_VERSION="1.21.0"
+MADLIB_VERSION="1.22.0-dev"
MADLIB_INSTALL_PATH="InstallPathNotFound"
# Fetching configuration from debconf
diff --git a/deploy/postflight.sh b/deploy/postflight.sh
index cecd023a..2750f93c 100755
--- a/deploy/postflight.sh
+++ b/deploy/postflight.sh
@@ -2,7 +2,7 @@
# $0 - Script Path, $1 - Package Path, $2 - Target Location, and $3 - Target
Volume
-MADLIB_VERSION=1.21.0
+MADLIB_VERSION=1.22.0-dev
# Remove existing soft links
find $2/usr/local/madlib/bin -depth -type l -exec rm {} \; 2>/dev/null
diff --git a/pom.xml b/pom.xml
index 738d3e54..6d642c0e 100644
--- a/pom.xml
+++ b/pom.xml
@@ -22,7 +22,7 @@
org.apache.madlib
madlib
- 1.21.0
+ 1.22.0-dev
pom
diff --git a/src/config/Version.yml b/src/config/Version.yml
index 937913f4..8df8aefd 100644
--- a/src/config/Version.yml
+++ b/src/config/Version.yml
@@ -1 +1 @@
-version: 1.21.0
+version: 1.22.0-dev
diff --git a/src/ports/postgres/modules/deep_learning/madlib_keras.sql_in
b/src/ports/postgres/modules/deep_learning/madlib_keras.sql_in
index a30b394d..bf3779b5 100644
--- a/src/ports/postgres/modules/deep_learning/madlib_keras.sql_in
+++ b/src/ports/postgres/modules/deep_learning/madlib_keras.sql_in
@@ -1143,7 +1143,7 @@ model_size| 0.7900390625
start_training_time | 2021-02-01 15:58:43.760568
end_training_time | 2021-02-01 15:58:44.470054
metrics_elapsed_time | {0.709463119506836}
-madlib_version| 1.21.0
+madlib_version| 1.22.0-dev
num_classes | {3}
dependent_vartype | {"character varying"}
normalizing_const | 1
@@ -1457,7 +1457,7 @@ model_size| 0.7900390625
start_training_time | 2021-01-29 14:41:16.943861
end_training_time | 2021-01-29 14:41:19.478149
metrics_elapsed_time |
{2.3377411365509,2.42358803749084,2.49885511398315,2.53427410125732}
-madlib_version| 1.21.0
+madlib_version| 1.22.0-dev
num_classes | {3}
dependent_vartype | {"character varying"}
normalizing_const | 1
@@ -1627,7 +1627,7 @@ model_size| 0.7900390625
start_training_time | 2021-01-29 14:42:28.780276
end_training_time | 2021-01-29 14:42:31.177561
metrics_elapsed_time |
{2.24628114700317,2.28473520278931,2.32178020477295,2.35844302177429,2.39726710319519}
-madlib_version| 1.21.0
+madlib_version| 1.22.0-dev
num_classes | {3}
dependent_vartype | {"character varying"}
normalizing_const | 1
@@ -1742,7 +1742,7 @@ model_size| 0.7900390625
start_training_time | 2021-01-29 14:44:51.176983
end_training_time | 2021-01-29 14:44:53.666457
metrics_elapsed_time | {2.48945999145508}
-madlib_version| 1.21.0
+madlib_version| 1.22.0-dev
num_classes | {3}
dependent_vartype | {"character varying"}
normalizing_const | 1
diff --git
a/src/ports/postgres/modul
[madlib] branch madlib2-master created (now f06638ee)
This is an automated email from the ASF dual-hosted git repository. okislal pushed a change to branch madlib2-master in repository https://gitbox.apache.org/repos/asf/madlib.git at f06638ee Update version number to 2.0.0-dev This branch includes the following new commits: new f06638ee Update version number to 2.0.0-dev The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference.
[madlib] 01/01: Update version number to 2.0.0-dev
This is an automated email from the ASF dual-hosted git repository.
okislal pushed a commit to branch madlib2-master
in repository https://gitbox.apache.org/repos/asf/madlib.git
commit f06638ee2b173ec793dab66fd7cdbf1e68bd05c8
Author: Orhan Kislal
AuthorDate: Fri Mar 10 16:03:11 2023 +0300
Update version number to 2.0.0-dev
---
deploy/DEB/postinst | 2 +-
deploy/postflight.sh | 2 +-
pom.xml | 2 +-
src/config/Version.yml| 2 +-
src/ports/postgres/modules/deep_learning/madlib_keras.sql_in | 8
.../postgres/modules/deep_learning/madlib_keras_automl.sql_in | 4 ++--
.../modules/deep_learning/madlib_keras_fit_multiple_model.sql_in | 6 +++---
src/ports/postgres/modules/svm/svm.sql_in | 2 +-
8 files changed, 14 insertions(+), 14 deletions(-)
diff --git a/deploy/DEB/postinst b/deploy/DEB/postinst
index 8569abe5..8ffce914 100755
--- a/deploy/DEB/postinst
+++ b/deploy/DEB/postinst
@@ -22,7 +22,7 @@
# Source debconf library.
. /usr/share/debconf/confmodule
-MADLIB_VERSION="1.21.0"
+MADLIB_VERSION="2.0.0-dev"
MADLIB_INSTALL_PATH="InstallPathNotFound"
# Fetching configuration from debconf
diff --git a/deploy/postflight.sh b/deploy/postflight.sh
index cecd023a..dea1fd40 100755
--- a/deploy/postflight.sh
+++ b/deploy/postflight.sh
@@ -2,7 +2,7 @@
# $0 - Script Path, $1 - Package Path, $2 - Target Location, and $3 - Target
Volume
-MADLIB_VERSION=1.21.0
+MADLIB_VERSION=2.0.0-dev
# Remove existing soft links
find $2/usr/local/madlib/bin -depth -type l -exec rm {} \; 2>/dev/null
diff --git a/pom.xml b/pom.xml
index 738d3e54..dc24966e 100644
--- a/pom.xml
+++ b/pom.xml
@@ -22,7 +22,7 @@
org.apache.madlib
madlib
- 1.21.0
+ 2.0.0-dev
pom
diff --git a/src/config/Version.yml b/src/config/Version.yml
index 937913f4..9d51a0ae 100644
--- a/src/config/Version.yml
+++ b/src/config/Version.yml
@@ -1 +1 @@
-version: 1.21.0
+version: 2.0.0-dev
diff --git a/src/ports/postgres/modules/deep_learning/madlib_keras.sql_in
b/src/ports/postgres/modules/deep_learning/madlib_keras.sql_in
index a30b394d..b75d0fc5 100644
--- a/src/ports/postgres/modules/deep_learning/madlib_keras.sql_in
+++ b/src/ports/postgres/modules/deep_learning/madlib_keras.sql_in
@@ -1143,7 +1143,7 @@ model_size| 0.7900390625
start_training_time | 2021-02-01 15:58:43.760568
end_training_time | 2021-02-01 15:58:44.470054
metrics_elapsed_time | {0.709463119506836}
-madlib_version| 1.21.0
+madlib_version| 2.0.0-dev
num_classes | {3}
dependent_vartype | {"character varying"}
normalizing_const | 1
@@ -1457,7 +1457,7 @@ model_size| 0.7900390625
start_training_time | 2021-01-29 14:41:16.943861
end_training_time | 2021-01-29 14:41:19.478149
metrics_elapsed_time |
{2.3377411365509,2.42358803749084,2.49885511398315,2.53427410125732}
-madlib_version| 1.21.0
+madlib_version| 2.0.0-dev
num_classes | {3}
dependent_vartype | {"character varying"}
normalizing_const | 1
@@ -1627,7 +1627,7 @@ model_size| 0.7900390625
start_training_time | 2021-01-29 14:42:28.780276
end_training_time | 2021-01-29 14:42:31.177561
metrics_elapsed_time |
{2.24628114700317,2.28473520278931,2.32178020477295,2.35844302177429,2.39726710319519}
-madlib_version| 1.21.0
+madlib_version| 2.0.0-dev
num_classes | {3}
dependent_vartype | {"character varying"}
normalizing_const | 1
@@ -1742,7 +1742,7 @@ model_size| 0.7900390625
start_training_time | 2021-01-29 14:44:51.176983
end_training_time | 2021-01-29 14:44:53.666457
metrics_elapsed_time | {2.48945999145508}
-madlib_version| 1.21.0
+madlib_version| 2.0.0-dev
num_classes | {3}
dependent_vartype | {"character varying"}
normalizing_const | 1
diff --git
a/src/ports/postgres/modules/deep_learning/madlib_keras_automl.sql_in
b/src/ports/postgres/modules/deep_learning/madlib_keras_automl.sql_in
index 0447610f..1990491d 100644
--- a/src/ports/postgres/modules/deep_learning/madlib_keras_automl.sql_in
+++ b/src/ports/postgres/modules/deep_learning/madlib_keras_automl.sql_in
@@ -1091,7 +1091,7 @@ name |
description |
start_training_time | 2021-01-16 01:20:17
end_training_time | 2021-01-16 01:21:47
-madlib_version| 1.21.0
+madlib_version| 2.0.0-dev
num_classes | 3
class_values | {Iris-setosa,Iris-versicolor,Iris-virginica}
svn commit: r60400 - /release/madlib/KEYS
Author: okislal Date: Wed Mar 1 16:48:12 2023 New Revision: 60400 Log: Updating KEYS file for the release v1.21.0 Modified: release/madlib/KEYS Modified: release/madlib/KEYS == --- release/madlib/KEYS (original) +++ release/madlib/KEYS Wed Mar 1 16:48:12 2023 @@ -799,3 +799,61 @@ oP+vPXKF5xjhHWBfC6PCNDu+JwtB4UhnZ05GkHlb J9oa8m5x+PBIZQK5ZWvRivyOB9Cn3A== =8F1+ -END PGP PUBLIC KEY BLOCK- +pub rsa4096 2023-02-23 [SC] + FD36A42E059A032EB65D2DE3DCB18614D385D258 +uid [ultimate] Venkatesh Raghavan (G!) +sig 3DCB18614D385D258 2023-02-23 Venkatesh Raghavan (G!) +sub rsa4096 2023-02-23 [E] +sig DCB18614D385D258 2023-02-23 Venkatesh Raghavan (G!) +-BEGIN PGP PUBLIC KEY BLOCK- + +mQINBGP35egBEADeL98KrTWY/RZCptorj+KW25CCf/CNsh3AtjiX/duA57gGKn5y +SDOD5H2Du0oeMZqmcCFAvdx+tvyvlbm7ceVUYionBDVeEZK9bQ0VEOe4yqQydQFn +j/pO5S+Q77GsZKn/c15e/Cm7Rs9+SOhKDtMLR0W2AbIAG6B/yjfi1PkOu2C2cknB +LR1D4UTTT7JzgjUAgP9rjZnCnk9+XI7M5XAaatj3mlaNHGTLtiZdzdi7+P6LjtSF +poriy/9bI/BkbquffkHm4ePKyq+oiN1aEnMm8agofZrW+kOIXBP5qksKBsQK046M +QP5YUcth27saXIg4MufLMr3dgbzEiVsYnHMEctBmgLn+1j0Ns0cYiisyFswsvFH2 +NwQjCjCVKFJ3QAq2mqNcIc4ifecsxTJBGSBTz6eqLuBtYpoUNZUDOetHoOEmwiKH +Pfm8BqalvCKMy2QXoiutwrkcyzEcaoxsCxiWzPyOpBpRvHhpayXlKS0HG4QodiUL +hrnQ0ByxmX4Le+zB95LiVPZ9gXEyHXC9w4dXKE/PZPKrsXoTl15NBEBRkbnMKcHi +LUly8ph/NOHDhxnPKh7s8JVQ7m4G/PgpgiJRsq3V6PiXIhhchAOTC2NJnieE0WIP ++5YuYBix1/X5t1/q4IfSWsZCkSNv5/Iu4Iloh4zMadM2VRvozUGgUMiK5wARAQAB +tC9WZW5rYXRlc2ggUmFnaGF2YW4gKEchKSA8cmFnaGF2YW52ZUB2bXdhcmUuY29t +PokCUQQTAQgAOxYhBP02pC4FmgMutl0t49yxhhTThdJYBQJj9+XoAhsDBQsJCAcC +AiICBhUKCQgLAgQWAgMBAh4HAheAAAoJENyxhhTThdJY8cwP/2D1M1I15VWF+WB7 +KzjqAcjwF2wO7alWv+izrLzUME7lOMM7jSHDs0xCr/pMBk2fAepaRjUh1gZ34jWh +nJJKVddML4jCLE1TkuG9FLRu+N37A52BwUXCBfzxaeWiDYLqOjuQB+zqkkoFAbJa +E9VYfZcAT1Va7xPy+wPw3njv6KiV03vEmJ3TbRp1t2nM9LL4TfX88CYNIn8+/f96 +rZTrQcuYiSkrRfkayFHy/ErtgoC3bqKXCjNMu8vSKzTuK9oUZc0wdgrezV8pOHGX +snKrJ70HayzDBy3d05KaHlMKe4TZk2O8+Y4rHxmnxoqb9hbLQ08VeU4EAebWJyTF +TF2BdAbZJdBAdG8ah+oqM2XU+YbPL2iV/ymv47xMpVrLFT2Fy5M9/jpBZVv9c23S +vknBIzHOi6lFCJTmIXlLmj5BK3Y26x5Vk+BgzVHhFWrUX9leKNViwFw51hN/lo3D +9D0cWOAWXSyDQPtNpblbsRIjJfaiuJ39bkP8RQh3fjcCUgrrpg/PuHsuKvup0sDv +or/G5qf/kmYfuCYk2Y0oL4hwu5cTCuXw+CQqkTsOP9VozDvueFHN0ixsglKXUo0d +Hl8Mque2orYFD2E3vFPC3yZUltBJCVU6W6xaNLLpFJzXJid+wwvLE5bS0nZkilap +2/CTIuKBaomgg6tiVEHaHYoztCHxuQINBGP35egBEADKeiWpYxN/RD3OzXGOPUHF +oDGp2ssyBkMpjtFjL9ZYbGDlFEKnrJJeZNAbl4CPLMdQgpFWGfAGYDBzn3wyC9Rh +syCLcN7uL3R57PZoJOi62o+hU/TxgOwM2wAuX0VxcRPoP+X0ccyULLeuCcEgAbHl +0Ks5aVfQ3rIjBt6IbZXby2re9UuN536bWTloiejz3apqxS1hLAVTY1ZrB2aFku3Y +1zEvxaGgoPrrINmHuvMofNcdCl/3lQPVSAKxmJbkF/8lZ06xqXXhwNV+rgArtuhg +cXkui25cSJ9Ja2k9z2CDrEzU3mukItQGCz4GKfxrTmPljHSh/P399P6udX9nAcYU +aJXrabSkPi2HR1aglQ7O6C6BzZgzJB9Tjp5CpMODH4M3lq2REScIvxG00SujnHUz +D1/wn7pKNRGv0cm0J297RCkmYkDt9iIBhSBdFJqpIlyiECwL9us1jHdahTSR8G/Q +qAw3Ua/b+RdnopfcPPso9doL0SyHCFm/tJbEPEEY2Vjyjg02o09SVk20CasXsIYn +mZY25/mo5hXxvqyEBg6A/a/0TnhL3axSh1TUha1/xkMmfZK1prR6RGkdayB9XYrq +IfgHhNfEZVs//htQxXQI8BFF3geLxT/srWCGWW5RUazKjHAu9rz+rjqE6hAlTAD5 +ZW4JQYE2V+HVkexC57AGvQARAQABiQI2BBgBCAAgFiEE/TakLgWaAy62XS3j3LGG +FNOF0lgFAmP35egCGwwACgkQ3LGGFNOF0lgy7A/+IaA79VOkucqStA+Uug4FiS9r +YpbBWpZ2n6qZyzdp6Lf7qDl70Z/FvZ5HbifPTmiwcKrasdDIjEdmEw7K5sVDorrl +yxeQ2GRWALThyO8Ffw1pu18qV8U9brO+M0QNOJvGADU/LqturjW+2hLV3g8g/4Dl +/1CphCRapErowzfAy2GTmBMV7ezb7SCM3Yw3Vevc8gjBWvtJjmd0EN1LWmV4iSAU +7EbYqX+IVVugEoUc0Y+HLhhm3yR8KlYh5cOOI+oi9MLUs9xwb3mRKU41DHhVFdXl +J+l8rMjxRgguQ1+kZQ/CUXupbliPtHMLnFZweHlJ4vPeshyM1s4kX/JxZXAc2PYT +1FI7STgs6ouxGlZ1X+PtWLelYfvfRBQipvDsyoS4A/8VT5NBXBH3z0MGv3N9sqre +FFhsDGojc+xEoiVSagERy5cC0gpzyZ4SKPqKioVwK8bnZ+ZIaluteuQqKKxMp82n +n3IWRfmkq8+rkewO3P0qE6UfT0bwqgv7xqhcRPUUiBcLarF5FePlIZV+fvsSet6V +ocKcsZdByMBh3Pt2v1EJq0d5dh9iIGeEYjJIUy4QrYTc0WO5cs6E/maYOB2KLBTe +ttReZqaBKZlEBBsNoE5FFBOHFt2IBjoCsoeyBVHs4sSD7kEhI1E+kDjqb54MULJr +EReaDw8picFEfNfCg2Q= +=aVNg +-END PGP PUBLIC KEY BLOCK-
[madlib] annotated tag rel/v1.21.0 updated (68adc8b4 -> ba539945)
This is an automated email from the ASF dual-hosted git repository. okislal pushed a change to annotated tag rel/v1.21.0 in repository https://gitbox.apache.org/repos/asf/madlib.git *** WARNING: tag rel/v1.21.0 was modified! *** from 68adc8b4 (commit) to ba539945 (tag) tagging 68adc8b4af65efc4a660442230fd1e7a38cd90f0 (commit) replaces rc/1.21.0-rc1 by Orhan Kislal on Wed Mar 1 06:08:18 2023 +0300 - Log - Apache MADlib 1.21.0 -BEGIN PGP SIGNATURE- iQEzBAABCAAdFiEEGocCIKIuO+T6G3jEyiWcTeuzJCIFAmP+waIACgkQyiWcTeuz JCJn4gf+LJtueLXXR/+eWiQE3y1qoa2T91qb4q9Xx9Sy5EvnjScrkpioBrwb1VWs 60jMI1U/sWFGlFpLNFtfEkXbhKjj8xN0obagW/hE1J5bejs2Wi1Xv4ypnimdil95 BxpdBCeugUaza9zBWGus2AfSs/8efdz50WVGtH9ff6RpxmpYDylWH5QI7SKE58FQ HuWFI+vCacFmCkEMSnFhJmLSFbN+QBL+WFcHa4YjHUBVVbxaPMstNlkBwYoXSn8b mjp0cqS0hp//zzy5mKr5WpWkNzd9GqrnzPWHrRg7h78+ByNT23nq5P57Gxvb3sYZ AHuUFUeGIEX0qnlo1sIOIDAi1RmONg== =9ZWK -END PGP SIGNATURE- --- No new revisions were added by this update. Summary of changes:
svn commit: r60392 - /dev/madlib/1.21.0.RC2/ /release/madlib/1.21.0/
Author: okislal Date: Wed Mar 1 02:54:16 2023 New Revision: 60392 Log: Moving MADlib v1.21.0.RC2 from dist to release Added: release/madlib/1.21.0/ - copied from r60391, dev/madlib/1.21.0.RC2/ Removed: dev/madlib/1.21.0.RC2/
svn commit: r60288 - in /dev/madlib/1.21.0.RC2: ./ apache-madlib-1.21.0-src.tar.gz apache-madlib-1.21.0-src.tar.gz.asc apache-madlib-1.21.0-src.tar.gz.sha512
Author: okislal Date: Thu Feb 23 22:28:32 2023 New Revision: 60288 Log: Adding 1.21.0 RC2 candidate release artifacts Added: dev/madlib/1.21.0.RC2/ dev/madlib/1.21.0.RC2/apache-madlib-1.21.0-src.tar.gz (with props) dev/madlib/1.21.0.RC2/apache-madlib-1.21.0-src.tar.gz.asc dev/madlib/1.21.0.RC2/apache-madlib-1.21.0-src.tar.gz.sha512 Added: dev/madlib/1.21.0.RC2/apache-madlib-1.21.0-src.tar.gz == Binary file - no diff available. Propchange: dev/madlib/1.21.0.RC2/apache-madlib-1.21.0-src.tar.gz -- svn:mime-type = application/octet-stream Added: dev/madlib/1.21.0.RC2/apache-madlib-1.21.0-src.tar.gz.asc == --- dev/madlib/1.21.0.RC2/apache-madlib-1.21.0-src.tar.gz.asc (added) +++ dev/madlib/1.21.0.RC2/apache-madlib-1.21.0-src.tar.gz.asc Thu Feb 23 22:28:32 2023 @@ -0,0 +1,16 @@ +-BEGIN PGP SIGNATURE- + +iQIzBAABCAAdFiEE/TakLgWaAy62XS3j3LGGFNOF0lgFAmP35pQACgkQ3LGGFNOF +0lhs/A/+LExW7aaGj8EV5uvDiu/IiXdhnqZvbwlGMlryOulNzQOq5DAsofVavXh+ +rWywwc8mfoILXN8gddu2Nk3m/ENo+qEYrDsvDacZOK4iPmOuWT+v9udaSRDtFwL6 +0Yo9GnG4FHfKCOvlowDLgcM7ZhLwLtEz8t6TSXK4297hee5ulOE0XjuENAR71XZY +Pa9lKafAlkQYshxX21feafzc1r12rQzqS98zC9fDog6vMZyaaUHmPeP0cPZqBBSa +/t8E/E/uiUNavSgZCJ33xOkoblj55Gra2MhUsa/HBJD1lySMllqqp5FA4qrkY0fh +bt8VTt1inV2UJ5WEMnHB+T2R/is2kB8m+AvaUQFtN1ug8WKiPXY0oM2PxJ+c6ty6 +2lw5fROGRPO0NRfTWq2en4qIOmJUHcmlpdI32j93CtsU2ilM7881+E41NKSe/ffb +cIUZ/MHozUViGGQcr7evJGmXRnZBfDQN+XTdhWc9b+YtGZuYvi203WixDwu3uaod +n7/1Nf+LTLoTpBPSukxxg+22InfcOk3AUlXhnWtN0KgPmUzXpEN+3n9wrbdh/eWV +aoZ1h6kHqViosQnZbBRJeTbNx7Of+C4+OIfLXmrTbw1g0O2mGTpPxVKl1f993XNl +dZ/29i+Qm59k8hcafYyCgwYIrqKknOTgEwn39o33jJL2v6Cij88= +=aDEO +-END PGP SIGNATURE- Added: dev/madlib/1.21.0.RC2/apache-madlib-1.21.0-src.tar.gz.sha512 == --- dev/madlib/1.21.0.RC2/apache-madlib-1.21.0-src.tar.gz.sha512 (added) +++ dev/madlib/1.21.0.RC2/apache-madlib-1.21.0-src.tar.gz.sha512 Thu Feb 23 22:28:32 2023 @@ -0,0 +1 @@ +7f56a678f7dfa8dd7b9270c1b1cb2ae5c7cbc919beaee6fc5cf8ecf01f08372a34f70e9fe2ad51872a246a543fbf07f100a9eb08ba166de271339e6fabf12293 apache-madlib-1.21.0-src.tar.gz
svn commit: r60287 - /dev/madlib/KEYS
Author: okislal Date: Thu Feb 23 22:27:40 2023 New Revision: 60287 Log: Update KEYS with key for Venkatesh Raghavan Modified: dev/madlib/KEYS Modified: dev/madlib/KEYS == --- dev/madlib/KEYS (original) +++ dev/madlib/KEYS Thu Feb 23 22:27:40 2023 @@ -799,3 +799,61 @@ oP+vPXKF5xjhHWBfC6PCNDu+JwtB4UhnZ05GkHlb J9oa8m5x+PBIZQK5ZWvRivyOB9Cn3A== =8F1+ -END PGP PUBLIC KEY BLOCK- +pub rsa4096 2023-02-23 [SC] + FD36A42E059A032EB65D2DE3DCB18614D385D258 +uid [ultimate] Venkatesh Raghavan (G!) +sig 3DCB18614D385D258 2023-02-23 Venkatesh Raghavan (G!) +sub rsa4096 2023-02-23 [E] +sig DCB18614D385D258 2023-02-23 Venkatesh Raghavan (G!) +-BEGIN PGP PUBLIC KEY BLOCK- + +mQINBGP35egBEADeL98KrTWY/RZCptorj+KW25CCf/CNsh3AtjiX/duA57gGKn5y +SDOD5H2Du0oeMZqmcCFAvdx+tvyvlbm7ceVUYionBDVeEZK9bQ0VEOe4yqQydQFn +j/pO5S+Q77GsZKn/c15e/Cm7Rs9+SOhKDtMLR0W2AbIAG6B/yjfi1PkOu2C2cknB +LR1D4UTTT7JzgjUAgP9rjZnCnk9+XI7M5XAaatj3mlaNHGTLtiZdzdi7+P6LjtSF +poriy/9bI/BkbquffkHm4ePKyq+oiN1aEnMm8agofZrW+kOIXBP5qksKBsQK046M +QP5YUcth27saXIg4MufLMr3dgbzEiVsYnHMEctBmgLn+1j0Ns0cYiisyFswsvFH2 +NwQjCjCVKFJ3QAq2mqNcIc4ifecsxTJBGSBTz6eqLuBtYpoUNZUDOetHoOEmwiKH +Pfm8BqalvCKMy2QXoiutwrkcyzEcaoxsCxiWzPyOpBpRvHhpayXlKS0HG4QodiUL +hrnQ0ByxmX4Le+zB95LiVPZ9gXEyHXC9w4dXKE/PZPKrsXoTl15NBEBRkbnMKcHi +LUly8ph/NOHDhxnPKh7s8JVQ7m4G/PgpgiJRsq3V6PiXIhhchAOTC2NJnieE0WIP ++5YuYBix1/X5t1/q4IfSWsZCkSNv5/Iu4Iloh4zMadM2VRvozUGgUMiK5wARAQAB +tC9WZW5rYXRlc2ggUmFnaGF2YW4gKEchKSA8cmFnaGF2YW52ZUB2bXdhcmUuY29t +PokCUQQTAQgAOxYhBP02pC4FmgMutl0t49yxhhTThdJYBQJj9+XoAhsDBQsJCAcC +AiICBhUKCQgLAgQWAgMBAh4HAheAAAoJENyxhhTThdJY8cwP/2D1M1I15VWF+WB7 +KzjqAcjwF2wO7alWv+izrLzUME7lOMM7jSHDs0xCr/pMBk2fAepaRjUh1gZ34jWh +nJJKVddML4jCLE1TkuG9FLRu+N37A52BwUXCBfzxaeWiDYLqOjuQB+zqkkoFAbJa +E9VYfZcAT1Va7xPy+wPw3njv6KiV03vEmJ3TbRp1t2nM9LL4TfX88CYNIn8+/f96 +rZTrQcuYiSkrRfkayFHy/ErtgoC3bqKXCjNMu8vSKzTuK9oUZc0wdgrezV8pOHGX +snKrJ70HayzDBy3d05KaHlMKe4TZk2O8+Y4rHxmnxoqb9hbLQ08VeU4EAebWJyTF +TF2BdAbZJdBAdG8ah+oqM2XU+YbPL2iV/ymv47xMpVrLFT2Fy5M9/jpBZVv9c23S +vknBIzHOi6lFCJTmIXlLmj5BK3Y26x5Vk+BgzVHhFWrUX9leKNViwFw51hN/lo3D +9D0cWOAWXSyDQPtNpblbsRIjJfaiuJ39bkP8RQh3fjcCUgrrpg/PuHsuKvup0sDv +or/G5qf/kmYfuCYk2Y0oL4hwu5cTCuXw+CQqkTsOP9VozDvueFHN0ixsglKXUo0d +Hl8Mque2orYFD2E3vFPC3yZUltBJCVU6W6xaNLLpFJzXJid+wwvLE5bS0nZkilap +2/CTIuKBaomgg6tiVEHaHYoztCHxuQINBGP35egBEADKeiWpYxN/RD3OzXGOPUHF +oDGp2ssyBkMpjtFjL9ZYbGDlFEKnrJJeZNAbl4CPLMdQgpFWGfAGYDBzn3wyC9Rh +syCLcN7uL3R57PZoJOi62o+hU/TxgOwM2wAuX0VxcRPoP+X0ccyULLeuCcEgAbHl +0Ks5aVfQ3rIjBt6IbZXby2re9UuN536bWTloiejz3apqxS1hLAVTY1ZrB2aFku3Y +1zEvxaGgoPrrINmHuvMofNcdCl/3lQPVSAKxmJbkF/8lZ06xqXXhwNV+rgArtuhg +cXkui25cSJ9Ja2k9z2CDrEzU3mukItQGCz4GKfxrTmPljHSh/P399P6udX9nAcYU +aJXrabSkPi2HR1aglQ7O6C6BzZgzJB9Tjp5CpMODH4M3lq2REScIvxG00SujnHUz +D1/wn7pKNRGv0cm0J297RCkmYkDt9iIBhSBdFJqpIlyiECwL9us1jHdahTSR8G/Q +qAw3Ua/b+RdnopfcPPso9doL0SyHCFm/tJbEPEEY2Vjyjg02o09SVk20CasXsIYn +mZY25/mo5hXxvqyEBg6A/a/0TnhL3axSh1TUha1/xkMmfZK1prR6RGkdayB9XYrq +IfgHhNfEZVs//htQxXQI8BFF3geLxT/srWCGWW5RUazKjHAu9rz+rjqE6hAlTAD5 +ZW4JQYE2V+HVkexC57AGvQARAQABiQI2BBgBCAAgFiEE/TakLgWaAy62XS3j3LGG +FNOF0lgFAmP35egCGwwACgkQ3LGGFNOF0lgy7A/+IaA79VOkucqStA+Uug4FiS9r +YpbBWpZ2n6qZyzdp6Lf7qDl70Z/FvZ5HbifPTmiwcKrasdDIjEdmEw7K5sVDorrl +yxeQ2GRWALThyO8Ffw1pu18qV8U9brO+M0QNOJvGADU/LqturjW+2hLV3g8g/4Dl +/1CphCRapErowzfAy2GTmBMV7ezb7SCM3Yw3Vevc8gjBWvtJjmd0EN1LWmV4iSAU +7EbYqX+IVVugEoUc0Y+HLhhm3yR8KlYh5cOOI+oi9MLUs9xwb3mRKU41DHhVFdXl +J+l8rMjxRgguQ1+kZQ/CUXupbliPtHMLnFZweHlJ4vPeshyM1s4kX/JxZXAc2PYT +1FI7STgs6ouxGlZ1X+PtWLelYfvfRBQipvDsyoS4A/8VT5NBXBH3z0MGv3N9sqre +FFhsDGojc+xEoiVSagERy5cC0gpzyZ4SKPqKioVwK8bnZ+ZIaluteuQqKKxMp82n +n3IWRfmkq8+rkewO3P0qE6UfT0bwqgv7xqhcRPUUiBcLarF5FePlIZV+fvsSet6V +ocKcsZdByMBh3Pt2v1EJq0d5dh9iIGeEYjJIUy4QrYTc0WO5cs6E/maYOB2KLBTe +ttReZqaBKZlEBBsNoE5FFBOHFt2IBjoCsoeyBVHs4sSD7kEhI1E+kDjqb54MULJr +EReaDw8picFEfNfCg2Q= +=aVNg +-END PGP PUBLIC KEY BLOCK-
[madlib] annotated tag rc/1.21.0-rc2 updated (68adc8b4 -> da3a6a0b)
This is an automated email from the ASF dual-hosted git repository. okislal pushed a change to annotated tag rc/1.21.0-rc2 in repository https://gitbox.apache.org/repos/asf/madlib.git *** WARNING: tag rc/1.21.0-rc2 was modified! *** from 68adc8b4 (commit) to da3a6a0b (tag) tagging 68adc8b4af65efc4a660442230fd1e7a38cd90f0 (commit) replaces rc/1.21.0-rc1 by Orhan Kislal on Thu Feb 23 13:50:06 2023 -0500 - Log - Apache MADlib v1.21.0 RC2 -BEGIN PGP SIGNATURE- iHUEABYKAB0WIQQWb5K8Nsyi2QyY7ZHLGnOwrk5VpAUCY/e1XgAKCRDLGnOwrk5V pOnSAQD5j2J3cIO2hT2bvozFv4yJ3SjjZnhRI3Omkpvg+RFYYQD+Ld3VS0n8tLFg ZDJR03p87HSLNF0DtXriaeUdbPp/mgw= =fzuO -END PGP SIGNATURE- --- No new revisions were added by this update. Summary of changes:
[madlib] branch master updated: SSSP: Drop temp views
This is an automated email from the ASF dual-hosted git repository.
okislal pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/madlib.git
The following commit(s) were added to refs/heads/master by this push:
new 68adc8b4 SSSP: Drop temp views
68adc8b4 is described below
commit 68adc8b4af65efc4a660442230fd1e7a38cd90f0
Author: Orhan Kislal
AuthorDate: Thu Feb 23 12:25:16 2023 -0500
SSSP: Drop temp views
---
src/ports/postgres/modules/graph/sssp.py_in | 1 +
src/ports/postgres/modules/graph/test/sssp.sql_in | 2 +-
2 files changed, 2 insertions(+), 1 deletion(-)
diff --git a/src/ports/postgres/modules/graph/sssp.py_in
b/src/ports/postgres/modules/graph/sssp.py_in
index 09e4ce75..a1e640fa 100644
--- a/src/ports/postgres/modules/graph/sssp.py_in
+++ b/src/ports/postgres/modules/graph/sssp.py_in
@@ -445,6 +445,7 @@ def graph_sssp(schema_madlib, vertex_table, vertex_id,
edge_table,
plpy.execute("DROP TABLE IF EXISTS {0}".format(oldupdate))
if vertex_in != 'id' and vertex_type == "BIGINT":
plpy.execute("ALTER TABLE {0} RENAME COLUMN id TO
{1}".format(out_table, vertex_in))
+plpy.execute("DROP VIEW IF EXISTS {vertex_view},
{edge_view}".format(**locals()))
return None
diff --git a/src/ports/postgres/modules/graph/test/sssp.sql_in
b/src/ports/postgres/modules/graph/test/sssp.sql_in
index 92aceb23..1fa0a1cc 100644
--- a/src/ports/postgres/modules/graph/test/sssp.sql_in
+++ b/src/ports/postgres/modules/graph/test/sssp.sql_in
@@ -325,7 +325,7 @@ ascii('s'),
SELECT * FROM out_summary;
-- Test negative cycle
-DROP TABLE IF EXISTS vertex,"EDGE";
+DROP TABLE IF EXISTS vertex,"EDGE" CASCADE;
CREATE TABLE vertex(
id INTEGER
[madlib] annotated tag rc/1.21.0-rc1 updated (a7d182c8 -> 74b2a3a6)
This is an automated email from the ASF dual-hosted git repository. okislal pushed a change to annotated tag rc/1.21.0-rc1 in repository https://gitbox.apache.org/repos/asf/madlib.git *** WARNING: tag rc/1.21.0-rc1 was modified! *** from a7d182c8 (commit) to 74b2a3a6 (tag) tagging a7d182c872326133eda54b990094ad75d03e9d5e (commit) replaces rel/v1.20.0 by Orhan Kislal on Thu Feb 23 09:31:15 2023 -0500 - Log - Apache MADlib v1.21.0 RC1 -BEGIN PGP SIGNATURE- iHUEABYKAB0WIQQWb5K8Nsyi2QyY7ZHLGnOwrk5VpAUCY/d4swAKCRDLGnOwrk5V pAbhAQDcg6CtYKEFyqid5mGuE/f+DF6tbZMch65k8Xl9p9PkWwD+KVkAs3f9MIZa M64c5CSAlDD0B1aiyhm2d5EA9KsJgQg= =YaJV -END PGP SIGNATURE- --- No new revisions were added by this update. Summary of changes:
[madlib] branch master updated: Build: Update version to 1.21.0 and add release notes
This is an automated email from the ASF dual-hosted git repository.
okislal pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/madlib.git
The following commit(s) were added to refs/heads/master by this push:
new a7d182c8 Build: Update version to 1.21.0 and add release notes
a7d182c8 is described below
commit a7d182c872326133eda54b990094ad75d03e9d5e
Author: Orhan Kislal
AuthorDate: Wed Feb 22 13:33:11 2023 -0500
Build: Update version to 1.21.0 and add release notes
---
RELEASE_NOTES | 19 +++
deploy/DEB/postinst| 2 +-
deploy/postflight.sh | 2 +-
pom.xml| 2 +-
src/config/Version.yml | 2 +-
src/madpack/changelist_1.20.0_1.21.0.yaml | 58 ++
.../modules/deep_learning/madlib_keras.sql_in | 8 +--
.../deep_learning/madlib_keras_automl.sql_in | 4 +-
.../madlib_keras_fit_multiple_model.sql_in | 6 +--
src/ports/postgres/modules/svm/svm.sql_in | 2 +-
10 files changed, 91 insertions(+), 14 deletions(-)
diff --git a/RELEASE_NOTES b/RELEASE_NOTES
index 24cc9153..50f0f94e 100644
--- a/RELEASE_NOTES
+++ b/RELEASE_NOTES
@@ -9,6 +9,25 @@ commit history located at
https://github.com/apache/madlib/commits/master.
Current list of bugs and issues can be found at
https://issues.apache.org/jira/browse/MADLIB.
+—-
+MADlib v1.21.0:
+
+Release Date: 2023-March-XX
+
+New features:
+- Graph: Add warm start for weakly connected components
+- Graph: Add multicolumn identifier support for SSSP and APSP
+- Build: Add support for Photon3 OS
+
+Improvements:
+- XGBoost: Add support for bigint and varchar columns
+- XGBoost: Enable eval_metrics parameter
+
+Bug fixes:
+- XGBoost: Fix class label verification
+- Graph: Fix SSSP negative cycle check
+- Build: Disable TestIfNoUTF8BOM.py
+
—-
MADlib v1.20.0:
diff --git a/deploy/DEB/postinst b/deploy/DEB/postinst
index 6988df4d..8569abe5 100755
--- a/deploy/DEB/postinst
+++ b/deploy/DEB/postinst
@@ -22,7 +22,7 @@
# Source debconf library.
. /usr/share/debconf/confmodule
-MADLIB_VERSION="1.21.0-dev"
+MADLIB_VERSION="1.21.0"
MADLIB_INSTALL_PATH="InstallPathNotFound"
# Fetching configuration from debconf
diff --git a/deploy/postflight.sh b/deploy/postflight.sh
index c1de791b..cecd023a 100755
--- a/deploy/postflight.sh
+++ b/deploy/postflight.sh
@@ -2,7 +2,7 @@
# $0 - Script Path, $1 - Package Path, $2 - Target Location, and $3 - Target
Volume
-MADLIB_VERSION=1.21.0-dev
+MADLIB_VERSION=1.21.0
# Remove existing soft links
find $2/usr/local/madlib/bin -depth -type l -exec rm {} \; 2>/dev/null
diff --git a/pom.xml b/pom.xml
index ac46599b..738d3e54 100644
--- a/pom.xml
+++ b/pom.xml
@@ -22,7 +22,7 @@
org.apache.madlib
madlib
- 1.21.0-dev
+ 1.21.0
pom
diff --git a/src/config/Version.yml b/src/config/Version.yml
index d32d6df4..937913f4 100644
--- a/src/config/Version.yml
+++ b/src/config/Version.yml
@@ -1 +1 @@
-version: 1.21.0-dev
+version: 1.21.0
diff --git a/src/madpack/changelist_1.20.0_1.21.0.yaml
b/src/madpack/changelist_1.20.0_1.21.0.yaml
new file mode 100644
index ..1f96347b
--- /dev/null
+++ b/src/madpack/changelist_1.20.0_1.21.0.yaml
@@ -0,0 +1,58 @@
+#
--
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+#
--
+
+# Changelist for MADlib version rel/v1.20.0 to release-121
+
+# This file contains all changes that were introduced in a new version of
+# MADlib. This changelist is used by the upgrade script to detect what objects
+# should be upgraded (while retaining all other objects from the previous
version)
+
+# New modules (actually .sql_in files) added in upgrade version
+# For these files the sql_in code is retained as is with
[madlib] branch master updated: WCC: Add warm start
This is an automated email from the ASF dual-hosted git repository.
okislal pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/madlib.git
The following commit(s) were added to refs/heads/master by this push:
new f01d4338 WCC: Add warm start
f01d4338 is described below
commit f01d433886e9b943a22db3a24777e0ee113b0d79
Author: Orhan Kislal
AuthorDate: Mon Feb 13 11:35:00 2023 -0500
WCC: Add warm start
WCC creates a large number of subtransactions which may cause system
performance degredation in some cases. This cpmmit adds a parameter to
limit the number of iterations it runs as well as another one to
continue from the incomplete state.
---
src/ports/postgres/modules/graph/graph_utils.py_in | 12 +-
src/ports/postgres/modules/graph/test/wcc.sql_in | 89 +-
src/ports/postgres/modules/graph/wcc.py_in | 337 +
src/ports/postgres/modules/graph/wcc.sql_in| 51 +++-
4 files changed, 350 insertions(+), 139 deletions(-)
diff --git a/src/ports/postgres/modules/graph/graph_utils.py_in
b/src/ports/postgres/modules/graph/graph_utils.py_in
index 9c0a1c7a..8b0cf063 100644
--- a/src/ports/postgres/modules/graph/graph_utils.py_in
+++ b/src/ports/postgres/modules/graph/graph_utils.py_in
@@ -74,14 +74,18 @@ def validate_output_and_summary_tables(model_out_table,
module_name,
"Graph WCC: Output table {0} already
exists.".format(out_table))
def validate_graph_coding(vertex_table, vertex_id, edge_table, edge_params,
- out_table, func_name, **kwargs):
+ out_table, func_name, warm_start = False, **kwargs):
"""
Validates graph tables (vertex and edge) as well as the output table.
"""
_assert(out_table and out_table.strip().lower() not in ('null', ''),
-"Graph {func_name}: Invalid output table name!".format(**locals()))
-_assert(not table_exists(out_table),
-"Graph {func_name}: Output table already
exists!".format(**locals()))
+"Graph {func_name}: Invalid output table
name!".format(**locals()))
+if warm_start:
+_assert(table_exists(out_table),
+"Graph {func_name}: Output table is missing for warm
start!".format(**locals()))
+else:
+_assert(not table_exists(out_table),
+"Graph {func_name}: Output table already
exists!".format(**locals()))
_assert(vertex_table and vertex_table.strip().lower() not in ('null', ''),
"Graph {func_name}: Invalid vertex table name!".format(**locals()))
diff --git a/src/ports/postgres/modules/graph/test/wcc.sql_in
b/src/ports/postgres/modules/graph/test/wcc.sql_in
index f7af6868..5012246c 100644
--- a/src/ports/postgres/modules/graph/test/wcc.sql_in
+++ b/src/ports/postgres/modules/graph/test/wcc.sql_in
@@ -173,12 +173,12 @@ SELECT
weakly_connected_components('vertex','dest','"EDGE"',
'src=src_node,dest=dest_node','out','user_id');
SELECT * FROM out;
-ALTER TABLE vertex RENAME COLUMN dest TO id;
+ALTER TABLE vertex RENAME COLUMN dest TO vertex_id;
-- Test for bigint columns
-
-CREATE TABLE v2 AS SELECT (id+992147483647)::bigint as id FROM vertex;
-CREATE TABLE e2 AS SELECT (src_node+992147483647)::bigint as src,
(dest_node+992147483647)::bigint as dest FROM "EDGE";
+DROP TABLE IF EXISTS v2,e2;
+CREATE TABLE v2 AS SELECT (vertex_id+992147483647)::bigint as id FROM vertex;
+CREATE TABLE e2 AS SELECT (src_node+992147483647)::bigint as src,
(dest_node+992147483647)::bigint as dest, user_id FROM "EDGE";
SELECT weakly_connected_components('v2',NULL,'e2',NULL,'pg_temp.wcc_out');
SELECT count(*) from pg_temp.wcc_out;
@@ -188,7 +188,7 @@ SELECT count(*) from pg_temp.wcc_out_summary;
-- The datasets have the columns doubled so that the same tests can be run on
the output tables
DROP TABLE IF EXISTS vertex_mult, edge_mult CASCADE;
-CREATE TABLE vertex_mult AS SELECT id AS id1, id AS id2 FROM vertex;
+CREATE TABLE vertex_mult AS SELECT vertex_id AS id1, vertex_id AS id2 FROM
vertex;
CREATE TABLE edge_mult AS
SELECT src_node AS src1, src_node AS src2,
dest_node AS dest1, dest_node AS dest2,
@@ -276,3 +276,82 @@ SELECT graph_wcc_num_cpts(
SELECT assert(relative_error(num_components, 3) < 0.1,
'Weakly Connected Components: Incorrect largest component value.'
) FROM count_table WHERE user_id1=1;
+
+-- Warm Start
+
+-- Without grouping
+DROP TABLE IF EXISTS wcc_non_warm_start_out, wcc_non_warm_start_out_summary;
+SELECT
weakly_connected_components('v2',NULL,'e2',NULL,'wcc_non_warm_start_out');
+
+DROP TABLE IF EXISTS wcc_warm_start_out, wcc_warm_start_out_summary,
wcc_warm_start_out_message;
+SELECT weakly_connected_components('v2',NULL,'e2',NULL,'wcc_warm_start_out',
NULL, 1);
+
+SELECT assert(count(*) >
[madlib] branch master updated: SSSP: Fix negative cycle check
This is an automated email from the ASF dual-hosted git repository.
okislal pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/madlib.git
The following commit(s) were added to refs/heads/master by this push:
new ba772ae5 SSSP: Fix negative cycle check
ba772ae5 is described below
commit ba772ae58bd05edfe25e81d30c529cbcf6082c22
Author: Orhan Kislal
AuthorDate: Fri Jan 6 14:29:09 2023 +0300
SSSP: Fix negative cycle check
SSSP checks the iteration counter to identify negative cycles. If a
shortest path has the same length as the number of vertices, then the
check incorrectly identifies this as a negative cycle.
This commit fixes the issue and adds a relevant test.
---
src/ports/postgres/modules/graph/sssp.py_in | 9 +-
src/ports/postgres/modules/graph/test/sssp.sql_in | 110 ++
2 files changed, 115 insertions(+), 4 deletions(-)
diff --git a/src/ports/postgres/modules/graph/sssp.py_in
b/src/ports/postgres/modules/graph/sssp.py_in
index 05f12981..09e4ce75 100644
--- a/src/ports/postgres/modules/graph/sssp.py_in
+++ b/src/ports/postgres/modules/graph/sssp.py_in
@@ -325,7 +325,10 @@ def graph_sssp(schema_madlib, vertex_table, vertex_id,
edge_table,
WHERE {vertex_id} IS NOT NULL
""".format(**locals()))[0]['count']
-for i in range(0, v_cnt + 1):
+# The algorithm should converge in less than |V|+1 iterations.
+# Otherwise there is a negative cycle in the graph.
+# We check for v_cnt+2 because the first iteration is spent on the
setup
+for i in range(0, v_cnt + 2):
# Apply the updates calculated in the last iteration.
sql = """
@@ -400,9 +403,7 @@ def graph_sssp(schema_madlib, vertex_table, vertex_id,
edge_table,
newupdate = tmp
plpy.execute("DROP TABLE IF EXISTS {0}".format(newupdate))
-# The algorithm should converge in less than |V| iterations.
-# Otherwise there is a negative cycle in the graph.
-if i == v_cnt:
+if i == v_cnt+1:
if not grouping_cols:
plpy.execute("DROP TABLE IF EXISTS {0},{1},{2}".
format(out_table, summary_table, oldupdate))
diff --git a/src/ports/postgres/modules/graph/test/sssp.sql_in
b/src/ports/postgres/modules/graph/test/sssp.sql_in
index 63262247..92aceb23 100644
--- a/src/ports/postgres/modules/graph/test/sssp.sql_in
+++ b/src/ports/postgres/modules/graph/test/sssp.sql_in
@@ -256,3 +256,113 @@ SELECT assert(weight = 3, 'Wrong output in graph (SSSP)')
FROM sssp_mult_col_out WHERE id = ARRAY[6,6]::BIGINT[] AND grp1 = 0 AND
grp2 = 0;
SELECT assert(parent = ARRAY[5,5]::BIGINT[], 'Wrong parent in graph (SSSP)')
FROM sssp_mult_col_out WHERE id = ARRAY[6,6]::BIGINT[] AND grp1 = 0 AND
grp2 = 0;
+
+-- Test where |shortest path| = |V|
+DROP TABLE IF EXISTS out, out_summary;
+CREATE TABLE v(id INT);
+INSERT INTO v VALUES
+ (ascii('s')),
+ (ascii('t')),
+ (ascii('y')),
+ (ascii('x')),
+ (ascii('z'));
+
+CREATE TABLE e(src INT, dest INT, weight float8);
+INSERT INTO e VALUES
+(ascii('t'), ascii('x'), 5),
+(ascii('t'), ascii('y'), 8),
+(ascii('t'), ascii('z'), -4),
+(ascii('x'), ascii('t'), -2),
+(ascii('y'), ascii('x'), -3),
+(ascii('y'), ascii('z'), 9),
+(ascii('z'), ascii('x'), 7),
+(ascii('z'), ascii('s'), 2),
+(ascii('s'), ascii('t'), 6),
+(ascii('s'), ascii('y'), 7);
+
+CREATE TABLE e_grp(src INT, dest INT, weight float8, grp INT);
+INSERT INTO e_grp VALUES
+(ascii('t'), ascii('x'), 5, 1),
+(ascii('t'), ascii('y'), 8, 1),
+(ascii('t'), ascii('z'), -4, 1),
+(ascii('x'), ascii('t'), -2, 1),
+(ascii('y'), ascii('x'), -3, 1),
+(ascii('y'), ascii('z'), 9, 1),
+(ascii('z'), ascii('x'), 7, 1),
+(ascii('z'), ascii('s'), 2, 1),
+(ascii('s'), ascii('t'), 6, 1),
+(ascii('s'), ascii('y'), 7, 1),
+(ascii('t'), ascii('x'), 5, 2),
+(ascii('t'), ascii('y'), 8, 2),
+(ascii('t'), ascii('z'), -4, 2),
+(ascii('x'), ascii('t'), -2, 2),
+(ascii('y'), ascii('x'), -3, 2),
+(ascii('y'), ascii('z'), 9, 2),
+(ascii('z'), ascii('x'), 7, 2),
+(ascii('z'), ascii('s'), 2, 2),
+(ascii('s'), ascii('t'), 6, 2),
+(ascii('s'), ascii('y'), 7, 2);
+
+DROP TABLE IF EXISTS out, out_summary;
+SELECT graph_sssp('v',
+'id',
+'e',
+'src=src,dest=dest,weight=weight',
+ascii('s'),
+'out');
+
+SELECT * FROM out_summary;
+
+DROP TABLE IF EXISTS out, out_summary;
+SELECT graph_sssp('v',
+'id',
+'e_grp',
+'src=src,dest=dest,weight=weight',
+ascii('s'),
+'out',
+'grp');
+
+SELECT * FROM out_summary;
+
+-- Test negative cycle
+DROP TABLE IF EXISTS vertex,"EDGE";
+
+CREATE TABLE vertex(
+ id INTEGER
+);
+
+CREATE TABLE "EDGE"(
+ src INTEGER,
+ dest INTEGER,
+ weight DOUBLE PRECISION
[madlib] branch master updated: Disable TestIfNoUTF8BOM.py for all platforms
This is an automated email from the ASF dual-hosted git repository.
okislal pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/madlib.git
The following commit(s) were added to refs/heads/master by this push:
new d2c4d478 Disable TestIfNoUTF8BOM.py for all platforms
d2c4d478 is described below
commit d2c4d47894f8c28aa4dc81eb760383c27d7bafe5
Author: Orhan Kislal
AuthorDate: Mon Feb 13 11:32:09 2023 -0500
Disable TestIfNoUTF8BOM.py for all platforms
TestIfNoUTF8BOM.py file was needed for Postgres Versions before 9. Since
we don't support them any more, this commit comments them out. If there
are no problems, the commented lines and the file itself should be
removed in a later commit.
---
src/CMakeLists.txt | 2 +-
src/ports/greenplum/CMakeLists.txt | 2 +-
src/ports/postgres/CMakeLists.txt | 2 +-
3 files changed, 3 insertions(+), 3 deletions(-)
diff --git a/src/CMakeLists.txt b/src/CMakeLists.txt
index 0cf56614..fc4be6ba 100644
--- a/src/CMakeLists.txt
+++ b/src/CMakeLists.txt
@@ -215,7 +215,7 @@ macro(add_sql_files OUT_SQL_TARGET_FILES IN_SOURCE_DIR
IN_TARGET_DIR)
)
# psql of PostgreSQL < 9 does not like byte-order marks
set(_MADLIB_VERIFY_AND_COPY_COMMAND
-COMMAND "${CMAKE_SOURCE_DIR}/cmake/TestIfNoUTF8BOM.py"
"\"\${CURRENT_PATH}\""
+# COMMAND "${CMAKE_SOURCE_DIR}/cmake/TestIfNoUTF8BOM.py"
"\"\${CURRENT_PATH}\""
COMMAND "${CMAKE_COMMAND}" -E copy "\"\${CURRENT_PATH}\""
"\"\${OUTFILE}\""
)
batch_add_command(
diff --git a/src/ports/greenplum/CMakeLists.txt
b/src/ports/greenplum/CMakeLists.txt
index 07d19374..5634d4d9 100644
--- a/src/ports/greenplum/CMakeLists.txt
+++ b/src/ports/greenplum/CMakeLists.txt
@@ -100,7 +100,7 @@ add_sql_files(
string(REPLACE "/src/pg_gp" "" OUTFILE ${OUTFILE})
string(REPLACE "/sql/" "/test/" OUTFILE ${OUTFILE})
add_custom_command(OUTPUT "${OUTFILE}"
-COMMAND "${CMAKE_SOURCE_DIR}/cmake/TestIfNoUTF8BOM.py"
"${CURRENT_PATH}"
+# COMMAND "${CMAKE_SOURCE_DIR}/cmake/TestIfNoUTF8BOM.py"
"${CURRENT_PATH}"
COMMAND ${CMAKE_COMMAND} -E copy "${CURRENT_PATH}" "${OUTFILE}"
DEPENDS "${CURRENT_PATH}"
COMMENT "Validating and copying ${CURRENT_FILE}"
diff --git a/src/ports/postgres/CMakeLists.txt
b/src/ports/postgres/CMakeLists.txt
index 6c7b3a89..3bec9543 100644
--- a/src/ports/postgres/CMakeLists.txt
+++ b/src/ports/postgres/CMakeLists.txt
@@ -85,7 +85,7 @@ add_sql_files(
string(REPLACE "/src/pg_gp" "" OUTFILE ${OUTFILE})
string(REPLACE "/sql/" "/test/" OUTFILE ${OUTFILE})
add_custom_command(OUTPUT "${OUTFILE}"
-COMMAND "${CMAKE_SOURCE_DIR}/cmake/TestIfNoUTF8BOM.py"
"${CURRENT_PATH}"
+# COMMAND "${CMAKE_SOURCE_DIR}/cmake/TestIfNoUTF8BOM.py"
"${CURRENT_PATH}"
COMMAND ${CMAKE_COMMAND} -E copy "${CURRENT_PATH}" "${OUTFILE}"
DEPENDS "${CURRENT_PATH}"
COMMENT "Validating and copying ${CURRENT_FILE}"
[madlib] branch master updated: Update NOTICE for 2023
This is an automated email from the ASF dual-hosted git repository. okislal pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/madlib.git The following commit(s) were added to refs/heads/master by this push: new 475aba5b Update NOTICE for 2023 475aba5b is described below commit 475aba5bec6e95f9023db1ba4b8edb1c00e878c0 Author: Orhan Kislal AuthorDate: Thu Feb 16 12:25:48 2023 -0500 Update NOTICE for 2023 --- NOTICE | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/NOTICE b/NOTICE index a5a4367f..94bb576c 100644 --- a/NOTICE +++ b/NOTICE @@ -1,5 +1,5 @@ Apache MADlib -Copyright 2016-2022 The Apache Software Foundation. +Copyright 2016-2023 The Apache Software Foundation. This product includes software developed at The Apache Software Foundation (http://www.apache.org/).
[madlib] branch master updated: Fix TestIfNoUTF8BOM.py for newer versions of cmake
This is an automated email from the ASF dual-hosted git repository.
okislal pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/madlib.git
The following commit(s) were added to refs/heads/master by this push:
new ac151734 Fix TestIfNoUTF8BOM.py for newer versions of cmake
ac151734 is described below
commit ac1517347fd37fce4691dfe73ab25095d04ae723
Author: Orhan Kislal
AuthorDate: Wed Jan 4 15:15:44 2023 +0300
Fix TestIfNoUTF8BOM.py for newer versions of cmake
---
cmake/TestIfNoUTF8BOM.py | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/cmake/TestIfNoUTF8BOM.py b/cmake/TestIfNoUTF8BOM.py
index 47cd245f..135a0c00 100755
--- a/cmake/TestIfNoUTF8BOM.py
+++ b/cmake/TestIfNoUTF8BOM.py
@@ -27,7 +27,7 @@ def main(argv=None):
if BOM != "utf_8":
return 0
else:
-print 'Detected byte-order mark (%s) in file "%s".' % (BOM, argv[1])
+print ('Detected byte-order mark (%s) in file "%s".' % (BOM, argv[1]))
return 1
if __name__ == '__main__':
[madlib] branch master updated: XGBoost: Various fixes
This is an automated email from the ASF dual-hosted git repository.
okislal pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/madlib.git
The following commit(s) were added to refs/heads/master by this push:
new ee7c919d XGBoost: Various fixes
ee7c919d is described below
commit ee7c919d256322d66a3112a862d6123f713d7a45
Author: Orhan Kislal
AuthorDate: Thu Oct 13 15:06:45 2022 -0400
XGBoost: Various fixes
- Fix class label verification in predict
Class label is an optional argument for predict but the code checks to
ensure the column exists in the table. This commit fixes the issue and
adds a test to run predict with default values and no optional
parameters.
- Add support for bigint and varchar id col
XGBoost supports non-integer values as id columns (not features) in the
python implementation. This commit alters the surrounding code to
accomodate for such column types and adds/alters tests accordingly.
- Add eval_metrics as a parameter
eval_metrics is used by XGBoost used monitoring the training result and
early stopping. We expose this parameter to the user and parse it to
pass it to the fit function (instead of init).
---
.../postgres/modules/mxgboost/madlib_xgboost.py_in | 30 ++---
.../modules/mxgboost/madlib_xgboost.sql_in | 11 +++-
.../modules/mxgboost/test/madlib_xgboost.sql_in| 73 +++---
3 files changed, 94 insertions(+), 20 deletions(-)
diff --git a/src/ports/postgres/modules/mxgboost/madlib_xgboost.py_in
b/src/ports/postgres/modules/mxgboost/madlib_xgboost.py_in
index bd4d5d28..ce2a8e4f 100644
--- a/src/ports/postgres/modules/mxgboost/madlib_xgboost.py_in
+++ b/src/ports/postgres/modules/mxgboost/madlib_xgboost.py_in
@@ -40,6 +40,7 @@ from utilities.utilities import _assert
from utilities.utilities import add_postfix
from utilities.utilities import unique_string
from utilities.validate_args import get_cols
+from utilities.validate_args import get_expr_type
from utilities.validate_args import input_tbl_valid
from utilities.validate_args import output_tbl_valid
from utilities.validate_args import cols_in_tbl_valid
@@ -106,6 +107,13 @@ def expand_grid(params):
params_grid = [l for l in itertools.product(*params_list)]
return params_grid
+def try_literal_eval(t):
+try:
+ret = ast.literal_eval(t)
+except Exception:
+ret = t
+return ret
+
def xgboost_train(schema_madlib, dframe, features_all, class_label, params,
class_weights, train_set_size, id_column,
train_set_split_var):
"""
@@ -137,6 +145,7 @@ def xgboost_train(schema_madlib, dframe, features_all,
class_label, params,
X_test = X[numpy.array(df[train_set_split_var]==0),]
y_train = y[numpy.array(df[train_set_split_var]==1)]
y_test = y[numpy.array(df[train_set_split_var]==0)]
+
#save off and remove the id_column for later output. Make sure to get rid
of id_column from features!
test_ids = X_test [:,len(features)-1]
X_train = numpy.delete(X_train,len(features)-1,1)
@@ -167,14 +176,14 @@ def xgboost_train(schema_madlib, dframe, features_all,
class_label, params,
#Train gradient boosted trees
p_list = [p.split('=') for p in
ast.literal_eval(re.sub("[\\t]","",params).strip())]
-params_dict = dict([(k, ast.literal_eval(v.strip())) for k,v in p_list])
+params_dict = dict([(k, try_literal_eval(v.strip())) for k,v in p_list])
+eval_metric = params_dict.pop('eval_metric') if 'eval_metric' in
params_dict else 'auc'
gbm = xgb.XGBClassifier(**params_dict)
-
#Fit model
gbm.fit(
X_train,
y_train,
-eval_metric = 'auc',
+eval_metric = eval_metric,
sample_weight = sample_weights
)
#Compute and return model metrics score
@@ -199,7 +208,6 @@ def xgboost_train(schema_madlib, dframe, features_all,
class_label, params,
fnames, f_importance_scores = zip(*fnames_importances)
important_features = pd.DataFrame(fnames_importances)
-test_ids = [int(x) for x in test_ids]
return (features, pickle.dumps(gbm), params, fnames, f_importance_scores,
model_metrics.iloc[:,1].values.tolist(),
model_metrics.iloc[:,2].values.tolist(),
model_metrics.iloc[:,3].values.tolist(),model_metrics.iloc[:,4].values.tolist(),
@@ -240,7 +248,8 @@ def xgboost_grid_search(schema_madlib, source_table,
id_column, class_label,
{
'learning_rate': [0.3], #Regularization on weights (eta). For
smaller values, increase n_estimators
'max_depth': [6],#Larger values could lead to overfitting
-'n_estimators':[100] #More estimators, lesser variance (better fit
on test set)
+'n_estimators':[100], #More estimators, lesser variance (better
fit on test set)
+'eval_metric':['auc']
}
[madlib-site] branch asf-site updated: Add graph multicolumn 1
This is an automated email from the ASF dual-hosted git repository.
okislal pushed a commit to branch asf-site
in repository https://gitbox.apache.org/repos/asf/madlib-site.git
The following commit(s) were added to refs/heads/asf-site by this push:
new 5b6a895 Add graph multicolumn 1
5b6a895 is described below
commit 5b6a8952f189d7369a09ec0aa711e1d227f1437e
Author: Orhan Kislal
AuthorDate: Wed Aug 24 13:22:12 2022 +0300
Add graph multicolumn 1
---
community-artifacts/Graph/Multicolumn-v1.ipynb | 963 +
.../Supervised-learning/xgboost-v1.ipynb | 549 +---
2 files changed, 990 insertions(+), 522 deletions(-)
diff --git a/community-artifacts/Graph/Multicolumn-v1.ipynb
b/community-artifacts/Graph/Multicolumn-v1.ipynb
new file mode 100644
index 000..17bb852
--- /dev/null
+++ b/community-artifacts/Graph/Multicolumn-v1.ipynb
@@ -0,0 +1,963 @@
+{
+ "cells": [
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+"# Multicolumn Identifier Support for Graph functions\n",
+"MADlib graph functions are designed to have a vertex and an edge table.
Each vertex has an id in the vertex table and a pair of ids in the edge table
define a directed edge. Starting with version 1.20.0, multiple columns can be
used as the vertex identifier for select graph functions."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 1,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+"%load_ext sql"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 2,
+ "metadata": {},
+ "outputs": [
+{
+ "data": {
+ "text/plain": [
+ "u'Connected: okislal@madlib'"
+ ]
+ },
+ "execution_count": 2,
+ "metadata": {},
+ "output_type": "execute_result"
+}
+ ],
+ "source": [
+"%sql postgresql://okislal@localhost:6600/madlib\n",
+"\n",
+"# PostgreSQL local\n",
+"#%sql postgresql://fmcquillan@localhost:5432/madlib"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 3,
+ "metadata": {},
+ "outputs": [
+{
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ " * postgresql://okislal@localhost:6600/madlib\n",
+ "1 rows affected.\n"
+ ]
+},
+{
+ "data": {
+ "text/html": [
+ "\n",
+ "\n",
+ "version\n",
+ "\n",
+ "\n",
+ "MADlib version: 1.21.0-dev, git revision:
rel/v1.20.0-4-g195895cc, cmake configuration time: Thu Aug 18 13:37:23 UTC
2022, build type: RelWithDebInfo, build system: Darwin-20.6.0, C compiler:
Clang, C++ compiler: Clang\n",
+ "\n",
+ ""
+ ],
+ "text/plain": [
+ "[(u'MADlib version: 1.21.0-dev, git revision: rel/v1.20.0-4-g195895cc,
cmake configuration time: Thu Aug 18 13:37:23 UTC 2022, build type:
RelWithDebInfo, build system: Darwin-20.6.0, C compiler: Clang, C++ compiler:
Clang',)]"
+ ]
+ },
+ "execution_count": 3,
+ "metadata": {},
+ "output_type": "execute_result"
+}
+ ],
+ "source": [
+"%sql select madlib.version();\n",
+"#%sql select version();"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+"# 1. Create vertex and edge tables"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 4,
+ "metadata": {},
+ "outputs": [
+{
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ " * postgresql://okislal@localhost:6600/madlib\n",
+ "Done.\n",
+ "Done.\n",
+ "Done.\n",
+ "7 rows affected.\n",
+ "22 rows affected.\n",
+ "22 rows affected.\n"
+ ]
+},
+{
+ "data": {
+ "text/html": [
+ "\n",
+ "\n",
+ "conn_src_major\n",
+ "conn_dest_major\n",
+ "user_id_major\n",
+ "conn_src_minor\n",
+ "conn_dest_minor\n",
+ "user_id_minor\n",
+ "\n",
+ "\n",
[madlib] branch master updated: Build: Fix photon rpm creation
This is an automated email from the ASF dual-hosted git repository.
okislal pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/madlib.git
The following commit(s) were added to refs/heads/master by this push:
new 92db2aef Build: Fix photon rpm creation
92db2aef is described below
commit 92db2aef512bed83afec9b4cb3f253c8420967fe
Author: Orhan Kislal
AuthorDate: Thu Aug 18 20:59:07 2022 +0300
Build: Fix photon rpm creation
JIRA: MADLIB-1510
Creating RPMs on Photon 3 systems missed rpm scripts. This commit
ensures that it is treated like a redhat system and necessary scripts
are included in the metadata.
---
cmake/LinuxUtils.cmake| 10 +++---
deploy/CMakeLists.txt | 9 -
deploy/RPM/CMakeLists.txt | 5 +
3 files changed, 4 insertions(+), 20 deletions(-)
diff --git a/cmake/LinuxUtils.cmake b/cmake/LinuxUtils.cmake
index 9c12f7eb..525df6e8 100644
--- a/cmake/LinuxUtils.cmake
+++ b/cmake/LinuxUtils.cmake
@@ -8,19 +8,15 @@ macro(rh_version OUT_VERSION)
else(EXISTS "/etc/redhat-release")
set(${OUT_VERSION} "${OUT_VERSION}-NOTFOUND")
endif(EXISTS "/etc/redhat-release")
-endmacro(rh_version)
-# Get the Photon OS version
-macro(ph_version OUT_VERSION)
if(EXISTS "/etc/photon-release")
file(READ "/etc/photon-release" _PHOTON_RELEASE_CONTENT)
-string(REGEX REPLACE "[^0-9.]*([0-9.]+)[^0-9.]*\$" "\\1" ${OUT_VERSION}
+string(REGEX REPLACE "VMware Photon OS ([0-9.]+)[0-9.]*$" "\\1"
+${OUT_VERSION}
"${_PHOTON_RELEASE_CONTENT}"
)
-else(EXISTS "/etc/photon-release")
-set(${OUT_VERSION} "${OUT_VERSION}-NOTFOUND")
endif(EXISTS "/etc/photon-release")
-endmacro(ph_version)
+endmacro(rh_version)
# Get the Debian version
# DEB_OUT_VERSION will have a number if /etc/issue exists, with an entry for
Debian.
diff --git a/deploy/CMakeLists.txt b/deploy/CMakeLists.txt
index 831985dc..e727e926 100644
--- a/deploy/CMakeLists.txt
+++ b/deploy/CMakeLists.txt
@@ -18,21 +18,12 @@ elseif(UNIX)
if(RH_VERSION AND NOT (RH_VERSION STREQUAL "RH_VERSION-NOTFOUND"))
set(IS_REDHAT "True")
endif()
-ph_version(PH_VERSION)
-if(PH_VERSION AND NOT (PH_VERSION STREQUAL "PH_VERSION-NOTFOUND"))
- set(IS_PHOTON "True")
-endif()
if(IS_REDHAT)
message(STATUS "Detected RH version ${RH_VERSION}")
list(APPEND CPACK_GENERATOR
RPM
)
-elseif(IS_PHOTON)
-message(STATUS "Detected Photon version ${PH_VERSION}")
-list(APPEND CPACK_GENERATOR
-RPM
-)
elseif(IS_DEBIAN)
message(STATUS "Detected Debian version ${DEB_VERSION}")
# By default, the cmake flag -DCREATE_RPM_FOR_UBUNTU is not
diff --git a/deploy/RPM/CMakeLists.txt b/deploy/RPM/CMakeLists.txt
index 88254c5a..da31da2c 100644
--- a/deploy/RPM/CMakeLists.txt
+++ b/deploy/RPM/CMakeLists.txt
@@ -8,7 +8,6 @@
# Get information about the environment
rh_version(RH_VERSION)
-ph_version(PH_VERSION)
# -- Set RPM-specific variables
@@ -18,9 +17,7 @@ set(CPACK_RPM_PACKAGE_GROUP "Development/Libraries"
PARENT_SCOPE)
set(CPACK_PACKAGING_INSTALL_PREFIX
"/usr/local/madlib/Versions/${MADLIB_VERSION_STRING}" PARENT_SCOPE)
set(_PACKAGE_REQUIRES "m4 >= 1.4")
-if(PH_VERSION)
-set(CPACK_RPM_PACKAGE_REQUIRES "python >= 2.6, ${_PACKAGE_REQUIRES}"
PARENT_SCOPE)
-elseif(RH_VERSION AND RH_VERSION VERSION_LESS "6.0")
+if(RH_VERSION AND RH_VERSION VERSION_LESS "6.0")
# on RH/CentOS 5, there is no Python 2.6 or higher in the default
# repositories.
set(CPACK_RPM_PACKAGE_REQUIRES "python, ${_PACKAGE_REQUIRES}" PARENT_SCOPE)
[madlib] branch master updated: Support Photon OS in the build system.
This is an automated email from the ASF dual-hosted git repository.
okislal pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/madlib.git
The following commit(s) were added to refs/heads/master by this push:
new 16e3e865 Support Photon OS in the build system.
16e3e865 is described below
commit 16e3e865509b9059e360f0215356f4393b2149dd
Author: Jianwen Dong
AuthorDate: Tue Aug 16 11:39:56 2022 -0700
Support Photon OS in the build system.
JIRA: MADLIB-1510
Previously the building system only support Centos and Debian OS.
In order to run MADlib on Photon OS, we need the support to build it
with cmake on Photon OS.
Co-authored-by: Jianwen Dong
Co-authored-by: Gaurab Dey
---
cmake/LinuxUtils.cmake| 12
deploy/CMakeLists.txt | 9 +
deploy/RPM/CMakeLists.txt | 5 -
3 files changed, 25 insertions(+), 1 deletion(-)
diff --git a/cmake/LinuxUtils.cmake b/cmake/LinuxUtils.cmake
index 0547ef4d..9c12f7eb 100644
--- a/cmake/LinuxUtils.cmake
+++ b/cmake/LinuxUtils.cmake
@@ -10,6 +10,18 @@ macro(rh_version OUT_VERSION)
endif(EXISTS "/etc/redhat-release")
endmacro(rh_version)
+# Get the Photon OS version
+macro(ph_version OUT_VERSION)
+if(EXISTS "/etc/photon-release")
+file(READ "/etc/photon-release" _PHOTON_RELEASE_CONTENT)
+string(REGEX REPLACE "[^0-9.]*([0-9.]+)[^0-9.]*\$" "\\1" ${OUT_VERSION}
+"${_PHOTON_RELEASE_CONTENT}"
+)
+else(EXISTS "/etc/photon-release")
+set(${OUT_VERSION} "${OUT_VERSION}-NOTFOUND")
+endif(EXISTS "/etc/photon-release")
+endmacro(ph_version)
+
# Get the Debian version
# DEB_OUT_VERSION will have a number if /etc/issue exists, with an entry for
Debian.
# DEB_OUT_VERSION will have 'DEB_OUT_VERSION-NOTFOUND' if /etc/issue does not
exist.
diff --git a/deploy/CMakeLists.txt b/deploy/CMakeLists.txt
index e727e926..831985dc 100644
--- a/deploy/CMakeLists.txt
+++ b/deploy/CMakeLists.txt
@@ -18,12 +18,21 @@ elseif(UNIX)
if(RH_VERSION AND NOT (RH_VERSION STREQUAL "RH_VERSION-NOTFOUND"))
set(IS_REDHAT "True")
endif()
+ph_version(PH_VERSION)
+if(PH_VERSION AND NOT (PH_VERSION STREQUAL "PH_VERSION-NOTFOUND"))
+ set(IS_PHOTON "True")
+endif()
if(IS_REDHAT)
message(STATUS "Detected RH version ${RH_VERSION}")
list(APPEND CPACK_GENERATOR
RPM
)
+elseif(IS_PHOTON)
+message(STATUS "Detected Photon version ${PH_VERSION}")
+list(APPEND CPACK_GENERATOR
+RPM
+)
elseif(IS_DEBIAN)
message(STATUS "Detected Debian version ${DEB_VERSION}")
# By default, the cmake flag -DCREATE_RPM_FOR_UBUNTU is not
diff --git a/deploy/RPM/CMakeLists.txt b/deploy/RPM/CMakeLists.txt
index da31da2c..88254c5a 100644
--- a/deploy/RPM/CMakeLists.txt
+++ b/deploy/RPM/CMakeLists.txt
@@ -8,6 +8,7 @@
# Get information about the environment
rh_version(RH_VERSION)
+ph_version(PH_VERSION)
# -- Set RPM-specific variables
@@ -17,7 +18,9 @@ set(CPACK_RPM_PACKAGE_GROUP "Development/Libraries"
PARENT_SCOPE)
set(CPACK_PACKAGING_INSTALL_PREFIX
"/usr/local/madlib/Versions/${MADLIB_VERSION_STRING}" PARENT_SCOPE)
set(_PACKAGE_REQUIRES "m4 >= 1.4")
-if(RH_VERSION AND RH_VERSION VERSION_LESS "6.0")
+if(PH_VERSION)
+set(CPACK_RPM_PACKAGE_REQUIRES "python >= 2.6, ${_PACKAGE_REQUIRES}"
PARENT_SCOPE)
+elseif(RH_VERSION AND RH_VERSION VERSION_LESS "6.0")
# on RH/CentOS 5, there is no Python 2.6 or higher in the default
# repositories.
set(CPACK_RPM_PACKAGE_REQUIRES "python, ${_PACKAGE_REQUIRES}" PARENT_SCOPE)
[madlib] branch master updated: Graph: Add multi column support for SSSP and APSP
This is an automated email from the ASF dual-hosted git repository.
okislal pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/madlib.git
The following commit(s) were added to refs/heads/master by this push:
new 195895cc Graph: Add multi column support for SSSP and APSP
195895cc is described below
commit 195895cc68f85648cb7e2b5fdc92f6069e006dfb
Author: Orhan Kislal
AuthorDate: Wed Jul 20 18:02:25 2022 +0300
Graph: Add multi column support for SSSP and APSP
JIRA: MADLIB-1506
This is a follow up commit for 8a24663ccf2ee76ef538c416ed505485b740ff04,
applying the same logic to add multi column vertex identification
support to SSSP, APSP, and their respective get path functions.
---
src/ports/postgres/modules/graph/apsp.py_in| 112 +++-
src/ports/postgres/modules/graph/apsp.sql_in | 11 +
src/ports/postgres/modules/graph/graph_utils.py_in | 2 +
src/ports/postgres/modules/graph/sssp.py_in| 702 -
src/ports/postgres/modules/graph/sssp.sql_in | 35 +
src/ports/postgres/modules/graph/test/apsp.sql_in | 79 ++-
src/ports/postgres/modules/graph/test/sssp.sql_in | 79 +++
7 files changed, 680 insertions(+), 340 deletions(-)
diff --git a/src/ports/postgres/modules/graph/apsp.py_in
b/src/ports/postgres/modules/graph/apsp.py_in
index e854ab9e..9e0d4929 100644
--- a/src/ports/postgres/modules/graph/apsp.py_in
+++ b/src/ports/postgres/modules/graph/apsp.py_in
@@ -69,8 +69,8 @@ def graph_apsp(schema_madlib, vertex_table, vertex_id,
edge_table,
INFINITY = "'Infinity'"
EPSILON = 0.01
-params_types = {'src': str, 'dest': str, 'weight': str}
-default_args = {'src': 'src', 'dest': 'dest', 'weight': 'weight'}
+params_types = {'src': list, 'dest': list, 'weight': str}
+default_args = {'src': ['src'], 'dest': ['dest'], 'weight': 'weight'}
edge_params = extract_keyvalue_params(edge_args, params_types,
default_args)
# Prepare the input for recording in the summary table
@@ -89,19 +89,60 @@ def graph_apsp(schema_madlib, vertex_table, vertex_id,
edge_table,
if not grouping_cols:
g_st = ''
glist = None
+grouping_sql = ''
else:
g_st = grouping_cols
glist = split_quoted_delimited_str(grouping_cols)
+grouping_sql = ', {0}'.format(grouping_cols)
-src = edge_params["src"]
-dest = edge_params["dest"]
-weight = edge_params["weight"]
+vertex_in = vertex_id
-distribution = '' if is_platform_pg() else "DISTRIBUTED BY
({0})".format(src)
+if vertex_id[0] == '[' and vertex_id[-1] == ']':
+is_multicol = True
+vertex_id = split_quoted_delimited_str(vertex_id[1:-1])
+vertex_sql = "ARRAY[{0}]::BIGINT[]".format(','.join(vertex_id))
+vertex_type = "BIGINT[]"
+src = "ARRAY[{0}]::BIGINT[]".format(','.join(edge_params["src"]))
+dest = "ARRAY[{0}]::BIGINT[]".format(','.join(edge_params["dest"]))
+else:
+is_multicol = False
+vertex_sql = vertex_id
+vertex_id = [vertex_id]
+vertex_type = "BIGINT"
+edge_params["src"] = edge_params["src"][0]
+src = edge_params["src"]
+edge_params["dest"] = edge_params["dest"][0]
+dest = edge_params["dest"]
-_validate_apsp(vertex_table, vertex_id, edge_table,
+weight = edge_params["weight"]
+
+_validate_apsp(vertex_table, vertex_id, vertex_sql, edge_table,
edge_params, out_table, glist)
+vertex_view = unique_string('vertex_view')
+edge_view = unique_string('edge_view')
+
+sql = """
+CREATE VIEW {vertex_view} AS
+SELECT {vertex_sql} AS id
+FROM {vertex_table}
+""".format(**locals())
+plpy.execute(sql)
+sql = """
+CREATE VIEW {edge_view} AS
+SELECT {src} AS src, {dest} AS dest, {weight} {grouping_sql}
+FROM {edge_table}
+""".format(**locals())
+plpy.execute(sql)
+
+vertex_table = vertex_view
+edge_table = edge_view
+vertex_id = 'id'
+src = 'src'
+dest = 'dest'
+
+distribution = '' if is_platform_pg() else "DISTRIBUTED BY
({0})".format(src)
+
out_table_1 = unique_string(desp='out_table_1')
out_table_2 = unique_string(desp='out_table_2')
tmp_view = unique_string(desp='tmp_view')
@@ -202,7 +243,7 @@ def graph_apsp(schema_madlib, vertex_table, vertex_id,
edge_table,
plpy.execut
[madlib] branch master updated: XGBoost: Fix documentation example
This is an automated email from the ASF dual-hosted git repository. okislal pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/madlib.git The following commit(s) were added to refs/heads/master by this push: new 76a5db2f XGBoost: Fix documentation example 76a5db2f is described below commit 76a5db2f17b1338ef9acfba49fb40b458fe172f3 Author: Orhan Kislal AuthorDate: Fri Aug 5 16:16:55 2022 +0300 XGBoost: Fix documentation example --- doc/example/madlib_xgboost_example.sql | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/doc/example/madlib_xgboost_example.sql b/doc/example/madlib_xgboost_example.sql index 457d4206..36c928cf 100644 --- a/doc/example/madlib_xgboost_example.sql +++ b/doc/example/madlib_xgboost_example.sql @@ -4217,6 +4217,7 @@ INSERT INTO abalone VALUES DROP TABLE IF EXISTS xgb_out, xgb_out_summary; SELECT madlib.xgboost( 'abalone', -- Training table +'xgb_out', -- Grid search results table. 'id', -- Id column 'sex', -- Class label column '*',-- Independent variables @@ -4231,7 +4232,6 @@ SELECT madlib.xgboost( 'n_estimators':[100] #More estimators, lesser variance (better fit on test set) } $$, -- XGBoost grid search parameters -'xgb_out', -- Grid search results table. '', -- Class weights 0.8,-- Training set size ratio NULL-- Variable used to do the test/train split. @@ -4254,6 +4254,7 @@ SELECT * FROM xgb_score_out_metrics; DROP TABLE IF EXISTS xgb_out, xgb_out_summary; SELECT madlib.xgboost( 'abalone', -- Training table +'xgb_out', -- Grid search results table. 'id', -- Id column 'sex', -- Class label column '*',-- Independent variables @@ -4268,11 +4269,11 @@ SELECT madlib.xgboost( 'n_estimators':[100] #More estimators, lesser variance (better fit on test set) } $$, -- XGBoost grid search parameters -'xgb_out', -- Grid search results table. '', -- Class weights 0.8,-- Training set size ratio NULL-- Variable used to do the test/train split. ); + SELECT * FROM xgb_out_summary; DROP TABLE IF EXISTS xgb_score_out, xgb_score_out_metrics, xgb_score_out_roc_curve;
[madlib] 01/02: Update version number to 1.21.0-dev
This is an automated email from the ASF dual-hosted git repository.
okislal pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/madlib.git
commit 86e9ad28f6c65b42423898d07de92380e28760b6
Author: Orhan Kislal
AuthorDate: Fri Aug 5 11:45:08 2022 +0300
Update version number to 1.21.0-dev
---
RELEASE_NOTES | 2 +-
deploy/DEB/postinst | 2 +-
deploy/postflight.sh | 2 +-
pom.xml | 2 +-
src/config/Version.yml| 2 +-
src/ports/postgres/modules/deep_learning/madlib_keras.sql_in | 8
.../postgres/modules/deep_learning/madlib_keras_automl.sql_in | 4 ++--
.../modules/deep_learning/madlib_keras_fit_multiple_model.sql_in | 6 +++---
src/ports/postgres/modules/svm/svm.sql_in | 2 +-
9 files changed, 15 insertions(+), 15 deletions(-)
diff --git a/RELEASE_NOTES b/RELEASE_NOTES
index c9b8a68c..24cc9153 100644
--- a/RELEASE_NOTES
+++ b/RELEASE_NOTES
@@ -12,7 +12,7 @@ Current list of bugs and issues can be found at
https://issues.apache.org/jira/b
—-
MADlib v1.20.0:
-Release Date: 2022-July-XX
+Release Date: 2022-August-05
New features:
- XGBoost: Python based XGBoost with single and grid search executions
(MADLIB-1425, MADLIB-1490)
diff --git a/deploy/DEB/postinst b/deploy/DEB/postinst
index 3a564f33..6988df4d 100755
--- a/deploy/DEB/postinst
+++ b/deploy/DEB/postinst
@@ -22,7 +22,7 @@
# Source debconf library.
. /usr/share/debconf/confmodule
-MADLIB_VERSION="1.20.0"
+MADLIB_VERSION="1.21.0-dev"
MADLIB_INSTALL_PATH="InstallPathNotFound"
# Fetching configuration from debconf
diff --git a/deploy/postflight.sh b/deploy/postflight.sh
index 39056568..c1de791b 100755
--- a/deploy/postflight.sh
+++ b/deploy/postflight.sh
@@ -2,7 +2,7 @@
# $0 - Script Path, $1 - Package Path, $2 - Target Location, and $3 - Target
Volume
-MADLIB_VERSION=1.20.0
+MADLIB_VERSION=1.21.0-dev
# Remove existing soft links
find $2/usr/local/madlib/bin -depth -type l -exec rm {} \; 2>/dev/null
diff --git a/pom.xml b/pom.xml
index a4041793..ac46599b 100644
--- a/pom.xml
+++ b/pom.xml
@@ -22,7 +22,7 @@
org.apache.madlib
madlib
- 1.20.0
+ 1.21.0-dev
pom
diff --git a/src/config/Version.yml b/src/config/Version.yml
index d332d9ee..d32d6df4 100644
--- a/src/config/Version.yml
+++ b/src/config/Version.yml
@@ -1 +1 @@
-version: 1.20.0
+version: 1.21.0-dev
diff --git a/src/ports/postgres/modules/deep_learning/madlib_keras.sql_in
b/src/ports/postgres/modules/deep_learning/madlib_keras.sql_in
index d17bbc51..edca262f 100644
--- a/src/ports/postgres/modules/deep_learning/madlib_keras.sql_in
+++ b/src/ports/postgres/modules/deep_learning/madlib_keras.sql_in
@@ -1143,7 +1143,7 @@ model_size| 0.7900390625
start_training_time | 2021-02-01 15:58:43.760568
end_training_time | 2021-02-01 15:58:44.470054
metrics_elapsed_time | {0.709463119506836}
-madlib_version| 1.20.0
+madlib_version| 1.21.0-dev
num_classes | {3}
dependent_vartype | {"character varying"}
normalizing_const | 1
@@ -1457,7 +1457,7 @@ model_size| 0.7900390625
start_training_time | 2021-01-29 14:41:16.943861
end_training_time | 2021-01-29 14:41:19.478149
metrics_elapsed_time |
{2.3377411365509,2.42358803749084,2.49885511398315,2.53427410125732}
-madlib_version| 1.20.0
+madlib_version| 1.21.0-dev
num_classes | {3}
dependent_vartype | {"character varying"}
normalizing_const | 1
@@ -1627,7 +1627,7 @@ model_size| 0.7900390625
start_training_time | 2021-01-29 14:42:28.780276
end_training_time | 2021-01-29 14:42:31.177561
metrics_elapsed_time |
{2.24628114700317,2.28473520278931,2.32178020477295,2.35844302177429,2.39726710319519}
-madlib_version| 1.20.0
+madlib_version| 1.21.0-dev
num_classes | {3}
dependent_vartype | {"character varying"}
normalizing_const | 1
@@ -1742,7 +1742,7 @@ model_size| 0.7900390625
start_training_time | 2021-01-29 14:44:51.176983
end_training_time | 2021-01-29 14:44:53.666457
metrics_elapsed_time | {2.48945999145508}
-madlib_version| 1.20.0
+madlib_version| 1.21.0-dev
num_classes | {3}
dependent_vartype | {"character varying"}
normalizing_const | 1
diff --git
a/src/ports/postgres/modules/deep_learning/madlib_keras_automl.sql_in
b/src/ports/postgres/modules/deep_learning/madlib_keras_autom
[madlib] 02/02: Remove deep learning tests from jenkins
This is an automated email from the ASF dual-hosted git repository. okislal pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/madlib.git commit 918cc1466f4cea17c08a938199d8b74216dd7489 Author: Orhan Kislal AuthorDate: Fri Aug 5 15:51:16 2022 +0300 Remove deep learning tests from jenkins --- tool/jenkins/jenkins_build.sh | 2 ++ 1 file changed, 2 insertions(+) diff --git a/tool/jenkins/jenkins_build.sh b/tool/jenkins/jenkins_build.sh index 9e486085..904a102c 100755 --- a/tool/jenkins/jenkins_build.sh +++ b/tool/jenkins/jenkins_build.sh @@ -85,6 +85,8 @@ docker exec madlib bash -c '/build/src/bin/madpack -s mad -p postgres -c postgr EOF docker exec madlib bash -c 'mkdir -p /tmp' +docker exec madlib bash -c 'rm -rf /build/src/ports/postgres/modules/deep_learning/test' +docker exec madlib bash -c 'rm -rf /build/src/ports/postgres/11/modules/deep_learning/test' # Run dev check docker exec madlib bash -c '/build/src/bin/madpack -s mad -p postgres -c postgres/postgres@localhost:5432/postgres -d /tmp dev-check' | tee $workdir/logs/madlib_dev_check.log # Run unit tests, and append output to dev_check's log file
[madlib] branch master updated (f9761f49 -> 918cc146)
This is an automated email from the ASF dual-hosted git repository. okislal pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/madlib.git from f9761f49 Update RELEASE_NOTES new 86e9ad28 Update version number to 1.21.0-dev new 918cc146 Remove deep learning tests from jenkins The 2 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: RELEASE_NOTES | 2 +- deploy/DEB/postinst | 2 +- deploy/postflight.sh | 2 +- pom.xml | 2 +- src/config/Version.yml| 2 +- src/ports/postgres/modules/deep_learning/madlib_keras.sql_in | 8 .../postgres/modules/deep_learning/madlib_keras_automl.sql_in | 4 ++-- .../modules/deep_learning/madlib_keras_fit_multiple_model.sql_in | 6 +++--- src/ports/postgres/modules/svm/svm.sql_in | 2 +- tool/jenkins/jenkins_build.sh | 2 ++ 10 files changed, 17 insertions(+), 15 deletions(-)
[madlib-site] branch asf-site updated: Add xgboost ipython file
This is an automated email from the ASF dual-hosted git repository.
okislal pushed a commit to branch asf-site
in repository https://gitbox.apache.org/repos/asf/madlib-site.git
The following commit(s) were added to refs/heads/asf-site by this push:
new a8bbe0d Add xgboost ipython file
a8bbe0d is described below
commit a8bbe0d77f16a520fd884e2f275f55438fd68151
Author: Orhan Kislal
AuthorDate: Fri Aug 5 15:36:12 2022 +0300
Add xgboost ipython file
---
.../Supervised-learning/xgboost-v1.ipynb | 727 +
docs/v1.20.0/example/madlib_xgboost_example.sql| 4 +-
2 files changed, 729 insertions(+), 2 deletions(-)
diff --git a/community-artifacts/Supervised-learning/xgboost-v1.ipynb
b/community-artifacts/Supervised-learning/xgboost-v1.ipynb
new file mode 100644
index 000..2bc41b7
--- /dev/null
+++ b/community-artifacts/Supervised-learning/xgboost-v1.ipynb
@@ -0,0 +1,727 @@
+{
+ "cells": [
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+"# XGBoost\n",
+"XGBoost is an optimized distributed gradient boosting library designed to
be highly efficient, flexible and portable. It implements machine learning
algorithms under the Gradient Boosting framework. XGBoost provides a parallel
tree boosting (also known as GBDT, GBM) that solve many data science problems
in a fast and accurate way. XGBoost was first added in MADlib 1.20.0."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 1,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+"%load_ext sql"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 4,
+ "metadata": {},
+ "outputs": [
+{
+ "data": {
+ "text/plain": [
+ "u'Connected: okislal@madlib'"
+ ]
+ },
+ "execution_count": 4,
+ "metadata": {},
+ "output_type": "execute_result"
+}
+ ],
+ "source": [
+"# Greenplum Database 6.X\n",
+"%sql postgresql://okislal@localhost:6600/madlib"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 5,
+ "metadata": {},
+ "outputs": [
+{
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ " * postgresql://okislal@localhost:6600/madlib\n",
+ "1 rows affected.\n"
+ ]
+},
+{
+ "data": {
+ "text/html": [
+ "\n",
+ "\n",
+ "version\n",
+ "\n",
+ "\n",
+ "MADlib version: 1.20.0, git revision:
rc/1.20.0-rc2-6-gb07f7466, cmake configuration time: Fri Jul 29 14:31:52 UTC
2022, build type: RelWithDebInfo, build system: Darwin-20.6.0, C compiler:
Clang, C++ compiler: Clang\n",
+ "\n",
+ ""
+ ],
+ "text/plain": [
+ "[(u'MADlib version: 1.20.0, git revision: rc/1.20.0-rc2-6-gb07f7466,
cmake configuration time: Fri Jul 29 14:31:52 UTC 2022, build type:
RelWithDebInfo, build system: Darwin-20.6.0, C compiler: Clang, C++ compiler:
Clang',)]"
+ ]
+ },
+ "execution_count": 5,
+ "metadata": {},
+ "output_type": "execute_result"
+}
+ ],
+ "source": [
+"%sql select madlib.version();\n",
+"#%sql select version();"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+"# 1. Load data"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+"The sample data for XGBoost can be downloaded from the examples section
of the MADlib documentation. Direct link:
https://madlib.apache.org/docs/latest/example/madlib_xgboost_example.sql;
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 6,
+ "metadata": {},
+ "outputs": [
+{
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ " * postgresql://okislal@localhost:6600/madlib\n",
+ "10 rows affected.\n"
+ ]
+},
+{
+ "data": {
+ "text/html": [
+ "\n",
+ "\n",
+ "id\n",
+ "sex\n",
+ "length\n",
+ "diameter\n",
+ "height\n",
+ "whole\n",
+
[madlib] branch latest_release updated (13aa1cb2 -> f9761f49)
This is an automated email from the ASF dual-hosted git repository. okislal pushed a change to branch latest_release in repository https://gitbox.apache.org/repos/asf/madlib.git from 13aa1cb2 Add changelist files & update version add af363686 Update version from 1.16 to 1.17-dev add 8a612fe1 Add input params to last part of WCC impl add 640392dc Correlation: Process deconstruction in chunks for grouping add 0f6073cc Association Rules: Improve performance add ed202b10 Assoc Rules: Minor updates to user docs for new params add df03bc20 DL: Add function for predict byom add cb42f530 Quick fix for bison issue add 96885c89 DL: Split madlib_keras devcheck file add e4206866 KNN: Add distances to the output table add 205bdbaf updated user docs for madlib-keras with BYOM inference add 264a4d8d Build: Add cmake variable for gppkg filename add 33988ce4 Add generate mst table utility. This commit adds a new function to generate the model selection table. It generates such table as the naive combinations of the three input lists of parameter choices. This utility also validates and de-duplicates the inputs. add c89f42d5 madlib_keras_fit_multiple_model.* -> madlib_keras_model_selection.* add 6b6716f8 Catch error when fit_params_list or compile_params_list is NULL add f4e2ab69 Merge pull request #435 from reductionista/dl/mst-separate-module add a4f8ae1a DL: User docs for setting up model selection table add 0b1f3bf2 DL: Remove model_arch_table from output of load_model_selection_table() add 19ea7f76 update user docs to consider summary table add 5bc8ea10 DL: Update jenkins to install keras 2.2.4 add ac148f63 DL: Add support in preprocessor to evenly distribute data for GPDB add 7d484047 DL: Update training_preprocessor_dl to use bytea add bc93e25c Kmeans: Add automatic optimal cluster estimation add 72df6502 Use -p while creating tmp dir for jenkins build. add 7a130ec5 Kmeans: Add simple silhouette score for every point add dae72a0c update user docs for auto-k and per-point silh and generally reorganize add c416cd74 DL: Improve performance for madlib_keras_predict() add 9edd7458 DL: Update jenkins to install tensorflow 1.14. add 63f40e70 updated DL preprocessor docs for bytea (#445) add 1b5ba4af SVM: Lower bound the default for n_components add 828f9fec Kmeans: Use squared dist norm2 as a default for silh functions add 91923af3 Pivot: Use output_table instead of out_table add 610cf6de DL: Add training for multiple models add 913b66e4 DL: Use REAL[] instead of anyarray for aggregating arrays (#449) add 35e959d2 DL: Remove quote_ident to allow tables on schemas add 24c6e730 Add keras version to the docs and release notes add 72dfd30d Address review comments add 1e7aba37 Add pull request template add 22fabc8f DL: Update evaluate and predict for multi model outputs add 67813672 DL: Check for NULL in class_values for fit_multiple_model add 988c4ecb DL: Avoid having lingering processes from previous query add 509a6138 DL: Using distribution_key instead of gp_segment_id add 94648eab DL: Add param is_minibatched to validate_input_shape add ef2f0e90 DL: Add E2E dev-check tests add 828c2651 DL: Enable transfer learning for multi model training add 0f50490e DL: Make model_id naming consistent add 77ee7456 DL: Add new helper function for gpu_configuration (#455) add 5a1717ee MADLIB-1351 : Added stopping criteria on perplexity to LDA add d68e355d LDA: Turn off create table notices add ef11e2b5 DL: Add support for asymmetric segment distribution to preprocessor add 857049ef DL: Add optional parameters for multi model training add 9d2558fb DL: Add asymmetric cluster support for fit and evaluate add ea570674 DL: Add asymmetric cluster check for predict add ca17da42 DL: Update user docs for multi-model add ec5614fe misc user doc updates for 1dot17 add fc9cd64e correct fit function call in user docs for multi fit add 87758d70 clarify warm start vs transfer learning in user docs add eb771166 DL: Fix transfer learning dev-check test add 5f10bc8e DL: Modify multi-fit warm start to accept non-matching mst tables add 1fa020b0 clarify warm start with model selection in user docs add 7d2b02ca DL: Update tests calling trap_error add 687473ea Build: Add support for pg 12 add 85eba296 DL: Improve performance of mini-batch preprocessor (#467) add 7625ae01 DL: Fix failure on GPDB6 for preprocessor add 96a44244 Decrease the learning rate for transfer learning test add 273301e3 Update Apache Copyright date add 515dc257 misc user doc clarifications add dbae7223 DL: Fix metrics_elapsed_time for fit multi model add 5ddec472 DL: Drop leftover tables add 6b0240e2 Kmeans: Convert python inf
[madlib] annotated tag rel/v1.20.0 updated (f9761f49 -> edfd5e2a)
This is an automated email from the ASF dual-hosted git repository. okislal pushed a change to annotated tag rel/v1.20.0 in repository https://gitbox.apache.org/repos/asf/madlib.git *** WARNING: tag rel/v1.20.0 was modified! *** from f9761f49 (commit) to edfd5e2a (tag) tagging f9761f497bf29ae52f1406d4bd7b046d2726b6ab (commit) replaces rel/v1.19.0 by Orhan Kislal on Thu Aug 4 15:10:19 2022 +0300 - Log - Apache MADlib 1.20.0 -BEGIN PGP SIGNATURE- iQEzBAABCAAdFiEEGocCIKIuO+T6G3jEyiWcTeuzJCIFAmLrtysACgkQyiWcTeuz JCI8hgf9HZcv7egy1nhmq+0afXnjD3oJYAEoBRR3KIJZO9z6Wpzallvm/V24S0Kj 568AFH0xtPXcDBclgr57wuYKw3FPs3w50DbxxFj/6GoHonEZ32TfJKrBPQR+3oQ6 14dMTgEgMdGyU5uyvqLqaLlo6tLkZQTNBNEMQ/EIcxZcJ2edZjUuSU3xKhClTA1A maMAfiNgTTjAN8YqVgUa45p5rLQ0ZZMoylJT152pbptb925vsgSv+9lT4oGXiEY3 j+t/r6OupSovJTYzPRAj4d7MVot/J7ixk0BX3K79RGVTTxhpxljzNTWjM2vxudqz VJGRDl6SGWaw0rjek2HEVOLHYdDRIg== =DNf1 -END PGP SIGNATURE- --- No new revisions were added by this update. Summary of changes:
[madlib-site] branch asf-site updated: Update download page
This is an automated email from the ASF dual-hosted git repository. okislal pushed a commit to branch asf-site in repository https://gitbox.apache.org/repos/asf/madlib-site.git The following commit(s) were added to refs/heads/asf-site by this push: new 910bbee Update download page 910bbee is described below commit 910bbeea09180b4b53499ac405bc7d0a6683a4dc Author: Orhan Kislal AuthorDate: Thu Aug 4 15:08:24 2022 +0300 Update download page --- download.html | 17 +++-- 1 file changed, 15 insertions(+), 2 deletions(-) diff --git a/download.html b/download.html index ff62528..d3adb38 100644 --- a/download.html +++ b/download.html @@ -58,7 +58,7 @@ Current Release - v1.19.0 + v1.20.0 Source Code and Convenience Binaries MADlib source code is available from the Apache distribution site. @@ -66,7 +66,9 @@ Latest stable release: - https://dist.apache.org/repos/dist/release/madlib/1.19.0/apache-madlib-1.19.0-src.tar.gz;>Source code tar.gz (https://dist.apache.org/repos/dist/release/madlib/1.19.0/apache-madlib-1.19.0-src.tar.gz.asc;>pgp, https://dist.apache.org/repos/dist/release/madlib/1.19.0/apache-madlib-1.19.0-src.tar.gz.sha512;>sha512) + https://dist.apache.org/repos/dist/release/madlib/1.20.0/apache-madlib-1.20.0-src.tar.gz;>Source code tar.gz (https://dist.apache.org/repos/dist/release/madlib/1.20.0/apache-madlib-1.20.0-src.tar.gz.asc;>pgp, https://dist.apache.org/repos/dist/release/madlib/1.20.0/apache-madlib-1.20.0-src.tar.gz.sha512;>sha512) + + https://dist.apache.org/repos/dist/release/madlib/1.20.0/apache-madlib-1.20.0-CentOS7.rpm;>Linux (https://dist.apache.org/repos/dist/release/madlib/1.20.0/apache-madlib-1.20.0-CentOS7.rpm.asc;>pgp, https://dist.apache.org/repos/dist/release/madlib/1.20.0/apache-madlib-1.20.0-CentOS7.rpm.sha512;>sha512) — CentOS7 / Red Hat 7 (64 bit). G [...] @@ -93,6 +95,17 @@ Previous Apache Releases + v1.19.0 + Source Code and Convenience Binaries + + MADlib source code is available from the Apache distribution site. + + Latest stable release: + + + https://dist.apache.org/repos/dist/release/madlib/1.19.0/apache-madlib-1.19.0-src.tar.gz;>Source code tar.gz (https://dist.apache.org/repos/dist/release/madlib/1.19.0/apache-madlib-1.19.0-src.tar.gz.asc;>pgp, https://dist.apache.org/repos/dist/release/madlib/1.19.0/apache-madlib-1.19.0-src.tar.gz.sha512;>sha512) + + v1.18.0 Source Code and Convenience Binaries
[madlib-site] 01/01: Move 1.20.0 docs from rc
This is an automated email from the ASF dual-hosted git repository.
okislal pushed a commit to branch asf-site
in repository https://gitbox.apache.org/repos/asf/madlib-site.git
commit 844eeb3db09bc59d926f783f593536e02104dcc4
Author: Orhan Kislal
AuthorDate: Thu Aug 4 15:03:36 2022 +0300
Move 1.20.0 docs from rc
---
docs/latest | 2 +-
docs/{rc => v1.20.0}/apsp_8sql__in.html | 0
docs/{rc => v1.20.0}/arima_8sql__in.html| 0
docs/{rc => v1.20.0}/array__ops_8sql__in.html | 0
docs/{rc => v1.20.0}/assoc__rules_8sql__in.html | 0
docs/{rc => v1.20.0}/balance__sample_8sql__in.html | 0
docs/{rc => v1.20.0}/bayes_8sql__in.html| 0
docs/{rc => v1.20.0}/bc_s.png | Bin
docs/{rc => v1.20.0}/bdwn.png | Bin
docs/{rc => v1.20.0}/bfs_8sql__in.html | 0
docs/{rc => v1.20.0}/closed.png | Bin
.../clustered__variance_8sql__in.html | 0
.../clustered__variance__coxph_8sql__in.html| 0
docs/{rc => v1.20.0}/cols2vec_8sql__in.html | 0
.../conjugate__gradient_8sql__in.html | 0
docs/{rc => v1.20.0}/correlation_8sql__in.html | 0
.../cox__prop__hazards_8sql__in.html| 0
.../create__indicators_8sql__in.html| 0
docs/{rc => v1.20.0}/crf_8sql__in.html | 0
.../{rc => v1.20.0}/crf__data__loader_8sql__in.html | 0
.../{rc => v1.20.0}/crf__feature__gen_8sql__in.html | 0
.../{rc => v1.20.0}/cross__validation_8sql__in.html | 0
docs/{rc => v1.20.0}/dbscan_8sql__in.html | 0
docs/{rc => v1.20.0}/decision__tree_8sql__in.html | 0
.../dense__linear__systems_8sql__in.html| 0
.../dir_097d0c47873b413f914e0be0fb8e0c70.html | 0
.../dir_16b90e9a6c080d7fc2617cfecd7f223b.html | 0
.../dir_1960c8c5a7f1e7616d29e5a0c180bcf2.html | 0
.../dir_1c3de670490e30b2adf792227ef51e32.html | 0
.../dir_2e2767a73e2d710192c730fd426a8c90.html | 0
.../dir_3749b33f41e0a82f4dfb20be7f91dfa3.html | 0
.../dir_3ee26ceeadb429f807bd6cd822eca1b9.html | 0
.../dir_42e4eb27424bda9fbbfa95509de09bad.html | 0
.../dir_49426dfa1e413b9e4a461cc826e399ac.html | 0
.../dir_4f54709f5fc5d0f64da91428555e2469.html | 0
.../dir_505cd743a8a717435eca324f49291a46.html | 0
.../dir_5341dbb6d7f0a427749c6136276fa506.html | 0
.../dir_5cd13365c66443c54a8f3b4c82c03aee.html | 0
.../dir_5ff04c1d4b766f0a4dd68c6f19ed2092.html | 0
.../dir_60056874394c01e6c892c7492b3fbe27.html | 0
.../dir_68267d1309a1af8e8297ef4c3efbcdba.html | 0
.../dir_6ff79b0655deb26abf8f86290b84a97c.html | 0
.../dir_704eb8350b43e1ca74c0f90ed1ba450e.html | 0
.../dir_70e9abe17564e15776adcab8550c7667.html | 0
.../dir_73ccba3aa44ce35463f879b4ebbd3f46.html | 0
.../dir_7513a8b3b7336e99c30a333cfb144104.html | 0
.../dir_7592ceb856cf018f9811e11407c61fda.html | 0
.../dir_834e55cf992733df24fbe86b49356157.html | 0
.../dir_87d2e9694998410c1b0408bb775833a4.html | 0
.../dir_8b8525adb4e4d2ae68fa3d70bbeff8da.html | 0
.../dir_8cdb0d06bc3d9fdbe366fbcc82abc3e3.html | 0
.../dir_8d53e1c0026e7e34b4cd68f8b91426d6.html | 0
.../dir_93c42bb4df0f3e1302223b6dfd48c81e.html | 0
.../dir_9db0190ed89ab407304aec250d1ef64f.html | 0
.../dir_9e42ee0a0235722f482630aa6cc99334.html | 0
.../dir_9f5e6edf0db58b0627c46b41d27f.html | 0
.../dir_a4a48839224ef8488facbffa8a397967.html | 0
.../dir_ac52a4b89b7b1b1591f2952b5cbd041a.html | 0
.../dir_b0d8a62e5a4314de84ce58e7cd249850.html | 0
.../dir_b10c028a6a58d6e0840bc150baba59b9.html | 0
.../dir_b19a2d10787fafb5edd4ef1da0d2cacf.html | 0
.../dir_c2d3987992538edb8f634d5f4e0fae80.html | 0
.../dir_c8a9890f716ab6621b63d44c36a168eb.html | 0
.../dir_d06d449079888c21a85aa00b7664e813.html | 0
.../dir_d662e943e8adcb86abe4b822d2348c9e.html | 0
.../dir_d84ff4a1f4eeae069cebd336452cd47c.html | 0
.../dir_dc596537ad427a4d866006d1a3e1fe29.html | 0
.../dir_df86748cb94fb6c2fa09e991cce090c0.html | 0
.../dir_e2e6d80cc57daae810c80cbdae687e33.html | 0
.../dir_e502304d8bc7f1ef2456a474c6d92acd.html | 0
.../dir_e6c126a997181663ba81c11cbf416bb1.html | 0
.../dir_efbcf68973d247bbf15f9eecae7f24e3.html | 0
.../dir_fe647384fff178c2cff5ce75fb4044e5.html | 0
docs/{rc => v1.20.0}/distribution_8sql__in.html | 0
docs/{rc => v1.20.0}/doc.png| Bin
docs/{rc => v1.20.0}/dot_inline_dotgraph_1.svg | 0
docs/{rc => v1.20.0}/doxygen.css| 0
docs/{rc => v1.20.0}/doxygen.png| Bin
[madlib-site] branch asf-site updated (ebbacf5 -> 844eeb3)
This is an automated email from the ASF dual-hosted git repository. okislal pushed a change to branch asf-site in repository https://gitbox.apache.org/repos/asf/madlib-site.git discard ebbacf5 Move 1.20.0 docs from rc new 844eeb3 Move 1.20.0 docs from rc This update added new revisions after undoing existing revisions. That is to say, some revisions that were in the old version of the branch are not in the new version. This situation occurs when a user --force pushes a change and generates a repository containing something like this: * -- * -- B -- O -- O -- O (ebbacf5) \ N -- N -- N refs/heads/asf-site (844eeb3) You should already have received notification emails for all of the O revisions, and so the following emails describe only the N revisions from the common base, B. Any revisions marked "omit" are not gone; other references still refer to them. Any revisions marked "discard" are gone forever. The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: index.html | 11 --- 1 file changed, 11 deletions(-)
[madlib-site] branch asf-site updated: Move 1.20.0 docs from rc
This is an automated email from the ASF dual-hosted git repository.
okislal pushed a commit to branch asf-site
in repository https://gitbox.apache.org/repos/asf/madlib-site.git
The following commit(s) were added to refs/heads/asf-site by this push:
new ebbacf5 Move 1.20.0 docs from rc
ebbacf5 is described below
commit ebbacf57306b3bf307699f708537187d2d25e179
Author: Orhan Kislal
AuthorDate: Thu Aug 4 15:03:36 2022 +0300
Move 1.20.0 docs from rc
---
docs/latest| 2 +-
docs/{rc => v1.20.0}/apsp_8sql__in.html| 0
docs/{rc => v1.20.0}/arima_8sql__in.html | 0
docs/{rc => v1.20.0}/array__ops_8sql__in.html | 0
docs/{rc => v1.20.0}/assoc__rules_8sql__in.html| 0
docs/{rc => v1.20.0}/balance__sample_8sql__in.html | 0
docs/{rc => v1.20.0}/bayes_8sql__in.html | 0
docs/{rc => v1.20.0}/bc_s.png | Bin
docs/{rc => v1.20.0}/bdwn.png | Bin
docs/{rc => v1.20.0}/bfs_8sql__in.html | 0
docs/{rc => v1.20.0}/closed.png| Bin
.../clustered__variance_8sql__in.html | 0
.../clustered__variance__coxph_8sql__in.html | 0
docs/{rc => v1.20.0}/cols2vec_8sql__in.html| 0
.../conjugate__gradient_8sql__in.html | 0
docs/{rc => v1.20.0}/correlation_8sql__in.html | 0
.../cox__prop__hazards_8sql__in.html | 0
.../create__indicators_8sql__in.html | 0
docs/{rc => v1.20.0}/crf_8sql__in.html | 0
.../crf__data__loader_8sql__in.html| 0
.../crf__feature__gen_8sql__in.html| 0
.../cross__validation_8sql__in.html| 0
docs/{rc => v1.20.0}/dbscan_8sql__in.html | 0
docs/{rc => v1.20.0}/decision__tree_8sql__in.html | 0
.../dense__linear__systems_8sql__in.html | 0
.../dir_097d0c47873b413f914e0be0fb8e0c70.html | 0
.../dir_16b90e9a6c080d7fc2617cfecd7f223b.html | 0
.../dir_1960c8c5a7f1e7616d29e5a0c180bcf2.html | 0
.../dir_1c3de670490e30b2adf792227ef51e32.html | 0
.../dir_2e2767a73e2d710192c730fd426a8c90.html | 0
.../dir_3749b33f41e0a82f4dfb20be7f91dfa3.html | 0
.../dir_3ee26ceeadb429f807bd6cd822eca1b9.html | 0
.../dir_42e4eb27424bda9fbbfa95509de09bad.html | 0
.../dir_49426dfa1e413b9e4a461cc826e399ac.html | 0
.../dir_4f54709f5fc5d0f64da91428555e2469.html | 0
.../dir_505cd743a8a717435eca324f49291a46.html | 0
.../dir_5341dbb6d7f0a427749c6136276fa506.html | 0
.../dir_5cd13365c66443c54a8f3b4c82c03aee.html | 0
.../dir_5ff04c1d4b766f0a4dd68c6f19ed2092.html | 0
.../dir_60056874394c01e6c892c7492b3fbe27.html | 0
.../dir_68267d1309a1af8e8297ef4c3efbcdba.html | 0
.../dir_6ff79b0655deb26abf8f86290b84a97c.html | 0
.../dir_704eb8350b43e1ca74c0f90ed1ba450e.html | 0
.../dir_70e9abe17564e15776adcab8550c7667.html | 0
.../dir_73ccba3aa44ce35463f879b4ebbd3f46.html | 0
.../dir_7513a8b3b7336e99c30a333cfb144104.html | 0
.../dir_7592ceb856cf018f9811e11407c61fda.html | 0
.../dir_834e55cf992733df24fbe86b49356157.html | 0
.../dir_87d2e9694998410c1b0408bb775833a4.html | 0
.../dir_8b8525adb4e4d2ae68fa3d70bbeff8da.html | 0
.../dir_8cdb0d06bc3d9fdbe366fbcc82abc3e3.html | 0
.../dir_8d53e1c0026e7e34b4cd68f8b91426d6.html | 0
.../dir_93c42bb4df0f3e1302223b6dfd48c81e.html | 0
.../dir_9db0190ed89ab407304aec250d1ef64f.html | 0
.../dir_9e42ee0a0235722f482630aa6cc99334.html | 0
.../dir_9f5e6edf0db58b0627c46b41d27f.html | 0
.../dir_a4a48839224ef8488facbffa8a397967.html | 0
.../dir_ac52a4b89b7b1b1591f2952b5cbd041a.html | 0
.../dir_b0d8a62e5a4314de84ce58e7cd249850.html | 0
.../dir_b10c028a6a58d6e0840bc150baba59b9.html | 0
.../dir_b19a2d10787fafb5edd4ef1da0d2cacf.html | 0
.../dir_c2d3987992538edb8f634d5f4e0fae80.html | 0
.../dir_c8a9890f716ab6621b63d44c36a168eb.html | 0
.../dir_d06d449079888c21a85aa00b7664e813.html | 0
.../dir_d662e943e8adcb86abe4b822d2348c9e.html | 0
.../dir_d84ff4a1f4eeae069cebd336452cd47c.html | 0
.../dir_dc596537ad427a4d866006d1a3e1fe29.html | 0
.../dir_df86748cb94fb6c2fa09e991cce090c0.html | 0
.../dir_e2e6d80cc57daae810c80cbdae687e33.html | 0
.../dir_e502304d8bc7f1ef2456a474c6d92acd.html | 0
.../dir_e6c126a997181663ba81c11cbf416bb1.html | 0
.../dir_efbcf68973d247bbf15f9eecae7f24e3.html | 0
.../dir_fe647384fff178c2cff5ce75fb4044e5.html | 0
docs/{rc => v1.20.0}/distribution_8sql__in.html| 0
docs/{rc => v1.20.0}/doc.png | Bin
docs/{rc => v1.20.0}/dot_inline_dotgraph_1.svg | 0
docs/{rc => v1.20.0}/doxygen.css | 0
svn commit: r56124 - /dev/madlib/1.20.0.RC1/
Author: okislal Date: Thu Aug 4 11:54:48 2022 New Revision: 56124 Log: Removing 1.20.0 RC1 artifacts Removed: dev/madlib/1.20.0.RC1/
svn commit: r56123 - /release/madlib/KEYS
Author: okislal Date: Thu Aug 4 11:17:19 2022 New Revision: 56123 Log: Update KEYS for 1.20.0 release Modified: release/madlib/KEYS Modified: release/madlib/KEYS == --- release/madlib/KEYS (original) +++ release/madlib/KEYS Thu Aug 4 11:17:19 2022 @@ -707,6 +707,44 @@ gWTRrr7ATrcLQfWhrjPg/klXfvQ+6OCy0EQJA7Ok Qw== =xsMj -END PGP PUBLIC KEY BLOCK- +pub rsa2048 2021-12-13 [SC] + 1A870220A22E3BE4FA1B78C4CA259C4DEBB32422 +uid [ultimate] Orhan Kislal (Code Signing Key Dec 2021) +sig 3CA259C4DEBB32422 2021-12-13 Orhan Kislal (Code Signing Key Dec 2021) +sub rsa2048 2021-12-13 [E] +sig CA259C4DEBB32422 2021-12-13 Orhan Kislal (Code Signing Key Dec 2021) + +-BEGIN PGP PUBLIC KEY BLOCK- + +mQENBGG33IABCADnkXCHSx3Vy5gXB4mSHZM4MxdsDBIBJlKV5+ZUSSH1UUWmXH0p +bDDNou1rbsqBdtKkA5P+H1M84rrACiX5mh+cmpWMMmwRu9rciUJR6IxuRIv0LYvG +LgAKcT1uFMeGQg9QBeSJRTnulKkPab/ELJeuAmNJywndAQpeUDm1dkh3mMp1WuiT +fVFKjBj2xhZ+4jHpaVK6Hzx+1dDX55X06GTSyMOBV1uGSrz7lq6RfycfiSwimq1M +pmfUsf1gEO9/fqnkyNaGmk7DxKEcK7HkuzILsRCCj9hgOqZsl8YZWHaN50HTGP0K +pf7KsEiCW4SaHIAXHMQX7lAicZnqSRtMZVHtABEBAAG0PU9yaGFuIEtpc2xhbCAo +Q29kZSBTaWduaW5nIEtleSBEZWMgMjAyMSkgPG9raXNsYWxAdm13YXJlLmNvbT6J +AVIEEwEIADwWIQQahwIgoi475PobeMTKJZxN67MkIgUCYbfcgAIbAwULCQgHAgMi +AgEGFQoJCAsCBBYCAwECHgcCF4AACgkQyiWcTeuzJCIjsgf+Mv6+aj+o9UExx3kt +AWux5cKTy43kFcIjzFtgExSdQu8VHU5OMHVmTrHbyXsMFok4syEELbY7ijXZ621B +G6/fLYygzzFcy4dh63vb9bGi8is9Ojji2Qcc/zVkMVRdsxXFzMj6tDI5o4wMLRQ0 +N5U5xRZk5qmh0pcKv0edhatKLmMYRbNQ8gdyOd0PxKXkH3W1hsOVbuLaIy1w4GiI +cEntJbRG/OSzW6LqVwxRuMbuQtno739ubv6qMsexCMgh50YM8K2P3O0RDLwHgkg8 +FLld44ChDC7zg+hzBk1//GrdrnnQZFOYf9iF4og2s+AbCoSMiUvohLUokf25gPcc +y1mvybkBDQRht9yAAQgAtU1kxCk4Shvi/rcS8YWxNCLHuONdxACwrIPa8Kh9Q7St +N+SDC6DdaoxnBtzJUB4MOW6PkbZaQGK3PkOs/iMEWx6ZkrwmDeBL51kd57xqpcgz +YOSQYHJ4taOGRwXic0XwtN+JXQhY2xS1dSbXgQHPnIJJv8PHE7+A+nIr/Yf5gk3Q +WHi2BCYSVnzkJ46KYC24qD7D8EQt4KGCHoqVMpKiG19sRzbHjZOA8QFYfnKAgCqH +aKfUsaVA/3pJhbw4XZeYVEbKDSc5XY0s0UGqqMeLDnjpN3A3rGeXwr4GmXyy21sr +UUsSPvfqGU3m1Cq3jww+HtbdbLEngxwDKavCqfMzAwARAQABiQE2BBgBCAAgFiEE +GocCIKIuO+T6G3jEyiWcTeuzJCIFAmG33IACGwwACgkQyiWcTeuzJCJh2gf7B7I+ +tDb2lVFDFVRDfIpwWBEUcTOyZEH2Mjy0TN06wc1uVXbQ2GVqz3MQYaRM/gBr5lpt +oUBdKr3jaIz2ApY2igSsg0wD0MsXEYGoXUKochG+pw181euYM8CiPCzll+Zud2NP +jLfGoQzLWBvHvirYgYJVZOPmudrDd7iKoSUDzHrgCHKML7Lve3wGqiYuBNyc6wv2 +dmidqOLDyay4sNc6maYghmRf9tRuJ8wL9NS00602JoEb5jxWZFifm4OZzotvPfT7 +gwOAEyG/RbtVJ0JULAvHfEtH/EUvdFOrKahfCRtpU2ukZOPie/IjL2kFWf4GhjlA +/YvtGBljYEFDKjQMwQ== +=GfRx +-END PGP PUBLIC KEY BLOCK- pub ed25519 2021-07-26 [SC] [expires: 2023-07-26] 166F92BC36CCA2D90C98ED91CB1A73B0AE4E55A4 uid [ultimate] Orhan Kislal @@ -727,3 +765,37 @@ K3GivgGU3eEoVQMBCAeIfgQYFgoAJhYhBBZvkrw2 vByyFZSe6tw0rW9UAP9+wJnxYOYqlI6FZ7+biGUTM/Iytelpkygt9rC5+QyQDA== =sF3T -END PGP PUBLIC KEY BLOCK- +pub rsa4096 2022-07-26 [SC] [expires: 2024-07-25] + FE43DE7208EA2EC04DF3B50DF96F3BD09B0DF688 +uid [ultimate] Venkatesh Raghavan +sig 3F96F3BD09B0DF688 2022-07-26 Venkatesh Raghavan + +-BEGIN PGP PUBLIC KEY BLOCK- + +mQINBGLgNN4BEAComvMi0+DfxpNyuCJTvY556xebWOcmXV3QluqpI/YnMr2QLwIT +5asUIvh2wUk0EVpRQaC5GsXc/PmWUCIj6kft7A8xZ9ixEZs4AUhubMu4G8ITkHva +Yw3wJcJ3yRJ5BDAFq5+tiTefc/sLtj74dkutLBbyPDDOB2H5r5b15Yai/ajU4ULN +hI6d7kMJjxZ8ew7RAmYiYhqRpXx5ojj4hrL/gmvwy9TbiOc81/gAXmzrZR8eRQgM +ZAU9SeOHp3oPzQdHQLY3BX/Pjb+3h6WxlmvAADczCdL6pOwZMyGH2bMc/LmLlFHw +iKZvFBecr9dIref14eCqnvpFbXa2XPnDJ/JkpIB9uclaT4uZb7SgiRo89W9SunzT +XPZdcZ3KN8R8l2LnMIiIVv0fzPvPR5v9WBc+3IoVwkpK78ZkhG3aoMGLX3ftEOsc +Sc3kvdYTh0eVcYW3TnF+ZHDtLlRIfYkARQgky/fXtte1OgKTOXApCmrIjs8GZ8SB +dU0pi6cugP3LPNqz/9TXy3osh8lmklJjA5zVs95Y83Yj6kLR1C8SFOwdQ13cNvtq +TfC7svQ+VrAz1w4snAlRmUbXtHvLayQ5mHvObGfec11CYKlPEUlQOwZEmJ4s5OOT +OxaVnJcQcW63IzjbWwIoRrdKCYMR7iLrtR4uP3Nzg+ivgjW2FMi0ntx1qQARAQAB +tCpWZW5rYXRlc2ggUmFnaGF2YW4gPHJhZ2hhdmFudmVAdm13YXJlLmNvbT6JAlcE +EwEIAEEWIQT+Q95yCOouwE3ztQ35bzvQmw32iAUCYuA03gIbAwUJA8JnAAULCQgH +AgIiAgYVCgkICwIEFgIDAQIeBwIXgAAKCRD5bzvQmw32iE5JD/0W6/QMLgyjGr4d +Ye5RGwMj2AqQVyFDEvHsPzBnU/arJ2eReYJlJCyLQCyIqyUyI5JiWFzuhePbktHa +ikYkbLoZxPGsdbjK5QdWmIYydHGXlNjy0+FuAasvur4EFYmPY3/XX40leJplWkPb +sCfBuL7mCKLzn5o+Dsdiyzs1Uvf1Osf3oj40fNyg96BN+FjIhILdxJ6rcUlQQiRc +dBouvWSxT61qtWCDfLI71ptITolbfSwaG8OrL6B40X4KdvClhlMSPBcr3mide+TX +HmJom1iQMupkJv2WNa2c32sgNAsDvJ4/K5DI5NX7WE/lcbGLBYS+dF3VzzHCeVby +4tunWb7tWSTLvoH9pszIiSyI7fKBW1qDGD7d3dBy0NnheGsjt/NlDWsphPwcpZUU +N1hEWZv1zdOugeiA7Gfpmn1hdvzWQvcHigRXsgHoFEeebe8i5mg6nS3xtsxMqLPi +IWVhoAiAV7TvF6pig3LkNIb4D16cZKvBtmNXHXBh9vztRtPcNkxcYEcvetwpcAfw +QojqZJwbUFIzcH3TOXuNPdg4jFWBE34E5CGGnUfTJy7RattajziQL5O26cyta73T +oP+vPXKF5xjhHWBfC6PCNDu+JwtB4UhnZ05GkHlbUpAXXxJWDkNSAKG7ILC9FT0I +J9oa8m5x+PBIZQK5ZWvRivyOB9Cn3A== +=8F1+ +-END PGP PUBLIC KEY BLOCK-
svn commit: r56122 - /dev/madlib/1.20.0.RC2/ /release/madlib/1.20.0/
Author: okislal Date: Thu Aug 4 11:14:13 2022 New Revision: 56122 Log: (empty) Added: release/madlib/1.20.0/ - copied from r56121, dev/madlib/1.20.0.RC2/ Removed: dev/madlib/1.20.0.RC2/
[madlib] annotated tag rc/1.20.0-rc2 updated (f9761f49 -> 041809be)
This is an automated email from the ASF dual-hosted git repository. okislal pushed a change to annotated tag rc/1.20.0-rc2 in repository https://gitbox.apache.org/repos/asf/madlib.git *** WARNING: tag rc/1.20.0-rc2 was modified! *** from f9761f49 (commit) to 041809be (tag) tagging f9761f497bf29ae52f1406d4bd7b046d2726b6ab (commit) replaces rel/v1.19.0 by Orhan Kislal on Thu Jul 28 14:43:26 2022 +0300 - Log - Apache MADlib 1.20.0 RC2 -BEGIN PGP SIGNATURE- iQEzBAABCAAdFiEEGocCIKIuO+T6G3jEyiWcTeuzJCIFAmLidl4ACgkQyiWcTeuz JCI71AgAjG5nNgjgiWCC5/mTUUMfixYaLlD8aYSRNf0oZwtKy2qy4bxAocW+cJ1d nVXlBVoyfANx0DyT9UdO9exkXFOhvzETlg+NfZuD0hCH5KoAt4vYLY+oKDY2G6P/ CcHMn/r3SCLnq0oaZsHc4Ci64KX/SJZUm4XpNrh7NX7oZU2zILbrnlpqHXJGSADw nelP58VkkVJX4WeIGcFy8DzSvxfzqP5wZ7jVi8bXe1+z9+u+8eCEvxyJb0ty2NrO v+xC3nGQzvI9dnyqUgVfAgjcPgc2F+bMHtgue3B8gx7S/v67myQPU0fhu3dKgpWq y8Fp6tFxb5BVB6uPSIv/xNrY4aFLig== =Wt/y -END PGP SIGNATURE- --- No new revisions were added by this update. Summary of changes:
svn commit: r56007 - in /dev/madlib: ./ 1.20.0.RC2/
Author: okislal Date: Thu Jul 28 11:40:19 2022 New Revision: 56007 Log: Adding 1.20.0 RC2 candidate release artifacts and new KEYS file Added: dev/madlib/1.20.0.RC2/ dev/madlib/1.20.0.RC2/apache-madlib-1.20.0-CentOS7.rpm (with props) dev/madlib/1.20.0.RC2/apache-madlib-1.20.0-CentOS7.rpm.asc dev/madlib/1.20.0.RC2/apache-madlib-1.20.0-CentOS7.rpm.sha512 dev/madlib/1.20.0.RC2/apache-madlib-1.20.0-src.tar.gz (with props) dev/madlib/1.20.0.RC2/apache-madlib-1.20.0-src.tar.gz.asc dev/madlib/1.20.0.RC2/apache-madlib-1.20.0-src.tar.gz.sha512 Modified: dev/madlib/KEYS Added: dev/madlib/1.20.0.RC2/apache-madlib-1.20.0-CentOS7.rpm == Binary file - no diff available. Propchange: dev/madlib/1.20.0.RC2/apache-madlib-1.20.0-CentOS7.rpm -- svn:mime-type = application/octet-stream Added: dev/madlib/1.20.0.RC2/apache-madlib-1.20.0-CentOS7.rpm.asc == --- dev/madlib/1.20.0.RC2/apache-madlib-1.20.0-CentOS7.rpm.asc (added) +++ dev/madlib/1.20.0.RC2/apache-madlib-1.20.0-CentOS7.rpm.asc Thu Jul 28 11:40:19 2022 @@ -0,0 +1,16 @@ +-BEGIN PGP SIGNATURE- + +iQIzBAABCAAdFiEE/kPecgjqLsBN87UN+W870JsN9ogFAmLgNdcACgkQ+W870JsN +9ogQgQ/9GBw8gy+Iiz8wwmM9vWv/Opnyph5ogtuuHUslPrFM2YFj3EpSH5PWd5qq +qc5maFy9lG0cGckcnLATd3C008Zpq0dZJoe+ik8HYCwQZ6/nv8R8R5/DktWk48Av +MG94PpCWHh2pWt6FSthGw5XBjzLlI0QuwxhDRf/XUaoK3KY82axLgXPrpBkIBaFV +PEjsR+q1yDQlVZ27cytKHSGbFyxwEnUQAQjGT/jcOVjCwCFkT3XH5C5ZwE88L/di +vPu16aiJemyT5si+ZFVczwhIe7x1/h7ST/P9SiL45tGFhbDk5/PxVXwsuWZwomTj +rY8nJ6aFYc1FNLdmn+c4oL2w95Sm1Pdme8XfasGFvti9N7Pnmhl5v/5jFjVr08pk +IRxeWFTsRmgtl/lSNP+20BACxua/xtvtWhdB7efpP/5jRW+BDcj8cxHZNQDaHDBT +lHmEjeP1CZon/Ve8bhD0liZuIpymNMdCEVLchVFrp+lGJz58DQ3srz9IL6QBZk5E +uD5+hI2lJbcg0ynuXIqFVtGcXnpG/+QE7XbEEVvmR5E/RcleD/7hfSBhWyLaYxV7 +tD40UuAk/w0ASCg7WNpLcFXu9SAN7g2GKdm0eb5UMKG/gFEg0lGw/cmLSxtqqHVb +SsixteYVI921HHc63h1cP8Hbv36+KM+lNYeejpwZSCx/wPI9TFI= +=q+R4 +-END PGP SIGNATURE- Added: dev/madlib/1.20.0.RC2/apache-madlib-1.20.0-CentOS7.rpm.sha512 == --- dev/madlib/1.20.0.RC2/apache-madlib-1.20.0-CentOS7.rpm.sha512 (added) +++ dev/madlib/1.20.0.RC2/apache-madlib-1.20.0-CentOS7.rpm.sha512 Thu Jul 28 11:40:19 2022 @@ -0,0 +1 @@ +32bd3d0cc78384f5fd74ed2273b47784f9b6ca13ab24310718dd4cd3e851565374c1008ac9d9789de1c64a306bef6ece6f70cfcd596bab89a9c037a55487da0d apache-madlib-1.20.0-CentOS7.rpm Added: dev/madlib/1.20.0.RC2/apache-madlib-1.20.0-src.tar.gz == Binary file - no diff available. Propchange: dev/madlib/1.20.0.RC2/apache-madlib-1.20.0-src.tar.gz -- svn:mime-type = application/octet-stream Added: dev/madlib/1.20.0.RC2/apache-madlib-1.20.0-src.tar.gz.asc == --- dev/madlib/1.20.0.RC2/apache-madlib-1.20.0-src.tar.gz.asc (added) +++ dev/madlib/1.20.0.RC2/apache-madlib-1.20.0-src.tar.gz.asc Thu Jul 28 11:40:19 2022 @@ -0,0 +1,16 @@ +-BEGIN PGP SIGNATURE- + +iQIzBAABCAAdFiEE/kPecgjqLsBN87UN+W870JsN9ogFAmLgNckACgkQ+W870JsN +9ohNgg//dabRpgtF5as5wKUDWAzsmGhUEdUM/mP9jiJOqJVueq09rfaJwnedoUqf +Hn8JJQeyT+1mkLulHoHrvq7sFXQ0RHu372YnsMy95R+PZw7dhxVJU2F+z0Jr5/nK ++ok8DhHlIkmggKbD8GHzcHIyoJNKJAHymRgXB95wV7G0JoxcNPcy5D0YbWFMk7Ke +Mpj3nuAE8GdD0+UwzbJ5M0WJMINiwk8dRdGnJ0WyD9S3RkFVleVny2OiLPUU/opd +loOmc/Plqhfc4sObet1x3T/aunbfB/KJD1VRhZaYdaImOLezBKB9c2F1Gd7uMQK1 +791HlLcuggElNLSoQXsN5W3kM4eCoNaW/AEZABOVIqrGysSkiMB0SfJxBy7MsrAB +13IKeRvVF7JokBPE4BRS8uhE7oQIZYdD/gBal3DPI8vwBeUvQFUxvQ6QeNIdY9+g +p1smMp3QMdz8foWWA5JFgIVgz+xRcjJEmGMmANDYVyO6lFo+8NT4+gpZQMKIBxu6 +yfsHRJOSwlQAh1SHy+4qlD+fDNIr2yyjlBq7BVgiqbxVyBR7oRUzLJbph9pMStxv +vY4qBXlh3yQH2seQCIi78VfDVimHddan4UEvqDTuPZslOSzBurjkWhKmsZxO2eil +tBi2+wCoJ9+YumxQ9YiZOBiaevd9wMvaR0FrvL4nrcC/K8AIxMI= +=ca/Y +-END PGP SIGNATURE- Added: dev/madlib/1.20.0.RC2/apache-madlib-1.20.0-src.tar.gz.sha512 == --- dev/madlib/1.20.0.RC2/apache-madlib-1.20.0-src.tar.gz.sha512 (added) +++ dev/madlib/1.20.0.RC2/apache-madlib-1.20.0-src.tar.gz.sha512 Thu Jul 28 11:40:19 2022 @@ -0,0 +1 @@ +462577c67957cfb6ddc28791cd55e70070d37338e2afd832306419c6990ab0843a530a3e5739351ca7ce1f4cca036ea5bd7eccfd0c31a3d003dc4441243ab554 apache-madlib-1.20.0-src.tar.gz Modified: dev/madlib/KEYS == --- dev/madlib/KEYS (original) +++ dev/madlib/KEYS Thu Jul 28 11:40:19 2022 @@ -765,3 +765,37 @@ K3GivgGU3eEoVQMBCAeIfgQYFgoAJhYhBBZvkrw2 vByyFZSe6tw0rW9UAP9+wJnxYOYqlI6FZ7+biGUTM/Iytelpkygt9rC5+QyQDA== =sF3T -END PGP
svn commit: r55820 - in /dev/madlib: 1.20.0.RC1/ 1.20.0.RC1/apache-madlib-1.20.0-src.tar.gz 1.20.0.RC1/apache-madlib-1.20.0-src.tar.gz.asc 1.20.0.RC1/apache-madlib-1.20.0-src.tar.gz.sha512 KEYS
Author: okislal Date: Mon Jul 18 13:02:07 2022 New Revision: 55820 Log: adding 1.20.0 RC1 candidate release artifacts Added: dev/madlib/1.20.0.RC1/ dev/madlib/1.20.0.RC1/apache-madlib-1.20.0-src.tar.gz (with props) dev/madlib/1.20.0.RC1/apache-madlib-1.20.0-src.tar.gz.asc dev/madlib/1.20.0.RC1/apache-madlib-1.20.0-src.tar.gz.sha512 Modified: dev/madlib/KEYS Added: dev/madlib/1.20.0.RC1/apache-madlib-1.20.0-src.tar.gz == Binary file - no diff available. Propchange: dev/madlib/1.20.0.RC1/apache-madlib-1.20.0-src.tar.gz -- svn:mime-type = application/octet-stream Added: dev/madlib/1.20.0.RC1/apache-madlib-1.20.0-src.tar.gz.asc == --- dev/madlib/1.20.0.RC1/apache-madlib-1.20.0-src.tar.gz.asc (added) +++ dev/madlib/1.20.0.RC1/apache-madlib-1.20.0-src.tar.gz.asc Mon Jul 18 13:02:07 2022 @@ -0,0 +1,7 @@ +-BEGIN PGP SIGNATURE- + +iHUEABYKAB0WIQQWb5K8Nsyi2QyY7ZHLGnOwrk5VpAUCYtVZZAAKCRDLGnOwrk5V +pDvlAP48VtAG+4eM5NmdOHowr4vJKu1yf1v3777v7O2/JKpEuwEAj5uocjYLdqx9 +2dvjk7ZqIQTICN8Vhr587jzJwaKo+gs= +=7pgd +-END PGP SIGNATURE- Added: dev/madlib/1.20.0.RC1/apache-madlib-1.20.0-src.tar.gz.sha512 == --- dev/madlib/1.20.0.RC1/apache-madlib-1.20.0-src.tar.gz.sha512 (added) +++ dev/madlib/1.20.0.RC1/apache-madlib-1.20.0-src.tar.gz.sha512 Mon Jul 18 13:02:07 2022 @@ -0,0 +1 @@ +462577c67957cfb6ddc28791cd55e70070d37338e2afd832306419c6990ab0843a530a3e5739351ca7ce1f4cca036ea5bd7eccfd0c31a3d003dc4441243ab554 apache-madlib-1.20.0-src.tar.gz Modified: dev/madlib/KEYS == --- dev/madlib/KEYS (original) +++ dev/madlib/KEYS Mon Jul 18 13:02:07 2022 @@ -745,3 +745,23 @@ gwOAEyG/RbtVJ0JULAvHfEtH/EUvdFOrKahfCRtp /YvtGBljYEFDKjQMwQ== =GfRx -END PGP PUBLIC KEY BLOCK- +pub ed25519 2021-07-26 [SC] [expires: 2023-07-26] + 166F92BC36CCA2D90C98ED91CB1A73B0AE4E55A4 +uid [ultimate] Orhan Kislal +sig 3CB1A73B0AE4E55A4 2021-07-26 Orhan Kislal +sub cv25519 2021-07-26 [E] [expires: 2023-07-26] +sig CB1A73B0AE4E55A4 2021-07-26 Orhan Kislal + +-BEGIN PGP PUBLIC KEY BLOCK- + +mDMEYP8m8BYJKwYBBAHaRw8BAQdANGa1Obo0IS7BpXOVvn3PcoV4As5xxTAlb8/A +VKwupMO0IU9yaGFuIEtpc2xhbCA8b2tpc2xhbEB2bXdhcmUuY29tPoiaBBMWCgBC +FiEEFm+SvDbMotkMmO2RyxpzsK5OVaQFAmD/JvACGwMFCQPCZwAFCwkIBwIDIgIB +BhUKCQgLAgQWAgMBAh4HAheAAAoJEMsac7CuTlWkjQYBAMWrkp6Vzcn5MPw2W/AQ +w0krZtgBaCLvxRQ/JKv0psmlAP43W9Um+/A80nEWTggajtea5uA0k5j/ukEZWUEv +V+QVALg4BGD/JvASCisGAQQBl1UBBQEBB0BzKMhSNNN2H4qu6KdNAsgNdHvxuGPX +K3GivgGU3eEoVQMBCAeIfgQYFgoAJhYhBBZvkrw2zKLZDJjtkcsac7CuTlWkBQJg +/ybwAhsMBQkDwmcAAAoJEMsac7CuTlWkXjYBAKPnTwLvrk7dAMEQdZpT3CzJ5mvT +vByyFZSe6tw0rW9UAP9+wJnxYOYqlI6FZ7+biGUTM/Iytelpkygt9rC5+QyQDA== +=sF3T +-END PGP PUBLIC KEY BLOCK-
[madlib] annotated tag rc/1.20.0-rc1 updated (f9761f49 -> 603b9b5d)
This is an automated email from the ASF dual-hosted git repository. okislal pushed a change to annotated tag rc/1.20.0-rc1 in repository https://gitbox.apache.org/repos/asf/madlib.git *** WARNING: tag rc/1.20.0-rc1 was modified! *** from f9761f49 (commit) to 603b9b5d (tag) tagging f9761f497bf29ae52f1406d4bd7b046d2726b6ab (commit) replaces rel/v1.19.0 by Orhan Kislal on Mon Jul 18 15:54:32 2022 +0300 - Log - Apache MADlib 1.20.0 RC1 -BEGIN PGP SIGNATURE- iQEzBAABCAAdFiEEGocCIKIuO+T6G3jEyiWcTeuzJCIFAmLVWAgACgkQyiWcTeuz JCLzvAf+N0dJxtwI8v84zRl/s+Vo4AlM61ZnS5fxFqQwDmGCtjBJAyGJaB9/5elM DTOhAR+5ffpSoNvFDBDXkEJDudR/JA6+LyME7BdSpHuqtyMbC8b6js3DcDXiYuhI NJtQtU7e45f9UvEvvKVAV4OyrOh8npGeuIqo2LOrORP2xdroge4fegqDrDdk63nw UCM3IF+wHadUN734NQ+CQfoEowDPQGktHbhxrNTQcXNx7SS3mWMPCI7HgQOLJwUt KNIhkLbztlnqrE+zupWCOkrydHS5cKvRUS7x7P1ZfSz3QbbcNeXh37ltAlAkkOCY Jy242U+yfsEd/8A+txtpPTejk9AOMg== =avtY -END PGP SIGNATURE- --- No new revisions were added by this update. Summary of changes:
[madlib] 02/03: Build: Update version number to 1.20.0
This is an automated email from the ASF dual-hosted git repository.
okislal pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/madlib.git
commit 05d0a9bd64cb8d421af4ce96971fa9520ebc8707
Author: Orhan Kislal
AuthorDate: Wed Jul 6 15:45:14 2022 +0300
Build: Update version number to 1.20.0
---
deploy/DEB/postinst| 2 +-
deploy/postflight.sh | 2 +-
pom.xml| 2 +-
src/config/Version.yml | 2 +-
src/madpack/changelist_1.19.0_1.20.0.yaml | 61 ++
.../modules/deep_learning/madlib_keras.sql_in | 8 +--
.../deep_learning/madlib_keras_automl.sql_in | 4 +-
.../madlib_keras_fit_multiple_model.sql_in | 6 +--
src/ports/postgres/modules/svm/svm.sql_in | 2 +-
9 files changed, 75 insertions(+), 14 deletions(-)
diff --git a/deploy/DEB/postinst b/deploy/DEB/postinst
index da5fcf3c..3a564f33 100755
--- a/deploy/DEB/postinst
+++ b/deploy/DEB/postinst
@@ -22,7 +22,7 @@
# Source debconf library.
. /usr/share/debconf/confmodule
-MADLIB_VERSION="1.20.0-dev"
+MADLIB_VERSION="1.20.0"
MADLIB_INSTALL_PATH="InstallPathNotFound"
# Fetching configuration from debconf
diff --git a/deploy/postflight.sh b/deploy/postflight.sh
index 8e053804..39056568 100755
--- a/deploy/postflight.sh
+++ b/deploy/postflight.sh
@@ -2,7 +2,7 @@
# $0 - Script Path, $1 - Package Path, $2 - Target Location, and $3 - Target
Volume
-MADLIB_VERSION=1.20.0-dev
+MADLIB_VERSION=1.20.0
# Remove existing soft links
find $2/usr/local/madlib/bin -depth -type l -exec rm {} \; 2>/dev/null
diff --git a/pom.xml b/pom.xml
index a66ade40..a4041793 100644
--- a/pom.xml
+++ b/pom.xml
@@ -22,7 +22,7 @@
org.apache.madlib
madlib
- 1.20.0-dev
+ 1.20.0
pom
diff --git a/src/config/Version.yml b/src/config/Version.yml
index 7bd62100..d332d9ee 100644
--- a/src/config/Version.yml
+++ b/src/config/Version.yml
@@ -1 +1 @@
-version: 1.20.0-dev
+version: 1.20.0
diff --git a/src/madpack/changelist_1.19.0_1.20.0.yaml
b/src/madpack/changelist_1.19.0_1.20.0.yaml
new file mode 100644
index ..268b78dd
--- /dev/null
+++ b/src/madpack/changelist_1.19.0_1.20.0.yaml
@@ -0,0 +1,61 @@
+#
--
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+#
--
+
+# Changelist for MADlib version rel/v1.19.0 to release-120-prep
+
+# This file contains all changes that were introduced in a new version of
+# MADlib. This changelist is used by the upgrade script to detect what objects
+# should be upgraded (while retaining all other objects from the previous
version)
+
+# New modules (actually .sql_in files) added in upgrade version
+# For these files the sql_in code is retained as is with the functions in the
+# file installed on the upgrade version. All other files (that don't have
+# updates), are cleaned up to remove object replacements
+new module:
+madlib_xgboost:
+
+# Changes in the types (UDT) including removal and modification
+udt:
+
+# List of the UDF changes that affect the user externally. This includes change
+# in function name, return type, argument order or types, or removal of
+# the function. In each case, the original function is as good as removed and a
+# new function is created. In such cases, we should abort the upgrade if there
+# are user views dependent on this function, since the original function will
+# not be present in the upgraded version.
+udf:
+- graph_wcc_reachable_vertices:
+rettype: void
+argument: text, text, text
+- graph_wcc_vertex_check:
+rettype: void
+argument: text, text, text
+
+# Changes to aggregates (UDA) including removal and modification
+# Overloaded functions should be mentioned separately
+uda:
+
+# List of the UDC, UDO and UDOC changes.
+udc:
+
+# Changes in the operators (UDO)
+udo:
+
+# Changes in the operator classes (UDOC)
+udoc:
diff --git a/src/ports/postgres/modules/deep_learning/madlib_ker
[madlib] 03/03: Update RELEASE_NOTES
This is an automated email from the ASF dual-hosted git repository. okislal pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/madlib.git commit f9761f497bf29ae52f1406d4bd7b046d2726b6ab Author: Orhan Kislal AuthorDate: Wed Jul 6 15:57:35 2022 +0300 Update RELEASE_NOTES --- RELEASE_NOTES | 17 - 1 file changed, 16 insertions(+), 1 deletion(-) diff --git a/RELEASE_NOTES b/RELEASE_NOTES index a9457ccb..c9b8a68c 100644 --- a/RELEASE_NOTES +++ b/RELEASE_NOTES @@ -9,6 +9,22 @@ commit history located at https://github.com/apache/madlib/commits/master. Current list of bugs and issues can be found at https://issues.apache.org/jira/browse/MADLIB. +—- +MADlib v1.20.0: + +Release Date: 2022-July-XX + +New features: +- XGBoost: Python based XGBoost with single and grid search executions (MADLIB-1425, MADLIB-1490) +- Graph: Add multicolumn support for WCC and Pagerank (MADLIB-1502, MADLIB-1503) + +Improvements: +- Utilities: Reuse update plan in GroupIterationController +- Documentation: Update online examples for various modules + +Bug fixes: +- Elastic Net - GLM - SVM: Adjust ORCA to reduce planning time + —- MADlib v1.19.0: @@ -22,7 +38,6 @@ before upgrading. New features: - DBSCAN: Fast parallel-optimized DBSCAN (MADLIB-1017, MADLIB-867) - MLP: Add rmsprop and Adam optimization techniques (MADLIB-1434, MADLIB-1435) -- Build: Add OSX tarball for release (MADLIB-1493) Improvements: - Graph: Improve WCC subtx count and catalog entry frequency (MADLIB-1492)
[madlib] branch master updated (8a24663c -> f9761f49)
This is an automated email from the ASF dual-hosted git repository.
okislal pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/madlib.git
from 8a24663c Graph: Add multi column support for Pagerank and WCC
new 954d0f78 Fix create_changelist udoc command
new 05d0a9bd Build: Update version number to 1.20.0
new f9761f49 Update RELEASE_NOTES
The 3 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
RELEASE_NOTES | 17 -
deploy/DEB/postinst | 2 +-
deploy/postflight.sh| 2 +-
pom.xml | 2 +-
src/config/Version.yml | 2 +-
...t_1.15.1_1.16.yaml => changelist_1.19.0_1.20.0.yaml} | 12
src/madpack/create_changelist.py| 2 +-
.../postgres/modules/deep_learning/madlib_keras.sql_in | 8
.../modules/deep_learning/madlib_keras_automl.sql_in| 4 ++--
.../madlib_keras_fit_multiple_model.sql_in | 6 +++---
src/ports/postgres/modules/svm/svm.sql_in | 2 +-
11 files changed, 39 insertions(+), 20 deletions(-)
copy src/madpack/{changelist_1.15.1_1.16.yaml =>
changelist_1.19.0_1.20.0.yaml} (89%)
[madlib] 01/03: Fix create_changelist udoc command
This is an automated email from the ASF dual-hosted git repository.
okislal pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/madlib.git
commit 954d0f78b915b687763e6609e4ded35e5786cbaf
Author: Orhan Kislal
AuthorDate: Wed Jul 6 15:45:03 2022 +0300
Fix create_changelist udoc command
---
src/madpack/create_changelist.py | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/src/madpack/create_changelist.py b/src/madpack/create_changelist.py
index 74bea42a..67ccdd39 100644
--- a/src/madpack/create_changelist.py
+++ b/src/madpack/create_changelist.py
@@ -300,7 +300,7 @@ try:
os.system("""psql {database} -c "SELECT get_udocs('__tmp__madlib__',
'madlib_old_vers', '{t}')" > /dev/null """.format(**locals()))
-os.system("""psql {database} -x -c "SELECT opfamily_name, index_method
FROM __tmp__madlib__ ORDER BY name DESC" > /tmp/madlib_tmp_typedep_udoc.txt
""".format(**locals()))
+os.system("""psql {database} -x -c "SELECT opfamily_name, index_method
FROM __tmp__madlib__ ORDER BY opfamily_name DESC" >
/tmp/madlib_tmp_typedep_udoc.txt """.format(**locals()))
os.system("""psql {database} -c "DROP TABLE IF EXISTS __tmp__madlib__
" > /dev/null """.format(**locals()))
svn commit: r55584 - /release/madlib/KEYS
Author: okislal Date: Thu Jul 7 15:16:28 2022 New Revision: 55584 Log: Update KEYS file Modified: release/madlib/KEYS Modified: release/madlib/KEYS == --- release/madlib/KEYS (original) +++ release/madlib/KEYS Thu Jul 7 15:16:28 2022 @@ -707,3 +707,23 @@ gWTRrr7ATrcLQfWhrjPg/klXfvQ+6OCy0EQJA7Ok Qw== =xsMj -END PGP PUBLIC KEY BLOCK- +pub ed25519 2021-07-26 [SC] [expires: 2023-07-26] + 166F92BC36CCA2D90C98ED91CB1A73B0AE4E55A4 +uid [ultimate] Orhan Kislal +sig 3CB1A73B0AE4E55A4 2021-07-26 Orhan Kislal +sub cv25519 2021-07-26 [E] [expires: 2023-07-26] +sig CB1A73B0AE4E55A4 2021-07-26 Orhan Kislal + +-BEGIN PGP PUBLIC KEY BLOCK- + +mDMEYP8m8BYJKwYBBAHaRw8BAQdANGa1Obo0IS7BpXOVvn3PcoV4As5xxTAlb8/A +VKwupMO0IU9yaGFuIEtpc2xhbCA8b2tpc2xhbEB2bXdhcmUuY29tPoiaBBMWCgBC +FiEEFm+SvDbMotkMmO2RyxpzsK5OVaQFAmD/JvACGwMFCQPCZwAFCwkIBwIDIgIB +BhUKCQgLAgQWAgMBAh4HAheAAAoJEMsac7CuTlWkjQYBAMWrkp6Vzcn5MPw2W/AQ +w0krZtgBaCLvxRQ/JKv0psmlAP43W9Um+/A80nEWTggajtea5uA0k5j/ukEZWUEv +V+QVALg4BGD/JvASCisGAQQBl1UBBQEBB0BzKMhSNNN2H4qu6KdNAsgNdHvxuGPX +K3GivgGU3eEoVQMBCAeIfgQYFgoAJhYhBBZvkrw2zKLZDJjtkcsac7CuTlWkBQJg +/ybwAhsMBQkDwmcAAAoJEMsac7CuTlWkXjYBAKPnTwLvrk7dAMEQdZpT3CzJ5mvT +vByyFZSe6tw0rW9UAP9+wJnxYOYqlI6FZ7+biGUTM/Iytelpkygt9rC5+QyQDA== +=sF3T +-END PGP PUBLIC KEY BLOCK-
[madlib] branch master updated: Graph: Add multi column support for Pagerank and WCC
This is an automated email from the ASF dual-hosted git repository. okislal pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/madlib.git The following commit(s) were added to refs/heads/master by this push: new 8a24663c Graph: Add multi column support for Pagerank and WCC 8a24663c is described below commit 8a24663ccf2ee76ef538c416ed505485b740ff04 Author: Orhan Kislal AuthorDate: Mon Jun 13 20:03:05 2022 -0400 Graph: Add multi column support for Pagerank and WCC JIRA: MADLIB-1502, MADLIB-1503 This commit adds support for identifying vertices with multiple columns for Pagerank and WCC modules. The rest of the graph modules will add the same functionality in later commits. The multi column support is done by creating views in which we collect the given BIGINT columns into a single BIGINT array. Co-authored-by: Bhuvnesh Chaudhary --- doc/example/madlib_pagerank_example.sql| 172 + doc/example/madlib_wcc_example.sql | 170 + .../postgres/modules/dbscan/test/dbscan.sql_in | 4 +- src/ports/postgres/modules/graph/graph_utils.py_in | 21 +- src/ports/postgres/modules/graph/pagerank.py_in| 267 +--- src/ports/postgres/modules/graph/pagerank.sql_in | 270 ++--- .../postgres/modules/graph/test/pagerank.sql_in| 98 +++- src/ports/postgres/modules/graph/test/wcc.sql_in | 101 +++- src/ports/postgres/modules/graph/wcc.py_in | 190 +-- src/ports/postgres/modules/graph/wcc.sql_in| 205 +++- .../postgres/modules/utilities/validate_args.py_in | 4 + 11 files changed, 1226 insertions(+), 276 deletions(-) diff --git a/doc/example/madlib_pagerank_example.sql b/doc/example/madlib_pagerank_example.sql new file mode 100644 index ..a1369c46 --- /dev/null +++ b/doc/example/madlib_pagerank_example.sql @@ -0,0 +1,172 @@ +/* --- *//** + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + * + *//* --- */ + +DROP TABLE IF EXISTS vertex, edge; +CREATE TABLE vertex( +node_id INTEGER +); +CREATE TABLE edge( +conn_src INTEGER, +conn_dest INTEGER, +user_id INTEGER +); +INSERT INTO vertex VALUES +(0), +(1), +(2), +(3), +(4), +(5), +(6); +INSERT INTO edge VALUES +(0, 1, 1), +(0, 2, 1), +(0, 4, 1), +(1, 2, 1), +(1, 3, 1), +(2, 3, 1), +(2, 5, 1), +(2, 6, 1), +(3, 0, 1), +(4, 0, 1), +(5, 6, 1), +(6, 3, 1), +(0, 1, 2), +(0, 2, 2), +(0, 4, 2), +(1, 2, 2), +(1, 3, 2), +(2, 3, 2), +(3, 0, 2), +(4, 0, 2), +(5, 6, 2), +(6, 3, 2); + +DROP TABLE IF EXISTS pagerank_out, pagerank_out_summary; +SELECT madlib.pagerank( + 'vertex',-- Vertex table + 'node_id', -- Vertex id column + 'edge', -- Edge table + 'src=conn_src, dest=conn_dest', -- Comma delimted string of edge arguments + 'pagerank_out'); -- Output table of PageRank +SELECT * FROM pagerank_out ORDER BY pagerank DESC; + +SELECT * FROM pagerank_out_summary; + +DROP TABLE IF EXISTS pagerank_out, pagerank_out_summary; +SELECT madlib.pagerank( + 'vertex',-- Vertex table + 'node_id', -- Vertex id column + 'edge', -- Edge table + 'src=conn_src, dest=conn_dest', -- Comma delimted string of edge arguments + 'pagerank_out', -- Output table of PageRank + 0.5);-- Damping factor +SELECT * FROM pagerank_out ORDER BY pagerank DESC; + +DROP TABLE IF EXISTS pagerank_out, pagerank_out_summary; +SELECT madlib.pagerank( + 'vertex',-- Vertex table + 'node_id', -- Vertex id column +
[madlib] 02/03: SVM: Fix dev-check test typo
This is an automated email from the ASF dual-hosted git repository.
okislal pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/madlib.git
commit 45f533b2c79ea27fd101f00da8b2f0b5d2d9946c
Author: Orhan Kislal
AuthorDate: Thu Jun 9 16:59:54 2022 -0400
SVM: Fix dev-check test typo
---
src/ports/postgres/modules/svm/test/svm.sql_in | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/src/ports/postgres/modules/svm/test/svm.sql_in
b/src/ports/postgres/modules/svm/test/svm.sql_in
index 9dea0bb5..a8224014 100644
--- a/src/ports/postgres/modules/svm/test/svm.sql_in
+++ b/src/ports/postgres/modules/svm/test/svm.sql_in
@@ -108,9 +108,9 @@ SELECT svm_regression(
NULL,
'init_stepsize=0.01, max_iter=50, lambda=2, norm=l2, epsilon=0.01',
false);
-SELECT svm_predict('svr_model', 'svr_train_data', 'id', 'svr_test_result');
+SELECT svm_predict('svr_model2', 'svr_train_data', 'id', 'svr_test_result');
\x on
-SELECT * FROM svr_model;
+SELECT * FROM svr_model2;
\x off
SELECT
assert(
[madlib] 03/03: SVM: Disable ORCA to reduce planning time cost
This is an automated email from the ASF dual-hosted git repository. okislal pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/madlib.git commit 49bb11f04ebfb015f8b4f96947e0e8174929c841 Author: Orhan Kislal AuthorDate: Thu Jun 9 18:46:42 2022 -0400 SVM: Disable ORCA to reduce planning time cost --- src/ports/postgres/modules/svm/svm.py_in | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/src/ports/postgres/modules/svm/svm.py_in b/src/ports/postgres/modules/svm/svm.py_in index ceddb392..6106629f 100644 --- a/src/ports/postgres/modules/svm/svm.py_in +++ b/src/ports/postgres/modules/svm/svm.py_in @@ -7,6 +7,7 @@ from collections import defaultdict from kernel_approximation import create_kernel, load_kernel from utilities.control import MinWarning +from utilities.control import OptimizerControl from utilities.in_mem_group_control import GroupIterationController from utilities.utilities import _assert from utilities.utilities import _string_to_array @@ -815,7 +816,8 @@ def svm(schema_madlib, source_table, model_table, if transformer.transformed_table: args.update(transformer.transformed_table) _cross_validate_svm(args) -_svm_parsed_params(use_transformer_for_output=True, **args) +with OptimizerControl(False): +_svm_parsed_params(use_transformer_for_output=True, **args) transformer.clear() # --
[madlib] branch master updated (ac22af57 -> 49bb11f0)
This is an automated email from the ASF dual-hosted git repository. okislal pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/madlib.git from ac22af57 XGBoost: Add new module xgboost new f37a9292 Utilities: Reuse update plan in GroupIterationController new 45f533b2 SVM: Fix dev-check test typo new 49bb11f0 SVM: Disable ORCA to reduce planning time cost The 3 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: src/ports/postgres/modules/svm/svm.py_in | 5 +- src/ports/postgres/modules/svm/test/svm.sql_in | 4 +- .../modules/utilities/in_mem_group_control.py_in | 68 -- 3 files changed, 42 insertions(+), 35 deletions(-)
[madlib] 01/03: Utilities: Reuse update plan in GroupIterationController
This is an automated email from the ASF dual-hosted git repository.
okislal pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/madlib.git
commit f37a92924d21204a83566f66e2c814ad22b88588
Author: Orhan Kislal
AuthorDate: Thu Jun 9 16:57:33 2022 -0400
Utilities: Reuse update plan in GroupIterationController
The group iteration controller prepares a plan and executes it during
the update phase. For some modules, this plan does not change between
iterations. With this commit, the plan gets saved and reused as needed.
Co-authored-by: Bhuvnesh Chaudhary
---
src/ports/postgres/modules/svm/svm.py_in | 1 +
.../modules/utilities/in_mem_group_control.py_in | 68 --
2 files changed, 37 insertions(+), 32 deletions(-)
diff --git a/src/ports/postgres/modules/svm/svm.py_in
b/src/ports/postgres/modules/svm/svm.py_in
index 1532cb24..ceddb392 100644
--- a/src/ports/postgres/modules/svm/svm.py_in
+++ b/src/ports/postgres/modules/svm/svm.py_in
@@ -62,6 +62,7 @@ def _compute_svm(args):
it.kwargs['stepsize'] *= it.kwargs['decay_factor']
else:
it.kwargs['stepsize'] = init_stepsize / (it.iteration + 1)
+it.update_plan = None
has_converged = it.test(
"""
{iteration} >= {max_iter}
diff --git a/src/ports/postgres/modules/utilities/in_mem_group_control.py_in
b/src/ports/postgres/modules/utilities/in_mem_group_control.py_in
index 68834c04..31f7f374 100644
--- a/src/ports/postgres/modules/utilities/in_mem_group_control.py_in
+++ b/src/ports/postgres/modules/utilities/in_mem_group_control.py_in
@@ -240,6 +240,7 @@ class GroupIterationController:
self.finished_states = state_factory(self.is_state_type_bytea8)
self.group_param = self._init_group_param()
+self.update_plan = None
def _init_group_param(self):
_grp_key = ("array_to_string(ARRAY[{grouping_str}], ',')"
@@ -549,40 +550,43 @@ class GroupIterationController:
self.iteration = self.iteration + 1
group_param = self.group_param
-run_sql = """
-SELECT
-{_grp_key} AS {col_grp_key},
-{grouping_col},
-{iteration} AS {col_grp_iteration},
-({newState}) AS {col_grp_state}
-FROM (
-SELECT *,
-array_to_string(ARRAY[{grouping_str}], ',') AS
{col_grp_key}
-FROM {rel_source}
-) AS {as_rel_source}
-JOIN ( {select_rel_state} ) AS {rel_state}
-{using_str}
-JOIN ( {select_n_tuples} ) AS _rel_n_tuples
-{using_str}
-{groupby_str}
-""".format(
-newState=newState,
-iteration=self.iteration,
-using_str=group_param.using_str,
-groupby_str=group_param.groupby_str,
-_grp_key=group_param.grp_key,
-select_rel_state=group_param.select_rel_state,
-select_n_tuples=group_param.select_n_tuples,
-**self.kwargs)
-update_plan = plpy.prepare(run_sql,
- ["text[]", group_param.grouped_state_type,
-"text[]", "integer[]"])
-res_tuples = plpy.execute(update_plan, [self.new_states.keys,
-self.new_states.values,
-self.grp_to_n_tuples.keys(),
-self.grp_to_n_tuples.values()])
+if self.update_plan is None:
+run_sql = """
+SELECT
+{_grp_key} AS {col_grp_key},
+{grouping_col},
+({newState}) AS {col_grp_state}
+FROM (
+SELECT *,
+array_to_string(ARRAY[{grouping_str}], ',') AS
{col_grp_key}
+FROM {rel_source}
+) AS {as_rel_source}
+JOIN ( {select_rel_state} ) AS {rel_state}
+{using_str}
+JOIN ( {select_n_tuples} ) AS _rel_n_tuples
+{using_str}
+{groupby_str}
+""".format(
+newState=newState,
+iteration=self.iteration,
+using_str=group_param.using_str,
+groupby_str=group_param.groupby_str,
+_grp_key=group_param.grp_key,
+select_rel_state=group_param.select_rel_state,
+select_n_tuples=group_param.select_n_tuples,
+**self.kwargs)
+
+self.update_plan = plpy.prepare(run_sql,
+ ["text[]",
group_param.grouped_st
[madlib] branch master updated (bdf8cae0 -> ac22af57)
This is an automated email from the ASF dual-hosted git repository.
okislal pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/madlib.git
from bdf8cae0 Doc: Update online examples for various model selection
modules
new 011bddf9 Update gitignore to exclude /cmake-build-debug
new 9f1c5029 Add users for docker commands
new 4a98bc47 Fix jenkins
new ac22af57 XGBoost: Add new module xgboost
The 4 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
.gitignore | 1 +
CMakeLists.txt | 1 -
doc/CMakeLists.txt | 6 +
.../example/madlib_xgboost_example.sql | 121 +++-
doc/mainpage.dox.in| 1 +
src/config/Modules.yml | 2 +
.../postgres/modules/mxgboost}/__init__.py_in | 0
.../postgres/modules/mxgboost/madlib_xgboost.py_in | 601 +
.../modules/mxgboost/madlib_xgboost.sql_in | 720 +
.../modules/mxgboost/test/madlib_xgboost.sql_in| 119 +++-
tool/docker_start.sh | 4 +-
tool/jenkins/jenkins_build.sh | 4 +-
12 files changed, 1513 insertions(+), 67 deletions(-)
copy methods/sketch/src/pg_gp/sql/mfv.sql_in =>
doc/example/madlib_xgboost_example.sql (98%)
copy {methods/sketch/src/pg_gp =>
src/ports/postgres/modules/mxgboost}/__init__.py_in (100%)
create mode 100644 src/ports/postgres/modules/mxgboost/madlib_xgboost.py_in
create mode 100644 src/ports/postgres/modules/mxgboost/madlib_xgboost.sql_in
copy methods/sketch/src/pg_gp/sql/mfv.sql_in =>
src/ports/postgres/modules/mxgboost/test/madlib_xgboost.sql_in (98%)
[madlib] 02/04: Add users for docker commands
This is an automated email from the ASF dual-hosted git repository.
okislal pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/madlib.git
commit 9f1c50291846a6710261873d1475204c95a5ccd3
Author: Orhan Kislal
AuthorDate: Tue May 17 16:03:26 2022 +0300
Add users for docker commands
---
tool/docker_start.sh | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/tool/docker_start.sh b/tool/docker_start.sh
index bd193e43..bb7bd7f6 100755
--- a/tool/docker_start.sh
+++ b/tool/docker_start.sh
@@ -83,13 +83,13 @@ docker exec "${CONTAINER_NAME}" bash -c "rm -rf
/madlib/build_docker; \
echo "-- Installing and running install-check "
# Install MADlib and run install check
-docker exec "${CONTAINER_NAME}" bash -c "PATH=$PATH:/usr/lib/postgresql/10/bin
\
+docker exec --user=postgres "${CONTAINER_NAME}" bash -c
"PATH=$PATH:/usr/lib/postgresql/10/bin \
/madlib/build_docker/src/bin/madpack -p postgres \
-c
postgres/postgres@localhost:5432/postgres \
install" \
| tee "${workdir}/build_docker_logs/madlib_install.log"
-docker exec "${CONTAINER_NAME}" bash -c "/madlib/build_docker/src/bin/madpack
-p postgres \
+docker exec --user=postgres "${CONTAINER_NAME}" bash -c
"/madlib/build_docker/src/bin/madpack -p postgres \
-c
postgres/postgres@localhost:5432/postgres \
install-check" \
| tee
"${workdir}/build_docker_logs/madlib_install_check.log"
[madlib] 03/04: Fix jenkins
This is an automated email from the ASF dual-hosted git repository. okislal pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/madlib.git commit 4a98bc47a73d729e5fbe9f1ccb19acb9bdb8f17d Author: Orhan Kislal AuthorDate: Wed May 18 15:21:43 2022 +0300 Fix jenkins --- tool/jenkins/jenkins_build.sh | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/tool/jenkins/jenkins_build.sh b/tool/jenkins/jenkins_build.sh index 6a1c0b86..9e486085 100755 --- a/tool/jenkins/jenkins_build.sh +++ b/tool/jenkins/jenkins_build.sh @@ -60,9 +60,9 @@ sleep 15 echo "-- Install pip, and mock ---" # cmake, make, make install, and make package cat <
