[asterixdb] branch master updated: [NO ISSUE][DOC] Refresh and update text-example
This is an automated email from the ASF dual-hosted git repository. imaxon pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/asterixdb.git The following commit(s) were added to refs/heads/master by this push: new 1b0ded5 [NO ISSUE][DOC] Refresh and update text-example 1b0ded5 is described below commit 1b0ded56a2f5d14e14adb96e5abe6d7196b28286 Author: Ian Maxon AuthorDate: Thu May 16 11:35:26 2019 -0700 [NO ISSUE][DOC] Refresh and update text-example Change-Id: Ie756e790994cd6d8dad69cce1bb728923cc0398c Reviewed-on: https://asterix-gerrit.ics.uci.edu/3356 Sonar-Qube: Jenkins Tested-by: Jenkins Integration-Tests: Jenkins Reviewed-by: Michael Blow --- .../text-example/textclient/pom.xml| 16 + .../src/main/assembly/binary-assembly.xml | 7 ++ .../text-example/textclient/src/main/conf/README | 10 .../src/main/assembly/binary-assembly.xml | 12 ++ .../text-example/textserver/src/main/conf/README | 10 .../textserver/src/main/conf/text.conf | 28 ++ 6 files changed, 83 insertions(+) diff --git a/hyracks-fullstack/hyracks/hyracks-examples/text-example/textclient/pom.xml b/hyracks-fullstack/hyracks/hyracks-examples/text-example/textclient/pom.xml index c00ffc1..4412fe0 100644 --- a/hyracks-fullstack/hyracks/hyracks-examples/text-example/textclient/pom.xml +++ b/hyracks-fullstack/hyracks/hyracks-examples/text-example/textclient/pom.xml @@ -100,6 +100,22 @@ true + +maven-assembly-plugin + + + + +src/main/assembly/binary-assembly.xml + + +package + + single + + + + diff --git a/hyracks-fullstack/hyracks/hyracks-examples/text-example/textclient/src/main/assembly/binary-assembly.xml b/hyracks-fullstack/hyracks/hyracks-examples/text-example/textclient/src/main/assembly/binary-assembly.xml index 3271f1d..f8abe31 100644 --- a/hyracks-fullstack/hyracks/hyracks-examples/text-example/textclient/src/main/assembly/binary-assembly.xml +++ b/hyracks-fullstack/hyracks/hyracks-examples/text-example/textclient/src/main/assembly/binary-assembly.xml @@ -49,5 +49,12 @@ NOTICE* + + src/main/conf + . + +README + + diff --git a/hyracks-fullstack/hyracks/hyracks-examples/text-example/textclient/src/main/conf/README b/hyracks-fullstack/hyracks/hyracks-examples/text-example/textclient/src/main/conf/README new file mode 100644 index 000..3af2693 --- /dev/null +++ b/hyracks-fullstack/hyracks/hyracks-examples/text-example/textclient/src/main/conf/README @@ -0,0 +1,10 @@ +This package holds a very simple Hyracks client program that counts the words in a supplied text file. A prerequisite +is a running Hyracks cluster from the textserver example. + +In the bin directory of the unzipped textclient package, run + +textclient -host localhost \ + -infile-splits textserver-nc1:data/file1.txt \ + -outfile-splits textserver-nc1:data/file1.txt.out \ + -algo -hash + diff --git a/hyracks-fullstack/hyracks/hyracks-examples/text-example/textserver/src/main/assembly/binary-assembly.xml b/hyracks-fullstack/hyracks/hyracks-examples/text-example/textserver/src/main/assembly/binary-assembly.xml index e89db9c..ac2e6f9 100644 --- a/hyracks-fullstack/hyracks/hyracks-examples/text-example/textserver/src/main/assembly/binary-assembly.xml +++ b/hyracks-fullstack/hyracks/hyracks-examples/text-example/textserver/src/main/assembly/binary-assembly.xml @@ -53,5 +53,17 @@ NOTICE* + + src/main/conf + . + +text.conf +README + + + + data/ + data/ + diff --git a/hyracks-fullstack/hyracks/hyracks-examples/text-example/textserver/src/main/conf/README b/hyracks-fullstack/hyracks/hyracks-examples/text-example/textserver/src/main/conf/README new file mode 100644 index 000..22cf5c5 --- /dev/null +++ b/hyracks-fullstack/hyracks/hyracks-examples/text-example/textserver/src/main/conf/README @@ -0,0 +1,10 @@ +This example sets up a very basic Hyracks cluster to be used for counting words in a file with the textclient example. +To run, simply execute: + +bin/hyrackscc -config-file conf/text.conf + +to start the cluster controller. Once this process is running, execute: + +bin/hyracksnc -config-file conf/text.conf + +and you will have a Hyracks cluster running locally for the textclient to use. \ No newline at end of file diff --git a/hyracks-fullstack/hyracks/hyracks-examples/text-example/textserver/src/main/conf/text.conf b/hyracks-fullstack/hyracks/hyracks-examples/text-example/textserver/src/main/conf/text.conf new file mode 100644 index 000
svn commit: r34540 - /dev/asterixdb/
Author: imaxon Date: Sat Jun 15 00:48:17 2019 New Revision: 34540 Log: upload 0.9.5 and 0.3.5 Added: dev/asterixdb/apache-asterixdb-0.9.5-source-release.zip (with props) dev/asterixdb/apache-asterixdb-0.9.5-source-release.zip.asc dev/asterixdb/apache-asterixdb-0.9.5-source-release.zip.sha256 dev/asterixdb/apache-hyracks-0.3.5-source-release.zip (with props) dev/asterixdb/apache-hyracks-0.3.5-source-release.zip.asc dev/asterixdb/apache-hyracks-0.3.5-source-release.zip.sha256 dev/asterixdb/asterix-server-0.9.5-binary-assembly.zip (with props) dev/asterixdb/asterix-server-0.9.5-binary-assembly.zip.asc dev/asterixdb/asterix-server-0.9.5-binary-assembly.zip.sha256 Removed: dev/asterixdb/apache-asterixdb-0.9.4.1-source-release.zip dev/asterixdb/apache-asterixdb-0.9.4.1-source-release.zip.asc dev/asterixdb/apache-asterixdb-0.9.4.1-source-release.zip.sha256 dev/asterixdb/apache-hyracks-0.3.4.1-source-release.zip dev/asterixdb/apache-hyracks-0.3.4.1-source-release.zip.asc dev/asterixdb/apache-hyracks-0.3.4.1-source-release.zip.sha256 dev/asterixdb/asterix-server-0.9.4.1-binary-assembly.zip dev/asterixdb/asterix-server-0.9.4.1-binary-assembly.zip.asc dev/asterixdb/asterix-server-0.9.4.1-binary-assembly.zip.sha256 Added: dev/asterixdb/apache-asterixdb-0.9.5-source-release.zip == Binary file - no diff available. Propchange: dev/asterixdb/apache-asterixdb-0.9.5-source-release.zip -- svn:mime-type = application/octet-stream Added: dev/asterixdb/apache-asterixdb-0.9.5-source-release.zip.asc == --- dev/asterixdb/apache-asterixdb-0.9.5-source-release.zip.asc (added) +++ dev/asterixdb/apache-asterixdb-0.9.5-source-release.zip.asc Sat Jun 15 00:48:17 2019 @@ -0,0 +1,17 @@ +-BEGIN PGP SIGNATURE- +Version: GnuPG v2.0.14 (GNU/Linux) + +iQIcBAABAgAGBQJdBD3WAAoJEKmP6oelbqmAb+UP/1YFLohuvEzrCGdEly02bqqb +UhzNdAChgRwraiscINc/Rk0v8D23WqaM8HEbkbPBBmIP2Js4ihF95M6dHbWQ7lgW +VNvkXIG4B5TdkfzTlsSxHdCWLISjEEv2rfS9+rjJVfCLfd47hvj6OMBN19MQGc83 +bKfEVP4aIJWWx8/lWeGA9r4Dme6wrKT5UIIiKhMBhyH4ujUH2TjbBR0WTKhib5J5 +fNpdV+CV02/gqptbWAEVYdgbGDOy8wWB1zbKakfI7fxgWKgwxfM7kLx/MM2AXjUb +GTDumYrfUoKghoWG2D0Vo2y6hxoLpbRChV6cuP5kybDm3VjcRVt1PKI8KVFD0Dr1 +xS9ICu8irxr4f11l1o4jPki5myH90XRpSrV4x7LoaV7OHC0tgNN/7TOiOFUCK7dZ +66RueShT5ThmXIWt/+3np/J8z8y3j5v6tX13y79vFJFALJttn3EAd2JDaxOpI860 +soBrB05KzSVBmjtplJhLC9U3Lm47Kx0HmdUFcJ3Tog5A1zCEnhVIcPmKMXBmkmnk +368FXMKEzf5Ya1Wqld/PUbXGmvsx6snMbYyTOGvs8hBesZeH60pQ2NhOQyJM4/3f +HpupzjIKUEAR6ty1v4qs2upBSTRFTT6hPjKNHTWZ2lViu6HO6jPXGv4gGcDaAU7b +wOE37MytcO2LWCQ8XJDQ +=OPhX +-END PGP SIGNATURE- Added: dev/asterixdb/apache-asterixdb-0.9.5-source-release.zip.sha256 == --- dev/asterixdb/apache-asterixdb-0.9.5-source-release.zip.sha256 (added) +++ dev/asterixdb/apache-asterixdb-0.9.5-source-release.zip.sha256 Sat Jun 15 00:48:17 2019 @@ -0,0 +1 @@ +9ec3582e5d163658673824f1a648fb9223773b1d64a14879b7274c3e74a9ce82 apache-asterixdb-0.9.5-source-release.zip Added: dev/asterixdb/apache-hyracks-0.3.5-source-release.zip == Binary file - no diff available. Propchange: dev/asterixdb/apache-hyracks-0.3.5-source-release.zip -- svn:mime-type = application/octet-stream Added: dev/asterixdb/apache-hyracks-0.3.5-source-release.zip.asc == --- dev/asterixdb/apache-hyracks-0.3.5-source-release.zip.asc (added) +++ dev/asterixdb/apache-hyracks-0.3.5-source-release.zip.asc Sat Jun 15 00:48:17 2019 @@ -0,0 +1,17 @@ +-BEGIN PGP SIGNATURE- +Version: GnuPG v2.0.14 (GNU/Linux) + +iQIcBAABAgAGBQJdBDyjAAoJEKmP6oelbqmAYMUQANjTRzFS/tlMnmrUktQXhrXl +ov5Bmy/+Klk/P5sGeS5MBfAzqIJHtSM48/aGhUbb/U4hJ0SbmmwpJyff3y+sPaVo +lYKNUTKovJlGttZXx8/NNaH3Eow6A8uzYZBp6R0apcNFlc3jdLC89XqBvQlNypT5 +IW0AESXtptaGhQZqD/hsrTY0uxeEmECmoH8fHR0iKzxX+5YSOmOMCv6gPudKXoIN ++6faCGI7yAho8M3vbJGmRmjSo2Qx8bO0p0ERZbydiB2+mR5sfaix1G3VxixjCg4B +CBBE+7qQq7kCYuiUoBxIJgveJpBzqSjuvllM0O1NKmLlVNidnQfl1E93yb1W1liz +hU4vXLGcWweifeXo9rgpj1wUQy66FW65WDHfVR8WTWnykscZ4TjndmSW0yiBgNXY +eTnlyN9X8I/AaiPyFcJ2sec7RMTLkrorpVq1qehaoaprZMCXV+L+/86Xx53F6ykt +UQp+HHVBe/LNcxXASXSo65t5EJo2nMweZBAc0fbUl4cm+dAB6GIUtrEwE/Aq2N2D +jNkbdSCWz74pnam2jNXNDqblSjYyVCGxZBsgszRDR2Y7UNv6UPQJSCxHfDUbsSnT +nb/zNjL+rUPYDTDAdiTrTuORLNKEadqrLdOU15a3YOCmfJcvUaI1OKiMGo9Ui6I+ +Mu4OXQlVlhZ7zOPbGB8G +=O5F/ +-END PGP SIGNATURE- Added: dev/asterixdb/apache-hyracks-0.3.5-source-release.zip.sha256 == --- dev/asterixdb
svn commit: r34638 - /dev/asterixdb/
Author: imaxon Date: Wed Jun 26 01:04:09 2019 New Revision: 34638 Log: upload 0.9.5 and 0.3.5 Added: dev/asterixdb/apache-asterixdb-0.9.5-source-release.zip (with props) dev/asterixdb/apache-asterixdb-0.9.5-source-release.zip.asc dev/asterixdb/apache-asterixdb-0.9.5-source-release.zip.sha256 dev/asterixdb/apache-hyracks-0.3.5-source-release.zip (with props) dev/asterixdb/apache-hyracks-0.3.5-source-release.zip.asc dev/asterixdb/apache-hyracks-0.3.5-source-release.zip.sha256 dev/asterixdb/asterix-server-0.9.5-binary-assembly.zip (with props) dev/asterixdb/asterix-server-0.9.5-binary-assembly.zip.asc dev/asterixdb/asterix-server-0.9.5-binary-assembly.zip.sha256 Added: dev/asterixdb/apache-asterixdb-0.9.5-source-release.zip == Binary file - no diff available. Propchange: dev/asterixdb/apache-asterixdb-0.9.5-source-release.zip -- svn:mime-type = application/octet-stream Added: dev/asterixdb/apache-asterixdb-0.9.5-source-release.zip.asc == --- dev/asterixdb/apache-asterixdb-0.9.5-source-release.zip.asc (added) +++ dev/asterixdb/apache-asterixdb-0.9.5-source-release.zip.asc Wed Jun 26 01:04:09 2019 @@ -0,0 +1,17 @@ +-BEGIN PGP SIGNATURE- +Version: GnuPG v2.0.14 (GNU/Linux) + +iQIcBAABAgAGBQJdEsIFAAoJEKmP6oelbqmAXvYQAMyqaDEHVHIDO+3vmaDUAWxi +JETwN5U4Klq8BUyNLtvOHb5DXtkiMHdfcv1mIzPWBDA8Jf3087HbzsWfd2JeT3Rd +octvdYGkQ132s7q2SxMwlppa7IlPs744cttLShPfnhs8RCsSTMDoVwXItUCol3zs +KtcOqHcEA7hTw7xwrHwxRU29q5u1IOcN5YQr3JP9UTDt+MgHsVpIEQMBAkVMfipQ +/9IpasYgAsgcP4rmcHWVxlyDLaBwtubshUdA+D7WVV0mwOFEQCl/7RTX8tZlmW1w +vVT0l9Y8hc69RcYGX+OsYxJtP+aL240DDcHYcDLpK29K87a6//si03IlfI0BUS/D +27r4l9+rN8FGSjdWtj6sTFiSJgbe/hKXVCEEXZjvrMWpZPfO+ea4xc8Oi2tbiEpb +vjt7Le3iys94tIUe+kurdKvyT4rRx63A5PJafyh1zA4OcjmBydPgONqwAbyL4bBF +m4NDSuPHaALogk2leqyiNrJr8EUcxBUuUsRLIwe/yGEM1uttAdZU87aDT9nAEu3u +0dr2E6O5t174qipZJQwCyy5hmq++2FPQIYeZZwwb5CgNlPFPDBPFcOJOS2tfOR6T +DOZfDyz0Lsp5ts7RSVpaffM8xPMszMXE1NGaiZVHmaqTGzwYgVHwCKANFwbFuGsP +CMSqUpxD4umcg0+0rBPP +=ib2Y +-END PGP SIGNATURE- Added: dev/asterixdb/apache-asterixdb-0.9.5-source-release.zip.sha256 == --- dev/asterixdb/apache-asterixdb-0.9.5-source-release.zip.sha256 (added) +++ dev/asterixdb/apache-asterixdb-0.9.5-source-release.zip.sha256 Wed Jun 26 01:04:09 2019 @@ -0,0 +1 @@ +8d29df132ff841608f8e4220759118624cf19dadf37591e9bd60ac425b5d9f3e apache-asterixdb-0.9.5-source-release.zip Added: dev/asterixdb/apache-hyracks-0.3.5-source-release.zip == Binary file - no diff available. Propchange: dev/asterixdb/apache-hyracks-0.3.5-source-release.zip -- svn:mime-type = application/octet-stream Added: dev/asterixdb/apache-hyracks-0.3.5-source-release.zip.asc == --- dev/asterixdb/apache-hyracks-0.3.5-source-release.zip.asc (added) +++ dev/asterixdb/apache-hyracks-0.3.5-source-release.zip.asc Wed Jun 26 01:04:09 2019 @@ -0,0 +1,17 @@ +-BEGIN PGP SIGNATURE- +Version: GnuPG v2.0.14 (GNU/Linux) + +iQIcBAABAgAGBQJdEsDFAAoJEKmP6oelbqmAXFQQANEVsFhouQvS3heldqKxLLIF +lJF7MM8tX27jTTHt3XuiWKPYWgwHVgNUXarF3Tjq29llRcLmq0cuKeT1XzPs6Qvh +5dLjzlN2aZqoLoJR9HAdTnd2uXyWOvHABDj/Q3HfegGValnTRbGoXQKOyi827eGQ +x4ePQHxTGC8W5ZX8jzSLiYSq+LCSxxBgxtmAYI5jSx0LiRK458TSMASev4sDzLr1 +psuVf93N6lbhvSxOoe/XIarQ2O+8tWCxt/SnsZpGmQ2Ks/3OOX0Upsj0o6AF16+2 +4SXQA4hy5T/6NCtnmTryeZulPiFtti+wOP6mljIY7R4sgnoGumjFjepdhXhc/e4M +KNQGpbqAx8DV7/RDMCpcKi6q0xZsQkafgq7jVAxyk+Mt8E1zXXVtvF+hvRyPdR6v +r6K1f6y0I2qRLN+oI+Ywu1OXBfjlBIaPYKjSkLTfrq2n1/k4eDF8k3vFRXzpGeDC +4ipn003lw6cXOuWo7Y8tmIla/vJawAZ0OZ1u7LwW62TaxgNizq4iFb33zPdrr95f +cfK1LXvns3gR7zuydgbsYg5zEeda/o3CCQ037UtfTFk8LuI3gC0A/WsueBdH53xg +a5q1ZNzPNEqxcIxcflXBuRzIi/df0VxyQR9SGyH4fa1Spnwc/XOUcnoizHMDVnhX +kWPxxw6j9xDDdbJe4fNS +=G4iG +-END PGP SIGNATURE- Added: dev/asterixdb/apache-hyracks-0.3.5-source-release.zip.sha256 == --- dev/asterixdb/apache-hyracks-0.3.5-source-release.zip.sha256 (added) +++ dev/asterixdb/apache-hyracks-0.3.5-source-release.zip.sha256 Wed Jun 26 01:04:09 2019 @@ -0,0 +1 @@ +ae308851da139b62b10a9e093727c0dc039264d962efc9c67051d7b07add0eff apache-hyracks-0.3.5-source-release.zip Added: dev/asterixdb/asterix-server-0.9.5-binary-assembly.zip == Binary file - no diff available. Propchange: dev/asterixdb/asterix-server-0.9.5-binary-assembly.zip
svn commit: r34722 - /dev/asterixdb/
Author: imaxon Date: Mon Jul 1 23:25:25 2019 New Revision: 34722 Log: upload 0.9.5 and 0.3.5 Added: dev/asterixdb/apache-asterixdb-0.9.5-source-release.zip (with props) dev/asterixdb/apache-asterixdb-0.9.5-source-release.zip.asc dev/asterixdb/apache-asterixdb-0.9.5-source-release.zip.sha256 dev/asterixdb/apache-hyracks-0.3.5-source-release.zip (with props) dev/asterixdb/apache-hyracks-0.3.5-source-release.zip.asc dev/asterixdb/apache-hyracks-0.3.5-source-release.zip.sha256 dev/asterixdb/asterix-server-0.9.5-binary-assembly.zip (with props) dev/asterixdb/asterix-server-0.9.5-binary-assembly.zip.asc dev/asterixdb/asterix-server-0.9.5-binary-assembly.zip.sha256 Added: dev/asterixdb/apache-asterixdb-0.9.5-source-release.zip == Binary file - no diff available. Propchange: dev/asterixdb/apache-asterixdb-0.9.5-source-release.zip -- svn:mime-type = application/octet-stream Added: dev/asterixdb/apache-asterixdb-0.9.5-source-release.zip.asc == --- dev/asterixdb/apache-asterixdb-0.9.5-source-release.zip.asc (added) +++ dev/asterixdb/apache-asterixdb-0.9.5-source-release.zip.asc Mon Jul 1 23:25:25 2019 @@ -0,0 +1,17 @@ +-BEGIN PGP SIGNATURE- +Version: GnuPG v2.0.14 (GNU/Linux) + +iQIcBAABAgAGBQJdGpOmAAoJEKmP6oelbqmAVscQAK295NcbcqyvHMTR9LVEYwjk +QgbO2aiCkEAvzOk3NDzshPQPN9PTbGN08jkxX7z1DjluA59oDiQqLfoWl9CPRkkF +J4/6gamFWeYxyEVUP1BcfWGKXWv4mO2zpAWB+IHvXrjqkVRHjksv/F6bHBG46kH4 +deboYUxjklGQXQqabfdq5BfLlQBIgg8thMetlvWliWp44QNvlYChjFbwKG3ZlH82 +btGkB+IoIRxXZZ0wwCs82w8+lxNDl9Ty3syupmAdLPlTDC9TngK4It2QGfaStGSu +naR2f9eTHBdZgf4A3tC+IH95AISEfle5vKw7PTpVVKVbCZp6gnaSo7RlKlgzP+ta +EbQVbPUFew8F8hAhC8R5ziFV6IZowFaRsPmGnKNl4u3oAmUBY9qejxXai+6zDZ0f +KSVC0P7H1kwNzztNvjmZwpDn+KeosHSpvMa7ug5easO4crURignUDaXXxXU7dbBs +BiWBfjKF57loLSehwGpiVnGj8bvSUDE+vGgDz640rjPNsN7fi2bKAMcPbXcCaeTC +moIvXo9EGmOV5wSLKvq2K+CSuPN42aD1t63PZG916S49a80PPJv55n/DMTvwtXnq +v3bYc0PHzu91MyqN4Syzi3gmlBzIgOLZiWm6j0vdpAAMrBFDkUXtzytAnNeQy4Bg +qhcmVRMG6QQ2Q6T/RiJk +=7xwE +-END PGP SIGNATURE- Added: dev/asterixdb/apache-asterixdb-0.9.5-source-release.zip.sha256 == --- dev/asterixdb/apache-asterixdb-0.9.5-source-release.zip.sha256 (added) +++ dev/asterixdb/apache-asterixdb-0.9.5-source-release.zip.sha256 Mon Jul 1 23:25:25 2019 @@ -0,0 +1 @@ +35c0249bf7d8bb5868016589018eefb91c0dfde1f6b06001e859ffb3d9144638 apache-asterixdb-0.9.5-source-release.zip Added: dev/asterixdb/apache-hyracks-0.3.5-source-release.zip == Binary file - no diff available. Propchange: dev/asterixdb/apache-hyracks-0.3.5-source-release.zip -- svn:mime-type = application/octet-stream Added: dev/asterixdb/apache-hyracks-0.3.5-source-release.zip.asc == --- dev/asterixdb/apache-hyracks-0.3.5-source-release.zip.asc (added) +++ dev/asterixdb/apache-hyracks-0.3.5-source-release.zip.asc Mon Jul 1 23:25:25 2019 @@ -0,0 +1,17 @@ +-BEGIN PGP SIGNATURE- +Version: GnuPG v2.0.14 (GNU/Linux) + +iQIcBAABAgAGBQJdGpJxAAoJEKmP6oelbqmAhy0QALDVIE6oWOFuVFGNP+4UX6ku +SF/DwZ9QTiAAFyse3lZfza5pBWsgLCmFuiu2MF5ixMi7gNi+73+KkqWgxr1CzH3r +1Zlic20TAaqIiV+Q9bXhuXDdY6qPpWbgVErr7vgRfTMMvG8m8Hb4hWxl0OxduCsN +spOniPN60n776m0Us2iFZ7pe+3QmhB3tmfH+7RANpdudpDr2y7T1H9BJvJNUCxo0 +vwRaAXBtCLmSRmaqrsqD6dEV0YYPSLPrPiFaDqI3RXA1voW5Yc5ZnWXB7impYz7W +DpVdOX9IO+o1cjZjgLqoDEPMKnodGLVAjsZ2RjFiS4nSHGJhq6B6+kwmmKebfI5N +6J937//yeiPlu/9A0wqXLO/opiGEx5dwBq2j7JYO6awiiB/LGmXYeR+5p6eslNcE +YLiYfekffYCyGoo2DxgvsxPcIp7xwTlamJuqg1T2PVHclcpYJ+y02PHz5ogZMLtW +ef/wgDvHf2QrkxueuHyh4PV0rtsBIPVbhuHqcBWToBTVQH9tXploJ9tlUdzX5EV5 +N3+qKQJ/zVFtikkvqYwlLSD4CIEJUzlSV2qCv3S13X8zRZfRDcauyX5O2ChGdv00 +WAgd3wAj8hDM961FiKE9abl18YE8RBX8L/DrLcS65rUs92jGboKKRwQUobd77hvW +ORZdgGa8dFnsVtUtdmx1 +=uf4u +-END PGP SIGNATURE- Added: dev/asterixdb/apache-hyracks-0.3.5-source-release.zip.sha256 == --- dev/asterixdb/apache-hyracks-0.3.5-source-release.zip.sha256 (added) +++ dev/asterixdb/apache-hyracks-0.3.5-source-release.zip.sha256 Mon Jul 1 23:25:25 2019 @@ -0,0 +1 @@ +ee2eda7e9ff03978e21b4fc0db33854475d6dba70b5346b5e78308bf6d8efc72 apache-hyracks-0.3.5-source-release.zip Added: dev/asterixdb/asterix-server-0.9.5-binary-assembly.zip == Binary file - no diff available. Propchange: dev/asterixdb/asterix-server-0.9.5-binary-assembly.zip
[asterixdb] branch master updated: [ASTERIXDB-2597] Load UDFs via HTTP
This is an automated email from the ASF dual-hosted git repository. imaxon pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/asterixdb.git The following commit(s) were added to refs/heads/master by this push: new 580b81a [ASTERIXDB-2597] Load UDFs via HTTP 580b81a is described below commit 580b81aa5eb8e1b0620521a1c9680e54df73 Author: Ian Maxon AuthorDate: Mon Jul 1 13:16:43 2019 -0700 [ASTERIXDB-2597] Load UDFs via HTTP - POST existing UDF format to /admin/udf/$DATAVERSE/$LIBNAME - DELETE against that URL to remove UDFs Change-Id: I6be9fef54c010bdb32f5c78af9b973f9843f442f Reviewed-on: https://asterix-gerrit.ics.uci.edu/3386 Contrib: Jenkins Tested-by: Jenkins Integration-Tests: Jenkins Reviewed-by: Ian Maxon --- .../asterix/api/http/server/UdfApiServlet.java | 161 + .../asterix/app/external/ExternalLibraryUtils.java | 81 ++- .../asterix/app/external/ExternalUDFLibrarian.java | 122 +--- .../app/external/IExternalUDFLibrarian.java| 7 +- .../asterix/app/message/DeleteUdfMessage.java | 55 +++ .../apache/asterix/app/message/LoadUdfMessage.java | 67 + .../asterix/app/message/UdfResponseMessage.java| 70 + .../app/nc/task/ExternalLibrarySetupTask.java | 3 +- .../asterix/hyracks/bootstrap/CCApplication.java | 6 +- .../api/common/AsterixHyracksIntegrationUtil.java | 3 +- .../asterix/app/bootstrap/TestNodeController.java | 4 - .../apache/asterix/test/common/TestExecutor.java | 2 +- .../asterix/test/runtime/LangExecutionUtil.java| 11 +- .../type_validation/type_validation.0.ddl.sqlpp} | 15 +- .../asterix/common/library/ILibraryManager.java| 5 +- .../org/apache/asterix/common/utils/Servlets.java | 1 + .../external/library/ExternalLibraryManager.java | 17 ++- .../hyracks/api/application/IServerContext.java} | 22 +-- .../hyracks/api/application/IServiceContext.java | 2 + .../client/HyracksClientInterfaceFunctions.java| 8 +- .../api/client/IHyracksClientConnection.java | 18 ++- .../api/client/IHyracksClientInterface.java| 3 +- .../hyracks/control/cc/ClientInterfaceIPCI.java| 2 +- .../control/cc/work/CliDeployBinaryWork.java | 8 +- .../control/common/application/ServiceContext.java | 11 +- .../control/common/base/INodeController.java | 2 +- .../control/common/context/ServerContext.java | 14 +- .../ClassLoaderJobSerializerDeserializer.java | 2 +- .../control/common/deployment/DeploymentUtils.java | 52 ++- .../hyracks/control/common/ipc/CCNCFunctions.java | 9 +- .../common/ipc/NodeControllerRemoteProxy.java | 5 +- .../hyracks/control/nc/NodeControllerIPCI.java | 4 +- .../hyracks/control/nc/NodeControllerService.java | 2 +- .../apache/hyracks/control/nc/io/IOManager.java| 4 + .../hyracks/control/nc/work/DeployBinaryWork.java | 7 +- .../impl/HyracksClientInterfaceRemoteProxy.java| 6 +- .../apache/hyracks/ipc/impl/HyracksConnection.java | 24 +-- .../hyracks/test/support/TestNCServiceContext.java | 6 + 38 files changed, 613 insertions(+), 228 deletions(-) diff --git a/asterixdb/asterix-app/src/main/java/org/apache/asterix/api/http/server/UdfApiServlet.java b/asterixdb/asterix-app/src/main/java/org/apache/asterix/api/http/server/UdfApiServlet.java new file mode 100644 index 000..d293ee8 --- /dev/null +++ b/asterixdb/asterix-app/src/main/java/org/apache/asterix/api/http/server/UdfApiServlet.java @@ -0,0 +1,161 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ +package org.apache.asterix.api.http.server; + +import java.io.File; +import java.io.RandomAccessFile; +import java.nio.ByteBuffer; +import java.nio.channels.FileChannel; +import java.util.ArrayList; +import java.util.Arrays; +import java.util.List; +import java.util.concurrent.ConcurrentMap; + +import org.apache.asterix.app.external.ExternalLibraryUtils; +import org.apache.asterix.app.message.DeleteUdfMessage; +import org.apache.asterix.app.message.LoadUdfMessage; +import org.apache.asterix.common.dataflow.I
svn commit: r34857 - /dev/asterixdb/
Author: imaxon Date: Fri Jul 12 02:02:45 2019 New Revision: 34857 Log: upload 0.9.5 and 0.3.5 Added: dev/asterixdb/apache-asterixdb-0.9.5-source-release.zip (with props) dev/asterixdb/apache-asterixdb-0.9.5-source-release.zip.asc dev/asterixdb/apache-asterixdb-0.9.5-source-release.zip.sha256 dev/asterixdb/apache-hyracks-0.3.5-source-release.zip (with props) dev/asterixdb/apache-hyracks-0.3.5-source-release.zip.asc dev/asterixdb/apache-hyracks-0.3.5-source-release.zip.sha256 dev/asterixdb/asterix-server-0.9.5-binary-assembly.zip (with props) dev/asterixdb/asterix-server-0.9.5-binary-assembly.zip.asc dev/asterixdb/asterix-server-0.9.5-binary-assembly.zip.sha256 Added: dev/asterixdb/apache-asterixdb-0.9.5-source-release.zip == Binary file - no diff available. Propchange: dev/asterixdb/apache-asterixdb-0.9.5-source-release.zip -- svn:mime-type = application/octet-stream Added: dev/asterixdb/apache-asterixdb-0.9.5-source-release.zip.asc == --- dev/asterixdb/apache-asterixdb-0.9.5-source-release.zip.asc (added) +++ dev/asterixdb/apache-asterixdb-0.9.5-source-release.zip.asc Fri Jul 12 02:02:45 2019 @@ -0,0 +1,17 @@ +-BEGIN PGP SIGNATURE- +Version: GnuPG v2.0.14 (GNU/Linux) + +iQIcBAABAgAGBQJdJ+e4AAoJEKmP6oelbqmAaJgP/1+NGyPuuORmtmoCO6PaKrw3 +A8muTmBAAehOI8kVEjLzzQvdzDjIe6Bnk328hbCkCfRoesGeR3WBouu5BZ0trA77 +wan+nShowfPlkGCL2yN2OXWE+OdGIyN7StARdtaWEepRl3OPaBoYThj1OXNobEQk +9TJ4JXTyDaEd7O9i/EH9bAoSMhxYmQJvi45O56KtvDN//nqQCSLm67cw7Kd1/PrS +39PJWfIjm4qvASLV/fApEa2GUBhtU/BKvF492ckCetD6eswucITI/XhMGf8fNXC7 +8HQ7Udgs3HwHZB0CLQS1aDB+r5YlBe0ijQ7Mrxl0F8aMMqPLikKdjShu5VaOuEyC +ieWjHFYT5ctWPsDqSCB/X0ia2MJzXIwFuupb4+vbDgsaDbtACtN+5Nas0tXosXfj +KaTzsaBcaNcrL8c33SeG8QFON8Sp8BiEk0ME+nMmKrpecOFj03JOI4VW434AH4b9 +lVPNCF1tHTehvbSOiKuBmyrnS+YgvNXC/x1kkTdztAjHdXiDIMhixp/ybiFg0nk2 +qRaS/+NqKWbWHtckU+UFBiOQC+Lx1/6nXrX8d7dFC7JwxqcrHY1KCD1pj9VTK4hp +8hEg5buT1KjV784Jr4N/vgMKm4a8DBCiVtPOWzaA8VT5oCeGDrGHhKQrvRutpSpT +5wTDGddzLtonqwFEU1t5 +=6PgL +-END PGP SIGNATURE- Added: dev/asterixdb/apache-asterixdb-0.9.5-source-release.zip.sha256 == --- dev/asterixdb/apache-asterixdb-0.9.5-source-release.zip.sha256 (added) +++ dev/asterixdb/apache-asterixdb-0.9.5-source-release.zip.sha256 Fri Jul 12 02:02:45 2019 @@ -0,0 +1 @@ +71178eb3017b454f7b6d68f5dafbf665dc52bd4ba7407e265f8ecf378b6a59f7 apache-asterixdb-0.9.5-source-release.zip Added: dev/asterixdb/apache-hyracks-0.3.5-source-release.zip == Binary file - no diff available. Propchange: dev/asterixdb/apache-hyracks-0.3.5-source-release.zip -- svn:mime-type = application/octet-stream Added: dev/asterixdb/apache-hyracks-0.3.5-source-release.zip.asc == --- dev/asterixdb/apache-hyracks-0.3.5-source-release.zip.asc (added) +++ dev/asterixdb/apache-hyracks-0.3.5-source-release.zip.asc Fri Jul 12 02:02:45 2019 @@ -0,0 +1,17 @@ +-BEGIN PGP SIGNATURE- +Version: GnuPG v2.0.14 (GNU/Linux) + +iQIcBAABAgAGBQJdJ+Z8AAoJEKmP6oelbqmATNQP/2lpohsXdYADJdD2FauWHb88 +evNY960QwrbULdKAz2Cj4LKbvGkZNhXXw2Avire4Vs/1pMio338tt4PzuehMfv93 +sA6mhakjhv/UtG1awGE7f5aGb7cUaIqPcaWltNsa18AZfnX38Q5qUBiT/Az4o6rK +GnPdrwQnEqY9MIwpFtuPD0HwUWQv6ftyNvVFhWRkjWLW9bNhFNWIi2TaZvrnOMQM +N/H8FgoQ5XHtUbs+pyPnX5AUJADBdDl62/2yVZ3I2g0gPwwY/IFpZG3WQAR3CYsl +hlf9Zsk2ae5rQtn28lMrvVt/U/NX23TTzVF7H8hVlu+jNLHbuh+sDRusOb+XqlXi +X49gNO1rc9wwqoA3avg/hR/p6A43C1cy6K+thnbzrBO/7lx0XhcPgRXwMpdcIVl8 +pN3SNbAxYaUuHhupBgcK/V3rtSy8ckIzpwrDXyISWI+3Cq8x5h6unMWr14wXzaFa +KviNIWfrXfOQoLZaMSatochoS5ACS+OqfGulPG/TqVRZWfuCc2GZ2XFfyJX/+MFl +T6oEr7NdxqSl5SzTzMb1ZXq0eXnyaXm7WrK8vyUoATYJ3BGc/BdAqzQnX6VeSbPV +iwNEeYO24TbrD1azOGNajZD7Gfaq0hMEc6zfIHWOZiR2uefiDXJJtvzotwDl2bab +hIPTI+mljsk1R4N3xLlk +=Al9e +-END PGP SIGNATURE- Added: dev/asterixdb/apache-hyracks-0.3.5-source-release.zip.sha256 == --- dev/asterixdb/apache-hyracks-0.3.5-source-release.zip.sha256 (added) +++ dev/asterixdb/apache-hyracks-0.3.5-source-release.zip.sha256 Fri Jul 12 02:02:45 2019 @@ -0,0 +1 @@ +f927c469cb25e8685b865277eb36091f89f82a1fcc08dd0c0bf4cb67d60aee35 apache-hyracks-0.3.5-source-release.zip Added: dev/asterixdb/asterix-server-0.9.5-binary-assembly.zip == Binary file - no diff available. Propchange: dev/asterixdb/asterix-server-0.9.5-binary-assembly.zip
svn commit: r34860 - /dev/asterixdb/
Author: imaxon Date: Fri Jul 12 07:43:58 2019 New Revision: 34860 Log: upload 0.9.5 and 0.3.5 Added: dev/asterixdb/apache-asterixdb-0.9.5-source-release.zip (with props) dev/asterixdb/apache-asterixdb-0.9.5-source-release.zip.asc dev/asterixdb/apache-asterixdb-0.9.5-source-release.zip.sha256 dev/asterixdb/apache-hyracks-0.3.5-source-release.zip (with props) dev/asterixdb/apache-hyracks-0.3.5-source-release.zip.asc dev/asterixdb/apache-hyracks-0.3.5-source-release.zip.sha256 dev/asterixdb/asterix-server-0.9.5-binary-assembly.zip (with props) dev/asterixdb/asterix-server-0.9.5-binary-assembly.zip.asc dev/asterixdb/asterix-server-0.9.5-binary-assembly.zip.sha256 Added: dev/asterixdb/apache-asterixdb-0.9.5-source-release.zip == Binary file - no diff available. Propchange: dev/asterixdb/apache-asterixdb-0.9.5-source-release.zip -- svn:mime-type = application/octet-stream Added: dev/asterixdb/apache-asterixdb-0.9.5-source-release.zip.asc == --- dev/asterixdb/apache-asterixdb-0.9.5-source-release.zip.asc (added) +++ dev/asterixdb/apache-asterixdb-0.9.5-source-release.zip.asc Fri Jul 12 07:43:58 2019 @@ -0,0 +1,17 @@ +-BEGIN PGP SIGNATURE- +Version: GnuPG v2.0.14 (GNU/Linux) + +iQIcBAABAgAGBQJdKDbaAAoJEKmP6oelbqmAlCYP/1pNN0EY/gg/NrW1JAthR35Z +420LzxBCf12kzJxCpwZRLZsT7rDmY1kvNiz2UPTY4Pwo2UQtuBOf04shU/qdyHmX +mi9WCZ+qwRoG/3gsnbTjLZZRfLkb6y0OlRH8r2rDwk1ZXC9iE2Fptfc5QhnLQ1us +WTfywNYFI9jHQDN7j9j7lipoAn47k9txnCZISLGmsGrWBSU+r6dQvrzkYHkkKhpO +SlQGXr7ncqxAh7VOkr6rWCnrYvalja8uBaEfjaovqsF7YelH8RJsy9DftFOifbwI +qrfQPrE2YDbP7IhBTVOS8xVCnprfOK5OCkr6exu26bqpcashGBMcaTMZjCuiPs29 +MCTFwNCdnai+19zdV42f3Z33vbUjjc7ZutKsmDX8vwwI23zy4FD6Qqsxlb/XDzlO +K+A1J0uyY2pE33gNy0iJfIhFNvROJoS68omLEyShSfQMB4dCq+IzVmT8FxbGMLRd +puYyWvqj1jxSl9KSURiiW52LNQGthzOlNHomXQYCxQjBQRrF0ZQPk8+L7nQxye8H +89PH4oy1YXAS2+F0QsTfVuR2K0dbVSjlyPibxbKy8EQgipP0HxY36eocckIglpDx +NHI08+ZxqIrKsvZUFgwRKs34is2KtO2eBONV484a2xH5FmOz+v++IV+3gnbwpQre +MgksWiRJeWFvxQYjMXM1 +=BZVO +-END PGP SIGNATURE- Added: dev/asterixdb/apache-asterixdb-0.9.5-source-release.zip.sha256 == --- dev/asterixdb/apache-asterixdb-0.9.5-source-release.zip.sha256 (added) +++ dev/asterixdb/apache-asterixdb-0.9.5-source-release.zip.sha256 Fri Jul 12 07:43:58 2019 @@ -0,0 +1 @@ +1eecef9152ec2e39083383070456a38876e4bf6127cb6800b1e2e365f207 apache-asterixdb-0.9.5-source-release.zip Added: dev/asterixdb/apache-hyracks-0.3.5-source-release.zip == Binary file - no diff available. Propchange: dev/asterixdb/apache-hyracks-0.3.5-source-release.zip -- svn:mime-type = application/octet-stream Added: dev/asterixdb/apache-hyracks-0.3.5-source-release.zip.asc == --- dev/asterixdb/apache-hyracks-0.3.5-source-release.zip.asc (added) +++ dev/asterixdb/apache-hyracks-0.3.5-source-release.zip.asc Fri Jul 12 07:43:58 2019 @@ -0,0 +1,17 @@ +-BEGIN PGP SIGNATURE- +Version: GnuPG v2.0.14 (GNU/Linux) + +iQIcBAABAgAGBQJdKDWAAAoJEKmP6oelbqmAHisP/0l3TxeBUlXXFctq+QY8tGn2 +K/4PzTmF5aCQUwudoyGoM4r+S9kghsjDcidt+H3ea+Twgn+VK66818466xGXghDc +p5+ev4p/hk0UHjL5HYhuCGoNOCuWrC+Hwwo3n60zSDix4GY2CrI386GI5DQJzune +eoIcU0OoF10oifxdGwQezG5eVMUKp37sXyd9uTQpi/fs5+WvtzEhZUdaRzkNDKWn +UrobEDty4bKCsZ9sanbwkL2rY8adh217SC4k9zQNPT56Q6Umg6sKBfnlOA/t3Nxg +ABjduyjuVaQDJFzIhzls8k6TIm1b7qx+ssB1cf7vhTfzJLDEEJWc1MfuU3zj/TNq +U10s2FpvmIcv6EZLNUSk9WdSY+0aKYepPMvKEeYs+NUzleNmy76ljvQFDFzh74DJ +Rfq19XZu5V+PvpNwRiQ3kO2oJapn6hKEwOR6uSrca73InqkMs+OxkfYPmUbGNoG2 +xFqSPBjhwcyIAjXEBj59/3MeYkWNW0pXC2WmuLj00jg2u7HiOj9IeOKoIdsxswFw +ore0n4EqU1aTI2RycWuLDova76yWBSS5V86DPYxguJjPmfBi02xYeQeur0dPXx5Q +EU0CtCBJXsF3xFjlaChCIkU0tIhC6vAH/yp8RTr4FNXTUUGvdQAxDVdHLDIJPl9v +gFICdIrngDe3BXd7u3F0 +=IxYS +-END PGP SIGNATURE- Added: dev/asterixdb/apache-hyracks-0.3.5-source-release.zip.sha256 == --- dev/asterixdb/apache-hyracks-0.3.5-source-release.zip.sha256 (added) +++ dev/asterixdb/apache-hyracks-0.3.5-source-release.zip.sha256 Fri Jul 12 07:43:58 2019 @@ -0,0 +1 @@ +a90e1f8e06ed78bfc9d2d3dd3bcdf996d738d8193a4d94fe3f5cd8b4c447ee2e apache-hyracks-0.3.5-source-release.zip Added: dev/asterixdb/asterix-server-0.9.5-binary-assembly.zip == Binary file - no diff available. Propchange: dev/asterixdb/asterix-server-0.9.5-binary-assembly.zip
[asterixdb] branch master updated: [NO ISSUE] Fix UDF delete message
This is an automated email from the ASF dual-hosted git repository. imaxon pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/asterixdb.git The following commit(s) were added to refs/heads/master by this push: new 44a13bc [NO ISSUE] Fix UDF delete message 44a13bc is described below commit 44a13bc9a7343c211a39a2b2a66bab760c735cf0 Author: Ian Maxon AuthorDate: Tue Jul 16 18:13:55 2019 -0700 [NO ISSUE] Fix UDF delete message The delete method for UDFs is using the async message format, but this isn't correct. It should use the synchronous request form and require a response, just as the udf load does. Change-Id: I4c18e62bdca2fe6b9239d740b9040171b799a3a7 Reviewed-on: https://asterix-gerrit.ics.uci.edu/3487 Tested-by: Jenkins Integration-Tests: Jenkins Reviewed-by: Ian Maxon Reviewed-by: Michael Blow --- .../asterix/api/http/server/UdfApiServlet.java | 9 ++--- ...LoadUdfMessage.java => AbstractUdfMessage.java} | 26 +++--- .../asterix/app/message/DeleteUdfMessage.java | 31 +--- .../apache/asterix/app/message/LoadUdfMessage.java | 41 -- 4 files changed, 34 insertions(+), 73 deletions(-) diff --git a/asterixdb/asterix-app/src/main/java/org/apache/asterix/api/http/server/UdfApiServlet.java b/asterixdb/asterix-app/src/main/java/org/apache/asterix/api/http/server/UdfApiServlet.java index d293ee8..79b78a8 100644 --- a/asterixdb/asterix-app/src/main/java/org/apache/asterix/api/http/server/UdfApiServlet.java +++ b/asterixdb/asterix-app/src/main/java/org/apache/asterix/api/http/server/UdfApiServlet.java @@ -130,10 +130,11 @@ public class UdfApiServlet extends AbstractServlet { } private void deleteUdf(String dataverse, String resourceName) throws Exception { -DeleteUdfMessage msg = new DeleteUdfMessage(dataverse, resourceName); -for (String nc : appCtx.getClusterStateManager().getParticipantNodes()) { -broker.sendApplicationMessageToNC(msg, nc); -} +long reqId = broker.newRequestId(); +List requests = new ArrayList<>(); +List ncs = new ArrayList<>(appCtx.getClusterStateManager().getParticipantNodes()); +ncs.forEach(s -> requests.add(new DeleteUdfMessage(dataverse, resourceName, reqId))); +broker.sendSyncRequestToNCs(reqId, ncs, requests, UDF_RESPONSE_TIMEOUT); appCtx.getLibraryManager().deregisterLibraryClassLoader(dataverse, resourceName); appCtx.getHcc().unDeployBinary(new DeploymentId(resourceName)); } diff --git a/asterixdb/asterix-app/src/main/java/org/apache/asterix/app/message/LoadUdfMessage.java b/asterixdb/asterix-app/src/main/java/org/apache/asterix/app/message/AbstractUdfMessage.java similarity index 71% copy from asterixdb/asterix-app/src/main/java/org/apache/asterix/app/message/LoadUdfMessage.java copy to asterixdb/asterix-app/src/main/java/org/apache/asterix/app/message/AbstractUdfMessage.java index 66714bf..90bd2a6 100644 --- a/asterixdb/asterix-app/src/main/java/org/apache/asterix/app/message/LoadUdfMessage.java +++ b/asterixdb/asterix-app/src/main/java/org/apache/asterix/app/message/AbstractUdfMessage.java @@ -18,26 +18,25 @@ */ package org.apache.asterix.app.message; -import org.apache.asterix.app.external.ExternalLibraryUtils; import org.apache.asterix.common.api.INcApplicationContext; import org.apache.asterix.common.library.ILibraryManager; import org.apache.asterix.common.messaging.CcIdentifiedMessage; import org.apache.asterix.common.messaging.api.INCMessageBroker; import org.apache.asterix.common.messaging.api.INcAddressedMessage; -import org.apache.hyracks.util.file.FileUtil; import org.apache.logging.log4j.LogManager; import org.apache.logging.log4j.Logger; -public class LoadUdfMessage extends CcIdentifiedMessage implements INcAddressedMessage { +public abstract class AbstractUdfMessage extends CcIdentifiedMessage implements INcAddressedMessage { -private final String dataverseName; -private final String libraryName; -private static final Logger LOGGER = LogManager.getLogger(); +protected final String dataverseName; +protected final String libraryName; +protected static final Logger LOGGER = LogManager.getLogger(); + +private static final long serialVersionUID = 2L; -private static final long serialVersionUID = -4529473341458281271L; private final long reqId; -public LoadUdfMessage(String dataverseName, String libraryName, long reqId) { +public AbstractUdfMessage(String dataverseName, String libraryName, long reqId) { this.dataverseName = dataverseName; this.libraryName = libraryName; this.reqId = reqId; @@ -51,12 +50,11 @@ public class LoadUdfMessage extends CcIdentifiedMessage implements INcAddressedM INCMessageBroker broker = (INCMessageBroker) appCtx.getServiceContext().getMessageBro
[asterixdb] branch master updated: [NO ISSUE] Make asterix-docker work again
This is an automated email from the ASF dual-hosted git repository. imaxon pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/asterixdb.git The following commit(s) were added to refs/heads/master by this push: new 739acc1 [NO ISSUE] Make asterix-docker work again 739acc1 is described below commit 739acc1d1bff0d8107b2bbf990a2d859a3cae795 Author: Ian Maxon AuthorDate: Wed Aug 21 14:44:09 2019 -0700 [NO ISSUE] Make asterix-docker work again Change-Id: I6fef4413e43e683238b46dc735c075ec02ac9e92 Reviewed-on: https://asterix-gerrit.ics.uci.edu/3529 Contrib: Jenkins Sonar-Qube: Jenkins Tested-by: Jenkins Integration-Tests: Jenkins Reviewed-by: Michael Blow --- asterixdb/asterix-docker/docker/Dockerfile | 6 +++--- asterixdb/asterix-docker/docker/supervisord.conf | 12 ++-- 2 files changed, 9 insertions(+), 9 deletions(-) diff --git a/asterixdb/asterix-docker/docker/Dockerfile b/asterixdb/asterix-docker/docker/Dockerfile index 58c2c1a..8fa3ab4 100644 --- a/asterixdb/asterix-docker/docker/Dockerfile +++ b/asterixdb/asterix-docker/docker/Dockerfile @@ -22,12 +22,12 @@ MAINTAINER AsterixDB Team RUN echo 'LANG="en_US.UTF-8"' > /etc/sysconfig/i18n ;echo 'ZONE="America/Los_Angeles"' > /etc/sysconfig/clock ;cp -a /usr/share/zoneinfo/America/Los_Angeles /etc/localtime RUN echo "include_only=.us" >> /etc/yum/pluginconf.d/fastestmirror.conf RUN yum -y update -RUN yum install -y unzip java-1.8.0-openjdk openssh-server openssh-clients python-setuptools wget curl +RUN yum install -y unzip java-11-openjdk-headless python-setuptools wget curl RUN easy_install supervisor RUN mkdir /asterixdb COPY asterix-server*.zip . RUN unzip asterix-server*.zip -d /asterixdb/ -COPY asterix-configuration.xml /asterixdb/bin/asterix-configuration.xml +RUN mv /asterixdb/apache*/* /asterixdb/ COPY supervisord.conf /etc/supervisord.conf COPY twu.adm /asterixdb/twu.adm COPY twm.adm /asterixdb/twm.adm @@ -35,7 +35,7 @@ COPY fbu.adm /asterixdb/fbu.adm COPY fbm.adm /asterixdb/fbm.adm WORKDIR /asterixdb/bin -ENV JAVA_HOME /usr/lib/jvm/jre-1.8.0 +ENV JAVA_HOME /usr/lib/jvm/jre ENV JAVA_OPTS -Xmx1536m EXPOSE 19001 19002 19003 50031 diff --git a/asterixdb/asterix-docker/docker/supervisord.conf b/asterixdb/asterix-docker/docker/supervisord.conf index b6ce1e0..20f1797 100644 --- a/asterixdb/asterix-docker/docker/supervisord.conf +++ b/asterixdb/asterix-docker/docker/supervisord.conf @@ -18,22 +18,22 @@ [supervisord] nodaemon=true -[program:asterixcc] -command=/asterixdb/bin/asterixcc -address localhost -client-listen-address localhost +[program:asterixnc1] +command=/asterixdb/bin/asterixncservice -logdir - -config-file "/asterixdb/opt/local/conf/blue.conf" stdout_logfile=/dev/stdout stdout_logfile_maxbytes=0 stderr_logfile=/dev/stderr stderr_logfile_maxbytes=0 -[program:asterixnc1] -command=/asterixdb/bin/asterixnc -node-id nc1 -cluster-address localhost -iodevices /asterixdb/nc1 -address localhost -data-listen-address localhost -result-listen-address localhost +[program:asterixnc2] +command=/asterixdb/bin/asterixncservice -logdir - stdout_logfile=/dev/stdout stdout_logfile_maxbytes=0 stderr_logfile=/dev/stderr stderr_logfile_maxbytes=0 -[program:asterixnc2] -command=/asterixdb/bin/asterixnc -node-id nc2 -cluster-address localhost -iodevices /asterixdb/nc2 -address localhost -data-listen-address localhost -result-listen-address localhost +[program:asterixcc] +command=/asterixdb/bin/asterixcc -config-file "/asterixdb/opt/local/conf/cc.conf" stdout_logfile=/dev/stdout stdout_logfile_maxbytes=0 stderr_logfile=/dev/stderr
svn commit: r35533 - /dev/asterixdb/
Author: imaxon Date: Tue Sep 3 21:32:46 2019 New Revision: 35533 Log: upload 0.9.5 and 0.3.5 Added: dev/asterixdb/apache-asterixdb-0.9.5-source-release.zip (with props) dev/asterixdb/apache-asterixdb-0.9.5-source-release.zip.asc dev/asterixdb/apache-asterixdb-0.9.5-source-release.zip.sha256 dev/asterixdb/apache-hyracks-0.3.5-source-release.zip (with props) dev/asterixdb/apache-hyracks-0.3.5-source-release.zip.asc dev/asterixdb/apache-hyracks-0.3.5-source-release.zip.sha256 Removed: dev/asterixdb/asterix-server-0.9.5-binary-assembly.zip dev/asterixdb/asterix-server-0.9.5-binary-assembly.zip.asc dev/asterixdb/asterix-server-0.9.5-binary-assembly.zip.sha256 Added: dev/asterixdb/apache-asterixdb-0.9.5-source-release.zip == Binary file - no diff available. Propchange: dev/asterixdb/apache-asterixdb-0.9.5-source-release.zip -- svn:mime-type = application/octet-stream Added: dev/asterixdb/apache-asterixdb-0.9.5-source-release.zip.asc == --- dev/asterixdb/apache-asterixdb-0.9.5-source-release.zip.asc (added) +++ dev/asterixdb/apache-asterixdb-0.9.5-source-release.zip.asc Tue Sep 3 21:32:46 2019 @@ -0,0 +1,17 @@ +-BEGIN PGP SIGNATURE- +Version: GnuPG v2.0.14 (GNU/Linux) + +iQIcBAABAgAGBQJdbtk+AAoJEKmP6oelbqmA+NYQAIxMOzmOJFo3Z0oJwAnmIlxa +yV2boaXUtjtg+3pEblwjcLVhwuXwwNqMghOMCuAVcz9+IzBU0irvzG6f+NxaCvcf +yUWVioyP7beA5nXIF3jHuNjoXJUt7QUmXR/N/RGhdnN1s4Zu/lDmTVIq1ET5FTLB +OCxDsGnrIJ6C8l2c+fc0h0g/21GAABzuHEHoT+NrL+ulkjfc5BqMjeUOcE2bFDWO +90S6HGfsXPUHKK+YfBmo0nFD127dBHHqA1mNzyf7x6PogD5C3DHizRVkMWctjUzT +5rKG2u5/SCnlGKXp9yPjDxqbM+IjC/4IOOn6iNQv0RFeDrfJPaEii+WBDoFS+86t +9RcwGEtfpP9M/helphMZfTmG2gqvMHrRdjipHRMs1sWFMrxnVl1ZDhq1cmerHe/m +TRt61huEB1mCx1m1Ta0nTtDkJU/F25rSbAuSN8hNs4hiAGrXgELgfXS7+VeZDmIy +vEobHtI7TTZnYoJl3ODNIm7BZwe+cQm6kwf4Q17tbOn60mxMbCxCNxC7al3L6DH4 +USWOz57jUClrl2nzKjPBDe/6TqTTDuwqNTRhbl/l6gkQUgQJ14Z5ottT5P/rMKvW +/UiPq6bYbE4S5Up5Gl/ziXR/jrau44r9sPd2aOcAH0uoM3K/HKGhyORJ8OzDZlU6 +RBtemmP0RM2vET+SntFL +=T0RP +-END PGP SIGNATURE- Added: dev/asterixdb/apache-asterixdb-0.9.5-source-release.zip.sha256 == --- dev/asterixdb/apache-asterixdb-0.9.5-source-release.zip.sha256 (added) +++ dev/asterixdb/apache-asterixdb-0.9.5-source-release.zip.sha256 Tue Sep 3 21:32:46 2019 @@ -0,0 +1 @@ +be41051e803e5ada2c64f608614c6476c6686e043c47a2a0291ccfd25239a679 apache-asterixdb-0.9.5-source-release.zip Added: dev/asterixdb/apache-hyracks-0.3.5-source-release.zip == Binary file - no diff available. Propchange: dev/asterixdb/apache-hyracks-0.3.5-source-release.zip -- svn:mime-type = application/octet-stream Added: dev/asterixdb/apache-hyracks-0.3.5-source-release.zip.asc == --- dev/asterixdb/apache-hyracks-0.3.5-source-release.zip.asc (added) +++ dev/asterixdb/apache-hyracks-0.3.5-source-release.zip.asc Tue Sep 3 21:32:46 2019 @@ -0,0 +1,17 @@ +-BEGIN PGP SIGNATURE- +Version: GnuPG v2.0.14 (GNU/Linux) + +iQIcBAABAgAGBQJdbtf9AAoJEKmP6oelbqmANFwP/RZ6VQMIQkO5Go/meAN4PXkq +2zeu9z2+94Qi6+MuYejMoSxHKoIyKzM7WMIRCnOAZqGjHoEwEMH16Zta49im6YmM +Wo25D1LGmKR6NGCEqV1fZFXv9/C80v3R2PGTZMUKV4VQlU++zOhrin5nKlBxy6k1 +Eo8KvXEou2EUCHdQAdLXM8hNrOISC4AsKxOym1qTleg9EpmkzlDvM5hYmIaFY98u +Y2PW7fFnaREHEfi5WNeKZHmOfc5x20nlwo/R/yiK1lMZx94gLnd+zbvfHLTKOtjM +VmSrYCzZTvWKVHoDFE1JpH91H28wSA37DcruTTltqfPobUTuu6yuikTmWu2Ih0xF +P5axqtCO3lyRlD2QSBdU3J/8LpKK5s7oh7MDooCpZej86V9zTSMx6/CUOnq5JFv3 +8dCuTcaBV+UihOcWbAjxPSQNddmYvB1+WSf4LfnFYASYtUUFkQeuFeAwQ/oVJqo6 ++2ErB5kOSuoEPnF0P7gwrCHVlpBBkAuBtPM39NCDUbO4hJZgYtHqP14bWENQMbeb +7kkL3iVVe0vpPxbygYptuMKRqzEN7cJFSVmODvEKrK8qNWdcT5pjeNVSAYub+ARL +JEXH8nLCWNNBRmQt4TU7i1Kik9woF3Vm47dd86RYXuZKWuI2ShHFHNnT21ps9Mbm +YjiZLOJxxlNCUc89sBmX +=Q2BC +-END PGP SIGNATURE- Added: dev/asterixdb/apache-hyracks-0.3.5-source-release.zip.sha256 == --- dev/asterixdb/apache-hyracks-0.3.5-source-release.zip.sha256 (added) +++ dev/asterixdb/apache-hyracks-0.3.5-source-release.zip.sha256 Tue Sep 3 21:32:46 2019 @@ -0,0 +1 @@ +b06fe983aa6837abe3460a157d7600662ec56181a43db317579f5c7ddf9bfc08 apache-hyracks-0.3.5-source-release.zip
svn commit: r35535 - in /dev/asterixdb: apache-asterixdb-0.9.5.zip apache-asterixdb-0.9.5.zip.asc apache-asterixdb-0.9.5.zip.sha256
Author: imaxon Date: Wed Sep 4 00:57:54 2019 New Revision: 35535 Log: amend Added: dev/asterixdb/apache-asterixdb-0.9.5.zip (with props) dev/asterixdb/apache-asterixdb-0.9.5.zip.asc dev/asterixdb/apache-asterixdb-0.9.5.zip.sha256 Added: dev/asterixdb/apache-asterixdb-0.9.5.zip == Binary file - no diff available. Propchange: dev/asterixdb/apache-asterixdb-0.9.5.zip -- svn:mime-type = application/octet-stream Added: dev/asterixdb/apache-asterixdb-0.9.5.zip.asc == --- dev/asterixdb/apache-asterixdb-0.9.5.zip.asc (added) +++ dev/asterixdb/apache-asterixdb-0.9.5.zip.asc Wed Sep 4 00:57:54 2019 @@ -0,0 +1,17 @@ +-BEGIN PGP SIGNATURE- +Version: GnuPG v2.0.14 (GNU/Linux) + +iQIcBAABAgAGBQJdbtspAAoJEKmP6oelbqmALKsP+wSI+YIfSyWSiwOuMDaaI3VP +0lHM6Jv6adgnmVbKk6JZoJJwOj8sv2ntiIASNV6X5qWFAeP6QCAg93N6u8fAu5ZB +CfOJffv/KWo+RRE6aCusE32J02wb665MIRpAfjQqJcmCBzKUw6NHKSvKy75/v1Nb +mm8710W5UbUG5yW+fDA1aENera4yw0FKE/aQVF5ihAhIEt5bg1MD9aBJfSgprCKa +Jx8u5p+APWeWK6MZQoiy9DCJRSFwyhVpic5oCJ6jumrnbotXomGb9hbsfjNI+rNv +Aypz2D+U4mP0spheJzF5b939Ea3RLQXjC64XyA5EEujozP14a3Oz5kIoIrVV6X4j +W04mopBpjV1KV9PZKODwTWZXy+f4qZSnyU7no5FDQdIar1HOGCaGyKXshAz7GHkL ++9bhrQBzPfbRMsT7mJ7HWENEPRMJNeZXi4gdsH7I06FqGaiTVZ7CebalQIP/x1pR +BCsXWY4Jwdr0AJP7iKs3FZdthZXxtvrdWS+aQccaHeuOXFWTQO7oPqJvXqbM5htu +wwl1D2UecNCOnu98JgPUg2e5OlT0d8k5IkWcR5D1J3PgGi5tFlD7L4MwfgwT4QOM +ORgSmwzfWo5nHLaAoGYGEbrl0ZUkO9hdqjMRfMP03K+pbOoOJH/7YjsVPkctq6nv +HhhOhb1c2a4dBLIKcG8R +=5CqJ +-END PGP SIGNATURE- Added: dev/asterixdb/apache-asterixdb-0.9.5.zip.sha256 == --- dev/asterixdb/apache-asterixdb-0.9.5.zip.sha256 (added) +++ dev/asterixdb/apache-asterixdb-0.9.5.zip.sha256 Wed Sep 4 00:57:54 2019 @@ -0,0 +1 @@ +f02b833b7399c2eacf404ae1d84b4be3a0ae88dc0d7053b6e7da8020df80da89 apache-asterixdb-0.9.5.zip
[asterixdb] branch master updated: [ASTERIXDB-2532][RT] per-operator profiling
This is an automated email from the ASF dual-hosted git repository. imaxon pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/asterixdb.git The following commit(s) were added to refs/heads/master by this push: new c20bf9a [ASTERIXDB-2532][RT] per-operator profiling c20bf9a is described below commit c20bf9a74fcb35ee8dae9c355657235ef19297cc Author: Ian Maxon AuthorDate: Thu Sep 12 12:12:04 2019 -0700 [ASTERIXDB-2532][RT] per-operator profiling Enables profiling in queries at the operator-level when the analyze variable is set in a query. Change-Id: Ie16f3901ae5b32920d8552d5fd1ec8bb6e2ec8ae Reviewed-on: https://asterix-gerrit.ics.uci.edu/3226 Contrib: Jenkins Tested-by: Jenkins Integration-Tests: Jenkins Reviewed-by: Till Westmann --- .../asterix/translator/IStatementExecutor.java | 58 - .../apache/asterix/api/common/ResultMetadata.java | 11 ++ .../api/http/server/NCQueryServiceServlet.java | 2 +- .../api/http/server/QueryResultApiServlet.java | 10 +- .../http/server/QueryServiceRequestParameters.java | 10 ++ .../api/http/server/QueryServiceServlet.java | 33 -- .../message/ExecuteStatementRequestMessage.java| 10 +- .../asterix/app/result/JobResultCallback.java | 17 +++ .../{ResponseMertics.java => ResponseMetrics.java} | 26 ++--- .../asterix/app/result/fields/MetricsPrinter.java | 30 ++--- .../asterix/app/result/fields/NcResultPrinter.java | 8 +- .../asterix/app/result/fields/ProfilePrinter.java | 73 .../asterix/app/translator/QueryTranslator.java| 6 + .../asterix/test/common/ResultExtractor.java | 6 + .../apache/asterix/test/common/TestExecutor.java | 5 + .../org/apache/asterix/test/common/TestHelper.java | 12 ++ .../test/runtime/ProfiledExecutionTest.java| 65 +++ .../asterix-app/src/test/resources/cc-single.conf | 50 .../src/test/resources/runtimets/profiled.xml | 30 + .../profile/full-scan/full-scan.1.ddl.sqlpp| 41 --- .../profile/full-scan/full-scan.2.update.sqlpp | 24 ++-- .../profile/full-scan/full-scan.3.profile.sqlpp| 23 ++-- .../profile/full-scan/full-scan.4.ddl.sqlpp| 21 +--- .../profile/full-scan/full-scan.3.regexjson| 64 ++ .../apache/asterix/builders/IARecordBuilder.java | 7 +- .../OpenRecordConstructorDescriptor.java | 5 + .../NestedPlansRunningAggregatorFactory.java | 15 ++- .../meta/AlgebricksMetaOperatorDescriptor.java | 5 + .../runtime/operators/meta/PipelineAssembler.java | 7 +- .../operators/std/AssignRuntimeFactory.java| 2 +- .../api/com}/job/profiling/counters/Counter.java | 3 +- .../hyracks/api/dataflow/IPassableTimer.java} | 24 ++-- .../hyracks/api/dataflow/TimedFrameWriter.java | 130 + .../api/dataflow/TimedOperatorNodePushable.java| 123 +++ .../hyracks/api/job/profiling/IStatsCollector.java | 27 - .../hyracks/api}/job/profiling/OperatorStats.java | 11 +- .../runtime/SuperActivityOperatorNodePushable.java | 28 - .../hyracks/client/stats/AggregateCounter.java | 2 +- .../client/stats/impl/ClientCounterContext.java| 2 +- .../common/job/profiling/StatsCollector.java | 44 +-- .../common/job/profiling/om/TaskProfile.java | 18 ++- .../java/org/apache/hyracks/control/nc/Joblet.java | 2 +- .../java/org/apache/hyracks/control/nc/Task.java | 2 +- .../hyracks/control/nc/work/StartTasksWork.java| 3 +- .../group/sort/SortGroupByOperatorDescriptor.java | 22 +++- .../OptimizedHybridHashJoinOperatorDescriptor.java | 10 ++ .../std/result/ResultWriterOperatorDescriptor.java | 5 + .../std/sort/AbstractSorterOperatorDescriptor.java | 15 ++- .../std/sort/ExternalSortOperatorDescriptor.java | 9 +- .../hyracks/dataflow/std/sort/IRunGenerator.java | 2 + .../dataflow/std/sort/TimedRunGenerator.java | 51 .../std/sort/TopKSorterOperatorDescriptor.java | 15 ++- .../tests/integration/AbstractIntegrationTest.java | 4 +- .../dataflow/BTreeSearchOperatorDescriptor.java| 5 + .../storage/am/btree/test/FramewriterTest.java | 3 + .../hyracks/hyracks-storage-am-common/pom.xml | 5 - .../dataflow/IndexSearchOperatorNodePushable.java | 16 ++- .../hyracks/test/support/CounterContext.java | 2 +- 58 files changed, 1069 insertions(+), 190 deletions(-) diff --git a/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/translator/IStatementExecutor.java b/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/translator/IStatementExecutor.java index 3718340..606abd2 100644 --- a/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/translator/IStatementExecutor.java +++ b/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/translator/IStatementExecutor.java @@ -1
[asterixdb] branch master updated: [NO ISSUE] Fix typo in docs
This is an automated email from the ASF dual-hosted git repository. imaxon pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/asterixdb.git The following commit(s) were added to refs/heads/master by this push: new 8897bfa [NO ISSUE] Fix typo in docs 8897bfa is described below commit 8897bfa513413d82de383402341d8ac169ee3979 Author: Ian Maxon AuthorDate: Tue Sep 3 18:21:19 2019 -0700 [NO ISSUE] Fix typo in docs Change-Id: I8e394025b80b2c3d6d03a53cd863cd534d55e049 Reviewed-on: https://asterix-gerrit.ics.uci.edu/3541 Sonar-Qube: Jenkins Contrib: Jenkins Integration-Tests: Jenkins Tested-by: Jenkins Reviewed-by: Till Westmann --- asterixdb/asterix-doc/src/site/markdown/aql/primer.md | 2 +- asterixdb/asterix-doc/src/site/markdown/sqlpp/filters.md | 2 +- asterixdb/asterix-doc/src/site/markdown/sqlpp/fulltext.md | 2 +- asterixdb/asterix-doc/src/site/markdown/sqlpp/primer-sqlpp.md | 2 +- asterixdb/asterix-doc/src/site/markdown/sqlpp/similarity.md | 2 +- 5 files changed, 5 insertions(+), 5 deletions(-) diff --git a/asterixdb/asterix-doc/src/site/markdown/aql/primer.md b/asterixdb/asterix-doc/src/site/markdown/aql/primer.md index de187b0..72406c0 100644 --- a/asterixdb/asterix-doc/src/site/markdown/aql/primer.md +++ b/asterixdb/asterix-doc/src/site/markdown/aql/primer.md @@ -44,7 +44,7 @@ As you read through this document, you should try each step for yourself on your Once you have reached the end, you will be fully armed and dangerous, with all the basic AsterixDB knowledge that you'll need to start down the path of modeling, storing, and querying your own semistructured data. -## ADM: Modeling Semistructed Data in AsterixDB ## +## ADM: Modeling Semistructured Data in AsterixDB ## In this section you will learn all about modeling Big Data using ADM, the data model of the AsterixDB BDMS. diff --git a/asterixdb/asterix-doc/src/site/markdown/sqlpp/filters.md b/asterixdb/asterix-doc/src/site/markdown/sqlpp/filters.md index 6b8e00f..be1e2d4 100644 --- a/asterixdb/asterix-doc/src/site/markdown/sqlpp/filters.md +++ b/asterixdb/asterix-doc/src/site/markdown/sqlpp/filters.md @@ -63,7 +63,7 @@ components, it is possible, when answering certain queries, to exploit partitioning to only access some components and safely filter out the remaining components, thus reducing query times. For instance, referring to our -[TinySocial](primer.html#ADM:_Modeling_Semistructed_Data_in_AsterixDB) +[TinySocial](primer.html#ADM:_Modeling_Semistructured_Data_in_AsterixDB) example, suppose a user always retrieves tweets from the `TweetMessages` dataset based on the `send-time` field (e.g., tweets posted in the last 24 hours). Since there is not a secondary index on diff --git a/asterixdb/asterix-doc/src/site/markdown/sqlpp/fulltext.md b/asterixdb/asterix-doc/src/site/markdown/sqlpp/fulltext.md index 1328ed9..1d5b8da 100644 --- a/asterixdb/asterix-doc/src/site/markdown/sqlpp/fulltext.md +++ b/asterixdb/asterix-doc/src/site/markdown/sqlpp/fulltext.md @@ -54,7 +54,7 @@ Thus, "Voice" or "voice" will be evaluated as the same word. from ChirpMessages msg where ftcontains(msg.messageText, "voice", {"mode":"any"}); -The DDL and DML of TinySocial can be found in [ADM: Modeling Semistructed Data in AsterixDB](../sqlpp/primer-sqlpp.html#ADM:_Modeling_Semistructed_Data_in_AsterixDB). +The DDL and DML of TinySocial can be found in [ADM: Modeling Semistructured Data in AsterixDB](../sqlpp/primer-sqlpp.html#ADM:_Modeling_Semistructured_Data_in_AsterixDB). The `Expression1` is an expression that should be evaluable as a string at runtime as in the above example where `msg.messageText` is a string field. The `Expression2` can be a string, an (un)ordered list diff --git a/asterixdb/asterix-doc/src/site/markdown/sqlpp/primer-sqlpp.md b/asterixdb/asterix-doc/src/site/markdown/sqlpp/primer-sqlpp.md index 40b81d6..48eba96 100644 --- a/asterixdb/asterix-doc/src/site/markdown/sqlpp/primer-sqlpp.md +++ b/asterixdb/asterix-doc/src/site/markdown/sqlpp/primer-sqlpp.md @@ -45,7 +45,7 @@ You will use the AsterixDB web interface to do this, and for SQL++ you will need Once you have reached the end of this tutorial, you will be fully armed and dangerous, with all the basic AsterixDB knowledge that you'll need to start down the path of modeling, storing, and querying your own semistructured data. -## ADM: Modeling Semistructed Data in AsterixDB ## +## ADM: Modeling Semistructured Data in AsterixDB ## In this section you will learn all about modeling Big Data using ADM, the data model of the AsterixDB BDMS. diff --git a/asterixdb/asterix-doc/src/site/markdown/sqlpp/similarity.md b/asterixdb/asterix-doc/src/site/markdown/sqlpp/similarity.md index 8118126..a0e6ab0 100644 --- a/asterixdb/asterix-doc/src/site/markdown/sqlpp/similarity.md +++ b
svn commit: r35939 - /release/asterixdb/KEYS
Author: imaxon Date: Fri Sep 20 00:05:36 2019 New Revision: 35939 Log: remove compromised key Modified: release/asterixdb/KEYS Modified: release/asterixdb/KEYS == --- release/asterixdb/KEYS (original) +++ release/asterixdb/KEYS Fri Sep 20 00:05:36 2019 @@ -1,70 +0,0 @@ -pub 4096R/A56EA980 2015-07-21 -uid Ian Maxon (CODE SIGNING KEY) -sig 3A56EA980 2015-07-21 Ian Maxon (CODE SIGNING KEY) -sig BAED10BC 2015-09-01 Till Westmann (CODE SIGNING KEY) -sub 4096R/511F67F0 2015-07-21 -sig A56EA980 2015-07-21 Ian Maxon (CODE SIGNING KEY) - --BEGIN PGP PUBLIC KEY BLOCK- -Version: GnuPG v1 - -mQINBFWupxcBEADrlyK+mKtzw4yva3BfJQVrcIa1iHO33HQHaalhSkIy9gm2+Tgf -3rRzIAJ+JOVRaVBMFSixUvDuRt522Oa3wEr7cCpfZCSexN6e7DHV4Fdeb0VqQftq -w08fhn/4Su87gtDfKiS4ALU+XlEx0W5MMNNoQMa90QG8VpHLLrlisw7IqzJ++0Qz -Dj3BUe8s8JOsuhXj/UhpMPqLC5FbC1aZRv6n3n9uN3rrPlGoDEzTqcjPQ4Fo60x/ -GuP19lIjeNetTEA4uHYO2/A/AN4HfdJbyai+cj4U2kKOqLiLpf6mJ3pqDNGotTo2 -Lb0Oet2Dja2RRPrP1j436C3nqKAxwyttqKgcQZVXUhRDpffRVpd+Ny3BoSDNhZPL -hfqsPZKGdyUUrxiJoKTkTePvXNh0oLNBnodRswSBEpIaJlh8/EjZz/NHL8FYNt2t -+GfqjjziMnQsqYZS4HECXw/B6fALfUdvwwkKZ6jg9/PjVvRHF598PsL1Wuv7I5Op -4O2duqfudzdiA4MzGKEhcjn2KvbCRuFtUeukL0JlVb6ECDKvyW3M7Z3sgDP9kuH4 -w02G2A9QwFpivRQhr0lZTGfzCMJOvGYGcJX0VHVnZ5ymVpvYCnovlEjJL5N6GrmU -is8Cel87HKTLzDFYGZ1awe8KqbPZEQpdWrygJ4dsdlRYKgyFBIAG68sOEQARAQAB -tDBJYW4gTWF4b24gKENPREUgU0lHTklORyBLRVkpIDxpbWF4b25AYXBhY2hlLm9y -Zz6JAjcEEwECACECGwMCHgECF4AFAlWurlkFCwkIBwMFFQoJCAsFFgIDAQAACgkQ -qY/qh6VuqYDbCA//YfnFdmI7G++D8M9EkHg5TT65EAOAbcECwSxIohIq17Ea11Df -NrQfsLR5cXaEhQF60w2OWaTBDYnFwLqP+1/hElsoXYPJNpfBfk5eRkIZbRwJWji1 -eShRxWvs827qzxIeRP7tyYVsl3FotgzEZmm1/LusDZC3JZ75dO0Y04zcAyX55XWS -80w9velYXitZ3slazNe35HwoFtawjkYZKKqMKAzJau0Y8g9dW1CT3JC41Mr8o385 -szE46YGXcnHeleT2rRA7Lpjg0Pg5orFRsPHeznlVrMVOM33g+09Q51rWeL6XpD0l -txUI5EGlgJXI73mSV1rQd5mXs+MAlRG0W5y47nMKUqX8MAqZ3MsEhQpsanp6A9QN -UYI0ZMGg842XOUq1FMieNuko+uU0GNeIxodVoD1NsKAGcdrAqFy8cBi8xkLtignw -YDXOElSwghRPGObmKR2ahf31/LmiV0xXZnnPsmB44uLXkxFQl7qRc6UjMM2w3ary -ydXW/pst7vtBr5WWAxgdc37IS/TTf+EcJWKtGPc6sXePi8AsMUnfjkmGmIYIgXlp -8niUxMbJbbqJOEyx/N7FkYGy9v/lb2hvwfOZrkwzDMkhCfDiUDvD/Qw6SqhH6ZRp -n6DjcbcEjtY9/JO1iTIBOllWXS4PHiBbv74CyZtamg4+cPEg/FLDlOmsGkyJAhwE -EAEIAAYFAlXlKIwACgkQJQHka7rtELygcBAAoSftrtuuDNYouDehOCYr8Z+w2jaJ -Shx4tbDG0f248wjf0nlk41MwaPULZHR0J9/tS3UxIi7kPOl2rpCXhc0fIkEOSHFT -RaHRpdMTh70DYDnBiej+2ZCBvBf9S3p11lJKwr1t+IL5cCQ4Wv58EH0ZVsjmC8A3 -H/h8uWnQROJCtJ+q4vrxijrpXu41wKUrYRVE8o0s+FVOdeiTjFhKJ+EgIhCmiRga -NZWAZHY8CvO30NMPUFryWr6h/L1/D9TMyNhG4iG56s1JVcESiQ5R7YXtNi1HHlLL -AXXIR2NW9FmrjR83xWoOjikls6X3HNUNY9Kh8aNU/1NU2U3rm9FtOu4Fx7ArliGR -nDVkvOKzDV3GCWGgwXsolUT7tsmibIBRpbj0rWLRcaALog8N1e3U7/qZo8ZIU3uZ -GJ2jK8AGiuQ/wMn7oojAklBg+IvU7k9mTkQW3q9XmkfA4u0WxyKnWU1KREUeDxdl -rR8k4BJi3Q6UZ+uhbzYJyz0pVFMTG8erPfdHlfT80KjFiNq8GvmkLGOS6rJ7d9nf -5YQtHTV1kUV5WaZUgzUBVa58plarfj9dBQTLx8vwSrtcdFv62QBA1+3rEBqai2uw -6KXNPwHcCarOQ9WsuMIqocZoHhhLe6YTmzrF7nrIy8FJHXC88TUCNHcTIFy6AYUF -DcMHiQVgaynWxt65Ag0EVa6nFwEQAN9L+rZPtHEusNO/cK5eTp8pH4in5W+wceHP -y+eTBD1VC2o2kRGVsswm+z3R8A+ev2g2BA2G5faLxAGEZTSnBC4eq0BeDvPsJwv5 -m+mHovTsvQBlbzygbRVnrYlXNTCQsVvHNeXW7hHgkszVlXE6vJwDHnU0PzlzQRRg -QyoNs9yU18Htt7o79rScO4jDYsvx5bx7uMOc8p3duABJsr9d5LatGnMCAsd+tqaW -6QYBl1HjVWUIUu41crIXzsKy2JIsBz2YR7RqcQ9p9xYASQu4cKH8nz6oXZevoK9c -yinTdW4J1dgtpEQk/yOJPxsZOlOYR8JXwVLmfC84piUuBk2ov3QLOlbGJSY57D1G -09KpAJ6Z/pERoy2fPM/gVyqr6romw2saFtqaKh8uT0suaZ9qf13dJMYzFKKwG4Im -Aw+mnqLNgfA3ErFuS3gpDNen0GwfrsH6vLeS4hbJsaO38hFDpSXn0GxcChFyzn+B -TtyddNcqQ4Gu6z03dXjc7gACyaSV/7fotsI2KeHGmMR8JapshZPmik2W2ghar/SS -oZbly7VeHMGEGa1MOFQjeg2uEfUI/3Su3knWXGGV7d5QtZT8Z075h1KeR+5H8Azd -X50WpnLb3tyWl3G1px2Zm/DSKGlcRW3seK5kWHsE4o6/cYPPVZFpgZ7iSEdjd8mv -g1wlHDC5ABEBAAGJAh8EGAECAAkFAlWupxcCGwwACgkQqY/qh6VuqYCTdxAAttwa -qwZZLDm7OKAABW8s2ylxNzluPtJBoB9bzW22tCL9Xrno8mItU0u0DW70VktH9EAT -u8q/j0/cuqAA4nRJwug2HWF13PA52RZRaxnCmSJY9TsMcTsCTuaXmbeQQ4DGVFCh -/i9CHqLyS7c9B9Fs5/kcth3Uf4kpxMuvciDmD2aC/CC6TLVVU7Qd3SLbfAF5bvpC -V4+U8UjQyyhghxmVItSoYcln0Bnnz9patnvMXNtpC+UG22wXUyXTWWHvotJw9YhJ -Sjs0ZdG9kGASO0fvOnOibx7b5jk2A9rM7Z7j7XqIamTdzZizyEJ14djOWM16HlNH -m5d4tnkVrtuYU+GaZEbLHCO1JN45ti0mcogePPz8YC7t9A8rKA/K3yTcPHf6naPT -jObQ2ctpd1CnB6uGX8kyvBadWz/+muicVn88cFPSi8UFtQTragxW/T0VssriRJZz -tv9IA01pjUCAZQaShWI1BBOLsf+aE19gecv9FrO2VHjoKyFf9EFSmUe1bFzWTNHB -jziwWmaLeH81GGAH8JRDnW6uHfoLEsduYUtQPSGp6PQ8mQRD+5MmHd3o0wWK5bE9 -qFhEBeHfQJZWwCISZw4zJp0GrFnE916PsasN3E5JKwNUYUW76TVAuffLOwPckaCf -sOlbyO/L1rckBpirTBm3tfemQp41k54f3Kff4qU= -=MY4N --END PGP PUBLIC KEY BLOCK-
svn commit: r36338 - /release/asterixdb/hyracks-0.3.4.1/apache-hyracks-0.3.4.1-source-release.zip.asc
Author: imaxon Date: Tue Oct 15 00:57:15 2019 New Revision: 36338 Log: add revocation certificate for compromised key. add new keys. resign releases with new key. Modified: release/asterixdb/hyracks-0.3.4.1/apache-hyracks-0.3.4.1-source-release.zip.asc Modified: release/asterixdb/hyracks-0.3.4.1/apache-hyracks-0.3.4.1-source-release.zip.asc == Binary files release/asterixdb/hyracks-0.3.4.1/apache-hyracks-0.3.4.1-source-release.zip.asc (original) and release/asterixdb/hyracks-0.3.4.1/apache-hyracks-0.3.4.1-source-release.zip.asc Tue Oct 15 00:57:15 2019 differ
svn commit: r36339 - in /release/asterixdb: ./ asterixdb-0.9.1/ asterixdb-0.9.2/ asterixdb-0.9.3/ asterixdb-0.9.4.1/ asterixdb-0.9.4/ hyracks-0.3.1/ hyracks-0.3.2/ hyracks-0.3.3/ hyracks-0.3.4/
Author: imaxon Date: Tue Oct 15 00:57:27 2019 New Revision: 36339 Log: add revocation certificate for compromised key. add new keys. resign releases with new key. Modified: release/asterixdb/KEYS release/asterixdb/asterixdb-0.9.1/apache-asterixdb-0.9.1-source-release.zip.asc release/asterixdb/asterixdb-0.9.1/asterix-installer-0.9.1-binary-assembly.zip.asc release/asterixdb/asterixdb-0.9.1/asterix-server-0.9.1-binary-assembly.zip.asc release/asterixdb/asterixdb-0.9.1/asterix-yarn-0.9.1-binary-assembly.zip.asc release/asterixdb/asterixdb-0.9.2/apache-asterixdb-0.9.2-source-release.zip.asc release/asterixdb/asterixdb-0.9.2/asterix-installer-0.9.2-binary-assembly.zip.asc release/asterixdb/asterixdb-0.9.2/asterix-server-0.9.2-binary-assembly.zip.asc release/asterixdb/asterixdb-0.9.2/asterix-yarn-0.9.2-binary-assembly.zip.asc release/asterixdb/asterixdb-0.9.3/apache-asterixdb-0.9.3-source-release.zip.asc release/asterixdb/asterixdb-0.9.3/asterix-installer-0.9.3-binary-assembly.zip.asc release/asterixdb/asterixdb-0.9.3/asterix-server-0.9.3-binary-assembly.zip.asc release/asterixdb/asterixdb-0.9.3/asterix-yarn-0.9.3-binary-assembly.zip.asc release/asterixdb/asterixdb-0.9.4.1/apache-asterixdb-0.9.4.1-source-release.zip.asc release/asterixdb/asterixdb-0.9.4.1/asterix-server-0.9.4.1-binary-assembly.zip.asc release/asterixdb/asterixdb-0.9.4/apache-asterixdb-0.9.4-source-release.zip.asc release/asterixdb/asterixdb-0.9.4/apache-asterixdb-0.9.4.zip.asc release/asterixdb/asterixdb-0.9.4/apache-asterixdb_0.9.4_all.deb.asc release/asterixdb/hyracks-0.3.1/apache-hyracks-0.3.1-source-release.zip.asc release/asterixdb/hyracks-0.3.2/apache-hyracks-0.3.2-source-release.zip.asc release/asterixdb/hyracks-0.3.3/apache-hyracks-0.3.3-source-release.zip.asc release/asterixdb/hyracks-0.3.4/apache-hyracks-0.3.4-source-release.zip.asc Modified: release/asterixdb/KEYS == --- release/asterixdb/KEYS (original) +++ release/asterixdb/KEYS Tue Oct 15 00:57:27 2019 @@ -0,0 +1,398 @@ +This file contains the PGP keys of various developers. + +Users: pgp < KEYS + gpg --import KEYS +Developers: +pgp -kxa and append it to this file. +(pgpk -ll && pgpk -xa ) >> this file. +(gpg --list-sigs + && gpg --armor --export ) >> this file. + + +Revocations: + +pub rsa4096 2015-07-21 [SC] + 3C4D3F19B2BDE3178423BF56A98FEA87A56EA980 +uid [ultimate] Ian Maxon (CODE SIGNING KEY) +sub rsa4096 2015-07-21 [E] + + +-BEGIN PGP PUBLIC KEY BLOCK- +Comment: This is a revocation certificate + +iQI2BCABCgAgFiEEPE0/GbK94xeEI79WqY/qh6VuqYAFAl2hDJgCHQIACgkQqY/q +h6VuqYC4bA//WGoqALkv8Qw+ZZMxkQtoiovXMvNekNBOSJk7zRf5oz7a+4BvAYHs +/VLX/QL+uDaYvE5Vo+RshoKKxvzyv9H8XhiK+MWZA6Yo4rYh6LBLvICUZ9KoAQCo +NI047NGMQFpgYh1jEbg6WPZFuzOZkkrNvRWBxXOMxJy+RGQ3XLAtBw9B/zhgw8y0 +sBZRYYsewocUz63z7EbpG6u4mAmhjFEISZKDK3PsCPRlTcAFNxy4mRwzLv0047U3 +Keum+Vorldg4m2AeeAbdijf28GRbDBlcdS9cBOwHPS6g1dJSACCctgy+smAcnbHX +wMTMvP1WXbq++M1JqwvttHWMuv9P1n1BoIW81yP9/MRP73FmgKkuB7KsuUqSA1pu +TR+8XhmHNThSOjZ5cCWvhkOiT4/1T19vJTRZpDTuUHjo/2xRUEh3BopdsR8tlSRW +Gz3cv7WuDlnSheIG4s9SdGGOwCY4N6nEOKYq123nc/vymL0uN7jxKwlX5/4eb/2y +hNQeH1enAru6Uar6xZC5j5KQ+YiMmbGRylKBBttjwuKuVVVLTir8maghvISutCD2 +h0AFJWjomfksve2VJqQXiJt+7+PjffSJP3ONgjzaJ2LZGrg73YvDE+sHHMkFkCi2 +jHCsiV0vluVbBB78kbswYRZCkfdF2bBtcbEo37srLei8OQu0VRvWj/E= +=Y4sG +-END PGP PUBLIC KEY BLOCK- + +Keys: + +pub rsa4096 2019-10-12 [C] + 216697FEF8AA7F48E4255D50C79BA55A768C8392 +uid [ unknown] Ian Maxon +sig 3C79BA55A768C8392 2019-10-12 Ian Maxon +sig AC2A2F6F04B1335D 2019-10-14 Xikui Wang (CODE SIGNING KEY) +sig E88FEED2DEDF69A1 2019-10-14 Chen Luo +uid [ unknown] Ian Maxon (CODE SIGNING KEY) +sig 3C79BA55A768C8392 2019-10-12 Ian Maxon +sig AC2A2F6F04B1335D 2019-10-14 Xikui Wang (CODE SIGNING KEY) +sig E88FEED2DEDF69A1 2019-10-14 Chen Luo +sub rsa4096 2019-10-12 [S] [expires: 2020-10-11] +sig C79BA55A768C8392 2019-10-12 Ian Maxon +sub rsa4096 2019-10-12 [E] [expires: 2020-10-11] +sig C79BA55A768C8392 2019-10-12 Ian Maxon +sub rsa4096 2019-10-12 [A] [expires: 2020-10-11] +sig C79BA55A768C8392 2019-10-12 Ian Maxon + +-BEGIN PGP PUBLIC KEY BLOCK- + +mQINBF2hJG4BEADS4dlWylcc1KUkSvDB+sLuO+YDOoII9LXYsoUkFD5YRf3SnDz1 +Zp74Fyhvj1uCR6RzSuGKt51OT87M4F8klJVXBBxnfn73wlYQh902kmM+QyiV6mdO +Uo6fFQMavVUAbN3SWLDUYVCb3n+Yu+2yIHsjauHP7nDWE8xqN0SxUMAz13ZyuuUI +v9b6X1DDRhXix2Foiud8zDppvxCW878LObgvtiDAQBP5ayQSFcrgwZmemDmgwPv5 +4GCqLOJvqPNnSn71nMdp28QvAtFGpQcm61hWr+IsD4fW85bvu8c8LMUeau0DV5J1 +9uQ8gQqzu/NJ6gEuyvGhefSOHnMj7g4vrTqsFK/PzcT5FcK3THSN3Wmo1bzmvV7w +AS6rhLdSiHDKtOpvgB1eTsWOBntsPPuJr687K2uJ2hbrlGNgEstoaCcf5BGQ1N/E +QNW+i7J/I/Y/zujQhO5VEtLahrlTIc6x+
svn commit: r36348 - /release/asterixdb/KEYS
Author: imaxon Date: Tue Oct 15 17:40:19 2019 New Revision: 36348 Log: add wyk's key Modified: release/asterixdb/KEYS Modified: release/asterixdb/KEYS == --- release/asterixdb/KEYS (original) +++ release/asterixdb/KEYS Tue Oct 15 17:40:19 2019 @@ -39,14 +39,16 @@ Keys: pub rsa4096 2019-10-12 [C] 216697FEF8AA7F48E4255D50C79BA55A768C8392 -uid [ unknown] Ian Maxon +uid [ultimate] Ian Maxon sig 3C79BA55A768C8392 2019-10-12 Ian Maxon sig AC2A2F6F04B1335D 2019-10-14 Xikui Wang (CODE SIGNING KEY) sig E88FEED2DEDF69A1 2019-10-14 Chen Luo -uid [ unknown] Ian Maxon (CODE SIGNING KEY) +sig CFDA9461C5B1A725 2019-10-15 Wail Y.Alkowaileet +uid [ultimate] Ian Maxon (CODE SIGNING KEY) sig 3C79BA55A768C8392 2019-10-12 Ian Maxon sig AC2A2F6F04B1335D 2019-10-14 Xikui Wang (CODE SIGNING KEY) sig E88FEED2DEDF69A1 2019-10-14 Chen Luo +sig CFDA9461C5B1A725 2019-10-15 Wail Y.Alkowaileet sub rsa4096 2019-10-12 [S] [expires: 2020-10-11] sig C79BA55A768C8392 2019-10-12 Ian Maxon sub rsa4096 2019-10-12 [E] [expires: 2020-10-11] @@ -103,126 +105,149 @@ nUqthppmxfRd/1yYXbmgNJQ8CSxCMp2qQ/a2Pt+d 1brCraTTETWBGw3hrUf+X2INR3l942vSY4Y1kPyltboLXfKFJmpTGMUVxzZMk6qr D+wkB7Djw/bcKKyWI03cq4EcPb9tbo3s5ZVGzSfAXp7rb3oBXH9EngNm0p8cTO0q V6tZvXvwNB9xi4/Ygesou7eTSrZ8N36ggrG+CvlgcTiVZ/eYp2fWpii64peJcr2k -sQFnP0/F00zUQPAF142pwDYuvEU8C/q9jXAwS4/y/VG6sir5LR0qRbtpgsi0HUlh -biBNYXhvbiA8aW1heG9uQGFwYWNoZS5vcmc+iQJMBBMBCgA2FiEEIWaX/viqf0jk -JV1Qx5ulWnaMg5IFAl2hJXwCGwEECwkIBwQVCgkIBRYCAwEAAh4BAheAAAoJEMeb -pVp2jIOS+iYP+wa09i+x1F/hSbMob3EbtniLZb3yFN0e1IvN4+U/d/nAZzjNWfgj -ZhX2s3f9HRkAxIUKD0/cyNvSkRJKMNYcfheiccghMykPqf0eCXxxSkvp4btn37WI -tgufY3YZ7IjpGleLZ3ql4BUPZmgtFa0CTl7/hIwfqG0RalN5349rK+7hwxdr/Ybb -C7kSLDXh7bOfs7JL5kuSIzTX/N+QQ3qm16FEKxvObsbkLt4sBPzS4mRSVgNVLehz -kXifIoUYqONkeuUMqSkTvojxQRcAXnQfZw3axDCEhha5kkBZCR7HprSnLmMfNfrH -HCsGmovHyOWafUc40xAXdxqrPp26kb6vq4NoYjnJ882kADxZ+XZ4eVTln6FoD94C -qulIAcKb9dAf/Qvpm1oCLWpLCromTygqgmmcDkqAf/6SqMUIbzemQcxml5yCYGlF -8zStm5e4AcpU8KvOCPNSY81vu6C0uClyodsE3YCPr0zoU+UB/8JMnwxCRbxdHSRF -tkbxyZxHEPMKtYV6Cn2Bd5j/xHIVc3heWyAAGO9+5cJj/4A0IHme4DBdaBzSOw+D -9WEpZfMt4yhjHOO6wvJ/1W++w6a7jtz2RBiWIi+Uy1XwNqt42ZVIpMt1UCj3LQw9 -Sqy6TR+aE3/kwrGJBcQaRNKmarVaOlFjnmfu7tuGaUIxsOkeO1Roa268iQIzBBAB -CAAdFiEEu0fz0XfzmVKy4leqrCovbwSxM10FAl2lBtUACgkQrCovbwSxM13swxAA -jcn1L0f0oINDPmFr9fkM+kisEapoih8VCba2JBbeSdihRyeLJmJk6Jf4mLy/g1dX -Ly/6ZOjhDchwlnoP5Rk7sk0gPFW2VgmG2ciAjIp6wxQPkAThEvPdTQEXC6gZtbWl -htc7QlTs8jyo4L0L0/noi7KCSWtp9ijoVhNGafMb/LbmX5pjR5jWuCM38+7tDuyo -hOiRVC4mmtZLqHEACH6srLDATxZ8AukAHTcHORmuBvj45Am/aE1B8KHz1eaFP2ia -3AMJgr528ZxJqEPcKeH5CwECmGJY00WQGfaDdQDBMPGz8hS8h1ajic+5Xh6cNzDQ -/4/Nd5MM7uODXRY/78mKZxj+IME0pD4csPGUuYSBm+ouwNv2IoWyHisPW/A1wYih -1oxyuuFkTWPcX4luVTtIxNSvt9K4YXltYdvRBzGVAaALMMZQwEAlwJQyp9ut2N3W -0XKJJEqnCXZVn3bAxG7hccZnWMZZ5BMyykIiRGWoe2GKhpGW5hnbPILldHGzxNMz -9AsDa6POQi5S3zAsNFWTCUuo+BL02Dhy/ZkahANB3Tdy+hOBnaZgD3I9SdbOG7On -FdlA66W7yT23XsUST2onNvK228vhytbQTfA3oRlxeSCC0nHnpJu4U9fGcmfrdL27 -ZuUI0cRj98SAEck117Q1HBj9ncu4PLzQ7ntlJsPbYYuJAjMEEAEIAB0WIQQ0x0wS -uZMQ1K5XDCLoj+7S3t9poQUCXaUHcwAKCRDoj+7S3t9poTfaD/9DlbDE5vl5PJko -j8sZQLW6wjyFabwFPBFVmsEAjWuXeYJc05VQCEbESht9KyP12d81DCw9zJkboYoJ -YC11jYFzzzsWHUB7F7d5uwZotR/LInwOS2q1TY0Fer+33oob2Ixhqef8g3CeBV4N -CVpKQC9iODquOUlt4c8BUMQT2FIovY+H07npE/WVjME6BGvzESscQ/CBQoWR1b/C -vn6zGn8SpO1zNVcpUdvRVWzLMLgpIn1OoN4u8dUtTLIkKiC9ZZtX4TKShykyTAiL -NtkQwVWqBHRvfAxa9L0nE72KUOyqdVphtHDF9bTjyWetp7YaAF022Pu1WxmefrEZ -qGTqFo7DpoYZIbSHvx7s8sqCUUJeLipxKT2dUYm1kNxso0IKtzoJQnX12J6DEQms -48eHlYAAP0B/PgO0i8izLQH+/NCRwKu7Fbr1b9xjKiXnshS3MfAFw1lMXubGFDwM -dPNDKVOLCl7zyl650X3g+F42jV2AjC7Vbfw59cq5eGRUu+R+Wv9WHrLo48j58buj -SyH/kxy8zl8FHfZpnJJm/jOdgWkPvFllP7ZG9eIs84Dl9f2nwt+SbURjeAuf/9Fg -Ltc9thRtaF6n02Q5Y9LAgrZOCxPV4rPeOqRZvr5RA7zzdYzXuYIxKEtMwsmKLyA1 -MYmaStVu/SlloeeJacytLIpIK+KZZLkCDQRdoSTqARAApWsDdqzXi4IKZo6Dltvm -za724WwwfI+DwRKheb4pYA6fKfxG83y7PUW0TUwIG7V5eSQuZh2bPKHgaJlpdviH -WFOhcHApi9ln+jO/RZcjBxpOGTYMoyGupiuuXsjuzQ26cdF4UYMyftNO2YcrgcXt -T1GgJcdO/yOoEEknIYGJ9Pg4IvqbR2BE4pZ2BF4vgzjHM0sV5Fm1QY7FjGmDWA3k -8Jfws6xiBgl3fsGw8dWBXLUYx/jhOmSgtGgZEMQY/dSEZP2CoEFhM60zVtAIME1q -zYqPwdaceaUsvDz6uZvoX8jwY9YDxe/017ZvIh6BA2XgmzroYqtetelS1q20SrEa -CVCq6keobETkKXShGVPq35OMeIAnzNucrISzKW3BOSdxNCO6XMwTVBXMcNA2WImV -N126aNjYr8GPXUiaQ+zlNMVN5q0L+95f24GagNTTID60ctXsA+aTYitHVPe2qXvR -Os0VyGO3k28qrVXQZZ0+9WzTyJLnUBIZhGBNFqlrRVMRyhcBRzdy9cigVjbMweHt -bi3tVVNbbQkjypAYQ/9MJeXvz4LSRnAc7B+44po5glTOd+eDvlWljwOCDlIObSVV -T0Tx8kH4+VQ3S3ctVux2Gm/W1hKjK5p8u54AcD/tFHmD6l8xXlq86bSlNAinOvaB -kCWrdM6kh8v0R3HnITrt1YcAEQEAAYkEcgQYAQoAJhYhBCFml/74qn9I5CVdUMeb -pVp2jIOSBQJdoSTqAhsCBQkB4TOAAkAJEMebpVp2jIOSwXQgBBkBCgAdFiEEIf3g -bxaFXS93IFrkwLkA3UcwFf0FAl2hJOoACgkQwLkA3UcwFf2yPg//YLSeiQvQ3u68 -kCeuPkTwpjrw7vdlg23HSyFfTc2gdJh2KuxULpd38nOtGJcmUfBIAiqHeMAXcLSA -6SZHPjJYeBWXyPcmT1A
[asterixdb] branch master updated: [NO ISSUE][RT] Profiling fixes
This is an automated email from the ASF dual-hosted git repository. imaxon pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/asterixdb.git The following commit(s) were added to refs/heads/master by this push: new c775dd9 [NO ISSUE][RT] Profiling fixes c775dd9 is described below commit c775dd9d2c01a66b9f2da8447c75400bac6ddf90 Author: Ian Maxon AuthorDate: Wed Nov 20 18:43:33 2019 -0800 [NO ISSUE][RT] Profiling fixes - Fix profile serialization for profiles > 64kb in size - Break deadlock for OperatorNodePushables that keep initialize() open and call downstream operators that block until other operators have initialized (e.g. Intersect). Change-Id: I87ec970eaf2d5db76e7bfaa60be9190efb1a70ae Reviewed-on: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/4263 Tested-by: Jenkins Integration-Tests: Jenkins Reviewed-by: Sandeep Gupta --- .../asterix/translator/IStatementExecutor.java | 31 +-- .../api/dataflow/TimedOperatorNodePushable.java| 36 +++--- 2 files changed, 47 insertions(+), 20 deletions(-) diff --git a/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/translator/IStatementExecutor.java b/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/translator/IStatementExecutor.java index c0cf8eb..a5c8162 100644 --- a/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/translator/IStatementExecutor.java +++ b/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/translator/IStatementExecutor.java @@ -54,11 +54,15 @@ import org.apache.hyracks.api.result.ResultSetId; import com.fasterxml.jackson.databind.JsonNode; import com.fasterxml.jackson.databind.ObjectMapper; import com.fasterxml.jackson.databind.node.ObjectNode; +import com.google.common.base.Splitter; +import com.google.common.collect.Lists; /** * An interface that takes care of executing a list of statements that are submitted through an Asterix API */ public interface IStatementExecutor { +public static final char UNIT_SEPARATOR = 31; +public static final char END_OF_BLOCK = 23; /** * Specifies result delivery of executed statements @@ -200,12 +204,35 @@ public interface IStatementExecutor { private void writeObject(ObjectOutputStream out) throws IOException { ObjectMapper om = new ObjectMapper(); -out.writeUTF(om.writeValueAsString(profile)); +java.lang.String prof = om.writeValueAsString(profile); +//split the string if it is >=64K to avoid writeUTF limit +List pieces; +if (prof.length() > 65534L) { +pieces = Lists.newArrayList(Splitter.fixedLength(32768).split(prof)); +} else { +pieces = Lists.newArrayList(prof); +} + +for (int i = 0; i < pieces.size(); i++) { +if (i == pieces.size() - 1) { +out.writeChar(UNIT_SEPARATOR); +} else { +out.writeChar(END_OF_BLOCK); +} +out.writeUTF(pieces.get(i)); +} + } private void readObject(ObjectInputStream in) throws IOException, ClassNotFoundException { ObjectMapper om = new ObjectMapper(); -JsonNode inNode = om.readTree(in.readUTF()); +StringBuilder objSplits = new StringBuilder(); +for (char cmd = in.readChar(); cmd != END_OF_BLOCK && cmd == UNIT_SEPARATOR; cmd = in.readChar()) { +objSplits.append(in.readUTF()); +} +objSplits.append(in.readUTF()); + +JsonNode inNode = om.readTree(objSplits.toString()); if (!inNode.isObject()) { throw new IOException("Deserialization error"); } diff --git a/hyracks-fullstack/hyracks/hyracks-api/src/main/java/org/apache/hyracks/api/dataflow/TimedOperatorNodePushable.java b/hyracks-fullstack/hyracks/hyracks-api/src/main/java/org/apache/hyracks/api/dataflow/TimedOperatorNodePushable.java index 2d46bea..1d47c98 100644 --- a/hyracks-fullstack/hyracks/hyracks-api/src/main/java/org/apache/hyracks/api/dataflow/TimedOperatorNodePushable.java +++ b/hyracks-fullstack/hyracks/hyracks-api/src/main/java/org/apache/hyracks/api/dataflow/TimedOperatorNodePushable.java @@ -33,7 +33,7 @@ public class TimedOperatorNodePushable extends TimedFrameWriter implements IOper HashMap inputs; long frameStart; -TimedOperatorNodePushable(IOperatorNodePushable op, IStatsCollector collector) throws HyracksDataException { +private TimedOperatorNodePushable(IOperatorNodePushable op, IStatsCollector collector) { super(null, collector, op.getDisplayName()); this.op = op; inputs = new HashMap<>(); @@ -41,20 +41,16 @@ public class TimedOperatorNodePushable extends TimedFrameWriter implements IOp
[asterixdb] branch master updated: [NO ISSUE] HTTP Basic Auth servlet wrapper
This is an automated email from the ASF dual-hosted git repository. imaxon pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/asterixdb.git The following commit(s) were added to refs/heads/master by this push: new 694ffd1 [NO ISSUE] HTTP Basic Auth servlet wrapper 694ffd1 is described below commit 694ffd194ce5c6e610f61368c1511778d0bff254 Author: Ian Maxon AuthorDate: Mon Mar 9 19:08:05 2020 -0700 [NO ISSUE] HTTP Basic Auth servlet wrapper - Basic auth servlet that checks a /etc/passwd style file on the CC - Only enabled for UDF API servlet for now - Associated code in client helper for adding file to CC path - Config variable to manually set path if desired Change-Id: Ibea23a2e8308937f343d80eff04ede9a235aa0d1 Reviewed-on: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/5204 Contrib: Jenkins Integration-Tests: Jenkins Tested-by: Jenkins Reviewed-by: Michael Blow --- asterixdb/asterix-app/pom.xml | 8 ++ .../asterix/api/http/server/BasicAuthServlet.java | 105 + .../asterix/api/http/server/ServletConstants.java | 1 + .../asterix/api/http/server/UdfApiServlet.java | 3 +- .../asterix/hyracks/bootstrap/CCApplication.java | 29 ++ .../api/common/AsterixHyracksIntegrationUtil.java | 8 +- .../asterix/app/external/ExternalUDFLibrarian.java | 38 ++-- .../app/external/IExternalUDFLibrarian.java| 7 +- .../apache/asterix/test/common/TestExecutor.java | 13 ++- .../asterix/test/runtime/SqlppExecutionIT.java | 2 +- asterixdb/asterix-app/src/test/resources/cc.conf | 1 + .../deterministic/deterministic.1.lib.sqlpp| 2 +- .../deterministic/deterministic.5.lib.sqlpp| 2 +- .../getCapital/getCapital.1.lib.sqlpp | 2 +- .../getCapital/getCapital.4.lib.sqlpp | 2 +- .../getCapital_open/getCapital_open.1.lib.sqlpp| 2 +- .../getCapital_open/getCapital_open.4.lib.sqlpp| 2 +- .../keyword_detector/keyword_detector.1.lib.sqlpp | 2 +- .../my_array_sum/my_array_sum.1.lib.sqlpp | 2 +- .../my_array_sum/my_array_sum.4.lib.sqlpp | 2 +- .../external-library/mysum/mysum.1.lib.sqlpp | 2 +- .../external-library/mysum/mysum.7.lib.sqlpp | 2 +- .../mysum_bad_credential.0.ddl.sqlpp} | 3 +- .../mysum_bad_credential.1.lib.sqlpp} | 2 +- .../mysum_bad_credential.2.lib.sqlpp} | 2 +- .../mysum_bad_credential.3.lib.sqlpp} | 2 +- .../mysum_bad_credential.4.lib.sqlpp} | 2 +- .../mysum_bad_credential.5.lib.sqlpp} | 2 +- .../mysum_bad_credential.6.lib.sqlpp} | 2 +- .../return_invalid_type.1.lib.sqlpp| 2 +- .../type_validation/type_validation.1.lib.sqlpp| 2 +- .../type_validation/type_validation.4.lib.sqlpp| 2 +- .../udf_filter_on_feed.3.lib.sqlpp | 2 +- .../udf_metadata/udf_metadata.1.lib.sqlpp | 2 +- .../udf_metadata/udf_metadata.4.lib.sqlpp | 2 +- .../upperCase/upperCase.1.lib.sqlpp| 2 +- .../upperCase/upperCase.4.lib.sqlpp| 2 +- .../feed-with-external-function.1.lib.sqlpp| 2 +- .../feed-with-external-function.6.lib.sqlpp| 2 +- ...ith-external-parser-with-open-index.2.lib.sqlpp | 2 +- ...ith-external-parser-with-open-index.6.lib.sqlpp | 2 +- ...ternal-parser-with-two-open-indexes.2.lib.sqlpp | 2 +- ...ternal-parser-with-two-open-indexes.6.lib.sqlpp | 2 +- .../feed-with-external-parser.2.lib.sqlpp | 2 +- .../feed-with-external-parser.6.lib.sqlpp | 2 +- .../feed-with-multiple-indexes.2.lib.sqlpp | 2 +- .../feed-with-multiple-indexes.6.lib.sqlpp | 2 +- .../resources/runtimets/testsuite_it_sqlpp.xml | 9 ++ .../asterix-app/src/test/resources/security/passwd | 1 + asterixdb/asterix-client-helper/pom.xml| 12 +++ .../java/org/apache/asterix/clienthelper/Args.java | 21 + .../clienthelper/AsterixHelperExecution.java | 3 + .../clienthelper/commands/ClientCommand.java | 5 +- .../commands/GenerateCredentialCommand.java| 57 +++ .../asterix/common/config/ExternalProperties.java | 17 +++- asterixdb/asterix-server/pom.xml | 10 ++ asterixdb/pom.xml | 10 ++ .../appended-resources/supplemental-models.xml | 12 +++ .../opensource.org_licenses_isc-license.txt| 3 + .../www.mindrot.org_files_jBCrypt_LICENSE.txt | 17 .../control/common/controllers/CCConfig.java | 1 + 61 files changed, 405 insertions(+), 61 deletions(-) diff --git a/asterixdb/asterix-app/pom.xml b/asterixdb/asterix-app/pom.xml index ac2c303..e39bb1f 100644 --- a/asterixdb/asterix-app/pom.xml +++ b/asterixdb/asterix-app/pom.xml @@ -699,5 +699,13
[asterixdb] branch master updated: [NO ISSUE] Add UTF8StringUtil.getUTF8StringInArray() for tokenizer scenario
This is an automated email from the ASF dual-hosted git repository. imaxon pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/asterixdb.git The following commit(s) were added to refs/heads/master by this push: new 38f652a [NO ISSUE] Add UTF8StringUtil.getUTF8StringInArray() for tokenizer scenario 38f652a is described below commit 38f652a0dc144d0064f54c0505968eddc362e45b Author: Rui Guo AuthorDate: Fri Apr 3 22:04:01 2020 -0700 [NO ISSUE] Add UTF8StringUtil.getUTF8StringInArray() for tokenizer scenario Change-Id: I273a776f14a2846e5380f2bdc4a3168a1dac052c Reviewed-on: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/5565 Contrib: Jenkins Tested-by: Jenkins Reviewed-by: Ian Maxon Integration-Tests: Jenkins --- .../apache/hyracks/util/string/UTF8StringUtil.java | 14 + .../hyracks/util/string/UTF8StringUtilTest.java| 23 ++ 2 files changed, 37 insertions(+) diff --git a/hyracks-fullstack/hyracks/hyracks-util/src/main/java/org/apache/hyracks/util/string/UTF8StringUtil.java b/hyracks-fullstack/hyracks/hyracks-util/src/main/java/org/apache/hyracks/util/string/UTF8StringUtil.java index f50fa90..2b0e49e 100644 --- a/hyracks-fullstack/hyracks/hyracks-util/src/main/java/org/apache/hyracks/util/string/UTF8StringUtil.java +++ b/hyracks-fullstack/hyracks/hyracks-util/src/main/java/org/apache/hyracks/util/string/UTF8StringUtil.java @@ -324,6 +324,20 @@ public class UTF8StringUtil { return builder; } +// Different from the above toString() methods, here we assume the byte[] doesn't contain NumBytesToStoreLength +// In fact, this is used for string tokenizer: get "hello" and "world" from the bytes of "hello world" +public static String getUTF8StringInArray(byte[] b, int start, int len) { +StringBuilder builder = new StringBuilder(); + +for (int i = start; i < start + len;) { +char c = UTF8StringUtil.charAt(b, i); +builder.append(c); +i += UTF8StringUtil.charSize(b, i); +} + +return builder.toString(); +} + public static void printUTF8StringWithQuotes(byte[] b, int s, int l, OutputStream os) throws IOException { printUTF8String(b, s, l, os, true); } diff --git a/hyracks-fullstack/hyracks/hyracks-util/src/test/java/org/apache/hyracks/util/string/UTF8StringUtilTest.java b/hyracks-fullstack/hyracks/hyracks-util/src/test/java/org/apache/hyracks/util/string/UTF8StringUtilTest.java index b75d68c..2c99104 100644 --- a/hyracks-fullstack/hyracks/hyracks-util/src/test/java/org/apache/hyracks/util/string/UTF8StringUtilTest.java +++ b/hyracks-fullstack/hyracks/hyracks-util/src/test/java/org/apache/hyracks/util/string/UTF8StringUtilTest.java @@ -31,6 +31,7 @@ import static org.apache.hyracks.util.string.UTF8StringUtil.compareTo; import static org.apache.hyracks.util.string.UTF8StringUtil.getModifiedUTF8Len; import static org.apache.hyracks.util.string.UTF8StringUtil.getNumBytesToStoreLength; import static org.apache.hyracks.util.string.UTF8StringUtil.getStringLength; +import static org.apache.hyracks.util.string.UTF8StringUtil.getUTF8StringInArray; import static org.apache.hyracks.util.string.UTF8StringUtil.getUTFLength; import static org.apache.hyracks.util.string.UTF8StringUtil.hash; import static org.apache.hyracks.util.string.UTF8StringUtil.lowerCaseCompareTo; @@ -42,6 +43,7 @@ import static org.junit.Assert.assertEquals; import static org.junit.Assert.assertTrue; import java.io.IOException; +import java.util.List; import org.junit.Test; @@ -153,4 +155,25 @@ public class UTF8StringUtilTest { assertTrue(familyOne != familyTwo); } +@Test +public void testGetUTF8StringInArray() { +String str = null; +byte[] bytes = null; +List answer = null; + +str = "database group at university of California, Irvine 2"; +bytes = writeStringToBytes(str); +// First byte in bytes is for the number of bytes of the entire string, +// and it should be skipped in getUTF8StringInArray +assertEquals("database", getUTF8StringInArray(bytes, 1, 8)); +assertEquals("at", getUTF8StringInArray(bytes, 16, 2)); +// test upper case +assertEquals("California", getUTF8StringInArray(bytes, 33, 10)); +// test non-english char +assertEquals(",", getUTF8StringInArray(bytes, 43, 1)); +assertEquals("Irvine", getUTF8StringInArray(bytes, 45, 6)); +// test number +assertEquals("2", getUTF8StringInArray(bytes, 52, 5)); +} + }
[asterixdb] 03/03: [NO ISSUE] Remove out-of-date tokenizer
This is an automated email from the ASF dual-hosted git repository. imaxon pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/asterixdb.git commit f85cee9e60c4a68bc2a7de87284c6b812794f76b Author: Rui Guo AuthorDate: Mon Apr 13 10:04:19 2020 -0700 [NO ISSUE] Remove out-of-date tokenizer The string-based Tokenizer should be replaced with the array-based IBinaryTokenizer. The Tokenizer is not used in the codebase in a meaningful way, so let's remove it to make things clear. Change-Id: I483604bf2a5e20c18f6224ac2a153667828dabfb Reviewed-on: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/5763 Integration-Tests: Jenkins Tested-by: Jenkins Reviewed-by: Ian Maxon --- .../asterix/fuzzyjoin/FuzzyJoinAppendLength.java | 58 - .../asterix/fuzzyjoin/FuzzyJoinTokenize.java | 133 - .../fuzzyjoin/similarity/SimilarityMetric.java | 4 - .../similarity/SimilarityMetricJaccard.java| 21 .../fuzzyjoin/tokenizer/NGramTokenizer.java| 90 -- .../apache/asterix/fuzzyjoin/tokenizer/Token.java | 118 -- .../asterix/fuzzyjoin/tokenizer/Tokenizer.java | 27 - .../fuzzyjoin/tokenizer/TokenizerBuffered.java | 30 - .../tokenizer/TokenizerBufferedFactory.java| 34 -- .../fuzzyjoin/tokenizer/TokenizerFactory.java | 31 - .../asterix/fuzzyjoin/tokenizer/WordTokenizer.java | 68 --- .../fuzzyjoin/tokenizer/WordTokenizerBuffered.java | 92 -- .../fuzzyjoin/tokenorder/IntTokenCountRank.java| 28 - .../tokenorder/IntTokenCountRankFrequency.java | 58 - .../asterix/fuzzyjoin/tokenorder/IntTokenRank.java | 28 - .../tokenorder/IntTokenRankFrequency.java | 54 - .../asterix/fuzzyjoin/tokenorder/TokenLoad.java| 61 -- .../asterix/fuzzyjoin/tokenorder/TokenRank.java| 31 - .../tokenorder/TokenRankBufferedFrequency.java | 75 .../fuzzyjoin/tokenorder/TokenRankFrequency.java | 61 -- 20 files changed, 1102 deletions(-) diff --git a/asterixdb/asterix-fuzzyjoin/src/main/java/org/apache/asterix/fuzzyjoin/FuzzyJoinAppendLength.java b/asterixdb/asterix-fuzzyjoin/src/main/java/org/apache/asterix/fuzzyjoin/FuzzyJoinAppendLength.java deleted file mode 100644 index 8be6f0c..000 --- a/asterixdb/asterix-fuzzyjoin/src/main/java/org/apache/asterix/fuzzyjoin/FuzzyJoinAppendLength.java +++ /dev/null @@ -1,58 +0,0 @@ -/** - * Licensed to the Apache Software Foundation (ASF) under one - * or more contributor license agreements. See the NOTICE file - * distributed with this work for additional information - * regarding copyright ownership. The ASF licenses this file - * to you under the Apache License, Version 2.0 (the - * "License"); you may not use this file except in compliance - * with the License. You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, - * software distributed under the License is distributed on an - * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY - * KIND, either express or implied. See the License for the - * specific language governing permissions and limitations - * under the License. - */ - -package org.apache.asterix.fuzzyjoin; - -import java.io.BufferedReader; -import java.io.BufferedWriter; -import java.io.FileReader; -import java.io.FileWriter; -import java.io.IOException; -import java.util.Collection; - -import org.apache.asterix.fuzzyjoin.tokenizer.Tokenizer; -import org.apache.asterix.fuzzyjoin.tokenizer.TokenizerFactory; - -public class FuzzyJoinAppendLength { -public static void main(String args[]) throws IOException { -final String inputFileName = args[0]; -final String outputFileName = args[1]; - -BufferedReader input = new BufferedReader(new FileReader(inputFileName)); -BufferedWriter output = new BufferedWriter(new FileWriter(outputFileName)); - -Tokenizer tokenizer = TokenizerFactory.getTokenizer(FuzzyJoinConfig.TOKENIZER_VALUE, -FuzzyJoinConfig.WORD_SEPARATOR_REGEX, FuzzyJoinConfig.TOKEN_SEPARATOR); - -int[] dataColumns = FuzzyJoinUtil.getDataColumns("2,3"); - -String line; -while ((line = input.readLine()) != null) { -String[] splits = line.split(FuzzyJoinConfig.RECORD_SEPARATOR_REGEX); -Collection tokens = -tokenizer.tokenize(FuzzyJoinUtil.getData(splits, dataColumns, FuzzyJoinConfig.TOKEN_SEPARATOR)); -output.write(splits[0] + FuzzyJoinConfig.RECORD_SEPARATOR + splits[1] + FuzzyJoinConfig.RECORD_SEPARATOR -+ splits[2] + FuzzyJoinConfig.RECORD_SEPARATOR + splits[3] + FuzzyJoinConfig.RECORD_SEPARATOR -+ tokens.size() + "
[asterixdb] 01/03: [ASTERIXDB-2709][COMP] Fix registration of st_union aggregates
This is an automated email from the ASF dual-hosted git repository. imaxon pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/asterixdb.git commit 3d0897b6532b276d20fb7fc255f91de58002d7b8 Author: Dmitry Lychagin AuthorDate: Fri Apr 10 11:20:49 2020 -0700 [ASTERIXDB-2709][COMP] Fix registration of st_union aggregates - user model changes: no - storage format changes: no - interface changes: no Details: - Fix registration of st_union() aggregate function in BuiltinFunctions Change-Id: Iff0e141a5874b87916bd07335594fb6350ed93c7 Reviewed-on: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/5704 Contrib: Jenkins Integration-Tests: Jenkins Tested-by: Jenkins Reviewed-by: Dmitry Lychagin Reviewed-by: Ali Alsuliman --- .../two-geometries/aggregate.17.query.sqlpp| 23 ++ .../results/geojson/two-geometries/result.17.adm | 1 + .../asterix/om/functions/BuiltinFunctions.java | 28 ++ 3 files changed, 48 insertions(+), 4 deletions(-) diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/geojson/two-geometries/aggregate.17.query.sqlpp b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/geojson/two-geometries/aggregate.17.query.sqlpp new file mode 100644 index 000..578ba95 --- /dev/null +++ b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/geojson/two-geometries/aggregate.17.query.sqlpp @@ -0,0 +1,23 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ +use GeoJSON1; + +strict_st_union(( + SELECT VALUE gbu FROM [st_make_point(1.0,1.0),st_make_point(1.0,2.0)] as gbu +)); \ No newline at end of file diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/results/geojson/two-geometries/result.17.adm b/asterixdb/asterix-app/src/test/resources/runtimets/results/geojson/two-geometries/result.17.adm new file mode 100644 index 000..5123e09 --- /dev/null +++ b/asterixdb/asterix-app/src/test/resources/runtimets/results/geojson/two-geometries/result.17.adm @@ -0,0 +1 @@ +{"type":"MultiPoint","coordinates":[[1,1],[1,2]],"crs":{"type":"name","properties":{"name":"EPSG:4326"}}} \ No newline at end of file diff --git a/asterixdb/asterix-om/src/main/java/org/apache/asterix/om/functions/BuiltinFunctions.java b/asterixdb/asterix-om/src/main/java/org/apache/asterix/om/functions/BuiltinFunctions.java index 3570aff..19c1b51 100644 --- a/asterixdb/asterix-om/src/main/java/org/apache/asterix/om/functions/BuiltinFunctions.java +++ b/asterixdb/asterix-om/src/main/java/org/apache/asterix/om/functions/BuiltinFunctions.java @@ -1326,13 +1326,22 @@ public class BuiltinFunctions { public static final FunctionIdentifier ST_LENGTH = new FunctionIdentifier(FunctionConstants.ASTERIX_NS, "st-length", 1); public static final FunctionIdentifier SCALAR_ST_UNION_AGG = -new FunctionIdentifier(FunctionConstants.ASTERIX_NS, "st-union", 1); +new FunctionIdentifier(FunctionConstants.ASTERIX_NS, "st_union", 1); public static final FunctionIdentifier SCALAR_ST_UNION_AGG_DISTINCT = -new FunctionIdentifier(FunctionConstants.ASTERIX_NS, "st-union-distinct", 1); +new FunctionIdentifier(FunctionConstants.ASTERIX_NS, "st_union-distinct", 1); public static final FunctionIdentifier ST_UNION_AGG = -new FunctionIdentifier(FunctionConstants.ASTERIX_NS, "st-union-agg", 1); +new FunctionIdentifier(FunctionConstants.ASTERIX_NS, "agg-st_union", 1); public static final FunctionIdentifier ST_UNION_AGG_DISTINCT = -new FunctionIdentifier(FunctionConstants.ASTERIX_NS, "st-union-agg-distinct", 1); +new FunctionIdentifier(FunctionConstants.ASTERIX_NS, "agg-st_union-distinct", 1); +public static final FunctionIdentifier SCALAR_ST_UNION_SQL_AGG = +new FunctionIdentifier(FunctionConstants.ASTERIX_NS, "sql-st_union&
[asterixdb] 02/03: [NO ISSUE] Rename tokenizer factories
This is an automated email from the ASF dual-hosted git repository. imaxon pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/asterixdb.git commit 934ce9d903d36990d22febd69e08a81d0c8e40b2 Author: Rui Guo AuthorDate: Tue Apr 14 12:17:13 2020 -0700 [NO ISSUE] Rename tokenizer factories Rename the variables to make things clear. Previously, tokenizer factories were named tokenizer which could be confusing. Change-Id: Ie72d420e18509489d0fc0f9c98b162202a62be55 Reviewed-on: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/5804 Contrib: Jenkins Integration-Tests: Jenkins Tested-by: Jenkins Reviewed-by: Ian Maxon --- .../nontagged/BinaryTokenizerFactoryProvider.java | 27 -- 1 file changed, 15 insertions(+), 12 deletions(-) diff --git a/asterixdb/asterix-om/src/main/java/org/apache/asterix/formats/nontagged/BinaryTokenizerFactoryProvider.java b/asterixdb/asterix-om/src/main/java/org/apache/asterix/formats/nontagged/BinaryTokenizerFactoryProvider.java index 5ef7702..6333e92 100644 --- a/asterixdb/asterix-om/src/main/java/org/apache/asterix/formats/nontagged/BinaryTokenizerFactoryProvider.java +++ b/asterixdb/asterix-om/src/main/java/org/apache/asterix/formats/nontagged/BinaryTokenizerFactoryProvider.java @@ -30,26 +30,29 @@ import org.apache.hyracks.storage.am.lsm.invertedindex.tokenizers.NGramUTF8Strin import org.apache.hyracks.storage.am.lsm.invertedindex.tokenizers.UTF8NGramTokenFactory; import org.apache.hyracks.storage.am.lsm.invertedindex.tokenizers.UTF8WordTokenFactory; +// ToDo: maybe we can make the constructor method of the tokenizers private so that tokenizers have to be generated via this provider +// Currently, different call sites of tokenizers use **different parameters**, and this can be error-prone +// A centralized provider can avoid the bugs due to different parameters. public class BinaryTokenizerFactoryProvider implements IBinaryTokenizerFactoryProvider { public static final BinaryTokenizerFactoryProvider INSTANCE = new BinaryTokenizerFactoryProvider(); -private static final IBinaryTokenizerFactory aqlStringTokenizer = +private static final IBinaryTokenizerFactory stringTokenizerFactory = new DelimitedUTF8StringBinaryTokenizerFactory(true, true, new UTF8WordTokenFactory(ATypeTag.SERIALIZED_STRING_TYPE_TAG, ATypeTag.SERIALIZED_INT32_TYPE_TAG)); -private static final IBinaryTokenizerFactory aqlStringNoTypeTagTokenizer = +private static final IBinaryTokenizerFactory stringNoTypeTagTokenizerFactory = new DelimitedUTF8StringBinaryTokenizerFactory(true, false, new UTF8WordTokenFactory(ATypeTag.STRING.serialize(), ATypeTag.INTEGER.serialize())); -private static final IBinaryTokenizerFactory aqlHashingStringTokenizer = +private static final IBinaryTokenizerFactory stringHashingTokenizerFactory = new DelimitedUTF8StringBinaryTokenizerFactory(true, true, new HashedUTF8WordTokenFactory( ATypeTag.SERIALIZED_INT32_TYPE_TAG, ATypeTag.SERIALIZED_INT32_TYPE_TAG)); -private static final IBinaryTokenizerFactory orderedListTokenizer = +private static final IBinaryTokenizerFactory orderedListTokenizerFactory = new AOrderedListBinaryTokenizerFactory(new AListElementTokenFactory()); -private static final IBinaryTokenizerFactory unorderedListTokenizer = +private static final IBinaryTokenizerFactory unorderedListTokenizerFactory = new AUnorderedListBinaryTokenizerFactory(new AListElementTokenFactory()); @Override @@ -58,16 +61,16 @@ public class BinaryTokenizerFactoryProvider implements IBinaryTokenizerFactoryPr switch (typeTag) { case STRING: if (hashedTokens) { -return aqlHashingStringTokenizer; +return stringHashingTokenizerFactory; } else if (!typeTageAlreadyRemoved) { -return aqlStringTokenizer; +return stringTokenizerFactory; } else { -return aqlStringNoTypeTagTokenizer; +return stringNoTypeTagTokenizerFactory; } case ARRAY: -return orderedListTokenizer; +return orderedListTokenizerFactory; case MULTISET: -return unorderedListTokenizer; +return unorderedListTokenizerFactory; default: return null; } @@ -86,9 +89,9 @@ public class BinaryTokenizerFactoryProvider implements IBinaryTokenizerFactoryPr ATypeTag.SERIALIZED_INT32_TYPE_TAG)); } case ARRAY: -return orderedListTokenizer; +return orderedListTokenizerFactory; case MULTISET
[asterixdb] branch master updated (6e6b342 -> f85cee9)
This is an automated email from the ASF dual-hosted git repository. imaxon pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/asterixdb.git. from 6e6b342 [ASTERIXDB-2708] Introduce batch and stateful point cursors new 3d0897b [ASTERIXDB-2709][COMP] Fix registration of st_union aggregates new 934ce9d [NO ISSUE] Rename tokenizer factories new f85cee9 [NO ISSUE] Remove out-of-date tokenizer The 3 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: .../two-geometries/aggregate.17.query.sqlpp} | 4 + .../results/geojson/two-geometries/result.17.adm | 1 + .../asterix/fuzzyjoin/FuzzyJoinAppendLength.java | 58 - .../asterix/fuzzyjoin/FuzzyJoinTokenize.java | 133 - .../fuzzyjoin/similarity/SimilarityMetric.java | 4 - .../similarity/SimilarityMetricJaccard.java| 21 .../fuzzyjoin/tokenizer/NGramTokenizer.java| 90 -- .../apache/asterix/fuzzyjoin/tokenizer/Token.java | 118 -- .../asterix/fuzzyjoin/tokenizer/Tokenizer.java | 27 - .../fuzzyjoin/tokenizer/TokenizerBuffered.java | 30 - .../tokenizer/TokenizerBufferedFactory.java| 34 -- .../fuzzyjoin/tokenizer/TokenizerFactory.java | 31 - .../asterix/fuzzyjoin/tokenizer/WordTokenizer.java | 68 --- .../fuzzyjoin/tokenizer/WordTokenizerBuffered.java | 92 -- .../fuzzyjoin/tokenorder/IntTokenCountRank.java| 28 - .../tokenorder/IntTokenCountRankFrequency.java | 58 - .../asterix/fuzzyjoin/tokenorder/IntTokenRank.java | 28 - .../tokenorder/IntTokenRankFrequency.java | 54 - .../asterix/fuzzyjoin/tokenorder/TokenLoad.java| 61 -- .../asterix/fuzzyjoin/tokenorder/TokenRank.java| 31 - .../tokenorder/TokenRankBufferedFrequency.java | 75 .../fuzzyjoin/tokenorder/TokenRankFrequency.java | 61 -- .../nontagged/BinaryTokenizerFactoryProvider.java | 27 +++-- .../asterix/om/functions/BuiltinFunctions.java | 28 - 24 files changed, 44 insertions(+), 1118 deletions(-) copy asterixdb/asterix-app/src/test/resources/{metadata/queries/basic/dataset_with_meta-1/dataset_with_meta-1.2.update.aql => runtimets/queries_sqlpp/geojson/two-geometries/aggregate.17.query.sqlpp} (87%) create mode 100644 asterixdb/asterix-app/src/test/resources/runtimets/results/geojson/two-geometries/result.17.adm delete mode 100644 asterixdb/asterix-fuzzyjoin/src/main/java/org/apache/asterix/fuzzyjoin/FuzzyJoinAppendLength.java delete mode 100644 asterixdb/asterix-fuzzyjoin/src/main/java/org/apache/asterix/fuzzyjoin/FuzzyJoinTokenize.java delete mode 100644 asterixdb/asterix-fuzzyjoin/src/main/java/org/apache/asterix/fuzzyjoin/tokenizer/NGramTokenizer.java delete mode 100644 asterixdb/asterix-fuzzyjoin/src/main/java/org/apache/asterix/fuzzyjoin/tokenizer/Token.java delete mode 100644 asterixdb/asterix-fuzzyjoin/src/main/java/org/apache/asterix/fuzzyjoin/tokenizer/Tokenizer.java delete mode 100644 asterixdb/asterix-fuzzyjoin/src/main/java/org/apache/asterix/fuzzyjoin/tokenizer/TokenizerBuffered.java delete mode 100644 asterixdb/asterix-fuzzyjoin/src/main/java/org/apache/asterix/fuzzyjoin/tokenizer/TokenizerBufferedFactory.java delete mode 100644 asterixdb/asterix-fuzzyjoin/src/main/java/org/apache/asterix/fuzzyjoin/tokenizer/TokenizerFactory.java delete mode 100644 asterixdb/asterix-fuzzyjoin/src/main/java/org/apache/asterix/fuzzyjoin/tokenizer/WordTokenizer.java delete mode 100644 asterixdb/asterix-fuzzyjoin/src/main/java/org/apache/asterix/fuzzyjoin/tokenizer/WordTokenizerBuffered.java delete mode 100644 asterixdb/asterix-fuzzyjoin/src/main/java/org/apache/asterix/fuzzyjoin/tokenorder/IntTokenCountRank.java delete mode 100644 asterixdb/asterix-fuzzyjoin/src/main/java/org/apache/asterix/fuzzyjoin/tokenorder/IntTokenCountRankFrequency.java delete mode 100644 asterixdb/asterix-fuzzyjoin/src/main/java/org/apache/asterix/fuzzyjoin/tokenorder/IntTokenRank.java delete mode 100644 asterixdb/asterix-fuzzyjoin/src/main/java/org/apache/asterix/fuzzyjoin/tokenorder/IntTokenRankFrequency.java delete mode 100644 asterixdb/asterix-fuzzyjoin/src/main/java/org/apache/asterix/fuzzyjoin/tokenorder/TokenLoad.java delete mode 100644 asterixdb/asterix-fuzzyjoin/src/main/java/org/apache/asterix/fuzzyjoin/tokenorder/TokenRank.java delete mode 100644 asterixdb/asterix-fuzzyjoin/src/main/java/org/apache/asterix/fuzzyjoin/tokenorder/TokenRankBufferedFrequency.java delete mode 100644 asterixdb/asterix-fuzzyjoin/src/main/java/org/apache/asterix/fuzzyjoin/tokenorder/TokenRankFrequency.java
[asterixdb] branch master updated: [NO ISSUE] Fix UDF Metadata Transactions
This is an automated email from the ASF dual-hosted git repository. imaxon pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/asterixdb.git The following commit(s) were added to refs/heads/master by this push: new f2c9308 [NO ISSUE] Fix UDF Metadata Transactions f2c9308 is described below commit f2c93085c81628eb9bc7e563259eaf9e4fe7a3bc Author: Ian Maxon AuthorDate: Wed Apr 22 01:43:28 2020 -0700 [NO ISSUE] Fix UDF Metadata Transactions - Move UDF Metadata Transactions to be initiated from the CC directly - Remove old UDF initialization code - Make DROP DATAVERSE remove libraries - Make NCs properly read libraries on init with multipart dataverse names Change-Id: Ibda23a2e8308937f343d80eff04ede9a235da0d2 Reviewed-on: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/4923 Contrib: Jenkins Integration-Tests: Jenkins Tested-by: Jenkins Reviewed-by: Murtadha Hubail --- .../asterix/api/http/server/UdfApiServlet.java | 166 --- .../asterix/app/external/ExternalLibraryUtils.java | 225 +++-- .../asterix/app/message/AbstractUdfMessage.java| 8 +- .../asterix/app/message/DeleteUdfMessage.java | 6 +- .../apache/asterix/app/message/LoadUdfMessage.java | 4 +- .../app/nc/task/ExternalLibrarySetupTask.java | 2 +- .../asterix/app/translator/QueryTranslator.java| 14 ++ .../asterix/hyracks/bootstrap/CCApplication.java | 2 +- .../asterix/app/external/ExternalUDFLibrarian.java | 16 +- .../deterministic.5.ddl.sqlpp} | 0 .../getCapital.4.ddl.sqlpp}| 0 .../getCapital_open.4.ddl.sqlpp} | 0 .../getCapital_open/getCapital_open.4.lib.sqlpp| 19 -- ...ay_sum.5.ddl.sqlpp => my_array_sum.4.ddl.sqlpp} | 0 .../my_array_sum/my_array_sum.4.lib.sqlpp | 19 -- .../external-library/mysum/mysum.7.lib.sqlpp | 19 -- .../mysum_dropinuse.0.ddl.sqlpp} | 3 +- .../mysum_dropinuse.1.lib.sqlpp} | 2 +- .../mysum_dropinuse.2.ddl.sqlpp} | 3 +- .../mysum_dropinuse.3.query.sqlpp} | 3 +- .../mysum_dropinuse.4.lib.sqlpp} | 0 .../mysum_dropinuse.5.ddl.sqlpp} | 3 +- .../mysum_dropinuse.6.lib.sqlpp} | 0 .../mysum_dropinuse.7.ddl.sqlpp} | 0 ...ion.5.ddl.sqlpp => type_validation.4.ddl.sqlpp} | 0 .../udf_metadata.4.ddl.sqlpp} | 0 .../upperCase.4.ddl.sqlpp} | 2 +- ...lpp => feed-with-external-function.6.ddl.sqlpp} | 0 .../feed-with-external-function.6.lib.sqlpp| 19 -- .../mysum_dropinuse/mysum_dropinuse.1.adm | 1 + .../resources/runtimets/testsuite_it_sqlpp.xml | 6 + .../asterix/common/exceptions/ErrorCode.java | 1 + .../src/main/resources/asx_errormsg/en.properties | 1 + .../org/apache/asterix/metadata/MetadataNode.java | 5 + 34 files changed, 220 insertions(+), 329 deletions(-) diff --git a/asterixdb/asterix-app/src/main/java/org/apache/asterix/api/http/server/UdfApiServlet.java b/asterixdb/asterix-app/src/main/java/org/apache/asterix/api/http/server/UdfApiServlet.java index 54a62df..0f5d542 100644 --- a/asterixdb/asterix-app/src/main/java/org/apache/asterix/api/http/server/UdfApiServlet.java +++ b/asterixdb/asterix-app/src/main/java/org/apache/asterix/api/http/server/UdfApiServlet.java @@ -19,21 +19,34 @@ package org.apache.asterix.api.http.server; import java.io.File; +import java.io.PrintWriter; import java.io.RandomAccessFile; import java.nio.ByteBuffer; import java.nio.channels.FileChannel; +import java.rmi.RemoteException; import java.util.ArrayList; import java.util.Arrays; import java.util.List; import java.util.concurrent.ConcurrentMap; import org.apache.asterix.app.external.ExternalLibraryUtils; -import org.apache.asterix.app.message.DeleteUdfMessage; import org.apache.asterix.app.message.LoadUdfMessage; import org.apache.asterix.common.dataflow.ICcApplicationContext; +import org.apache.asterix.common.exceptions.AsterixException; +import org.apache.asterix.common.exceptions.ErrorCode; import org.apache.asterix.common.messaging.api.ICCMessageBroker; import org.apache.asterix.common.messaging.api.INcAddressedMessage; import org.apache.asterix.common.metadata.DataverseName; +import org.apache.asterix.common.metadata.IMetadataLockUtil; +import org.apache.asterix.common.metadata.LockList; +import org.apache.asterix.metadata.MetadataManager; +import org.apache.asterix.metadata.MetadataTransactionContext; +import org.apache.asterix.metadata.declared.MetadataProvider; +import org.apache.asterix.metadata.entities.DatasourceAdapter; +import org.apache.asterix.metadata.entities.Dataverse; +import org.apache.asterix.metadata.entities.Function; +import org.apache.asterix.metadata.entities.Library
[asterixdb] branch master updated: [NO ISSUE] Update README.md
This is an automated email from the ASF dual-hosted git repository. imaxon pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/asterixdb.git The following commit(s) were added to refs/heads/master by this push: new d7072db [NO ISSUE] Update README.md d7072db is described below commit d7072db8fab4b11d367e62953504fd4be69d0229 Author: Ian Maxon AuthorDate: Thu May 7 19:23:46 2020 -0700 [NO ISSUE] Update README.md Change-Id: Ie273594326b296c7a8257816dfb1f91c9b951fb8 Reviewed-on: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/6226 Contrib: Jenkins Tested-by: Jenkins Integration-Tests: Jenkins Reviewed-by: Wael Alkowaileet --- README.md | 7 +-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/README.md b/README.md index a555946..483b108 100644 --- a/README.md +++ b/README.md @@ -59,7 +59,8 @@ To build AsterixDB from source, you should have a platform with the following: * A Unix-ish environment (Linux, OS X, will all do). * git * Maven 3.3.9 or newer. -* Oracle JDK 8 or newer. +* JDK 11 or newer. +* Python 3.6+ with pip and venv Instructions for building the master: @@ -83,13 +84,15 @@ Here are steps to get AsterixDB running on your local machine: * Good to go and run queries in your browser at: -http://localhost:19001 +http://localhost:19006 * Read more [documentation](https://ci.apache.org/projects/asterixdb/index.html) to learn the data model, query language, and how to create a cluster instance. ## Documentation * [master](https://ci.apache.org/projects/asterixdb/index.html) | + [0.9.4.1](http://asterixdb.apache.org/docs/0.9.4.1/index.html) | + [0.9.4](http://asterixdb.apache.org/docs/0.9.4/index.html) | [0.9.3](http://asterixdb.apache.org/docs/0.9.3/index.html) | [0.9.2](http://asterixdb.apache.org/docs/0.9.2/index.html) | [0.9.1](http://asterixdb.apache.org/docs/0.9.1/index.html) |
svn commit: r39800 - /dev/asterixdb/
Author: imaxon Date: Thu May 28 09:29:41 2020 New Revision: 39800 Log: upload new RC Added: dev/asterixdb/apache-asterixdb-0.9.5-source-release.zip (with props) dev/asterixdb/apache-asterixdb-0.9.5-source-release.zip.asc dev/asterixdb/apache-asterixdb-0.9.5-source-release.zip.sha256 dev/asterixdb/apache-hyracks-0.3.5-source-release.zip (with props) dev/asterixdb/apache-hyracks-0.3.5-source-release.zip.asc dev/asterixdb/apache-hyracks-0.3.5-source-release.zip.sha256 dev/asterixdb/asterix-server-0.9.5-binary-assembly.zip (with props) dev/asterixdb/asterix-server-0.9.5-binary-assembly.zip.asc dev/asterixdb/asterix-server-0.9.5-binary-assembly.zip.sha256 Removed: dev/asterixdb/apache-asterixdb-0.9.5.zip dev/asterixdb/apache-asterixdb-0.9.5.zip.asc dev/asterixdb/apache-asterixdb-0.9.5.zip.sha256 Added: dev/asterixdb/apache-asterixdb-0.9.5-source-release.zip == Binary file - no diff available. Propchange: dev/asterixdb/apache-asterixdb-0.9.5-source-release.zip -- svn:mime-type = application/octet-stream Added: dev/asterixdb/apache-asterixdb-0.9.5-source-release.zip.asc == --- dev/asterixdb/apache-asterixdb-0.9.5-source-release.zip.asc (added) +++ dev/asterixdb/apache-asterixdb-0.9.5-source-release.zip.asc Thu May 28 09:29:41 2020 @@ -0,0 +1,16 @@ +-BEGIN PGP SIGNATURE- + +iQIzBAABCgAdFiEEIf3gbxaFXS93IFrkwLkA3UcwFf0FAl7PfrYACgkQwLkA3Ucw +Ff34iw/+OdeyVfMC1ZG37PDUUHdCXjPe5maTUtoDriT1N9IqopaF3TY816KPMH3l +4G/Qgp8pFDRpPmtADn3h5ssizvBVHZzJQJcHL/jVQ3x8Ju+2WabgdXhudDeJeqPZ +5TnTxPWLZnqf3sHlEuqWFm26csXhWSP/UAhQzM8fy0VJE8VpFFvhU5KYvucLTDED +G8K7XEZwEbECXhrts9XctOkEgoXgZT55uyI5BvGAI8SkYDiAkecWOFB1+SBqJHJv +e1C5sElAuDnbn7dY0flZZ+kldb5PDOdO57dI4kj23uW5fGcnZO4fTyQFRf1XgU/9 +vQnc40QMihAShcxKw+ogyxXWddgTiCX0BX3lAEcAJQss6DMp/L1C9knU4HO0NYxD +FQbRwD9dFkBn8AZW45/3yLd/Q9XhSIVM29mc15WEmtdCLU2QvCwSwYwXmKbtNxgE +kF/R5BBklYKvGgO7jXa3YSgLkmvSJOhzyZgV/tUkR/qvyHO7Be/jYm4z1Up01K7i +dqn5JiGCuKCmN/kewmaGIhxWnokRdoYU8RFwiuguJBwlqsPQTA62TDy/a8ONwiZh +Ln5NoyCUhUovUQqfrGTEY4O2j7YX+d9RH0g5KU0sRtKOfayDRZbE8AFP1zTWpab4 +6XZj2/gH22irgeuTDC6pn8hmNj+utHSBKhdUesSEnRFJ46Gocjg= +=JGJS +-END PGP SIGNATURE- Added: dev/asterixdb/apache-asterixdb-0.9.5-source-release.zip.sha256 == --- dev/asterixdb/apache-asterixdb-0.9.5-source-release.zip.sha256 (added) +++ dev/asterixdb/apache-asterixdb-0.9.5-source-release.zip.sha256 Thu May 28 09:29:41 2020 @@ -0,0 +1 @@ +0e2f7e20fbebc1420edce62027b9de7a25aa0718a81bfaf68c15ebf11544d03d apache-asterixdb-0.9.5-source-release.zip Added: dev/asterixdb/apache-hyracks-0.3.5-source-release.zip == Binary file - no diff available. Propchange: dev/asterixdb/apache-hyracks-0.3.5-source-release.zip -- svn:mime-type = application/octet-stream Added: dev/asterixdb/apache-hyracks-0.3.5-source-release.zip.asc == --- dev/asterixdb/apache-hyracks-0.3.5-source-release.zip.asc (added) +++ dev/asterixdb/apache-hyracks-0.3.5-source-release.zip.asc Thu May 28 09:29:41 2020 @@ -0,0 +1,16 @@ +-BEGIN PGP SIGNATURE- + +iQIzBAABCgAdFiEEIf3gbxaFXS93IFrkwLkA3UcwFf0FAl7PfMQACgkQwLkA3Ucw +Ff2sLg//VkU+j4LFpBRQbeYzBw6OToES8her62iRtpV/EM4GXjicUq4gPd0pg3tH +FFZ/6ZL4LI4CpaaK+YCuVBTy/dm6ughQcYvVnhTiCcxJKurGVLBv1pXWV+QZOnjg +io97L/fzrylJ4xhEeiK0Hg4HlaWqFv9q4m0kOyqUkDTO5Jiz+x4qDkLauJqQ/pJS +XZ7DlbDRwbKs98D0x4YbbxfdPJhvKvbxZbc1oFMLQWbuYjNySW1kpZ71zg6l0iUZ +FQOIPwp/Q8po2P3Ia1SEjSJUXuP0AUqmNuDNxNHyB/eRfhfX+bNxxUNtKiMblFvg +Iq2mt3xOgtka3gRgb68ryxejPemkyu8GlrZgapPxHK6C/EadcllXbpoRdwryHLNe +Tw1HXYRgD2HOFL0D/SfMTZ92FNB2o/OCFVgv+43GxV59AIeXPmsbWn+wcWOr9EhU +k6CkOF6qe00JewlhEMTmicb0GhVA1Bzc+xYYIagZGcyh3SSGYwFy9JBziJEC8FdB +srxJwI4X0FxoOuV/fqCEuywWkFhMduuSygvCvJQdsxwLcQE78X30d+FZGbFxjuX9 +2Q+5x1dPu+MNCVKVuPNyJiyCCKQEYUyMe9yXRKgYAFbtVgpoi8KwmJHzeXZtW1Uk +lS36bZsP2lVUH7tl7M47ChemKveakWhJTrpmh8//5gRYpApzaoU= +=oOkb +-END PGP SIGNATURE- Added: dev/asterixdb/apache-hyracks-0.3.5-source-release.zip.sha256 == --- dev/asterixdb/apache-hyracks-0.3.5-source-release.zip.sha256 (added) +++ dev/asterixdb/apache-hyracks-0.3.5-source-release.zip.sha256 Thu May 28 09:29:41 2020 @@ -0,0 +1 @@ +37a68609c556cb9b09ab193f3b67370e1634ca3378dc4477a08ff4f670073206 apache-hyracks-0.3.5-source-release.zip Added: dev/asterixdb/asterix-server-0.9.5-binary-assembly.zip == Binary file - no diff available. Propchange
[asterixdb] branch master updated: [NO ISSUE] Update UDF documentation
This is an automated email from the ASF dual-hosted git repository. imaxon pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/asterixdb.git The following commit(s) were added to refs/heads/master by this push: new d17b091 [NO ISSUE] Update UDF documentation d17b091 is described below commit d17b0911a1fe735051358ff5ef44914adca20c75 Author: Ian Maxon AuthorDate: Wed May 27 13:28:50 2020 -0700 [NO ISSUE] Update UDF documentation Change-Id: Ibdc65eaecef122b24b4795c8949931a37ad90f47 Reviewed-on: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/6444 Integration-Tests: Jenkins Reviewed-by: Dmitry Lychagin Tested-by: Jenkins --- .../src/main/user-defined_function/udf.md | 214 ++--- 1 file changed, 146 insertions(+), 68 deletions(-) diff --git a/asterixdb/asterix-doc/src/main/user-defined_function/udf.md b/asterixdb/asterix-doc/src/main/user-defined_function/udf.md index 2431448..dc21c30 100644 --- a/asterixdb/asterix-doc/src/main/user-defined_function/udf.md +++ b/asterixdb/asterix-doc/src/main/user-defined_function/udf.md @@ -17,48 +17,139 @@ ! under the License. !--> -## Introduction## +## Introduction -Apache AsterixDB supports two languages for writing user-defined functions (UDFs): SQL++ and Java. +Apache AsterixDB supports three languages for writing user-defined functions (UDFs): SQL++, Java and Python A user can encapsulate data processing logic into a UDF and invoke it later repeatedly. For SQL++ functions, a user can refer to [SQL++ Functions](sqlpp/manual.html#Functions) -for their usages. In this document, we -focus on how to install/invoke/uninstall a Java function library using the Ansible script that we provide. +for their usages. This document will focus on UDFs in languages other than SQL++ -## Installing an UDF Library## +## Endpoints and Authentication -UDFs have to be installed offline. -This section describes the process assuming that you have followed the preceding [ansible installation instructions](ansible.html) -to deploy an AsterixDB instance on your local machine or cluster. Here are the -instructions to install an UDF library: +The UDF endpoint is not enabled by default until authentication has been configured properly. To enable it, we +will need to set the path to the credential file and populate it with our username and password. -- Step 1: Stop the AsterixDB instance if it is ACTIVE. +The credential file is a simple `/etc/passwd` style text file with usernames and corresponding `bcrypt` hashed and salted +passwords. You can populate this on your own if you would like, but the `asterixhelper` utility can write the entries as +well. We can invoke `asterixhelper` like so: -$ bin/stop.sh +$ bin/asterixhelper -u admin -p admin -cp opt/local/conf add_credential -- Step 2: Deploy the UDF package. +Then, in your `cc.conf`, in the `[cc]` section, add the correct `credential.file` path -$ bin/udf.sh -m i -d DATAVERSE_NAME -l LIBRARY_NAME -p UDF_PACKAGE_PATH +[cc] +address = 127.0.0.1 +... +... +credential.file = conf/passwd -- Step 3: Start AsterixDB +Now,restart the cluster if it was already started to allow the Cluster Controller to find the new credentials. -$ bin/start.sh -After AsterixDB starts, you can use the following query to check whether your UDFs have been sucessfully registered with the system. +## Installing a Java UDF Library -SELECT * FROM Metadata.`Function`; +To install a UDF package to the cluster, we need to send a Multipart Form-data HTTP request to the `/admin/udf` endpoint +of the CC at the normal API port (`19002` by default). The request should use HTTP Basic authentication. This means your +credentials will *not* be obfuscated or encrypted *in any way*, so submit to this endpoint over localhost or a network +where you know your traffic is safe from eavesdropping. Any suitable tool will do, but for the example here I will use +`curl` which is widely available. + +For example, to install a library with the following criteria: + +* `udfs` dataverse name +* with a new Library name of `testlib` +* from `lib.zip` in the present working directory +* to the cluster at `localhost` with API port `19002` +* with credentials being a username and password of `admin:admin` + +we would execute + +curl -v -u admin:admin -X POST -F 'data=@./lib.zip' localhost:19002/admin/udf/udfs/testlib + +Any response other than `200` indicates an error in deployment. In the AsterixDB source release, we provide several sample UDFs that you can try out. You need to build the AsterixDB source to get the compiled UDF package. It can be found under the `asterixdb-external` sub-project. Assuming that these UDFs have been installed into the `udfs` dataverse and `testlib` library, here is an example that uses the sample UDF `mysum` to compute the sum of two i
[asterixdb] branch master updated: [NO ISSUE] Validate paths within library archives
This is an automated email from the ASF dual-hosted git repository. imaxon pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/asterixdb.git The following commit(s) were added to refs/heads/master by this push: new 28c0ee8 [NO ISSUE] Validate paths within library archives 28c0ee8 is described below commit 28c0ee84f1387ab5d0659e9e822f4e3923ddc22d Author: Ian Maxon AuthorDate: Mon Apr 6 18:32:53 2020 -0700 [NO ISSUE] Validate paths within library archives Change-Id: I8f4a82c43b950fc3573cae5aa7c0782b475f962c Reviewed-on: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/5624 Integration-Tests: Jenkins Tested-by: Jenkins Reviewed-by: Ian Maxon Contrib: Ian Maxon --- .../control/common/deployment/DeploymentUtils.java| 19 ++- 1 file changed, 18 insertions(+), 1 deletion(-) diff --git a/hyracks-fullstack/hyracks/hyracks-control/hyracks-control-common/src/main/java/org/apache/hyracks/control/common/deployment/DeploymentUtils.java b/hyracks-fullstack/hyracks/hyracks-control/hyracks-control-common/src/main/java/org/apache/hyracks/control/common/deployment/DeploymentUtils.java index d07b648..e0150f7 100644 --- a/hyracks-fullstack/hyracks/hyracks-control/hyracks-control-common/src/main/java/org/apache/hyracks/control/common/deployment/DeploymentUtils.java +++ b/hyracks-fullstack/hyracks/hyracks-control/hyracks-control-common/src/main/java/org/apache/hyracks/control/common/deployment/DeploymentUtils.java @@ -25,6 +25,8 @@ import java.io.IOException; import java.io.InputStream; import java.io.OutputStream; import java.net.URL; +import java.nio.file.Path; +import java.nio.file.Paths; import java.util.ArrayList; import java.util.Enumeration; import java.util.List; @@ -32,6 +34,7 @@ import java.util.zip.ZipEntry; import java.util.zip.ZipFile; import org.apache.commons.io.FileUtils; +import org.apache.commons.io.FilenameUtils; import org.apache.commons.io.IOUtils; import org.apache.http.HttpEntity; import org.apache.http.HttpResponse; @@ -45,6 +48,7 @@ import org.apache.hyracks.api.job.IJobSerializerDeserializer; import org.apache.hyracks.api.job.IJobSerializerDeserializerContainer; import org.apache.hyracks.api.util.JavaSerializationUtils; import org.apache.hyracks.control.common.context.ServerContext; +import org.apache.hyracks.util.file.FileUtil; import org.apache.logging.log4j.LogManager; import org.apache.logging.log4j.Logger; @@ -241,12 +245,25 @@ public class DeploymentUtils { throw HyracksException.create(trace); } -public static void unzip(String sourceFile, String outputDir) throws IOException { +public static void unzip(String sourceFile, String outputDirName) throws IOException { try (ZipFile zipFile = new ZipFile(sourceFile)) { +Path outputPath = Paths.get(FilenameUtils.normalize(outputDirName)); +File outputDir = outputPath.toFile(); +if (!outputDir.exists()) { +throw new IOException("Output path doesn't exist"); +} +if (!outputDir.isDirectory()) { +throw new IOException("Output path is not a directory"); +} Enumeration entries = zipFile.entries(); List createdFiles = new ArrayList<>(); while (entries.hasMoreElements()) { ZipEntry entry = entries.nextElement(); +String normalizedPath = FilenameUtils.normalize(FileUtil.joinPath(outputDirName, entry.getName())); +Path candidatePath = Paths.get(normalizedPath); +if (!candidatePath.startsWith(outputPath)) { +throw new IOException("Malformed ZIP archive"); +} File entryDestination = new File(outputDir, entry.getName()); if (!entry.isDirectory()) { entryDestination.getParentFile().mkdirs();
svn commit: r40326 - /dev/asterixdb/
Author: imaxon Date: Mon Jul 6 22:35:28 2020 New Revision: 40326 Log: upload new RC Added: dev/asterixdb/apache-asterixdb-0.9.5-source-release.zip (with props) dev/asterixdb/apache-asterixdb-0.9.5-source-release.zip.asc dev/asterixdb/apache-asterixdb-0.9.5-source-release.zip.sha256 dev/asterixdb/apache-hyracks-0.3.5-source-release.zip (with props) dev/asterixdb/apache-hyracks-0.3.5-source-release.zip.asc dev/asterixdb/apache-hyracks-0.3.5-source-release.zip.sha256 dev/asterixdb/asterix-server-0.9.5-binary-assembly.zip (with props) dev/asterixdb/asterix-server-0.9.5-binary-assembly.zip.asc dev/asterixdb/asterix-server-0.9.5-binary-assembly.zip.sha256 Added: dev/asterixdb/apache-asterixdb-0.9.5-source-release.zip == Binary file - no diff available. Propchange: dev/asterixdb/apache-asterixdb-0.9.5-source-release.zip -- svn:mime-type = application/octet-stream Added: dev/asterixdb/apache-asterixdb-0.9.5-source-release.zip.asc == --- dev/asterixdb/apache-asterixdb-0.9.5-source-release.zip.asc (added) +++ dev/asterixdb/apache-asterixdb-0.9.5-source-release.zip.asc Mon Jul 6 22:35:28 2020 @@ -0,0 +1,16 @@ +-BEGIN PGP SIGNATURE- + +iQIzBAABCgAdFiEEIf3gbxaFXS93IFrkwLkA3UcwFf0FAl8DodwACgkQwLkA3Ucw +Ff3BwA/9HfvQZCNAtST95WtwE8k5AW9QCTnxS+hWAImIPcPCWDtGua91NKaEPBiz +JVhnaecPOTy5AFENTDx2yNJSQQhcOGT2hXfg6aCIEDGkMD5PBbFJp9TyYUUD2Xex +fHZ6ns9qeJjPuhV4gogebiTEccWrFK55vgAeCVZ0JI3j0TuKkO6WDq68Rmd7LXA6 +71KhJwphsu9qBhMcncR31TxHc9YM68HIMLW5TMSRO0yZvV17hxkTOqB8asbGM9dW +LnWHiCSnRGgepGF6BkLo8O0RINLT9jiL5Sr5QUEu5MCCATBe+kyoAH/bBqDFFP8N +vui82fLDsaqm2/XmtTWBD1cTbH5G86ft3XjleNRDY6OQlJAa1x6Q8brqAyjjTpBl +r+g0J7+oPkVOYPXfAs4PEuAYsWiSa67Ee3bbnOJJ3z05G5MlmzmKGliKOS2zOCEH +Kc6BozqsumI2S1aqQWsqy3DO120F1U+itbESjBGQ9/s9/ckxLy8gqDN1qpxr4ard +bIhfKOBuSytE1dKePEjIWSDb7pe3bR0Z4+RNCn4QgfIWgy3U4uVZl0P8c93KSV0p +8i9qt/XxujsuEHvwH3k3xvtDo2naMcfmlAd4i5Wuh66/5vKqLZTE0peRbJ77n9MC +tfAca7Wy5JKl23L0qJAKKn69OKvlwEJlCI54AmJa/4HueGItxG0= +=r8Lf +-END PGP SIGNATURE- Added: dev/asterixdb/apache-asterixdb-0.9.5-source-release.zip.sha256 == --- dev/asterixdb/apache-asterixdb-0.9.5-source-release.zip.sha256 (added) +++ dev/asterixdb/apache-asterixdb-0.9.5-source-release.zip.sha256 Mon Jul 6 22:35:28 2020 @@ -0,0 +1 @@ +09affe9ce5aa75add6c5a75c51505e619f85cb7a87eb3b9d977ac472d5387bd1 apache-asterixdb-0.9.5-source-release.zip Added: dev/asterixdb/apache-hyracks-0.3.5-source-release.zip == Binary file - no diff available. Propchange: dev/asterixdb/apache-hyracks-0.3.5-source-release.zip -- svn:mime-type = application/octet-stream Added: dev/asterixdb/apache-hyracks-0.3.5-source-release.zip.asc == --- dev/asterixdb/apache-hyracks-0.3.5-source-release.zip.asc (added) +++ dev/asterixdb/apache-hyracks-0.3.5-source-release.zip.asc Mon Jul 6 22:35:28 2020 @@ -0,0 +1,16 @@ +-BEGIN PGP SIGNATURE- + +iQIzBAABCgAdFiEEIf3gbxaFXS93IFrkwLkA3UcwFf0FAl8DoAsACgkQwLkA3Ucw +Ff3+LQ/8DKFp4oQx5HyIrABvC9GtSwLIRbi7J0hPdbVDpOCoNxTA8cWVssn/U5ft +VyaY7Z36hK+xfHIA59WUO2ZiFMmZEoN83FIuZYQn1/vXqM24usuzC8u5SaGTJHJ1 +UlLQhSjXL1bbut2N6kJipq9LtZlrLNt/0oWrejBSqS786PNnyuuWJHSJoASuvzrm +k+e3sXf2X9lQ3FMSJBIGSSsj8gabNn5zmCXNiR9/uW8lAU4YirjQsNZdQ7JeuVS3 +CQ7uITa8Yjtg8Zpo+8eHHw1EQZoLgfhei55xAA3ymuGOY1iMb/ua7BebG5vuzafO +n8lXRbM96Nos2l8KI/bsX/ki3dneDDyLbIAVthl3OhIHoGaDafK2OwfpzHFdsORp +S8MEKoMY1gsp6kmGV8tt0kjqItr0Y+8779cNJac4QFQCCBlCJkSQLkzYHpjni+Eb +cmY0oN4FXaEhhiZ3h6NB/QUlm6luqJhd8XB2UqqCslEnvIRYq3C8wuHDgMqMl0IP +LkopjrYrzFJM1ujSzXWeTTsXHtQOMpg3pTo0B2UOd+jLa4+NGYcH7kpnZGo8UHdn +VlWlCUX/g6ONWBGGo43LsD+XJhKzHJSsCDcgDkM49BZhWyaTFevqwQflLEb6pzCb +IRywTQwdkFxCkFtu/cw2/d5NUGrAXtmaNwStfGDHAkmo5ZcNyv8= +=Hguy +-END PGP SIGNATURE- Added: dev/asterixdb/apache-hyracks-0.3.5-source-release.zip.sha256 == --- dev/asterixdb/apache-hyracks-0.3.5-source-release.zip.sha256 (added) +++ dev/asterixdb/apache-hyracks-0.3.5-source-release.zip.sha256 Mon Jul 6 22:35:28 2020 @@ -0,0 +1 @@ +577d2b3da91ebfa37c113bae18561dcbfae0bdd526edee604b747f6044f4a03b apache-hyracks-0.3.5-source-release.zip Added: dev/asterixdb/asterix-server-0.9.5-binary-assembly.zip == Binary file - no diff available. Propchange: dev/asterixdb/asterix-server-0.9.5-binary-assembly.zip -- svn:mime-type
svn commit: r40906 - /release/asterixdb/asterixdb-0.9.5/
Author: imaxon Date: Fri Aug 7 18:37:58 2020 New Revision: 40906 Log: Release Apache AsterixDB 0.9.5 and hyracks 0.9.5 Added: release/asterixdb/asterixdb-0.9.5/ release/asterixdb/asterixdb-0.9.5/apache-asterixdb-0.9.5-source-release.zip (with props) release/asterixdb/asterixdb-0.9.5/apache-asterixdb-0.9.5-source-release.zip.asc release/asterixdb/asterixdb-0.9.5/apache-asterixdb-0.9.5-source-release.zip.sha256 release/asterixdb/asterixdb-0.9.5/apache-hyracks-0.3.5-source-release.zip (with props) release/asterixdb/asterixdb-0.9.5/apache-hyracks-0.3.5-source-release.zip.asc release/asterixdb/asterixdb-0.9.5/apache-hyracks-0.3.5-source-release.zip.sha256 release/asterixdb/asterixdb-0.9.5/asterix-server-0.9.5-binary-assembly.zip (with props) release/asterixdb/asterixdb-0.9.5/asterix-server-0.9.5-binary-assembly.zip.asc release/asterixdb/asterixdb-0.9.5/asterix-server-0.9.5-binary-assembly.zip.sha256 Added: release/asterixdb/asterixdb-0.9.5/apache-asterixdb-0.9.5-source-release.zip == Binary file - no diff available. Propchange: release/asterixdb/asterixdb-0.9.5/apache-asterixdb-0.9.5-source-release.zip -- svn:mime-type = application/octet-stream Added: release/asterixdb/asterixdb-0.9.5/apache-asterixdb-0.9.5-source-release.zip.asc == --- release/asterixdb/asterixdb-0.9.5/apache-asterixdb-0.9.5-source-release.zip.asc (added) +++ release/asterixdb/asterixdb-0.9.5/apache-asterixdb-0.9.5-source-release.zip.asc Fri Aug 7 18:37:58 2020 @@ -0,0 +1,16 @@ +-BEGIN PGP SIGNATURE- + +iQIzBAABCgAdFiEEIf3gbxaFXS93IFrkwLkA3UcwFf0FAl8DodwACgkQwLkA3Ucw +Ff3BwA/9HfvQZCNAtST95WtwE8k5AW9QCTnxS+hWAImIPcPCWDtGua91NKaEPBiz +JVhnaecPOTy5AFENTDx2yNJSQQhcOGT2hXfg6aCIEDGkMD5PBbFJp9TyYUUD2Xex +fHZ6ns9qeJjPuhV4gogebiTEccWrFK55vgAeCVZ0JI3j0TuKkO6WDq68Rmd7LXA6 +71KhJwphsu9qBhMcncR31TxHc9YM68HIMLW5TMSRO0yZvV17hxkTOqB8asbGM9dW +LnWHiCSnRGgepGF6BkLo8O0RINLT9jiL5Sr5QUEu5MCCATBe+kyoAH/bBqDFFP8N +vui82fLDsaqm2/XmtTWBD1cTbH5G86ft3XjleNRDY6OQlJAa1x6Q8brqAyjjTpBl +r+g0J7+oPkVOYPXfAs4PEuAYsWiSa67Ee3bbnOJJ3z05G5MlmzmKGliKOS2zOCEH +Kc6BozqsumI2S1aqQWsqy3DO120F1U+itbESjBGQ9/s9/ckxLy8gqDN1qpxr4ard +bIhfKOBuSytE1dKePEjIWSDb7pe3bR0Z4+RNCn4QgfIWgy3U4uVZl0P8c93KSV0p +8i9qt/XxujsuEHvwH3k3xvtDo2naMcfmlAd4i5Wuh66/5vKqLZTE0peRbJ77n9MC +tfAca7Wy5JKl23L0qJAKKn69OKvlwEJlCI54AmJa/4HueGItxG0= +=r8Lf +-END PGP SIGNATURE- Added: release/asterixdb/asterixdb-0.9.5/apache-asterixdb-0.9.5-source-release.zip.sha256 == --- release/asterixdb/asterixdb-0.9.5/apache-asterixdb-0.9.5-source-release.zip.sha256 (added) +++ release/asterixdb/asterixdb-0.9.5/apache-asterixdb-0.9.5-source-release.zip.sha256 Fri Aug 7 18:37:58 2020 @@ -0,0 +1 @@ +09affe9ce5aa75add6c5a75c51505e619f85cb7a87eb3b9d977ac472d5387bd1 apache-asterixdb-0.9.5-source-release.zip Added: release/asterixdb/asterixdb-0.9.5/apache-hyracks-0.3.5-source-release.zip == Binary file - no diff available. Propchange: release/asterixdb/asterixdb-0.9.5/apache-hyracks-0.3.5-source-release.zip -- svn:mime-type = application/octet-stream Added: release/asterixdb/asterixdb-0.9.5/apache-hyracks-0.3.5-source-release.zip.asc == --- release/asterixdb/asterixdb-0.9.5/apache-hyracks-0.3.5-source-release.zip.asc (added) +++ release/asterixdb/asterixdb-0.9.5/apache-hyracks-0.3.5-source-release.zip.asc Fri Aug 7 18:37:58 2020 @@ -0,0 +1,16 @@ +-BEGIN PGP SIGNATURE- + +iQIzBAABCgAdFiEEIf3gbxaFXS93IFrkwLkA3UcwFf0FAl8DoAsACgkQwLkA3Ucw +Ff3+LQ/8DKFp4oQx5HyIrABvC9GtSwLIRbi7J0hPdbVDpOCoNxTA8cWVssn/U5ft +VyaY7Z36hK+xfHIA59WUO2ZiFMmZEoN83FIuZYQn1/vXqM24usuzC8u5SaGTJHJ1 +UlLQhSjXL1bbut2N6kJipq9LtZlrLNt/0oWrejBSqS786PNnyuuWJHSJoASuvzrm +k+e3sXf2X9lQ3FMSJBIGSSsj8gabNn5zmCXNiR9/uW8lAU4YirjQsNZdQ7JeuVS3 +CQ7uITa8Yjtg8Zpo+8eHHw1EQZoLgfhei55xAA3ymuGOY1iMb/ua7BebG5vuzafO +n8lXRbM96Nos2l8KI/bsX/ki3dneDDyLbIAVthl3OhIHoGaDafK2OwfpzHFdsORp +S8MEKoMY1gsp6kmGV8tt0kjqItr0Y+8779cNJac4QFQCCBlCJkSQLkzYHpjni+Eb +cmY0oN4FXaEhhiZ3h6NB/QUlm6luqJhd8XB2UqqCslEnvIRYq3C8wuHDgMqMl0IP +LkopjrYrzFJM1ujSzXWeTTsXHtQOMpg3pTo0B2UOd+jLa4+NGYcH7kpnZGo8UHdn +VlWlCUX/g6ONWBGGo43LsD+XJhKzHJSsCDcgDkM49BZhWyaTFevqwQflLEb6pzCb +IRywTQwdkFxCkFtu/cw2/d5NUGrAXtmaNwStfGDHAkmo5ZcNyv8= +=Hguy +-END PGP SIGNATURE- Added: release/asterixdb/asterixdb-0.9.5/apache-hyracks-0.3.5-source-release.zip.sha256 == --- release/asterixdb/asterixdb-0.9.5/apache-hyracks-0.3.5-source
[asterixdb] branch release-0.9.5 created (now c95de7a)
This is an automated email from the ASF dual-hosted git repository. imaxon pushed a change to branch release-0.9.5 in repository https://gitbox.apache.org/repos/asf/asterixdb.git. at c95de7a Advance version to AsterixDB 0.9.6-SNAPSHOT and Hyracks 0.3.6-SNAPSHOT This branch includes the following new commits: new 4a20fb2 [NO ISSUE] excise log4j 1.x new 30b08c5 [ASTERIXDB-2742] Relax Java version check new ed3f425 Merge branch 'master' into release-0.9.5-rc0-manual new fadb35b Release Apache AsterixDB 0.9.5 and Hyracks 0.3.5 new c95de7a Advance version to AsterixDB 0.9.6-SNAPSHOT and Hyracks 0.3.6-SNAPSHOT The 5 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference.
[asterixdb] 05/05: Advance version to AsterixDB 0.9.6-SNAPSHOT and Hyracks 0.3.6-SNAPSHOT
This is an automated email from the ASF dual-hosted git repository. imaxon pushed a commit to branch release-0.9.5 in repository https://gitbox.apache.org/repos/asf/asterixdb.git commit c95de7a9548e6868e97cc664cdb12e6721b508c3 Author: Ian Maxon AuthorDate: Fri Aug 7 11:58:39 2020 -0700 Advance version to AsterixDB 0.9.6-SNAPSHOT and Hyracks 0.3.6-SNAPSHOT Change-Id: I31cf6146b35a461d522c87f67e4d8304f6f9eb12 Reviewed-on: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/7503 Reviewed-by: Ian Maxon Tested-by: Ian Maxon --- asterixdb/asterix-active/pom.xml| 2 +- asterixdb/asterix-algebra/pom.xml | 2 +- asterixdb/asterix-app/pom.xml | 2 +- asterixdb/asterix-benchmark/pom.xml | 2 +- asterixdb/asterix-client-helper/pom.xml | 2 +- asterixdb/asterix-common/pom.xml| 2 +- asterixdb/asterix-coverage/pom.xml | 2 +- asterixdb/asterix-dashboard/pom.xml | 2 +- asterixdb/asterix-doc/pom.xml | 2 +- asterixdb/asterix-docker/pom.xml| 2 +- asterixdb/asterix-examples/pom.xml | 2 +- asterixdb/asterix-external-data/pom.xml | 2 +- asterixdb/asterix-fuzzyjoin/pom.xml | 2 +- asterixdb/asterix-geo/pom.xml | 2 +- asterixdb/asterix-hivecompat/pom.xml| 2 +- asterixdb/asterix-lang-aql/pom.xml | 2 +- asterixdb/asterix-lang-common/pom.xml | 2 +- asterixdb/asterix-lang-sqlpp/pom.xml| 2 +- asterixdb/asterix-license/pom.xml | 2 +- .../asterix-grammar-extension-maven-plugin/pom.xml | 2 +- .../asterix-test-datagenerator-maven-plugin/pom.xml | 2 +- .../asterix-maven-plugins/lexer-generator-maven-plugin/pom.xml | 2 +- asterixdb/asterix-maven-plugins/pom.xml | 2 +- .../record-manager-generator-maven-plugin/pom.xml | 2 +- asterixdb/asterix-metadata/pom.xml | 2 +- asterixdb/asterix-om/pom.xml| 2 +- asterixdb/asterix-replication/pom.xml | 2 +- asterixdb/asterix-runtime/pom.xml | 2 +- asterixdb/asterix-server/pom.xml| 2 +- asterixdb/asterix-spidersilk/pom.xml| 2 +- asterixdb/asterix-test-framework/pom.xml| 2 +- asterixdb/asterix-tools/pom.xml | 2 +- asterixdb/asterix-transactions/pom.xml | 2 +- asterixdb/pom.xml | 6 +++--- hyracks-fullstack/algebricks/algebricks-common/pom.xml | 2 +- hyracks-fullstack/algebricks/algebricks-compiler/pom.xml| 2 +- hyracks-fullstack/algebricks/algebricks-core/pom.xml| 2 +- hyracks-fullstack/algebricks/algebricks-data/pom.xml| 2 +- hyracks-fullstack/algebricks/algebricks-rewriter/pom.xml| 2 +- hyracks-fullstack/algebricks/algebricks-runtime/pom.xml | 2 +- hyracks-fullstack/algebricks/algebricks-tests/pom.xml | 2 +- hyracks-fullstack/algebricks/pom.xml| 2 +- hyracks-fullstack/hyracks-fullstack-license/pom.xml | 2 +- hyracks-fullstack/hyracks/hyracks-api/pom.xml | 2 +- hyracks-fullstack/hyracks/hyracks-client/pom.xml| 2 +- hyracks-fullstack/hyracks/hyracks-comm/pom.xml | 2 +- .../hyracks/hyracks-control/hyracks-control-cc/pom.xml | 2 +- .../hyracks/hyracks-control/hyracks-control-common/pom.xml | 2 +- .../hyracks/hyracks-control/hyracks-control-nc/pom.xml | 2 +- .../hyracks/hyracks-control/hyracks-nc-service/pom.xml | 2 +- hyracks-fullstack/hyracks/hyracks-control/pom.xml | 2 +- hyracks-fullstack/hyracks/hyracks-data/hyracks-data-std/pom.xml | 2 +- hyracks-fullstack/hyracks/hyracks-data/pom.xml | 2 +- hyracks-fullstack/hyracks/hyracks-dataflow-common/pom.xml | 2 +- hyracks-fullstack/hyracks/hyracks-dataflow-std/pom.xml | 2 +- hyracks-fullstack/hyracks/hyracks-dist/pom.xml | 2 +- hyracks-fullstack/hyracks/hyracks-documentation/pom.xml | 2 +- .../hyracks/hyracks-examples/btree-example/btreeclient/pom.xml | 2 +- .../hyracks/hyracks-examples
[asterixdb] 04/05: Release Apache AsterixDB 0.9.5 and Hyracks 0.3.5
This is an automated email from the ASF dual-hosted git repository. imaxon pushed a commit to branch release-0.9.5 in repository https://gitbox.apache.org/repos/asf/asterixdb.git commit fadb35b22014bdced66b2a00ab085da59ca7393d Author: Ian Maxon AuthorDate: Mon Jul 6 14:35:59 2020 -0700 Release Apache AsterixDB 0.9.5 and Hyracks 0.3.5 Change-Id: I31cf6146b35a461d522c87f67e4d8304f6f9ab11 Reviewed-on: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/7124 Integration-Tests: Jenkins Tested-by: Ian Maxon Reviewed-by: Ian Maxon --- asterixdb/asterix-active/pom.xml| 2 +- asterixdb/asterix-algebra/pom.xml | 2 +- asterixdb/asterix-app/pom.xml | 2 +- asterixdb/asterix-benchmark/pom.xml | 2 +- asterixdb/asterix-client-helper/pom.xml | 2 +- asterixdb/asterix-common/pom.xml| 2 +- asterixdb/asterix-coverage/pom.xml | 2 +- asterixdb/asterix-dashboard/pom.xml | 2 +- asterixdb/asterix-doc/pom.xml | 2 +- asterixdb/asterix-docker/pom.xml| 2 +- asterixdb/asterix-examples/pom.xml | 2 +- asterixdb/asterix-external-data/pom.xml | 2 +- asterixdb/asterix-fuzzyjoin/pom.xml | 2 +- asterixdb/asterix-geo/pom.xml | 2 +- asterixdb/asterix-hivecompat/pom.xml| 2 +- asterixdb/asterix-lang-aql/pom.xml | 2 +- asterixdb/asterix-lang-common/pom.xml | 2 +- asterixdb/asterix-lang-sqlpp/pom.xml| 2 +- asterixdb/asterix-license/pom.xml | 2 +- .../asterix-grammar-extension-maven-plugin/pom.xml | 2 +- .../asterix-test-datagenerator-maven-plugin/pom.xml | 2 +- .../asterix-maven-plugins/lexer-generator-maven-plugin/pom.xml | 2 +- asterixdb/asterix-maven-plugins/pom.xml | 2 +- .../record-manager-generator-maven-plugin/pom.xml | 2 +- asterixdb/asterix-metadata/pom.xml | 2 +- asterixdb/asterix-om/pom.xml| 2 +- asterixdb/asterix-replication/pom.xml | 2 +- asterixdb/asterix-runtime/pom.xml | 2 +- asterixdb/asterix-server/pom.xml| 2 +- asterixdb/asterix-spidersilk/pom.xml| 2 +- asterixdb/asterix-test-framework/pom.xml| 2 +- asterixdb/asterix-tools/pom.xml | 2 +- asterixdb/asterix-transactions/pom.xml | 2 +- asterixdb/pom.xml | 6 +++--- hyracks-fullstack/algebricks/algebricks-common/pom.xml | 2 +- hyracks-fullstack/algebricks/algebricks-compiler/pom.xml| 2 +- hyracks-fullstack/algebricks/algebricks-core/pom.xml| 2 +- hyracks-fullstack/algebricks/algebricks-data/pom.xml| 2 +- hyracks-fullstack/algebricks/algebricks-rewriter/pom.xml| 2 +- hyracks-fullstack/algebricks/algebricks-runtime/pom.xml | 2 +- hyracks-fullstack/algebricks/algebricks-tests/pom.xml | 2 +- hyracks-fullstack/algebricks/pom.xml| 2 +- hyracks-fullstack/hyracks-fullstack-license/pom.xml | 2 +- hyracks-fullstack/hyracks/hyracks-api/pom.xml | 2 +- hyracks-fullstack/hyracks/hyracks-client/pom.xml| 2 +- hyracks-fullstack/hyracks/hyracks-comm/pom.xml | 2 +- .../hyracks/hyracks-control/hyracks-control-cc/pom.xml | 2 +- .../hyracks/hyracks-control/hyracks-control-common/pom.xml | 2 +- .../hyracks/hyracks-control/hyracks-control-nc/pom.xml | 2 +- .../hyracks/hyracks-control/hyracks-nc-service/pom.xml | 2 +- hyracks-fullstack/hyracks/hyracks-control/pom.xml | 2 +- hyracks-fullstack/hyracks/hyracks-data/hyracks-data-std/pom.xml | 2 +- hyracks-fullstack/hyracks/hyracks-data/pom.xml | 2 +- hyracks-fullstack/hyracks/hyracks-dataflow-common/pom.xml | 2 +- hyracks-fullstack/hyracks/hyracks-dataflow-std/pom.xml | 2 +- hyracks-fullstack/hyracks/hyracks-dist/pom.xml | 2 +- hyracks-fullstack/hyracks/hyracks-documentation/pom.xml | 2 +- .../hyracks/hyracks-examples/btree-example/btreeclient/pom.xml | 2 +- .../hyracks
[asterixdb] 03/05: Merge branch 'master' into release-0.9.5-rc0-manual
This is an automated email from the ASF dual-hosted git repository. imaxon pushed a commit to branch release-0.9.5 in repository https://gitbox.apache.org/repos/asf/asterixdb.git commit ed3f425c2186e0385e35be7a765fca40b3d1d2de Merge: 30b08c5 866d4c8 Author: Ian Maxon AuthorDate: Sun Jul 5 20:12:00 2020 -0700 Merge branch 'master' into release-0.9.5-rc0-manual Change-Id: I493dfffd77c596bb3485eccb00b417d40f47b642 .../org/apache/asterix/active/ActiveManager.java | 2 +- .../algebra/extension/ExtensionStatement.java | 2 + .../optimizer/rules/ConstantFoldingRule.java | 10 +- .../optimizer/rules/am/BTreeAccessMethod.java | 3 +- .../asterix/translator/IRequestParameters.java | 7 + .../asterix/translator/IStatementExecutor.java | 33 ++ .../translator/LangExpressionToPlanTranslator.java | 12 +- .../SqlppExpressionToPlanTranslator.java | 28 +- .../apache/asterix/translator/TypeTranslator.java | 20 +- .../asterix-app/data/json/malformed-json-2.json| 7 + .../data/json/single-line/array_of_objects.json| 1 + .../asterix-app/data/jsonl/malformed-jsonl-1.json | 4 + .../asterix-app/data/jsonl/malformed-jsonl-2.json | 14 + asterixdb/asterix-app/pom.xml | 5 - .../apache/asterix/api/http/server/ApiServlet.java | 2 +- .../asterix/api/http/server/BasicAuthServlet.java | 18 +- .../api/http/server/NCQueryServiceServlet.java | 27 +- .../http/server/QueryServiceRequestParameters.java | 10 + .../api/http/server/QueryServiceServlet.java | 106 ++-- .../api/http/server/RebalanceApiServlet.java | 3 +- .../asterix/api/http/server/UdfApiServlet.java | 458 + .../apache/asterix/api/java/AsterixJavaClient.java | 4 +- .../apache/asterix/app/active/RecoveryTask.java| 6 +- .../asterix/app/external/ExternalLibraryUtil.java | 140 ++ .../asterix/app/external/ExternalLibraryUtils.java | 55 --- .../asterix/app/message/AbstractUdfMessage.java| 66 --- .../message/ExecuteStatementRequestMessage.java| 7 +- .../message/ExecuteStatementResponseMessage.java | 10 + .../apache/asterix/app/message/LoadUdfMessage.java | 41 -- .../asterix/app/message/UdfResponseMessage.java| 70 --- .../apache/asterix/app/nc/NCAppRuntimeContext.java | 25 +- .../app/nc/task/ExternalLibrarySetupTask.java | 50 -- .../app/replication/NcLifecycleCoordinator.java| 2 - .../asterix/app/translator/QueryTranslator.java| 540 ++--- .../asterix/app/translator/RequestParameters.java | 27 +- .../asterix/hyracks/bootstrap/CCApplication.java | 24 +- .../org/apache/asterix/utils/FeedOperations.java | 5 +- .../api/common/AsterixHyracksIntegrationUtil.java | 9 +- .../http/servlet/QueryCancellationServletTest.java | 8 +- .../asterix/test/common/ResultExtractor.java | 4 +- .../apache/asterix/test/common/TestExecutor.java | 79 ++- .../aws/AwsS3ExternalDatasetOnePartitionTest.java | 6 +- .../aws/AwsS3ExternalDatasetTest.java | 172 +-- .../asterix/test/runtime/ExecutionTestUtil.java| 20 +- .../test/runtime/SqlppRQGGroupingSetsIT.java | 7 +- .../agg_filter_01/agg_filter_01.3.sqlpp} | 70 ++- .../agg_filter_01/agg_filter_01.4.sqlpp} | 72 ++- .../agg_filter_01/agg_filter_01.5.sqlpp} | 72 ++- .../agg_filter_01/agg_filter_01.6.sqlpp| 58 +++ .../agg_filter_01/agg_filter_01.7.sqlpp| 58 +++ .../agg_filter_01/agg_filter_01.8.sqlpp} | 71 ++- .../btree-composite-key-non-enforced-04.sqlpp} | 24 +- .../agg_filter_01/agg_filter_01.3.plan | 34 ++ .../agg_filter_01/agg_filter_01.4.plan | 38 ++ .../agg_filter_01/agg_filter_01.5.plan | 94 .../agg_filter_01/agg_filter_01.6.plan | 26 + .../agg_filter_01/agg_filter_01.7.plan | 44 ++ .../agg_filter_01/agg_filter_01.8.plan | 28 ++ .../btree-composite-key-non-enforced-04.plan | 17 + .../agg_filter_01/agg_filter_01.1.ddl.sqlpp} | 39 +- .../agg_filter_01/agg_filter_01.2.update.sqlpp}| 4 +- .../agg_filter_01/agg_filter_01.3.query.sqlpp} | 11 +- .../agg_filter_01/agg_filter_01.4.query.sqlpp} | 13 +- .../agg_filter_01/agg_filter_01.5.query.sqlpp} | 13 +- .../agg_filter_01/agg_filter_01.6.query.sqlpp} | 29 +- .../agg_filter_01/agg_filter_01.7.query.sqlpp} | 29 +- .../agg_filter_01/agg_filter_01.8.query.sqlpp} | 12 +- .../agg_filter_02.neg.5.query.sqlpp} | 9 +- .../agg_filter_02_neg.1.query.sqlpp} | 7 +- .../agg_filter_02_neg.2.query.sqlpp} | 7 +- .../agg_filter_02_neg.3.query.sqlpp} | 8 +- .../agg_filter_02_neg.4.query.sqlpp} | 9 +- .../bad-type-ddl/bad-type-ddl.1.ddl.sqlpp} | 13 +- .../create-dataset-inline-type-2.5.ddl.sqlpp}
[asterixdb] 01/05: [NO ISSUE] excise log4j 1.x
This is an automated email from the ASF dual-hosted git repository. imaxon pushed a commit to branch release-0.9.5 in repository https://gitbox.apache.org/repos/asf/asterixdb.git commit 4a20fb28d9e4ac55fa1b61bdc3baadc599aac201 Author: Ian Maxon AuthorDate: Mon Jun 8 17:23:07 2020 -0700 [NO ISSUE] excise log4j 1.x Change-Id: I1d1d6c5e3e5a2f051552f3518f302f6bb8b4e83d Reviewed-on: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/6684 Integration-Tests: Jenkins Tested-by: Jenkins Reviewed-by: Michael Blow --- asterixdb/asterix-active/pom.xml | 2 +- asterixdb/asterix-app/pom.xml | 4 + asterixdb/asterix-external-data/pom.xml| 8 +- asterixdb/asterix-hivecompat/pom.xml | 4 + asterixdb/asterix-server/pom.xml | 8 +- asterixdb/pom.xml | 89 ++--- hyracks-fullstack/hyracks/hyracks-hdfs/pom.xml | 50 ++--- .../apache/hyracks/hdfs/dataflow/DataflowTest.java | 217 - .../hyracks/hdfs2/dataflow/DataflowTest.java | 134 - hyracks-fullstack/pom.xml | 27 +++ 10 files changed, 132 insertions(+), 411 deletions(-) diff --git a/asterixdb/asterix-active/pom.xml b/asterixdb/asterix-active/pom.xml index 9b5dcbf..210345f 100644 --- a/asterixdb/asterix-active/pom.xml +++ b/asterixdb/asterix-active/pom.xml @@ -55,7 +55,7 @@ org.apache.logging.log4j - log4j-api + log4j-1.2-api diff --git a/asterixdb/asterix-app/pom.xml b/asterixdb/asterix-app/pom.xml index a26a335..67285d5 100644 --- a/asterixdb/asterix-app/pom.xml +++ b/asterixdb/asterix-app/pom.xml @@ -865,5 +865,9 @@ akka-http-core_2.12 test + + org.apache.logging.log4j + log4j-1.2-api + diff --git a/asterixdb/asterix-external-data/pom.xml b/asterixdb/asterix-external-data/pom.xml index f6ebdba..ea8afac 100644 --- a/asterixdb/asterix-external-data/pom.xml +++ b/asterixdb/asterix-external-data/pom.xml @@ -413,10 +413,6 @@ 4.4 - org.apache.logging.log4j - log4j-api - - com.sun.xml.bind jaxb-core @@ -468,5 +464,9 @@ software.amazon.awssdk auth + + org.apache.logging.log4j + log4j-1.2-api + diff --git a/asterixdb/asterix-hivecompat/pom.xml b/asterixdb/asterix-hivecompat/pom.xml index bd5df93..57575c5 100644 --- a/asterixdb/asterix-hivecompat/pom.xml +++ b/asterixdb/asterix-hivecompat/pom.xml @@ -72,5 +72,9 @@ org.apache.hadoop hadoop-mapreduce-client-core + + org.apache.logging.log4j + log4j-1.2-api + diff --git a/asterixdb/asterix-server/pom.xml b/asterixdb/asterix-server/pom.xml index 0db37fc..2fea765 100644 --- a/asterixdb/asterix-server/pom.xml +++ b/asterixdb/asterix-server/pom.xml @@ -698,10 +698,6 @@ hyracks-util - org.apache.logging.log4j - log4j-api - - org.apache.asterix asterix-fuzzyjoin ${project.version} @@ -725,5 +721,9 @@ com.fasterxml.jackson.core jackson-databind + + org.apache.logging.log4j + log4j-1.2-api + diff --git a/asterixdb/pom.xml b/asterixdb/pom.xml index bb3a018..3f0 100644 --- a/asterixdb/pom.xml +++ b/asterixdb/pom.xml @@ -937,23 +937,25 @@ org.apache.hadoop -hadoop-yarn-common -${hadoop.version} - - -org.apache.hadoop -hadoop-yarn-client -${hadoop.version} - - -org.apache.hadoop hadoop-client ${hadoop.version} + + +log4j +log4j + + org.apache.hadoop hadoop-hdfs ${hadoop.version} + + +log4j +log4j + + org.apache.hadoop @@ -980,18 +982,34 @@ jdk.tools jdk.tools + +log4j +log4j + org.apache.hadoop hadoop-hdfs-client ${hadoop.version} + + +log4j +log4j + + org.apache.hadoop hadoop-common ${hadoop.version} tests + + +log4j +log4j + + org.apache.hadoop @@ -1002,17 +1020,33 @@ jdk.tools jdk.tools + +log4j +log4j + org.apache.hadoop hadoop-annotations ${hadoop.version} + + +log4j +log4j + + org.apache.hadoop hadoop-minicluster
[asterixdb] 02/05: [ASTERIXDB-2742] Relax Java version check
This is an automated email from the ASF dual-hosted git repository. imaxon pushed a commit to branch release-0.9.5 in repository https://gitbox.apache.org/repos/asf/asterixdb.git commit 30b08c5cf767e9ec68c1b5610815373e90f4a445 Author: Ian Maxon AuthorDate: Fri Jun 12 10:56:56 2020 -0700 [ASTERIXDB-2742] Relax Java version check Change-Id: I5cbbc367944b489aee651ea050e74990dcf65531 Reviewed-on: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/6763 Integration-Tests: Jenkins Tested-by: Jenkins Reviewed-by: Michael Blow --- .../asterix-server/src/main/opt/local/bin/start-sample-cluster.sh | 5 ++--- .../asterix-server/src/main/opt/local/bin/stop-sample-cluster.sh | 7 --- 2 files changed, 6 insertions(+), 6 deletions(-) diff --git a/asterixdb/asterix-server/src/main/opt/local/bin/start-sample-cluster.sh b/asterixdb/asterix-server/src/main/opt/local/bin/start-sample-cluster.sh index 34c3a0f..65ef608 100755 --- a/asterixdb/asterix-server/src/main/opt/local/bin/start-sample-cluster.sh +++ b/asterixdb/asterix-server/src/main/opt/local/bin/start-sample-cluster.sh @@ -86,10 +86,9 @@ if [ -z "$JAVACMD" ] ; then fi export JAVA_VERSION=$($JAVACMD -version 2>&1 | head -1 | awk '{ print $3 }' | tr -d '"') + case $JAVA_VERSION in - 1.8*|1.9*|10*|11*) -;; - *) + 1.[0-7]*) echo JAVA_HOME must be at version 1.8 or later, but is: $JAVA_VERSION exit 2 esac diff --git a/asterixdb/asterix-server/src/main/opt/local/bin/stop-sample-cluster.sh b/asterixdb/asterix-server/src/main/opt/local/bin/stop-sample-cluster.sh index 97473ac..b79ca0b 100755 --- a/asterixdb/asterix-server/src/main/opt/local/bin/stop-sample-cluster.sh +++ b/asterixdb/asterix-server/src/main/opt/local/bin/stop-sample-cluster.sh @@ -99,14 +99,15 @@ if [ -z "$JAVACMD" ] ; then JAVACMD=`which java` fi fi + export JAVA_VERSION=$($JAVACMD -version 2>&1 | head -1 | awk '{ print $3 }' | tr -d '"') + case $JAVA_VERSION in - 1.8*|1.9*|10*|11*) -;; - *) + 1.[0-7]*) echo JAVA_HOME must be at version 1.8 or later, but is: $JAVA_VERSION exit 2 esac + DIRNAME=$(dirname "$0") [ $(echo $DIRNAME | wc -l) -ne 1 ] && { echo "Paths with spaces are not supported"
[asterixdb] annotated tag apache-asterixdb-0.9.5 created (now adcb4c0)
This is an automated email from the ASF dual-hosted git repository. imaxon pushed a change to annotated tag apache-asterixdb-0.9.5 in repository https://gitbox.apache.org/repos/asf/asterixdb.git. at adcb4c0 (tag) tagging c95de7a9548e6868e97cc664cdb12e6721b508c3 (commit) replaces apache-asterixdb-0.9.4 by Ian Maxon on Fri Aug 7 12:49:34 2020 -0700 - Log - Released RC with final tag -BEGIN PGP SIGNATURE- iQIzBAABCgAdFiEEIf3gbxaFXS93IFrkwLkA3UcwFf0FAl8tsE4ACgkQwLkA3Ucw Ff3Ekw//cfihRbyDgu2mI1mKhd2wpH8hLBIxk+ZF/+R1tJRQ3K1ljUerXjORZ0fA nGuq1Q27Y5L2mWlferxIyOT9eAV8t4lw5gmBVgDzY8OJjpje1hb4cwLj1bTU9sdO scuVg5F/2PcgSg9QSwKEo5/u8krQXP1LAzxGdcBrxHl1ZufCwpHyK6D9ZiPx6Y/r y1DqEh09NBCSQKIaiJPs0j7A6Q+yV47GNQ3n1k1Q153hY/k+TpXzzjnQ37ohrSso Re5oVhL36AFBDsMo4aKHoNJS1JqvXzOYHIOqW9/gjh32wAUfdGdRXF23btZu/Nkl 2aHHrFuZu/a8ocOe7ipgwQB26XIk1FG7Z9sBYKyT9YITAa4hq/CQyJd8vyzAsROl oHdkbiGzWID7et3Y59JHu7FvBbI9zthb3BymIh/TXTjIwJ7H7ygKL0/h59AkYYyf yywNufnyscgR1nUCff7lM6nYkvH2KNyE77Ctu+99IVJtXbG6BRvdyIK6ZCK+v7QA Kki/0icRd+GQ3GCudiO96hp1wscY+cgN4c5fhenCCgb7e4DHGO2WYRbs1hFrYioC NQao3Tcj7wKnlZW1w/i/pbc4b3Tlb97hjb6mC8WDNeZIMCCAHAlwpYpfkYZMC7eL wrGCo5qyzYBjpCttKBzZVyovz474VlhrxQK/SZKWSyv0lRMTL5Q= =jPcq -END PGP SIGNATURE- --- No new revisions were added by this update.
[asterixdb] annotated tag apache-hyracks-0.3.5 created (now 6deeaa8)
This is an automated email from the ASF dual-hosted git repository. imaxon pushed a change to annotated tag apache-hyracks-0.3.5 in repository https://gitbox.apache.org/repos/asf/asterixdb.git. at 6deeaa8 (tag) tagging c95de7a9548e6868e97cc664cdb12e6721b508c3 (commit) replaces apache-asterixdb-0.9.4 by Ian Maxon on Fri Aug 7 12:50:04 2020 -0700 - Log - Released RC with final tag -BEGIN PGP SIGNATURE- iQIzBAABCgAdFiEEIf3gbxaFXS93IFrkwLkA3UcwFf0FAl8tsG0ACgkQwLkA3Ucw Ff3qpxAAg4hiY7m64yCwH1qhSMhHbIjMduk5w1VbOFbhx4sPxNLyzHe8P7YAqCab 8XfgWzJ2mhAjPlZW7EJjuMVbtRTbJ+QA5yEBv6cnKbBlJv9vyNyT14Q1dRge743Y 2SGSl7G0CoVanXb2cE3QL/FH5q8cjBshfnAxDkBzYQaVxW4X+Xr86SFOt27N8rjb PascwxeJ6vzjq8WYOtw3GTEL5RhRs/Uhhy401pJ24nGP1PxO69xvQP1NaGiztSJc sPJqFG8cp8GUdcZgngLqjJPplOMBKckH7jUxNWvOck3MBbPuowfgbsJlJ6E2Lx7h yN6KToDCClLMhgNQcRbtfroqhjXBTpPBetOM7jVvRJIZr1gS3xLFqzBuS7rfOvqp PCzZQz2MjETzw7KTDhmNyZW179EciqJA3qIhXwygEfxq3I3Mt8WRfNHrNqOs03k7 bIwW4RoH662ZnKmJkkU2HxGxxXjcpsIRsu3Qqr1Go2APCg6Ch/U6yOHsnfw5I+qL NgvshEZduPygaXlx3vcmAufBPjVk0KefH2L0WAqsGxyYTV6Rakny+akBeH5vqOHz Gh9eQfy25Wof6qbHq/2sETkQd/LC/ca86xh8Bzu/xPsQqvKM8dRegSxErzmXy+VA fK90NGz5B7vA8nKgBB0CRsihd2rR1Otumtq50OzaKfTsNdZiESw= =PJ3o -END PGP SIGNATURE- --- No new revisions were added by this update.
[asterixdb] branch master updated: [NO ISSUE] Fix bug to block illegal inverted index
This is an automated email from the ASF dual-hosted git repository. imaxon pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/asterixdb.git The following commit(s) were added to refs/heads/master by this push: new e336845 [NO ISSUE] Fix bug to block illegal inverted index e336845 is described below commit e33684506b73368f1f3cd93e82ac03bf9861d21d Author: Rui Guo AuthorDate: Tue Aug 11 23:25:28 2020 -0700 [NO ISSUE] Fix bug to block illegal inverted index Fix bug to block inverted index creation with a variable-length primary key (e.g. string as the primary key). Currently, we don't support such an inverted index in this case. Change-Id: I715d55d48246fd08817b72f7104bc4d436578600 Reviewed-on: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/7523 Integration-Tests: Jenkins Tested-by: Jenkins Reviewed-by: Ian Maxon --- .../asterix/app/translator/QueryTranslator.java| 2 +- ...ex-with-variable-length-primary-key.1.ddl.sqlpp | 33 ++ .../test/resources/runtimets/testsuite_sqlpp.xml | 6 3 files changed, 40 insertions(+), 1 deletion(-) diff --git a/asterixdb/asterix-app/src/main/java/org/apache/asterix/app/translator/QueryTranslator.java b/asterixdb/asterix-app/src/main/java/org/apache/asterix/app/translator/QueryTranslator.java index 04a2f2d..1c9bfb3 100644 --- a/asterixdb/asterix-app/src/main/java/org/apache/asterix/app/translator/QueryTranslator.java +++ b/asterixdb/asterix-app/src/main/java/org/apache/asterix/app/translator/QueryTranslator.java @@ -1095,7 +1095,7 @@ public class QueryTranslator extends AbstractLangTranslator implements IStatemen ITypeTraits typeTrait = TypeTraitProvider.INSTANCE.getTypeTrait(keyType); // If it is not a fixed length -if (typeTrait.getFixedLength() < 0) { +if (!typeTrait.isFixedLength()) { throw new CompilationException(ErrorCode.COMPILATION_ERROR, sourceLoc, "The keyword or ngram index -" + indexName + " cannot be created on the dataset -" + datasetName + " due to its variable-length primary key field - " diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/ddl/create-index/create-inverted-index-with-variable-length-primary-key/create-inverted-index-with-variable-length-primary-key.1.ddl.sqlpp b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/ddl/create-index/create-inverted-index-with-variable-length-primary-key/create-inverted-index-with-variable-length-primary-key.1.ddl.sqlpp new file mode 100644 index 000..d6e9bbb --- /dev/null +++ b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/ddl/create-index/create-inverted-index-with-variable-length-primary-key/create-inverted-index-with-variable-length-primary-key.1.ddl.sqlpp @@ -0,0 +1,33 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ + +DROP DATAVERSE MyDataverse IF EXISTS; +CREATE DATAVERSE MyDataverse; +USE MyDataverse; + +DROP TYPE MyDataset IF EXISTS; +CREATE TYPE MyDataType AS OPEN +{ myKey: string, myValue: string? }; + +DROP DATASET MyDataset IF EXISTS; +CREATE DATASET MyDataset(MyDataType) PRIMARY KEY myKey; + +DROP INDEX MyDataset.MyIndex IF EXISTS; +// should fail because inverted index with a variable length primary key are not supported yet +CREATE INDEX MyIndex ON MyDataset(myValue) type fulltext; diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/testsuite_sqlpp.xml b/asterixdb/asterix-app/src/test/resources/runtimets/testsuite_sqlpp.xml index e04f341..19f5614 100644 --- a/asterixdb/asterix-app/src/test/resources/runtimets/testsuite_sqlpp.xml +++ b/asterixdb/asterix-app/src/test/resources/runtimets/testsuite_sqlpp.xml @@ -3967,6 +3967,12 @@ Syntax error: In line 53 >>create primary index if exists sec_primary_idx1 on LineItem;<< Encountered "exists" at column 26. + + +create-inverted-index-with-variable-length-primary-key +
[asterixdb] branch master updated: [NO ISSUE] Polish exception message
This is an automated email from the ASF dual-hosted git repository. imaxon pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/asterixdb.git The following commit(s) were added to refs/heads/master by this push: new d86e7cb [NO ISSUE] Polish exception message d86e7cb is described below commit d86e7cbb924b68f76174a75ac4cdd7e5a749938e Author: Rui Guo AuthorDate: Mon Aug 17 21:21:57 2020 -0700 [NO ISSUE] Polish exception message Polish the exception message when creating inverted index with a variable-length primary key which is not support yet. Change-Id: I95b6a0088ca4934a6ac4e0fc84d3f8c0b264a56a Reviewed-on: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/7583 Integration-Tests: Jenkins Tested-by: Jenkins Reviewed-by: Ian Maxon --- .../main/java/org/apache/asterix/app/translator/QueryTranslator.java | 4 ++-- .../asterix-app/src/test/resources/runtimets/testsuite_sqlpp.xml | 2 +- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/asterixdb/asterix-app/src/main/java/org/apache/asterix/app/translator/QueryTranslator.java b/asterixdb/asterix-app/src/main/java/org/apache/asterix/app/translator/QueryTranslator.java index 1c9bfb3..78a82d6 100644 --- a/asterixdb/asterix-app/src/main/java/org/apache/asterix/app/translator/QueryTranslator.java +++ b/asterixdb/asterix-app/src/main/java/org/apache/asterix/app/translator/QueryTranslator.java @@ -1097,8 +1097,8 @@ public class QueryTranslator extends AbstractLangTranslator implements IStatemen // If it is not a fixed length if (!typeTrait.isFixedLength()) { throw new CompilationException(ErrorCode.COMPILATION_ERROR, sourceLoc, -"The keyword or ngram index -" + indexName + " cannot be created on the dataset -" -+ datasetName + " due to its variable-length primary key field - " +"The keyword or ngram index " + indexName + " cannot be created on the dataset " ++ datasetName + " due to its variable-length primary key field " + partitioningKey); } diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/testsuite_sqlpp.xml b/asterixdb/asterix-app/src/test/resources/runtimets/testsuite_sqlpp.xml index 58af775..1495ce0 100644 --- a/asterixdb/asterix-app/src/test/resources/runtimets/testsuite_sqlpp.xml +++ b/asterixdb/asterix-app/src/test/resources/runtimets/testsuite_sqlpp.xml @@ -3970,7 +3970,7 @@ create-inverted-index-with-variable-length-primary-key -ASX1079: Compilation error: The keyword or ngram index -MyIndex cannot be created on the dataset -MyDataset due to its variable-length primary key field - [myKey] (in line 33, at column 1) +ASX1079: Compilation error: The keyword or ngram index MyIndex cannot be created on the dataset MyDataset due to its variable-length primary key field [myKey] (in line 33, at column 1)
[asterixdb] branch master updated: [ASTERIXDB-2176][RT] Flexible python pathing
This is an automated email from the ASF dual-hosted git repository. imaxon pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/asterixdb.git The following commit(s) were added to refs/heads/master by this push: new ae092dd [ASTERIXDB-2176][RT] Flexible python pathing ae092dd is described below commit ae092dd7cb3eec160950703edbf2967ba2b31e34 Author: Ian Maxon AuthorDate: Thu Aug 20 13:51:02 2020 -0700 [ASTERIXDB-2176][RT] Flexible python pathing - Allow python path and args to be specified in config Change-Id: I6f75b29be59090250b2ff9992279187babd97fff Reviewed-on: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/7423 Integration-Tests: Jenkins Tested-by: Jenkins Reviewed-by: Michael Blow --- asterixdb/asterix-app/pom.xml | 14 ++--- .../asterix/api/http/server/BasicAuthServlet.java | 51 +--- .../asterix/api/http/server/ServletConstants.java | 1 + .../asterix/api/http/server/UdfApiServlet.java | 70 +- .../asterix/hyracks/bootstrap/CCApplication.java | 8 ++- .../asterix-app/src/main/resources/entrypoint.py | 10 ++-- .../ExternalScalarPythonFunctionEvaluator.java | 64 .../LibraryDeployPrepareOperatorDescriptor.java| 30 ++ .../control/common/controllers/NCConfig.java | 16 - 9 files changed, 175 insertions(+), 89 deletions(-) diff --git a/asterixdb/asterix-app/pom.xml b/asterixdb/asterix-app/pom.xml index 27c0215..ef443ea 100644 --- a/asterixdb/asterix-app/pom.xml +++ b/asterixdb/asterix-app/pom.xml @@ -37,8 +37,8 @@ ${basedir}/.. ${basedir}/src/main/appended-resources pom.xml,src/main/java,src/main/resources - ${project.build.directory}${file.separator}bin${file.separator}pip3 - ${project.build.directory}${file.separator}bin${file.separator}shiv +${project.build.directory}/bin/pip3 +${project.build.directory}/bin/shiv @@ -382,8 +382,8 @@ - ${project.build.directory}${file.separator}Scripts${file.separator}pip3.exe - ${project.build.directory}${file.separator}Scripts${file.separator}shiv.exe +${project.build.directory}\Scripts\pip3.exe +${project.build.directory}\Scripts\shiv.exe @@ -827,9 +827,9 @@ commons-csv -org.apache.httpcomponents -httpmime -test + org.apache.httpcomponents + httpmime + test diff --git a/asterixdb/asterix-app/src/main/java/org/apache/asterix/api/http/server/BasicAuthServlet.java b/asterixdb/asterix-app/src/main/java/org/apache/asterix/api/http/server/BasicAuthServlet.java index d8e167f..f25d223 100644 --- a/asterixdb/asterix-app/src/main/java/org/apache/asterix/api/http/server/BasicAuthServlet.java +++ b/asterixdb/asterix-app/src/main/java/org/apache/asterix/api/http/server/BasicAuthServlet.java @@ -18,35 +18,68 @@ */ package org.apache.asterix.api.http.server; -import static org.apache.asterix.api.http.server.ServletConstants.CREDENTIAL_MAP; +import static org.apache.asterix.api.http.server.ServletConstants.*; +import java.io.IOException; import java.nio.charset.StandardCharsets; import java.util.Base64; +import java.util.Collections; import java.util.Map; import java.util.concurrent.ConcurrentMap; +import org.apache.commons.lang3.RandomStringUtils; +import org.apache.hyracks.http.api.IServlet; import org.apache.hyracks.http.api.IServletRequest; import org.apache.hyracks.http.api.IServletResponse; -import org.apache.hyracks.http.server.AbstractServlet; import org.apache.logging.log4j.Level; import org.apache.logging.log4j.LogManager; import org.apache.logging.log4j.Logger; import org.mindrot.jbcrypt.BCrypt; import io.netty.handler.codec.http.HttpHeaderNames; +import io.netty.handler.codec.http.HttpMethod; import io.netty.handler.codec.http.HttpResponseStatus; -public abstract class BasicAuthServlet extends AbstractServlet { +public class BasicAuthServlet implements IServlet { private static final Logger LOGGER = LogManager.getLogger(); public static String BASIC_AUTH_METHOD_NAME = "Basic"; private Base64.Decoder b64Decoder; Map storedCredentials; +Map ephemeralCredentials; +private String sysAuthHeader; +private final IServlet delegate; +private ConcurrentMap ctx; -protected BasicAuthServlet(ConcurrentMap ctx, String... paths) { -super(ctx, paths); +public BasicAuthServlet(ConcurrentMap ctx, IServlet delegate) { +this.delegate = delegate; b64Decoder = Base64.getDecoder(); storedCredentials = (Map) ctx.get(CREDENTIAL_MAP); +this.ctx = ctx; +// generate internal user +String sysUser; +do { +sysUser = generateRandomString(32); +} while (storedCredentials.containsKey(sysUser)); +String sysPassword = generateRa
[asterixdb] branch master updated: [ASTERIXDB-2781] Allow multi-options
This is an automated email from the ASF dual-hosted git repository. imaxon pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/asterixdb.git The following commit(s) were added to refs/heads/master by this push: new bdea5ef [ASTERIXDB-2781] Allow multi-options bdea5ef is described below commit bdea5ef2d07df9215c9b31c2aecfb3053bdcb542 Author: Ian Maxon AuthorDate: Mon Sep 21 20:53:43 2020 -0700 [ASTERIXDB-2781] Allow multi-options - Allow options that are array typed to be specified with multi-options in INI configs - Change PYTHON_ADDITIONAL_PKGS to be STRING_ARRAY Change-Id: I5ff9338524407e14a16640a08fc0abeb74a3ebde Reviewed-on: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/8024 Integration-Tests: Jenkins Tested-by: Jenkins Reviewed-by: Dmitry Lychagin Reviewed-by: Michael Blow --- asterixdb/asterix-app/src/main/resources/cc.conf | 6 -- asterixdb/asterix-app/src/test/resources/cc.conf | 3 ++- .../ExternalScalarPythonFunctionEvaluator.java | 8 +++- .../control/common/config/ConfigManager.java | 24 -- .../hyracks/control/common/config/ConfigUtils.java | 7 +++ .../hyracks/control/common/config/OptionTypes.java | 10 + .../control/common/controllers/NCConfig.java | 2 +- 7 files changed, 45 insertions(+), 15 deletions(-) diff --git a/asterixdb/asterix-app/src/main/resources/cc.conf b/asterixdb/asterix-app/src/main/resources/cc.conf index 9e9ac51..ccd35f8 100644 --- a/asterixdb/asterix-app/src/main/resources/cc.conf +++ b/asterixdb/asterix-app/src/main/resources/cc.conf @@ -18,7 +18,8 @@ [nc/asterix_nc1] txn.log.dir=target/tmp/asterix_nc1/txnlog core.dump.dir=target/tmp/asterix_nc1/coredump -iodevices=asterix_nc1/iodevice1,asterix_nc1/iodevice2 +iodevices=asterix_nc1/iodevice1 +iodevices=asterix_nc1/iodevice2 nc.api.port=19004 #jvm.args=-agentlib:jdwp=transport=dt_socket,server=y,suspend=y,address=5006 @@ -26,7 +27,8 @@ nc.api.port=19004 ncservice.port=9091 txn.log.dir=target/tmp/asterix_nc2/txnlog core.dump.dir=target/tmp/asterix_nc2/coredump -iodevices=asterix_nc2/iodevice1,asterix_nc2/iodevice2 +iodevices=asterix_nc2/iodevice1 +iodevices=asterix_nc1/iodevice2 nc.api.port=19005 #jvm.args=-agentlib:jdwp=transport=dt_socket,server=y,suspend=y,address=5007 diff --git a/asterixdb/asterix-app/src/test/resources/cc.conf b/asterixdb/asterix-app/src/test/resources/cc.conf index a113e3a..158eaa0 100644 --- a/asterixdb/asterix-app/src/test/resources/cc.conf +++ b/asterixdb/asterix-app/src/test/resources/cc.conf @@ -18,7 +18,8 @@ [nc/asterix_nc1] txn.log.dir=target/tmp/asterix_nc1/txnlog core.dump.dir=target/tmp/asterix_nc1/coredump -iodevices=target/tmp/asterix_nc1/iodevice1,../asterix-server/target/tmp/asterix_nc1/iodevice2 +iodevices=target/tmp/asterix_nc1/iodevice1, +iodevices=../asterix-server/target/tmp/asterix_nc1/iodevice2 nc.api.port=19004 #jvm.args=-agentlib:jdwp=transport=dt_socket,server=y,suspend=y,address=5006 diff --git a/asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/library/ExternalScalarPythonFunctionEvaluator.java b/asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/library/ExternalScalarPythonFunctionEvaluator.java index a5eccce..2353c6d 100644 --- a/asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/library/ExternalScalarPythonFunctionEvaluator.java +++ b/asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/library/ExternalScalarPythonFunctionEvaluator.java @@ -91,11 +91,9 @@ class ExternalScalarPythonFunctionEvaluator extends ExternalScalarFunctionEvalua File pythonPath = new File(pythonPathCmd); List sitePkgs = new ArrayList<>(); sitePkgs.add(SITE_PACKAGES); -String addlSitePackagesRaw = - ctx.getServiceContext().getAppConfig().getString((NCConfig.Option.PYTHON_ADDITIONAL_PACKAGES)); -if (addlSitePackagesRaw != null) { - sitePkgs.addAll(Arrays.asList(addlSitePackagesRaw.split(File.pathSeparator))); -} +String[] addlSitePackages = + ctx.getServiceContext().getAppConfig().getStringArray((NCConfig.Option.PYTHON_ADDITIONAL_PACKAGES)); +sitePkgs.addAll(Arrays.asList(addlSitePackages)); if (cfg.getBoolean(NCConfig.Option.PYTHON_USE_BUNDLED_MSGPACK)) { sitePkgs.add("ipc" + File.separator + SITE_PACKAGES + File.separator); } diff --git a/hyracks-fullstack/hyracks/hyracks-control/hyracks-control-common/src/main/java/org/apache/hyracks/control/common/config/ConfigManager.java b/hyracks-fullstack/hyracks/hyracks-control/hyracks-control-common/src/main/java/org/apache/hyracks/control/common/config/ConfigManager.java index c175dd4..0ba9090 100644 --- a/hyracks-fullstack/hyracks/hyracks-control/hyracks-control-common/src/main/java/org/apache/hyrack
[asterixdb] branch master updated: [NO ISSUE] Add raw SQL++ grammar to docs
This is an automated email from the ASF dual-hosted git repository. imaxon pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/asterixdb.git The following commit(s) were added to refs/heads/master by this push: new 731fa70 [NO ISSUE] Add raw SQL++ grammar to docs 731fa70 is described below commit 731fa7048669a4d2ccfd9b2a96e2a8d5da856484 Author: Ian Maxon AuthorDate: Fri Sep 25 13:36:24 2020 -0700 [NO ISSUE] Add raw SQL++ grammar to docs Change-Id: Ie8dd906de9369da9db810c94e891d50d3266c190 Reviewed-on: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/8103 Integration-Tests: Jenkins Tested-by: Jenkins Reviewed-by: Ian Maxon Reviewed-by: Dmitry Lychagin Contrib: Ian Maxon --- asterixdb/asterix-doc/pom.xml | 12 +++ asterixdb/asterix-doc/src/site/site.xml | 1 + asterixdb/asterix-lang-sqlpp/pom.xml| 36 +++-- 3 files changed, 20 insertions(+), 29 deletions(-) diff --git a/asterixdb/asterix-doc/pom.xml b/asterixdb/asterix-doc/pom.xml index f7d914e..c54f439 100644 --- a/asterixdb/asterix-doc/pom.xml +++ b/asterixdb/asterix-doc/pom.xml @@ -81,6 +81,18 @@ run + + sqlpp + prepare-package + + + + + + + run + + diff --git a/asterixdb/asterix-doc/src/site/site.xml b/asterixdb/asterix-doc/src/site/site.xml index 6db028e..f1cd93b 100644 --- a/asterixdb/asterix-doc/src/site/site.xml +++ b/asterixdb/asterix-doc/src/site/site.xml @@ -80,6 +80,7 @@ + diff --git a/asterixdb/asterix-lang-sqlpp/pom.xml b/asterixdb/asterix-lang-sqlpp/pom.xml index 2a2783b..b04aa66 100644 --- a/asterixdb/asterix-lang-sqlpp/pom.xml +++ b/asterixdb/asterix-lang-sqlpp/pom.xml @@ -52,6 +52,13 @@ true + +javacc-jjdoc + + jjdoc + +generate-test-sources + @@ -106,35 +113,6 @@ - - - jjdoc - - - jjdoc.skip - !true - - - - - -org.codehaus.mojo -javacc-maven-plugin - - -javacc-jjdoc - - jjdoc - -generate-test-sources - - - - - - - - org.apache.hyracks
[asterixdb] 01/02: Merge 0.9.5/0.3.5 release branch
This is an automated email from the ASF dual-hosted git repository. imaxon pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/asterixdb.git commit 44b4dd8eb1101b90957e8f2d16c823a63077d160 Merge: 731fa70 c95de7a Author: Ian Maxon AuthorDate: Thu Oct 1 12:58:06 2020 -0700 Merge 0.9.5/0.3.5 release branch Change-Id: I32cf2146a35e461d522c87f67e4d8304f6f9ab11 asterixdb/asterix-active/pom.xml | 4 +- asterixdb/asterix-algebra/pom.xml | 2 +- asterixdb/asterix-app/pom.xml | 6 +- asterixdb/asterix-benchmark/pom.xml| 2 +- asterixdb/asterix-client-helper/pom.xml| 2 +- asterixdb/asterix-common/pom.xml | 2 +- asterixdb/asterix-coverage/pom.xml | 2 +- asterixdb/asterix-dashboard/pom.xml| 2 +- asterixdb/asterix-doc/pom.xml | 2 +- asterixdb/asterix-docker/pom.xml | 2 +- asterixdb/asterix-examples/pom.xml | 2 +- asterixdb/asterix-external-data/pom.xml| 10 +- asterixdb/asterix-fuzzyjoin/pom.xml| 2 +- asterixdb/asterix-geo/pom.xml | 2 +- asterixdb/asterix-hivecompat/pom.xml | 6 +- asterixdb/asterix-lang-aql/pom.xml | 2 +- asterixdb/asterix-lang-common/pom.xml | 2 +- asterixdb/asterix-lang-sqlpp/pom.xml | 2 +- asterixdb/asterix-license/pom.xml | 2 +- .../asterix-grammar-extension-maven-plugin/pom.xml | 2 +- .../pom.xml| 2 +- .../lexer-generator-maven-plugin/pom.xml | 2 +- asterixdb/asterix-maven-plugins/pom.xml| 2 +- .../record-manager-generator-maven-plugin/pom.xml | 2 +- asterixdb/asterix-metadata/pom.xml | 2 +- asterixdb/asterix-om/pom.xml | 2 +- asterixdb/asterix-replication/pom.xml | 2 +- asterixdb/asterix-runtime/pom.xml | 2 +- asterixdb/asterix-server/pom.xml | 10 +- .../src/main/opt/local/bin/start-sample-cluster.sh | 5 +- .../src/main/opt/local/bin/stop-sample-cluster.sh | 7 +- asterixdb/asterix-spidersilk/pom.xml | 2 +- asterixdb/asterix-test-framework/pom.xml | 2 +- asterixdb/asterix-tools/pom.xml| 2 +- asterixdb/asterix-transactions/pom.xml | 2 +- asterixdb/pom.xml | 95 ++--- .../algebricks/algebricks-common/pom.xml | 2 +- .../algebricks/algebricks-compiler/pom.xml | 2 +- .../algebricks/algebricks-core/pom.xml | 2 +- .../algebricks/algebricks-data/pom.xml | 2 +- .../algebricks/algebricks-rewriter/pom.xml | 2 +- .../algebricks/algebricks-runtime/pom.xml | 2 +- .../algebricks/algebricks-tests/pom.xml| 2 +- hyracks-fullstack/algebricks/pom.xml | 2 +- .../hyracks-fullstack-license/pom.xml | 2 +- hyracks-fullstack/hyracks/hyracks-api/pom.xml | 2 +- hyracks-fullstack/hyracks/hyracks-client/pom.xml | 2 +- hyracks-fullstack/hyracks/hyracks-comm/pom.xml | 2 +- .../hyracks-control/hyracks-control-cc/pom.xml | 2 +- .../hyracks-control/hyracks-control-common/pom.xml | 2 +- .../hyracks-control/hyracks-control-nc/pom.xml | 2 +- .../hyracks-control/hyracks-nc-service/pom.xml | 2 +- hyracks-fullstack/hyracks/hyracks-control/pom.xml | 2 +- .../hyracks/hyracks-data/hyracks-data-std/pom.xml | 2 +- hyracks-fullstack/hyracks/hyracks-data/pom.xml | 2 +- .../hyracks/hyracks-dataflow-common/pom.xml| 2 +- .../hyracks/hyracks-dataflow-std/pom.xml | 2 +- hyracks-fullstack/hyracks/hyracks-dist/pom.xml | 2 +- .../hyracks/hyracks-documentation/pom.xml | 2 +- .../btree-example/btreeclient/pom.xml | 2 +- .../btree-example/btreehelper/pom.xml | 2 +- .../btree-example/btreeserver/pom.xml | 2 +- .../hyracks/hyracks-examples/btree-example/pom.xml | 2 +- .../hyracks-integration-tests/pom.xml | 2 +- .../hyracks-examples/hyracks-shutdown-test/pom.xml | 2 +- hyracks-fullstack/hyracks/hyracks-examples/pom.xml | 2 +- .../hyracks/hyracks-examples/text-example/pom.xml | 2 +- .../text-example/textclient/pom.xml| 2 +- .../text-example/texthelper/pom.xml| 2 +- .../text-example/textserver/pom.xml| 2 +- .../hyracks/hyracks-examples/tpch-example/pom.xml | 2 +- .../tpch-example/tpchclient/pom.xml| 2 +- .../tpch-example/tpchserver/pom.xml| 2 +- hyracks-fullstack/hyracks/hyracks-hdfs/pom.xml | 52 ++--- .../apache/hyracks/hdfs/dataflow/DataflowTest.java | 217
[asterixdb] 02/02: Merge "Merge 0.9.5/0.3.5 release branch"
This is an automated email from the ASF dual-hosted git repository. imaxon pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/asterixdb.git commit 30b246fd8db4152252850d0588b02ec2ff057146 Merge: c908ed1 44b4dd8 Author: Apache AsterixDB Gerrit AuthorDate: Fri Oct 2 04:09:59 2020 + Merge "Merge 0.9.5/0.3.5 release branch" asterixdb/asterix-active/pom.xml | 4 +- asterixdb/asterix-algebra/pom.xml | 2 +- asterixdb/asterix-app/pom.xml | 6 +- asterixdb/asterix-benchmark/pom.xml| 2 +- asterixdb/asterix-client-helper/pom.xml| 2 +- asterixdb/asterix-common/pom.xml | 2 +- asterixdb/asterix-coverage/pom.xml | 2 +- asterixdb/asterix-dashboard/pom.xml| 2 +- asterixdb/asterix-doc/pom.xml | 2 +- asterixdb/asterix-docker/pom.xml | 2 +- asterixdb/asterix-examples/pom.xml | 2 +- asterixdb/asterix-external-data/pom.xml| 10 +- asterixdb/asterix-fuzzyjoin/pom.xml| 2 +- asterixdb/asterix-geo/pom.xml | 2 +- asterixdb/asterix-hivecompat/pom.xml | 6 +- asterixdb/asterix-lang-aql/pom.xml | 2 +- asterixdb/asterix-lang-common/pom.xml | 2 +- asterixdb/asterix-lang-sqlpp/pom.xml | 2 +- asterixdb/asterix-license/pom.xml | 2 +- .../asterix-grammar-extension-maven-plugin/pom.xml | 2 +- .../pom.xml| 2 +- .../lexer-generator-maven-plugin/pom.xml | 2 +- asterixdb/asterix-maven-plugins/pom.xml| 2 +- .../record-manager-generator-maven-plugin/pom.xml | 2 +- asterixdb/asterix-metadata/pom.xml | 2 +- asterixdb/asterix-om/pom.xml | 2 +- asterixdb/asterix-replication/pom.xml | 2 +- asterixdb/asterix-runtime/pom.xml | 2 +- asterixdb/asterix-server/pom.xml | 10 +- .../src/main/opt/local/bin/start-sample-cluster.sh | 5 +- .../src/main/opt/local/bin/stop-sample-cluster.sh | 7 +- asterixdb/asterix-spidersilk/pom.xml | 2 +- asterixdb/asterix-test-framework/pom.xml | 2 +- asterixdb/asterix-tools/pom.xml| 2 +- asterixdb/asterix-transactions/pom.xml | 2 +- asterixdb/pom.xml | 95 ++--- .../algebricks/algebricks-common/pom.xml | 2 +- .../algebricks/algebricks-compiler/pom.xml | 2 +- .../algebricks/algebricks-core/pom.xml | 2 +- .../algebricks/algebricks-data/pom.xml | 2 +- .../algebricks/algebricks-rewriter/pom.xml | 2 +- .../algebricks/algebricks-runtime/pom.xml | 2 +- .../algebricks/algebricks-tests/pom.xml| 2 +- hyracks-fullstack/algebricks/pom.xml | 2 +- .../hyracks-fullstack-license/pom.xml | 2 +- hyracks-fullstack/hyracks/hyracks-api/pom.xml | 2 +- hyracks-fullstack/hyracks/hyracks-client/pom.xml | 2 +- hyracks-fullstack/hyracks/hyracks-comm/pom.xml | 2 +- .../hyracks-control/hyracks-control-cc/pom.xml | 2 +- .../hyracks-control/hyracks-control-common/pom.xml | 2 +- .../hyracks-control/hyracks-control-nc/pom.xml | 2 +- .../hyracks-control/hyracks-nc-service/pom.xml | 2 +- hyracks-fullstack/hyracks/hyracks-control/pom.xml | 2 +- .../hyracks/hyracks-data/hyracks-data-std/pom.xml | 2 +- hyracks-fullstack/hyracks/hyracks-data/pom.xml | 2 +- .../hyracks/hyracks-dataflow-common/pom.xml| 2 +- .../hyracks/hyracks-dataflow-std/pom.xml | 2 +- hyracks-fullstack/hyracks/hyracks-dist/pom.xml | 2 +- .../hyracks/hyracks-documentation/pom.xml | 2 +- .../btree-example/btreeclient/pom.xml | 2 +- .../btree-example/btreehelper/pom.xml | 2 +- .../btree-example/btreeserver/pom.xml | 2 +- .../hyracks/hyracks-examples/btree-example/pom.xml | 2 +- .../hyracks-integration-tests/pom.xml | 2 +- .../hyracks-examples/hyracks-shutdown-test/pom.xml | 2 +- hyracks-fullstack/hyracks/hyracks-examples/pom.xml | 2 +- .../hyracks/hyracks-examples/text-example/pom.xml | 2 +- .../text-example/textclient/pom.xml| 2 +- .../text-example/texthelper/pom.xml| 2 +- .../text-example/textserver/pom.xml| 2 +- .../hyracks/hyracks-examples/tpch-example/pom.xml | 2 +- .../tpch-example/tpchclient/pom.xml| 2 +- .../tpch-example/tpchserver/pom.xml| 2 +- hyracks-fullstack/hyracks/hyracks-hdfs/pom.xml | 52 ++--- .../apache/hyracks/hdfs/dataflow/DataflowTest.java | 217 - .../hyracks/hdfs
[asterixdb] branch master updated (c908ed1 -> 30b246f)
This is an automated email from the ASF dual-hosted git repository. imaxon pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/asterixdb.git. from c908ed1 Merge "Merge branch gerrit/mad-hatter" add 4a20fb2 [NO ISSUE] excise log4j 1.x add 30b08c5 [ASTERIXDB-2742] Relax Java version check add ed3f425 Merge branch 'master' into release-0.9.5-rc0-manual add fadb35b Release Apache AsterixDB 0.9.5 and Hyracks 0.3.5 add c95de7a Advance version to AsterixDB 0.9.6-SNAPSHOT and Hyracks 0.3.6-SNAPSHOT new 44b4dd8 Merge 0.9.5/0.3.5 release branch new 30b246f Merge "Merge 0.9.5/0.3.5 release branch" The 2 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: asterixdb/asterix-active/pom.xml | 4 +- asterixdb/asterix-algebra/pom.xml | 2 +- asterixdb/asterix-app/pom.xml | 6 +- asterixdb/asterix-benchmark/pom.xml| 2 +- asterixdb/asterix-client-helper/pom.xml| 2 +- asterixdb/asterix-common/pom.xml | 2 +- asterixdb/asterix-coverage/pom.xml | 2 +- asterixdb/asterix-dashboard/pom.xml| 2 +- asterixdb/asterix-doc/pom.xml | 2 +- asterixdb/asterix-docker/pom.xml | 2 +- asterixdb/asterix-examples/pom.xml | 2 +- asterixdb/asterix-external-data/pom.xml| 10 +- asterixdb/asterix-fuzzyjoin/pom.xml| 2 +- asterixdb/asterix-geo/pom.xml | 2 +- asterixdb/asterix-hivecompat/pom.xml | 6 +- asterixdb/asterix-lang-aql/pom.xml | 2 +- asterixdb/asterix-lang-common/pom.xml | 2 +- asterixdb/asterix-lang-sqlpp/pom.xml | 2 +- asterixdb/asterix-license/pom.xml | 2 +- .../asterix-grammar-extension-maven-plugin/pom.xml | 2 +- .../pom.xml| 2 +- .../lexer-generator-maven-plugin/pom.xml | 2 +- asterixdb/asterix-maven-plugins/pom.xml| 2 +- .../record-manager-generator-maven-plugin/pom.xml | 2 +- asterixdb/asterix-metadata/pom.xml | 2 +- asterixdb/asterix-om/pom.xml | 2 +- asterixdb/asterix-replication/pom.xml | 2 +- asterixdb/asterix-runtime/pom.xml | 2 +- asterixdb/asterix-server/pom.xml | 10 +- .../src/main/opt/local/bin/start-sample-cluster.sh | 5 +- .../src/main/opt/local/bin/stop-sample-cluster.sh | 7 +- asterixdb/asterix-spidersilk/pom.xml | 2 +- asterixdb/asterix-test-framework/pom.xml | 2 +- asterixdb/asterix-tools/pom.xml| 2 +- asterixdb/asterix-transactions/pom.xml | 2 +- asterixdb/pom.xml | 95 ++--- .../algebricks/algebricks-common/pom.xml | 2 +- .../algebricks/algebricks-compiler/pom.xml | 2 +- .../algebricks/algebricks-core/pom.xml | 2 +- .../algebricks/algebricks-data/pom.xml | 2 +- .../algebricks/algebricks-rewriter/pom.xml | 2 +- .../algebricks/algebricks-runtime/pom.xml | 2 +- .../algebricks/algebricks-tests/pom.xml| 2 +- hyracks-fullstack/algebricks/pom.xml | 2 +- .../hyracks-fullstack-license/pom.xml | 2 +- hyracks-fullstack/hyracks/hyracks-api/pom.xml | 2 +- hyracks-fullstack/hyracks/hyracks-client/pom.xml | 2 +- hyracks-fullstack/hyracks/hyracks-comm/pom.xml | 2 +- .../hyracks-control/hyracks-control-cc/pom.xml | 2 +- .../hyracks-control/hyracks-control-common/pom.xml | 2 +- .../hyracks-control/hyracks-control-nc/pom.xml | 2 +- .../hyracks-control/hyracks-nc-service/pom.xml | 2 +- hyracks-fullstack/hyracks/hyracks-control/pom.xml | 2 +- .../hyracks/hyracks-data/hyracks-data-std/pom.xml | 2 +- hyracks-fullstack/hyracks/hyracks-data/pom.xml | 2 +- .../hyracks/hyracks-dataflow-common/pom.xml| 2 +- .../hyracks/hyracks-dataflow-std/pom.xml | 2 +- hyracks-fullstack/hyracks/hyracks-dist/pom.xml | 2 +- .../hyracks/hyracks-documentation/pom.xml | 2 +- .../btree-example/btreeclient/pom.xml | 2 +- .../btree-example/btreehelper/pom.xml | 2 +- .../btree-example/btreeserver/pom.xml | 2 +- .../hyracks/hyracks-examples/btree-example/pom.xml | 2 +- .../hyracks-integration-tests/pom.xml | 2 +- .../hyracks-examples/hyracks-shutdown-test/pom.xml | 2 +- hyracks-fullstack/hyracks/hyracks-examples/p
svn commit: r41700 - /dev/asterixdb/
Author: imaxon Date: Mon Oct 5 06:49:36 2020 New Revision: 41700 Log: upload new RC Added: dev/asterixdb/apache-asterixdb-0.9.6-source-release.zip (with props) dev/asterixdb/apache-asterixdb-0.9.6-source-release.zip.asc dev/asterixdb/apache-asterixdb-0.9.6-source-release.zip.sha256 dev/asterixdb/apache-hyracks-0.3.6-source-release.zip (with props) dev/asterixdb/apache-hyracks-0.3.6-source-release.zip.asc dev/asterixdb/apache-hyracks-0.3.6-source-release.zip.sha256 dev/asterixdb/asterix-server-0.9.6-binary-assembly.zip (with props) dev/asterixdb/asterix-server-0.9.6-binary-assembly.zip.asc dev/asterixdb/asterix-server-0.9.6-binary-assembly.zip.sha256 Removed: dev/asterixdb/apache-asterixdb-0.9.5-source-release.zip dev/asterixdb/apache-asterixdb-0.9.5-source-release.zip.asc dev/asterixdb/apache-asterixdb-0.9.5-source-release.zip.sha256 dev/asterixdb/apache-hyracks-0.3.5-source-release.zip dev/asterixdb/apache-hyracks-0.3.5-source-release.zip.asc dev/asterixdb/apache-hyracks-0.3.5-source-release.zip.sha256 dev/asterixdb/asterix-server-0.9.5-binary-assembly.zip dev/asterixdb/asterix-server-0.9.5-binary-assembly.zip.asc dev/asterixdb/asterix-server-0.9.5-binary-assembly.zip.sha256 Added: dev/asterixdb/apache-asterixdb-0.9.6-source-release.zip == Binary file - no diff available. Propchange: dev/asterixdb/apache-asterixdb-0.9.6-source-release.zip -- svn:mime-type = application/octet-stream Added: dev/asterixdb/apache-asterixdb-0.9.6-source-release.zip.asc == --- dev/asterixdb/apache-asterixdb-0.9.6-source-release.zip.asc (added) +++ dev/asterixdb/apache-asterixdb-0.9.6-source-release.zip.asc Mon Oct 5 06:49:36 2020 @@ -0,0 +1,16 @@ +-BEGIN PGP SIGNATURE- + +iQIzBAABCgAdFiEEIf3gbxaFXS93IFrkwLkA3UcwFf0FAl96uy4ACgkQwLkA3Ucw +Ff1y8Q/+NaDqg7yP2fjoxiHKP9XXbmIMViLFCS3eknWNHSjVBFvrOzuePd7lXzex +YbkiNpqYnp2gWaQDVdzEj/Svi5ppYTbeF7v9F1n6H4UIKKcAM6nNtJXKBRJtzw6T +XqQ0lGC3s5dZxCWdWbt+H+sIVgOaCxRMwexqiJCW3smHrmrtjVAGRDYHjsHJV1fO +YPFB0FewPPaW+ARVZd85ACOXGl/WX8q45VVjISZWqPXNpoxwLnTzSVgpLuWh24XD +t7wt7ugqFbbdx164v63C8YAiOeFNn2zud0yneWC2d9tRIu6OnAPGGDZ9A0hcpIoO +XI/SzxuQIsI1YrEy9vhJlt3LuYmoe/C1ISSqG+9c5oj8sMWOsCm3W9DZjMttHJvn +qLMm5AbvuACGUEhmEd2HJ7klf6ypkGvV6WqKh8idPeVbQs2ZV8+qSD8EPDDPt8jJ +h0xRrkrCeB6haYBXjYS+86Kbglt08wSMkUnFKsmgOz4ulF6eNorLWzxSBCnhhZuQ +0uieWs5xr9lZgggNMC5rWIk8awFXRgaU21jEv088yFI6HWysCNn6Kk8z7+FH9r44 +eg6+J/U8W1juRqmpwHsa9RWAL2TphciNqqUc8ABUz2LcrcRxfTxohQqkdL1XUYVI +ucUbVfypjwjivpUsBxn4tYN7GYgrzDtyx3j8a4HJo8fMKTUDmoY= +=Pyrc +-END PGP SIGNATURE- Added: dev/asterixdb/apache-asterixdb-0.9.6-source-release.zip.sha256 == --- dev/asterixdb/apache-asterixdb-0.9.6-source-release.zip.sha256 (added) +++ dev/asterixdb/apache-asterixdb-0.9.6-source-release.zip.sha256 Mon Oct 5 06:49:36 2020 @@ -0,0 +1 @@ +98443ff5a8bb5b25b38fa81b1a4fb43aeb1522742164462909062d1cdf7cd88d apache-asterixdb-0.9.6-source-release.zip Added: dev/asterixdb/apache-hyracks-0.3.6-source-release.zip == Binary file - no diff available. Propchange: dev/asterixdb/apache-hyracks-0.3.6-source-release.zip -- svn:mime-type = application/octet-stream Added: dev/asterixdb/apache-hyracks-0.3.6-source-release.zip.asc == --- dev/asterixdb/apache-hyracks-0.3.6-source-release.zip.asc (added) +++ dev/asterixdb/apache-hyracks-0.3.6-source-release.zip.asc Mon Oct 5 06:49:36 2020 @@ -0,0 +1,16 @@ +-BEGIN PGP SIGNATURE- + +iQIzBAABCgAdFiEEIf3gbxaFXS93IFrkwLkA3UcwFf0FAl96uUUACgkQwLkA3Ucw +Ff0fLhAApEesc2oOJ4H/AZo8z/1CYvc0i6Mc4Bhjg8KzvFBBfejHxTiJibRBuTav +H6IlTZvCRJmILLlWFoeTMSCXOO1l4jkM3bbJvYqGyFQ8JmTKs/YDlDqIpIevEGiB +GCeeluxZF0YL691f3Hn+wt4MhdAWIVLdNzthRcxfdpKX6ejgoKBWrA6uvQnanYvd ++IaO5Hal6q1tRYcZMwWJfcFiblPj+Sl0cqlgCIiz4ko1RXfEv/BP6RwPJSMi7kpQ +V5Im3kPh1A5ad7hlJwdrfb2KEwsoqqQpZcn4iY/Uf1cFZ3xwHTvmmZRxUAUrA27r +V+GpMqOcdFtMEo3ZbeDK2NXppQd0nQ27ZwdYzXgygL6ntQefpg9zW26hLpcRgIKa +L+lQbeKUCp4Sll266MX9DxY8c8VrCpfmV9cjH9mR3r1GiX7vziiRhjAtA3erpzZl +LqUF+nvdusUOwIYCNxh4d6oaZ8xxLSS3CPI/8BFMudxQrBUgTvdu/WZHTugbyhHC +AjJ8P2xXqmsWPqARMekLUM3RP7885q6UYo9oWjVCd++3zcrkxpNlP2+jpb39gODT +0KE1h2DZymfc6ugsFI0DR7gn40p0JsnykhFwWKmcsylE/hoBKaWID3D3HrezvfrL +Xf7Z3SYp6f/l7Yo/E2kSlbRON0hLGc0JUB7IgxIXmlMQJvVMzAw= +=1Wux +-END PGP SIGNATURE- Added: dev/asterixdb/apache-hyracks-0.3.6-source-release.zip.sha256 == --- dev/asterixdb/apache-hyracks-0.3.6-source
[asterixdb] branch master updated: [NO ISSUE] Fix msgpack LICENSE/NOTICE inclusions
This is an automated email from the ASF dual-hosted git repository. imaxon pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/asterixdb.git The following commit(s) were added to refs/heads/master by this push: new a556b84 [NO ISSUE] Fix msgpack LICENSE/NOTICE inclusions a556b84 is described below commit a556b84c8f3c2208c08ef4d5af7ac7b0f1ee9413 Author: Ian Maxon AuthorDate: Wed Oct 7 01:48:02 2020 -0700 [NO ISSUE] Fix msgpack LICENSE/NOTICE inclusions Change-Id: I7ee331ee5372791fc8373eab0e0eac76d14b91a3 Reviewed-on: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/8287 Integration-Tests: Jenkins Reviewed-by: Ian Maxon Reviewed-by: Hussain Towaileb Reviewed-by: Michael Blow Tested-by: Jenkins --- asterixdb/asterix-server/pom.xml | 4 .../main/appended-resources/supplemental-models.xml | 19 +++ ...content.com_msgpack_msgpack-java_0.8.20_NOTICE.txt | 5 + 3 files changed, 28 insertions(+) diff --git a/asterixdb/asterix-server/pom.xml b/asterixdb/asterix-server/pom.xml index 8dd3222..2c12ab0 100644 --- a/asterixdb/asterix-server/pom.xml +++ b/asterixdb/asterix-server/pom.xml @@ -205,6 +205,10 @@ org.mindrot:jbcrypt:0.4 http://www.mindrot.org/files/jBCrypt/LICENSE + + org.msgpack:msgpack-core:0.8.20 + https://raw.githubusercontent.com/msgpack/msgpack-java/0.8.20/NOTICE + diff --git a/asterixdb/src/main/appended-resources/supplemental-models.xml b/asterixdb/src/main/appended-resources/supplemental-models.xml index 060ce3a..e457628 100644 --- a/asterixdb/src/main/appended-resources/supplemental-models.xml +++ b/asterixdb/src/main/appended-resources/supplemental-models.xml @@ -500,6 +500,20 @@ + org.msgpack + msgpack-core + + 0.8.20 + 0.8.20 +0.8.20 + + + + + + + + org.mindrot jbcrypt @@ -508,6 +522,11 @@ http://www.mindrot.org/files/jBCrypt/LICENSE + + 0.4 + 0.4 +0.4 + diff --git a/asterixdb/src/main/licenses/content/raw.githubusercontent.com_msgpack_msgpack-java_0.8.20_NOTICE.txt b/asterixdb/src/main/licenses/content/raw.githubusercontent.com_msgpack_msgpack-java_0.8.20_NOTICE.txt new file mode 100644 index 000..bc6328d --- /dev/null +++ b/asterixdb/src/main/licenses/content/raw.githubusercontent.com_msgpack_msgpack-java_0.8.20_NOTICE.txt @@ -0,0 +1,5 @@ +This product includes the software developed by third-party: + + * Google Guava https://code.google.com/p/guava-libraries/ (APL2) + * sbt-extras: https://github.com/paulp/sbt-extras (BSD) (LICENSE.sbt-extras.txt) +
svn commit: r41925 - in /release/asterixdb: asterixdb-0.9.6/ hyracks-0.3.6/
Author: imaxon Date: Wed Oct 14 06:18:57 2020 New Revision: 41925 Log: Release Apache AsterixDB 0.9.6 and hyracks 0.9.6 Added: release/asterixdb/asterixdb-0.9.6/ release/asterixdb/asterixdb-0.9.6/apache-asterixdb-0.9.6-source-release.zip (with props) release/asterixdb/asterixdb-0.9.6/apache-asterixdb-0.9.6-source-release.zip.asc release/asterixdb/asterixdb-0.9.6/apache-asterixdb-0.9.6-source-release.zip.sha256 release/asterixdb/asterixdb-0.9.6/asterix-server-0.9.6-binary-assembly.zip (with props) release/asterixdb/asterixdb-0.9.6/asterix-server-0.9.6-binary-assembly.zip.asc release/asterixdb/asterixdb-0.9.6/asterix-server-0.9.6-binary-assembly.zip.sha256 release/asterixdb/hyracks-0.3.6/ release/asterixdb/hyracks-0.3.6/apache-hyracks-0.3.6-source-release.zip (with props) release/asterixdb/hyracks-0.3.6/apache-hyracks-0.3.6-source-release.zip.asc release/asterixdb/hyracks-0.3.6/apache-hyracks-0.3.6-source-release.zip.sha256 Added: release/asterixdb/asterixdb-0.9.6/apache-asterixdb-0.9.6-source-release.zip == Binary file - no diff available. Propchange: release/asterixdb/asterixdb-0.9.6/apache-asterixdb-0.9.6-source-release.zip -- svn:mime-type = application/octet-stream Added: release/asterixdb/asterixdb-0.9.6/apache-asterixdb-0.9.6-source-release.zip.asc == --- release/asterixdb/asterixdb-0.9.6/apache-asterixdb-0.9.6-source-release.zip.asc (added) +++ release/asterixdb/asterixdb-0.9.6/apache-asterixdb-0.9.6-source-release.zip.asc Wed Oct 14 06:18:57 2020 @@ -0,0 +1,16 @@ +-BEGIN PGP SIGNATURE- + +iQIzBAABCgAdFiEEIf3gbxaFXS93IFrkwLkA3UcwFf0FAl96uy4ACgkQwLkA3Ucw +Ff1y8Q/+NaDqg7yP2fjoxiHKP9XXbmIMViLFCS3eknWNHSjVBFvrOzuePd7lXzex +YbkiNpqYnp2gWaQDVdzEj/Svi5ppYTbeF7v9F1n6H4UIKKcAM6nNtJXKBRJtzw6T +XqQ0lGC3s5dZxCWdWbt+H+sIVgOaCxRMwexqiJCW3smHrmrtjVAGRDYHjsHJV1fO +YPFB0FewPPaW+ARVZd85ACOXGl/WX8q45VVjISZWqPXNpoxwLnTzSVgpLuWh24XD +t7wt7ugqFbbdx164v63C8YAiOeFNn2zud0yneWC2d9tRIu6OnAPGGDZ9A0hcpIoO +XI/SzxuQIsI1YrEy9vhJlt3LuYmoe/C1ISSqG+9c5oj8sMWOsCm3W9DZjMttHJvn +qLMm5AbvuACGUEhmEd2HJ7klf6ypkGvV6WqKh8idPeVbQs2ZV8+qSD8EPDDPt8jJ +h0xRrkrCeB6haYBXjYS+86Kbglt08wSMkUnFKsmgOz4ulF6eNorLWzxSBCnhhZuQ +0uieWs5xr9lZgggNMC5rWIk8awFXRgaU21jEv088yFI6HWysCNn6Kk8z7+FH9r44 +eg6+J/U8W1juRqmpwHsa9RWAL2TphciNqqUc8ABUz2LcrcRxfTxohQqkdL1XUYVI +ucUbVfypjwjivpUsBxn4tYN7GYgrzDtyx3j8a4HJo8fMKTUDmoY= +=Pyrc +-END PGP SIGNATURE- Added: release/asterixdb/asterixdb-0.9.6/apache-asterixdb-0.9.6-source-release.zip.sha256 == --- release/asterixdb/asterixdb-0.9.6/apache-asterixdb-0.9.6-source-release.zip.sha256 (added) +++ release/asterixdb/asterixdb-0.9.6/apache-asterixdb-0.9.6-source-release.zip.sha256 Wed Oct 14 06:18:57 2020 @@ -0,0 +1 @@ +98443ff5a8bb5b25b38fa81b1a4fb43aeb1522742164462909062d1cdf7cd88d apache-asterixdb-0.9.6-source-release.zip Added: release/asterixdb/asterixdb-0.9.6/asterix-server-0.9.6-binary-assembly.zip == Binary file - no diff available. Propchange: release/asterixdb/asterixdb-0.9.6/asterix-server-0.9.6-binary-assembly.zip -- svn:mime-type = application/octet-stream Added: release/asterixdb/asterixdb-0.9.6/asterix-server-0.9.6-binary-assembly.zip.asc == --- release/asterixdb/asterixdb-0.9.6/asterix-server-0.9.6-binary-assembly.zip.asc (added) +++ release/asterixdb/asterixdb-0.9.6/asterix-server-0.9.6-binary-assembly.zip.asc Wed Oct 14 06:18:57 2020 @@ -0,0 +1,16 @@ +-BEGIN PGP SIGNATURE- + +iQIzBAABCgAdFiEEIf3gbxaFXS93IFrkwLkA3UcwFf0FAl96vNsACgkQwLkA3Ucw +Ff2E2Q//buhPOiKshDdHLh2icmObRaKAkFWkH14tKy+a4fWzedYpXEsJYYqIP/uL +yf31nIIoloOsyrx034PkRxQdKvk7DDwy83HCE7rhjs5rYSMd4vluZefvvA4/atWk +3DY1zLzo6EiLhXYkn+4e8/gBeIKNffsk61FP8A1ikJ8WtYLNVpXbaU8mItIfHgy/ +DKPuS9fQ2KtGrK/WaZSPpYWiS40Dx7jqC3rtpz13f7lzSxNj5+u6ZMJHV4n2dNcN +QBw3WYEz43415wp1Efq+w6pTCJvdwEODh8YGJEB3BYGb8fAq9biUJcjJzU66kOUw +5GHxcvAuVNn7dygxU+2LT/Xc5K4qfbODoJ8Lxrmng27WMd4TrGplIteUhKz+M4Nc +nM65q6ResVrKTKNfGV6ZpTpIZ4ev9uXnEUihDjg9qgSOQzfm8ZxM9c8GQN02C4ha +g4hbuRis17v2RfR2BaMSYUrp418GMKiPM/vhiAhScegBdquNVPaIP8NO4zBRu/Dj +uWM6vffuFwJHOyaZu79KbBL2QwcKCsDiX0nb8W0no4sWW8Ig2HA3ZGc4seKQcijf +L7frKBA9u+k6d3IVfoE7J9hEBA8DMKk9kdNZL8PRn/Zz9gK+GOZTmrhPNTlr7hIJ +DmPhQwDf/z4VLwc09u2YmLJ5o++VbR8i6O0qCOTtrqfH0mezPg4= +=JKIR +-END PGP SIGNATURE- Added: release/asterixdb/asterixdb-0.9.6/asterix-server-0.9.6-binary-assembly.zip.sha256 == --- release/asterixdb
[asterixdb] branch master updated: [NO ISSUE] Allow UDF Requests from NC
This is an automated email from the ASF dual-hosted git repository. imaxon pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/asterixdb.git The following commit(s) were added to refs/heads/master by this push: new 68cbe16 [NO ISSUE] Allow UDF Requests from NC 68cbe16 is described below commit 68cbe169d4ec4ca257a0fe9ec3f25b4aaaf686d7 Author: Ian Maxon AuthorDate: Tue Oct 27 16:43:55 2020 -0700 [NO ISSUE] Allow UDF Requests from NC - Route UDF requests on NCs to the CC - Enable library tests to round-robin betwen NC and CC Change-Id: I16557c2efb4622c9639c2992c8b2ef0624bd650e Reviewed-on: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/8186 Integration-Tests: Jenkins Tested-by: Jenkins Reviewed-by: Dmitry Lychagin --- .../asterix/api/http/server/NCUdfApiServlet.java | 168 + .../asterix/api/http/server/UdfApiServlet.java | 64 .../message/AbstractInternalRequestMessage.java| 123 +++ .../app/message/CreateLibraryRequestMessage.java | 55 +++ .../app/message/DropLibraryRequestMessage.java | 48 ++ .../message/ExecuteStatementRequestMessage.java| 9 +- .../app/message/InternalRequestResponse.java | 53 +++ .../asterix/hyracks/bootstrap/CCApplication.java | 3 +- .../asterix/hyracks/bootstrap/NCApplication.java | 36 + .../asterix/app/external/ExternalUDFLibrarian.java | 28 +--- .../app/external/IExternalUDFLibrarian.java| 7 +- .../apache/asterix/test/common/TestExecutor.java | 9 +- .../asterix/test/runtime/SqlppExecutionIT.java | 23 ++- asterixdb/asterix-app/src/test/resources/cc.conf | 1 + .../asterix/common/config/ExternalProperties.java | 16 +- .../control/common/controllers/CCConfig.java | 15 +- .../common/controllers/ControllerConfig.java | 3 +- .../control/common/controllers/NCConfig.java | 14 +- 18 files changed, 598 insertions(+), 77 deletions(-) diff --git a/asterixdb/asterix-app/src/main/java/org/apache/asterix/api/http/server/NCUdfApiServlet.java b/asterixdb/asterix-app/src/main/java/org/apache/asterix/api/http/server/NCUdfApiServlet.java new file mode 100644 index 000..6d7a847 --- /dev/null +++ b/asterixdb/asterix-app/src/main/java/org/apache/asterix/api/http/server/NCUdfApiServlet.java @@ -0,0 +1,168 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ +package org.apache.asterix.api.http.server; + +import java.io.IOException; +import java.io.InputStream; +import java.net.URI; +import java.nio.file.Files; +import java.nio.file.Path; +import java.nio.file.Paths; +import java.util.concurrent.ConcurrentMap; +import java.util.concurrent.TimeUnit; + +import org.apache.asterix.app.message.CreateLibraryRequestMessage; +import org.apache.asterix.app.message.DropLibraryRequestMessage; +import org.apache.asterix.app.message.InternalRequestResponse; +import org.apache.asterix.common.api.IApplicationContext; +import org.apache.asterix.common.api.INcApplicationContext; +import org.apache.asterix.common.api.IRequestReference; +import org.apache.asterix.common.functions.ExternalFunctionLanguage; +import org.apache.asterix.common.messaging.api.ICcAddressedMessage; +import org.apache.asterix.common.messaging.api.INCMessageBroker; +import org.apache.asterix.common.messaging.api.MessageFuture; +import org.apache.asterix.common.metadata.DataverseName; +import org.apache.asterix.compiler.provider.ILangCompilationProvider; +import org.apache.asterix.external.library.ExternalLibraryManager; +import org.apache.commons.io.IOUtils; +import org.apache.hyracks.api.application.INCServiceContext; +import org.apache.hyracks.control.common.context.ServerContext; +import org.apache.hyracks.control.common.work.SynchronizableWork; +import org.apache.hyracks.control.nc.NodeControllerService; +import org.apache.hyracks.http.api.IServletRequest; +import org.apache.hyracks.http.api.IServletResponse; +import org.apache.hyracks.http.server.utils.HttpUtil; + +import io.netty.handler.codec.http.HttpResponseStatus; +import io.netty.handler.codec.http.HttpScheme; + +public class NCUdfApiServlet e
[asterixdb] branch master updated: [NO ISSUE] Remove UDF API servlet from CC
This is an automated email from the ASF dual-hosted git repository. imaxon pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/asterixdb.git The following commit(s) were added to refs/heads/master by this push: new 517a4bc [NO ISSUE] Remove UDF API servlet from CC 517a4bc is described below commit 517a4bcee3e5ec90da146c71d6c5371ab6d47df3 Author: Ian Maxon AuthorDate: Mon Oct 19 21:37:40 2020 -0700 [NO ISSUE] Remove UDF API servlet from CC Also refactor the woringDir to be given from the library manager Change-Id: I5f8bc2f0bc4b68f9d4b3ebafc3266e9584d8535b Reviewed-on: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/8483 Integration-Tests: Jenkins Tested-by: Jenkins Reviewed-by: Ian Maxon --- .../asterix/api/http/server/NCUdfApiServlet.java | 212 +++- .../asterix/api/http/server/UdfApiServlet.java | 363 - .../asterix/hyracks/bootstrap/CCApplication.java | 8 - .../asterix/common/library/ILibraryManager.java| 2 + .../external/library/ExternalLibraryManager.java | 9 + 5 files changed, 211 insertions(+), 383 deletions(-) diff --git a/asterixdb/asterix-app/src/main/java/org/apache/asterix/api/http/server/NCUdfApiServlet.java b/asterixdb/asterix-app/src/main/java/org/apache/asterix/api/http/server/NCUdfApiServlet.java index 6d7a847..7af7fd4 100644 --- a/asterixdb/asterix-app/src/main/java/org/apache/asterix/api/http/server/NCUdfApiServlet.java +++ b/asterixdb/asterix-app/src/main/java/org/apache/asterix/api/http/server/NCUdfApiServlet.java @@ -18,12 +18,21 @@ */ package org.apache.asterix.api.http.server; +import static org.apache.asterix.api.http.server.ServletConstants.HYRACKS_CONNECTION_ATTR; +import static org.apache.asterix.api.http.server.ServletConstants.SYS_AUTH_HEADER; +import static org.apache.asterix.common.functions.ExternalFunctionLanguage.JAVA; +import static org.apache.asterix.common.functions.ExternalFunctionLanguage.PYTHON; + import java.io.IOException; import java.io.InputStream; +import java.io.PrintWriter; import java.net.URI; import java.nio.file.Files; import java.nio.file.Path; import java.nio.file.Paths; +import java.util.Arrays; +import java.util.Collections; +import java.util.Map; import java.util.concurrent.ConcurrentMap; import java.util.concurrent.TimeUnit; @@ -32,49 +41,102 @@ import org.apache.asterix.app.message.DropLibraryRequestMessage; import org.apache.asterix.app.message.InternalRequestResponse; import org.apache.asterix.common.api.IApplicationContext; import org.apache.asterix.common.api.INcApplicationContext; +import org.apache.asterix.common.api.IReceptionist; import org.apache.asterix.common.api.IRequestReference; +import org.apache.asterix.common.exceptions.ErrorCode; +import org.apache.asterix.common.exceptions.RuntimeDataException; import org.apache.asterix.common.functions.ExternalFunctionLanguage; +import org.apache.asterix.common.library.LibraryDescriptor; import org.apache.asterix.common.messaging.api.ICcAddressedMessage; import org.apache.asterix.common.messaging.api.INCMessageBroker; import org.apache.asterix.common.messaging.api.MessageFuture; import org.apache.asterix.common.metadata.DataverseName; import org.apache.asterix.compiler.provider.ILangCompilationProvider; -import org.apache.asterix.external.library.ExternalLibraryManager; +import org.apache.commons.io.FileUtils; +import org.apache.commons.io.FilenameUtils; import org.apache.commons.io.IOUtils; +import org.apache.commons.lang3.StringUtils; +import org.apache.hyracks.algebricks.common.utils.Pair; import org.apache.hyracks.api.application.INCServiceContext; -import org.apache.hyracks.control.common.context.ServerContext; +import org.apache.hyracks.api.client.IHyracksClientConnection; +import org.apache.hyracks.api.exceptions.IFormattedException; import org.apache.hyracks.control.common.work.SynchronizableWork; import org.apache.hyracks.control.nc.NodeControllerService; import org.apache.hyracks.http.api.IServletRequest; import org.apache.hyracks.http.api.IServletResponse; +import org.apache.hyracks.http.server.AbstractServlet; import org.apache.hyracks.http.server.utils.HttpUtil; +import org.apache.hyracks.util.file.FileUtil; +import org.apache.logging.log4j.LogManager; +import org.apache.logging.log4j.Logger; +import io.netty.handler.codec.http.HttpRequest; import io.netty.handler.codec.http.HttpResponseStatus; import io.netty.handler.codec.http.HttpScheme; +import io.netty.handler.codec.http.multipart.FileUpload; +import io.netty.handler.codec.http.multipart.HttpPostRequestDecoder; +import io.netty.handler.codec.http.multipart.InterfaceHttpData; -public class NCUdfApiServlet extends UdfApiServlet { +public class NCUdfApiServlet extends AbstractServlet { INcApplicationContext appCtx; INCServiceContext srvCtx; +protected final IApplicationContext plainAppCtx; +private final HttpScheme
[asterixdb-site] branch asf-site updated: Update site for release
This is an automated email from the ASF dual-hosted git repository. imaxon pushed a commit to branch asf-site in repository https://gitbox.apache.org/repos/asf/asterixdb-site.git The following commit(s) were added to refs/heads/asf-site by this push: new 76ff9d4 Update site for release 76ff9d4 is described below commit 76ff9d4d9a3956d6d2f1ba6e5edc16d99026c48d Author: Ian Maxon AuthorDate: Thu Oct 22 17:03:09 2020 -0700 Update site for release Change-Id: I8aa5dce0b9a9c51df2fcafa91d2e4f47d9fb4cf5 Reviewed-on: https://asterix-gerrit.ics.uci.edu/c/incubator-asterixdb-site/+/8544 Reviewed-by: Ian Maxon --- _config.yml| 8 content/about.html | 4 ++-- content/community.html | 4 ++-- content/dev-setup.html | 6 +++--- content/download.html | 6 +++--- content/extension.html | 4 ++-- content/index.html | 22 +++--- content/pushing.html | 4 ++-- content/site.html | 4 ++-- content/tools.html | 4 ++-- 10 files changed, 33 insertions(+), 33 deletions(-) diff --git a/_config.yml b/_config.yml index 5b61816..9806fb3 100644 --- a/_config.yml +++ b/_config.yml @@ -1,10 +1,10 @@ url: "" name: "Apache AsterixDB" -STABLE_VERSION: 0.9.5 -UNSTABLE_VERSION: 0.9.5-SNAPSHOT -STABLE_NCSERVICE_URL: "http://www.apache.org/dyn/closer.lua/asterixdb/asterixdb-0.9.5/asterix-server-0.9.5-binary-assembly.zip"; -UNSTABLE_NCSERVICE_URL: "https://asterix-jenkins.ics.uci.edu/job/asterix-integration-tests/lastSuccessfulBuild/org.apache.asterix$asterix-server/artifact/org.apache.asterix/asterix-server/0.9.5-SNAPSHOT/asterix-server-0.9.5-SNAPSHOT-binary-assembly.zip"; +STABLE_VERSION: 0.9.6 +UNSTABLE_VERSION: 0.9.6-SNAPSHOT +STABLE_NCSERVICE_URL: "http://www.apache.org/dyn/closer.lua/asterixdb/asterixdb-0.9.6/asterix-server-0.9.6-binary-assembly.zip"; +UNSTABLE_NCSERVICE_URL: "https://asterix-jenkins.ics.uci.edu/job/asterix-integration-tests/lastSuccessfulBuild/org.apache.asterix$asterix-server/artifact/org.apache.asterix/asterix-server/0.9.6-SNAPSHOT/asterix-server-0.9.6-SNAPSHOT-binary-assembly.zip"; DOCS_BASE_URL: "docs" UNSTABLE_DOCS_BASE_URL: "https://ci.apache.org/projects/asterixdb/"; diff --git a/content/about.html b/content/about.html index f79be01..537bb33 100644 --- a/content/about.html +++ b/content/about.html @@ -53,10 +53,10 @@ Documentation Stable release - Documentation for 0.9.5 + Documentation for 0.9.6 Snapshot release - https://ci.apache.org/projects/asterixdb/index.html";>Documentation for 0.9.5-SNAPSHOT + https://ci.apache.org/projects/asterixdb/index.html";>Documentation for 0.9.6-SNAPSHOT Tutorial for SQL Users Don Chamberlin's SQL++ for SQL Users diff --git a/content/community.html b/content/community.html index fc00153..8e3a35f 100644 --- a/content/community.html +++ b/content/community.html @@ -53,10 +53,10 @@ Documentation Stable release - Documentation for 0.9.5 + Documentation for 0.9.6 Snapshot release - https://ci.apache.org/projects/asterixdb/index.html";>Documentation for 0.9.5-SNAPSHOT + https://ci.apache.org/projects/asterixdb/index.html";>Documentation for 0.9.6-SNAPSHOT Tutorial for SQL Users Don Chamberlin's SQL++ for SQL Users diff --git a/content/dev-setup.html b/content/dev-setup.html index 7d543bf..1a05df2 100644 --- a/content/dev-setup.html +++ b/content/dev-setup.html @@ -53,10 +53,10 @@ Documentation Stable release - Documentation for 0.9.5 + Documentation for 0.9.6 Snapshot release - https://ci.apache.org/projects/asterixdb/index.html";>Documentation for 0.9.5-SNAPSHOT + https://ci.apache.org/projects/asterixdb/index.html";>Documentation for 0.9.6-SNAPSHOT Tutorial for SQL Users Don Chamberlin's SQL++ for SQL Users @@ -149,7 +149,7 @@ A great way to start contributing is to pick a bug labelled starter in JIRA and If you are a developer simply looking for a snapshot of the latest development version of AsterixDB to test out your application that is based on top of AsterixDB, snapshots from the latest successful push to our master branch are available below. These are non ASF-release, automatic builds. Server Package -https://asterix-jenkins.ics.uci.edu/job/as
[asterixdb] branch master updated: [NO ISSUE] Check if Java UDF classes are assignable to IFunctionFactory
This is an automated email from the ASF dual-hosted git repository. imaxon pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/asterixdb.git The following commit(s) were added to refs/heads/master by this push: new d5ca10b [NO ISSUE] Check if Java UDF classes are assignable to IFunctionFactory d5ca10b is described below commit d5ca10bfc2599373cc348f87388ccb915b953b5d Author: Ian Maxon AuthorDate: Tue Nov 24 17:35:01 2020 -0800 [NO ISSUE] Check if Java UDF classes are assignable to IFunctionFactory As of now, we instantiate whatever class is specified in an external UDF upon invocation before checking if it can be cast to an IFunctionFactory. This might be dangerous if some class in our classloaders contains exploitable code in static intializers. So we should check if the class is assignable to IFunctionFactory before attempting to instantiate it. Change-Id: Id14581cca775b54de6f3fd8c0cf032d7c352bbbe Reviewed-on: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/9043 Integration-Tests: Jenkins Tested-by: Jenkins Reviewed-by: Dmitry Lychagin --- .../library/ExternalScalarJavaFunctionEvaluator.java| 13 ++--- 1 file changed, 10 insertions(+), 3 deletions(-) diff --git a/asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/library/ExternalScalarJavaFunctionEvaluator.java b/asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/library/ExternalScalarJavaFunctionEvaluator.java index 33b0369..a8d246e 100755 --- a/asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/library/ExternalScalarJavaFunctionEvaluator.java +++ b/asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/library/ExternalScalarJavaFunctionEvaluator.java @@ -53,9 +53,16 @@ class ExternalScalarJavaFunctionEvaluator extends ExternalScalarFunctionEvaluato String classname = finfo.getExternalIdentifier().get(0); try { -Class clazz = Class.forName(classname, true, library.getClassLoader()); -IFunctionFactory externalFunctionFactory = (IFunctionFactory) clazz.newInstance(); -externalFunctionInstance = (IExternalScalarFunction) externalFunctionFactory.getExternalFunction(); +//first, check if this class is assignable to the correct interface before running static initializers that +//may be dangerous +Class clazz = Class.forName(classname, false, library.getClassLoader()); +if (IFunctionFactory.class.isAssignableFrom(clazz)) { +//check if clazz implements IFunctionFactory +IFunctionFactory externalFunctionFactory = (IFunctionFactory) clazz.newInstance(); +externalFunctionInstance = (IExternalScalarFunction) externalFunctionFactory.getExternalFunction(); +} else { +throw new ClassCastException("Specified class does not implement IFunctionFactory"); +} } catch (Exception e) { throw new RuntimeDataException(ErrorCode.LIBRARY_EXTERNAL_FUNCTION_UNABLE_TO_LOAD_CLASS, e, classname); }
[asterixdb] branch master updated: [NO ISSUE] Polish UDF document
This is an automated email from the ASF dual-hosted git repository. imaxon pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/asterixdb.git The following commit(s) were added to refs/heads/master by this push: new 582d7c6 [NO ISSUE] Polish UDF document 582d7c6 is described below commit 582d7c60a73c5dbfcf7b619dee2e163baa46dfb4 Author: Rui Guo AuthorDate: Mon Dec 14 11:57:57 2020 -0800 [NO ISSUE] Polish UDF document Fix typos and update API port. Change-Id: Ib5fd3467560ea0ca06d5ec8044da8c3ee1dbcce9 Reviewed-on: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/9103 Integration-Tests: Jenkins Tested-by: Jenkins Reviewed-by: Ian Maxon --- .../asterix-doc/src/main/user-defined_function/udf.md| 16 +++- 1 file changed, 11 insertions(+), 5 deletions(-) diff --git a/asterixdb/asterix-doc/src/main/user-defined_function/udf.md b/asterixdb/asterix-doc/src/main/user-defined_function/udf.md index 7ca23bb..fe72789 100644 --- a/asterixdb/asterix-doc/src/main/user-defined_function/udf.md +++ b/asterixdb/asterix-doc/src/main/user-defined_function/udf.md @@ -19,7 +19,7 @@ ## Introduction -Apache AsterixDB supports three languages for writing user-defined functions (UDFs): SQL++, Java and Python +Apache AsterixDB supports three languages for writing user-defined functions (UDFs): SQL++, Java and Python. A user can encapsulate data processing logic into a UDF and invoke it later repeatedly. For SQL++ functions, a user can refer to [SQL++ Functions](sqlpp/manual.html#Functions) for their usages. This document will focus on UDFs in languages other than SQL++ @@ -50,7 +50,7 @@ Now,restart the cluster if it was already started to allow the Cluster Controlle ## Installing a Java UDF Library To install a UDF package to the cluster, we need to send a Multipart Form-data HTTP request to the `/admin/udf` endpoint -of the CC at the normal API port (`19002` by default). The request should use HTTP Basic authentication. This means your +of the CC at the normal API port (`19004` by default). The request should use HTTP Basic authentication. This means your credentials will *not* be obfuscated or encrypted *in any way*, so submit to this endpoint over localhost or a network where you know your traffic is safe from eavesdropping. Any suitable tool will do, but for the example here I will use `curl` which is widely available. @@ -60,18 +60,24 @@ For example, to install a library with the following criteria: * `udfs` dataverse name * with a new Library name of `testlib` * from `lib.zip` in the present working directory -* to the cluster at `localhost` with API port `19002` +* to the cluster at `localhost` with API port `19004` of the Asterix CC * with credentials being a username and password of `admin:admin` we would execute -curl -v -u admin:admin -X POST -F 'data=@./lib.zip' localhost:19002/admin/udf/udfs/testlib +curl -v -u admin:admin -X POST -F 'data=@./lib.zip' localhost:19004/admin/udf/udfs/testlib Any response other than `200` indicates an error in deployment. In the AsterixDB source release, we provide several sample UDFs that you can try out. You need to build the AsterixDB source to get the compiled UDF package. It can be found under -the `asterixdb-external` sub-project. Assuming that these UDFs have been installed into the `testlib` library in`udfs` dataverse, +the `asterix-external-data` sub-project under the path +`asterixdb/asterix-external-data/src/test/java/org/apache/asterix/external/library`. +After compilation, the UDFs will be packed in a zip file at +`asterixdb/asterix-external-data/target/asterix-external-data-$VERSION-testlib.zip` +which you can use to upload to the AsterixDB cluster. + +Assuming that these UDFs have been installed into the `testlib` library in`udfs` dataverse, here is an example that uses the sample UDF `mysum` to compute the sum of two input integers. USE udfs;
[asterixdb] 03/38: [NO ISSUE][HYR][MISC] Clear interrupted state on halt
This is an automated email from the ASF dual-hosted git repository. imaxon pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/asterixdb.git commit 33647b6519671b5074c158f822193ce2db41b7a0 Author: Michael Blow AuthorDate: Thu Apr 15 18:08:02 2021 -0400 [NO ISSUE][HYR][MISC] Clear interrupted state on halt Clear the calling thread's interrupted state on halt, to enable collection of thread dump at exit. Should the thread be interrupted again before the thread dump has completed, the collection will be aborted and the halt will proceed. Change-Id: I29584c059bdae32baff67ba6f6d435cc8458b107 Reviewed-on: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/11083 Reviewed-by: Murtadha Hubail Reviewed-by: Michael Blow Integration-Tests: Jenkins Tested-by: Michael Blow --- .../src/main/java/org/apache/hyracks/util/ExitUtil.java | 6 -- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/hyracks-fullstack/hyracks/hyracks-util/src/main/java/org/apache/hyracks/util/ExitUtil.java b/hyracks-fullstack/hyracks/hyracks-util/src/main/java/org/apache/hyracks/util/ExitUtil.java index abd9fda..f4c4183 100644 --- a/hyracks-fullstack/hyracks/hyracks-util/src/main/java/org/apache/hyracks/util/ExitUtil.java +++ b/hyracks-fullstack/hyracks/hyracks-util/src/main/java/org/apache/hyracks/util/ExitUtil.java @@ -103,9 +103,11 @@ public class ExitUtil { public static synchronized void halt(int status, Level logLevel) { try { +boolean interrupted = Thread.interrupted(); +LOGGER.log(logLevel, "JVM halting with status {} (halting thread {}, interrupted {})", status, +Thread.currentThread(), interrupted); Future future = haltThreadDumpExecutor.submit(() -> { -LOGGER.log(logLevel, "JVM halting with status {}; thread dump at halt: {}", status, -ThreadDumpUtil.takeDumpString()); +LOGGER.log(logLevel, "Thread dump at halt: {}", ThreadDumpUtil.takeDumpString()); // try to give time for the log to be emitted... LogManager.shutdown(); });
[asterixdb] branch master updated (e75c8ef -> e7974ce)
This is an automated email from the ASF dual-hosted git repository. imaxon pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/asterixdb.git. from e75c8ef [ASTERIXDB-2892][IDX] Fixing bug w/ range queries for array indexes. new 40be6e7 [NO ISSUE][HYR] Increase thread name brevity new 94b1306 [NO ISSUE][REP] Account for SSL socket data transfer behavior new 33647b6 [NO ISSUE][HYR][MISC] Clear interrupted state on halt new c70687f [ASTERIXDB-2874][EXT] S3 external datasets: add support to anonymous authentication new ef5467b [NO ISSUE][HYR][MISC] Elimate hashtable for thread name save/restore new 605b115 [NO ISSUE][EXT] Provide direct access to controller stat primitives new 2323a4c [ASTERIXDB-2886][COMP] Fix RemoveRedundantVariablesRule new d5eb016 [NO ISSUE][OTH] Add operator field to subplan json plan new d5edad2 [NO ISSUE][MISC] Misc / minor diagnostic improvement new 0d88d11 [NO ISSUE][API] Make the AbstractServlet look for /* wild card new 0da78d6 [NO ISSUE][OTH] Add request parameter to allow for immediate execution new 7dfb0b3 [NO ISSUE][MISC] Ensure safe serialVersionUIDs for anon. inners new 902f63c [NO ISSUE][HYR][HTTP] Set thread name on interruptable stream handler new 61aad74 [NO ISSUE][FUN] Remove decode_dataverse_display_name() new 82f1931 [NO ISSUE][OTH] Add util method to get request http scheme new 3856cb3 [NO ISSUE] Update third-party dependencies new 2fe0e14 Merge tracking branch 'gerrit/stabilization-f69489' new 6abbeea Merge branch 'gerrit/stabilization-5949a1cb71' new 8c67f9c Merge branch 'gerrit/mad-hatter' new 404e382 [ASTERIXDB-2883][COMP] Improve null handling in UDF calls new d28f9ad [NO ISSUE][MISC] Avoid URI.getHost/Port where hostnames may contain underscore new 30c4a78 [NO ISSUE][OTH] Only order nodes in node group for new datasets new ec5195a [ASTERIXDB-2897][FUN] Fix unknown handling in PyUDF new 91f91d2 [NO ISSUE][EXT] Refactor JSONDataParser new 5d02b03 [NO ISSUE][*DB] Enable large parsing resources to be freed on memory pressure new a0d1fb7 [ASTERIXDB-2896] Increase UDF argument buffer size new ed56b86 [NO ISSUE][TX] Ensure lock manager computes valid indexes for hash table new b69e30e [NO ISSUE][MISC] Fix typo on active state log, += HyracksThrowingSupplier new 0e7e4bd [NO ISSUE][STO] Close datasets of flushed indexes after recovery new 86da3f8 [ASTERIXDB-2884] Compiler error with nested UDF calls new dec8426 [NO ISSUE][MISC] Preserve Error instances on retryUntilSuccessOrExhausted new d5de9eb [NO ISSUE] Fixed storage unit util false positives + added tests new 2782be4 Merge branch 'gerrit/mad-hatter' new eed4941 [ASTERIXDB-2894] Update UDF docs new 528ee18 [NO ISSUE][*DB][EXT] Fail with helpful error message on non-JSON object new 3b6982c [NO ISSUE][MISC] Improve parser error reporting new 3435172 [NO ISSUE][HYR][HTTP] http stream handler -> consumer / processor new e7974ce Merge branch 'cheshire-cat' The 38 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: .../active/message/ActiveManagerMessage.java | 2 +- .../ExtractBatchableExternalFunctionCallsRule.java | 12 +- ...duceDynamicTypeCastForExternalFunctionRule.java | 13 +- .../asterix/translator/IRequestParameters.java | 2 + .../asterix/translator/IStatementExecutor.java | 6 +- .../apache/asterix/api/common/APIFramework.java| 24 +- .../api/http/server/AbstractNCUdfServlet.java | 2 +- .../app/active/ActiveEntityEventsListener.java | 7 +- .../asterix/app/function/DumpIndexDatasource.java | 14 +- .../asterix/app/function/DumpIndexRewriter.java| 6 +- .../message/ExecuteStatementRequestMessage.java| 20 +- .../org/apache/asterix/app/nc/RecoveryManager.java | 20 +- .../asterix/app/nc/task/RetrieveLibrariesTask.java | 7 +- .../asterix/app/translator/QueryTranslator.java| 16 +- .../asterix/app/translator/RequestParameters.java | 17 + .../org/apache/asterix/utils/FeedOperations.java | 2 +- .../asterix/app/external/ExternalUDFLibrarian.java | 3 +- .../apache/asterix/test/common/TestExecutor.java | 9 +- .../test/dataflow/LSMFlushRecoveryTest.java| 12 +- .../s3/anonymous_no_auth/test.000.ddl.sqlpp} | 18 +- .../s3/anonymous_no_auth/test.001.ddl.sqlpp} | 18 +- .../s3/anonymous_no_auth/test.002.ddl.sqlpp} | 17 +- .../test.099.ddl.sqlpp | 0 query.sqlpp => type_validation.10.query.sqlpp} | 7 +- ...on.4.ddl.s
[asterixdb] 11/38: [NO ISSUE][OTH] Add request parameter to allow for immediate execution
This is an automated email from the ASF dual-hosted git repository. imaxon pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/asterixdb.git commit 0da78d6f840836af98004ad129fcb10b6d166e72 Author: Ali Alsuliman AuthorDate: Thu Apr 22 18:29:22 2021 +0300 [NO ISSUE][OTH] Add request parameter to allow for immediate execution - user model changes: no - storage format changes: no - interface changes: yes Details: Add a request parameter to allow certain queries to run immediately. Change-Id: I60208c07200326a4957e908d8e7c9dcdb7bc3204 Reviewed-on: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/11145 Integration-Tests: Jenkins Tested-by: Jenkins Reviewed-by: Ali Alsuliman Reviewed-by: Michael Blow --- .../asterix/translator/IRequestParameters.java | 2 ++ .../asterix/translator/IStatementExecutor.java | 6 +- .../apache/asterix/api/common/APIFramework.java| 24 +- .../message/ExecuteStatementRequestMessage.java| 20 +++--- .../asterix/app/translator/QueryTranslator.java| 16 +++ .../asterix/app/translator/RequestParameters.java | 17 +++ .../org/apache/asterix/utils/FeedOperations.java | 2 +- 7 files changed, 64 insertions(+), 23 deletions(-) diff --git a/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/translator/IRequestParameters.java b/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/translator/IRequestParameters.java index 4ad1040..5c743ee 100644 --- a/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/translator/IRequestParameters.java +++ b/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/translator/IRequestParameters.java @@ -73,4 +73,6 @@ public interface IRequestParameters extends ICommonRequestParameters { * inconsistent. */ boolean isForceDropDataset(); + +boolean isSkipAdmissionPolicy(); } diff --git a/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/translator/IStatementExecutor.java b/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/translator/IStatementExecutor.java index 083fa83..d60b791 100644 --- a/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/translator/IStatementExecutor.java +++ b/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/translator/IStatementExecutor.java @@ -273,6 +273,9 @@ public interface IStatementExecutor { * @param statementParameters *Statement parameters * @param statementRewriter + *The statement rewriter + * @param requestParameters + *The request parameters * @return the compiled {@code JobSpecification} * @throws AsterixException * @throws RemoteException @@ -281,7 +284,8 @@ public interface IStatementExecutor { */ JobSpecification rewriteCompileQuery(IClusterInfoCollector clusterInfoCollector, MetadataProvider metadataProvider, Query query, ICompiledDmlStatement dmlStatement, Map statementParameters, -IStatementRewriter statementRewriter) throws RemoteException, AlgebricksException, ACIDException; +IStatementRewriter statementRewriter, IRequestParameters requestParameters) +throws RemoteException, AlgebricksException, ACIDException; /** * returns the active dataverse for an entity or a statement diff --git a/asterixdb/asterix-app/src/main/java/org/apache/asterix/api/common/APIFramework.java b/asterixdb/asterix-app/src/main/java/org/apache/asterix/api/common/APIFramework.java index b3c7f54..920b0d9 100644 --- a/asterixdb/asterix-app/src/main/java/org/apache/asterix/api/common/APIFramework.java +++ b/asterixdb/asterix-app/src/main/java/org/apache/asterix/api/common/APIFramework.java @@ -77,6 +77,7 @@ import org.apache.asterix.optimizer.rules.util.EquivalenceClassUtils; import org.apache.asterix.runtime.job.listener.JobEventListenerFactory; import org.apache.asterix.translator.CompiledStatements.ICompiledDmlStatement; import org.apache.asterix.translator.ExecutionPlans; +import org.apache.asterix.translator.IRequestParameters; import org.apache.asterix.translator.SessionConfig; import org.apache.asterix.translator.SessionOutput; import org.apache.asterix.translator.SqlppExpressionToPlanTranslator; @@ -205,7 +206,8 @@ public class APIFramework { public JobSpecification compileQuery(IClusterInfoCollector clusterInfoCollector, MetadataProvider metadataProvider, Query query, int varCounter, String outputDatasetName, SessionOutput output, ICompiledDmlStatement statement, Map externalVars, IResponsePrinter printer, -IWarningCollector warningCollector) throws AlgebricksException, ACIDException { +IWarningCollector warningCollector, IRequestParameters requestParameters) +throws AlgebricksException, ACIDException
[asterixdb] 26/38: [ASTERIXDB-2896] Increase UDF argument buffer size
This is an automated email from the ASF dual-hosted git repository. imaxon pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/asterixdb.git commit a0d1fb7acb588f4eab940501deee738789e9af50 Author: Ian Maxon AuthorDate: Sun May 2 11:23:41 2021 -0700 [ASTERIXDB-2896] Increase UDF argument buffer size -user model changes: no -storage format changes: no -interface changes: yes Details: Bump buffer sizes in Python IPC to 1MB for individual arguments, and in the case of batching, match the buffer size of the Hyracks IPC layer for deserialization. Change-Id: If847ac3b09406d1e9e6a976a7e0193b6e81bcc8b Reviewed-on: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/11243 Reviewed-by: Ian Maxon Reviewed-by: Michael Blow Integration-Tests: Jenkins Tested-by: Jenkins Contrib: Ian Maxon --- .../ExternalScalarPythonFunctionEvaluator.java| 9 ++--- .../operators/ExternalAssignBatchRuntimeFactory.java | 19 --- .../asterix/external/util/ExternalDataUtils.java | 19 +++ 3 files changed, 41 insertions(+), 6 deletions(-) diff --git a/asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/library/ExternalScalarPythonFunctionEvaluator.java b/asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/library/ExternalScalarPythonFunctionEvaluator.java index eb87399..7c860a2 100644 --- a/asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/library/ExternalScalarPythonFunctionEvaluator.java +++ b/asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/library/ExternalScalarPythonFunctionEvaluator.java @@ -29,6 +29,7 @@ import java.nio.ByteBuffer; import org.apache.asterix.common.exceptions.AsterixException; import org.apache.asterix.common.exceptions.ErrorCode; import org.apache.asterix.external.library.msgpack.MessageUnpackerToADM; +import org.apache.asterix.external.util.ExternalDataUtils; import org.apache.asterix.om.functions.IExternalFunctionInfo; import org.apache.asterix.om.types.ATypeTag; import org.apache.asterix.om.types.IAType; @@ -77,9 +78,11 @@ class ExternalScalarPythonFunctionEvaluator extends ExternalScalarFunctionEvalua for (int i = 0; i < argValues.length; i++) { argValues[i] = VoidPointable.FACTORY.createPointable(); } -//TODO: these should be dynamic -this.argHolder = ByteBuffer.wrap(new byte[Short.MAX_VALUE * 2]); -this.outputWrapper = ByteBuffer.wrap(new byte[Short.MAX_VALUE * 2]); +//TODO: these should be dynamic. this static size picking is a temporary bodge until this works like +// v-size frames do or these construction buffers are removed entirely +int maxArgSz = ExternalDataUtils.getArgBufferSize(); +this.argHolder = ByteBuffer.wrap(new byte[maxArgSz]); +this.outputWrapper = ByteBuffer.wrap(new byte[maxArgSz]); this.evaluatorContext = ctx; this.sourceLocation = sourceLoc; this.unpackerInput = new ArrayBufferInput(new byte[0]); diff --git a/asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/operators/ExternalAssignBatchRuntimeFactory.java b/asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/operators/ExternalAssignBatchRuntimeFactory.java index 39e480a..593bac6 100644 --- a/asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/operators/ExternalAssignBatchRuntimeFactory.java +++ b/asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/operators/ExternalAssignBatchRuntimeFactory.java @@ -36,6 +36,7 @@ import org.apache.asterix.common.exceptions.RuntimeDataException; import org.apache.asterix.external.library.PythonLibraryEvaluator; import org.apache.asterix.external.library.PythonLibraryEvaluatorFactory; import org.apache.asterix.external.library.msgpack.MessageUnpackerToADM; +import org.apache.asterix.external.util.ExternalDataUtils; import org.apache.asterix.om.functions.IExternalFunctionDescriptor; import org.apache.asterix.om.types.ATypeTag; import org.apache.hyracks.algebricks.common.utils.Pair; @@ -61,6 +62,8 @@ public final class ExternalAssignBatchRuntimeFactory extends AbstractOneInputOne private final IExternalFunctionDescriptor[] fnDescs; private final int[][] fnArgColumns; +private int rpcBufferSize; + public ExternalAssignBatchRuntimeFactory(int[] outColumns, IExternalFunctionDescriptor[] fnDescs, int[][] fnArgColumns, int[] projectionList) { super(projectionList); @@ -73,6 +76,9 @@ public final class ExternalAssignBatchRuntimeFactory extends AbstractOneInputOne public AbstractOneInputOneOutputPushRuntime createOneOutputPushRuntime(IHyracksTaskContext ctx) { final int[] projectionToOutColumns = new int[projectionList.length]; +//this is a tempor
[asterixdb] 21/38: [NO ISSUE][MISC] Avoid URI.getHost/Port where hostnames may contain underscore
This is an automated email from the ASF dual-hosted git repository. imaxon pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/asterixdb.git commit d28f9addb8d4e0ea80bb3c88f828aa60d5e3783f Author: Michael Blow AuthorDate: Mon May 3 16:48:08 2021 -0400 [NO ISSUE][MISC] Avoid URI.getHost/Port where hostnames may contain underscore Change-Id: Iff58db84cc4d11609e7d7c459003a8a50f058ac6 Reviewed-on: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/11305 Reviewed-by: Michael Blow Reviewed-by: Ian Maxon Integration-Tests: Jenkins Tested-by: Jenkins --- .../asterix/app/nc/task/RetrieveLibrariesTask.java | 7 +-- .../asterix/app/external/ExternalUDFLibrarian.java | 3 ++- .../apache/asterix/test/common/TestExecutor.java | 9 +++-- .../java/org/apache/hyracks/util/NetworkUtil.java | 22 ++ .../org/apache/hyracks/util/NetworkUtilTest.java | 11 +++ 5 files changed, 43 insertions(+), 9 deletions(-) diff --git a/asterixdb/asterix-app/src/main/java/org/apache/asterix/app/nc/task/RetrieveLibrariesTask.java b/asterixdb/asterix-app/src/main/java/org/apache/asterix/app/nc/task/RetrieveLibrariesTask.java index 18d303c..715e626 100644 --- a/asterixdb/asterix-app/src/main/java/org/apache/asterix/app/nc/task/RetrieveLibrariesTask.java +++ b/asterixdb/asterix-app/src/main/java/org/apache/asterix/app/nc/task/RetrieveLibrariesTask.java @@ -33,12 +33,14 @@ import org.apache.asterix.common.api.INCLifecycleTask; import org.apache.asterix.common.api.INcApplicationContext; import org.apache.asterix.common.library.ILibraryManager; import org.apache.http.client.utils.URIBuilder; +import org.apache.http.client.utils.URIUtils; import org.apache.hyracks.algebricks.common.utils.Pair; import org.apache.hyracks.api.control.CcId; import org.apache.hyracks.api.exceptions.ErrorCode; import org.apache.hyracks.api.exceptions.HyracksDataException; import org.apache.hyracks.api.io.FileReference; import org.apache.hyracks.api.service.IControllerService; +import org.apache.hyracks.util.NetworkUtil; import org.apache.hyracks.util.file.FileUtil; import org.apache.logging.log4j.LogManager; import org.apache.logging.log4j.Logger; @@ -51,7 +53,7 @@ public class RetrieveLibrariesTask implements INCLifecycleTask { public RetrieveLibrariesTask(List> nodes) { this.nodes = nodes; -if (nodes.size() <= 0) { +if (nodes.isEmpty()) { throw new IllegalArgumentException("No nodes specified to retrieve from"); } } @@ -62,7 +64,8 @@ public class RetrieveLibrariesTask implements INCLifecycleTask { boolean success = false; for (Pair referenceNode : nodes) { try { -LOGGER.info("Retrieving UDFs from " + referenceNode.getFirst().getHost()); +LOGGER.info("Retrieving UDFs from " ++ NetworkUtil.toHostPort(URIUtils.extractHost(referenceNode.getFirst(; retrieveLibrary(referenceNode.getFirst(), referenceNode.getSecond(), appContext); success = true; break; diff --git a/asterixdb/asterix-app/src/test/java/org/apache/asterix/app/external/ExternalUDFLibrarian.java b/asterixdb/asterix-app/src/test/java/org/apache/asterix/app/external/ExternalUDFLibrarian.java index 2450025..e32f8ea 100644 --- a/asterixdb/asterix-app/src/test/java/org/apache/asterix/app/external/ExternalUDFLibrarian.java +++ b/asterixdb/asterix-app/src/test/java/org/apache/asterix/app/external/ExternalUDFLibrarian.java @@ -33,6 +33,7 @@ import org.apache.http.client.HttpClient; import org.apache.http.client.methods.HttpDelete; import org.apache.http.client.methods.HttpPost; import org.apache.http.client.protocol.HttpClientContext; +import org.apache.http.client.utils.URIUtils; import org.apache.http.entity.ContentType; import org.apache.http.entity.mime.HttpMultipartMode; import org.apache.http.entity.mime.MultipartEntityBuilder; @@ -78,7 +79,7 @@ public class ExternalUDFLibrarian implements IExternalUDFLibrarian { private HttpClientContext createHttpClientContext(URI path, Pair credentials) { HttpClientContext hcCtx = HttpClientContext.create(); -HttpHost h = new HttpHost(path.getHost(), path.getPort(), "http"); +HttpHost h = URIUtils.extractHost(path); CredentialsProvider cp = new BasicCredentialsProvider(); cp.setCredentials(AuthScope.ANY, new UsernamePasswordCredentials(credentials.first, credentials.second)); hcCtx.setCredentialsProvider(cp); diff --git a/asterixdb/asterix-app/src/test/java/org/apache/asterix/test/common/TestExecutor.java b/asterixdb/asterix-app/src/test/java/org/apache/asterix/test/common/TestExecutor.java index 28b7fb3..40946d3 100644 --- a/asterixdb/asterix-app/src/test/java/org/apache/asterix/test/common/TestExecutor.java +++ b/asterixd
[asterixdb] 17/38: Merge tracking branch 'gerrit/stabilization-f69489'
This is an automated email from the ASF dual-hosted git repository. imaxon pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/asterixdb.git commit 2fe0e14266c32fbb2379b859acb08d095f109255 Merge: f9f467a 3856cb3 Author: Michael Blow AuthorDate: Sat May 1 15:00:00 2021 -0400 Merge tracking branch 'gerrit/stabilization-f69489' Change-Id: Ide3d9da3f2c2518afd0b488f6db5b2e3377ee8a7 asterixdb/asterix-server/pom.xml | 4 ++-- asterixdb/pom.xml | 4 ++-- .../main/appended-resources/supplemental-models.xml| 8 ...tent.com_netty_netty_netty-4.1.63.Final_NOTICE.txt} | 0 hyracks-fullstack/pom.xml | 18 +- 5 files changed, 17 insertions(+), 17 deletions(-) diff --cc asterixdb/pom.xml index f57d71e,ed486fd..36b4581 --- a/asterixdb/pom.xml +++ b/asterixdb/pom.xml @@@ -73,15 -71,13 +73,15 @@@ ${global.itest.includes} ${global.itest.excludes} +compile +process-classes -0.3.4.2-SNAPSHOT -0.3.4.2-SNAPSHOT +0.3.5-SNAPSHOT +0.3.5-SNAPSHOT 2.8.5 0.7.6.201602180812 - 2.13.3 + 2.14.1 Apache AsterixDB - ${project.name} https://asterixdb.apache.org/
[asterixdb] 31/38: [ASTERIXDB-2884] Compiler error with nested UDF calls
This is an automated email from the ASF dual-hosted git repository. imaxon pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/asterixdb.git commit 86da3f820f9e59fe5382dc42c54b0828cc9aecb8 Author: Dmitry Lychagin AuthorDate: Wed May 12 22:25:50 2021 -0700 [ASTERIXDB-2884] Compiler error with nested UDF calls - user model changes: no - storage format changes: no - interface changes: no Details: - Fix invalid plan produced by ExtractBatchableExternalFunctionCallsRule - IntroduceDynamicTypeCastForExternalFunctionRule should not introduce redundant cast if function argument type is ANY - Add testcase Change-Id: I472aa8f828cd548d14d7bda9893984cc4bf25737 Reviewed-on: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/11445 Integration-Tests: Jenkins Tested-by: Jenkins Reviewed-by: Michael Blow --- .../rules/ExtractBatchableExternalFunctionCallsRule.java | 12 +--- .../IntroduceDynamicTypeCastForExternalFunctionRule.java | 4 .../{mysentiment.0.ddl.sqlpp => toplevel_fn.0.ddl.sqlpp} | 0 .../{mysentiment.1.lib.sqlpp => toplevel_fn.1.lib.sqlpp} | 0 .../{mysentiment.2.ddl.sqlpp => toplevel_fn.2.ddl.sqlpp} | 0 .../{mysentiment.3.query.sqlpp => toplevel_fn.3.query.sqlpp} | 0 .../{mysentiment.3.query.sqlpp => toplevel_fn.4.query.sqlpp} | 7 ++- .../{mysentiment.4.ddl.sqlpp => toplevel_fn.5.ddl.sqlpp} | 0 .../toplevel_fn/{toplevel_fn.1.adm => toplevel_fn.3.adm} | 0 .../results/external-library/toplevel_fn/toplevel_fn.4.adm | 1 + 10 files changed, 16 insertions(+), 8 deletions(-) diff --git a/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/ExtractBatchableExternalFunctionCallsRule.java b/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/ExtractBatchableExternalFunctionCallsRule.java index 1e5d805..54a1bb7 100644 --- a/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/ExtractBatchableExternalFunctionCallsRule.java +++ b/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/ExtractBatchableExternalFunctionCallsRule.java @@ -190,13 +190,11 @@ public class ExtractBatchableExternalFunctionCallsRule implements IAlgebraicRewr if (!assignVars.isEmpty()) { usedVarList.clear(); expr.getUsedVariables(usedVarList); -for (int i = 0, ln = assignVars.size(); i < ln; i++) { -List candidateVarList = assignVars.get(i); -if (OperatorPropertiesUtil.disjoint(candidateVarList, usedVarList)) { -assignVarList = candidateVarList; -assignExprList = assignExprs.get(i); -break; -} +int candidateVarListIdx = assignVars.size() - 1; +List candidateVarList = assignVars.get(candidateVarListIdx); +if (OperatorPropertiesUtil.disjoint(candidateVarList, usedVarList)) { +assignVarList = candidateVarList; +assignExprList = assignExprs.get(candidateVarListIdx); } } diff --git a/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/IntroduceDynamicTypeCastForExternalFunctionRule.java b/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/IntroduceDynamicTypeCastForExternalFunctionRule.java index 2c5a07d..22d87ac 100644 --- a/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/IntroduceDynamicTypeCastForExternalFunctionRule.java +++ b/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/IntroduceDynamicTypeCastForExternalFunctionRule.java @@ -22,6 +22,7 @@ package org.apache.asterix.optimizer.rules; import org.apache.asterix.om.functions.BuiltinFunctions; import org.apache.asterix.om.functions.ExternalFunctionInfo; import org.apache.asterix.om.typecomputer.base.TypeCastUtils; +import org.apache.asterix.om.typecomputer.impl.TypeComputeUtils; import org.apache.asterix.om.types.ARecordType; import org.apache.asterix.om.types.ATypeTag; import org.apache.asterix.om.types.AUnionType; @@ -82,6 +83,9 @@ public class IntroduceDynamicTypeCastForExternalFunctionRule implements IAlgebra if (reqArgType.getTypeTag() == ATypeTag.OBJECT) { castFlag = !IntroduceDynamicTypeCastRule.compatible((ARecordType) reqArgType, inputType, argExpr.getValue().getSourceLocation()); +} else if (reqArgType.getTypeTag() == ATypeTag.ANY) { +IAType inputPrimeType = TypeComputeUtils.getActualType(inputType); +castFlag = inputPrimeType.getTypeTag().isDerivedType(); } else { castFlag = !reqArgType.equals(inputType); } diff --git a/asterixdb/asteri
[asterixdb] 18/38: Merge branch 'gerrit/stabilization-5949a1cb71'
This is an automated email from the ASF dual-hosted git repository. imaxon pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/asterixdb.git commit 6abbeeaff95cd6a7d1ce27df7b94fdff9e9c5aa1 Merge: 3ea8ef1 2fe0e14 Author: Michael Blow AuthorDate: Sun May 2 09:27:39 2021 -0400 Merge branch 'gerrit/stabilization-5949a1cb71' Change-Id: Id8c5f6de8ce3738a89994e87c3fc6d0ca7150d8e asterixdb/asterix-server/pom.xml | 4 ++-- asterixdb/pom.xml | 4 ++-- .../main/appended-resources/supplemental-models.xml| 8 ...tent.com_netty_netty_netty-4.1.63.Final_NOTICE.txt} | 0 hyracks-fullstack/pom.xml | 18 +- 5 files changed, 17 insertions(+), 17 deletions(-) diff --cc asterixdb/asterix-server/pom.xml index ba0abdd,5a279d0..266866e --- a/asterixdb/asterix-server/pom.xml +++ b/asterixdb/asterix-server/pom.xml @@@ -169,38 -169,9 +169,38 @@@ https://raw.githubusercontent.com/mojohaus/appassembler/appassembler-2.0.0/LICENSE.txt - io.netty:netty-all:4.1.59.Final - https://raw.githubusercontent.com/netty/netty/netty-4.1.59.Final/NOTICE.txt + io.netty:netty-all:4.1.63.Final + https://raw.githubusercontent.com/netty/netty/netty-4.1.63.Final/NOTICE.txt + + org.reactivestreams:reactive-streams:1.0.2 + https://raw.githubusercontent.com/reactive-streams/reactive-streams-jvm/v1.0.2/COPYING.txt + https://raw.githubusercontent.com/reactive-streams/reactive-streams-jvm/v1.0.2/LICENSE.txt + + + +software.amazon.awssdk:sdk-core:2.10.83 +software.amazon.awssdk:aws-core:2.10.83 +software.amazon.awssdk:auth:2.10.83 +software.amazon.awssdk:arns:2.10.83 +software.amazon.awssdk:annotations:2.10.83 +software.amazon.awssdk:utils:2.10.83 +software.amazon.awssdk:aws-query-protocol:2.10.83 +software.amazon.awssdk:profiles:2.10.83 +software.amazon.awssdk:protocol-core:2.10.83 +software.amazon.awssdk:s3:2.10.83 +software.amazon.awssdk:netty-nio-client:2.10.83 +software.amazon.awssdk:apache-client:2.10.83 +software.amazon.awssdk:aws-xml-protocol:2.10.83 +software.amazon.awssdk:regions:2.10.83 +software.amazon.awssdk:http-client-spi:2.10.83 + + https://raw.githubusercontent.com/aws/aws-sdk-java-v2/2.10.83/NOTICE.txt + + + software.amazon.eventstream:eventstream:1.0.1 + https://raw.githubusercontent.com/awslabs/aws-eventstream-java/7be2dd80e12f8835674c8ffb0f4a2efb64c7b585/NOTICE + diff --cc asterixdb/pom.xml index 5b7e6c9,36b4581..43e0dfd --- a/asterixdb/pom.xml +++ b/asterixdb/pom.xml @@@ -81,8 -81,7 +81,8 @@@ 0.3.5-SNAPSHOT 2.8.5 0.7.6.201602180812 - 2.13.3 + 2.14.1 +2.10.83 Apache AsterixDB - ${project.name} https://asterixdb.apache.org/
[asterixdb] 34/38: [ASTERIXDB-2894] Update UDF docs
This is an automated email from the ASF dual-hosted git repository. imaxon pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/asterixdb.git commit eed4941d2d8dfb3395ae17a36c349ecac12c9cce Author: Ian Maxon AuthorDate: Thu Apr 29 12:46:44 2021 -0700 [ASTERIXDB-2894] Update UDF docs - user model changes: no - storage format changes: no - interface changes: no Details: - Update API examples to include type - Include details about typing and execution model Change-Id: Id9780d72960f9094c29f7f5766185782069fe7cf Reviewed-on: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/11225 Reviewed-by: Ian Maxon Reviewed-by: Dmitry Lychagin Integration-Tests: Jenkins Tested-by: Jenkins --- .../src/main/user-defined_function/udf.md | 63 +- 1 file changed, 50 insertions(+), 13 deletions(-) diff --git a/asterixdb/asterix-doc/src/main/user-defined_function/udf.md b/asterixdb/asterix-doc/src/main/user-defined_function/udf.md index 7ca23bb..655113b 100644 --- a/asterixdb/asterix-doc/src/main/user-defined_function/udf.md +++ b/asterixdb/asterix-doc/src/main/user-defined_function/udf.md @@ -19,7 +19,7 @@ ## Introduction -Apache AsterixDB supports three languages for writing user-defined functions (UDFs): SQL++, Java and Python +Apache AsterixDB supports three languages for writing user-defined functions (UDFs): SQL++, Java, and Python A user can encapsulate data processing logic into a UDF and invoke it later repeatedly. For SQL++ functions, a user can refer to [SQL++ Functions](sqlpp/manual.html#Functions) for their usages. This document will focus on UDFs in languages other than SQL++ @@ -27,8 +27,10 @@ for their usages. This document will focus on UDFs in languages other than SQL++ ## Endpoints and Authentication -The UDF endpoint is not enabled by default until authentication has been configured properly. To enable it, we -will need to set the path to the credential file and populate it with our username and password. +The UDF API endpoint used to deploy functions is not enabled by default until authentication has been configured properly. +Even if the endpoint is enabled, it is only accessible on the loopback interface on each NC to restrict access. + +To enable it, we need to set the path to the credential file and populate it with our username and password. The credential file is a simple `/etc/passwd` style text file with usernames and corresponding `bcrypt` hashed and salted passwords. You can populate this on your own if you would like, but the `asterixhelper` utility can write the entries as @@ -50,9 +52,7 @@ Now,restart the cluster if it was already started to allow the Cluster Controlle ## Installing a Java UDF Library To install a UDF package to the cluster, we need to send a Multipart Form-data HTTP request to the `/admin/udf` endpoint -of the CC at the normal API port (`19002` by default). The request should use HTTP Basic authentication. This means your -credentials will *not* be obfuscated or encrypted *in any way*, so submit to this endpoint over localhost or a network -where you know your traffic is safe from eavesdropping. Any suitable tool will do, but for the example here I will use +of the CC at the normal API port (`19004` by default). Any suitable tool will do, but for the example here I will use `curl` which is widely available. For example, to install a library with the following criteria: @@ -65,7 +65,7 @@ For example, to install a library with the following criteria: we would execute -curl -v -u admin:admin -X POST -F 'data=@./lib.zip' localhost:19002/admin/udf/udfs/testlib +curl -v -u admin:admin -X POST -F 'data=@./lib.zip' -F 'type=java' localhost:19004/admin/udf/udfs/testlib Any response other than `200` indicates an error in deployment. @@ -119,7 +119,7 @@ scikit-learn here (our method is obviously better!), but it's just included as a Then, deploy it the same as the Java UDF was, with the library name `pylib` in `udfs` dataverse -curl -v -u admin:admin -X POST -F 'data=@./lib.pyz' localhost:19002/admin/udf/udfs/pylib +curl -v -u admin:admin -X POST -F 'data=@./lib.pyz' -F 'type=python' localhost:19002/admin/udf/udfs/pylib With the library deployed, we can define a function within it for use. For example, to expose the Python function `sentiment` in the module `sentiment_mod` in the class `sent_model`, the `CREATE FUNCTION` would be as follows @@ -131,14 +131,14 @@ With the library deployed, we can define a function within it for use. For examp AS "sentiment_mod", "sent_model.sentiment" AT pylib; By default, AsterixDB will treat all external functions as deterministic. It means the function must return the same -result for the same input, irrespective of when or how many tim
[asterixdb] 05/38: [NO ISSUE][HYR][MISC] Elimate hashtable for thread name save/restore
This is an automated email from the ASF dual-hosted git repository. imaxon pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/asterixdb.git commit ef5467be498a284dde179973a5d9f404cd0c95e9 Author: Michael Blow AuthorDate: Thu Apr 15 18:22:57 2021 -0400 [NO ISSUE][HYR][MISC] Elimate hashtable for thread name save/restore Change-Id: Ifeea72fb253601c214a18e8f4053d7f7d2b31135 Reviewed-on: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/11084 Reviewed-by: Michael Blow Reviewed-by: Ian Maxon Integration-Tests: Jenkins Tested-by: Jenkins --- .../hyracks/util/MaintainedThreadNameExecutorService.java| 12 +++- 1 file changed, 3 insertions(+), 9 deletions(-) diff --git a/hyracks-fullstack/hyracks/hyracks-util/src/main/java/org/apache/hyracks/util/MaintainedThreadNameExecutorService.java b/hyracks-fullstack/hyracks/hyracks-util/src/main/java/org/apache/hyracks/util/MaintainedThreadNameExecutorService.java index a9ebb50..9adac04 100644 --- a/hyracks-fullstack/hyracks/hyracks-util/src/main/java/org/apache/hyracks/util/MaintainedThreadNameExecutorService.java +++ b/hyracks-fullstack/hyracks/hyracks-util/src/main/java/org/apache/hyracks/util/MaintainedThreadNameExecutorService.java @@ -18,8 +18,6 @@ */ package org.apache.hyracks.util; -import java.util.Map; -import java.util.concurrent.ConcurrentHashMap; import java.util.concurrent.ExecutorService; import java.util.concurrent.SynchronousQueue; import java.util.concurrent.ThreadFactory; @@ -28,7 +26,7 @@ import java.util.concurrent.TimeUnit; public class MaintainedThreadNameExecutorService extends ThreadPoolExecutor { -private final Map threadNames = new ConcurrentHashMap<>(); +private static final ThreadLocal savedName = new ThreadLocal<>(); private MaintainedThreadNameExecutorService(ThreadFactory threadFactory) { super(0, Integer.MAX_VALUE, 60L, TimeUnit.SECONDS, new SynchronousQueue<>(), threadFactory); @@ -40,17 +38,13 @@ public class MaintainedThreadNameExecutorService extends ThreadPoolExecutor { @Override protected void beforeExecute(Thread t, Runnable r) { -threadNames.put(t, t.getName()); +savedName.set(t.getName()); super.beforeExecute(t, r); } @Override protected void afterExecute(Runnable r, Throwable t) { super.afterExecute(r, t); -final Thread thread = Thread.currentThread(); -final String originalThreadName = threadNames.remove(thread); -if (originalThreadName != null) { -thread.setName(originalThreadName); -} +Thread.currentThread().setName(savedName.get()); } }
[asterixdb] 04/38: [ASTERIXDB-2874][EXT] S3 external datasets: add support to anonymous authentication
This is an automated email from the ASF dual-hosted git repository. imaxon pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/asterixdb.git commit c70687f57f1bf93727b746a6bb787e374dfec0f4 Author: Hussain Towaileb AuthorDate: Thu Apr 15 17:47:31 2021 +0300 [ASTERIXDB-2874][EXT] S3 external datasets: add support to anonymous authentication - user model changes: no - storage format changes: no - interface changes: no Details: - Added support to creating external datasets with no credentials (anonymous). Change-Id: I9b0b83ac51b4c262b52d657f5cdcf6c0f818451e Reviewed-on: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/11023 Reviewed-by: Hussain Towaileb Reviewed-by: Till Westmann Integration-Tests: Jenkins Tested-by: Jenkins --- .../s3/anonymous_no_auth/test.000.ddl.sqlpp| 36 + .../s3/anonymous_no_auth/test.001.ddl.sqlpp| 36 + .../s3/anonymous_no_auth/test.002.ddl.sqlpp| 35 + .../s3/anonymous_no_auth/test.099.ddl.sqlpp| 20 ++ .../runtimets/testsuite_external_dataset_s3.xml| 7 .../asterix/common/exceptions/ErrorCode.java | 1 + .../src/main/resources/asx_errormsg/en.properties | 1 + .../asterix/external/util/ExternalDataUtils.java | 45 +- 8 files changed, 172 insertions(+), 9 deletions(-) diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/external-dataset/s3/anonymous_no_auth/test.000.ddl.sqlpp b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/external-dataset/s3/anonymous_no_auth/test.000.ddl.sqlpp new file mode 100644 index 000..13cfe8a --- /dev/null +++ b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/external-dataset/s3/anonymous_no_auth/test.000.ddl.sqlpp @@ -0,0 +1,36 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ + +drop dataverse test if exists; +create dataverse test; +use test; + +drop type test if exists; +create type test as open { +}; + +drop dataset test if exists; +CREATE EXTERNAL DATASET test(test) USING S3 ( +("accessKeyId"="dummyAccessKey"), +("region"="us-west-2"), +("serviceEndpoint"="http://localhost:8001";), +("container"="playground"), +("definition"="json-data/reviews/single-line/json"), +("format"="json") +); \ No newline at end of file diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/external-dataset/s3/anonymous_no_auth/test.001.ddl.sqlpp b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/external-dataset/s3/anonymous_no_auth/test.001.ddl.sqlpp new file mode 100644 index 000..b8d0945 --- /dev/null +++ b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/external-dataset/s3/anonymous_no_auth/test.001.ddl.sqlpp @@ -0,0 +1,36 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ + +drop dataverse test if exists; +create dataverse test; +use test; + +drop type test if exists; +create type test as open { +}; + +drop dataset test if exists; +CREATE EXTERNAL DATASET test(test) USING S3 ( +("secretAccessKey"="dummySecretKey"), +("region"="us-west-2"), +("serviceEndpoint"="
[asterixdb] 35/38: [NO ISSUE][*DB][EXT] Fail with helpful error message on non-JSON object
This is an automated email from the ASF dual-hosted git repository. imaxon pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/asterixdb.git commit 528ee18a5a7d9c01aa1f3f94282fbff2063f688a Author: Michael Blow AuthorDate: Wed May 19 20:10:46 2021 -0400 [NO ISSUE][*DB][EXT] Fail with helpful error message on non-JSON object Change-Id: Ia096e108aa062e87a92f29ad3812f118390f411e Reviewed-on: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/11525 Integration-Tests: Jenkins Reviewed-by: Till Westmann Tested-by: Jenkins --- .../java/org/apache/asterix/common/exceptions/ErrorCode.java | 3 ++- .../src/main/resources/asx_errormsg/en.properties | 1 + .../asterix/external/parser/AbstractJsonDataParser.java | 11 --- .../asterix/external/parser/AbstractNestedDataParser.java | 2 +- .../org/apache/asterix/external/parser/JSONDataParser.java| 6 +- .../org/apache/asterix/external/parser/ParseException.java| 4 .../java/org/apache/asterix/external/parser/TweetParser.java | 2 +- .../java/org/apache/hyracks/api/util/ErrorMessageUtil.java| 11 +++ 8 files changed, 29 insertions(+), 11 deletions(-) diff --git a/asterixdb/asterix-common/src/main/java/org/apache/asterix/common/exceptions/ErrorCode.java b/asterixdb/asterix-common/src/main/java/org/apache/asterix/common/exceptions/ErrorCode.java index 7e32e0b..4078ea2 100644 --- a/asterixdb/asterix-common/src/main/java/org/apache/asterix/common/exceptions/ErrorCode.java +++ b/asterixdb/asterix-common/src/main/java/org/apache/asterix/common/exceptions/ErrorCode.java @@ -307,7 +307,7 @@ public enum ErrorCode implements IError { PARSER_ADM_DATA_PARSER_CAST_ERROR(3072), PARSER_ADM_DATA_PARSER_CONSTRUCTOR_MISSING_DESERIALIZER(3073), PARSER_ADM_DATA_PARSER_WRONG_INSTANCE(3074), -PARSER_TWEET_PARSER_CLOSED_FIELD_NULL(3075), +PARSER_EXT_DATA_PARSER_CLOSED_FIELD_NULL(3075), UTIL_FILE_SYSTEM_WATCHER_NO_FILES_FOUND(3076), UTIL_LOCAL_FILE_SYSTEM_UTILS_PATH_NOT_FOUND(3077), UTIL_HDFS_UTILS_CANNOT_OBTAIN_HDFS_SCHEDULER(3078), @@ -349,6 +349,7 @@ public enum ErrorCode implements IError { FAILED_TO_PARSE_MALFORMED_LOG_RECORD(3117), ACTIVE_ENTITY_NOT_RUNNING(3118), REQUIRED_PARAM_IF_PARAM_IS_PRESENT(3119), +PARSER_DATA_PARSER_UNEXPECTED_TOKEN(3120), // Lifecycle management errors DUPLICATE_PARTITION_ID(4000), diff --git a/asterixdb/asterix-common/src/main/resources/asx_errormsg/en.properties b/asterixdb/asterix-common/src/main/resources/asx_errormsg/en.properties index 345b9d8..18898bb 100644 --- a/asterixdb/asterix-common/src/main/resources/asx_errormsg/en.properties +++ b/asterixdb/asterix-common/src/main/resources/asx_errormsg/en.properties @@ -354,6 +354,7 @@ 3117 = Failed to parse record, malformed log record 3118 = Active Entity %1$s is not running (it is %2$s) 3119 = Parameter '%1$s' is required if '%2$s' is provided +3120 = Unexpected token %s: was expecting %s # Lifecycle management errors 4000 = Partition id %1$s for node %2$s already in use by node %3$s diff --git a/asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/parser/AbstractJsonDataParser.java b/asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/parser/AbstractJsonDataParser.java index a5d5a08..2bf0df4 100644 --- a/asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/parser/AbstractJsonDataParser.java +++ b/asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/parser/AbstractJsonDataParser.java @@ -212,7 +212,7 @@ public abstract class AbstractJsonDataParser extends AbstractNestedDataParser extends AbstractDataParser { protected void checkOptionalConstraints(ARecordType recordType, BitSet nullBitmap) throws RuntimeDataException { for (int i = 0; i < recordType.getFieldTypes().length; i++) { if (!nullBitmap.get(i) && !isMissableType(recordType.getFieldTypes()[i])) { -throw new RuntimeDataException(ErrorCode.PARSER_TWEET_PARSER_CLOSED_FIELD_NULL, +throw new RuntimeDataException(ErrorCode.PARSER_EXT_DATA_PARSER_CLOSED_FIELD_NULL, recordType.getFieldNames()[i]); } } diff --git a/asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/parser/JSONDataParser.java b/asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/parser/JSONDataParser.java index b4ec46e..b2cffa9 100644 --- a/asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/parser/JSONDataParser.java +++ b/asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/parser/JSONDataParser.java @@ -18,6 +18,8 @@ */ package org.apache.asterix.external.parser; +import static org.apache.asterix.common.exceptions.ErrorCode.PARSER_DATA_PAR
[asterixdb] 33/38: Merge branch 'gerrit/mad-hatter'
This is an automated email from the ASF dual-hosted git repository. imaxon pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/asterixdb.git commit 2782be4244396a3dbc4ddfd1dc58ffd27f786e8c Merge: dec8426 d5de9eb Author: Michael Blow AuthorDate: Wed May 19 07:35:35 2021 -0400 Merge branch 'gerrit/mad-hatter' Change-Id: I78240c264b37e3d5be3fc40ed1f2b50f20c8689c .../java/org/apache/hyracks/util/StorageUtil.java | 53 ++- .../org/apache/hyracks/util/StorageUnitTest.java | 79 ++ 2 files changed, 99 insertions(+), 33 deletions(-)
[asterixdb] 38/38: Merge branch 'cheshire-cat'
This is an automated email from the ASF dual-hosted git repository. imaxon pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/asterixdb.git commit e7974ce1adf788c93868736704ce3d755d51de22 Merge: e75c8ef 3435172 Author: Ian Maxon AuthorDate: Fri May 21 12:41:46 2021 -0700 Merge branch 'cheshire-cat' Change-Id: I72241c264b37ead5ae1fc40ed1f2b50f20c8689c .../active/message/ActiveManagerMessage.java | 2 +- .../ExtractBatchableExternalFunctionCallsRule.java | 12 +- ...duceDynamicTypeCastForExternalFunctionRule.java | 13 +- .../asterix/translator/IRequestParameters.java | 2 + .../asterix/translator/IStatementExecutor.java | 6 +- .../apache/asterix/api/common/APIFramework.java| 24 +- .../api/http/server/AbstractNCUdfServlet.java | 2 +- .../app/active/ActiveEntityEventsListener.java | 7 +- .../asterix/app/function/DumpIndexDatasource.java | 14 +- .../asterix/app/function/DumpIndexRewriter.java| 6 +- .../message/ExecuteStatementRequestMessage.java| 20 +- .../org/apache/asterix/app/nc/RecoveryManager.java | 20 +- .../asterix/app/nc/task/RetrieveLibrariesTask.java | 7 +- .../asterix/app/translator/QueryTranslator.java| 16 +- .../asterix/app/translator/RequestParameters.java | 17 + .../org/apache/asterix/utils/FeedOperations.java | 2 +- .../asterix/app/external/ExternalUDFLibrarian.java | 3 +- .../apache/asterix/test/common/TestExecutor.java | 9 +- .../test/dataflow/LSMFlushRecoveryTest.java| 12 +- .../s3/anonymous_no_auth/test.000.ddl.sqlpp} | 18 +- .../s3/anonymous_no_auth/test.001.ddl.sqlpp} | 18 +- .../s3/anonymous_no_auth/test.002.ddl.sqlpp} | 17 +- .../s3/anonymous_no_auth/test.099.ddl.sqlpp} | 4 +- query.sqlpp => type_validation.10.query.sqlpp} | 7 +- ...on.4.ddl.sqlpp => type_validation.11.ddl.sqlpp} | 0 .../type_validation.2.ddl.sqlpp| 4 +- .../type_validation.3.query.sqlpp | 3 +- ...3.query.sqlpp => type_validation.4.query.sqlpp} | 5 +- ...3.query.sqlpp => type_validation.5.query.sqlpp} | 5 +- ...3.query.sqlpp => type_validation.6.query.sqlpp} | 7 +- ...3.query.sqlpp => type_validation.7.query.sqlpp} | 3 +- ...3.query.sqlpp => type_validation.8.query.sqlpp} | 5 +- ...3.query.sqlpp => type_validation.9.query.sqlpp} | 5 +- ...ntiment.0.ddl.sqlpp => toplevel_fn.0.ddl.sqlpp} | 0 ...ntiment.1.lib.sqlpp => toplevel_fn.1.lib.sqlpp} | 0 ...ntiment.2.ddl.sqlpp => toplevel_fn.2.ddl.sqlpp} | 0 ...ent.3.query.sqlpp => toplevel_fn.3.query.sqlpp} | 0 ...ent.3.query.sqlpp => toplevel_fn.4.query.sqlpp} | 7 +- ...ntiment.4.ddl.sqlpp => toplevel_fn.5.ddl.sqlpp} | 0 .../type_name.0.ddl.sqlpp} | 0 .../type_name.1.lib.sqlpp} | 4 +- .../type_name.2.ddl.sqlpp} | 6 +- .../type_name.3.query.sqlpp} | 11 +- .../type_name.4.ddl.sqlpp} | 0 .../query-ASTERIXDB-2886.1.ddl.sqlpp} | 25 +- .../query-ASTERIXDB-2886.2.ddl.sqlpp} | 25 +- .../query-ASTERIXDB-2886.3.query.sqlpp}| 14 +- .../special_chars_2/special_chars_2.2.query.sqlpp | 1 - .../type_validation.1.adm | 2 +- .../type_validation.2.adm | 1 + .../type_validation.3.adm | 1 + .../type_validation.4.adm | 1 + .../type_validation.5.adm | 1 + .../type_validation.6.adm | 1 + .../type_validation.7.adm | 1 + .../type_validation.8.adm | 1 + .../{toplevel_fn.1.adm => toplevel_fn.3.adm} | 0 .../external-library/toplevel_fn/toplevel_fn.4.adm | 1 + .../external-library/type_name/type_name.3.adm | 1 + .../push-limit-to-primary-scan-select.11.adm | 4 +- .../push-limit-to-primary-scan.7.adm | 4 +- .../query-ASTERIXDB-2886.3.adm | 4 + .../special_chars_2/special_chars_2.2.adm | 12 +- .../results/union/union_opt_1/union_opt_1.11.adm | 8 +- ...stsuite_external_dataset_azure_blob_storage.xml | 10 +- .../runtimets/testsuite_external_dataset_s3.xml| 17 +- ...testsuite_external_dataset_s3_one_partition.xml | 20 +- .../resources/runtimets/testsuite_it_sqlpp.xml | 5 + .../test/resources/runtimets/testsuite_sqlpp.xml | 5 + .../common/api/IDatasetLifecycleManager.java | 9 + .../common/context/DatasetLifecycleManager.java| 11 + .../asterix/common/exceptions/ErrorCode.java | 4 +- .../common/transactions/IRecoveryManager.java | 11 - .../src/main/resources/asx_errormsg/en.properties | 2 + .../src/main/user-defined_function/udf.md | 63 ++- .
[asterixdb] 37/38: [NO ISSUE][HYR][HTTP] http stream handler -> consumer / processor
This is an automated email from the ASF dual-hosted git repository. imaxon pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/asterixdb.git commit 34351729058b19a530b9b4aa193433141840c352 Author: Michael Blow AuthorDate: Wed May 19 08:00:00 2021 -0400 [NO ISSUE][HYR][HTTP] http stream handler -> consumer / processor Change-Id: I7cd7622dbee880845d0b4233ce3a3b17af15eebc Reviewed-on: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/11524 Reviewed-by: Michael Blow Reviewed-by: Ian Maxon Integration-Tests: Jenkins Tested-by: Jenkins --- .../org/apache/hyracks/http/server/utils/HttpUtil.java | 17 - 1 file changed, 12 insertions(+), 5 deletions(-) diff --git a/hyracks-fullstack/hyracks/hyracks-http/src/main/java/org/apache/hyracks/http/server/utils/HttpUtil.java b/hyracks-fullstack/hyracks/hyracks-http/src/main/java/org/apache/hyracks/http/server/utils/HttpUtil.java index c61e0d2..835cd54 100644 --- a/hyracks-fullstack/hyracks/hyracks-http/src/main/java/org/apache/hyracks/http/server/utils/HttpUtil.java +++ b/hyracks-fullstack/hyracks/hyracks-http/src/main/java/org/apache/hyracks/http/server/utils/HttpUtil.java @@ -45,6 +45,7 @@ import org.apache.hyracks.http.server.BaseRequest; import org.apache.hyracks.http.server.FormUrlEncodedRequest; import org.apache.hyracks.http.server.HttpServer; import org.apache.hyracks.util.ThrowingConsumer; +import org.apache.hyracks.util.ThrowingFunction; import org.apache.logging.log4j.LogManager; import org.apache.logging.log4j.Logger; @@ -191,16 +192,23 @@ public class HttpUtil { return i < 0 ? uri : uri.substring(0, i); } -public static void handleStreamInterruptibly(CloseableHttpResponse response, +public static void consumeStreamInterruptibly(CloseableHttpResponse response, ThrowingConsumer streamProcessor, ExecutorService executor, Supplier descriptionSupplier) +throws InterruptedException, ExecutionException, IOException { +processStreamInterruptibly(response, ThrowingConsumer.asFunction(streamProcessor), executor, +descriptionSupplier); +} + +public static T processStreamInterruptibly(CloseableHttpResponse response, +ThrowingFunction streamProcessor, ExecutorService executor, Supplier descriptionSupplier) throws IOException, InterruptedException, ExecutionException { // we have to consume the stream in a separate thread, as it not stop on interrupt; we need to // instead close the connection to achieve the interrupt String description = descriptionSupplier.get(); -Future readFuture = executor.submit(() -> { +Future readFuture = executor.submit(() -> { Thread.currentThread().setName(description); InputStreamReader reader = new InputStreamReader(response.getEntity().getContent(), StandardCharsets.UTF_8); -streamProcessor.process(new Reader() { +return streamProcessor.process(new Reader() { @Override public int read(char[] cbuf, int off, int len) throws IOException { return reader.read(cbuf, off, len); @@ -213,10 +221,9 @@ public class HttpUtil { LOGGER.debug("ignoring close on {}", reader); } }); -return null; }); try { -readFuture.get(); +return readFuture.get(); } catch (InterruptedException ex) { // NOSONAR -- interrupt or rethrow response.close(); try {
[asterixdb] 16/38: [NO ISSUE] Update third-party dependencies
This is an automated email from the ASF dual-hosted git repository. imaxon pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/asterixdb.git commit 3856cb39f60d86f492330bd42e899c2cd13c5813 Author: Michael Blow AuthorDate: Sat May 1 11:36:15 2021 -0400 [NO ISSUE] Update third-party dependencies - com.fasterxml.jackson.core:jackson-*:2.12.1 -> 2.12.3 - commons-codec:commons-codec:1.14 -> 1.15 - commons-io:commons-io:2.6 -> 2.8.0 - io.netty:netty-all:4.1.59.Final -> 4.1.63.Final - org.apache.commons:commons-lang3:3.9 -> 3.12.0 - org.apache.commons:commons-text:1.8 -> 1.9 - org.apache.httpcomponents:httpcore:4.4.13 -> 4.4.14 - org.apache.logging.log4j:log4j-*:2.13.3 -> 2.14.1 Change-Id: I9aa3ea779ccc84ab2fafbf33345d391f34f0696d Reviewed-on: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/11263 Tested-by: Jenkins Integration-Tests: Jenkins Reviewed-by: Michael Blow Reviewed-by: Murtadha Hubail --- asterixdb/asterix-server/pom.xml | 4 ++-- asterixdb/pom.xml | 4 ++-- .../main/appended-resources/supplemental-models.xml| 8 ...tent.com_netty_netty_netty-4.1.63.Final_NOTICE.txt} | 0 hyracks-fullstack/pom.xml | 18 +- 5 files changed, 17 insertions(+), 17 deletions(-) diff --git a/asterixdb/asterix-server/pom.xml b/asterixdb/asterix-server/pom.xml index 877293c..ee58e2d 100644 --- a/asterixdb/asterix-server/pom.xml +++ b/asterixdb/asterix-server/pom.xml @@ -169,8 +169,8 @@ https://raw.githubusercontent.com/mojohaus/appassembler/appassembler-2.0.0/LICENSE.txt - io.netty:netty-all:4.1.59.Final - https://raw.githubusercontent.com/netty/netty/netty-4.1.59.Final/NOTICE.txt + io.netty:netty-all:4.1.63.Final + https://raw.githubusercontent.com/netty/netty/netty-4.1.63.Final/NOTICE.txt diff --git a/asterixdb/pom.xml b/asterixdb/pom.xml index 81e53c5..ed486fd 100644 --- a/asterixdb/pom.xml +++ b/asterixdb/pom.xml @@ -77,7 +77,7 @@ 0.3.4.2-SNAPSHOT 2.8.5 0.7.6.201602180812 -2.13.3 +2.14.1 Apache AsterixDB - ${project.name} https://asterixdb.apache.org/ @@ -1281,7 +1281,7 @@ commons-codec commons-codec -1.14 +1.15 it.unimi.dsi diff --git a/asterixdb/src/main/appended-resources/supplemental-models.xml b/asterixdb/src/main/appended-resources/supplemental-models.xml index 3f176a7..dd56718 100644 --- a/asterixdb/src/main/appended-resources/supplemental-models.xml +++ b/asterixdb/src/main/appended-resources/supplemental-models.xml @@ -160,9 +160,9 @@ netty-all - 4.1.59.Final - 4.1.59.Final - 4.1.59.Final + 4.1.63.Final + 4.1.63.Final + 4.1.63.Final @@ -229,7 +229,7 @@ jackson-annotations - 2.8.4,2.9.7,2.9.10,2.10.0,2.10.3,2.12.1 + 2.12.3 diff --git a/asterixdb/src/main/licenses/content/raw.githubusercontent.com_netty_netty_netty-4.1.59.Final_NOTICE.txt b/asterixdb/src/main/licenses/content/raw.githubusercontent.com_netty_netty_netty-4.1.63.Final_NOTICE.txt similarity index 100% rename from asterixdb/src/main/licenses/content/raw.githubusercontent.com_netty_netty_netty-4.1.59.Final_NOTICE.txt rename to asterixdb/src/main/licenses/content/raw.githubusercontent.com_netty_netty_netty-4.1.63.Final_NOTICE.txt diff --git a/hyracks-fullstack/pom.xml b/hyracks-fullstack/pom.xml index 2f34980..0ac307d 100644 --- a/hyracks-fullstack/pom.xml +++ b/hyracks-fullstack/pom.xml @@ -69,7 +69,7 @@ 2.8.5 0.7.6.201602180812 -2.13.3 +2.14.1 Apache Hyracks and Algebricks - ${project.name} https://asterixdb.apache.org/ @@ -81,7 +81,7 @@ io.netty netty-all -4.1.59.Final +4.1.63.Final junit @@ -142,27 +142,27 @@ commons-io commons-io -2.6 +2.8.0 org.apache.commons commons-text -1.8 +1.9 com.fasterxml.jackson.core jackson-databind -2.12.1 +2.12.3 com.fasterxml.jackson.core jackson-core -2.12.1 +2.12.3 com.fasterxml.jackson.core jackson-annotations -2.12.1 +2.12.3 com.google.guava @@ -194,7 +194,7 @@ org.apache.commons commons-lang3 -3.9 +3.12.0 org.apache.commons @@ -204,7 +204,7 @@ org.apache.httpcomponents httpcore -
[asterixdb] 09/38: [NO ISSUE][MISC] Misc / minor diagnostic improvement
This is an automated email from the ASF dual-hosted git repository. imaxon pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/asterixdb.git commit d5edad2eccb0a7fa0ca3e99f19b165deae2417b9 Author: Michael Blow AuthorDate: Thu Apr 22 08:50:01 2021 -0400 [NO ISSUE][MISC] Misc / minor diagnostic improvement - include target runtimeId & message kind for ActiveManagerMessage.toString() - add ThrowingFunction.asUnchecked() helper Change-Id: I9e1cccb216033f08b69d3000a68dc8593704af9f Reviewed-on: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/11143 Integration-Tests: Jenkins Reviewed-by: Michael Blow Reviewed-by: Till Westmann Tested-by: Jenkins --- .../asterix/active/message/ActiveManagerMessage.java | 2 +- .../org/apache/hyracks/util/ThrowingFunction.java | 19 +++ 2 files changed, 20 insertions(+), 1 deletion(-) diff --git a/asterixdb/asterix-active/src/main/java/org/apache/asterix/active/message/ActiveManagerMessage.java b/asterixdb/asterix-active/src/main/java/org/apache/asterix/active/message/ActiveManagerMessage.java index 4d726cf..bad3f79 100644 --- a/asterixdb/asterix-active/src/main/java/org/apache/asterix/active/message/ActiveManagerMessage.java +++ b/asterixdb/asterix-active/src/main/java/org/apache/asterix/active/message/ActiveManagerMessage.java @@ -64,6 +64,6 @@ public class ActiveManagerMessage extends CcIdentifiedMessage implements INcAddr @Override public String toString() { -return getClass().getSimpleName(); +return getClass().getSimpleName() + "{" + "kind=" + kind + ", runtimeId=" + runtimeId + '}'; } } diff --git a/hyracks-fullstack/hyracks/hyracks-util/src/main/java/org/apache/hyracks/util/ThrowingFunction.java b/hyracks-fullstack/hyracks/hyracks-util/src/main/java/org/apache/hyracks/util/ThrowingFunction.java index d18b41b..f60efbc 100644 --- a/hyracks-fullstack/hyracks/hyracks-util/src/main/java/org/apache/hyracks/util/ThrowingFunction.java +++ b/hyracks-fullstack/hyracks/hyracks-util/src/main/java/org/apache/hyracks/util/ThrowingFunction.java @@ -18,7 +18,26 @@ */ package org.apache.hyracks.util; +import java.util.function.Function; + +import com.google.common.util.concurrent.UncheckedExecutionException; + @FunctionalInterface public interface ThrowingFunction { R process(I input) throws Exception; + +@SuppressWarnings("Duplicates") +static Function asUnchecked(ThrowingFunction consumer) { +return input -> { +try { +return consumer.process(input); +} catch (InterruptedException e) { +Thread.currentThread().interrupt(); +throw new UncheckedExecutionException(e); +} catch (Exception e) { +throw new UncheckedExecutionException(e); +} +}; +} + }
[asterixdb] 27/38: [NO ISSUE][TX] Ensure lock manager computes valid indexes for hash table
This is an automated email from the ASF dual-hosted git repository. imaxon pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/asterixdb.git commit ed56b863625ec6e9ed05ca58e13329e332ebe457 Author: Ali Alsuliman AuthorDate: Mon May 10 19:53:45 2021 +0300 [NO ISSUE][TX] Ensure lock manager computes valid indexes for hash table - user model changes: no - storage format changes: no - interface changes: no Details: Given a dataset id and an entity hash value, ensure that the lock manager computes a valid index for looking up the hash table. Change-Id: I33d342455a79d0d69b49e126ff6a40c47b07d0dd Reviewed-on: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/11403 Reviewed-by: Ali Alsuliman Reviewed-by: Michael Blow Tested-by: Michael Blow Tested-by: Jenkins Integration-Tests: Jenkins --- .../transaction/management/service/locking/ResourceGroupTable.java | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/asterixdb/asterix-transactions/src/main/java/org/apache/asterix/transaction/management/service/locking/ResourceGroupTable.java b/asterixdb/asterix-transactions/src/main/java/org/apache/asterix/transaction/management/service/locking/ResourceGroupTable.java index 93cda83..a82b302 100644 --- a/asterixdb/asterix-transactions/src/main/java/org/apache/asterix/transaction/management/service/locking/ResourceGroupTable.java +++ b/asterixdb/asterix-transactions/src/main/java/org/apache/asterix/transaction/management/service/locking/ResourceGroupTable.java @@ -41,7 +41,7 @@ class ResourceGroupTable { ResourceGroup get(int dId, int entityHashValue) { // TODO ensure good properties of hash function -return table[Math.abs(dId ^ entityHashValue) % size]; +return table[Math.abs((dId ^ entityHashValue) % size)]; } ResourceGroup get(int i) {
[asterixdb] 15/38: [NO ISSUE][OTH] Add util method to get request http scheme
This is an automated email from the ASF dual-hosted git repository. imaxon pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/asterixdb.git commit 82f19315dde3cf25191314ea558ba0e7d3c474d2 Author: Murtadha Hubail AuthorDate: Thu Apr 29 17:44:38 2021 +0300 [NO ISSUE][OTH] Add util method to get request http scheme - user model changes: no - storage format changes: no - interface changes: no Details: - Add util method to get request http scheme. - Allow exceptions to be thrown from servletNotFound. Change-Id: I85e88223a0ec3291be4423cf5dc146d7d7d09903 Reviewed-on: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/11224 Integration-Tests: Jenkins Tested-by: Jenkins Reviewed-by: Murtadha Hubail Reviewed-by: Ali Alsuliman --- .../java/org/apache/hyracks/http/server/HttpServerHandler.java| 8 ++-- .../main/java/org/apache/hyracks/http/server/utils/HttpUtil.java | 6 ++ 2 files changed, 8 insertions(+), 6 deletions(-) diff --git a/hyracks-fullstack/hyracks/hyracks-http/src/main/java/org/apache/hyracks/http/server/HttpServerHandler.java b/hyracks-fullstack/hyracks/hyracks-http/src/main/java/org/apache/hyracks/http/server/HttpServerHandler.java index 77d3493..a7ace7a 100644 --- a/hyracks-fullstack/hyracks/hyracks-http/src/main/java/org/apache/hyracks/http/server/HttpServerHandler.java +++ b/hyracks-fullstack/hyracks/hyracks-http/src/main/java/org/apache/hyracks/http/server/HttpServerHandler.java @@ -18,8 +18,6 @@ */ package org.apache.hyracks.http.server; -import static org.apache.hyracks.http.server.utils.HttpUtil.X_FORWARDED_PROTO; - import java.io.IOException; import java.util.concurrent.Future; import java.util.concurrent.RejectedExecutionException; @@ -143,9 +141,7 @@ public class HttpServerHandler extends SimpleChannelInboun private void submit(ChannelHandlerContext ctx, IServlet servlet, FullHttpRequest request) throws IOException { IServletRequest servletRequest; try { -HttpScheme scheme = -server.getScheme() == HttpScheme.HTTPS || "https".equals(request.headers().get(X_FORWARDED_PROTO)) -? HttpScheme.HTTPS : HttpScheme.HTTP; +HttpScheme scheme = HttpUtil.getScheme(server, request); servletRequest = createServletRequest(ctx, request, scheme); } catch (IllegalArgumentException e) { LOGGER.log(Level.WARN, "Failure Decoding Request", e); @@ -166,7 +162,7 @@ public class HttpServerHandler extends SimpleChannelInboun } } -protected void handleServletNotFound(ChannelHandlerContext ctx, FullHttpRequest request) { +protected void handleServletNotFound(ChannelHandlerContext ctx, FullHttpRequest request) throws IOException { if (LOGGER.isDebugEnabled()) { LOGGER.debug("No servlet for " + request.uri()); } diff --git a/hyracks-fullstack/hyracks/hyracks-http/src/main/java/org/apache/hyracks/http/server/utils/HttpUtil.java b/hyracks-fullstack/hyracks/hyracks-http/src/main/java/org/apache/hyracks/http/server/utils/HttpUtil.java index 4c0942b..c61e0d2 100644 --- a/hyracks-fullstack/hyracks/hyracks-http/src/main/java/org/apache/hyracks/http/server/utils/HttpUtil.java +++ b/hyracks-fullstack/hyracks/hyracks-http/src/main/java/org/apache/hyracks/http/server/utils/HttpUtil.java @@ -43,6 +43,7 @@ import org.apache.hyracks.http.api.IServletRequest; import org.apache.hyracks.http.api.IServletResponse; import org.apache.hyracks.http.server.BaseRequest; import org.apache.hyracks.http.server.FormUrlEncodedRequest; +import org.apache.hyracks.http.server.HttpServer; import org.apache.hyracks.util.ThrowingConsumer; import org.apache.logging.log4j.LogManager; import org.apache.logging.log4j.Logger; @@ -230,6 +231,11 @@ public class HttpUtil { } } +public static HttpScheme getScheme(HttpServer server, FullHttpRequest request) { +return server.getScheme() == HttpScheme.HTTPS || "https".equals(request.headers().get(X_FORWARDED_PROTO)) +? HttpScheme.HTTPS : HttpScheme.HTTP; +} + public static class ContentType { public static final String ADM = "adm"; public static final String JSON = "json";
[asterixdb] 08/38: [NO ISSUE][OTH] Add operator field to subplan json plan
This is an automated email from the ASF dual-hosted git repository. imaxon pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/asterixdb.git commit d5eb016761601e66a2fdd66c8cd8cc72cf248a03 Author: Murtadha Hubail AuthorDate: Wed Apr 21 18:28:38 2021 +0300 [NO ISSUE][OTH] Add operator field to subplan json plan - user model changes: no - storage format changes: no - interface changes: no Details: - Add "operator" field to subplan operators json format plan. Change-Id: I1df5a97367286663b002547997c555b20e56697d Reviewed-on: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/11123 Integration-Tests: Jenkins Tested-by: Jenkins Reviewed-by: Murtadha Hubail Reviewed-by: Ali Alsuliman --- .../prettyprint/LogicalOperatorPrettyPrintVisitorJson.java | 9 +++-- 1 file changed, 7 insertions(+), 2 deletions(-) diff --git a/hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/prettyprint/LogicalOperatorPrettyPrintVisitorJson.java b/hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/prettyprint/LogicalOperatorPrettyPrintVisitorJson.java index f404058..6ba4c0c 100644 --- a/hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/prettyprint/LogicalOperatorPrettyPrintVisitorJson.java +++ b/hyracks-fullstack/algebricks/algebricks-core/src/main/java/org/apache/hyracks/algebricks/core/algebra/prettyprint/LogicalOperatorPrettyPrintVisitorJson.java @@ -411,8 +411,13 @@ public class LogicalOperatorPrettyPrintVisitorJson extends AbstractLogicalOperat @Override public Void visitSubplanOperator(SubplanOperator op, Void indent) throws AlgebricksException { -if (!op.getNestedPlans().isEmpty()) { -writeNestedPlans(op, indent); +try { +if (!op.getNestedPlans().isEmpty()) { +jsonGenerator.writeStringField(OPERATOR_FIELD, "subplan"); +writeNestedPlans(op, indent); +} +} catch (IOException e) { +throw AlgebricksException.create(ErrorCode.ERROR_PRINTING_PLAN, e, String.valueOf(e)); } return null; }
[asterixdb] 14/38: [NO ISSUE][FUN] Remove decode_dataverse_display_name()
This is an automated email from the ASF dual-hosted git repository. imaxon pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/asterixdb.git commit 61aad744bcb446f976bc79a9feedce88256bbc26 Author: Dmitry Lychagin AuthorDate: Tue Apr 27 12:19:43 2021 -0700 [NO ISSUE][FUN] Remove decode_dataverse_display_name() - user model changes: no - storage format changes: no - interface changes: no Details: - Remove decode_dataverse_display_name() function Change-Id: Iae38424c92ba237b58032958b67a6e2026615f0d Reviewed-on: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/11203 Integration-Tests: Jenkins Reviewed-by: Michael Blow Tested-by: Jenkins --- .../special_chars_2/special_chars_2.2.query.sqlpp | 1 - .../special_chars_2/special_chars_2.2.adm | 12 +-- .../asterix/om/functions/BuiltinFunctions.java | 3 - .../DecodeDataverseDisplayNameDescriptor.java | 88 -- .../runtime/functions/FunctionCollection.java | 2 - 5 files changed, 6 insertions(+), 100 deletions(-) diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/multipart-dataverse/special_chars_2/special_chars_2.2.query.sqlpp b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/multipart-dataverse/special_chars_2/special_chars_2.2.query.sqlpp index 4dc4e3b..08a41b5 100644 --- a/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/multipart-dataverse/special_chars_2/special_chars_2.2.query.sqlpp +++ b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/multipart-dataverse/special_chars_2/special_chars_2.2.query.sqlpp @@ -18,7 +18,6 @@ */ select DataverseName as CanonicalName, - decode_dataverse_display_name(DataverseName) as DisplayName, decode_dataverse_name(DataverseName) as NameParts from Metadata.`Dataverse` order by DataverseName; \ No newline at end of file diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/results/multipart-dataverse/special_chars_2/special_chars_2.2.adm b/asterixdb/asterix-app/src/test/resources/runtimets/results/multipart-dataverse/special_chars_2/special_chars_2.2.adm index 7edd3eb..805247c 100644 --- a/asterixdb/asterix-app/src/test/resources/runtimets/results/multipart-dataverse/special_chars_2/special_chars_2.2.adm +++ b/asterixdb/asterix-app/src/test/resources/runtimets/results/multipart-dataverse/special_chars_2/special_chars_2.2.adm @@ -1,6 +1,6 @@ -{ "CanonicalName": "A", "DisplayName": "A", "NameParts": [ "A" ] } -{ "CanonicalName": "B/C", "DisplayName": "B.C", "NameParts": [ "B", "C" ] } -{ "CanonicalName": "C.D.E", "DisplayName": "`C.D.E`", "NameParts": [ "C.D.E" ] } -{ "CanonicalName": "Default", "DisplayName": "Default", "NameParts": [ "Default" ] } -{ "CanonicalName": "Metadata", "DisplayName": "Metadata", "NameParts": [ "Metadata" ] } -{ "CanonicalName": "a-A/b_B/c$C/z.Z", "DisplayName": "`a-A`.b_B.c$C.`z.Z`", "NameParts": [ "a-A", "b_B", "c$C", "z.Z" ] } \ No newline at end of file +{ "CanonicalName": "A", "NameParts": [ "A" ] } +{ "CanonicalName": "B/C", "NameParts": [ "B", "C" ] } +{ "CanonicalName": "C.D.E", "NameParts": [ "C.D.E" ] } +{ "CanonicalName": "Default", "NameParts": [ "Default" ] } +{ "CanonicalName": "Metadata", "NameParts": [ "Metadata" ] } +{ "CanonicalName": "a-A/b_B/c$C/z.Z", "NameParts": [ "a-A", "b_B", "c$C", "z.Z" ] } \ No newline at end of file diff --git a/asterixdb/asterix-om/src/main/java/org/apache/asterix/om/functions/BuiltinFunctions.java b/asterixdb/asterix-om/src/main/java/org/apache/asterix/om/functions/BuiltinFunctions.java index aac01f9..a2e235e 100644 --- a/asterixdb/asterix-om/src/main/java/org/apache/asterix/om/functions/BuiltinFunctions.java +++ b/asterixdb/asterix-om/src/main/java/org/apache/asterix/om/functions/BuiltinFunctions.java @@ -1592,8 +1592,6 @@ public class BuiltinFunctions { public static final FunctionIdentifier DECODE_DATAVERSE_NAME = new FunctionIdentifier(FunctionConstants.ASTERIX_NS, "decode-dataverse-name", 1); -public static final FunctionIdentifier DECODE_DATAVERSE_DISPLAY_NAME = -new FunctionIdentifier(FunctionConstants.ASTERIX_NS, "decode-dataverse-display-name", 1); static {
[asterixdb] 29/38: [NO ISSUE][MISC] Fix typo on active state log, += HyracksThrowingSupplier
This is an automated email from the ASF dual-hosted git repository. imaxon pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/asterixdb.git commit b69e30e1c0b55d6216647f28e3b68a16a66028e0 Author: Michael Blow AuthorDate: Wed May 12 12:06:33 2021 -0400 [NO ISSUE][MISC] Fix typo on active state log, += HyracksThrowingSupplier Change-Id: I19ba60f6a362b06451be331449d21f3c37653570 Reviewed-on: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/11443 Reviewed-by: Michael Blow Reviewed-by: Ian Maxon Integration-Tests: Jenkins Tested-by: Jenkins --- .../app/active/ActiveEntityEventsListener.java | 7 ++ .../hyracks/api/util/HyracksThrowingSupplier.java | 26 ++ 2 files changed, 28 insertions(+), 5 deletions(-) diff --git a/asterixdb/asterix-app/src/main/java/org/apache/asterix/app/active/ActiveEntityEventsListener.java b/asterixdb/asterix-app/src/main/java/org/apache/asterix/app/active/ActiveEntityEventsListener.java index 2891710..e18e50c 100644 --- a/asterixdb/asterix-app/src/main/java/org/apache/asterix/app/active/ActiveEntityEventsListener.java +++ b/asterixdb/asterix-app/src/main/java/org/apache/asterix/app/active/ActiveEntityEventsListener.java @@ -133,15 +133,12 @@ public abstract class ActiveEntityEventsListener implements IActiveEntityControl this.locations = locations; this.numRegistered = 0; this.numDeRegistered = 0; -this.handler = -(ActiveNotificationHandler) metadataProvider.getApplicationContext().getActiveNotificationHandler(); +this.handler = (ActiveNotificationHandler) appCtx.getActiveNotificationHandler(); handler.registerListener(this); } protected synchronized void setState(ActivityState newState) { -if (LOGGER.isEnabled(level)) { -LOGGER.log(level, "State of " + getEntityId() + "is being set to " + newState + " from " + state); -} +LOGGER.log(level, "State of {} is being set to {} from {}", getEntityId(), newState, state); this.prevState = state; this.state = newState; if (newState == ActivityState.STARTING || newState == ActivityState.RECOVERING diff --git a/hyracks-fullstack/hyracks/hyracks-api/src/main/java/org/apache/hyracks/api/util/HyracksThrowingSupplier.java b/hyracks-fullstack/hyracks/hyracks-api/src/main/java/org/apache/hyracks/api/util/HyracksThrowingSupplier.java new file mode 100644 index 000..c6c3922 --- /dev/null +++ b/hyracks-fullstack/hyracks/hyracks-api/src/main/java/org/apache/hyracks/api/util/HyracksThrowingSupplier.java @@ -0,0 +1,26 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ +package org.apache.hyracks.api.util; + +import org.apache.hyracks.api.exceptions.HyracksDataException; + +@FunctionalInterface +public interface HyracksThrowingSupplier { +T get() throws HyracksDataException; +}
[asterixdb] 06/38: [NO ISSUE][EXT] Provide direct access to controller stat primitives
This is an automated email from the ASF dual-hosted git repository. imaxon pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/asterixdb.git commit 605b115c2abd9bc4570c349afde6a7c485bab32c Author: Michael Blow AuthorDate: Thu Apr 1 15:08:05 2021 -0400 [NO ISSUE][EXT] Provide direct access to controller stat primitives Change-Id: I40281b2cc682e0c32359963733909caca95f511f Reviewed-on: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/10825 Integration-Tests: Jenkins Tested-by: Jenkins Reviewed-by: Michael Blow --- .../asterix/external/dataflow/AbstractFeedDataFlowController.java| 5 + .../asterix/external/dataflow/FeedRecordDataFlowController.java | 5 - .../asterix/external/dataflow/FeedStreamDataFlowController.java | 1 - 3 files changed, 9 insertions(+), 2 deletions(-) diff --git a/asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/dataflow/AbstractFeedDataFlowController.java b/asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/dataflow/AbstractFeedDataFlowController.java index 94d9e6e..bd422ef 100644 --- a/asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/dataflow/AbstractFeedDataFlowController.java +++ b/asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/dataflow/AbstractFeedDataFlowController.java @@ -35,6 +35,7 @@ public abstract class AbstractFeedDataFlowController implements IDataFlowControl protected final ArrayTupleBuilder tb; protected final FeedLogManager feedLogManager; protected boolean flushing; +protected long incomingRecordsCount = 0; public AbstractFeedDataFlowController(IHyracksTaskContext ctx, FeedLogManager feedLogManager, int numOfFields) { this.feedLogManager = feedLogManager; @@ -50,6 +51,10 @@ public abstract class AbstractFeedDataFlowController implements IDataFlowControl flushing = false; } +public long getIncomingRecordsCount() { +return incomingRecordsCount; +} + public abstract String getStats(); @Override diff --git a/asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/dataflow/FeedRecordDataFlowController.java b/asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/dataflow/FeedRecordDataFlowController.java index 8cec5de..4279ebd 100644 --- a/asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/dataflow/FeedRecordDataFlowController.java +++ b/asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/dataflow/FeedRecordDataFlowController.java @@ -57,7 +57,6 @@ public class FeedRecordDataFlowController extends AbstractFeedDataFlowControl protected final AtomicBoolean closed = new AtomicBoolean(false); protected static final long INTERVAL = 1000; protected State state = State.CREATED; -protected long incomingRecordsCount = 0; protected long failedRecordsCount = 0; public FeedRecordDataFlowController(IHyracksTaskContext ctx, FeedLogManager feedLogManager, int numOfOutputFields, @@ -267,6 +266,10 @@ public class FeedRecordDataFlowController extends AbstractFeedDataFlowControl return dataParser; } +public long getFailedRecordsCount() { +return failedRecordsCount; +} + @Override public String getStats() { String readerStats = recordReader.getStats(); diff --git a/asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/dataflow/FeedStreamDataFlowController.java b/asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/dataflow/FeedStreamDataFlowController.java index b42e6de..9f4a3b0 100644 --- a/asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/dataflow/FeedStreamDataFlowController.java +++ b/asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/dataflow/FeedStreamDataFlowController.java @@ -32,7 +32,6 @@ public class FeedStreamDataFlowController extends AbstractFeedDataFlowController private final IStreamDataParser dataParser; private final AsterixInputStream stream; -protected long incomingRecordsCount = 0; public FeedStreamDataFlowController(IHyracksTaskContext ctx, FeedLogManager feedLogManager, IStreamDataParser streamParser, AsterixInputStream inputStream) {
[asterixdb] 02/38: [NO ISSUE][REP] Account for SSL socket data transfer behavior
This is an automated email from the ASF dual-hosted git repository. imaxon pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/asterixdb.git commit 94b1306e380dc923a964abf60900d55e545a4e4f Author: Murtadha Hubail AuthorDate: Thu Apr 15 02:53:08 2021 +0300 [NO ISSUE][REP] Account for SSL socket data transfer behavior - user model changes: no - storage format changes: no - interface changes: no Details: - Account for the fact that a call to an SSLSocketChannel read operation can return 0 read bytes even in a blocking socket due to incomplete SSL packet. Change-Id: I0e1e69cba7336e0cfca5def870ab16334ce8c19f Reviewed-on: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/11064 Integration-Tests: Jenkins Tested-by: Jenkins Reviewed-by: Murtadha Hubail Reviewed-by: Ali Alsuliman --- .../replication/management/NetworkingUtil.java | 34 -- 1 file changed, 18 insertions(+), 16 deletions(-) diff --git a/asterixdb/asterix-replication/src/main/java/org/apache/asterix/replication/management/NetworkingUtil.java b/asterixdb/asterix-replication/src/main/java/org/apache/asterix/replication/management/NetworkingUtil.java index 97dd049..7f6439c 100644 --- a/asterixdb/asterix-replication/src/main/java/org/apache/asterix/replication/management/NetworkingUtil.java +++ b/asterixdb/asterix-replication/src/main/java/org/apache/asterix/replication/management/NetworkingUtil.java @@ -60,30 +60,32 @@ public class NetworkingUtil { public static void sendFile(FileChannel fileChannel, ISocketChannel socketChannel) throws IOException { long pos = 0; -long fileSize = fileChannel.size(); -long remainingBytes = fileSize; -long transferredBytes = 0; - -while ((transferredBytes += fileChannel.transferTo(pos, remainingBytes, socketChannel)) < fileSize) { -pos += transferredBytes; -remainingBytes -= transferredBytes; +long remainingBytes = fileChannel.size(); +try { +while (remainingBytes > 0) { +long sentBytes = fileChannel.transferTo(pos, remainingBytes, socketChannel); +pos += sentBytes; +remainingBytes -= sentBytes; +} + socketChannel.getSocketChannel().socket().getOutputStream().flush(); +} catch (Exception e) { +LOGGER.info("failed to send file; file size {}, pos {}, remainingBytes {}", fileChannel.size(), pos, +remainingBytes); } -socketChannel.getSocketChannel().socket().getOutputStream().flush(); } public static void downloadFile(FileChannel fileChannel, ISocketChannel socketChannel) throws IOException { +long remainingBytes = fileChannel.size(); long pos = 0; -long fileSize = fileChannel.size(); -long count = fileSize; -long numTransferred = 0; try { -while ((numTransferred += fileChannel.transferFrom(socketChannel, pos, count)) < fileSize) { -pos += numTransferred; -count -= numTransferred; +while (remainingBytes > 0) { +long readBytes = fileChannel.transferFrom(socketChannel, pos, remainingBytes); +pos += readBytes; +remainingBytes -= readBytes; } } catch (Exception e) { -LOGGER.info("failed to download file; file size {}, pos {}, count {}, numTransferred {}", fileSize, pos, -count, numTransferred); +LOGGER.info("failed to download file; file size {}, pos {}, remainingBytes {}", fileChannel.size(), pos, +remainingBytes); throw e; } }
[asterixdb] 23/38: [ASTERIXDB-2897][FUN] Fix unknown handling in PyUDF
This is an automated email from the ASF dual-hosted git repository. imaxon pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/asterixdb.git commit ec5195a4f58b2ce3971b537b48078f3bc696e0bf Author: Ian Maxon AuthorDate: Tue May 4 09:35:21 2021 -0700 [ASTERIXDB-2897][FUN] Fix unknown handling in PyUDF - user model changes: no - storage format changes: no - interface changes: no Details: Carry the fix from ASTERIXDB-2883 to Python UDFs, and add a corresponding test for both this new code in the non-batched execution, as well as checking that it works the same in batched mode Change-Id: I64ac16d094caafcc657a21650011ed41d0c3c6d6 Reviewed-on: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/11324 Reviewed-by: Ian Maxon Reviewed-by: Dmitry Lychagin Integration-Tests: Jenkins Tested-by: Jenkins --- ...3.query.sqlpp => type_validation.10.query.sqlpp} | 7 +-- ...ion.4.ddl.sqlpp => type_validation.11.ddl.sqlpp} | 0 .../type_validation.2.ddl.sqlpp | 4 +++- .../type_validation.3.query.sqlpp | 3 ++- 3.query.sqlpp => type_validation.4.query.sqlpp} | 5 - 3.query.sqlpp => type_validation.5.query.sqlpp} | 5 +++-- 3.query.sqlpp => type_validation.6.query.sqlpp} | 7 +-- 3.query.sqlpp => type_validation.7.query.sqlpp} | 3 ++- 3.query.sqlpp => type_validation.8.query.sqlpp} | 5 - 3.query.sqlpp => type_validation.9.query.sqlpp} | 5 +++-- .../type_validation.1.adm | 2 +- .../type_validation.2.adm | 1 + .../type_validation.3.adm | 1 + .../type_validation.4.adm | 1 + .../type_validation.5.adm | 1 + .../type_validation.6.adm | 1 + .../type_validation.7.adm | 1 + .../type_validation.8.adm | 1 + .../ExternalScalarPythonFunctionEvaluator.java | 21 ++--- 19 files changed, 57 insertions(+), 17 deletions(-) diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/external-library/python_open_type_validation/type_validation.3.query.sqlpp b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/external-library/python_open_type_validation/type_validation.10.query.sqlpp similarity index 83% copy from asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/external-library/python_open_type_validation/type_validation.3.query.sqlpp copy to asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/external-library/python_open_type_validation/type_validation.10.query.sqlpp index 0ae7d0c..69a0b25 100644 --- a/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/external-library/python_open_type_validation/type_validation.3.query.sqlpp +++ b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/external-library/python_open_type_validation/type_validation.10.query.sqlpp @@ -17,6 +17,9 @@ * under the License. */ use externallibtest; -typeValidation(907, 9.07, "907", 9.07, true, unix_time_from_date_in_days(date("2013-01-01")), - unix_time_from_datetime_in_secs(datetime("1989-09-07T12:13:14.039Z"))); + +set `rewrite_attempt_batch_assign` "false"; + +typeValidationNullCall(907, 9.07, "907", 9.07, true, unix_time_from_date_in_days(date("2013-01-01")), + unix_time_from_datetime_in_secs(datetime("1989-09-07T12:13:14.039Z")), missing); diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/external-library/python_open_type_validation/type_validation.4.ddl.sqlpp b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/external-library/python_open_type_validation/type_validation.11.ddl.sqlpp similarity index 100% rename from asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/external-library/python_open_type_validation/type_validation.4.ddl.sqlpp rename to asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/external-library/python_open_type_validation/type_validation.11.ddl.sqlpp diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/external-library/python_open_type_validation/type_validation.2.ddl.sqlpp b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/external-library/python_open_type_validation/type_validation.2.ddl.sqlpp index a8ba8a1..8041b5e 100644 --- a/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/external-library/python_open_type_validation/type_validation.2.ddl.sqlpp +++ b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/external-library/python_open_type_validation/type_validation.2.ddl.sqlpp @@ -18,6 +18,8 @@ */ use externallibtest; -cre
[asterixdb] 36/38: [NO ISSUE][MISC] Improve parser error reporting
This is an automated email from the ASF dual-hosted git repository. imaxon pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/asterixdb.git commit 3b6982ce7fa50ff4005d50847a944ecbaf3ecb30 Author: Michael Blow AuthorDate: Thu May 20 16:43:43 2021 -0400 [NO ISSUE][MISC] Improve parser error reporting Change-Id: I8707d9e2a952693b6501e9e63aff9162a77541cc Reviewed-on: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/11543 Integration-Tests: Jenkins Tested-by: Jenkins Reviewed-by: Michael Blow Reviewed-by: Till Westmann --- ...stsuite_external_dataset_azure_blob_storage.xml | 10 - .../runtimets/testsuite_external_dataset_s3.xml| 10 - ...testsuite_external_dataset_s3_one_partition.xml | 20 - .../reader/stream/QuotedLineRecordReader.java | 2 +- .../reader/stream/SemiStructuredRecordReader.java | 2 +- .../external/parser/AbstractJsonDataParser.java| 18 +++- .../external/parser/DelimitedDataParser.java | 2 +- .../java/org/apache/hyracks}/util/ParseUtil.java | 25 +++--- .../src/main/resources/errormsg/en.properties | 2 +- .../file/FieldCursorForDelimitedDataParser.java| 5 ++--- 10 files changed, 56 insertions(+), 40 deletions(-) diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/testsuite_external_dataset_azure_blob_storage.xml b/asterixdb/asterix-app/src/test/resources/runtimets/testsuite_external_dataset_azure_blob_storage.xml index df60e60..1e302da 100644 --- a/asterixdb/asterix-app/src/test/resources/runtimets/testsuite_external_dataset_azure_blob_storage.xml +++ b/asterixdb/asterix-app/src/test/resources/runtimets/testsuite_external_dataset_azure_blob_storage.xml @@ -143,11 +143,11 @@ common/malformed-json -Parsing error at malformed-data/duplicate-fields.json line 1 field field: Duplicate field 'field' -Parsing error at malformed-data/malformed-json.json line 1 field field: Unexpected character ('}' (code 125)): was expecting double-quote to start field name -Parsing error at malformed-data/malformed-json-2.json line 4 field array_f: Unexpected character (']' (code 93)): expected a valid value (JSON String, Number, Array, Object or token 'null', 'true' or 'false') -Parsing error at malformed-data/malformed-jsonl-1.json line 3 field field2: Unrecognized token 'truee': was expecting (JSON String, Number, Array, Object or token 'null', 'true' or 'false') -Parsing error at malformed-data/malformed-jsonl-2.json line 11 field array_f: Unexpected character (']' (code 93)): expected a valid value (JSON String, Number, Array, Object or token 'null', 'true' or 'false') +Parsing error at malformed-data/duplicate-fields.json line 1 field 'field': Duplicate field 'field' +Parsing error at malformed-data/malformed-json.json line 1 field 'field': Unexpected character ('}' (code 125)): was expecting double-quote to start field name +Parsing error at malformed-data/malformed-json-2.json line 4 field 'array_f': Unexpected character (']' (code 93)): expected a valid value (JSON String, Number, Array, Object or token 'null', 'true' or 'false') +Parsing error at malformed-data/malformed-jsonl-1.json line 3 field 'field2': Unrecognized token 'truee': was expecting (JSON String, Number, Array, Object or token 'null', 'true' or 'false') +Parsing error at malformed-data/malformed-jsonl-2.json line 11 field 'array_f': Unexpected character (']' (code 93)): expected a valid value (JSON String, Number, Array, Object or token 'null', 'true' or 'false') diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/testsuite_external_dataset_s3.xml b/asterixdb/asterix-app/src/test/resources/runtimets/testsuite_external_dataset_s3.xml index 9bc463c..b354e65 100644 --- a/asterixdb/asterix-app/src/test/resources/runtimets/testsuite_external_dataset_s3.xml +++ b/asterixdb/asterix-app/src/test/resources/runtimets/testsuite_external_dataset_s3.xml @@ -92,11 +92,11 @@ common/malformed-json -Parsing error at malformed-data/duplicate-fields.json line 1 field field: Duplicate field 'field' -Parsing error at malformed-data/malformed-json.json line 1 field field: Unexpected character ('}' (code 125)): was expecting double-quote to start field name -Parsing error at malformed-data/malformed-json-2.json line 4 field array_f: Unexpected character (']' (code 93)): expected a valid value (JSON String
[asterixdb] 19/38: Merge branch 'gerrit/mad-hatter'
This is an automated email from the ASF dual-hosted git repository. imaxon pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/asterixdb.git commit 8c67f9cb4a380d23cdb3d3c04a85ff00e31b439d Merge: 82f1931 6abbeea Author: Michael Blow AuthorDate: Sun May 2 12:32:23 2021 -0400 Merge branch 'gerrit/mad-hatter' Change-Id: I9fb18d2460cc83005380020c52a294c06c4b264c asterixdb/asterix-server/pom.xml | 4 ++-- asterixdb/pom.xml | 4 ++-- .../main/appended-resources/supplemental-models.xml| 8 ...tent.com_netty_netty_netty-4.1.63.Final_NOTICE.txt} | 0 hyracks-fullstack/pom.xml | 18 +- 5 files changed, 17 insertions(+), 17 deletions(-) diff --cc asterixdb/pom.xml index 6067ef9,43e0dfd..a930643 --- a/asterixdb/pom.xml +++ b/asterixdb/pom.xml @@@ -75,19 -75,14 +75,19 @@@ ${global.itest.excludes} compile process-classes +none +none +none -0.3.5-SNAPSHOT -0.3.5-SNAPSHOT +0.3.6-SNAPSHOT +0.3.6-SNAPSHOT 2.8.5 0.7.6.201602180812 - 2.13.3 + 2.14.1 2.10.83 +12.6.0 +1.8.2 Apache AsterixDB - ${project.name} https://asterixdb.apache.org/
[asterixdb] 13/38: [NO ISSUE][HYR][HTTP] Set thread name on interruptable stream handler
This is an automated email from the ASF dual-hosted git repository. imaxon pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/asterixdb.git commit 902f63c6683cee08c6340a6a6fe007277722d1be Author: Michael Blow AuthorDate: Sun Apr 25 10:52:25 2021 -0400 [NO ISSUE][HYR][HTTP] Set thread name on interruptable stream handler Change-Id: I26b718cb0373c6c4d04d7fb0666de472c0419b29 Reviewed-on: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/11183 Integration-Tests: Jenkins Tested-by: Jenkins Reviewed-by: Hussain Towaileb --- .../main/java/org/apache/hyracks/http/server/utils/HttpUtil.java | 8 +--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/hyracks-fullstack/hyracks/hyracks-http/src/main/java/org/apache/hyracks/http/server/utils/HttpUtil.java b/hyracks-fullstack/hyracks/hyracks-http/src/main/java/org/apache/hyracks/http/server/utils/HttpUtil.java index 01baa69..4c0942b 100644 --- a/hyracks-fullstack/hyracks/hyracks-http/src/main/java/org/apache/hyracks/http/server/utils/HttpUtil.java +++ b/hyracks-fullstack/hyracks/hyracks-http/src/main/java/org/apache/hyracks/http/server/utils/HttpUtil.java @@ -191,11 +191,13 @@ public class HttpUtil { } public static void handleStreamInterruptibly(CloseableHttpResponse response, -ThrowingConsumer streamProcessor, ExecutorService executor, Supplier taskDescription) +ThrowingConsumer streamProcessor, ExecutorService executor, Supplier descriptionSupplier) throws IOException, InterruptedException, ExecutionException { // we have to consume the stream in a separate thread, as it not stop on interrupt; we need to // instead close the connection to achieve the interrupt +String description = descriptionSupplier.get(); Future readFuture = executor.submit(() -> { +Thread.currentThread().setName(description); InputStreamReader reader = new InputStreamReader(response.getEntity().getContent(), StandardCharsets.UTF_8); streamProcessor.process(new Reader() { @Override @@ -219,10 +221,10 @@ public class HttpUtil { try { readFuture.get(1, TimeUnit.SECONDS); } catch (TimeoutException te) { -LOGGER.warn("{} did not exit on stream close due to interrupt after 1s", taskDescription); +LOGGER.warn("{} did not exit on stream close due to interrupt after 1s", description); readFuture.cancel(true); } catch (ExecutionException ee) { -LOGGER.debug("ignoring exception awaiting aborted {} shutdown", taskDescription, ee); +LOGGER.debug("ignoring exception awaiting aborted {} shutdown", description, ee); } throw ex; }
[asterixdb] 32/38: [NO ISSUE][MISC] Preserve Error instances on retryUntilSuccessOrExhausted
This is an automated email from the ASF dual-hosted git repository. imaxon pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/asterixdb.git commit dec84265a802476cf31d25d374cfd6511e0016c7 Author: Michael Blow AuthorDate: Sun May 16 09:24:27 2021 -0400 [NO ISSUE][MISC] Preserve Error instances on retryUntilSuccessOrExhausted Prior to this change, InvokeUtil.retryUntilSuccessOrExhausted() would wrap instances of java.lang.Error with HyracksDataException upon exhaustion of retry attempts. Errors are typically handled differently than non-Errors, so preserve the Error when propagating the failure to the caller. Change-Id: Idfe1d443addaed342b0c0ed3a0a3835ad226dbe7 Reviewed-on: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/11483 Integration-Tests: Jenkins Reviewed-by: Michael Blow Reviewed-by: Hussain Towaileb Tested-by: Jenkins --- .../src/main/java/org/apache/hyracks/api/util/InvokeUtil.java | 3 +++ 1 file changed, 3 insertions(+) diff --git a/hyracks-fullstack/hyracks/hyracks-api/src/main/java/org/apache/hyracks/api/util/InvokeUtil.java b/hyracks-fullstack/hyracks/hyracks-api/src/main/java/org/apache/hyracks/api/util/InvokeUtil.java index 0b1c5a6..7d04cf2 100644 --- a/hyracks-fullstack/hyracks/hyracks-api/src/main/java/org/apache/hyracks/api/util/InvokeUtil.java +++ b/hyracks-fullstack/hyracks/hyracks-api/src/main/java/org/apache/hyracks/api/util/InvokeUtil.java @@ -280,6 +280,9 @@ public class InvokeUtil { long delayMs = delay.calculate(attempt); if (!policy.retry(th) || span.elapsed() || span.remaining(TimeUnit.MILLISECONDS) < delayMs) { onFailure.attemptFailed(action, attempt, true, span, failure); +if (th instanceof Error) { +throw (Error) th; +} throw HyracksDataException.create(failure); } else { onFailure.attemptFailed(action, attempt, false, span, failure);
[asterixdb] 30/38: [NO ISSUE][STO] Close datasets of flushed indexes after recovery
This is an automated email from the ASF dual-hosted git repository. imaxon pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/asterixdb.git commit 0e7e4bdb2c514fe7469a775c2ffc3d9e78e6a317 Author: Murtadha Hubail AuthorDate: Wed May 12 16:41:02 2021 +0300 [NO ISSUE][STO] Close datasets of flushed indexes after recovery - user model changes: no - storage format changes: no - interface changes: yes Details: - After performing redo of a flush log on any index, close its dataset to ensure any cached state that might have been changed during recovery is cleared (e.g. the component id generator). - Fix LSMFlushRecoveryTest total number of records to be inserted. - Update LSMFlushRecoveryTest to check for duplicate component ids. Change-Id: I29072f475cc7b4d7d6efde415be0329fc568443e Reviewed-on: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/11423 Integration-Tests: Jenkins Tested-by: Jenkins Reviewed-by: Murtadha Hubail Reviewed-by: Michael Blow --- .../org/apache/asterix/app/nc/RecoveryManager.java | 20 +--- .../asterix/test/dataflow/LSMFlushRecoveryTest.java | 12 ++-- .../asterix/common/api/IDatasetLifecycleManager.java | 9 + .../common/context/DatasetLifecycleManager.java | 11 +++ .../common/transactions/IRecoveryManager.java| 11 --- 5 files changed, 43 insertions(+), 20 deletions(-) diff --git a/asterixdb/asterix-app/src/main/java/org/apache/asterix/app/nc/RecoveryManager.java b/asterixdb/asterix-app/src/main/java/org/apache/asterix/app/nc/RecoveryManager.java index 65cb36a..0359cf1 100644 --- a/asterixdb/asterix-app/src/main/java/org/apache/asterix/app/nc/RecoveryManager.java +++ b/asterixdb/asterix-app/src/main/java/org/apache/asterix/app/nc/RecoveryManager.java @@ -173,15 +173,14 @@ public class RecoveryManager implements IRecoveryManager, ILifeCycleComponent { deleteRecoveryTemporaryFiles(); //get active partitions on this node -replayPartitionsLogs(partitions, logMgr.getLogReader(true), lowWaterMarkLSN); +replayPartitionsLogs(partitions, logMgr.getLogReader(true), lowWaterMarkLSN, true); } -@Override -public synchronized void replayPartitionsLogs(Set partitions, ILogReader logReader, long lowWaterMarkLSN) -throws IOException, ACIDException { +public synchronized void replayPartitionsLogs(Set partitions, ILogReader logReader, long lowWaterMarkLSN, +boolean closeOnFlushRedo) throws IOException, ACIDException { try { Set winnerJobSet = startRecoverysAnalysisPhase(partitions, logReader, lowWaterMarkLSN); -startRecoveryRedoPhase(partitions, logReader, lowWaterMarkLSN, winnerJobSet); +startRecoveryRedoPhase(partitions, logReader, lowWaterMarkLSN, winnerJobSet, closeOnFlushRedo); } finally { logReader.close(); deleteRecoveryTemporaryFiles(); @@ -277,7 +276,7 @@ public class RecoveryManager implements IRecoveryManager, ILifeCycleComponent { } private synchronized void startRecoveryRedoPhase(Set partitions, ILogReader logReader, -long lowWaterMarkLSN, Set winnerTxnSet) throws IOException, ACIDException { +long lowWaterMarkLSN, Set winnerTxnSet, boolean closeOnFlushRedo) throws IOException, ACIDException { int redoCount = 0; long txnId = 0; @@ -299,6 +298,7 @@ public class RecoveryManager implements IRecoveryManager, ILifeCycleComponent { TxnEntityId tempKeyTxnEntityId = new TxnEntityId(-1, -1, -1, null, -1, false); ILogRecord logRecord = null; +Set flushRedoDatasets = new HashSet<>(); try { logReader.setPosition(lowWaterMarkLSN); logRecord = logReader.next(); @@ -409,6 +409,7 @@ public class RecoveryManager implements IRecoveryManager, ILifeCycleComponent { && !index.isCurrentMutableComponentEmpty()) { // schedule flush redoFlush(index, logRecord); +flushRedoDatasets.add(datasetId); redoCount++; } else { // TODO: update checkpoint file? @@ -441,6 +442,11 @@ public class RecoveryManager implements IRecoveryManager, ILifeCycleComponent { for (long r : resourceIdList) { datasetLifecycleManager.close(resourcesMap.get(r).getPath()); } +if (closeOnFlushRedo) { +// close datasets of indexes to ensure any cached state that might've been changed by recovery is cleared +// e.g. when redoing a flush, the component id
[asterixdb] 20/38: [ASTERIXDB-2883][COMP] Improve null handling in UDF calls
This is an automated email from the ASF dual-hosted git repository. imaxon pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/asterixdb.git commit 404e382c28d50e7008ef1c1d5b7095412f9de5d8 Author: Dmitry Lychagin AuthorDate: Mon May 3 17:42:16 2021 -0700 [ASTERIXDB-2883][COMP] Improve null handling in UDF calls - user model changes: no - storage format changes: no - interface changes: no - IntroduceDynamicTypeCastForExternalFunctionRule should not apply twice on the same operator - Add null-call handling to ExternalTypeComputer and ExternalScalarJavaFunctionEvaluator Change-Id: I9c58e28f673606c5d0413bdc4cd3ba0c7c20eb8d Reviewed-on: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/11306 Integration-Tests: Jenkins Tested-by: Jenkins Reviewed-by: Dmitry Lychagin Reviewed-by: Ian Maxon --- ...duceDynamicTypeCastForExternalFunctionRule.java | 9 - .../type_name/type_name.0.ddl.sqlpp| 20 ++ .../type_name/type_name.1.lib.sqlpp| 20 ++ .../type_name/type_name.2.ddl.sqlpp| 25 .../type_name/type_name.3.query.sqlpp | 28 + .../type_name/type_name.4.ddl.sqlpp| 20 ++ .../external-library/type_name/type_name.3.adm | 1 + .../resources/runtimets/testsuite_it_sqlpp.xml | 5 +++ .../ExternalScalarJavaFunctionEvaluator.java | 45 ++--- .../asterix/external/library/TypeNameFactory.java | 30 ++ .../asterix/external/library/TypeNameFunction.java | 46 ++ .../functions/ExternalFunctionCompilerUtil.java| 2 +- .../metadata/functions/ExternalTypeComputer.java | 13 -- 13 files changed, 244 insertions(+), 20 deletions(-) diff --git a/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/IntroduceDynamicTypeCastForExternalFunctionRule.java b/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/IntroduceDynamicTypeCastForExternalFunctionRule.java index d3eb0c2..2c5a07d 100644 --- a/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/IntroduceDynamicTypeCastForExternalFunctionRule.java +++ b/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/IntroduceDynamicTypeCastForExternalFunctionRule.java @@ -115,6 +115,13 @@ public class IntroduceDynamicTypeCastForExternalFunctionRule implements IAlgebra if (op.getOperatorTag() != LogicalOperatorTag.ASSIGN) { return false; } -return op.acceptExpressionTransform(expr -> rewriteFunctionArgs(op, expr, context)); +if (context.checkIfInDontApplySet(this, op)) { +return false; +} +boolean applied = op.acceptExpressionTransform(expr -> rewriteFunctionArgs(op, expr, context)); +if (applied) { +context.addToDontApplySet(this, op); +} +return applied; } } diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/external-library/type_name/type_name.0.ddl.sqlpp b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/external-library/type_name/type_name.0.ddl.sqlpp new file mode 100644 index 000..76cc70d --- /dev/null +++ b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/external-library/type_name/type_name.0.ddl.sqlpp @@ -0,0 +1,20 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ +DROP DATAVERSE externallibtest if exists; +CREATE DATAVERSE externallibtest; diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/external-library/type_name/type_name.1.lib.sqlpp b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/external-library/type_name/type_name.1.lib.sqlpp new file mode 100644 index 000..b2bf929 --- /dev/null +++ b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/external-library/type_name/type_name.1.lib.sqlpp @@ -0,0 +1,20 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + *
[asterixdb] 07/38: [ASTERIXDB-2886][COMP] Fix RemoveRedundantVariablesRule
This is an automated email from the ASF dual-hosted git repository. imaxon pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/asterixdb.git commit 2323a4cd9b8a9c278797aefdff61552836d32a9c Author: Dmitry Lychagin AuthorDate: Tue Apr 20 14:38:53 2021 -0700 [ASTERIXDB-2886][COMP] Fix RemoveRedundantVariablesRule - user model changes: no - storage format changes: no - interface changes: no Details: - Change RemoveRedundantVariablesRule to operate on the whole plan at once instead of working incrementally on each operator Change-Id: Ia6cec065614ad5e2b14e7b160cf3c9de7215618a Reviewed-on: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/11103 Integration-Tests: Jenkins Tested-by: Jenkins Reviewed-by: Dmitry Lychagin Reviewed-by: Ali Alsuliman --- .../query-ASTERIXDB-2886.1.ddl.sqlpp | 43 ++ .../query-ASTERIXDB-2886.2.ddl.sqlpp | 43 ++ .../query-ASTERIXDB-2886.3.query.sqlpp | 30 +++ .../push-limit-to-primary-scan-select.11.adm | 4 +- .../push-limit-to-primary-scan.7.adm | 4 +- .../query-ASTERIXDB-2886.3.adm | 4 + .../results/union/union_opt_1/union_opt_1.11.adm | 8 +- .../test/resources/runtimets/testsuite_sqlpp.xml | 5 ++ .../rules/RemoveRedundantVariablesRule.java| 94 -- 9 files changed, 186 insertions(+), 49 deletions(-) diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/query-ASTERIXDB-2886/query-ASTERIXDB-2886.1.ddl.sqlpp b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/query-ASTERIXDB-2886/query-ASTERIXDB-2886.1.ddl.sqlpp new file mode 100644 index 000..ab6b723 --- /dev/null +++ b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/query-ASTERIXDB-2886/query-ASTERIXDB-2886.1.ddl.sqlpp @@ -0,0 +1,43 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ + +drop dataverse test1 if exists; +create dataverse test1; +use test1; + +create dataset a( + c_a1 bigint not unknown, + c_a2 bigint, + c_a3 string +) primary key c_a1; + +create index ia2 on a(c_a2); + +create index ia3 on a(c_a3); + +create dataset b( + c_b1 bigint not unknown, + c_b2 bigint, + c_b3 string +) primary key c_b1; + +create index ib2 on b(c_b2); + +create index ib3 on b(c_b3); + diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/query-ASTERIXDB-2886/query-ASTERIXDB-2886.2.ddl.sqlpp b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/query-ASTERIXDB-2886/query-ASTERIXDB-2886.2.ddl.sqlpp new file mode 100644 index 000..621d057 --- /dev/null +++ b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/misc/query-ASTERIXDB-2886/query-ASTERIXDB-2886.2.ddl.sqlpp @@ -0,0 +1,43 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ + +drop dataverse test2 if exists; +create dataverse test2; +use test2; + +create dataset a( + c_a1 bigint not unknown, + c_a2 bigint, + c_a3 string +) primary key c_a1; + +create index ia2 on a(c_a2); + +create index ia3 on a(c_a3); + +create dataset b( + c_b1 bigint not unknown, + c_b2 bigint, + c_b3 string +) primary key c_b1; + +create index ib2 on b(c_b2); + +create index ib3 on b(c_b3); + diff --git a/asterixdb/asterix-app/src/test/reso
[asterixdb] 01/38: [NO ISSUE][HYR] Increase thread name brevity
This is an automated email from the ASF dual-hosted git repository. imaxon pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/asterixdb.git commit 40be6e72254ea9a2192041eb357435bc838dee93 Author: Michael Blow AuthorDate: Tue Apr 13 17:28:31 2021 -0400 [NO ISSUE][HYR] Increase thread name brevity Change-Id: I55a7586980cf869edd383ab435952f10b3976b86 Reviewed-on: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/11045 Integration-Tests: Jenkins Tested-by: Jenkins Reviewed-by: Michael Blow Reviewed-by: Murtadha Hubail --- .../org/apache/hyracks/api/rewriter/runtime/SuperActivity.java | 5 + .../api/rewriter/runtime/SuperActivityOperatorNodePushable.java| 7 +++ 2 files changed, 8 insertions(+), 4 deletions(-) diff --git a/hyracks-fullstack/hyracks/hyracks-api/src/main/java/org/apache/hyracks/api/rewriter/runtime/SuperActivity.java b/hyracks-fullstack/hyracks/hyracks-api/src/main/java/org/apache/hyracks/api/rewriter/runtime/SuperActivity.java index ddcfd78..55eaf68 100644 --- a/hyracks-fullstack/hyracks/hyracks-api/src/main/java/org/apache/hyracks/api/rewriter/runtime/SuperActivity.java +++ b/hyracks-fullstack/hyracks/hyracks-api/src/main/java/org/apache/hyracks/api/rewriter/runtime/SuperActivity.java @@ -54,6 +54,11 @@ public class SuperActivity extends OneToOneConnectedActivityCluster implements I } @Override +public String getDisplayName() { +return "SA"; +} + +@Override public IOperatorNodePushable createPushRuntime(IHyracksTaskContext ctx, final IRecordDescriptorProvider recordDescProvider, final int partition, final int nPartitions) throws HyracksDataException { diff --git a/hyracks-fullstack/hyracks/hyracks-api/src/main/java/org/apache/hyracks/api/rewriter/runtime/SuperActivityOperatorNodePushable.java b/hyracks-fullstack/hyracks/hyracks-api/src/main/java/org/apache/hyracks/api/rewriter/runtime/SuperActivityOperatorNodePushable.java index 79687ab..e43d72a 100644 --- a/hyracks-fullstack/hyracks/hyracks-api/src/main/java/org/apache/hyracks/api/rewriter/runtime/SuperActivityOperatorNodePushable.java +++ b/hyracks-fullstack/hyracks/hyracks-api/src/main/java/org/apache/hyracks/api/rewriter/runtime/SuperActivityOperatorNodePushable.java @@ -55,7 +55,7 @@ import org.apache.hyracks.api.util.ExceptionUtils; * connected activities in a single thread. */ public class SuperActivityOperatorNodePushable implements IOperatorNodePushable { -private static final String CLASS_ABBREVIATION = "SA"; +private static final String CLASS_ABBREVIATION = "SAO"; private final Map operatorNodePushables = new HashMap<>(); private final List operatorNodePushablesBFSOrder = new ArrayList<>(); private final Map startActivities; @@ -222,9 +222,8 @@ public class SuperActivityOperatorNodePushable implements IOperatorNodePushable tasks.add(ctx.getExecutorService().submit(() -> { startSemaphore.release(); try { -Thread.currentThread() -.setName(Thread.currentThread().getName() + ":" + ctx.getJobletContext().getJobId() -+ ":" + ctx.getTaskAttemptId() + ":" + CLASS_ABBREVIATION); +Thread.currentThread().setName(CLASS_ABBREVIATION + ":" + ctx.getJobletContext().getJobId() ++ ":" + ctx.getTaskAttemptId()); action.run(op); } catch (Throwable th) { // NOSONAR: Must catch all causes of failure failures.offer(th);
[asterixdb] 12/38: [NO ISSUE][MISC] Ensure safe serialVersionUIDs for anon. inners
This is an automated email from the ASF dual-hosted git repository. imaxon pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/asterixdb.git commit 7dfb0b3625c4813e26ae6c4b4eb5b48823d6afc6 Author: Michael Blow AuthorDate: Fri Apr 23 08:26:56 2021 -0400 [NO ISSUE][MISC] Ensure safe serialVersionUIDs for anon. inners - user model changes: no - storage format changes: no - interface changes: yes Ensure anonymous inner classes have unique & non-trivial serialVersionUIDs when multiples exist of a given type in an enclosing class, to avoid making hard-to-detect binary-incompatible changes Change-Id: I073c8a820f76bea9e73064a9bca350c63f65a698 Reviewed-on: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/11163 Integration-Tests: Jenkins Reviewed-by: Murtadha Hubail Tested-by: Jenkins --- .../org/apache/asterix/om/types/BuiltinType.java | 60 +++--- .../apache/hyracks/control/cc/job/JobManager.java | 4 +- 2 files changed, 32 insertions(+), 32 deletions(-) diff --git a/asterixdb/asterix-om/src/main/java/org/apache/asterix/om/types/BuiltinType.java b/asterixdb/asterix-om/src/main/java/org/apache/asterix/om/types/BuiltinType.java index 76ba6a2..bdac9e9 100644 --- a/asterixdb/asterix-om/src/main/java/org/apache/asterix/om/types/BuiltinType.java +++ b/asterixdb/asterix-om/src/main/java/org/apache/asterix/om/types/BuiltinType.java @@ -45,7 +45,7 @@ public abstract class BuiltinType implements IAType { /** the type of all types */ public static final BuiltinType ALL_TYPE = new BuiltinType() { -private static final long serialVersionUID = 1L; +private static final long serialVersionUID = -6449893468153263063L; @Override public ATypeTag getTypeTag() { @@ -83,7 +83,7 @@ public abstract class BuiltinType implements IAType { public static final BuiltinType AINT8 = new LowerCaseConstructorType() { -private static final long serialVersionUID = 1L; +private static final long serialVersionUID = -3548486253352636441L; @Override public String getDisplayName() { @@ -116,7 +116,7 @@ public abstract class BuiltinType implements IAType { public static final BuiltinType AINT16 = new LowerCaseConstructorType() { -private static final long serialVersionUID = 1L; +private static final long serialVersionUID = 7372637344359144146L; @Override public String getDisplayName() { @@ -149,7 +149,7 @@ public abstract class BuiltinType implements IAType { public static final BuiltinType AINT32 = new LowerCaseConstructorType() { -private static final long serialVersionUID = 1L; +private static final long serialVersionUID = -2019098705812691759L; @Override public String getDisplayName() { @@ -182,7 +182,7 @@ public abstract class BuiltinType implements IAType { public static final BuiltinType AINT64 = new LowerCaseConstructorType() { -private static final long serialVersionUID = 1L; +private static final long serialVersionUID = -1512324312146380842L; @Override public ATypeTag getTypeTag() { @@ -215,7 +215,7 @@ public abstract class BuiltinType implements IAType { public static final BuiltinType ABINARY = new LowerCaseConstructorType() { -private static final long serialVersionUID = 1L; +private static final long serialVersionUID = 6267661034873713699L; @Override public String getDisplayName() { @@ -248,7 +248,7 @@ public abstract class BuiltinType implements IAType { public static final BuiltinType AFLOAT = new LowerCaseConstructorType() { -private static final long serialVersionUID = 1L; +private static final long serialVersionUID = 3677347396583352706L; @Override public String getDisplayName() { @@ -281,7 +281,7 @@ public abstract class BuiltinType implements IAType { public static final BuiltinType ADOUBLE = new LowerCaseConstructorType() { -private static final long serialVersionUID = 1L; +private static final long serialVersionUID = -4541634684505456336L; @Override public String getDisplayName() { @@ -314,7 +314,7 @@ public abstract class BuiltinType implements IAType { public static final BuiltinType ASTRING = new LowerCaseConstructorType() { -private static final long serialVersionUID = 1L; +private static final long serialVersionUID = -7772076532516609205L; @Override public String getDisplayName() { @@ -347,7 +347,7 @@ public abstract class BuiltinType implements IAType { public static final BuiltinType AMISSING = new LowerCaseConstructorType() { -private static final long serialVersionUID = 1L; +private static final long serialVersionUID = 7776297133205477953L; @Over
[asterixdb] 24/38: [NO ISSUE][EXT] Refactor JSONDataParser
This is an automated email from the ASF dual-hosted git repository. imaxon pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/asterixdb.git commit 91f91d25427257fac34ec17386f2ad5113c08242 Author: Michael Blow AuthorDate: Thu May 6 16:06:47 2021 -0400 [NO ISSUE][EXT] Refactor JSONDataParser Change-Id: Ia47f2fe449cfb0ca8e6e3a6308ef01ee6810ec03 Reviewed-on: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/11363 Integration-Tests: Jenkins Tested-by: Jenkins Reviewed-by: Michael Blow Reviewed-by: Till Westmann --- ...DataParser.java => AbstractJsonDataParser.java} | 78 +--- .../asterix/external/parser/JSONDataParser.java| 435 + 2 files changed, 27 insertions(+), 486 deletions(-) diff --git a/asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/parser/JSONDataParser.java b/asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/parser/AbstractJsonDataParser.java similarity index 85% copy from asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/parser/JSONDataParser.java copy to asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/parser/AbstractJsonDataParser.java index b2036c0..a5d5a08 100644 --- a/asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/parser/JSONDataParser.java +++ b/asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/parser/AbstractJsonDataParser.java @@ -20,7 +20,6 @@ package org.apache.asterix.external.parser; import java.io.DataOutput; import java.io.IOException; -import java.io.InputStream; import java.util.BitSet; import java.util.function.LongSupplier; import java.util.function.Supplier; @@ -29,9 +28,6 @@ import org.apache.asterix.builders.IARecordBuilder; import org.apache.asterix.builders.IAsterixListBuilder; import org.apache.asterix.common.exceptions.ErrorCode; import org.apache.asterix.common.exceptions.RuntimeDataException; -import org.apache.asterix.external.api.IRawRecord; -import org.apache.asterix.external.api.IRecordDataParser; -import org.apache.asterix.external.api.IStreamDataParser; import org.apache.asterix.external.parser.jackson.ADMToken; import org.apache.asterix.external.parser.jackson.GeometryCoParser; import org.apache.asterix.external.parser.jackson.ParserContext; @@ -56,21 +52,18 @@ import com.fasterxml.jackson.core.JsonParseException; import com.fasterxml.jackson.core.JsonParser; import com.fasterxml.jackson.core.JsonStreamContext; import com.fasterxml.jackson.core.JsonToken; -import com.fasterxml.jackson.databind.JsonNode; -import com.fasterxml.jackson.databind.node.TreeTraversingParser; /** - * JSON format parser using Jakson parser. + * JSON format parser using Jackson parser. */ -public class JSONDataParser extends AbstractNestedDataParser -implements IStreamDataParser, IRecordDataParser { +public abstract class AbstractJsonDataParser extends AbstractNestedDataParser { protected final ParserContext parserContext; protected final JsonFactory jsonFactory; protected final ARecordType rootType; protected final GeometryCoParser geometryCoParser; -private Supplier dataSourceName; -private LongSupplier lineNumber; +protected Supplier dataSourceName = ExternalDataConstants.EMPTY_STRING; +protected LongSupplier lineNumber = ExternalDataConstants.NO_LINES; protected JsonParser jsonParser; @@ -82,15 +75,13 @@ public class JSONDataParser extends AbstractNestedDataParser * @param jsonFactory *Jackson JSON parser factory. */ -public JSONDataParser(ARecordType recordType, JsonFactory jsonFactory) { +public AbstractJsonDataParser(ARecordType recordType, JsonFactory jsonFactory) { // recordType currently cannot be null, however this is to guarantee for any future changes. this.rootType = recordType != null ? recordType : RecordUtil.FULLY_OPEN_RECORD_TYPE; this.jsonFactory = jsonFactory; -//GeometyCoParser to parse GeoJSON objects to AsterixDB internal spatial types. +//GeometryCoParser to parse GeoJSON objects to AsterixDB internal spatial types. geometryCoParser = new GeometryCoParser(jsonParser); parserContext = new ParserContext(); -this.dataSourceName = ExternalDataConstants.EMPTY_STRING; -this.lineNumber = ExternalDataConstants.NO_LINES; } /* @@ -99,53 +90,6 @@ public class JSONDataParser extends AbstractNestedDataParser */ -@Override -public void configure(Supplier dataSourceName, LongSupplier lineNumber) { -this.dataSourceName = dataSourceName == null ? ExternalDataConstants.EMPTY_STRING : dataSourceName; -this.lineNumber = lineNumber == null ? ExternalDataConstants.NO_LINES : lineNumber; -} - -@Override -public final bool
[asterixdb] 25/38: [NO ISSUE][*DB] Enable large parsing resources to be freed on memory pressure
This is an automated email from the ASF dual-hosted git repository. imaxon pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/asterixdb.git commit 5d02b03854911d7fb889b078f0ada37b282261f9 Author: Michael Blow AuthorDate: Thu May 6 16:36:30 2021 -0400 [NO ISSUE][*DB] Enable large parsing resources to be freed on memory pressure Change-Id: Ieaca566f765a5d3531baff8128d6ff03c95956e2 Reviewed-on: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/11365 Integration-Tests: Jenkins Tested-by: Jenkins Reviewed-by: Michael Blow Reviewed-by: Till Westmann --- .../{ObjectPool.java => AbstractObjectPool.java} | 28 +++- .../external/parser/jackson/IObjectPool.java | 25 +- .../external/parser/jackson/ObjectPool.java| 34 +-- .../external/parser/jackson/ParserContext.java | 8 ++--- .../{ObjectPool.java => SoftObjectPool.java} | 38 +++--- .../apache/hyracks/util/string/UTF8StringUtil.java | 11 --- .../hyracks/util/string/UTF8StringWriter.java | 3 +- 7 files changed, 52 insertions(+), 95 deletions(-) diff --git a/asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/parser/jackson/ObjectPool.java b/asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/parser/jackson/AbstractObjectPool.java similarity index 75% copy from asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/parser/jackson/ObjectPool.java copy to asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/parser/jackson/AbstractObjectPool.java index 8945e71..ef75688 100644 --- a/asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/parser/jackson/ObjectPool.java +++ b/asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/parser/jackson/AbstractObjectPool.java @@ -27,39 +27,35 @@ import org.apache.asterix.om.util.container.IObjectFactory; * Object pool for DFS traversal mode, which allows to recycle objects * as soon as it is not needed. */ -public class ObjectPool { +public abstract class AbstractObjectPool implements IObjectPool { private final IObjectFactory objectFactory; -private final Queue recycledObjects; -private final T element; +private final Queue recycledObjects; +private final T param; -public ObjectPool() { -this(null, null); -} - -public ObjectPool(IObjectFactory objectFactory) { -this(objectFactory, null); -} - -public ObjectPool(IObjectFactory objectFactory, T element) { +protected AbstractObjectPool(IObjectFactory objectFactory, T param) { this.objectFactory = objectFactory; recycledObjects = new ArrayDeque<>(); -this.element = element; +this.param = param; } public E getInstance() { -E instance = recycledObjects.poll(); +E instance = unwrap(recycledObjects.poll()); if (objectFactory != null && instance == null) { -instance = objectFactory.create(element); +instance = objectFactory.create(param); } return instance; } public void recycle(E object) { if (object != null) { -recycledObjects.add(object); +recycledObjects.add(wrap(object)); } } +protected abstract E unwrap(Q element); + +protected abstract Q wrap(E element); + @Override public String toString() { return recycledObjects.toString(); diff --git a/hyracks-fullstack/hyracks/hyracks-util/src/main/java/org/apache/hyracks/util/string/UTF8StringWriter.java b/asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/parser/jackson/IObjectPool.java similarity index 57% copy from hyracks-fullstack/hyracks/hyracks-util/src/main/java/org/apache/hyracks/util/string/UTF8StringWriter.java copy to asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/parser/jackson/IObjectPool.java index 563f865..a2990ec 100644 --- a/hyracks-fullstack/hyracks/hyracks-util/src/main/java/org/apache/hyracks/util/string/UTF8StringWriter.java +++ b/asterixdb/asterix-external-data/src/main/java/org/apache/asterix/external/parser/jackson/IObjectPool.java @@ -16,23 +16,14 @@ * specific language governing permissions and limitations * under the License. */ -package org.apache.hyracks.util.string; +package org.apache.asterix.external.parser.jackson; -import java.io.DataOutput; -import java.io.IOException; -import java.io.Serializable; - -public class UTF8StringWriter implements Serializable { - -private static final long serialVersionUID = 1L; -transient byte[] tempBytes; - -public final void writeUTF8(CharSequence str, DataOutput out) throws IOException { -UTF8StringUtil.writeUTF8(str, out, this); -} - -public final void writeUTF8(char[] buffer, int start, int length, Da
[asterixdb] 28/38: [NO ISSUE] Fixed storage unit util false positives + added tests
This is an automated email from the ASF dual-hosted git repository. imaxon pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/asterixdb.git commit d5de9eb65fa935076d0bb82094d3fdf5357ea704 Author: Hussain Towaileb AuthorDate: Tue May 11 17:09:05 2021 +0300 [NO ISSUE] Fixed storage unit util false positives + added tests Change-Id: Ic263c4fea0de15803811c88e4d6b01df21e083c8 Reviewed-on: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/11303 Integration-Tests: Jenkins Tested-by: Jenkins Reviewed-by: Michael Blow --- .../java/org/apache/hyracks/util/StorageUtil.java | 53 ++- .../org/apache/hyracks/util/StorageUnitTest.java | 79 ++ 2 files changed, 99 insertions(+), 33 deletions(-) diff --git a/hyracks-fullstack/hyracks/hyracks-util/src/main/java/org/apache/hyracks/util/StorageUtil.java b/hyracks-fullstack/hyracks/hyracks-util/src/main/java/org/apache/hyracks/util/StorageUtil.java index e4969f0..0456dc7 100644 --- a/hyracks-fullstack/hyracks/hyracks-util/src/main/java/org/apache/hyracks/util/StorageUtil.java +++ b/hyracks-fullstack/hyracks/hyracks-util/src/main/java/org/apache/hyracks/util/StorageUtil.java @@ -20,10 +20,13 @@ package org.apache.hyracks.util; import java.util.HashMap; import java.util.Map; +import java.util.regex.Matcher; +import java.util.regex.Pattern; public class StorageUtil { public static final int BASE = 1024; +private static final Pattern PATTERN = Pattern.compile("^(-?[.0-9]+)([A-Z]{0,2})$"); public enum StorageUnit { BYTE("B", "b", 1), @@ -96,43 +99,27 @@ public class StorageUtil { * @throws IllegalArgumentException */ public static double getSizeInBytes(String s) { -String sSpaceRemoved = s.replaceAll(" ", ""); -String sUpper = sSpaceRemoved.toUpperCase(); - -// Default type -StorageUtil.StorageUnit unitType; - -// If the length is 1, it should only contain a digit number. -if (sUpper.length() == 1) { -if (Character.isDigit(sUpper.charAt(0))) { -unitType = StorageUnit.BYTE; -} else { -throw invalidFormatException(s); -} -} else if (sUpper.length() > 1) { -String checkStr = sUpper.substring(sUpper.length() - 2); -unitType = StorageUnit.lookupBySuffix(checkStr); - -if (unitType == null) { -// The last suffix should be at least "B" or a digit to be qualified as byte unit string. -char lastChar = sUpper.charAt(sUpper.length() - 1); -if (sUpper.substring(sUpper.length() - 1).equals(StorageUnit.BYTE.toString()) -|| Character.isDigit(lastChar)) { -unitType = StorageUnit.BYTE; -} else { -throw invalidFormatException(s); -} -} -} else { -// String length is zero. We can't parse this string. +String valueAndUnit = s.replace(" ", "").toUpperCase(); +Matcher matcher = PATTERN.matcher(valueAndUnit); +if (!matcher.find()) { throw invalidFormatException(s); } -// Strip all unit suffixes such as KB, MB ... -String sFinalVal = sUpper.replaceAll("[^-\\.0123456789]", ""); +String value = matcher.group(1); +String unit = matcher.group(2); + +// Default to bytes or find provided unit +StorageUnit unitType = !unit.isEmpty() ? StorageUnit.lookupBySuffix(unit) : StorageUnit.BYTE; +if (unitType == null) { +throw invalidFormatException(s); +} -// Return the bytes. -return unitType.toBytes(Double.parseDouble(sFinalVal)); +try { +// Return the bytes. +return unitType.toBytes(Double.parseDouble(value)); +} catch (NumberFormatException ex) { +throw invalidFormatException(s); +} } private static IllegalArgumentException invalidFormatException(String s) { diff --git a/hyracks-fullstack/hyracks/hyracks-util/src/test/java/org/apache/hyracks/util/StorageUnitTest.java b/hyracks-fullstack/hyracks/hyracks-util/src/test/java/org/apache/hyracks/util/StorageUnitTest.java new file mode 100644 index 000..445d15f --- /dev/null +++ b/hyracks-fullstack/hyracks/hyracks-util/src/test/java/org/apache/hyracks/util/StorageUnitTest.java @@ -0,0 +1,79 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except
[asterixdb] 22/38: [NO ISSUE][OTH] Only order nodes in node group for new datasets
This is an automated email from the ASF dual-hosted git repository. imaxon pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/asterixdb.git commit 30c4a78fcecf8d81a3cc070e348a38cae5600d1a Author: Ali Alsuliman AuthorDate: Thu May 6 01:39:41 2021 +0300 [NO ISSUE][OTH] Only order nodes in node group for new datasets - user model changes: no - storage format changes: no - interface changes: no Details: - Revert the change that sorts the cluster partitions. - Order the nodes in the node group for newly created datasets. - Make dump index function use the same locations as the index split files in the index dataflow helper. Change-Id: Id90a9c88ed44655631e44155c737459e4501c081 Reviewed-on: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/11343 Integration-Tests: Jenkins Tested-by: Jenkins Reviewed-by: Michael Blow --- .../asterix/app/function/DumpIndexDatasource.java | 14 +++--- .../apache/asterix/app/function/DumpIndexRewriter.java | 6 +- .../apache/asterix/metadata/entities/NodeGroup.java| 18 +- .../org/apache/asterix/metadata/utils/DatasetUtil.java | 2 +- .../metadata/utils/SecondaryIndexOperationsHelper.java | 4 .../asterix/runtime/utils/ClusterStateManager.java | 2 -- 6 files changed, 30 insertions(+), 16 deletions(-) diff --git a/asterixdb/asterix-app/src/main/java/org/apache/asterix/app/function/DumpIndexDatasource.java b/asterixdb/asterix-app/src/main/java/org/apache/asterix/app/function/DumpIndexDatasource.java index 6318f8e..3351075 100644 --- a/asterixdb/asterix-app/src/main/java/org/apache/asterix/app/function/DumpIndexDatasource.java +++ b/asterixdb/asterix-app/src/main/java/org/apache/asterix/app/function/DumpIndexDatasource.java @@ -18,6 +18,7 @@ */ package org.apache.asterix.app.function; +import org.apache.asterix.common.cluster.IClusterStateManager; import org.apache.asterix.metadata.api.IDatasourceFunction; import org.apache.asterix.metadata.declared.DataSourceId; import org.apache.asterix.metadata.declared.FunctionDataSource; @@ -36,19 +37,26 @@ public class DumpIndexDatasource extends FunctionDataSource { private final IndexDataflowHelperFactory indexDataflowHelperFactory; private final RecordDescriptor recDesc; private final IBinaryComparatorFactory[] comparatorFactories; +private final AlgebricksAbsolutePartitionConstraint storageLocations; public DumpIndexDatasource(INodeDomain domain, IndexDataflowHelperFactory indexDataflowHelperFactory, -RecordDescriptor recDesc, IBinaryComparatorFactory[] comparatorFactories) throws AlgebricksException { +RecordDescriptor recDesc, IBinaryComparatorFactory[] comparatorFactories, +AlgebricksAbsolutePartitionConstraint storageLocations) throws AlgebricksException { super(DUMP_INDEX_DATASOURCE_ID, DumpIndexRewriter.DUMP_INDEX, domain); this.indexDataflowHelperFactory = indexDataflowHelperFactory; this.recDesc = recDesc; this.comparatorFactories = comparatorFactories; +this.storageLocations = storageLocations; +} + +@Override +protected AlgebricksAbsolutePartitionConstraint getLocations(IClusterStateManager csm) { +return storageLocations; } @Override protected IDatasourceFunction createFunction(MetadataProvider metadataProvider, AlgebricksAbsolutePartitionConstraint locations) { -return new DumpIndexFunction(metadataProvider.getClusterLocations(), indexDataflowHelperFactory, recDesc, -comparatorFactories); +return new DumpIndexFunction(locations, indexDataflowHelperFactory, recDesc, comparatorFactories); } } diff --git a/asterixdb/asterix-app/src/main/java/org/apache/asterix/app/function/DumpIndexRewriter.java b/asterixdb/asterix-app/src/main/java/org/apache/asterix/app/function/DumpIndexRewriter.java index 000e910..30eaf93 100644 --- a/asterixdb/asterix-app/src/main/java/org/apache/asterix/app/function/DumpIndexRewriter.java +++ b/asterixdb/asterix-app/src/main/java/org/apache/asterix/app/function/DumpIndexRewriter.java @@ -26,6 +26,7 @@ import org.apache.asterix.metadata.declared.MetadataProvider; import org.apache.asterix.metadata.entities.Dataset; import org.apache.asterix.metadata.entities.Index; import org.apache.asterix.metadata.utils.SecondaryIndexOperationsHelper; +import org.apache.hyracks.algebricks.common.constraints.AlgebricksAbsolutePartitionConstraint; import org.apache.hyracks.algebricks.common.exceptions.AlgebricksException; import org.apache.hyracks.algebricks.core.algebra.base.IOptimizationContext; import org.apache.hyracks.algebricks.core.algebra.expressions.AbstractFunctionCallExpression; @@ -67,7 +68,10 @@ public class DumpIndexRewriter extends FunctionRewriter { IndexDataflowHelperFactory indexDataflowHelperFactory
[asterixdb] 10/38: [NO ISSUE][API] Make the AbstractServlet look for /* wild card
This is an automated email from the ASF dual-hosted git repository. imaxon pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/asterixdb.git commit 0d88d1187acfe1902e766e1bfeec6c3d2d3384a5 Author: Ali Alsuliman AuthorDate: Thu Apr 22 17:46:12 2021 +0300 [NO ISSUE][API] Make the AbstractServlet look for /* wild card - user model changes: no - storage format changes: no - interface changes: no Details: Make the AbstractServlet look for /* wild card to determine the servlet path length Change-Id: I74d393ede9e3bcf91222e2aa0e4a501a48ac0cf3 Reviewed-on: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/11144 Integration-Tests: Jenkins Tested-by: Jenkins Reviewed-by: Michael Blow --- .../api/http/server/AbstractNCUdfServlet.java | 2 +- .../hyracks/http/server/AbstractServlet.java | 23 +++--- 2 files changed, 13 insertions(+), 12 deletions(-) diff --git a/asterixdb/asterix-app/src/main/java/org/apache/asterix/api/http/server/AbstractNCUdfServlet.java b/asterixdb/asterix-app/src/main/java/org/apache/asterix/api/http/server/AbstractNCUdfServlet.java index cf287d9..5a3cde6 100644 --- a/asterixdb/asterix-app/src/main/java/org/apache/asterix/api/http/server/AbstractNCUdfServlet.java +++ b/asterixdb/asterix-app/src/main/java/org/apache/asterix/api/http/server/AbstractNCUdfServlet.java @@ -134,7 +134,7 @@ public abstract class AbstractNCUdfServlet extends AbstractServlet { } URI createDownloadURI(Path file) throws Exception { -String path = paths[0].substring(0, trims[0]) + GET_UDF_DIST_ENDPOINT + '/' + file.getFileName(); +String path = paths[0].substring(0, servletPathLengths[0]) + GET_UDF_DIST_ENDPOINT + '/' + file.getFileName(); String host = getHyracksClientConnection().getHost(); return new URI(httpServerProtocol.toString(), null, host, httpServerPort, path, null, null); } diff --git a/hyracks-fullstack/hyracks/hyracks-http/src/main/java/org/apache/hyracks/http/server/AbstractServlet.java b/hyracks-fullstack/hyracks/hyracks-http/src/main/java/org/apache/hyracks/http/server/AbstractServlet.java index 514a7dd..ed567f5 100644 --- a/hyracks-fullstack/hyracks/hyracks-http/src/main/java/org/apache/hyracks/http/server/AbstractServlet.java +++ b/hyracks-fullstack/hyracks/hyracks-http/src/main/java/org/apache/hyracks/http/server/AbstractServlet.java @@ -53,20 +53,20 @@ public abstract class AbstractServlet implements IServlet { protected final String[] paths; protected final ConcurrentMap ctx; -protected final int[] trims; +protected final int[] servletPathLengths; public AbstractServlet(ConcurrentMap ctx, String... paths) { this.paths = paths; this.ctx = ctx; -trims = new int[paths.length]; +servletPathLengths = new int[paths.length]; for (int i = 0; i < paths.length; i++) { String path = paths[i]; if (path.endsWith("/*")) { -trims[i] = path.indexOf("/*"); +servletPathLengths[i] = path.indexOf("/*"); } else if (path.endsWith("/")) { -trims[i] = path.length() - 1; +servletPathLengths[i] = path.length() - 1; } else { -trims[i] = path.length(); +servletPathLengths[i] = path.length(); } } } @@ -175,26 +175,27 @@ public abstract class AbstractServlet implements IServlet { public String localPath(IServletRequest request) { final String uri = request.getHttpRequest().uri(); int queryStart = uri.indexOf('?'); -return queryStart == -1 ? uri.substring(trim(uri)) : uri.substring(trim(uri), queryStart); +return queryStart == -1 ? uri.substring(servletLength(uri)) : uri.substring(servletLength(uri), queryStart); } public String servletPath(IServletRequest request) { final String uri = request.getHttpRequest().uri(); -return uri.substring(0, trim(uri)); +return uri.substring(0, servletLength(uri)); } -protected int trim(final String uri) { +protected int servletLength(final String uri) { int trim = -1; if (paths.length > 1) { for (int i = 0; i < paths.length; i++) { -String path = paths[i].indexOf('*') >= 0 ? paths[i].substring(0, paths[i].indexOf('*')) : paths[i]; +int wildCardIdx = paths[i].indexOf("/*"); +String path = wildCardIdx >= 0 ? paths[i].substring(0, wildCardIdx) : paths[i]; if (uri.indexOf(path) == 0) { -trim = trims[i]; +trim = servletPathLengths[i]; break; } } } else { -trim = trims[0]; +trim = servletPathLengths[0]; } return trim; }
[asterixdb] branch master updated (1a290d1 -> 594780e)
This is an automated email from the ASF dual-hosted git repository. imaxon pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/asterixdb.git. from 1a290d1 [ASTERIXDB-2891] Remove WRITE OUTPUT new 4057591 Release Apache AsterixDB 0.9.6 and Hyracks 0.3.6 new 03ccded Advance to Apache AsterixDB 0.9.7-SNAPSHOT and Hyracks 0.3.7-SNAPSHOT new 518b70c Merge branch 'release-0.9.6-rc0' into 'master' new 594780e Merge "Merge branch 'release-0.9.6-rc0' into 'master'" The 4 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: asterixdb/asterix-active/pom.xml| 2 +- asterixdb/asterix-algebra/pom.xml | 2 +- asterixdb/asterix-app/pom.xml | 2 +- asterixdb/asterix-benchmark/pom.xml | 2 +- asterixdb/asterix-client-helper/pom.xml | 2 +- asterixdb/asterix-common/pom.xml| 2 +- asterixdb/asterix-coverage/pom.xml | 2 +- asterixdb/asterix-dashboard/pom.xml | 2 +- asterixdb/asterix-doc/pom.xml | 2 +- asterixdb/asterix-docker/pom.xml| 2 +- asterixdb/asterix-examples/pom.xml | 2 +- asterixdb/asterix-external-data/pom.xml | 2 +- asterixdb/asterix-fuzzyjoin/pom.xml | 2 +- asterixdb/asterix-geo/pom.xml | 2 +- asterixdb/asterix-hivecompat/pom.xml| 2 +- asterixdb/asterix-lang-common/pom.xml | 2 +- asterixdb/asterix-lang-sqlpp/pom.xml| 2 +- asterixdb/asterix-license/pom.xml | 2 +- .../asterix-grammar-extension-maven-plugin/pom.xml | 2 +- .../asterix-test-datagenerator-maven-plugin/pom.xml | 2 +- .../asterix-maven-plugins/lexer-generator-maven-plugin/pom.xml | 2 +- asterixdb/asterix-maven-plugins/pom.xml | 2 +- .../record-manager-generator-maven-plugin/pom.xml | 2 +- asterixdb/asterix-metadata/pom.xml | 2 +- asterixdb/asterix-om/pom.xml| 2 +- asterixdb/asterix-replication/pom.xml | 2 +- asterixdb/asterix-runtime/pom.xml | 2 +- asterixdb/asterix-server/pom.xml| 2 +- asterixdb/asterix-spidersilk/pom.xml| 2 +- asterixdb/asterix-test-framework/pom.xml| 2 +- asterixdb/asterix-tools/pom.xml | 2 +- asterixdb/asterix-transactions/pom.xml | 2 +- asterixdb/pom.xml | 6 +++--- hyracks-fullstack/algebricks/algebricks-common/pom.xml | 2 +- hyracks-fullstack/algebricks/algebricks-compiler/pom.xml| 2 +- hyracks-fullstack/algebricks/algebricks-core/pom.xml| 2 +- hyracks-fullstack/algebricks/algebricks-data/pom.xml| 2 +- hyracks-fullstack/algebricks/algebricks-rewriter/pom.xml| 2 +- hyracks-fullstack/algebricks/algebricks-runtime/pom.xml | 2 +- hyracks-fullstack/algebricks/algebricks-tests/pom.xml | 2 +- hyracks-fullstack/algebricks/pom.xml| 2 +- hyracks-fullstack/hyracks-fullstack-license/pom.xml | 2 +- hyracks-fullstack/hyracks/hyracks-api/pom.xml | 2 +- hyracks-fullstack/hyracks/hyracks-client/pom.xml| 2 +- hyracks-fullstack/hyracks/hyracks-comm/pom.xml | 2 +- .../hyracks/hyracks-control/hyracks-control-cc/pom.xml | 2 +- .../hyracks/hyracks-control/hyracks-control-common/pom.xml | 2 +- .../hyracks/hyracks-control/hyracks-control-nc/pom.xml | 2 +- .../hyracks/hyracks-control/hyracks-nc-service/pom.xml | 2 +- hyracks-fullstack/hyracks/hyracks-control/pom.xml | 2 +- hyracks-fullstack/hyracks/hyracks-data/hyracks-data-std/pom.xml | 2 +- hyracks-fullstack/hyracks/hyracks-data/pom.xml | 2 +- hyracks-fullstack/hyracks/hyracks-dataflow-common/pom.xml | 2 +- hyracks-fullstack/hyracks/hyracks-dataflow-std/pom.xml | 2 +- hyracks-fullstack/hyracks/hyracks-dist/
[asterixdb] 04/04: Merge "Merge branch 'release-0.9.6-rc0' into 'master'"
This is an automated email from the ASF dual-hosted git repository. imaxon pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/asterixdb.git commit 594780ef291757b485f47739e4972845b0805656 Merge: 1a290d1 518b70c Author: Apache AsterixDB Gerrit AuthorDate: Tue Jun 1 17:40:58 2021 + Merge "Merge branch 'release-0.9.6-rc0' into 'master'" asterixdb/asterix-active/pom.xml| 2 +- asterixdb/asterix-algebra/pom.xml | 2 +- asterixdb/asterix-app/pom.xml | 2 +- asterixdb/asterix-benchmark/pom.xml | 2 +- asterixdb/asterix-client-helper/pom.xml | 2 +- asterixdb/asterix-common/pom.xml| 2 +- asterixdb/asterix-coverage/pom.xml | 2 +- asterixdb/asterix-dashboard/pom.xml | 2 +- asterixdb/asterix-doc/pom.xml | 2 +- asterixdb/asterix-docker/pom.xml| 2 +- asterixdb/asterix-examples/pom.xml | 2 +- asterixdb/asterix-external-data/pom.xml | 2 +- asterixdb/asterix-fuzzyjoin/pom.xml | 2 +- asterixdb/asterix-geo/pom.xml | 2 +- asterixdb/asterix-hivecompat/pom.xml| 2 +- asterixdb/asterix-lang-common/pom.xml | 2 +- asterixdb/asterix-lang-sqlpp/pom.xml| 2 +- asterixdb/asterix-license/pom.xml | 2 +- .../asterix-grammar-extension-maven-plugin/pom.xml | 2 +- .../asterix-test-datagenerator-maven-plugin/pom.xml | 2 +- .../asterix-maven-plugins/lexer-generator-maven-plugin/pom.xml | 2 +- asterixdb/asterix-maven-plugins/pom.xml | 2 +- .../record-manager-generator-maven-plugin/pom.xml | 2 +- asterixdb/asterix-metadata/pom.xml | 2 +- asterixdb/asterix-om/pom.xml| 2 +- asterixdb/asterix-replication/pom.xml | 2 +- asterixdb/asterix-runtime/pom.xml | 2 +- asterixdb/asterix-server/pom.xml| 2 +- asterixdb/asterix-spidersilk/pom.xml| 2 +- asterixdb/asterix-test-framework/pom.xml| 2 +- asterixdb/asterix-tools/pom.xml | 2 +- asterixdb/asterix-transactions/pom.xml | 2 +- asterixdb/pom.xml | 6 +++--- hyracks-fullstack/algebricks/algebricks-common/pom.xml | 2 +- hyracks-fullstack/algebricks/algebricks-compiler/pom.xml| 2 +- hyracks-fullstack/algebricks/algebricks-core/pom.xml| 2 +- hyracks-fullstack/algebricks/algebricks-data/pom.xml| 2 +- hyracks-fullstack/algebricks/algebricks-rewriter/pom.xml| 2 +- hyracks-fullstack/algebricks/algebricks-runtime/pom.xml | 2 +- hyracks-fullstack/algebricks/algebricks-tests/pom.xml | 2 +- hyracks-fullstack/algebricks/pom.xml| 2 +- hyracks-fullstack/hyracks-fullstack-license/pom.xml | 2 +- hyracks-fullstack/hyracks/hyracks-api/pom.xml | 2 +- hyracks-fullstack/hyracks/hyracks-client/pom.xml| 2 +- hyracks-fullstack/hyracks/hyracks-comm/pom.xml | 2 +- .../hyracks/hyracks-control/hyracks-control-cc/pom.xml | 2 +- .../hyracks/hyracks-control/hyracks-control-common/pom.xml | 2 +- .../hyracks/hyracks-control/hyracks-control-nc/pom.xml | 2 +- .../hyracks/hyracks-control/hyracks-nc-service/pom.xml | 2 +- hyracks-fullstack/hyracks/hyracks-control/pom.xml | 2 +- hyracks-fullstack/hyracks/hyracks-data/hyracks-data-std/pom.xml | 2 +- hyracks-fullstack/hyracks/hyracks-data/pom.xml | 2 +- hyracks-fullstack/hyracks/hyracks-dataflow-common/pom.xml | 2 +- hyracks-fullstack/hyracks/hyracks-dataflow-std/pom.xml | 2 +- hyracks-fullstack/hyracks/hyracks-dist/pom.xml | 2 +- hyracks-fullstack/hyracks/hyracks-documentation/pom.xml | 2 +- .../hyracks/hyracks-examples/btree-example/btreeclient/pom.xml | 2 +- .../hyracks/hyracks-examples/btree-example/btreehelper/pom.xml | 2 +- .../hyracks/hyracks-examples/btree-example/btreeserver/pom.xml | 2 +- hyracks-fullstack/hyracks/hyracks-examples/btree-example/pom.xml| 2 +- .../hyracks/hyrack
[asterixdb] 01/04: Release Apache AsterixDB 0.9.6 and Hyracks 0.3.6
This is an automated email from the ASF dual-hosted git repository. imaxon pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/asterixdb.git commit 4057591291bea2cd933666fde59bd8763ae5608f Author: Ian Maxon AuthorDate: Sun Oct 4 23:11:25 2020 -0700 Release Apache AsterixDB 0.9.6 and Hyracks 0.3.6 Change-Id:I9f3696ce5a15d9a2b08d635c3dcf531f75bab41b Reviewed-on: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/8225 Integration-Tests: Jenkins Tested-by: Jenkins Reviewed-by: Ian Maxon --- asterixdb/asterix-active/pom.xml| 2 +- asterixdb/asterix-algebra/pom.xml | 2 +- asterixdb/asterix-app/pom.xml | 2 +- asterixdb/asterix-benchmark/pom.xml | 2 +- asterixdb/asterix-client-helper/pom.xml | 2 +- asterixdb/asterix-common/pom.xml| 2 +- asterixdb/asterix-coverage/pom.xml | 2 +- asterixdb/asterix-dashboard/pom.xml | 2 +- asterixdb/asterix-doc/pom.xml | 2 +- asterixdb/asterix-docker/pom.xml| 2 +- asterixdb/asterix-examples/pom.xml | 2 +- asterixdb/asterix-external-data/pom.xml | 2 +- asterixdb/asterix-fuzzyjoin/pom.xml | 2 +- asterixdb/asterix-geo/pom.xml | 2 +- asterixdb/asterix-hivecompat/pom.xml| 2 +- asterixdb/asterix-lang-aql/pom.xml | 2 +- asterixdb/asterix-lang-common/pom.xml | 2 +- asterixdb/asterix-lang-sqlpp/pom.xml| 2 +- asterixdb/asterix-license/pom.xml | 2 +- .../asterix-grammar-extension-maven-plugin/pom.xml | 2 +- .../asterix-test-datagenerator-maven-plugin/pom.xml | 2 +- .../asterix-maven-plugins/lexer-generator-maven-plugin/pom.xml | 2 +- asterixdb/asterix-maven-plugins/pom.xml | 2 +- .../record-manager-generator-maven-plugin/pom.xml | 2 +- asterixdb/asterix-metadata/pom.xml | 2 +- asterixdb/asterix-om/pom.xml| 2 +- asterixdb/asterix-replication/pom.xml | 2 +- asterixdb/asterix-runtime/pom.xml | 2 +- asterixdb/asterix-server/pom.xml| 2 +- asterixdb/asterix-spidersilk/pom.xml| 2 +- asterixdb/asterix-test-framework/pom.xml| 2 +- asterixdb/asterix-tools/pom.xml | 2 +- asterixdb/asterix-transactions/pom.xml | 2 +- asterixdb/pom.xml | 6 +++--- hyracks-fullstack/algebricks/algebricks-common/pom.xml | 2 +- hyracks-fullstack/algebricks/algebricks-compiler/pom.xml| 2 +- hyracks-fullstack/algebricks/algebricks-core/pom.xml| 2 +- hyracks-fullstack/algebricks/algebricks-data/pom.xml| 2 +- hyracks-fullstack/algebricks/algebricks-rewriter/pom.xml| 2 +- hyracks-fullstack/algebricks/algebricks-runtime/pom.xml | 2 +- hyracks-fullstack/algebricks/algebricks-tests/pom.xml | 2 +- hyracks-fullstack/algebricks/pom.xml| 2 +- hyracks-fullstack/hyracks-fullstack-license/pom.xml | 2 +- hyracks-fullstack/hyracks/hyracks-api/pom.xml | 2 +- hyracks-fullstack/hyracks/hyracks-client/pom.xml| 2 +- hyracks-fullstack/hyracks/hyracks-comm/pom.xml | 2 +- .../hyracks/hyracks-control/hyracks-control-cc/pom.xml | 2 +- .../hyracks/hyracks-control/hyracks-control-common/pom.xml | 2 +- .../hyracks/hyracks-control/hyracks-control-nc/pom.xml | 2 +- .../hyracks/hyracks-control/hyracks-nc-service/pom.xml | 2 +- hyracks-fullstack/hyracks/hyracks-control/pom.xml | 2 +- hyracks-fullstack/hyracks/hyracks-data/hyracks-data-std/pom.xml | 2 +- hyracks-fullstack/hyracks/hyracks-data/pom.xml | 2 +- hyracks-fullstack/hyracks/hyracks-dataflow-common/pom.xml | 2 +- hyracks-fullstack/hyracks/hyracks-dataflow-std/pom.xml | 2 +- hyracks-fullstack/hyracks/hyracks-dist/pom.xml | 2 +- hyracks-fullstack/hyracks/hyracks-documentation/pom.xml | 2 +- .../hyracks/hyracks-examples/btree-example/btreeclient/pom.xml | 2 +- .../hyracks/hyracks-examples
[asterixdb] 02/04: Advance to Apache AsterixDB 0.9.7-SNAPSHOT and Hyracks 0.3.7-SNAPSHOT
This is an automated email from the ASF dual-hosted git repository. imaxon pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/asterixdb.git commit 03ccdeda74ba41f0fff05af399d4ff737064cb6b Author: Ian Maxon AuthorDate: Sun Mar 7 18:35:16 2021 -0800 Advance to Apache AsterixDB 0.9.7-SNAPSHOT and Hyracks 0.3.7-SNAPSHOT Change-Id:I9f3696ce5a15d9a2b08d635c3dcf131f75bab41a Reviewed-on: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/10403 Integration-Tests: Jenkins Tested-by: Jenkins Reviewed-by: Ian Maxon --- asterixdb/NOTICE| 2 +- asterixdb/asterix-active/pom.xml| 2 +- asterixdb/asterix-algebra/pom.xml | 2 +- asterixdb/asterix-app/pom.xml | 2 +- asterixdb/asterix-benchmark/pom.xml | 2 +- asterixdb/asterix-client-helper/pom.xml | 2 +- asterixdb/asterix-common/pom.xml| 2 +- asterixdb/asterix-coverage/pom.xml | 2 +- asterixdb/asterix-dashboard/pom.xml | 2 +- asterixdb/asterix-doc/pom.xml | 2 +- asterixdb/asterix-docker/pom.xml| 2 +- asterixdb/asterix-examples/pom.xml | 2 +- asterixdb/asterix-external-data/pom.xml | 2 +- asterixdb/asterix-fuzzyjoin/pom.xml | 2 +- asterixdb/asterix-geo/pom.xml | 2 +- asterixdb/asterix-hivecompat/pom.xml| 2 +- asterixdb/asterix-lang-aql/pom.xml | 2 +- asterixdb/asterix-lang-common/pom.xml | 2 +- asterixdb/asterix-lang-sqlpp/pom.xml| 2 +- asterixdb/asterix-license/pom.xml | 2 +- .../asterix-grammar-extension-maven-plugin/pom.xml | 2 +- .../asterix-test-datagenerator-maven-plugin/pom.xml | 2 +- .../asterix-maven-plugins/lexer-generator-maven-plugin/pom.xml | 2 +- asterixdb/asterix-maven-plugins/pom.xml | 2 +- .../record-manager-generator-maven-plugin/pom.xml | 2 +- asterixdb/asterix-metadata/pom.xml | 2 +- asterixdb/asterix-om/pom.xml| 2 +- asterixdb/asterix-replication/pom.xml | 2 +- asterixdb/asterix-runtime/pom.xml | 2 +- asterixdb/asterix-server/pom.xml| 2 +- asterixdb/asterix-spidersilk/pom.xml| 2 +- asterixdb/asterix-test-framework/pom.xml| 2 +- asterixdb/asterix-tools/pom.xml | 2 +- asterixdb/asterix-transactions/pom.xml | 2 +- asterixdb/pom.xml | 6 +++--- hyracks-fullstack/NOTICE| 2 +- hyracks-fullstack/algebricks/algebricks-common/pom.xml | 2 +- hyracks-fullstack/algebricks/algebricks-compiler/pom.xml| 2 +- hyracks-fullstack/algebricks/algebricks-core/pom.xml| 2 +- hyracks-fullstack/algebricks/algebricks-data/pom.xml| 2 +- hyracks-fullstack/algebricks/algebricks-rewriter/pom.xml| 2 +- hyracks-fullstack/algebricks/algebricks-runtime/pom.xml | 2 +- hyracks-fullstack/algebricks/algebricks-tests/pom.xml | 2 +- hyracks-fullstack/algebricks/pom.xml| 2 +- hyracks-fullstack/hyracks-fullstack-license/pom.xml | 2 +- hyracks-fullstack/hyracks/hyracks-api/pom.xml | 2 +- hyracks-fullstack/hyracks/hyracks-client/pom.xml| 2 +- hyracks-fullstack/hyracks/hyracks-comm/pom.xml | 2 +- .../hyracks/hyracks-control/hyracks-control-cc/pom.xml | 2 +- .../hyracks/hyracks-control/hyracks-control-common/pom.xml | 2 +- .../hyracks/hyracks-control/hyracks-control-nc/pom.xml | 2 +- .../hyracks/hyracks-control/hyracks-nc-service/pom.xml | 2 +- hyracks-fullstack/hyracks/hyracks-control/pom.xml | 2 +- hyracks-fullstack/hyracks/hyracks-data/hyracks-data-std/pom.xml | 2 +- hyracks-fullstack/hyracks/hyracks-data/pom.xml | 2 +- hyracks-fullstack/hyracks/hyracks-dataflow-common/pom.xml | 2 +- hyracks-fullstack/hyracks/hyracks-dataflow-std/pom.xml | 2 +- hyracks-fullstack/hyracks/hyracks-dist/pom.xml | 2
[asterixdb] 03/04: Merge branch 'release-0.9.6-rc0' into 'master'
This is an automated email from the ASF dual-hosted git repository. imaxon pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/asterixdb.git commit 518b70cd140d77c39eb56cff2c7c2e2f3eede211 Merge: 24b3e50 03ccded Author: Ian Maxon AuthorDate: Thu May 27 13:21:31 2021 -0700 Merge branch 'release-0.9.6-rc0' into 'master' Change-Id: Id9780d72960f9094c29f7f5766185782169fe7ca asterixdb/asterix-active/pom.xml| 2 +- asterixdb/asterix-algebra/pom.xml | 2 +- asterixdb/asterix-app/pom.xml | 2 +- asterixdb/asterix-benchmark/pom.xml | 2 +- asterixdb/asterix-client-helper/pom.xml | 2 +- asterixdb/asterix-common/pom.xml| 2 +- asterixdb/asterix-coverage/pom.xml | 2 +- asterixdb/asterix-dashboard/pom.xml | 2 +- asterixdb/asterix-doc/pom.xml | 2 +- asterixdb/asterix-docker/pom.xml| 2 +- asterixdb/asterix-examples/pom.xml | 2 +- asterixdb/asterix-external-data/pom.xml | 2 +- asterixdb/asterix-fuzzyjoin/pom.xml | 2 +- asterixdb/asterix-geo/pom.xml | 2 +- asterixdb/asterix-hivecompat/pom.xml| 2 +- asterixdb/asterix-lang-common/pom.xml | 2 +- asterixdb/asterix-lang-sqlpp/pom.xml| 2 +- asterixdb/asterix-license/pom.xml | 2 +- .../asterix-grammar-extension-maven-plugin/pom.xml | 2 +- .../asterix-test-datagenerator-maven-plugin/pom.xml | 2 +- .../asterix-maven-plugins/lexer-generator-maven-plugin/pom.xml | 2 +- asterixdb/asterix-maven-plugins/pom.xml | 2 +- .../record-manager-generator-maven-plugin/pom.xml | 2 +- asterixdb/asterix-metadata/pom.xml | 2 +- asterixdb/asterix-om/pom.xml| 2 +- asterixdb/asterix-replication/pom.xml | 2 +- asterixdb/asterix-runtime/pom.xml | 2 +- asterixdb/asterix-server/pom.xml| 2 +- asterixdb/asterix-spidersilk/pom.xml| 2 +- asterixdb/asterix-test-framework/pom.xml| 2 +- asterixdb/asterix-tools/pom.xml | 2 +- asterixdb/asterix-transactions/pom.xml | 2 +- asterixdb/pom.xml | 6 +++--- hyracks-fullstack/algebricks/algebricks-common/pom.xml | 2 +- hyracks-fullstack/algebricks/algebricks-compiler/pom.xml| 2 +- hyracks-fullstack/algebricks/algebricks-core/pom.xml| 2 +- hyracks-fullstack/algebricks/algebricks-data/pom.xml| 2 +- hyracks-fullstack/algebricks/algebricks-rewriter/pom.xml| 2 +- hyracks-fullstack/algebricks/algebricks-runtime/pom.xml | 2 +- hyracks-fullstack/algebricks/algebricks-tests/pom.xml | 2 +- hyracks-fullstack/algebricks/pom.xml| 2 +- hyracks-fullstack/hyracks-fullstack-license/pom.xml | 2 +- hyracks-fullstack/hyracks/hyracks-api/pom.xml | 2 +- hyracks-fullstack/hyracks/hyracks-client/pom.xml| 2 +- hyracks-fullstack/hyracks/hyracks-comm/pom.xml | 2 +- .../hyracks/hyracks-control/hyracks-control-cc/pom.xml | 2 +- .../hyracks/hyracks-control/hyracks-control-common/pom.xml | 2 +- .../hyracks/hyracks-control/hyracks-control-nc/pom.xml | 2 +- .../hyracks/hyracks-control/hyracks-nc-service/pom.xml | 2 +- hyracks-fullstack/hyracks/hyracks-control/pom.xml | 2 +- hyracks-fullstack/hyracks/hyracks-data/hyracks-data-std/pom.xml | 2 +- hyracks-fullstack/hyracks/hyracks-data/pom.xml | 2 +- hyracks-fullstack/hyracks/hyracks-dataflow-common/pom.xml | 2 +- hyracks-fullstack/hyracks/hyracks-dataflow-std/pom.xml | 2 +- hyracks-fullstack/hyracks/hyracks-dist/pom.xml | 2 +- hyracks-fullstack/hyracks/hyracks-documentation/pom.xml | 2 +- .../hyracks/hyracks-examples/btree-example/btreeclient/pom.xml | 2 +- .../hyracks/hyracks-examples/btree-example/btreehelper/pom.xml | 2 +- .../hyracks/hyracks-examples/btree-example/btreeserver/pom.xml | 2 +- hyracks-fullstack/hyracks/hyracks-examples/btree-example/pom.xml
[asterixdb] branch master updated: [NO ISSUE] Dashboard tweaks
This is an automated email from the ASF dual-hosted git repository. imaxon pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/asterixdb.git The following commit(s) were added to refs/heads/master by this push: new f6727a9 [NO ISSUE] Dashboard tweaks f6727a9 is described below commit f6727a96f2ed02f5adac7436d41795953e1dd2c9 Author: Ian Maxon AuthorDate: Thu Jun 3 09:16:25 2021 -0700 [NO ISSUE] Dashboard tweaks - user model changes: no - storage format changes: no - interface changes: no Details: - Update lodash - Add some modules to allowed CommonJS dependencies Change-Id: Iaffc8c8f2f51e6fe5ce30b03db1bca43b92b248b Reviewed-on: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/11683 Integration-Tests: Jenkins Tested-by: Jenkins Contrib: Ian Maxon Reviewed-by: Ian Maxon --- asterixdb/asterix-dashboard/src/node/angular.json | 3 +++ asterixdb/asterix-dashboard/src/node/package.json | 2 +- 2 files changed, 4 insertions(+), 1 deletion(-) diff --git a/asterixdb/asterix-dashboard/src/node/angular.json b/asterixdb/asterix-dashboard/src/node/angular.json index 839ff38..cbb4d83 100644 --- a/asterixdb/asterix-dashboard/src/node/angular.json +++ b/asterixdb/asterix-dashboard/src/node/angular.json @@ -33,6 +33,9 @@ "hammerjs", "lodash", "file-saver", + "codemirror", + "webcola", + "dagre" ] }, "configurations": { diff --git a/asterixdb/asterix-dashboard/src/node/package.json b/asterixdb/asterix-dashboard/src/node/package.json index da627b9..8aef8bd 100755 --- a/asterixdb/asterix-dashboard/src/node/package.json +++ b/asterixdb/asterix-dashboard/src/node/package.json @@ -34,7 +34,7 @@ "core-js": "3.8.3", "file-saver": "2.0.5", "hammerjs": "2.0.8", -"lodash": "^4.17.20", +"lodash": "^4.17.21", "roboto-fontface": "^0.10.0", "rxjs": "6.6.3", "rxjs-compat": "6.6.3",