Re: Jenkins PR build is failing
FYI: I have triggered a new MADlib PR and it has gone to another build slave and it looks happier. -=e Building remotely on H27 <https://builds.apache.org/computer/H27> (ubuntu xenial) in workspace /home/jenkins/jenkins-slave/workspace/madlib-pr-build On Fri, Aug 18, 2017 at 11:45 AM, Nandish Jayaram <njaya...@pivotal.io> wrote: > Thank you for investigating it Ed. :) > > NJ > > On Fri, Aug 18, 2017 at 11:42 AM, Ed Espino <esp...@apache.org> wrote: > > > NJ, > > > > I took a quick look at the build console output and it appears "qnode3 > > (ubuntu)" has exhausted it's available disk space. Additionally, it > looks > > as if this system has been indulging a bit too much lately (too many > Stan's > > donuts?) impacting other Jenkins jobs: > > https://issues.apache.org/jira/browse/INFRA-14838?jql= > > text%20~%20%22qnode3%22 > > > > I also triggered additional PR builds and they all failed as they went to > > the same Jenkins build resource. > > > > I have added a comment to INFRA-14838 for the MADlib PR build issue. You > > may need to open a new INFRA ticket to address this current issues. > > > > Top of build console output identifying the Jenkins build slave resource: > > > > Building remotely on qnode3 > > <https://builds.apache.org/computer/qnode3> (ubuntu) in workspace > > /home/jenkins/jenkins-slave/workspace/madlib-pr-build > > > > > > -=e > > > > On Fri, Aug 18, 2017 at 10:53 AM, Nandish Jayaram <njaya...@pivotal.io> > > wrote: > > > > > Hi, > > > > > > The latest PR build on Jenkins ( > > > https://builds.apache.org/user/riyer/my-views/view/ > > > MADlib-Monitor/job/madlib-pr-build/170/) > > > is failing with an > > > `java.io.IOException: No space left on device` error. > > > Can somebody with a Jenkins account have a look at > > > it please? > > > > > > NJ > > > > > > > > > > > -- > > *Ed Espino* > > > -- *Ed Espino*
Re: Jenkins PR build is failing
NJ, I took a quick look at the build console output and it appears "qnode3 (ubuntu)" has exhausted it's available disk space. Additionally, it looks as if this system has been indulging a bit too much lately (too many Stan's donuts?) impacting other Jenkins jobs: https://issues.apache.org/jira/browse/INFRA-14838?jql=text%20~%20%22qnode3%22 I also triggered additional PR builds and they all failed as they went to the same Jenkins build resource. I have added a comment to INFRA-14838 for the MADlib PR build issue. You may need to open a new INFRA ticket to address this current issues. Top of build console output identifying the Jenkins build slave resource: Building remotely on qnode3 <https://builds.apache.org/computer/qnode3> (ubuntu) in workspace /home/jenkins/jenkins-slave/workspace/madlib-pr-build -=e On Fri, Aug 18, 2017 at 10:53 AM, Nandish Jayaram <njaya...@pivotal.io> wrote: > Hi, > > The latest PR build on Jenkins ( > https://builds.apache.org/user/riyer/my-views/view/ > MADlib-Monitor/job/madlib-pr-build/170/) > is failing with an > `java.io.IOException: No space left on device` error. > Can somebody with a Jenkins account have a look at > it please? > > NJ > -- *Ed Espino*
Release Process section feedback request: Creating dmg package (macOS Sierra)
I have created a new Release Process section titled "Creating dmg package (macOS Sierra). It captures the essence of the macOS dmg release creation process. Feedback is welcomed. https://cwiki.apache.org/confluence/display/MADLIB/Release+Process#ReleaseProcess-Creatingdmgpackage(macOSSierra) Thanks, -=e -- *Ed Espino*
Re: Apache MADlib v1.12 status
Frankie, Are there Jiras for the remaining work? This work (minor changes to neural nets) is currently not visible on the release dashboard ( https://issues.apache.org/jira/secure/Dashboard.jspa?selectPageId=12331450) -=e On Wed, Aug 16, 2017 at 9:52 AM, Frank McQuillan <fmcquil...@pivotal.io> wrote: > Some doc changes coming for multiple modules, and minor changes to neural > nets in the next day or so. > > Frank > > On Wed, Aug 16, 2017 at 9:49 AM, Cooper Sloan <csl...@pivotal.io> wrote: > > > We shouldn't hold up the release. This is a no-op for now. > > If we get more information from the customer we can reopen it, but for > now > > we will do nothing. > > > > CS > > > > On Wed, Aug 16, 2017 at 9:41 AM Ed Espino <esp...@apache.org> wrote: > > > > > We have one outstanding Apache MADlib v1.12 Jira holding up the release > > > (MADLIB-1091). It appears Cooper has been working on it and is seeking > > > additional information. If it is not resolved soon, we need to decide > if > > we > > > will push this to a future release. > > > > > > FYI: Today I will be performing preliminary convenience binary builds > > > following the information provided in the Release Process section > titled > > > "Prepare rpm and dmg binaries" ( > > > > > > https://cwiki.apache.org/confluence/display/MADLIB/Release+Process# > > ReleaseProcess-Preparerpmanddmgbinaries > > > ). > > > I will undoubtedly be contributing additional information in the > section > > > and looking for guidance and confirmation of my understanding of the > > > convenience binary build environments. > > > > > > We're almost there! > > > > > > Cheerios, > > > -=e > > > > > > -- > > > *Ed Espino* > > > > > > -- *Ed Espino*
Re: Apache MADlib v1.12 status
Thanks for the update Cooper. I have updated the Fix Version/s to v2.0 for MADLIB-1091. -=e On Wed, Aug 16, 2017 at 9:52 AM, Frank McQuillan <fmcquil...@pivotal.io> wrote: > Some doc changes coming for multiple modules, and minor changes to neural > nets in the next day or so. > > Frank > > On Wed, Aug 16, 2017 at 9:49 AM, Cooper Sloan <csl...@pivotal.io> wrote: > > > We shouldn't hold up the release. This is a no-op for now. > > If we get more information from the customer we can reopen it, but for > now > > we will do nothing. > > > > CS > > > > On Wed, Aug 16, 2017 at 9:41 AM Ed Espino <esp...@apache.org> wrote: > > > > > We have one outstanding Apache MADlib v1.12 Jira holding up the release > > > (MADLIB-1091). It appears Cooper has been working on it and is seeking > > > additional information. If it is not resolved soon, we need to decide > if > > we > > > will push this to a future release. > > > > > > FYI: Today I will be performing preliminary convenience binary builds > > > following the information provided in the Release Process section > titled > > > "Prepare rpm and dmg binaries" ( > > > > > > https://cwiki.apache.org/confluence/display/MADLIB/Release+Process# > > ReleaseProcess-Preparerpmanddmgbinaries > > > ). > > > I will undoubtedly be contributing additional information in the > section > > > and looking for guidance and confirmation of my understanding of the > > > convenience binary build environments. > > > > > > We're almost there! > > > > > > Cheerios, > > > -=e > > > > > > -- > > > *Ed Espino* > > > > > > -- *Ed Espino*
Apache MADlib v1.12 status
We have one outstanding Apache MADlib v1.12 Jira holding up the release (MADLIB-1091). It appears Cooper has been working on it and is seeking additional information. If it is not resolved soon, we need to decide if we will push this to a future release. FYI: Today I will be performing preliminary convenience binary builds following the information provided in the Release Process section titled "Prepare rpm and dmg binaries" ( https://cwiki.apache.org/confluence/display/MADLIB/Release+Process#ReleaseProcess-Preparerpmanddmgbinaries). I will undoubtedly be contributing additional information in the section and looking for guidance and confirmation of my understanding of the convenience binary build environments. We're almost there! Cheerios, -=e -- *Ed Espino*
Apache MADlib v1.12 status
MADlib dev, We are winding down the number of outstanding issues for the Apache MADlib v1.12 release. The one outstanding issue is https://issues.apache.org/jira/browse/MADLIB-1091. Once this is resolved, I'm hoping to start the release process. Regards, -=e -- *Ed Espino*
MADLIB-1103 --> v2.0 (thoughts)
https://issues.apache.org/jira/browse/MADLIB-1103 (Remove pyxb GPL workaround) is dependent on the release of PyXB 1.2.6 (which is currently not scheduled). I'm inclined to move it to v2.0 and we can revisit at a later point. Thoughts? -=e -- *Ed Espino*
Re: Jira post v1.12 version?
Thanks Frank. I have moved them to v2.0. The main reason why I am interested in these issues is IMHO they tie directly to easing the dev user community adoption (lowers bar of entry - newer gcc versions supported). -=e On Mon, Aug 14, 2017 at 12:04 PM, Frank McQuillan <fmcquil...@pivotal.io> wrote: > Ed, > > I would suggest v2.0 for the next version, so you can add those 2 JIRAs to > v2.0 > > Once we get v1.12 out the door I was going to solicit comments from the > community on v2.0 features so we can get that backlog going. > > Frank > > On Mon, Aug 14, 2017 at 11:30 AM, Ed Espino <esp...@apache.org> wrote: > > > Dev, > > > > What are we setting the Jira Fix Version/s for issues to be addressed in > > the next release (post v1.12)? I noticed a v2.0 version (06/Oct/17) > > available in Jira. > > > > The two issues I'd like to set to the next release are the following: > > > > https://issues.apache.org/jira/browse/MADLIB-1025 - MADlib does not > > compile > > with gcc 6.2 > > https://issues.apache.org/jira/browse/MADLIB-1145 - Ubuntu 16.04 - Using > > GCC 5 (default gcc) causes Postgres 9.6 crash > > > > Any guidance is greatly appreciated. > > > > Regards > > -=e > > > > -- > > *Ed Espino* > > > -- *Ed Espino*
Jira post v1.12 version?
Dev, What are we setting the Jira Fix Version/s for issues to be addressed in the next release (post v1.12)? I noticed a v2.0 version (06/Oct/17) available in Jira. The two issues I'd like to set to the next release are the following: https://issues.apache.org/jira/browse/MADLIB-1025 - MADlib does not compile with gcc 6.2 https://issues.apache.org/jira/browse/MADLIB-1145 - Ubuntu 16.04 - Using GCC 5 (default gcc) causes Postgres 9.6 crash Any guidance is greatly appreciated. Regards -=e -- *Ed Espino*
Re: Jenkins madlib-master-build failed
An observant badminton birdie whispered in the wind "I couldn't find a way to re-trigger Jenkins master, is it because I don't have a Jenkins account?" It just so happens that I assist with Apache Jenkins support for the Apache HAWQ (incubating) project. I requested access from the mentor (The great, powerful and kind Roman). It is he who granted me access to the Apache Jenkins service. It is through that privilege that I was able to trigger a MADlib master build to get the project back to a green state. I'm not sure how many team members on the Apache MADlib project have access to this service, but I suggest there are at least a few to assist with its maintenance. Who on the team currently has access to the Apache Jenkins service? -=e On Thu, Aug 10, 2017 at 4:15 PM, Ed Espino <esp...@apache.org> wrote: > FYI: The manually triggered Jenkins master build passed: > https://builds.apache.org/view/M-R/view/MADlib/job/madlib-master-build/80/ > > -=e > > On Thu, Aug 10, 2017 at 4:14 PM, Ed Espino <esp...@apache.org> wrote: > >> Not sure what caused the MADlib master build to fail (git clone issue?). >> I have re-triggered it and it is beyond the previous failure point. >> >> -=e >> >> Here is the failure for future reference (https://builds.apache.org/vie >> w/M-R/view/MADlib/job/madlib-master-build/79/console): >> >> Checking out Revision 67b69eb8a5eec1ff5d4b947eabb90970d66b2ac5 >> (refs/remotes/origin/master) >> Commit message: "MADLIB-1133. TLP graduation - remove references to >> "incubating"." >> > git config core.sparsecheckout # timeout=10 >> > git checkout -f 67b69eb8a5eec1ff5d4b947eabb90970d66b2ac5 >> > git rev-list 0dc2df94358bb2ec3fd85865a6d53ae7cbde0226 # timeout=10 >> Extended Email Publisher is currently disabled in project settings >> FATAL: Unable to produce a script file >> java.io.IOException: Permission denied >> at java.io.UnixFileSystem.createFileExclusively(Native Method) >> at java.io.File.createTempFile(File.java:2024) >> at hudson.FilePath$17.invoke(FilePath.java:1373) >> at hudson.FilePath$17.invoke(FilePath.java:1363) >> at hudson.FilePath$FileCallableWrapper.call(FilePath.java:2739) >> at hudson.remoting.UserRequest.perform(UserRequest.java:153) >> at hudson.remoting.UserRequest.perform(UserRequest.java:50) >> at hudson.remoting.Request$2.run(Request.java:336) >> at hudson.remoting.InterceptingExecutorService$1.call(Intercept >> ingExecutorService.java:68) >> at java.util.concurrent.FutureTask.run(FutureTask.java:266) >> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPool >> Executor.java:1142) >> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoo >> lExecutor.java:617) >> at java.lang.Thread.run(Thread.java:748) >> Caused: java.io.IOException: Failed to create a temporary directory in >> /tmp >> at hudson.FilePath$17.invoke(FilePath.java:1375) >> at hudson.FilePath$17.invoke(FilePath.java:1363) >> at hudson.FilePath$FileCallableWrapper.call(FilePath.java:2739) >> at hudson.remoting.UserRequest.perform(UserRequest.java:153) >> at hudson.remoting.UserRequest.perform(UserRequest.java:50) >> at hudson.remoting.Request$2.run(Request.java:336) >> at hudson.remoting.InterceptingExecutorService$1.call(Intercept >> ingExecutorService.java:68) >> at java.util.concurrent.FutureTask.run(FutureTask.java:266) >> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPool >> Executor.java:1142) >> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoo >> lExecutor.java:617) >> at java.lang.Thread.run(Thread.java:748) >> at ..remote call to H21(Native Method) >> at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1545) >> at hudson.remoting.UserResponse.retrieve(UserRequest.java:253) >> at hudson.remoting.Channel.call(Channel.java:830) >> at hudson.FilePath.act(FilePath.java:986) >> Caused: java.io.IOException: remote file operation failed: >> /home/jenkins/jenkins-slave/workspace/madlib-master-build at >> hudson.remoting.Channel@4b715ff3:H21 >> at hudson.FilePath.act(FilePath.java:993) >> at hudson.FilePath.act(FilePath.java:975) >> at hudson.FilePath.createTextTempFile(FilePath.java:1363) >> Caused: java.io.IOException: Failed to create a temp file on >> /home/jenkins/jenkins-slave/workspace/madlib-master-build >> at hudson.FilePath.createTextTempFile(FilePath.java:1386) >> at hudson.tasks.CommandInterpreter.createScriptFile(CommandInte >> rpreter.java:162) >> at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:94) >> at hudson.t
Re: Jenkins madlib-master-build failed
FYI: The manually triggered Jenkins master build passed: https://builds.apache.org/view/M-R/view/MADlib/job/madlib-master-build/80/ -=e On Thu, Aug 10, 2017 at 4:14 PM, Ed Espino <esp...@apache.org> wrote: > Not sure what caused the MADlib master build to fail (git clone issue?). I > have re-triggered it and it is beyond the previous failure point. > > -=e > > Here is the failure for future reference (https://builds.apache.org/ > view/M-R/view/MADlib/job/madlib-master-build/79/console): > > Checking out Revision 67b69eb8a5eec1ff5d4b947eabb90970d66b2ac5 > (refs/remotes/origin/master) > Commit message: "MADLIB-1133. TLP graduation - remove references to > "incubating"." > > git config core.sparsecheckout # timeout=10 > > git checkout -f 67b69eb8a5eec1ff5d4b947eabb90970d66b2ac5 > > git rev-list 0dc2df94358bb2ec3fd85865a6d53ae7cbde0226 # timeout=10 > Extended Email Publisher is currently disabled in project settings > FATAL: Unable to produce a script file > java.io.IOException: Permission denied > at java.io.UnixFileSystem.createFileExclusively(Native Method) > at java.io.File.createTempFile(File.java:2024) > at hudson.FilePath$17.invoke(FilePath.java:1373) > at hudson.FilePath$17.invoke(FilePath.java:1363) > at hudson.FilePath$FileCallableWrapper.call(FilePath.java:2739) > at hudson.remoting.UserRequest.perform(UserRequest.java:153) > at hudson.remoting.UserRequest.perform(UserRequest.java:50) > at hudson.remoting.Request$2.run(Request.java:336) > at hudson.remoting.InterceptingExecutorService$1.call( > InterceptingExecutorService.java:68) > at java.util.concurrent.FutureTask.run(FutureTask.java:266) > at java.util.concurrent.ThreadPoolExecutor.runWorker( > ThreadPoolExecutor.java:1142) > at java.util.concurrent.ThreadPoolExecutor$Worker.run( > ThreadPoolExecutor.java:617) > at java.lang.Thread.run(Thread.java:748) > Caused: java.io.IOException: Failed to create a temporary directory in /tmp > at hudson.FilePath$17.invoke(FilePath.java:1375) > at hudson.FilePath$17.invoke(FilePath.java:1363) > at hudson.FilePath$FileCallableWrapper.call(FilePath.java:2739) > at hudson.remoting.UserRequest.perform(UserRequest.java:153) > at hudson.remoting.UserRequest.perform(UserRequest.java:50) > at hudson.remoting.Request$2.run(Request.java:336) > at hudson.remoting.InterceptingExecutorService$1.call( > InterceptingExecutorService.java:68) > at java.util.concurrent.FutureTask.run(FutureTask.java:266) > at java.util.concurrent.ThreadPoolExecutor.runWorker( > ThreadPoolExecutor.java:1142) > at java.util.concurrent.ThreadPoolExecutor$Worker.run( > ThreadPoolExecutor.java:617) > at java.lang.Thread.run(Thread.java:748) > at ..remote call to H21(Native Method) > at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1545) > at hudson.remoting.UserResponse.retrieve(UserRequest.java:253) > at hudson.remoting.Channel.call(Channel.java:830) > at hudson.FilePath.act(FilePath.java:986) > Caused: java.io.IOException: remote file operation failed: > /home/jenkins/jenkins-slave/workspace/madlib-master-build at > hudson.remoting.Channel@4b715ff3:H21 > at hudson.FilePath.act(FilePath.java:993) > at hudson.FilePath.act(FilePath.java:975) > at hudson.FilePath.createTextTempFile(FilePath.java:1363) > Caused: java.io.IOException: Failed to create a temp file on > /home/jenkins/jenkins-slave/workspace/madlib-master-build > at hudson.FilePath.createTextTempFile(FilePath.java:1386) > at hudson.tasks.CommandInterpreter.createScriptFile( > CommandInterpreter.java:162) > at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:94) > at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:66) > at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20) > at hudson.model.AbstractBuild$AbstractBuildExecution. > perform(AbstractBuild.java:735) > at hudson.model.Build$BuildExecution.build(Build.java:206) > at hudson.model.Build$BuildExecution.doRun(Build.java:163) > at hudson.model.AbstractBuild$AbstractBuildExecution.run( > AbstractBuild.java:490) > at hudson.model.Run.execute(Run.java:1735) > at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43) > at hudson.model.ResourceController.execute(ResourceController.java:97) > at hudson.model.Executor.run(Executor.java:405) > Build step 'Execute shell' marked build as failure > Skipped archiving because build is not successful > Recording test results > Extended Email Publisher is currently disabled in project settings > Warning: this build has no associated authentication, so build permissions > may be lacking, and downstream projects which cannot even be seen by an > anonymous user will be silently skipped > Finished: FAILURE > > > > -- > *Ed Espino* > -- *Ed Espino*
MADlib PR Jenkins jobs
It appears a few of the PRs are in a stalled state (Some checks haven’t completed yet) for the Jenkins "madlib-pr-build" ( https://builds.apache.org/view/M-R/view/MADlib/job/madlib-pr-build/) job. Any idea what might be going on? A sample of stalled build check PRs: https://github.com/apache/incubator-madlib/pull/161 https://github.com/apache/incubator-madlib/pull/163 -=e -- *Ed Espino*
Re: Migrating MADlib code base out of incubator.
NJ, I would probably start another with a clear Subject. Thanks, -=e On Wed, Aug 9, 2017 at 2:08 PM Nandish Jayaram <njaya...@pivotal.io> wrote: > > Shall we put the main MADlib repo(s) migration to GitBox to a vote? > Sure. Do we vote on this thread or on a different one? > > NJ > > On Wed, Aug 9, 2017 at 12:28 PM, Ed Espino <esp...@apache.org> wrote: > > > I believe there is another git repo "incubator-madlib-site" (pair) that > > needs to be migrated as well: > > > > https://git1-us-west.apache.org/repos/asf?p=incubator-madlib-site.git > > https://github.com/apache/incubator-madlib-site (GitHub mirror) > > > > Personally, I am in favor of Gitbox use. I believe there is a security > > requirement which I also support: "You are required to enable 2FA on > GitHub > > before you can gain write-access to repositories.". > > > > Shall we put the main MADlib repo(s) migration to GitBox to a vote? > > > > -=e > > > > On Wed, Aug 9, 2017 at 11:47 AM, Nandish Jayaram <njaya...@pivotal.io> > > wrote: > > > > > Hi All, > > > > > > I was planning on opening an Apache Infra JIRA to migrate MADlib's > > current > > > code repo out of "incubator", following the directions in: > > > https://incubator.apache.org/guides/transferring.html > > > http://www.apache.org/dev/infra-contact#requesting-graduation > > > An example Infra JIRA we can follow: > > > https://issues.apache.org/jira/browse/INFRA-5688 > > > > > > The following MADlib repos have to be migrated: > > > https://git1-us-west.apache.org/repos/asf?p=incubator-madlib.git > > > https://github.com/apache/incubator-madlib > > > > > > A recent email from Roman ( > > > https://mail-archives.apache.org/mod_mbox/incubator-madlib- > > > dev/201708.mbox/%3cCA+ULb+vP0ViWH4Nc=4eaXvbT0KOmeFtQzp4eAa3p0fKPP7c > > > 8...@mail.gmail.com%3e > > > ) > > > gave us another option in Gitbox for our source code repo. > > > > > > Does anybody in the community have any preferences/suggestions on > > > moving to Gitbox? > > > > > > NJ > > > > > > > > > > > -- > > *Ed Espino* > > > -- -- Ed Espino Director, Pivotal Technical Staff Cell: 925.389.4640 WeChat ID: EdEspino
MADLIB-1118 - IC is taking too long for elastic net
MADlibers, I took a stab at addressing MADLIB-1118 (IC is taking too long for elastic net). I'm not sure if the approach (updating elastic_net_train tolerance value (1e-6 --> 1e-2) is appropriate. Although it does deliver the expected reduction in execution. As the jira suggested, I tried to reduce the data set and found the execution time was not significantly reduced. Additionally, I added a comment to the PR which showed how I approached tweaking the test suite. If you have a chance, would you mind reviewing the PR? https://github.com/apache/incubator-madlib/pull/163 Regards, -=e -- *Ed Espino*
Re: Migrating MADlib code base out of incubator.
I believe there is another git repo "incubator-madlib-site" (pair) that needs to be migrated as well: https://git1-us-west.apache.org/repos/asf?p=incubator-madlib-site.git https://github.com/apache/incubator-madlib-site (GitHub mirror) Personally, I am in favor of Gitbox use. I believe there is a security requirement which I also support: "You are required to enable 2FA on GitHub before you can gain write-access to repositories.". Shall we put the main MADlib repo(s) migration to GitBox to a vote? -=e On Wed, Aug 9, 2017 at 11:47 AM, Nandish Jayaram <njaya...@pivotal.io> wrote: > Hi All, > > I was planning on opening an Apache Infra JIRA to migrate MADlib's current > code repo out of "incubator", following the directions in: > https://incubator.apache.org/guides/transferring.html > http://www.apache.org/dev/infra-contact#requesting-graduation > An example Infra JIRA we can follow: > https://issues.apache.org/jira/browse/INFRA-5688 > > The following MADlib repos have to be migrated: > https://git1-us-west.apache.org/repos/asf?p=incubator-madlib.git > https://github.com/apache/incubator-madlib > > A recent email from Roman ( > https://mail-archives.apache.org/mod_mbox/incubator-madlib- > dev/201708.mbox/%3cCA+ULb+vP0ViWH4Nc=4eaXvbT0KOmeFtQzp4eAa3p0fKPP7c > 8...@mail.gmail.com%3e > ) > gave us another option in Gitbox for our source code repo. > > Does anybody in the community have any preferences/suggestions on > moving to Gitbox? > > NJ > -- *Ed Espino*
GCC 5, 6 and 7 are not supported by MADlib
Sharing with the dev community: I've been working with several different Linux (Debian 8.9 & 9,1, Linux Mint 18.2, Fedora 26, Ubuntu 16.04 & 14.04) distros trying to see how MADlib builds and runs install-check against Postgres 9.6. I have had good success with GCC 4.x and zero success with GCC 5.x, 6.x and 7.x versions. There are distros in this list where finding a readily available GCC 4.x version isn't straight forward and I essentially had to revert to older distro versions. I can assist with validation once the other GCC versions (5, 6 & 7) are supported. -=e -- *Ed Espino*
Re: Confusion regarding the order of JIRAs to address
NJ, MADLIB-1132 is related to moving the https://dist.apache.org source tarball and convenience binary contents. These are managed in a "subversion" repository. There are subversion references in the "Release Process" section of the wiki: https://cwiki.apache.org/confluence/display/MADLIB/Release+Process -=e On Mon, Aug 7, 2017 at 3:26 PM, Nandish Jayaram <njaya...@pivotal.io> wrote: > Hi All, > > I was reviewing PR #158 (https://github.com/apache/ > incubator-madlib/pull/158), > and was also looking at the post graduation tasks. This PR seems to be very > relevant to one of the post graduation JIRAs ( > https://issues.apache.org/jira/browse/MADLIB-1132), and I was a little > confused about the order of doing things here. > Should we merge PR #158 before or after MADLIB-1132 is addressed? > > NJ > -- *Ed Espino*
Re: Using GDB with MADlib
I will add a new section "Debugging and Developing" to the wiki. And fortunately, I have permission to edit the wiki (Thanks Orhan). When I have the section created with content, I'll let the dev team know. Regards, -=e On Mon, Aug 7, 2017 at 4:39 PM, Roman Shaposhnik <ro...@shaposhnik.org> wrote: > On Mon, Aug 7, 2017 at 2:46 PM, Ed Espino <esp...@apache.org> wrote: > > While I was trying to track down the cause of MADLIB-1068, I managed to > put > > Postgres/MADlib under the debugger (gdb) and thought the configuration > > would be of some help to someone. Essentially, I built postgres with the > > following configure options: > > > > configure --enable-cassert --enable-debug --with-python > > > > I then was able to start a psql session and attach to the running session > > with gdb. This allows one to set breakpoints, review variable values and > > all the other cool gdb debugging techniques. There is a reasonable amount > > of information one can find on using gdb and on the web. It was much > easier > > than I originally envisioned. If anyone needs assistance with the setup, > > please don't hesitate to ping me. > > This is a pretty awesome tidbit! Any chance you can contribute it to the > wiki? >https://cwiki.apache.org/confluence/display/MADLIB/ > > Perhaps even a brand new section on Debugging and Developing? > > Thanks, > Roman. > -- *Ed Espino*
Using GDB with MADlib
While I was trying to track down the cause of MADLIB-1068, I managed to put Postgres/MADlib under the debugger (gdb) and thought the configuration would be of some help to someone. Essentially, I built postgres with the following configure options: configure --enable-cassert --enable-debug --with-python I then was able to start a psql session and attach to the running session with gdb. This allows one to set breakpoints, review variable values and all the other cool gdb debugging techniques. There is a reasonable amount of information one can find on using gdb and on the web. It was much easier than I originally envisioned. If anyone needs assistance with the setup, please don't hesitate to ping me. Happy MADilb Debugging, -=e -- *Ed Espino*
Re: MADlib Debugging (elastic_net) - MADLIB-1068
I have resolved MADLIB-1068 with a workaround (use GCC 4 and not the default GCC 5) for Ubuntu 16.04. I have created MADLIB-1145 to track the GCC 5 issue. Happy Monday, -=e On Fri, Aug 4, 2017 at 11:11 AM, Ed Espino <esp...@apache.org> wrote: > FYI: I have managed to get the issue to reproduce in gdb! It was > relatively painless. I will be adding my debugging environment and notes to > the dev list for future reference. > > I have come to understand Ubuntu isn't a supported platform. In the spirit > of growing and fostering MADlib adoption, I'll spend a bit more time on > this to see if I can identify what the subtleties are between the supported > platforms and Ubuntu. > > Cheers, > -=e > > On Thu, Aug 3, 2017 at 5:15 PM, Ed Espino <esp...@apache.org> wrote: > >> For MADLIB-1068, I have reproduced the elastic_net issue which causes a >> crash (core dump) in PostgreSQL 9.6 on Ubuntu 16.04.03 using MADlib master. >> Are there any general debugging techniques used to help track down these >> types of issues? I will be fiddling around with the elastic_net trying to >> track down this issue. Any helpful tips and tricks are greatly appreciated. >> >> -=e >> >> -- >> *Ed Espino* >> > > > > -- > *Ed Espino* > -- *Ed Espino*
Re: Regarding moving source repos from incubator-madlib to madlib
NJ (aka: Badminton man), Roman pointed me at the following "Guide :: Transferring Resources out of the Incubator" (https://incubator.apache.org/guides/transferring.html). There is mention of the git project renaming task. There are also several TLP graduation JIRAs filed under the "TLP graduation tasks" epic (https://issues.apache.org/jira/browse/MADLIB-1112). -=e On Mon, Aug 7, 2017 at 12:27 PM, Nandish Jayaram <njaya...@pivotal.io> wrote: > Hi All, > > Now that we have graduated to TLP, it's time to move all "incubator-madlib" > to "madlib", and I was looking at a PR ( > https://github.com/apache/incubator-madlib/pull/158) which does quite a > bit > towards it. > > I noticed that a few URLs such as our homepage and user docs ( > http://madlib.apache.org/docs/latest/index.html) have been been changed, > although we might still want to redirect > http://madlib.incubator.apache.org/docs/latest/index.html to > http://madlib.apache.org/docs/latest/index.html. > > The question I had was how and when to make URL changes to other resources > such as our source code repo (https://github.com/apache/incubator-madlib/ > and https://git1-us-west.apache.org/repos/asf?p=incubator-madlib.git). Any > idea how to go about this process? > > NJ > -- *Ed Espino*
Re: MADlib Debugging (elastic_net) - MADLIB-1068
FYI: I have managed to get the issue to reproduce in gdb! It was relatively painless. I will be adding my debugging environment and notes to the dev list for future reference. I have come to understand Ubuntu isn't a supported platform. In the spirit of growing and fostering MADlib adoption, I'll spend a bit more time on this to see if I can identify what the subtleties are between the supported platforms and Ubuntu. Cheers, -=e On Thu, Aug 3, 2017 at 5:15 PM, Ed Espino <esp...@apache.org> wrote: > For MADLIB-1068, I have reproduced the elastic_net issue which causes a > crash (core dump) in PostgreSQL 9.6 on Ubuntu 16.04.03 using MADlib master. > Are there any general debugging techniques used to help track down these > types of issues? I will be fiddling around with the elastic_net trying to > track down this issue. Any helpful tips and tricks are greatly appreciated. > > -=e > > -- > *Ed Espino* > -- *Ed Espino*
MADlib Debugging (elastic_net) - MADLIB-1068
For MADLIB-1068, I have reproduced the elastic_net issue which causes a crash (core dump) in PostgreSQL 9.6 on Ubuntu 16.04.03 using MADlib master. Are there any general debugging techniques used to help track down these types of issues? I will be fiddling around with the elastic_net trying to track down this issue. Any helpful tips and tricks are greatly appreciated. -=e -- *Ed Espino*
MADlib Jenkins project question (madlib-master-build & madlib-pr-build)
Dev,
In preparation for updating the Jenkins build environment for the MADlib
TLP move, I am reviewing the active Jenkins projects.
I noticed the Jenkins build projects (madlib-master-build &
madlib-pr-build) use a Source Code Management --> Git --> Additional
Behaviours --> Check out to a sub-directory (incubator-madlib)
configuration option. This currenlty places a hardcoded requirement on the
build scripts (jenkins_build.sh & rat_check.sh) with this value.
Question: is the value for the sub-directory arbitrary? I'd like to
suggest we set this value to match the repository name (minus .git suffix).
This will allow us to reference the ${GIT_URL} environment variable
available to the running shell process.
I have created https://issues.apache.org/jira/browse/MADLIB-1142 to track
the need to update the Jenkins projects and referenced the TLP graduation
epic (MADLIB-1112). The jira also has a recommended update to the command
and scripts being executed to be more dynamic.
Thoughts?
-=e
For reference - Jenkins build project links:
- https://builds.apache.org/view/M-R/view/MADlib/job/madlib-master-build
- https://builds.apache.org/view/M-R/view/MADlib/job/madlib-pr-build/
--
*Ed Espino*
MADLIB-1133 - TLP graduation - remove references to "incubating" in source tree
MADLIB-1133 is currently assigned to Frank McQuillan. I'd like to take a quick swing at this and generate a PR. Does anyone know if there has been any work performed on this task? -=e -- *Ed Espino*
MADlib v1.12 release update (plus TLP related tasks)
With Apache MADlib's graduation from incubation to a full fledged TLP complete, I will start tracking the v1.12 progress (including tasks required to support the TLP status). As I have mentioned in the past, I will be using the publicly available Jira Apache MADlib v1.12-incubating Release Dashboard to track the release: https://issues.apache.org/jira/secure/Dashboard.jspa?selectPageId=12331450 As of today, we have 21 outstanding issues (13 unassigned) currently targeted with Fix Version/s set to v1.12. I will be requesting Jira update privileges for the project so I can help with release updates as needed. Cheers, -=e -- *Ed Espino*
Re: MADlib Python module code coverage
NJ, I have a brute force hack working. I'm looking for a more general approach. If I do come up with an approach, I will definitely share it with the dev team. -=e On Tue, Jul 18, 2017 at 4:32 PM, Nandish Jayaram <njaya...@pivotal.io> wrote: > Hey Ed, > > I have not tried it. It'll be great if you could share your experiences > with it. Will go well with > https://github.com/apache/incubator-madlib/pull/151. > > NJ > > On Tue, Jul 18, 2017 at 3:34 PM, Ed Espino <esp...@apache.org> wrote: > > > MADlib dev, > > > > Has anyone tried to use the Python Coverage.py ( > > https://coverage.readthedocs.io/en/coverage-4.4.1/) module to > > generate MADlib Python module code coverage metrics? I want to run it > > against the python only code in the graph PR: > > https://github.com/apache/incubator-madlib/pull/152 > > > > Thanks, > > -=e > > > > -- > > *Ed Espino* > > > -- *Ed Espino*
MADlib Python module code coverage
MADlib dev, Has anyone tried to use the Python Coverage.py ( https://coverage.readthedocs.io/en/coverage-4.4.1/) module to generate MADlib Python module code coverage metrics? I want to run it against the python only code in the graph PR: https://github.com/apache/incubator-madlib/pull/152 Thanks, -=e -- *Ed Espino*
Re: MADlib Code coverage
Nandish,
For a very basic code coverage implementation using cmake, I have created:
* JIRA: https://issues.apache.org/jira/browse/MADLIB-1138
* PR: https://github.com/apache/incubator-madlib/pull/151
This is a basic code coverage implementation runs on my mac using the
following and a brew installed lcov module:
Configured with: --prefix=/Library/Developer/CommandLineTools/usr
--with-gxx-include-dir=/usr/include/c++/4.2.1
Apple LLVM version 8.1.0 (clang-802.0.42)
Target: x86_64-apple-darwin16.6.0
Thread model: posix
InstalledDir: /Library/Developer/CommandLineTools/usr/bin
Regards,
-=e
On Fri, Jul 14, 2017 at 1:50 PM, Ed Espino <esp...@apache.org> wrote:
> Sure NJ,
>
> I'll create both (Jira and PR).
>
> -=e
>
> On Fri, Jul 14, 2017 at 1:46 PM, Nandish Jayaram <njaya...@pivotal.io>
> wrote:
>
>> Thank you Ed, this is great. We will create a JIRA to incorporate this
>> (including documenting the steps to use it). Or, you can also create a PR.
>> :)
>>
>> NJ
>>
>> On Fri, Jul 14, 2017 at 12:31 PM, Ed Espino <esp...@apache.org> wrote:
>>
>> > NJ,
>> >
>> > I spent about an hour getting it setup on my mac. This supported the
>> > MLP PR review. I threw some info together below.
>> >
>> > Hope it helps,
>> > -=e
>> >
>> > --
>> >
>> > Add lcov through brew (brings in lcov and genhtml utilities):
>> > brew install lcov
>> >
>> > "Total HACK" to add code coverage support into the build for C/C++
>> > code. This is quick and dirty so I could get to the data I needed for
>> > the PR review.
>> >
>> > diff --git a/CMakeLists.txt b/CMakeLists.txt
>> > index b2172ef3..db80986c 100644
>> > --- a/CMakeLists.txt
>> > +++ b/CMakeLists.txt
>> > @@ -104,7 +104,7 @@ if(CMAKE_COMPILER_IS_GNUCXX)
>> >set(CMAKE_INCLUDE_SYSTEM_FLAG_CXX "-isystem ")
>> >endif(APPLE)
>> >elseif(CMAKE_C_COMPILER_ID MATCHES "Clang")
>> > -set(CMAKE_CXX_FLAGS "-stdlib=libstdc++")
>> > +set(CMAKE_CXX_FLAGS "-stdlib=libstdc++ -fprofile-arcs
>> > -ftest-coverage")
>> >endif(CMAKE_COMPILER_IS_GNUCXX)
>> >
>> ># force a `m4_' prefix to all builtins
>> > @@ -114,6 +114,8 @@ else()
>> >set(M4_ARGUMENTS "--prefix-builtins")
>> >endif()
>> >
>> > +set(CMAKE_C_FLAGS "-fprofile-arcs -ftest-coverage")
>> > +
>> ># Read and parse Version.yml file
>> >file(READ "${MADLIB_VERSION_YML}" _MADLIB_VERSION_CONTENTS)
>> >string(REGEX REPLACE "^.*version:[ \t]*([^\n]*)\n.*" "\\1"
>> > MADLIB_VERSION_STRING "${_MADLIB_VERSION_CONTENTS}")
>> >
>> > Once MADlib is built and installed, run your tests:
>> >
>> > ## Build and install MADlib
>> > mkdir build
>> > cd build
>> > cmake ..
>> > make -j8 install
>> > /usr/local/madlib/bin/madpack -s madlib -p postgres install
>> >
>> > ## At this point you can run your tests (I'm focusing on MLP)
>> >
>> > /usr/local/madlib/bin/madpack -s madlib -p postgres install-check -t
>> > convex/mlp
>> >
>> > ## Time to capture the results and generate html report.
>> >
>> > lcov --capture --directory . --output-file coverage.info
>> > genhtml coverage.info --output-directory gcov
>> >
>> > # You will notice there is some uninteresting system coverage info.
>> > # The following will filter them out.
>> >
>> > lcov --remove coverage.info '/usr/include/*' '/usr/local/include/*'
>> > '/usr/local/postgres/*' '*build/third_party/*' -o
>> coverage_filtered.info
>> > rm -rf gcov
>> > genhtml coverage_filtered.info --output-directory gcov
>> >
>> > It is possible to zero out the counters with the following.
>> >lcov --zerocounters --directory .
>> >
>> >
>> > On Fri, Jul 14, 2017 at 12:23 PM, Nandish Jayaram <njaya...@pivotal.io>
>> > wrote:
>> >
>> > > Hi Ed,
>> > >
>> > > We haven't set that up for MADlib yet, but we will be looking into it
>> > soon.
>> > > Any ideas?
>> > >
>> > > NJ
>> > >
>> > > On Fri, Jul 14, 2017 at 10:00 AM, Ed Espino <esp...@apache.org>
>> wrote:
>> > >
>> > > > Out of curiosity, do MADlib developers regularly use code coverage
>> > > > utilities to measure the coverage quality of their tests?
>> > > >
>> > > > -=e
>> > > >
>> > > > --
>> > > > *Ed Espino*
>> > > >
>> > >
>> >
>> >
>> >
>> > --
>> > *Ed Espino*
>> >
>>
>
>
>
> --
> *Ed Espino*
>
--
*Ed Espino*
Re: MADlib Code coverage
Sure NJ,
I'll create both (Jira and PR).
-=e
On Fri, Jul 14, 2017 at 1:46 PM, Nandish Jayaram <njaya...@pivotal.io>
wrote:
> Thank you Ed, this is great. We will create a JIRA to incorporate this
> (including documenting the steps to use it). Or, you can also create a PR.
> :)
>
> NJ
>
> On Fri, Jul 14, 2017 at 12:31 PM, Ed Espino <esp...@apache.org> wrote:
>
> > NJ,
> >
> > I spent about an hour getting it setup on my mac. This supported the
> > MLP PR review. I threw some info together below.
> >
> > Hope it helps,
> > -=e
> >
> > --
> >
> > Add lcov through brew (brings in lcov and genhtml utilities):
> > brew install lcov
> >
> > "Total HACK" to add code coverage support into the build for C/C++
> > code. This is quick and dirty so I could get to the data I needed for
> > the PR review.
> >
> > diff --git a/CMakeLists.txt b/CMakeLists.txt
> > index b2172ef3..db80986c 100644
> > --- a/CMakeLists.txt
> > +++ b/CMakeLists.txt
> > @@ -104,7 +104,7 @@ if(CMAKE_COMPILER_IS_GNUCXX)
> >set(CMAKE_INCLUDE_SYSTEM_FLAG_CXX "-isystem ")
> >endif(APPLE)
> >elseif(CMAKE_C_COMPILER_ID MATCHES "Clang")
> > -set(CMAKE_CXX_FLAGS "-stdlib=libstdc++")
> > +set(CMAKE_CXX_FLAGS "-stdlib=libstdc++ -fprofile-arcs
> > -ftest-coverage")
> >endif(CMAKE_COMPILER_IS_GNUCXX)
> >
> ># force a `m4_' prefix to all builtins
> > @@ -114,6 +114,8 @@ else()
> >set(M4_ARGUMENTS "--prefix-builtins")
> >endif()
> >
> > +set(CMAKE_C_FLAGS "-fprofile-arcs -ftest-coverage")
> > +
> ># Read and parse Version.yml file
> >file(READ "${MADLIB_VERSION_YML}" _MADLIB_VERSION_CONTENTS)
> >string(REGEX REPLACE "^.*version:[ \t]*([^\n]*)\n.*" "\\1"
> > MADLIB_VERSION_STRING "${_MADLIB_VERSION_CONTENTS}")
> >
> > Once MADlib is built and installed, run your tests:
> >
> > ## Build and install MADlib
> > mkdir build
> > cd build
> > cmake ..
> > make -j8 install
> > /usr/local/madlib/bin/madpack -s madlib -p postgres install
> >
> > ## At this point you can run your tests (I'm focusing on MLP)
> >
> > /usr/local/madlib/bin/madpack -s madlib -p postgres install-check -t
> > convex/mlp
> >
> > ## Time to capture the results and generate html report.
> >
> > lcov --capture --directory . --output-file coverage.info
> > genhtml coverage.info --output-directory gcov
> >
> > # You will notice there is some uninteresting system coverage info.
> > # The following will filter them out.
> >
> > lcov --remove coverage.info '/usr/include/*' '/usr/local/include/*'
> > '/usr/local/postgres/*' '*build/third_party/*' -o coverage_filtered.info
> > rm -rf gcov
> > genhtml coverage_filtered.info --output-directory gcov
> >
> > It is possible to zero out the counters with the following.
> >lcov --zerocounters --directory .
> >
> >
> > On Fri, Jul 14, 2017 at 12:23 PM, Nandish Jayaram <njaya...@pivotal.io>
> > wrote:
> >
> > > Hi Ed,
> > >
> > > We haven't set that up for MADlib yet, but we will be looking into it
> > soon.
> > > Any ideas?
> > >
> > > NJ
> > >
> > > On Fri, Jul 14, 2017 at 10:00 AM, Ed Espino <esp...@apache.org> wrote:
> > >
> > > > Out of curiosity, do MADlib developers regularly use code coverage
> > > > utilities to measure the coverage quality of their tests?
> > > >
> > > > -=e
> > > >
> > > > --
> > > > *Ed Espino*
> > > >
> > >
> >
> >
> >
> > --
> > *Ed Espino*
> >
>
--
*Ed Espino*
Re: MADlib Code coverage
NJ,
I spent about an hour getting it setup on my mac. This supported the
MLP PR review. I threw some info together below.
Hope it helps,
-=e
--
Add lcov through brew (brings in lcov and genhtml utilities):
brew install lcov
"Total HACK" to add code coverage support into the build for C/C++
code. This is quick and dirty so I could get to the data I needed for
the PR review.
diff --git a/CMakeLists.txt b/CMakeLists.txt
index b2172ef3..db80986c 100644
--- a/CMakeLists.txt
+++ b/CMakeLists.txt
@@ -104,7 +104,7 @@ if(CMAKE_COMPILER_IS_GNUCXX)
set(CMAKE_INCLUDE_SYSTEM_FLAG_CXX "-isystem ")
endif(APPLE)
elseif(CMAKE_C_COMPILER_ID MATCHES "Clang")
-set(CMAKE_CXX_FLAGS "-stdlib=libstdc++")
+set(CMAKE_CXX_FLAGS "-stdlib=libstdc++ -fprofile-arcs
-ftest-coverage")
endif(CMAKE_COMPILER_IS_GNUCXX)
# force a `m4_' prefix to all builtins
@@ -114,6 +114,8 @@ else()
set(M4_ARGUMENTS "--prefix-builtins")
endif()
+set(CMAKE_C_FLAGS "-fprofile-arcs -ftest-coverage")
+
# Read and parse Version.yml file
file(READ "${MADLIB_VERSION_YML}" _MADLIB_VERSION_CONTENTS)
string(REGEX REPLACE "^.*version:[ \t]*([^\n]*)\n.*" "\\1"
MADLIB_VERSION_STRING "${_MADLIB_VERSION_CONTENTS}")
Once MADlib is built and installed, run your tests:
## Build and install MADlib
mkdir build
cd build
cmake ..
make -j8 install
/usr/local/madlib/bin/madpack -s madlib -p postgres install
## At this point you can run your tests (I'm focusing on MLP)
/usr/local/madlib/bin/madpack -s madlib -p postgres install-check -t
convex/mlp
## Time to capture the results and generate html report.
lcov --capture --directory . --output-file coverage.info
genhtml coverage.info --output-directory gcov
# You will notice there is some uninteresting system coverage info.
# The following will filter them out.
lcov --remove coverage.info '/usr/include/*' '/usr/local/include/*'
'/usr/local/postgres/*' '*build/third_party/*' -o coverage_filtered.info
rm -rf gcov
genhtml coverage_filtered.info --output-directory gcov
It is possible to zero out the counters with the following.
lcov --zerocounters --directory .
On Fri, Jul 14, 2017 at 12:23 PM, Nandish Jayaram <njaya...@pivotal.io>
wrote:
> Hi Ed,
>
> We haven't set that up for MADlib yet, but we will be looking into it soon.
> Any ideas?
>
> NJ
>
> On Fri, Jul 14, 2017 at 10:00 AM, Ed Espino <esp...@apache.org> wrote:
>
> > Out of curiosity, do MADlib developers regularly use code coverage
> > utilities to measure the coverage quality of their tests?
> >
> > -=e
> >
> > --
> > *Ed Espino*
> >
>
--
*Ed Espino*
MADlib Code coverage
Out of curiosity, do MADlib developers regularly use code coverage utilities to measure the coverage quality of their tests? -=e -- *Ed Espino*
External references to MADlib incubator project content
When MADlib graduates, will the previous incubator links redirect to the TLP location? I noticed the following MADlib incubator references in the Pivotal Greenplum DB docs:: source page: Greenplum MADlib Extension for Analytics https://gpdb.docs.pivotal.io/4390/ref_guide/extensions/madlib.html#topic9 link references: MADlib web site is at http://madlib.incubator.apache.org/ MADlib documentation is at http://madlib.incubator.apache.org/documentation.html -=e -- *Ed Espino*
Question regarding tracking Release v1.12 specific changes.
MADlib dev, While reviewing a PR, I noticed the doc top-level generated html page has a reference to previous releases ( https://madlib.incubator.apache.org/docs/latest/). I found the corresponding v1.11 changelist ( https://github.com/apache/incubator-madlib/commit/648b05798826956e9621027447af501c194392b8) which updated the previous release versions to include v1.10 (in addition to other v1.11 related changes). Aside from this PR, how does the project track these types of release changes (Jira, wiki, email, other). I could not find a reference to them in the project's wiki. It might be staring me in the face so I apologize if it is. Any guidance is greatly appreciated. -=e p.s. And yes, this is in preparation for my v1.12 release manager role. Gaining context comes at a price. -- *Ed Espino*
Re: Apache Jira: MADLIB v1.12-incubating and Metrics dashboard
Thank you Frankerino! -=e On Tue, Jul 11, 2017 at 2:02 PM, Frank McQuillan <fmcquil...@pivotal.io> wrote: > I updated > https://issues.apache.org/jira/browse/MADLIB-1113 > > and created > https://issues.apache.org/jira/browse/MADLIB-1132 > https://issues.apache.org/jira/browse/MADLIB-1133 > > based on your findings. > > All of these TLP related stories are under the epic > https://issues.apache.org/jira/browse/MADLIB-1112 > > > On Mon, Jul 10, 2017 at 5:17 PM, Ed Espino <esp...@apache.org> wrote: > > > Frank, > > > > Thanks for the MADlib Top-Level Project (TLP) graduation update. Will > this > > mean the MADlib v1.12 release will be as an Apache TLP project? > > > > I have reviewed several TLP related Jiras thus far ( > > https://issues.apache.org/jira/issues/?jql=project%20% > > 3D%20%22Apache%20MADlib%22%20and%20summary%20~%20TLP). > > I saw the reference to details being added over time. Here are some that > > come to mind: > > > >- Update/remove "incubating" terminology: > > - Project's Release Process wiki documentation ( > > https://cwiki.apache.org/confluence/display/MADLIB/Release+Process > ). > > In addition to "incubating" terminology removal across entire wiki, > > there > > is no need for IPMC vote references. > >- Archive/Remove (not sure how) incubating subversion projects: > >- https://dist.apache.org/repos/dist/dev/incubator/madlib > > - https://dist.apache.org/repos/dist/release/incubator/madlib > > - Creation (not sure how) of TLP subversion projects: > > - https://dist.apache.org/repos/dist/dev/madlib > > - https://dist.apache.org/repos/dist/release/madlib > > - Source tree incubation references. The output (below) is from the > >"ack -i incubat" command on the master branch (c82c0f38). > > > > I'll continue to keep a list of items as I run across them. > > > > -=e > > > > > > -*- mode: ack; default-directory: "~/workspace/MADlib/incubator-madlib/" > > -*- > > Ack started at Mon Jul 10 17:13:16 > > > > ack -i incubat * > > DISCLAIMER > > 1:Apache MADlib is an effort undergoing incubation at the Apache Software > > 2:Foundation (ASF), sponsored by the Apache Incubator PMC. > > 4:Incubation is required of all newly accepted projects until a further > > 9:While incubation status is not necessarily a reflection of the > > > > HAWQ_Install.txt > > 1:Installing Apache MADlib (incubating) on Apache HAWQ (incubating) > > 4:Apache MADlib (incubating) is a library of statistics and machine > > learning > > 20:Apache MADlib is an effort undergoing incubation at the Apache > Software > > 21:Foundation (ASF), sponsored by the Apache Incubator PMC. > > 23:Incubation is required of all newly accepted projects until a further > > review > > 27:While incubation status is not necessarily a reflection of the > > completeness or > > > > LICENSE > > 204:Apache MADlib (incubating) Subcomponents: > > 206:The Apache MADlib (incubating) project contains subcomponents with > > separate > > > > NOTICE > > 1:Apache MADlib (incubating) > > > > README.md > > 12:See the project website [`MADlib Home`]( > > http://madlib.incubator.apache.org/) for links to the > > 30:-v (path to incubator-madlib directory):/incubator-madlib/ > > madlib/postgres_9.6 > > 31:# where incubator-madlib is the directory where the MADlib source code > > resides. > > 35:# make in the "incubator-madlib" folder inside the Docker container > will > > be > > 43:mkdir /incubator-madlib/build-docker > > 44:cd /incubator-madlib/build-docker > > 71:Docs`](http://madlib.incubator.apache.org/docs/latest/index.html). > > > > RELEASE_NOTES > > 8:commit history located at > > https://github.com/apache/incubator-madlib/commits/master. > > > > Release_Review_HOWTO.txt > > 18:Apache MADlib (incubating) is released as both source tarball and a > > series > > 77:Prior to the software grant to ASF on Sept 15, 2015 as an incubating > > project, > > 83:to ensure that the Apache MADlib (incubating) project was acting as a > > > > deploy/CMakeLists.txt > > 29:"Apache MADlib (incubating) is an Open-Source Library for Scalable > > in-Database Analytics") > > 34:set(CPACK_PACKAGE_VENDOR "Apache MADlib (incubating)") > > > > deploy/description.txt > > 1:Apache MADlib (incubating) is an open-source library
Re: Apache Jira: MADLIB v1.12-incubating and Metrics dashboard
deploy/PGXN/ReadMe.txt
1:Apache MADlib (incubating) Read Me
8:See the project web site located at http://madlib.incubator.apache.org/
for
14:The latest documentation of MADlib modules can be found at
http://madlib.incubator.apache.org/docs
27:
https://github.com/apache/incubator-madlib/blob/master/licenses/third_party/_M_widen_init.txt
65:Apache MADlib is an effort undergoing incubation at the Apache Software
66:Foundation (ASF), sponsored by the Apache Incubator PMC.
68:Incubation is required of all newly accepted projects until a further
73:While incubation status is not necessarily a reflection of the
doc/etc/developer.doxyfile.in
843:USE_MDFILE_AS_MAINPAGE = "
https://github.com/apache/incubator-madlib/blob/master/README.md;
doc/etc/header.html
30: ga('create', 'UA-45382226-1', 'madlib.incubator.apache.org');
44: http://madlib.incubator.apache.org;>
doc/mainpage.dox.in
3:Apache MADlib (incubating) is an open-source library for scalable
14:http://madlib.incubator.apache.org;>MADlib web site
17:https://mail-archives.apache.org/mod_mbox/incubator-madlib-user/;>User
mailing list
18:https://mail-archives.apache.org/mod_mbox/incubator-madlib-dev/;>Dev
mailing list
35:https://github.com/apache/incubator-madlib/blob/master/README.md
">ReadMe
38:https://github.com/apache/incubator-madlib/blob/master/LICENSE;>
src/madpack/madpack.py
698:<
http://madlib.incubator.apache.org/docs/latest/group__grp__linreg.html#warning
>
tool/docker/base/Dockerfile_gpdb_4_3_10
34:#ADD ./ /incubator-madlib
35:##RUN cd incubator-madlib && \
50:## 1) docker run -d -it --name gpdb -v
(path-to-incubator-madlib)/src:/incubator-madlib/src gpdb bash
53:## 2) docker exec -it gpdb /incubator-madlib/build/src/bin/madpack -p
greenplum -c gpadmin@127.0.0.1:5432/gpadmin install
59:## - cd /incubator-madlib/build
60:## - make (This can be run after changing code in the incubator-madlib
source code)
tool/docker/base/Dockerfile_postgres_9_6
56:## To build an image from this docker file, from incubator-madlib
folder, run:
tool/docker/base/Dockerfile_postgres_9_6_Jenkins
41:## To build an image from this docker file, from incubator-madlib
folder, run:
tool/jenkins/jenkins_build.sh
48:docker run -d --name madlib -v
"${workdir}/incubator-madlib":/incubator-madlib madlib/postgres_9.6:jenkins
| tee logs/docker_setup.log
50:docker run -d --name madlib -v
"${workdir}/incubator-madlib":/incubator-madlib madlib/postgres_9.6:jenkins
| tee logs/docker_setup.log
60:docker exec madlib bash -c 'rm -rf /build; mkdir /build; cd /build;
cmake ../incubator-madlib; make clean; make; make install; make package' |
tee $workdir/logs/madlib_compile.log
62:docker exec madlib bash -c 'rm -rf /build; mkdir /build; cd /build;
cmake ../incubator-madlib; make clean; make; make install; make package' |
tee $workdir/logs/madlib_compile.log
95:python incubator-madlib/tool/jenkins/junit_export.py
$workdir/logs/madlib_install_check.log
$workdir/logs/madlib_install_check.xml
97:python incubator-madlib/tool/jenkins/junit_export.py $workdir
$workdir/logs/madlib_install_check.log
$workdir/logs/madlib_install_check.xml
tool/jenkins/rat_check.sh
27:grep "Copyright 2016-$(date +"%Y") The Apache Software Foundation"
"${workdir}/incubator-madlib/NOTICE"
32:grep "$(cat
"${workdir}/incubator-madlib/src/config/Version.yml" | cut -d" "
-f2)" \
33:"${workdir}/incubator-madlib/pom.xml"
42:badfile_count=$(find . -name
"${workdir}/incubator-madlib/*.${extension}" | wc -l)
50:find . -name "${workdir}/incubator-madlib/*.${extension}"
Ack finished at Mon Jul 10 17:13:16
On Mon, Jul 10, 2017 at 3:41 PM, Frank McQuillan <fmcquil...@pivotal.io>
wrote:
> Thanks Ed, those dashboards are useful and give a good view of things.
>
> Regarding the 1.12 release timing, I suggest we move the release date until
> after the next ASF board meeting, which is scheduled for July 19, 2017. The
> reason is that MADlib graduation is on the agenda for the ASF meeting and
> hopefully it will pass fine. So I suggest the new release date for 1.12 is
> Aug 4, a couple weeks or so later. I updated the release date in JIRA.
>
> And yes, there is quite a lot of history on this project as it has been
> around since 2011 or so, well before the move the ASF in the fall of 2015.
>
> Frank
>
>
>
>
> On Mon, Jul 10, 2017 at 1:58 PM, Ed Espino <esp...@apache.org> wrote:
>
> > The automated Jira report for MADLIB Version v1.12 (UNRELEASED) is also
> > useful for getting a very quick view of the release status. It also
> > respects the tentative release date (14/Jul/17).
> >
> > https://issues.apache.org/jira/projects/MADLIB/versions/12340360
> >
> > -=e
> >
> > On Mon, Jul 10, 2017 at 1:04 PM, Ed Espino <esp...
Re: Apache Jira: MADLIB v1.12-incubating and Metrics dashboard
The automated Jira report for MADLIB Version v1.12 (UNRELEASED) is also useful for getting a very quick view of the release status. It also respects the tentative release date (14/Jul/17). https://issues.apache.org/jira/projects/MADLIB/versions/12340360 -=e On Mon, Jul 10, 2017 at 1:04 PM, Ed Espino <esp...@apache.org> wrote: > MADlibers, > > FYI: In order to get my head wrapped around the current Apache Jira state > for the MADlib v1.12 release, I have thrown together a quick dashboard. I > have made the dashboard and corresponding filters publicly available. This > will help me/us monitor the release convergence. > > Apache MADlib v1.12-incubating Release Dashboard: > https://issues.apache.org/jira/secure/Dashboard.jspa?selectPageId=12331450 > > Additionally, to get a status of the overall Jira state, I also threw > together a quick MADlib metrics dashboard. It appears there is a bit of > Jira legacy history with the project. : > https://issues.apache.org/jira/secure/Dashboard.jspa?selectPageId=12331451 > > Please take a quick look and let me know what you think. I can easily > adjust the dashboards if needed. > > Regards, > -=e > > > -- > *Ed Espino* > -- *Ed Espino*
Apache Jira: MADLIB v1.12-incubating and Metrics dashboard
MADlibers, FYI: In order to get my head wrapped around the current Apache Jira state for the MADlib v1.12 release, I have thrown together a quick dashboard. I have made the dashboard and corresponding filters publicly available. This will help me/us monitor the release convergence. Apache MADlib v1.12-incubating Release Dashboard: https://issues.apache.org/jira/secure/Dashboard.jspa?selectPageId=12331450 Additionally, to get a status of the overall Jira state, I also threw together a quick MADlib metrics dashboard. It appears there is a bit of Jira legacy history with the project. : https://issues.apache.org/jira/secure/Dashboard.jspa?selectPageId=12331451 Please take a quick look and let me know what you think. I can easily adjust the dashboards if needed. Regards, -=e -- *Ed Espino*
Re: Volunteer: Apache MADlib 1.12 (incubating) release manager
Thank you Frankie. I'm really looking forward to this opportunity. Thank you Trevor for the kind words. Don't be surprised if I take you up on your offer sometime in the future. To prepare, I will review MADlib's release process. Talk to you soon, -=e On Fri, Jul 7, 2017 at 1:58 PM Frank McQuillan <fmcquil...@pivotal.io> wrote: > Hi Ed, > > Thank you kindly for your offer to be release manager for 1.12! > We heartily accept your offer! > > And it is great that you have experience > on HAWQ - I think the MADlib release process will be very similar to > what you are used to. > > We have put together a wiki page on the MADlib release process > https://cwiki.apache.org/confluence/display/MADLIB/Release+Process > so you can have a look there and see the steps. Hopefully no surprises. > > We are looking at releasing 1.12 within the next month, depending on > community wishes, and we will be happy to work thru the steps with you. > > Again, thanks again for the offer, and we'll talk soon! > > Frank > > > > On Fri, Jul 7, 2017 at 12:49 PM, Trevor Grant <trevor.d.gr...@gmail.com> > wrote: > > > ... that's a very nice gesture- > > > > I'm only a lurker on this mailing list but I'm a PMC on a couple of other > > projects- would be happy to take you up if these folks don't :D > > > > > > > > On Fri, Jul 7, 2017 at 2:42 PM, Ed Espino <esp...@apache.org> wrote: > > > > > MADlib dev, > > > > > > I'm not sure if one has been identified and even though I am not a > > > committer on the project, I would like to volunteer my services to be > the > > > release manager for the upcoming Apache MADlib 1.12 (incubating). I > have > > > served in this capacity for the Apache HAWQ 2.1.0.0-incubating release > > > (references below). I have had the chance to review several of the > > > previous MADlib releases. I am looking forward to hone my ASF skill set > > and > > > this looks like a very good opportunity. > > > > > > Regards, > > > -=e > > > > > > My release manager participation references: > > > Apache HAWQ 2.1.0.0-incubating dev voting thread: > > > https://lists.apache.org/thread.html/9d3025c12dc032437d1317d662f0e4 > > > 434754c00258ca1abdd5c0ab9f@%3Cdev.hawq.apache.org%3E > > > > > > Apache HAWQ 2.1.0.0-incubating IPMC voting thread: > > > https://lists.apache.org/thread.html/1636e892b95475fe0af130d83fa457 > > > c3e8bfa0d26f695f6faac0@%3Cgeneral.incubator.apache.org%3E > > > > > > -- > > > *Ed Espino* > > > > > > -- -- Ed Espino Director, Pivotal Technical Staff Cell: 925.389.4640 WeChat ID: EdEspino
Re: [VOTE] MADlib v1.11-rc3
+1 (better late than never) 1) I apologize for falling off the radar. Upon my return to the US from Beijing, I came down with a lower respiratory tract infection. No fun. Finally, I'm feeling a wee bit better. 2) I am very satisfied with the resolution of my previous release candidate observations (for rc1 & rc2). I want to thank the community for allowing me to participate. -=e -- *Ed Espino* On Fri, May 5, 2017 at 9:34 AM, Frank McQuillan <fmcquil...@pivotal.io> wrote: > I just want to comment on a couple items raised in the RC1 and RC2 votes > that pertain to RC3: > > (1) > “I happened to open the file "CMakeLists.txt" in the root directory > and noticed it does not have the standard ASF header. I know there > were IP issues resolved globally for the project recently. I > noticed many of them are excluded in the pom.xml file. Regardless > of the IP issues, shouldn't these files contain the ASF header?” > > Since this file existed before MADlib’s move to ASF, it does not need an > ASF header as per the guidance from ASF on this topic > https://issues.apache.org/jira/browse/LEGAL-293 > > (2) > “The DMG（apache-madlib-1.11-incubating-bin-Darwin.dmg) contains a > pkg file named "madlib-1.11-Darwin.pkg". Shouldn't it be called > "apache-madlib-1.11-incubating-Darwin.pkg"? > > Similarly, the DMG base folder name is madlib-1.11.Darwin.“ > > As per guidance from Roman our mentor, it is not necessary to rename all > packages and files. Also, this may affect some functional tests that look > for certain file names. > > (3) > “There are still three outstanding Jira issues in an "Unresolved" state > with a fix version of v1.11. Are they going to be resolved soon? They can > be seen with the following url: > > https://issues.apache.org/jira/browse/MADLIB/fixforversion/12339592/? > selectedTab=com.atlassian.jira.jira-projects-plugin:version-summary-panel > ” > > Regarding the JIRAs that are not closed, the actual work has been done so > there is nothing material pending. But I did not close them because I > wanted Roman to do that, since he was the one overseeing them. > > (4) > Convenience binaries are being voted on, as Rashmi’s email calls out. > > (5) > I tried out the RC3 dmg and found that install, reinstall, upgrade work > fine with the soft link on my OS X box on PG 9.6 > > So... > > > +1 > > > > > On Thu, May 4, 2017 at 6:10 PM, Rashmi Raghu <rra...@pivotal.io> wrote: > > > Hello MADlib community, > > > > We have created a MADlib 1.11 RC-3, with the artifacts below (source and > > convenience binaries) up for a vote. > > > > Note that voting for the RC-2 release has been cancelled due to the need > > for minor corrections based on community feedback. Sorry for the > > inconvenience. > > > > RC-3 replaces RC-2 with the following minor changes: > > * Ensure product naming is consistently 'Apache MADlib (incubating)' > > * Git revision tag changed to rc/1.11-rc3 > > > > This will be the 5th release for Apache MADlib (incubating). > > > > The main goals of this release are: > > * new module (PageRank for graph analytics with grouping support > included) > > * improvements to existing modules (add grouping support to Single Source > > Shortest Path, reduce memory footprint of DT and RF, include NULL > features > > in training DT, add support for array and svec output for Pivot module, > > utility to unnest 2-D arrays into rows of 1-D arrays) > > * platform updates (GPDB 5) > > * updates for Apache Top Level Project readiness and build process on > > Apache infrastructure > > * bug fixes > > * doc improvements > > > > For more information including release notes, please see: > > https://cwiki.apache.org/confluence/display/MADLIB/MADlib+1.11 > > > > *** Please download, review and vote by Tue May 09, 2017 @ 6pm PDT *** > > > > We're voting upon the source and convenience binaries below: > > > > Source Repository (tag): rc/1.11-rc3 > > https://github.com/apache/incubator-madlib/tree/rc/1.11-rc3 > > > > Source Files and convenience Binaries: > > https://dist.apache.org/repos/dist/dev/incubator/madlib/1. > > 11-incubating-rc3/ > > > > Commit: > > https://github.com/apache/incubator-madlib/commit/ > > 8e2778a3921aa99f009962756881ce4bea5eee16 > > > > KEYS file containing PGP Keys we use to sign the release: > > https://dist.apache.org/repos/dist/dev/incubator/madlib/KEYS > > > > To help in tallying the vote, PMC members please be sure to indicate > > "(binding)" with the vote. > > > > [ ] +1 approve > > [ ] +0 no opinion > > [ ] -1 disapprove (and reason why) > > > > > > Regards, > > Rashmi Raghu > > > > -- > > Rashmi Raghu, Ph.D. > > Pivotal Data Science > > >
Re: [VOTE] MADlib v1.11-rc2
I have taken a quick look at the DMG and a Linux RPM binary artifacts (sorry haven't had time to build and/or test the binaries yet). But this info might be of some benefit to the team sooner than later. Regards, -=e -- *Ed Espino* == PGP signature (source and convenience binaries): good == Hashes (source and convenience binaries): good == Source directory layout: good == Source LICENSE, DISCLAIMER & NOTICE: good == Apache Release Audit Tool (RAT): Observation: - I happened to open the file "CMakeLists.txt" in the root directory and noticed it does not have the standard ASF header. I know there were IP issues resolved globally for the project recently. I noticed many of them are excluded in the pom.xml file. Regardless of the IP issues, shouldn't these files contain the ASF header? == Source miscelaneous: HAWQ_Install.txt Observation: - The file references the product name as "MADlib" and not "Apache MADlib (Incubating). Is this file still valid? == CONVENIENCE BINARIES -- -- Mac Installer DMG file: apache-madlib-1.11-incubating-bin-Darwin.dmg -- Observation: - The DMG（apache-madlib-1.11-incubating-bin-Darwin.dmg) contains a pkg file named "madlib-1.11-Darwin.pkg". Shouldn't it be called "apache-madlib-1.11-incubating-Darwin.pkg"? Similarly, the DMG base folder name is madlib-1.11.Darwin. Mac Installer Package o Introduction screen Observation: - The introduction screen identifies the product name as "MADlib". Shouldn't there be a mention of the project name being "Apache MADlib (Incubating)". o Read Me screen Observation: - Similar to initial screen, there is no mention to the Apache project except for the link to the project's wiki. o Remaining screens look reasonable (with exception of no Apache references). o The default application window name is "Install MADlib" Observation: - Similar to Introduction sreen, should the name be "Install Apache MADlib (Incubating)"? - Look for other opportunities to reference the product name as "Apache MADlib (Incubating)". -- Linux RPM: apache-madlib-1.11-incubating-bin-Linux.rpm -- Observation: - It appears the SPEC file used (possibly generated) references the product name as "madlib". Again, shouldn't there be references to the product name as "Apache MADlib" scattered about? Unfortunately, I am not sure if this should change or not. It might help for someone on the team to review other Apache projects convenience binary RPMs to see if something should be addressed. The podling's mentor might be able to provide additional direction as well. This can be seen in the following "rpm -qi madlib" output: [root@e0f4d3349d2d MADlib]# rpm -qi madlib Name: madlib Version : 1.11 Release : 1 Architecture: x86_64 Install Date: Wed May 3 04:00:10 2017 Group : Development/Libraries Size: 83575356 License : ASL 2.0 Signature : (none) Source RPM : madlib-1.11-1.src.rpm Build Date : Tue May 2 19:03:21 2017 Build Host : gpdb1.eng.pivotal.io Relocations : /usr/local Vendor : MADlib Summary : Open-Source Library for Scalable in-Database Analytics Description : MADlib is an open-source library for scalable in-database analytics. It provides data-parallel implementations of mathematical, statistical and machine learning methods for structured and unstructured data. The MADlib mission: to foster widespread development of scalable analytic skills, by harnessing efforts from commercial practice, academic research, and open-source development. To more information, please see the MADlib wiki at https://cwiki.apache.org/confluence/display/MADLIB -- On Wed, May 3, 2017 at 8:13 AM, Frank McQuillan &l
Re: [VOTE] MADlib v1.11-rc1
gt; > > >>>>>>>> Shortest Path, reduce memory footprint of DT and RF, include > > > >>>>>>>> NULL > > > >>>>>>> features > > > >>>>>>>> in training DT, add support for array and svec output for > > > >>>>>>>> Pivot > > > >>>>> module, > > > >>>>>>>> utility to unnest 2-D arrays into rows of 1-D arrays) > > > >>>>>>>> * platform updates (GPDB 5) > > > >>>>>>>> * updates for Apache Top Level Project readiness and build > > > >>>>>>>> process > > > >> on > > > >>>>>>>> Apache infrastructure > > > >>>>>>>> * bug fixes > > > >>>>>>>> * doc improvements > > > >>>>>>>> > > > >>>>>>>> For more information including release notes, please see: > > > >>>>>>>> https://cwiki.apache.org/confluence/display/MADLIB/ > MADlib+1.11 > > > >>>>>>>> > > > >>>>>>>> *** Please download, review and vote by Thu May 04, 2017 @ > > > >>>>>>>> 6pm PDT > > > >>>>> *** > > > >>>>>>>> > > > >>>>>>>> We're voting upon the source (tag): rc/1.11-rc1 > > > >>>>>>>> https://github.com/apache/incubator-madlib/tree/rc/1.11-rc1 > > > >>>>>>>> > > > >>>>>>>> Source Files: > > > >>>>>>>> https://dist.apache.org/repos/dist/dev/incubator/madlib/1. > > > >>>>>>>> 11-incubating-rc1/ > > > >>>>>>>> > > > >>>>>>>> Commit to be voted upon: > > > >>>>>>>> https://github.com/apache/incubator-madlib/commit/ > > > >>>>>>>> 0ff829a7060d08f284e8468ebf35c31b6e231d58 > > > >>>>>>>> > > > >>>>>>>> KEYS file containing PGP Keys we use to sign the release: > > > >>>>>>>> https://dist.apache.org/repos/dist/dev/incubator/madlib/KEYS > > > >>>>>>>> > > > >>>>>>>> To help in tallying the vote, PMC members please be sure to > > > >>>>>>>> indicate "(binding)" with the vote. > > > >>>>>>>> > > > >>>>>>>> [ ] +1 approve > > > >>>>>>>> [ ] +0 no opinion > > > >>>>>>>> [ ] -1 disapprove (and reason why) > > > >>>>>>>> > > > >>>>>>>> > > > >>>>>>>> Regards, > > > >>>>>>>> Rashmi Raghu > > > >>>>>>>> > > > >>>>>>>> -- > > > >>>>>>>> Rashmi Raghu, Ph.D. > > > >>>>>>>> Pivotal Data Science > > > >>>>>>>> > > > >>>>>>> > > > >>>>>> > > > >>>>>> > > > >>>>> > > > >>>> > > > >>>> > > > >>> > > > >> > > > > > > > > > > > > > > > > -- > > > > Ivan Novick > > > > Product Manager Pivotal Greenplum > > > > https://www.youtube.com/GreenplumDatabase > > > > > > -- > *Ed Espino* >
Re: [VOTE] MADlib v1.11-rc1
Requesting a point of vote clarification: Caveat: I am relatively new to the Apache HAWQ incubating project and my level of experience with general Apache processes (incubating or otherwise) is limited. Thus, my point of clarification and follow up question may be misguided. I apologize if this is the case. The "[VOTE] MADlib v1.11-rc1" email indicates: "We're voting upon the source (tag): rc/1.11-rc1". I have been wondering when the initial binary release would be coming up for vote. Especially, in light of the use of Apache Infrastructure for build processes note. As I was reviewing the source tarball in on the Apache dist server ( https://dist.apache.org/repos/dist/dev/incubator/madlib/1.11-incubating-rc1/), I noticed along with the source tarballs (and supporting pgp signature and hashes) there are also what looks like binary files (and supporting pgp signature and hashes). Previous votes only included source tarballs on the dist server. So the point of clarification I'm seeking, are the binaries also up for vote? Additionally, the Apache HAWQ incubating project is undergoing our first release that is including convenience binaries. This initial vote is explicitly calling out the source and convenience binaries being voted on. I haven't seen a reference to binaries being part of any previous Apache MADlib vote process. I happened to look at the previous 1.10.0 release directory on the Apache dist server ( https://dist.apache.org/repos/dist/release/incubator/madlib/1.10.0-incubating/) and was surprised to find binary files as well (this also applies to 1.9 & 1.9.1). Shouldn't the binary files have been voted on by the community before being added to the release area on the dist server? -=e -- *Ed Espino* On Tue, May 2, 2017 at 6:55 AM, Rashmi Raghu <rra...@pivotal.io> wrote: > Hello MADlib community, > > We have created a MADlib 1.11 RC-1, with the artifacts below up for a vote. > > This will be the 5th release for Apache MADlib (incubating). > > The main goals of this release are: > * new module (PageRank for graph analytics with grouping support included) > * improvements to existing modules (add grouping support to Single Source > Shortest Path, reduce memory footprint of DT and RF, include NULL features > in training DT, add support for array and svec output for Pivot module, > utility to unnest 2-D arrays into rows of 1-D arrays) > * platform updates (GPDB 5) > * updates for Apache Top Level Project readiness and build process on > Apache infrastructure > * bug fixes > * doc improvements > > For more information including release notes, please see: > https://cwiki.apache.org/confluence/display/MADLIB/MADlib+1.11 > > *** Please download, review and vote by Thu May 04, 2017 @ 6pm PDT *** > > We're voting upon the source (tag): rc/1.11-rc1 > https://github.com/apache/incubator-madlib/tree/rc/1.11-rc1 > > Source Files: > https://dist.apache.org/repos/dist/dev/incubator/madlib/1. > 11-incubating-rc1/ > > Commit to be voted upon: > https://github.com/apache/incubator-madlib/commit/ > 0ff829a7060d08f284e8468ebf35c31b6e231d58 > > KEYS file containing PGP Keys we use to sign the release: > https://dist.apache.org/repos/dist/dev/incubator/madlib/KEYS > > To help in tallying the vote, PMC members please be sure to indicate > "(binding)" with the vote. > > [ ] +1 approve > [ ] +0 no opinion > [ ] -1 disapprove (and reason why) > > > Regards, > Rashmi Raghu > > -- > Rashmi Raghu, Ph.D. > Pivotal Data Science >
Apache Jenkins MADlib projects
I see Apache Jenkins build service testing in madlib-test-build <https://builds.apache.org/job/madlib-test-build/> is being worked on. This is pretty cool for the dev community. Is there a set of projects and GitHub *master* branch and *Pull Request* (PR) integration points being worked on? For what it is worth, here are some integration points we have for the HAWQ project that may be of use to MADlib: - For each Pull Request (PR), perform the following checks (these go along with the default conflict check performed automatically by github): - Perform build (compilation) and Apache Release Audit Tool (RAT) check - For each master branch submission: - Perform build (compilation) - Perform Apache Release Audit Tool (RAT) check - Add "Embeddable Build Status Icon" to the project's README.md: https://builds.apache.org/job/madlib-test-build/badge/ Cheers, -=e -- *Ed Espino*
Re: [VOTE] MADlib v1.10-rc1
OK (Orhan Kislal), Thanks for the info (Apache ID signature preferred). That will come in handy for one of our future Apache HAWQ incubating releases. Cheers, -=e On Thu, Feb 16, 2017 at 12:06 PM, Orhan Kislal <okis...@pivotal.io> wrote: > Hi Ed, > > Thanks for the review. One of the comments from the previous release was a > preference towards a signature with an Apache id. Since Satoshi-san is not > an Apache committer yet, I took care of the signing process. > > Thanks, > > Orhan Kislal > > On Thu, Feb 16, 2017 at 11:58 AM, Ed Espino <esp...@apache.org> wrote: > > > A few MADlib v1.10-rc1 observations from a HAWQ incubator committer. > > > >- The Copyright year (2016) in the NOTICE file needs to be updated to > >2017. I believe this can be handled in next release. > >- As it still applies, similar to a past comment by Roman ([VOTE] > MADlib > >v1.9.1-rc2 > ><https://lists.apache.org/thread.html/981b4c24eaa2ab069b8e18f7aa4bdd > > c7a78d3a9dc26bf659af94fcfe@%3Cgeneral.incubator.apache.org%3E>) > >- *"* name of the top level folder in the archive is weird. The usual > >practice is to call the top level folder as - > ID>*"* > > (example: *apache-madlib-src-1.10-incubating* instead of > >*incubator-madlib*) > >- I'm more curious than anything. Why did Orhan sign the release? I > was > >expecting the release manager (Satoshi Nagayasu) to have signed the > > release. > >- Checksums and PGP signature are good. > >- ASF headers check: I spot checked files added (git whatchanged > >--diff-filter=A) since the last release. ASF headers look good. Nice > > Job! > > > > I was going to try and build but I ran past my allotted time limit for > this > > review. Hopefully, I can try this soon. > > > > Regards, > > -=ed espino > > > > On Thu, Feb 16, 2017 at 10:05 AM, Orhan Kislal <okis...@pivotal.io> > wrote: > > > > > +1 > > > > > > Orhan Kislal > > > > > > On Thu, Feb 16, 2017 at 9:23 AM, Joe Hellerstein < > > hellerst...@berkeley.edu > > > > > > > wrote: > > > > > > > +1 > > > > > > > > Sent from a telephone. > > > > > > > > > On Feb 16, 2017, at 9:17 AM, Frank McQuillan < > fmcquil...@pivotal.io> > > > > wrote: > > > > > > > > > > +1 > > > > > > > > > > Frank McQuillan > > > > > > > > > >> On Wed, Feb 15, 2017 at 7:27 PM, Satoshi Nagayasu < > sn...@uptime.jp> > > > > wrote: > > > > >> > > > > >> Hello MADlib community, > > > > >> > > > > >> We have created a MADlib 1.10 RC-1, with the artifacts below up > for > > a > > > > vote. > > > > >> > > > > >> This will be the 4th release for Apache MADlib (incubating). > > > > >> > > > > >> The main goals of this release are: > > > > >> * new modules (single source shortest path for graph analytics, > > encode > > > > >> categorical variables, K-nearest neighbors) > > > > >> * improvements to existing modules (add grouping support to > elastic > > > > >> net and PCA, add cross validation to elastic net, array input for > > > > >> K-means, verbose output option for DT and RF, limit itemset size > in > > > > >> association rules, various madpack installer improvements) > > > > >> * platform updates (PostgreSQL 9.6) > > > > >> * bug fixes > > > > >> * doc improvements > > > > >> > > > > >> For more information including release notes, please see: > > > > >> https://cwiki.apache.org/confluence/display/MADLIB/MADlib+1.10 > > > > >> > > > > >> *** Please download, review and vote by Sat Feb 18, 2017 @ 6pm PST > > *** > > > > >> > > > > >> We're voting upon the source (tag): rc/1.10.0-rc1 > > > > >> https://github.com/apache/incubator-madlib/tree/rc/1.10.0-rc1 > > > > >> > > > > >> Source Files: > > > > >> https://dist.apache.org/repos/dist/dev/incubator/madlib/1. > > > > >> 10.0-incubating-rc1/ > > > > >> > > > > >> Commit to be voted upon: > > > > >> https://github.com/apache/incubator-madlib/commit/ > > > > >> ea17530bfe22a1fde173d7fa83508cbcd9924c20 > > > > >> > > > > >> KEYS file containing PGP Keys we use to sign the release: > > > > >> https://dist.apache.org/repos/dist/dev/incubator/madlib/KEYS > > > > >> > > > > >> To help in tallying the vote, can PMC members please be sure to > > > > >> indicate "(binding)" with their vote. > > > > >> > > > > >> [ ] +1 approve > > > > >> [ ] +0 no opinion > > > > >> [ ] -1 disapprove (and reason why) > > > > >> > > > > >> -- > > > > >> Satoshi Nagayasu <sn...@uptime.jp> > > > > >> > > > > > > > > > > > > > > > -- > > *Ed Espino* > > *esp...@apache.org <esp...@apache.org>* > > > -- *Ed Espino* *esp...@apache.org <esp...@apache.org>*
