Re: [VOTE] Apache SystemML 0.11.0-incubating (RC1)

2016-10-02 Thread Matthias Boehm 

yes, I just closed them - I left them open for Mike to confirm, but we
resolved all known issues yesterday together. We should be good to go.

Regards,
Matthias



From:   Luciano Resende 
To: dev@systemml.incubator.apache.org
Date:   10/02/2016 08:30 PM
Subject:Re: [VOTE] Apache SystemML 0.11.0-incubating (RC1)



I still see the following jiras, which were mentioned on this thread, open:

https://issues.apache.org/jira/browse/SYSTEMML-993
https://issues.apache.org/jira/browse/SYSTEMML-994
https://issues.apache.org/jira/browse/SYSTEMML-995

Did folks forgot to clode the jiras ? Or are there things that still need
to be handled here ?


On Sat, Oct 1, 2016 at 2:41 PM, Matthias Boehm  wrote:

> ok the blocking issues SYSTEMML-993, 994, and 995 have been resolved -
> from my perspective we're ready to cut a new RC.
>
> Regards,
> Matthias
>
> [image: Inactive hide details for Matthias Boehm---09/29/2016 10:44:51
> PM---just a quick update: SYSTEMML-969 has been resolved too. Th]Matthias
> Boehm---09/29/2016 10:44:51 PM---just a quick update: SYSTEMML-969 has
been
> resolved too. The open issues are SYSTEMML-993, SYSTEMML-
>
> From: Matthias Boehm/Almaden/IBM@IBMUS
> To: dev@systemml.incubator.apache.org
> Date: 09/29/2016 10:44 PM
> Subject: Re: [VOTE] Apache SystemML 0.11.0-incubating (RC1)
> --
>
>
>
> just a quick update: SYSTEMML-969 has been resolved too. The open issues
> are SYSTEMML-993, SYSTEMML-994, and the new SYSTEMML-995. We should be
able
> to resolve them by tomorrow to give everybody a chance of testing a new
RC
> over the weekend.
>
> Regards,
> Matthias
>
> Acs S ---09/29/2016 05:31:23 PM---SYSTEMML-964 being addressed (I added
> changes and with UTF support Matthias added he reverted change
>
> From: Acs S 
> To: "dev@systemml.incubator.apache.org"
 >
> Date: 09/29/2016 05:31 PM
> Subject: Re: [VOTE] Apache SystemML 0.11.0-incubating (RC1)
> --
>
>
>
> SYSTEMML-964 being addressed (I added changes and with UTF support
> Matthias added he reverted changes)
>
> -Arvind
>
> From: "dusenberr...@gmail.com" 
> To: dev@systemml.incubator.apache.org
> Sent: Thursday, September 29, 2016 2:31 PM
> Subject: Re: [VOTE] Apache SystemML 0.11.0-incubating (RC1)
>
> I've also opened SYSTEMML-993 that relates to poor performance for vector
> DataFrame conversions, as well as SYSTEMML-994 for GC OOM on SystemML
> matrix to frame conversions that would both be good to work on.
>
> --
>
> Mike Dusenberry
> GitHub: github.com/dusenberrymw
> LinkedIn: linkedin.com/in/mikedusenberry
>
> Sent from my iPhone.
>
>
> > On Sep 29, 2016, at 12:32 PM, Luciano Resende 
> wrote:
> >
> >> On Thu, Sep 29, 2016 at 11:11 AM, Matthias Boehm 
> wrote:
> >>
> >> SYSTEMML-968 has been resolved too but we're still waiting for
> >> SYSTEMML-964. Furthermore, there is also a nice-to-have feature we
want
> to
> >> get it in: SYSTEMML-969 (extended dataframe - frame converter).
> >>
> >> Regards,
> >> Matthias
> > Great progress !!!
> >
> > Matthias, please let us know when these issues get resolved and I will
> work
> > on RC2.
> >
> > --
> > Luciano Resende
> > *http://twitter.com/lresende1975* 
> > *http://lresende.blogspot.com/* 
>
>
>
>
>


--
Luciano Resende
http://twitter.com/lresende1975
http://lresende.blogspot.com/

Re: [VOTE] Apache SystemML 0.11.0-incubating (RC1)

2016-10-02 Thread Luciano Resende 
I still see the following jiras, which were mentioned on this thread, open:

https://issues.apache.org/jira/browse/SYSTEMML-993
https://issues.apache.org/jira/browse/SYSTEMML-994
https://issues.apache.org/jira/browse/SYSTEMML-995

Did folks forgot to clode the jiras ? Or are there things that still need
to be handled here ?


On Sat, Oct 1, 2016 at 2:41 PM, Matthias Boehm  wrote:

> ok the blocking issues SYSTEMML-993, 994, and 995 have been resolved -
> from my perspective we're ready to cut a new RC.
>
> Regards,
> Matthias
>
> [image: Inactive hide details for Matthias Boehm---09/29/2016 10:44:51
> PM---just a quick update: SYSTEMML-969 has been resolved too. Th]Matthias
> Boehm---09/29/2016 10:44:51 PM---just a quick update: SYSTEMML-969 has been
> resolved too. The open issues are SYSTEMML-993, SYSTEMML-
>
> From: Matthias Boehm/Almaden/IBM@IBMUS
> To: dev@systemml.incubator.apache.org
> Date: 09/29/2016 10:44 PM
> Subject: Re: [VOTE] Apache SystemML 0.11.0-incubating (RC1)
> --
>
>
>
> just a quick update: SYSTEMML-969 has been resolved too. The open issues
> are SYSTEMML-993, SYSTEMML-994, and the new SYSTEMML-995. We should be able
> to resolve them by tomorrow to give everybody a chance of testing a new RC
> over the weekend.
>
> Regards,
> Matthias
>
> Acs S ---09/29/2016 05:31:23 PM---SYSTEMML-964 being addressed (I added
> changes and with UTF support Matthias added he reverted change
>
> From: Acs S 
> To: "dev@systemml.incubator.apache.org"  >
> Date: 09/29/2016 05:31 PM
> Subject: Re: [VOTE] Apache SystemML 0.11.0-incubating (RC1)
> --
>
>
>
> SYSTEMML-964 being addressed (I added changes and with UTF support
> Matthias added he reverted changes)
>
> -Arvind
>
> From: "dusenberr...@gmail.com" 
> To: dev@systemml.incubator.apache.org
> Sent: Thursday, September 29, 2016 2:31 PM
> Subject: Re: [VOTE] Apache SystemML 0.11.0-incubating (RC1)
>
> I've also opened SYSTEMML-993 that relates to poor performance for vector
> DataFrame conversions, as well as SYSTEMML-994 for GC OOM on SystemML
> matrix to frame conversions that would both be good to work on.
>
> --
>
> Mike Dusenberry
> GitHub: github.com/dusenberrymw
> LinkedIn: linkedin.com/in/mikedusenberry
>
> Sent from my iPhone.
>
>
> > On Sep 29, 2016, at 12:32 PM, Luciano Resende 
> wrote:
> >
> >> On Thu, Sep 29, 2016 at 11:11 AM, Matthias Boehm 
> wrote:
> >>
> >> SYSTEMML-968 has been resolved too but we're still waiting for
> >> SYSTEMML-964. Furthermore, there is also a nice-to-have feature we want
> to
> >> get it in: SYSTEMML-969 (extended dataframe - frame converter).
> >>
> >> Regards,
> >> Matthias
> > Great progress !!!
> >
> > Matthias, please let us know when these issues get resolved and I will
> work
> > on RC2.
> >
> > --
> > Luciano Resende
> > *http://twitter.com/lresende1975* 
> > *http://lresende.blogspot.com/* 
>
>
>
>
>


-- 
Luciano Resende
http://twitter.com/lresende1975
http://lresende.blogspot.com/

Re: Enhancing SystemML JavaDocs

2016-10-02 Thread Niketan Pansare 

Since we also now have Python APIs, it might be a good idea to consider
hosting them too along with javadocs. Here is a link to our current
documentation:
https://htmlpreview.github.io/?https://github.com/apache/incubator-systemml/blob/master/docs/devdocs/python_api.html

This documentation can be generated using following steps:
$ cd src/main/python
$ sphinx-apidoc --full --force --output-dir docs systemml
$ cd docs

Add following lines to conf.py
import sys
sys.path.append('...FULL PATH...src/main/python/')
autodoc_mock_imports = ['pyspark', 'py4j.java_gateway',
'pyspark.mllib.common', 'pyspark.context', 'pyspark.sql', 'pyspark.ml',
'pyspark.ml.feature']

$ make singlehtml

As an FYI, Spark moved to Sphinx in version 1.2.0 (please see SPARK-3420).

Thanks,

Niketan Pansare
IBM Almaden Research Center
E-mail: npansar At us.ibm.com
http://researcher.watson.ibm.com/researcher/view.php?person=us-npansar



From:   Frederick R Reiss/Almaden/IBM@IBMUS
To: dev@systemml.incubator.apache.org
Date:   09/30/2016 04:33 PM
Subject:Re: Enhancing SystemML JavaDocs



Wow, I was not aware that those empty JavaDocs were put in -- and left in
-- on purpose! To me they are the epitome of useless comments. At best they
convey exactly the same information as the line of code immediately below
them. Much of the time they end up conveying an incorrect, out-of-date
version of the same. And all of the time they limit the amount of useful
code that can be on screen at once. Are there many people on the project
who actually like the current JavaDoc policy?

Fred

Luciano Resende ---09/30/2016 03:18:41 PM---On Fri, Sep 30, 2016 at 1:42
PM, Deron Eriksson  wrote:

From: Luciano Resende 
To: dev@systemml.incubator.apache.org
Date: 09/30/2016 03:18 PM
Subject: Re: Enhancing SystemML JavaDocs



On Fri, Sep 30, 2016 at 1:42 PM, Deron Eriksson 
wrote:

> I do not see how these automatically generated javadocs are useful. For
> instance:
>
> /**
> *
> * @param pb
> * @param ec
> * @return
> * @throws DMLRuntimeException
> */
> public static double getTimeEstimate(ProgramBlock pb, ExecutionContext
ec,
> boolean recursive)
> throws DMLRuntimeException
>
> Here, someone has automatically generated a javadoc comment. The
developer
> has failed to correct the missing 'recursive' parameter. If a developer
has
> not created a blank javadoc comment in the first place, then the
recursive
> parameter mistake never would have been made because there never would
have
> been a blank javadoc comment to update in the first place.
>
> If automatically generated javadoc comments are decided to be part of our
> coding standard, then they should be applied to all methods, not just
> random methods.
>
> Deron
>
>

Completely agree Deron, these are becoming stale and obsolete with the APIs
being enhanced and the javadocs being left behind. In the past I actually
tried to fix some, but as you mentioned there is a lot and we need a
community effort here.

Another approach is to make javadocs update MANDATORY for PR acceptance,
meaning that, if you touch a given file, you fix any javadoc related to
that file.

Thoughts ?


--
Luciano Resende
http://twitter.com/lresende1975
http://lresende.blogspot.com/

Jenkins build is back to normal : SystemML-DailyTest #540

2016-10-02 Thread jenkins 
See