Github user gliptak closed the pull request at:
https://github.com/apache/spark/pull/9710
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is
Github user gliptak closed the pull request at:
https://github.com/apache/spark/pull/11853
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is
Github user gliptak commented on the pull request:
https://github.com/apache/spark/pull/11853#issuecomment-198951746
@srowen I asked for clarification on the other item listed ...
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub
Github user gliptak commented on the pull request:
https://github.com/apache/spark/pull/9710#issuecomment-198950986
@JoshRosen Are you still considering this for inclusion? Thanks
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub
GitHub user gliptak opened a pull request:
https://github.com/apache/spark/pull/11853
SPARK-13461 Delete duplicate example TrainValidationSplitExample.scala
https://issues.apache.org/jira/browse/SPARK-13461
Validated that
examples/src/main/scala/org/apache/spark/examples
Github user gliptak commented on the pull request:
https://github.com/apache/spark/pull/9710#issuecomment-187874408
@JoshRosen Could you offer pointers on how to move this forward? Thanks
---
If your project is set up for it, you can reply to this email and have your
reply appear on
Github user gliptak commented on the pull request:
https://github.com/apache/spark/pull/9710#issuecomment-183092452
Rebased.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user gliptak commented on the pull request:
https://github.com/apache/spark/pull/9532#issuecomment-181051318
@srowen Two tests failed with timeout. Unrelated?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If
Github user gliptak commented on the pull request:
https://github.com/apache/spark/pull/9710#issuecomment-181020787
@JoshRosen Could you offer ointers on how to move this forward? Thanks
---
If your project is set up for it, you can reply to this email and have your
reply appear on
Github user gliptak commented on the pull request:
https://github.com/apache/spark/pull/9532#issuecomment-180867438
OK, back to my initial confusion on this. So the pom.xml change is needed
to make this work ...
---
If your project is set up for it, you can reply to this email and
Github user gliptak commented on the pull request:
https://github.com/apache/spark/pull/9532#issuecomment-180853555
@srowen Yes I removed the pom.xml reference, and was successful building
locally. Maven dependencies might have changed since I initially issued the
patch ...
---
If
Github user gliptak commented on the pull request:
https://github.com/apache/spark/pull/9532#issuecomment-180826210
The deprecated call was updated with this commit.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If
Github user gliptak commented on the pull request:
https://github.com/apache/spark/pull/9532#issuecomment-177521123
@srowen if you are proposed changing dependencies, would a follow-up JIRA
work for that?
---
If your project is set up for it, you can reply to this email and have
Github user gliptak commented on the pull request:
https://github.com/apache/spark/pull/9532#issuecomment-176486910
@srowen commons-codec is already in the section
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user gliptak commented on the pull request:
https://github.com/apache/spark/pull/9532#issuecomment-175927249
Github is hiccuping ...
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user gliptak commented on the pull request:
https://github.com/apache/spark/pull/9710#issuecomment-174762219
@JoshRosen A number of the doctests do fail locally too. I'm also unsure
why only a single test result shows ...
---
If your project is set up for it, you can rep
Github user gliptak commented on the pull request:
https://github.com/apache/spark/pull/10909#issuecomment-174761272
@mengxr Does the test only fail under one of the Python versions?
---
If your project is set up for it, you can reply to this email and have your
reply appear on
Github user gliptak commented on the pull request:
https://github.com/apache/spark/pull/9710#issuecomment-174380026
@JoshRosen Please review accumulators.py
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project
Github user gliptak commented on the pull request:
https://github.com/apache/spark/pull/9532#issuecomment-174032690
Two failed tests
org.apache.spark.sql.hive.ParquetMetastoreSuite.Caching converted data
source Parquet Relations0.25 sec5
Github user gliptak commented on the pull request:
https://github.com/apache/spark/pull/9532#issuecomment-173770249
@srowen This built locally.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not
Github user gliptak commented on the pull request:
https://github.com/apache/spark/pull/9263#issuecomment-173336741
@mengxr Thank you (I didn't get to this last night)
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as wel
Github user gliptak commented on the pull request:
https://github.com/apache/spark/pull/9263#issuecomment-173015660
@JoshRosen I see. Thanks
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user gliptak commented on the pull request:
https://github.com/apache/spark/pull/9263#issuecomment-173004633
@mengxr I will keep that in mind. Do you usually squash commits before
committing to master? Thanks
---
If your project is set up for it, you can reply to this email
Github user gliptak commented on the pull request:
https://github.com/apache/spark/pull/9710#issuecomment-172242468
```
This patch adds the following public classes (experimental):
>>> class VectorAccumulatorParam(AccumulatorParam):
```
What is the re
Github user gliptak commented on the pull request:
https://github.com/apache/spark/pull/9710#issuecomment-172240675
@mengxr Locally I see a number of these doctests fail. Should failing ones
be commented and JIRAs opened against them? Thanks
---
If your project is set up for it, you
Github user gliptak commented on the pull request:
https://github.com/apache/spark/pull/9532#issuecomment-172238584
Would you like me to update this patch with 1.7?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user gliptak commented on a diff in the pull request:
https://github.com/apache/spark/pull/9263#discussion_r49932647
--- Diff: python/pyspark/mllib/tests.py ---
@@ -75,22 +75,23 @@
# No SciPy, but that's okay, we'll skip those tests
pass
Github user gliptak commented on the pull request:
https://github.com/apache/spark/pull/9263#issuecomment-171818494
@JoshRosen Would you like to see some other changes?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If
Github user gliptak commented on a diff in the pull request:
https://github.com/apache/spark/pull/9263#discussion_r49803019
--- Diff: python/pyspark/mllib/tests.py ---
@@ -75,22 +75,23 @@
# No SciPy, but that's okay, we'll skip those tests
pass
Github user gliptak commented on the pull request:
https://github.com/apache/spark/pull/9263#issuecomment-171512409
```
FAIL: test_trainOn_predictOn (pyspark.mllib.tests.StreamingKMeansTest)
Test that prediction happens on the updated model
Github user gliptak commented on the pull request:
https://github.com/apache/spark/pull/9710#issuecomment-171473452
I'm only foreseeing a library version bump.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If
Github user gliptak commented on the pull request:
https://github.com/apache/spark/pull/9263#issuecomment-171473008
Yes, the multiple reports might play into it. I have no visibility into
Jenkins to assess further.
---
If your project is set up for it, you can reply to this email
Github user gliptak commented on a diff in the pull request:
https://github.com/apache/spark/pull/9263#discussion_r49668651
--- Diff: python/pyspark/mllib/tests.py ---
@@ -75,22 +75,23 @@
# No SciPy, but that's okay, we'll skip those tests
pass
Github user gliptak commented on the pull request:
https://github.com/apache/spark/pull/9710#issuecomment-171455887
https://github.com/xmlrunner/unittest-xml-reporting/pull/100 was committed
but not yet released. The version 1.12.0 installed with
https://issues.apache.org/jira/browse
Github user gliptak commented on the pull request:
https://github.com/apache/spark/pull/9263#issuecomment-171454395
@JoshRosen Please let me know if you would like to see other description
changes.
---
If your project is set up for it, you can reply to this email and have your
reply
Github user gliptak commented on the pull request:
https://github.com/apache/spark/pull/9710#issuecomment-166046164
@JoshRosen Please review.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user gliptak commented on the pull request:
https://github.com/apache/spark/pull/9263#issuecomment-166042934
@zsxwing Are there some changes you would like to see to this pull request?
---
If your project is set up for it, you can reply to this email and have your
reply appear
Github user gliptak commented on the pull request:
https://github.com/apache/spark/pull/9263#issuecomment-164482952
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/47554/testReport/
shows 0 failures ...
---
If your project is set up for it, you can reply to this
Github user gliptak commented on the pull request:
https://github.com/apache/spark/pull/9710#issuecomment-157222548
@davies @mengxr @JoshRosen Please review. Thanks
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user gliptak commented on the pull request:
https://github.com/apache/spark/pull/9710#issuecomment-156887827
Working to improve tooling at
https://github.com/xmlrunner/unittest-xml-reporting/pull/100
---
If your project is set up for it, you can reply to this email and have
Github user gliptak commented on the pull request:
https://github.com/apache/spark/pull/9263#issuecomment-156748411
Unit test timed out?
```
==
ERROR: test_kafka_direct_stream
Github user gliptak commented on the pull request:
https://github.com/apache/spark/pull/9263#issuecomment-156742485
Rebased to current master
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user gliptak commented on the pull request:
https://github.com/apache/spark/pull/9710#issuecomment-156600819
https://github.com/apache/spark/pull/8323
WIP
Please review. Please note that results end up as
```target/test-reports/TEST-doctest.DocTestCase*.xml```
---
If
GitHub user gliptak opened a pull request:
https://github.com/apache/spark/pull/9710
SPARK-11741 Process doctests using TextTestRunner/XMLTestRunner
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/gliptak/spark SPARK-11741
Github user gliptak commented on the pull request:
https://github.com/apache/spark/pull/9550#issuecomment-156597058
@srowen Please review. Thanks
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not
Github user gliptak commented on the pull request:
https://github.com/apache/spark/pull/9532#issuecomment-156504764
@srowen I'm working this using ```mvn``` I will remove one of the excludes
to bring the dependency back (without direct reference)
---
If your project is set up f
Github user gliptak commented on the pull request:
https://github.com/apache/spark/pull/9532#issuecomment-156487704
@srowen Can you offer some pointers on how to move this forward?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub
Github user gliptak commented on the pull request:
https://github.com/apache/spark/pull/9263#issuecomment-156209306
https://github.com/apache/spark/pull/9669
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user gliptak commented on the pull request:
https://github.com/apache/spark/pull/9532#issuecomment-155643110
https://github.com/apache/spark/pull/3864
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user gliptak commented on a diff in the pull request:
https://github.com/apache/spark/pull/9532#discussion_r44494766
--- Diff: core/pom.xml ---
@@ -78,11 +84,23 @@
net.java.dev.jets3t
jets3t
+
+
+ commons
Github user gliptak commented on the pull request:
https://github.com/apache/spark/pull/9263#issuecomment-155273055
Are JUnit test results separated based on Java version running under?
---
If your project is set up for it, you can reply to this email and have your
reply appear on
Github user gliptak commented on the pull request:
https://github.com/apache/spark/pull/9532#issuecomment-155067478
@srowen So you are seeing this as a side effect if the Maven dependency
changes ... I will work this some more (locally).
---
If your project is set up for it, you can
Github user gliptak commented on a diff in the pull request:
https://github.com/apache/spark/pull/9532#discussion_r44275653
--- Diff: core/pom.xml ---
@@ -78,11 +84,23 @@
net.java.dev.jets3t
jets3t
+
+
+ commons
Github user gliptak commented on the pull request:
https://github.com/apache/spark/pull/9532#issuecomment-154905851
Number of errors related to signatures:
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/45323/testReport/org.apache.spark/JavaAPISuite/reduce
Github user gliptak commented on the pull request:
https://github.com/apache/spark/pull/9263#issuecomment-154899177
Failure for only one of the Pythons version?
```
FAIL: test_trainOn_predictOn (pyspark.mllib.tests.StreamingKMeansTest)
Test that prediction happens on
GitHub user gliptak opened a pull request:
https://github.com/apache/spark/pull/9550
SPARK-11573 Correct 'reflective access of structural type member methâ¦
â¦od should be enabled' Scala warnings
You can merge this pull request into a Git repository by running:
Github user gliptak commented on the pull request:
https://github.com/apache/spark/pull/9532#issuecomment-154853134
@srowen Which maven pom.xml this corresponds to (streaming?)?
[error] (streaming-kinesis-asl-assembly/*:update)
java.lang.IllegalStateException: impossible
Github user gliptak commented on the pull request:
https://github.com/apache/spark/pull/9263#issuecomment-154842219
@mengxr Could you trigger a build?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does
Github user gliptak commented on the pull request:
https://github.com/apache/spark/pull/9532#issuecomment-154770983
@srowen I'm running ```build/sbt dependency-tree``` and it is literally
running for several hours now. Is there an easier way to work this (or is there
somethi
Github user gliptak commented on the pull request:
https://github.com/apache/spark/pull/9532#issuecomment-154712261
@srowen From SparkQA: "This patch adds the following public classes
(experimental):" I do not see how my change adds a public class ...
As for the compile
Github user gliptak commented on the pull request:
https://github.com/apache/spark/pull/9263#issuecomment-154608217
@mengxr Thank you for the pointer. This worked locally with
$ ./run-tests --python-executables=python --modules=pyspark-mllib
---
If your project is set up
Github user gliptak commented on the pull request:
https://github.com/apache/spark/pull/9532#issuecomment-154594465
The error message displayed seems to have no relation to the pull request
change ...
The compile does fail with:
[error]
/home/jenkins/workspace
GitHub user gliptak opened a pull request:
https://github.com/apache/spark/pull/9532
SPARK-11565 Replace deprecated DigestUtils.shaHex call
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/gliptak/spark SPARK-11565
Alternatively
Github user gliptak commented on the pull request:
https://github.com/apache/spark/pull/9378#issuecomment-154513822
@yinxusen
Now I'm confused myself. Current master already have
mllib-frequent-pattern-mining.md corrected:
https://raw.githubusercontent.com/a
Github user gliptak commented on the pull request:
https://github.com/apache/spark/pull/9378#issuecomment-154389335
https://issues.apache.org/jira/browse/SPARK-11445 is already being worked
at https://github.com/apache/spark/pull/9407
---
If your project is set up for it, you can
Github user gliptak commented on the pull request:
https://github.com/apache/spark/pull/9378#issuecomment-153921216
This became a duplicate of https://github.com/apache/spark/pull/9340
commit 820064e613609bbf7edd726d982da1de60bf417a
Author: Pravin Gadakh
Date: Wed
Github user gliptak closed the pull request at:
https://github.com/apache/spark/pull/9378
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is
Github user gliptak commented on the pull request:
https://github.com/apache/spark/pull/9263#issuecomment-153384421
@JoshRosen Would you have some pointers on how to move this forward? Thanks
---
If your project is set up for it, you can reply to this email and have your
reply appear
Github user gliptak commented on the pull request:
https://github.com/apache/spark/pull/9378#issuecomment-152887869
Created https://issues.apache.org/jira/browse/SPARK-11445 and updated
description of https://issues.apache.org/jira/browse/SPARK-11382
---
If your project is set up
Github user gliptak commented on the pull request:
https://github.com/apache/spark/pull/9378#issuecomment-152886733
Can I add new JIRA covering mllib-ensembles.md?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user gliptak commented on the pull request:
https://github.com/apache/spark/pull/9378#issuecomment-152878051
I was planning to split into two commits (although I'm open to work the
other file in this same commit). Thanks
---
If your project is set up for it, you can rep
Github user gliptak commented on the pull request:
https://github.com/apache/spark/pull/9378#issuecomment-152680244
@mengxr Could you trigger a build (including site?)? Thanks
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as
GitHub user gliptak opened a pull request:
https://github.com/apache/spark/pull/9378
SPARK-11382 Replace example code in mllib-decision-tree.md using inclâ¦
â¦ude_example
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/gliptak
Github user gliptak commented on a diff in the pull request:
https://github.com/apache/spark/pull/9263#discussion_r43250312
--- Diff: python/pyspark/mllib/tests.py ---
@@ -76,7 +76,8 @@
pass
ser = PickleSerializer()
-sc = SparkContext('local[4]
Github user gliptak commented on the pull request:
https://github.com/apache/spark/pull/9263#issuecomment-151833067
This last run had a different failure than the previous run with the same
code ...
---
If your project is set up for it, you can reply to this email and have
Github user gliptak commented on a diff in the pull request:
https://github.com/apache/spark/pull/9263#discussion_r43125731
--- Diff: python/pyspark/mllib/tests.py ---
@@ -76,7 +76,8 @@
pass
ser = PickleSerializer()
-sc = SparkContext('local[4]
Github user gliptak commented on the pull request:
https://github.com/apache/spark/pull/9263#issuecomment-150880141
http://stackoverflow.com/questions/28259756/how-to-create-multiple-sparkcontexts-in-a-console
https://github.com/apache/spark/pull/3121
---
If your project is set
Github user gliptak commented on the pull request:
https://github.com/apache/spark/pull/9263#issuecomment-150855946
The test errors with:
```
Finished test(python2.6): pyspark.mllib.util (9s)
Traceback (most recent call last):
File "/usr/lib64/python2.6/run
Github user gliptak commented on the pull request:
https://github.com/apache/spark/pull/9263#issuecomment-150852107
pyspark.tests shows up
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44300/testReport/
---
If your project is set up for it, you can
Github user gliptak commented on the pull request:
https://github.com/apache/spark/pull/8323#issuecomment-150832851
Adding packages to XMLs at https://github.com/apache/spark/pull/9263
---
If your project is set up for it, you can reply to this email and have your
reply appear on
GitHub user gliptak opened a pull request:
https://github.com/apache/spark/pull/9263
SPARK-11295 Add packages to JUnit output for Python tests
WIP
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/gliptak/spark SPARK-11295
Github user gliptak commented on the pull request:
https://github.com/apache/spark/pull/8323#issuecomment-150386525
As per https://github.com/xmlrunner/unittest-xml-reporting/issues/95 the
Python tests do not have packages (or possibly not being recognized by
https://github.com
Github user gliptak commented on the pull request:
https://github.com/apache/spark/pull/8323#issuecomment-148865532
@JoshRosen I opened
https://github.com/xmlrunner/unittest-xml-reporting/issues/94 to discuss
doctests processing.
---
If your project is set up for it, you can reply
Github user gliptak commented on the pull request:
https://github.com/apache/spark/pull/8323#issuecomment-148865272
@davies Not that I see. I opened
https://github.com/xmlrunner/unittest-xml-reporting/issues/95 to discuss.
---
If your project is set up for it, you can reply to this
Github user gliptak commented on the pull request:
https://github.com/apache/spark/pull/8323#issuecomment-147903176
What is the next step on this pull request? Thanks
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If
Github user gliptak commented on the pull request:
https://github.com/apache/spark/pull/8323#issuecomment-146384917
@JoshRosen How are the doctest-s triggered on the commandline and in
Jenkins?
```
$ python/run-tests --help
No handlers could be found for logger "
Github user gliptak commented on the pull request:
https://github.com/apache/spark/pull/8323#issuecomment-146369057
@JoshRosen I do not have python2.6 installed (or available as a package)
locally
---
If your project is set up for it, you can reply to this email and have your
reply
Github user gliptak commented on the pull request:
https://github.com/apache/spark/pull/8323#issuecomment-144553667
@mengxr Thank you for guiding this
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does
Github user gliptak commented on the pull request:
https://github.com/apache/spark/pull/8323#issuecomment-144219524
@mengxr I pushed a commit with the updated location.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If
Github user gliptak commented on the pull request:
https://github.com/apache/spark/pull/8323#issuecomment-143914542
https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/1810/testReport/
doesn't seem to list the Python tests.
Can somebody with a
Github user gliptak commented on the pull request:
https://github.com/apache/spark/pull/8323#issuecomment-143087075
@mengxr Revised.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this
Github user gliptak commented on the pull request:
https://github.com/apache/spark/pull/8323#issuecomment-143051862
@mengxr Thanks for the local lint pointer
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user gliptak commented on the pull request:
https://github.com/apache/spark/pull/8323#issuecomment-142750808
I tried this locally and the files were generated. Somebody with visibility
on Jenkins might need to comment on file locations ...
---
If your project is set up for it
Github user gliptak commented on the pull request:
https://github.com/apache/spark/pull/8323#issuecomment-142464604
@mengxr Would installing on Jenkins be independent from the codebase (the
`import xmlrunner` just be available)? Thanks
---
If your project is set up for it, you can
Github user gliptak commented on the pull request:
https://github.com/apache/spark/pull/8323#issuecomment-133009165
Is this a no-go because of the license? Thanks
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user gliptak commented on the pull request:
https://github.com/apache/spark/pull/8323#issuecomment-132827862
I just added the headers, xmlrunner is LGPL (and only used to run the
tests).
---
If your project is set up for it, you can reply to this email and have your
reply
Github user gliptak commented on the pull request:
https://github.com/apache/spark/pull/8323#issuecomment-132825406
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/41290/console
Recording test results
ERROR: Publisher 'Publish JUnit test result r
GitHub user gliptak opened a pull request:
https://github.com/apache/spark/pull/8323
SPARK-7021 Add JUnit output for Python tests
WIP
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/gliptak/spark SPARK-7021
Alternatively you can
98 matches
Mail list logo
