Github user lresende commented on the issue:
https://github.com/apache/spark/pull/22867
@gss2002 Sorry I missed this initially, but great that @vanzin is helping
you with the fix.
---
-
To unsubscribe, e-mail
Github user lresende commented on the issue:
https://github.com/apache/spark/pull/15594
@vanzin I believe this might be your realm :) Could you please help review
this.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well
Github user lresende commented on a diff in the pull request:
https://github.com/apache/spark/pull/15594#discussion_r88752705
--- Diff:
sql/hive-thriftserver/src/main/scala/org/apache/spark/sql/hive/thriftserver/SparkSQLCLIService.scala
---
@@ -57,7 +59,24 @@ private[hive] class
Github user lresende commented on the issue:
https://github.com/apache/spark/pull/14601
@agsachin Are you planning to address these updates on this PR ? It would
be good to have this as part of Spark as it affects multiple usage scenarios in
cloud platforms and other cases as well
Github user lresende commented on the issue:
https://github.com/apache/spark/pull/15170
Mostly style related changes
LGTM
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user lresende closed the pull request at:
https://github.com/apache/spark/pull/14981
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature
Github user lresende commented on the issue:
https://github.com/apache/spark/pull/15114
I verified this works on native docker in linux with :
build/mvn -Pdocker-integration-tests -Pscala-2.11 -pl
:spark-docker-integration-tests_2.11 clean compile test
LGTM
Github user lresende commented on the issue:
https://github.com/apache/spark/pull/14981
@srowen Please don't get me wrong, I don't have any interest on this
extension either, but just want to make sure we start doing the right thing for
Apache Spark. I will try to ping some
Github user lresende commented on the issue:
https://github.com/apache/spark/pull/14981
The pointer is exactly your quote on the e-mail to legal-discuss:
http://www.apache.org/legal/resolved.html#prohibited says:
-
CAN APACHE PROJECTS RELY ON COMPONENTS UNDER
Github user lresende commented on the issue:
https://github.com/apache/spark/pull/14981
Yes, and this is the intent. It's ok to have these in the source release
(similar to ganglia) but we don't publish them in maven repository and it
becomes available only if people goes
Github user lresende commented on the issue:
https://github.com/apache/spark/pull/14981
@srowen @rxin My understanding is that the mvn deploy is what takes care of
actually publishing the files to maven staging repository :
`
$MVN -DzincPort=$ZINC_PORT --settings
Github user lresende commented on the issue:
https://github.com/apache/spark/pull/14981
Ok, reverting the commit to remove kinesis assembly as the python tests are
relying on it for the transient dependencies. Note that I was also trying to
overcome this requirement by appending all
Github user lresende commented on the issue:
https://github.com/apache/spark/pull/14981
@srowen should I update this PR with the removal of kinesis assembly then ?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user lresende commented on the issue:
https://github.com/apache/spark/pull/14981
I would still wait for the feedback from legal before removing anything.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user lresende commented on the issue:
https://github.com/apache/spark/pull/14981
Spark kinesis has dependency on the kinesis client which is category-x
com.amazonaws
amazon-kinesis-client
${aws.kinesis.client.version}
Thus
Github user lresende commented on the issue:
https://github.com/apache/spark/pull/14981
As for the the Ganglia one, I will create another jira, to track that
separately as this (Kinesis) one might involve more changes around the python
and examples.
---
If your project is set up
Github user lresende commented on the issue:
https://github.com/apache/spark/pull/14981
@srowen, The Kinesis assembly has been published by Spark releases for a
while. Here is the link to the 2.0 release on repository.apache.org :
https://repository.apache.org/service/local
GitHub user lresende opened a pull request:
https://github.com/apache/spark/pull/14981
[SPARK-17418] Remove Kinesis artifacts from Spark release scripts
## What changes were proposed in this pull request?
This PR removes Kinesis from the release scripts as Kinesis license
Github user lresende commented on the issue:
https://github.com/apache/spark/pull/14606
Couple of
[bugs](https://archive.apache.org/dist/kafka/0.10.0.1/RELEASE_NOTES.html)
seemed interesting to Spark, also good to maintain dependency currency similar
to what we have been doing
GitHub user lresende opened a pull request:
https://github.com/apache/spark/pull/14606
[SPARK-17023][BUILD] Upgrade to Kafka 0.10.0.1 release
## What changes were proposed in this pull request?
Update Kafka streaming connector to use Kafka 0.10.0.1 release
## How
Github user lresende commented on the issue:
https://github.com/apache/spark/pull/14606
Jenkins test this please
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled
Github user lresende commented on the issue:
https://github.com/apache/spark/pull/14601
Also, will this also fix the scenario where the user has provided the
properties programmatically ?
---
If your project is set up for it, you can reply to this email and have your
reply appear
Github user lresende commented on the issue:
https://github.com/apache/spark/pull/14601
Minor, the title should be [CORE] instead of [SPARK CORE]
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does
Github user lresende commented on a diff in the pull request:
https://github.com/apache/spark/pull/14601#discussion_r74441097
--- Diff: core/src/main/scala/org/apache/spark/deploy/SparkHadoopUtil.scala
---
@@ -107,6 +107,14 @@ class SparkHadoopUtil extends Logging
Github user lresende commented on a diff in the pull request:
https://github.com/apache/spark/pull/14601#discussion_r74440796
--- Diff: core/src/main/scala/org/apache/spark/deploy/SparkHadoopUtil.scala
---
@@ -107,6 +107,14 @@ class SparkHadoopUtil extends Logging
Github user lresende commented on the issue:
https://github.com/apache/spark/pull/14601
Jenkins test this please
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled
Github user lresende commented on the issue:
https://github.com/apache/spark/pull/14066
The issue here is that releases keep getting archived when new releases
comes up. For old releases (or by default) we could use
https://archive.apache.org/dist/maven/maven-3/, which is always
Github user lresende commented on the issue:
https://github.com/apache/spark/pull/9162
Jenkins retest this please
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled
Github user lresende commented on the pull request:
https://github.com/apache/spark/pull/9162#issuecomment-221358898
Other then the minor comment around BooleanType, it LGTM.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub
Github user lresende commented on a diff in the pull request:
https://github.com/apache/spark/pull/9162#discussion_r64446518
--- Diff: sql/core/src/test/scala/org/apache/spark/sql/jdbc/JDBCSuite.scala
---
@@ -579,6 +579,14 @@ class JDBCSuite extends SparkFunSuite
val
Github user lresende commented on the pull request:
https://github.com/apache/spark/pull/13229#issuecomment-220753618
LGTM, I also verified with a issue discussed on the mailing list where
writing to csv was taking about 1hr, and with this change it only took a few
minutes
Github user lresende closed the pull request at:
https://github.com/apache/spark/pull/13092
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature
Github user lresende commented on the pull request:
https://github.com/apache/spark/pull/13092#issuecomment-218951567
@srowen Please review.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
GitHub user lresende opened a pull request:
https://github.com/apache/spark/pull/13092
[SPARK-15309] Bump master to version 2.1.0-SNAPSHOT
## What changes were proposed in this pull request?
Update pom artifact version to 2.1.0-SNAPSHOT to avoid any conflicts with
2.0.0
Github user lresende commented on a diff in the pull request:
https://github.com/apache/spark/pull/12508#discussion_r62191641
--- Diff:
external/docker-integration-tests/src/test/scala/org/apache/spark/sql/jdbc/OracleIntegrationSuite.scala
---
@@ -46,12 +44,11 @@ import
Github user lresende commented on the pull request:
https://github.com/apache/spark/pull/12508#issuecomment-216679636
The SparkPullRequestBuilder Build basically calls ./dev/run-tests-jenkins
while the nightly snapshot build is running some scripts from @pwendell. So I
don't believe
Github user lresende commented on the pull request:
https://github.com/apache/spark/pull/12508#issuecomment-214993748
@vanzin @srowen I wouldn't recommend running when docker is installed, as
this would run on the Jenkins slaves, and @JoshRosen disable these because they
were failing
Github user lresende closed the pull request at:
https://github.com/apache/spark/pull/12705
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature
Github user lresende commented on the pull request:
https://github.com/apache/spark/pull/12705#issuecomment-214833015
@hvanhovell thanks for the comments, this is a wip and i should have a
better working code in a few hours.
---
If your project is set up for it, you can reply
GitHub user lresende opened a pull request:
https://github.com/apache/spark/pull/12705
[SPARK-12660][SQL][WIP] Rewrite except using anti-join
## What changes were proposed in this pull request?
(Please fill in changes proposed in this fix)
## How
Github user lresende commented on the pull request:
https://github.com/apache/spark/pull/12544#issuecomment-212732645
Is there any documentation that needs to be updated with the addition of
--packages or --jars to run the examples (e.g. running-on-yarn.md) ?
---
If your project
Github user lresende commented on the pull request:
https://github.com/apache/spark/pull/12552#issuecomment-212730550
LGTM
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user lresende commented on the pull request:
https://github.com/apache/spark/pull/12508#issuecomment-212722890
@JoshRosen @rxin ping
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user lresende commented on the pull request:
https://github.com/apache/spark/pull/12348#issuecomment-212722791
@JoshRosen ping
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user lresende commented on the pull request:
https://github.com/apache/spark/pull/12269#issuecomment-212198729
@davies @liancheng looks like after re basing to latest code, this issue
has been resolved. I am going to wait for a build to complete to double check.
---
If your
Github user lresende commented on the pull request:
https://github.com/apache/spark/pull/12508#issuecomment-212150753
@JoshRosen how would I go about updating pr build to run the integration
tests in Jenkins only ? Should we run the tests after the
./dev/run-tests-jenkins
GitHub user lresende opened a pull request:
https://github.com/apache/spark/pull/12508
[SPARK-14738][BUILD] Separate docker integration tests from main build
## What changes were proposed in this pull request?
Create a maven profile for executing the docker integration
Github user lresende commented on the pull request:
https://github.com/apache/spark/pull/12269#issuecomment-212137015
@davies @liancheng
So, when converting these tests, I noticed that the following :
test("uncorrelated scalar subquery on a DataFrame generated
Github user lresende commented on the pull request:
https://github.com/apache/spark/pull/12270#issuecomment-211732862
Ok, I will work with @JoshRosen on the trigger part.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well
Github user lresende commented on the pull request:
https://github.com/apache/spark/pull/12270#issuecomment-211731480
@rxin Let me move them to a specific docker profile. But I would still run
them on Jenkins, as the infrastructure is already setup there.
---
If your project is set
Github user lresende commented on the pull request:
https://github.com/apache/spark/pull/12270#issuecomment-211729901
@rxin, just trying to understand, is the oracle test the only one failing ?
Or you are suggesting we move the whole docker based tests to a separate
profile
Github user lresende commented on the pull request:
https://github.com/apache/spark/pull/12348#issuecomment-211705986
@JoshRosen All good now.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user lresende commented on a diff in the pull request:
https://github.com/apache/spark/pull/12270#discussion_r60103432
--- Diff: project/SparkBuild.scala ---
@@ -364,12 +364,15 @@ object Flume {
}
object DockerIntegrationTests {
+ // Ignore checksum
Github user lresende commented on the pull request:
https://github.com/apache/spark/pull/12270#issuecomment-211477523
Jenkins, retest this please.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does
Github user lresende commented on the pull request:
https://github.com/apache/spark/pull/12270#issuecomment-211084045
Jenkins, retest this
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user lresende commented on the pull request:
https://github.com/apache/spark/pull/12348#issuecomment-209744652
Jenkins, retest this please.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does
Github user lresende commented on the pull request:
https://github.com/apache/spark/pull/12348#issuecomment-209597909
@JoshRosen please review, these are the merge issues I mentioned yesterday,
and these changes are passing my local tests as well.
---
If your project is set up
Github user lresende closed the pull request at:
https://github.com/apache/spark/pull/12349
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature
Github user lresende commented on the pull request:
https://github.com/apache/spark/pull/12349#issuecomment-209225537
ok, I will leave it open for a day in case anyone else has interest on the
change, otherwise close it.
---
If your project is set up for it, you can reply
Github user lresende commented on the pull request:
https://github.com/apache/spark/pull/12349#issuecomment-209221983
@rxin It is, but you then have to search for the jira. This makes it much
more easy to just access the jira, from the link, similar to how the jira has
the link
GitHub user lresende opened a pull request:
https://github.com/apache/spark/pull/12349
[SPARK-14590] Update pull request template with JIRA link
## What changes were proposed in this pull request?
Update pull request template to have direct link to jira issue
You can
GitHub user lresende opened a pull request:
https://github.com/apache/spark/pull/12348
[SPARK-14589][SQL] Enhance DB2 JDBC Dialect docker tests
## What changes were proposed in this pull request?
Enhance the DB2 JDBC Dialect docker tests as they seemed to have had some
Github user lresende commented on a diff in the pull request:
https://github.com/apache/spark/pull/12265#discussion_r59119361
--- Diff:
mllib/src/main/scala/org/apache/spark/ml/feature/CountVectorizer.scala ---
@@ -152,17 +152,17 @@ class CountVectorizer(override val uid: String
Github user lresende commented on the pull request:
https://github.com/apache/spark/pull/12265#issuecomment-207827549
Jenkins retest this please
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does
Github user lresende commented on the pull request:
https://github.com/apache/spark/pull/12270#issuecomment-207706296
This PR is dependent on PR #9893
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does
GitHub user lresende opened a pull request:
https://github.com/apache/spark/pull/12270
[SPARK-14504] Enable Oracle docker tests
## What changes were proposed in this pull request?
Enable Oracle docker tests
## How was this patch tested?
Existing tests
You
Github user lresende commented on the pull request:
https://github.com/apache/spark/pull/9893#issuecomment-207697108
@JoshRosen All good, style regression fixed, all tests passing.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub
GitHub user lresende opened a pull request:
https://github.com/apache/spark/pull/12269
[SPARK-13419] Update SubquerySuite to use checkAnswer to validate resâ¦
## What changes were proposed in this pull request?
Change SubquerySuite to validate test results utilizing
Github user lresende commented on the pull request:
https://github.com/apache/spark/pull/9893#issuecomment-207676407
@JoshRosen, thanks for reviewing, I agree with your proposal regarding the
repository. Thanks.
---
If your project is set up for it, you can reply to this email
Github user lresende commented on the pull request:
https://github.com/apache/spark/pull/9893#issuecomment-207675856
Jenkins, retest this please.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does
Github user lresende commented on the pull request:
https://github.com/apache/spark/pull/11623#issuecomment-206479123
@cloud-fan Tested the latest commit in both 1.6 branch and trunk... fails
in 1.6 and works fine in trunk.
---
If your project is set up for it, you can reply
Github user lresende commented on a diff in the pull request:
https://github.com/apache/spark/pull/11623#discussion_r58633691
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/DatasetAggregatorSuite.scala ---
@@ -176,4 +186,13 @@ class DatasetAggregatorSuite extends QueryTest
Github user lresende commented on a diff in the pull request:
https://github.com/apache/spark/pull/11623#discussion_r58505416
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/DatasetAggregatorSuite.scala ---
@@ -176,4 +186,13 @@ class DatasetAggregatorSuite extends QueryTest
Github user lresende commented on a diff in the pull request:
https://github.com/apache/spark/pull/11623#discussion_r58487110
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/DatasetAggregatorSuite.scala ---
@@ -84,6 +84,16 @@ object ComplexBufferAgg extends Aggregator
Github user lresende commented on the pull request:
https://github.com/apache/spark/pull/11623#issuecomment-205629190
Jenkins retest this please
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does
Github user lresende commented on a diff in the pull request:
https://github.com/apache/spark/pull/11623#discussion_r58480607
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/DatasetAggregatorSuite.scala ---
@@ -176,4 +186,13 @@ class DatasetAggregatorSuite extends QueryTest
Github user lresende commented on a diff in the pull request:
https://github.com/apache/spark/pull/11623#discussion_r58480578
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/DatasetAggregatorSuite.scala ---
@@ -84,6 +84,16 @@ object ComplexBufferAgg extends Aggregator
Github user lresende commented on the pull request:
https://github.com/apache/spark/pull/11623#issuecomment-205420648
@cloud-fan could you please review
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project
Github user lresende commented on the pull request:
https://github.com/apache/spark/pull/9893#issuecomment-205413804
@JoshRosen could you please review this.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
GitHub user lresende opened a pull request:
https://github.com/apache/spark/pull/12151
[SPARK-14366] Remove sbt-idea plugin
## What changes were proposed in this pull request?
Remove sbt-idea plugin as importing sbt project provides much better
support.
You can merge
Github user lresende closed the pull request at:
https://github.com/apache/spark/pull/11542
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature
Github user lresende commented on the pull request:
https://github.com/apache/spark/pull/11542#issuecomment-203206216
@markhamstra The discussion seems to be getting consensus, so it's not that
I am jumping the gun and creating something completely new, but reopening
something
Github user lresende commented on the pull request:
https://github.com/apache/spark/pull/11542#issuecomment-203183327
@markhamstra I had done this is the past, and with the current discussion
of droping jdk 7 and scala 2.10 i think this might be useful now. Any concerns
reopening
Github user lresende commented on the pull request:
https://github.com/apache/spark/pull/11542#issuecomment-203179609
Jenkins retest this please
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does
GitHub user lresende reopened a pull request:
https://github.com/apache/spark/pull/11542
[SPARK-13703][BUILD] Remove obsolete scala-2.10 source files
## What changes were proposed in this pull request?
Remove obsolete scala-2.10 source files
## How was this patch
Github user lresende commented on the pull request:
https://github.com/apache/spark/pull/11623#issuecomment-200962644
Jenkins test this please
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user lresende commented on a diff in the pull request:
https://github.com/apache/spark/pull/11623#discussion_r57330036
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/DatasetAggregatorSuite.scala ---
@@ -206,4 +216,12 @@ class DatasetAggregatorSuite extends QueryTest
Github user lresende commented on a diff in the pull request:
https://github.com/apache/spark/pull/11623#discussion_r57098172
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/DatasetAggregatorSuite.scala ---
@@ -206,4 +216,12 @@ class DatasetAggregatorSuite extends QueryTest
Github user lresende commented on the pull request:
https://github.com/apache/spark/pull/11623#issuecomment-198151304
@rxin Please review this and merge/close the JIRA as I have checked with
author and having the fix on 2.0 is ok (so no need to backport it).
---
If your project
Github user lresende commented on the pull request:
https://github.com/apache/spark/pull/11623#issuecomment-194984207
Jenkins test this please
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user lresende commented on the pull request:
https://github.com/apache/spark/pull/11623#issuecomment-194983724
@rxin Updated the PR with more details
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project
GitHub user lresende opened a pull request:
https://github.com/apache/spark/pull/11623
[SPARK-12555][SQL] Add provided test scenario
## What changes were proposed in this pull request?
This PR add test case described in this issue to avoid future regressions.
This test
Github user lresende closed the pull request at:
https://github.com/apache/spark/pull/11542
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature
Github user lresende commented on the pull request:
https://github.com/apache/spark/pull/11542#issuecomment-192938438
@srowen I was under the impression we were not supporting 2.10 anymore, if
this is not the case I will close this PR.
---
If your project is set up for it, you can
GitHub user lresende opened a pull request:
https://github.com/apache/spark/pull/11542
[SPARK-13703][BUILD] Remove obsolete scala-2.10 source files
## What changes were proposed in this pull request?
Remove obsolete scala-2.10 source files
## How was this patch
Github user lresende commented on the pull request:
https://github.com/apache/spark/pull/11139#issuecomment-186560354
Jenkins, test this please
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does
Github user lresende commented on the pull request:
https://github.com/apache/spark/pull/9893#issuecomment-185382474
Ok, looks like the issue is that, when trying to start the docker db2
image, we are getting an memory error: SQL1220N The database manager failed to
allocate shared
Github user lresende commented on a diff in the pull request:
https://github.com/apache/spark/pull/11139#discussion_r53208178
--- Diff:
examples/src/main/java/org/apache/spark/examples/streaming/JavaRecoverableNetworkWordCount.java
---
@@ -204,13 +202,17 @@ public static void
Github user lresende commented on the pull request:
https://github.com/apache/spark/pull/9893#issuecomment-185326608
@JoshRosen @shaneknapp I believe this is ready, but Jenkins seems to be
having timeout issues probably when connecting to the db2 docker image. Is
there a way for any
Github user lresende commented on the pull request:
https://github.com/apache/spark/pull/11139#issuecomment-184811230
Jenkins, test this please
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does
1 - 100 of 147 matches
Mail list logo
