[
https://issues.apache.org/jira/browse/GEODE-194?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16037209#comment-16037209
]
ASF GitHub Bot commented on GEODE-194:
--------------------------------------
GitHub user metatype opened a pull request:
https://github.com/apache/geode/pull/558
GEODE-194: Remove spark connector
Remove the spark connector code until it can be updated
for the current spark release. We should also integrate
the build lifecycle and consider how to extract this into
a separate reo.
Thank you for submitting a contribution to Apache Geode.
In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:
### For all changes:
- [X] Is there a JIRA ticket associated with this PR? Is it referenced in
the commit message?
- [X] Has your PR been rebased against the latest commit within the target
branch (typically `develop`)?
- [X] Is your initial contribution a single, squashed commit?
- [X] Does `gradlew build` run cleanly?
- [ ] Have you written or updated unit tests to verify your changes?
- [ ] If adding new dependencies to the code, are these dependencies
licensed in a way that is compatible for inclusion under [ASF
2.0](http://www.apache.org/legal/resolved.html#category-a)?
### Note:
Please ensure that once the PR is submitted, you check travis-ci for build
issues and
submit an update to your PR as soon as possible. If you need help, please
send an
email to [email protected].
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/metatype/incubator-geode remove-spark
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/geode/pull/558.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #558
----
commit d95993b944d706baeb0c5e6c13a7d00754159123
Author: Anthony Baker <[email protected]>
Date: 2017-05-06T00:00:02Z
GEODE-194: Remove spark connector
Remove the spark connector code until it can be updated
for the current spark release. We should also integrate
the build lifecycle and consider how to extract this into
a separate reo.
----
> Geode Spark Connector does not support Spark 2.0
> ------------------------------------------------
>
> Key: GEODE-194
> URL: https://issues.apache.org/jira/browse/GEODE-194
> Project: Geode
> Issue Type: Bug
> Components: extensions
> Reporter: Jianxia Chen
> Labels: experimental, gsoc2016
>
> The BasicIntegrationTest fails when using spark 1.4. e.g.
> [info] - GemFire OQL query with more complex UDT: Partitioned Region ***
> FAILED ***
> [info] org.apache.spark.SparkException: Job aborted due to stage failure:
> Task 0 in stage 24.0 failed 1 times, most recent failure: Lost task 0.0 in
> stage 24.0 (TID 48, localhost): scala.MatchError:
> [info] Portfolio [id=3 status=active type=type3
> [info] AOL:Position [secId=AOL qty=978.0 mktValue=40.373],
> [info] MSFT:Position [secId=MSFT qty=98327.0 mktValue=23.32]]
> (of class ittest.io.pivotal.gemfire.spark.connector.Portfolio)
> [info] at
> org.apache.spark.sql.catalyst.CatalystTypeConverters$$anonfun$createToCatalystConverter$4.apply(CatalystTypeConverters.scala:178)
> [info] at
> org.apache.spark.sql.execution.RDDConversions$$anonfun$rowToRowRdd$1$$anonfun$apply$2.apply(ExistingRDD.scala:62)
> [info] at
> org.apache.spark.sql.execution.RDDConversions$$anonfun$rowToRowRdd$1$$anonfun$apply$2.apply(ExistingRDD.scala:59)
> [info] at scala.collection.Iterator$$anon$11.next(Iterator.scala:328)
> [info] at scala.collection.Iterator$$anon$11.next(Iterator.scala:328)
> [info] at scala.collection.Iterator$class.foreach(Iterator.scala:727)
> [info] at
> scala.collection.AbstractIterator.foreach(Iterator.scala:1157)
> [info] at
> scala.collection.generic.Growable$class.$plus$plus$eq(Growable.scala:48)
> [info] at
> scala.collection.mutable.ArrayBuffer.$plus$plus$eq(ArrayBuffer.scala:103)
> [info] at
> scala.collection.mutable.ArrayBuffer.$plus$plus$eq(ArrayBuffer.scala:47)
> [info] at
> scala.collection.TraversableOnce$class.to(TraversableOnce.scala:273)
> [info] at scala.collection.AbstractIterator.to(Iterator.scala:1157)
> [info] at
> scala.collection.TraversableOnce$class.toBuffer(TraversableOnce.scala:265)
> [info] at
> scala.collection.AbstractIterator.toBuffer(Iterator.scala:1157)
> [info] at
> scala.collection.TraversableOnce$class.toArray(TraversableOnce.scala:252)
> [info] at
> scala.collection.AbstractIterator.toArray(Iterator.scala:1157)
> [info] at
> org.apache.spark.rdd.RDD$$anonfun$collect$1$$anonfun$12.apply(RDD.scala:885)
> [info] at
> org.apache.spark.rdd.RDD$$anonfun$collect$1$$anonfun$12.apply(RDD.scala:885)
> [info] at
> org.apache.spark.SparkContext$$anonfun$runJob$5.apply(SparkContext.scala:1765)
> [info] at
> org.apache.spark.SparkContext$$anonfun$runJob$5.apply(SparkContext.scala:1765)
> [info] at
> org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:63)
> [info] at org.apache.spark.scheduler.Task.run(Task.scala:70)
> [info] at
> org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:213)
> [info] at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> [info] at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> [info] at java.lang.Thread.run(Thread.java:745)
> [info]
> [info] Driver stacktrace:
> [info] at
> org.apache.spark.scheduler.DAGScheduler.org$apache$spark$scheduler$DAGScheduler$$failJobAndIndependentStages(DAGScheduler.scala:1266)
> [info] at
> org.apache.spark.scheduler.DAGScheduler$$anonfun$abortStage$1.apply(DAGScheduler.scala:1257)
> [info] at
> org.apache.spark.scheduler.DAGScheduler$$anonfun$abortStage$1.apply(DAGScheduler.scala:1256)
> [info] at
> scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59)
> [info] at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:47)
> [info] at
> org.apache.spark.scheduler.DAGScheduler.abortStage(DAGScheduler.scala:1256)
> [info] at
> org.apache.spark.scheduler.DAGScheduler$$anonfun$handleTaskSetFailed$1.apply(DAGScheduler.scala:730)
> [info] at
> org.apache.spark.scheduler.DAGScheduler$$anonfun$handleTaskSetFailed$1.apply(DAGScheduler.scala:730)
> [info] at scala.Option.foreach(Option.scala:236)
> [info] at
> org.apache.spark.scheduler.DAGScheduler.handleTaskSetFailed(DAGScheduler.scala:730)
> [info] ...
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
