Github user witgo commented on the pull request:
https://github.com/apache/spark/pull/1574#issuecomment-50024391
This is a good idea.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this
Github user witgo commented on a diff in the pull request:
https://github.com/apache/spark/pull/1180#discussion_r15352693
--- Diff:
yarn/common/src/main/scala/org/apache/spark/scheduler/cluster/YarnClientSchedulerBackend.scala
---
@@ -115,7 +117,33 @@ private[spark] class
Github user witgo commented on a diff in the pull request:
https://github.com/apache/spark/pull/1180#discussion_r15352756
--- Diff:
yarn/alpha/src/main/scala/org/apache/spark/deploy/yarn/ExecutorLauncher.scala
---
@@ -282,15 +298,22 @@ class ExecutorLauncher(args
Github user witgo commented on the pull request:
https://github.com/apache/spark/pull/1180#issuecomment-50044511
This situation exists in theory.
[SparkContext.scala#L992](https://github.com/apache/spark/blob/master/core/src/main/scala/org/apache/spark/SparkContext.scala#L992)
deal
Github user witgo commented on the pull request:
https://github.com/apache/spark/pull/1578#issuecomment-50232324
Here also should throw an `FetchFailedException `?
```scala
override def next(): (BlockId, Option[Iterator[Any]]) = {
resultsGotten += 1
val
GitHub user witgo opened a pull request:
https://github.com/apache/spark/pull/1603
ConnectionManager throws out of "Could not find reference for received ack
message xxx" exception.
You can merge this pull request into a Git repository by running:
$ git
Github user witgo commented on the pull request:
https://github.com/apache/spark/pull/1565#issuecomment-50237538
@ScrapCodes
I think that the solution is simple and effective, is a better.
---
If your project is set up for it, you can reply to this email and have your
reply
Github user witgo commented on the pull request:
https://github.com/apache/spark/pull/1603#issuecomment-50239511
Throw an exception here cause
`System.exit(ExecutorExitCode.UNCAUGHT_EXCEPTION)` is called.
This is not necessary.
---
If your project is set up for it, you can reply
Github user witgo commented on a diff in the pull request:
https://github.com/apache/spark/pull/1608#discussion_r15437166
--- Diff: external/hbase/pom.xml ---
@@ -0,0 +1,217 @@
+http://maven.apache.org/POM/4.0.0";
xmlns:xsi="http://www.w3.org/2001/XMLSchem
Github user witgo commented on a diff in the pull request:
https://github.com/apache/spark/pull/1608#discussion_r15437173
--- Diff: external/hbase/pom.xml ---
@@ -0,0 +1,217 @@
+http://maven.apache.org/POM/4.0.0";
xmlns:xsi="http://www.w3.org/2001/XMLSchem
Github user witgo commented on a diff in the pull request:
https://github.com/apache/spark/pull/1608#discussion_r15437185
--- Diff: external/hbase/pom.xml ---
@@ -0,0 +1,217 @@
+http://maven.apache.org/POM/4.0.0";
xmlns:xsi="http://www.w3.org/2001/XMLSchem
Github user witgo commented on the pull request:
https://github.com/apache/spark/pull/940#issuecomment-50293501
@mengxr Done
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user witgo commented on the pull request:
https://github.com/apache/spark/pull/940#issuecomment-50297230
Jenkins, retest this please.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user witgo commented on a diff in the pull request:
https://github.com/apache/spark/pull/1609#discussion_r15446910
--- Diff:
core/src/main/scala/org/apache/spark/storage/BlockObjectWriter.scala ---
@@ -107,68 +109,296 @@ private[spark] class DiskBlockObjectWriter
Github user witgo commented on a diff in the pull request:
https://github.com/apache/spark/pull/1609#discussion_r15446921
--- Diff:
core/src/main/scala/org/apache/spark/storage/BlockObjectWriter.scala ---
@@ -176,10 +406,17 @@ private[spark] class DiskBlockObjectWriter
Github user witgo commented on a diff in the pull request:
https://github.com/apache/spark/pull/1609#discussion_r15447384
--- Diff:
core/src/main/scala/org/apache/spark/storage/BlockObjectWriter.scala ---
@@ -107,68 +109,296 @@ private[spark] class DiskBlockObjectWriter
Github user witgo commented on a diff in the pull request:
https://github.com/apache/spark/pull/1609#discussion_r15447402
--- Diff:
core/src/main/scala/org/apache/spark/storage/BlockObjectWriter.scala ---
@@ -107,68 +109,296 @@ private[spark] class DiskBlockObjectWriter
Github user witgo commented on a diff in the pull request:
https://github.com/apache/spark/pull/1609#discussion_r15447397
--- Diff:
core/src/main/scala/org/apache/spark/storage/BlockObjectWriter.scala ---
@@ -107,68 +109,296 @@ private[spark] class DiskBlockObjectWriter
Github user witgo commented on a diff in the pull request:
https://github.com/apache/spark/pull/1609#discussion_r15447401
--- Diff:
core/src/main/scala/org/apache/spark/storage/BlockObjectWriter.scala ---
@@ -107,68 +109,296 @@ private[spark] class DiskBlockObjectWriter
Github user witgo commented on a diff in the pull request:
https://github.com/apache/spark/pull/1609#discussion_r15447436
--- Diff:
core/src/main/scala/org/apache/spark/storage/BlockObjectWriter.scala ---
@@ -107,68 +109,296 @@ private[spark] class DiskBlockObjectWriter
Github user witgo commented on a diff in the pull request:
https://github.com/apache/spark/pull/1609#discussion_r15447430
--- Diff:
core/src/main/scala/org/apache/spark/storage/BlockObjectWriter.scala ---
@@ -107,68 +109,296 @@ private[spark] class DiskBlockObjectWriter
Github user witgo commented on a diff in the pull request:
https://github.com/apache/spark/pull/1609#discussion_r15447449
--- Diff:
core/src/main/scala/org/apache/spark/storage/BlockObjectWriter.scala ---
@@ -107,68 +109,296 @@ private[spark] class DiskBlockObjectWriter
Github user witgo commented on a diff in the pull request:
https://github.com/apache/spark/pull/1609#discussion_r15447510
--- Diff: core/src/main/scala/org/apache/spark/util/Utils.scala ---
@@ -935,15 +941,22 @@ private[spark] object Utils extends Logging {
* Currently
Github user witgo commented on a diff in the pull request:
https://github.com/apache/spark/pull/1609#discussion_r15447541
--- Diff:
core/src/main/scala/org/apache/spark/util/collection/ExternalAppendOnlyMap.scala
---
@@ -353,26 +368,53 @@ class ExternalAppendOnlyMap[K, V, C
Github user witgo commented on a diff in the pull request:
https://github.com/apache/spark/pull/1609#discussion_r15447535
--- Diff:
core/src/main/scala/org/apache/spark/util/collection/ExternalAppendOnlyMap.scala
---
@@ -353,26 +368,53 @@ class ExternalAppendOnlyMap[K, V, C
GitHub user witgo opened a pull request:
https://github.com/apache/spark/pull/1619
[WIP][SPARK-2677]BasicBlockFetchIterator#next can wait forever
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/witgo/spark SPARK-2677
Github user witgo commented on a diff in the pull request:
https://github.com/apache/spark/pull/1632#discussion_r15508224
--- Diff:
core/src/main/scala/org/apache/spark/storage/BlockFetcherIterator.scala ---
@@ -117,31 +121,45 @@ object BlockFetcherIterator
Github user witgo commented on the pull request:
https://github.com/apache/spark/pull/1619#issuecomment-50456598
@sarutak I think add a heartbeat detection mechanism is a good solution
---
If your project is set up for it, you can reply to this email and have your
reply appear on
Github user witgo commented on the pull request:
https://github.com/apache/spark/pull/1619#issuecomment-50458991
@sarutak
[ConnectionManager.scala#L259](https://github.com/apache/spark/blob/master/core/src/main/scala/org/apache/spark/network/ConnectionManager.scala#L259)
to deal
Github user witgo commented on the pull request:
https://github.com/apache/spark/pull/929#issuecomment-50492147
Jenkins, test this please.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user witgo commented on the pull request:
https://github.com/apache/spark/pull/929#issuecomment-50503214
@mengxr Done. Tomorrow, I will test in detail.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user witgo commented on a diff in the pull request:
https://github.com/apache/spark/pull/1269#discussion_r15560968
--- Diff:
mllib/src/test/scala/org/apache/spark/mllib/clustering/topicmodeling/topicmodels/RobustPLSASuite.scala
---
@@ -0,0 +1,40 @@
+/*
+ * Licensed
Github user witgo commented on the pull request:
https://github.com/apache/spark/pull/929#issuecomment-50573353
@mengxr The test pass.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this
Github user witgo commented on a diff in the pull request:
https://github.com/apache/spark/pull/929#discussion_r15566577
--- Diff:
mllib/src/main/scala/org/apache/spark/mllib/recommendation/ALS.scala ---
@@ -255,6 +255,9 @@ class ALS private (
rank, lambda, alpha
Github user witgo commented on a diff in the pull request:
https://github.com/apache/spark/pull/929#discussion_r15566754
--- Diff:
mllib/src/main/scala/org/apache/spark/mllib/recommendation/ALS.scala ---
@@ -255,6 +255,9 @@ class ALS private (
rank, lambda, alpha
Github user witgo commented on the pull request:
https://github.com/apache/spark/pull/929#issuecomment-50574926
Ok, I will try it.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this
Github user witgo commented on the pull request:
https://github.com/apache/spark/pull/1256#issuecomment-50586734
Jenkins, retest this please.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user witgo commented on the pull request:
https://github.com/apache/spark/pull/1678#issuecomment-50706018
We can also merge it to 1.0 branch.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does
GitHub user witgo opened a pull request:
https://github.com/apache/spark/pull/1683
SPARK-2766: ScalaReflectionSuite throw an llegalArgumentException in JDK 6
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/witgo/spark SPARK
Github user witgo commented on the pull request:
https://github.com/apache/spark/pull/1256#issuecomment-50718557
Jenkins, retest this please.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user witgo commented on the pull request:
https://github.com/apache/spark/pull/1683#issuecomment-50841416
Jenkins, test this please.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user witgo commented on the pull request:
https://github.com/apache/spark/pull/1369#issuecomment-50851936
Jenkins, retest this please.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user witgo commented on the pull request:
https://github.com/apache/spark/pull/940#issuecomment-50855187
@mengxr done
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user witgo commented on a diff in the pull request:
https://github.com/apache/spark/pull/940#discussion_r15684437
--- Diff: mllib/pom.xml ---
@@ -60,6 +60,14 @@
junit
junit
+
Github user witgo commented on the pull request:
https://github.com/apache/spark/pull/940#issuecomment-50860276
Jenkins, retest this please.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user witgo commented on the pull request:
https://github.com/apache/spark/pull/940#issuecomment-50906124
How do we resolve this issue?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user witgo commented on the pull request:
https://github.com/apache/spark/pull/1369#issuecomment-50910604
@srowen Thank you for your comments. And how to modify `GenerateMIMAIgnore`
file to exclude a method? An example?
---
If your project is set up for it, you can reply to
GitHub user witgo opened a pull request:
https://github.com/apache/spark/pull/1729
SPARK-2800: Exclude scalastyle-output.xml Apache RAT checks
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/witgo/spark SPARK-2800
Alternatively
Github user witgo commented on the pull request:
https://github.com/apache/spark/pull/1369#issuecomment-50951603
Jenkins, test this please.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user witgo commented on the pull request:
https://github.com/apache/spark/pull/1369#issuecomment-50953905
Jenkins, test this please.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
GitHub user witgo reopened a pull request:
https://github.com/apache/spark/pull/1208
[WIP][SPARK-1470]: Remove scalalogging-slf4j dependency
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/witgo/spark SPARK-1470
Alternatively
Github user witgo commented on the pull request:
https://github.com/apache/spark/pull/1208#issuecomment-50956956
@pwendell @marmbrus How to test the performance impact of the PR?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub
Github user witgo commented on the pull request:
https://github.com/apache/spark/pull/1369#issuecomment-50957000
How to re-open this ?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this
GitHub user witgo reopened a pull request:
https://github.com/apache/spark/pull/332
[SPARK-1470] remove scalalogging-slf4j dependency
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/witgo/spark remove_scalalogging
Alternatively
Github user witgo commented on the pull request:
https://github.com/apache/spark/pull/332#issuecomment-50961153
Jenkins, test this please.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user witgo commented on the pull request:
https://github.com/apache/spark/pull/1369#issuecomment-50965670
@pwendell #1208 fits your idea?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not
Github user witgo commented on the pull request:
https://github.com/apache/spark/pull/1208#issuecomment-50979440
Cool thanks
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user witgo closed the pull request at:
https://github.com/apache/spark/pull/332
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is
Github user witgo commented on the pull request:
https://github.com/apache/spark/pull/1341#issuecomment-50979611
@pwendell I think the PR can be merged into 1.1
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user witgo commented on the pull request:
https://github.com/apache/spark/pull/7279#issuecomment-122477288
This is a very good performance for ML.I will test this feature on
[cloudml/zen](https://github.com/cloudml/zen)
---
If your project is set up for it, you can reply
Github user witgo commented on the pull request:
https://github.com/apache/spark/pull/7279#issuecomment-119401465
This is very cool PR.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this
Github user witgo commented on the pull request:
https://github.com/apache/spark/pull/7621#issuecomment-126402467
LGTM
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled
Github user witgo commented on a diff in the pull request:
https://github.com/apache/spark/pull/7621#discussion_r35390543
--- Diff: mllib/src/main/scala/org/apache/spark/mllib/ann/Layer.scala ---
@@ -0,0 +1,856 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF
Github user witgo commented on a diff in the pull request:
https://github.com/apache/spark/pull/7621#discussion_r35408460
--- Diff: mllib/src/main/scala/org/apache/spark/mllib/ann/Layer.scala ---
@@ -0,0 +1,856 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF
Github user witgo commented on a diff in the pull request:
https://github.com/apache/spark/pull/7621#discussion_r35409451
--- Diff: mllib/src/main/scala/org/apache/spark/mllib/ann/Layer.scala ---
@@ -0,0 +1,856 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF
Github user witgo commented on a diff in the pull request:
https://github.com/apache/spark/pull/7621#discussion_r35410425
--- Diff: mllib/src/main/scala/org/apache/spark/mllib/ann/Layer.scala ---
@@ -0,0 +1,856 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF
Github user witgo commented on a diff in the pull request:
https://github.com/apache/spark/pull/7621#discussion_r35433813
--- Diff:
mllib/src/main/scala/org/apache/spark/ml/classification/MultilayerPerceptronClassifier.scala
---
@@ -0,0 +1,130 @@
+/*
+ * Licensed to the
Github user witgo commented on a diff in the pull request:
https://github.com/apache/spark/pull/7621#discussion_r35435404
--- Diff: mllib/src/main/scala/org/apache/spark/mllib/ann/Layer.scala ---
@@ -0,0 +1,857 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF
Github user witgo commented on a diff in the pull request:
https://github.com/apache/spark/pull/7621#discussion_r35436475
--- Diff: mllib/src/main/scala/org/apache/spark/mllib/ann/Layer.scala ---
@@ -0,0 +1,856 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF
Github user witgo commented on a diff in the pull request:
https://github.com/apache/spark/pull/7621#discussion_r35607344
--- Diff:
mllib/src/main/scala/org/apache/spark/ml/classification/MultilayerPerceptronClassifier.scala
---
@@ -0,0 +1,130 @@
+/*
+ * Licensed to the
Github user witgo commented on the pull request:
https://github.com/apache/spark/pull/6717#issuecomment-110612334
LGTM
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled
Github user witgo commented on the pull request:
https://github.com/apache/spark/pull/1290#issuecomment-104111324
@ankurdave
The `DBN` related code:
https://github.com/witgo/spark/tree/ann-interface-gemm-dbn
---
If your project is set up for it, you can reply to this email
Github user witgo commented on the pull request:
https://github.com/apache/spark/pull/6393#issuecomment-105154271
The code is useless ?
https://github.com/apache/spark/tree/master/sql/hive
---
If your project is set up for it, you can reply to this email and have your
reply
GitHub user witgo opened a pull request:
https://github.com/apache/spark/pull/11986
DAGScheduler should get map output statuses directly.
## What changes were proposed in this pull request?
Add a new method to `MapOutputTracker` class
```scala
def
Github user witgo commented on the pull request:
https://github.com/apache/spark/pull/11986#issuecomment-202179133
In actual use of MapStatus. The array stored in MapStatus is only removed
or added.The elements in this array are not modified. Therefore, this array can
be used outside
Github user witgo commented on the pull request:
https://github.com/apache/spark/pull/11986#issuecomment-202778045
retest please.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user witgo commented on the pull request:
https://github.com/apache/spark/pull/12423#issuecomment-211219477
I think the following should be deleted
1.
[FileShuffleBlockResolver.scala](https://github.com/apache/spark/blob/master/core/src/main/scala/org/apache/spark/shuffle
Github user witgo closed the pull request at:
https://github.com/apache/spark/pull/11986
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is
GitHub user witgo reopened a pull request:
https://github.com/apache/spark/pull/11986
[CORE][SPARK-14178]DAGScheduler should get map output statuses directly.
## What changes were proposed in this pull request?
Add a new method to `MapOutputTracker` class
```scala
Github user witgo commented on the pull request:
https://github.com/apache/spark/pull/11986#issuecomment-203695553
There are lots of map and reduce, MapStatus deserialization may take a few
seconds
---
If your project is set up for it, you can reply to this email and have your
reply
Github user witgo closed the pull request at:
https://github.com/apache/spark/pull/11986
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is
Github user witgo commented on a diff in the pull request:
https://github.com/apache/spark/pull/12113#discussion_r58324785
--- Diff: core/src/main/scala/org/apache/spark/MapOutputTracker.scala ---
@@ -428,40 +503,93 @@ private[spark] class MapOutputTrackerMaster(conf:
SparkConf
Github user witgo commented on a diff in the pull request:
https://github.com/apache/spark/pull/12113#discussion_r58325719
--- Diff: core/src/main/scala/org/apache/spark/MapOutputTracker.scala ---
@@ -296,10 +290,89 @@ private[spark] class MapOutputTrackerMaster(conf:
SparkConf
Github user witgo commented on a diff in the pull request:
https://github.com/apache/spark/pull/12113#discussion_r58502169
--- Diff: core/src/main/scala/org/apache/spark/MapOutputTracker.scala ---
@@ -296,10 +290,89 @@ private[spark] class MapOutputTrackerMaster(conf:
SparkConf
Github user witgo commented on a diff in the pull request:
https://github.com/apache/spark/pull/12113#discussion_r58503061
--- Diff: core/src/main/scala/org/apache/spark/MapOutputTracker.scala ---
@@ -428,40 +503,93 @@ private[spark] class MapOutputTrackerMaster(conf:
SparkConf
Github user witgo commented on a diff in the pull request:
https://github.com/apache/spark/pull/12113#discussion_r58637905
--- Diff: core/src/main/scala/org/apache/spark/MapOutputTracker.scala ---
@@ -428,40 +503,93 @@ private[spark] class MapOutputTrackerMaster(conf:
SparkConf
Github user witgo commented on a diff in the pull request:
https://github.com/apache/spark/pull/12113#discussion_r58641966
--- Diff: core/src/main/scala/org/apache/spark/MapOutputTracker.scala ---
@@ -296,10 +290,89 @@ private[spark] class MapOutputTrackerMaster(conf:
SparkConf
Github user witgo commented on a diff in the pull request:
https://github.com/apache/spark/pull/12113#discussion_r58643596
--- Diff: core/src/main/scala/org/apache/spark/MapOutputTracker.scala ---
@@ -492,16 +624,51 @@ private[spark] object MapOutputTracker extends
Logging
Github user witgo commented on a diff in the pull request:
https://github.com/apache/spark/pull/12113#discussion_r58711271
--- Diff: core/src/main/scala/org/apache/spark/MapOutputTracker.scala ---
@@ -492,16 +624,51 @@ private[spark] object MapOutputTracker extends
Logging
GitHub user witgo opened a pull request:
https://github.com/apache/spark/pull/14647
[WIP][Test only][SPARK-6235]Address various 2G limits
## What changes were proposed in this pull request?
The main changes.
1. Replace DiskStore method `def getBytes (blockId: BlockId
Github user witgo commented on the issue:
https://github.com/apache/spark/pull/14647
@hvanhovell
I will submit some small PRs and provide a more high level description of
them.
---
If your project is set up for it, you can reply to this email and have your
reply appear on
GitHub user witgo opened a pull request:
https://github.com/apache/spark/pull/14658
[WIP][SPARK-5928] Remote Shuffle Blocks cannot be more than 2 GB
## What changes were proposed in this pull request?
Add class `ChunkFetchInputStream` and it have the following effects
GitHub user witgo opened a pull request:
https://github.com/apache/spark/pull/14662
[WIP][SPARK-17082][CORE]Replace ByteBuffer with ChunkedByteBuffer
## What changes were proposed in this pull request?
Replace `ByteBuffer` with `ChunkedByteBuffer
GitHub user witgo opened a pull request:
https://github.com/apache/spark/pull/14664
Spark 6236 caching blocks 2G
## What changes were proposed in this pull request?
(Please fill in changes proposed in this fix)
## How was this patch tested?
(Please
Github user witgo commented on the issue:
https://github.com/apache/spark/pull/14647
@hvanhovell #14658 , #14662 and #14664
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
GitHub user witgo opened a pull request:
https://github.com/apache/spark/pull/14751
[WIP][SPARK-17184][[CORE]]Replace ByteBuf with InputStream
## What changes were proposed in this pull request?
The size of ByteBuf can not be greater than 2G, should be replaced by
Github user witgo commented on a diff in the pull request:
https://github.com/apache/spark/pull/17480#discussion_r110361779
--- Diff:
core/src/main/scala/org/apache/spark/ExecutorAllocationManager.scala ---
@@ -249,7 +249,14 @@ private[spark] class ExecutorAllocationManager
Github user witgo commented on the issue:
https://github.com/apache/spark/pull/17567
LGTM.
Are there any performance test reports?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this
Github user witgo commented on the issue:
https://github.com/apache/spark/pull/17567
OK, I see.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the
Github user witgo commented on a diff in the pull request:
https://github.com/apache/spark/pull/17480#discussion_r110796578
--- Diff:
core/src/main/scala/org/apache/spark/ExecutorAllocationManager.scala ---
@@ -249,7 +249,14 @@ private[spark] class ExecutorAllocationManager
801 - 900 of 916 matches
Mail list logo
