Github user zuotingbing commented on a diff in the pull request:
https://github.com/apache/spark/pull/22849#discussion_r229203395
--- Diff:
core/src/main/scala/org/apache/spark/scheduler/cluster/CoarseGrainedSchedulerBackend.scala
---
@@ -240,7 +240,7 @@ class
Github user zuotingbing commented on a diff in the pull request:
https://github.com/apache/spark/pull/22849#discussion_r229198651
--- Diff:
core/src/main/scala/org/apache/spark/scheduler/cluster/CoarseGrainedSchedulerBackend.scala
---
@@ -240,7 +240,7 @@ class
Github user zuotingbing commented on a diff in the pull request:
https://github.com/apache/spark/pull/22849#discussion_r228785046
--- Diff:
core/src/main/scala/org/apache/spark/scheduler/cluster/CoarseGrainedSchedulerBackend.scala
---
@@ -240,7 +240,7 @@ class
GitHub user zuotingbing opened a pull request:
https://github.com/apache/spark/pull/22849
[SPARK-25852][Core] we should filter the workOffers of which freeCores>0
when make fake resource offers on all executors
## What changes were proposed in this pull request?
Github user zuotingbing commented on the issue:
https://github.com/apache/spark/pull/22849
` , Could you give a help to check
it please? i find you are the latest
Github user zuotingbing commented on the issue:
https://github.com/apache/spark/pull/21789
retest this please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail
Github user zuotingbing commented on the issue:
https://github.com/apache/spark/pull/21789
@HyukjinKwon i am not sure what is the reason about the tests failed. i
checked the `unit-tests.log` and it seems the test "SPARK-24829 Checks cast as
float" , which i added, hav
Github user zuotingbing commented on the issue:
https://github.com/apache/spark/pull/21789
@mgaido91 yes , update it. Thanks.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user zuotingbing commented on a diff in the pull request:
https://github.com/apache/spark/pull/21789#discussion_r203682086
--- Diff:
sql/hive-thriftserver/src/test/scala/org/apache/spark/sql/hive/thriftserver/HiveThriftServer2Suites.scala
---
@@ -766,6 +774,14 @@ class
Github user zuotingbing commented on a diff in the pull request:
https://github.com/apache/spark/pull/21789#discussion_r203675324
--- Diff:
sql/hive-thriftserver/src/main/java/org/apache/hive/service/cli/Column.java ---
@@ -349,7 +349,7 @@ public void addValue(Type type, Object
Github user zuotingbing commented on a diff in the pull request:
https://github.com/apache/spark/pull/21789#discussion_r203613567
--- Diff:
sql/hive-thriftserver/src/test/scala/org/apache/spark/sql/hive/thriftserver/HiveThriftServer2Suites.scala
---
@@ -766,6 +774,14 @@ class
Github user zuotingbing commented on a diff in the pull request:
https://github.com/apache/spark/pull/21789#discussion_r203610279
--- Diff:
sql/hive-thriftserver/src/main/java/org/apache/hive/service/cli/Column.java ---
@@ -349,7 +349,7 @@ public void addValue(Type type, Object
Github user zuotingbing commented on the issue:
https://github.com/apache/spark/pull/21789
yes, only in STS, i will update the title
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user zuotingbing commented on a diff in the pull request:
https://github.com/apache/spark/pull/21789#discussion_r203239370
--- Diff:
sql/hive-thriftserver/src/test/scala/org/apache/spark/sql/hive/thriftserver/HiveThriftServer2Suites.scala
---
@@ -766,6 +774,14 @@ class
Github user zuotingbing closed the pull request at:
https://github.com/apache/spark/pull/18812
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h
Github user zuotingbing closed the pull request at:
https://github.com/apache/spark/pull/20864
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h
GitHub user zuotingbing opened a pull request:
https://github.com/apache/spark/pull/21789
[SPARK-24829][SQL]CAST AS FLOAT inconsistent with Hive
## What changes were proposed in this pull request?
SELECT CAST('4.56' AS FLOAT)
the result is 4.55942779541
Github user zuotingbing commented on the issue:
https://github.com/apache/spark/pull/20864
@felixcheung Thanks. The test result is 0 failed, what is meaning of
> This patch fails due to an unknown error code, -9
? what should i do to fix the test fa
Github user zuotingbing commented on the issue:
https://github.com/apache/spark/pull/20864
`add jar 'hdfs:///xxx.jar'`
or create UDF will download jars to the resource dir:
`CREATE FUNCTION xxx AS 'xxx' USING JAR 'hdfs:///xxx.jar'
SELECT xxx
Github user zuotingbing commented on the issue:
https://github.com/apache/spark/pull/20864
i take a look at [https://github.com/apache/spark/pull/18666]. i find it
can not clean all the ***_resources directories. because when we start
HiveThriftServer2, we created two resource
Github user zuotingbing commented on the issue:
https://github.com/apache/spark/pull/20864
@liufengdb would you please take a review. Thanks!
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
GitHub user zuotingbing opened a pull request:
https://github.com/apache/spark/pull/20864
[SPARK-23745][SQL]Remove the directories of the
âhive.downloaded.resources.dirâ when HiveThriftServer2 stopped
## What changes were proposed in this pull request?
![2018-03
Github user zuotingbing commented on the issue:
https://github.com/apache/spark/pull/20702
@cloud-fan @felixcheung would you please take a review. Thanks!
---
-
To unsubscribe, e-mail: reviews-unsubscr
Github user zuotingbing commented on the issue:
https://github.com/apache/spark/pull/20702
@gatorsmile @liufengdb please take a look at this, thanks!
---
-
To unsubscribe, e-mail: reviews-unsubscr
Github user zuotingbing commented on the issue:
https://github.com/apache/spark/pull/20702
Jenkins, ok to test
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail
Github user zuotingbing commented on a diff in the pull request:
https://github.com/apache/spark/pull/20702#discussion_r172166003
--- Diff:
sql/hive-thriftserver/src/main/java/org/apache/hive/service/cli/session/HiveSessionImpl.java
---
@@ -665,6 +667,25 @@ public void close
Github user zuotingbing commented on a diff in the pull request:
https://github.com/apache/spark/pull/20702#discussion_r171749296
--- Diff:
sql/hive-thriftserver/src/main/java/org/apache/hive/service/cli/session/HiveSessionImpl.java
---
@@ -665,6 +667,25 @@ public void close
GitHub user zuotingbing opened a pull request:
https://github.com/apache/spark/pull/20702
[SPARK-23547][SQL]Cleanup the .pipeout file when the Hive Session closed
## What changes were proposed in this pull request?
when the hive session closed, we should also cleanup
Github user zuotingbing commented on the issue:
https://github.com/apache/spark/pull/20025
@gatorsmile @liufengdb Anything else? Thanks!
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
Github user zuotingbing commented on a diff in the pull request:
https://github.com/apache/spark/pull/20025#discussion_r163136632
--- Diff:
sql/hive-thriftserver/src/main/java/org/apache/hive/service/cli/session/SessionManager.java
---
@@ -80,7 +76,6 @@ public synchronized void
Github user zuotingbing commented on a diff in the pull request:
https://github.com/apache/spark/pull/20025#discussion_r163127098
--- Diff:
sql/hive-thriftserver/src/main/java/org/apache/hive/service/cli/session/SessionManager.java
---
@@ -23,11 +23,7 @@
import
Github user zuotingbing commented on a diff in the pull request:
https://github.com/apache/spark/pull/20025#discussion_r162528506
--- Diff:
sql/hive-thriftserver/src/main/java/org/apache/hive/service/cli/session/SessionManager.java
---
@@ -79,35 +75,19 @@ public synchronized void
Github user zuotingbing commented on the issue:
https://github.com/apache/spark/pull/20025
@cloud-fan @gatorsmile @liufengdb @felixcheung @srowen @vanzin Is anybody
could make further contact and discuss this PR? Thanks
Github user zuotingbing commented on the issue:
https://github.com/apache/spark/pull/20025
ping @liufengdb
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews
Github user zuotingbing commented on the issue:
https://github.com/apache/spark/pull/20025
ping gatorsmile
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews
Github user zuotingbing closed the pull request at:
https://github.com/apache/spark/pull/19989
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h
Github user zuotingbing commented on the issue:
https://github.com/apache/spark/pull/20025
@gatorsmile What is your opinion about this? Thanks!
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
Github user zuotingbing commented on the issue:
https://github.com/apache/spark/pull/19989
ok, got it. Thanks!
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail
Github user zuotingbing commented on the issue:
https://github.com/apache/spark/pull/19989
@gatorsmile @liufengdb Could you please also check this PR ? it
[BACKPORT-2.0] from master/2.3 about
[?](https://github.com/apache/spark/pull/20029
Github user zuotingbing commented on the issue:
https://github.com/apache/spark/pull/20025
@rxin Could you please to review this? Thanks.
In my opinion we can create a new or follow-up PR if refactor is necessary.
This PR is to fix the bug about the Session Timeout Checker
Github user zuotingbing commented on the issue:
https://github.com/apache/spark/pull/20025
@liufengdb I think the class `SessionManager.java` is merged from Hive
originally, and in Spark we redesigned it by adding
`SparkSQLSessionManager.scala` with no affect to `SessionManager.java
Github user zuotingbing commented on the issue:
https://github.com/apache/spark/pull/20025
Currently we do not call the `super.init(hiveConf)` in
`SparkSQLSessionManager.init`, it means `SessionManager.init` which includes
`createBackgroundOperationPool()` will not be called
Github user zuotingbing commented on the issue:
https://github.com/apache/spark/pull/20029
Could you please to check this PR? Thanks @liufengdb
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
Github user zuotingbing commented on the issue:
https://github.com/apache/spark/pull/20025
Could you please merge this to Master? Thanks @felixcheung
---
-
To unsubscribe, e-mail: reviews-unsubscr
Github user zuotingbing commented on a diff in the pull request:
https://github.com/apache/spark/pull/20025#discussion_r158628287
--- Diff:
sql/hive-thriftserver/src/main/java/org/apache/hive/service/cli/session/SessionManager.java
---
@@ -148,6 +148,15 @@ public synchronized
Github user zuotingbing commented on the issue:
https://github.com/apache/spark/pull/19989
OK , please move to https://github.com/apache/spark/pull/20029. Thanks all.
---
-
To unsubscribe, e-mail: reviews-unsubscr
Github user zuotingbing commented on the issue:
https://github.com/apache/spark/pull/20029
`override protected lazy val resourceLoader: HiveSessionResourceLoader = {
val client: HiveClient = externalCatalog.client.newSession()
new HiveSessionResourceLoader(session
Github user zuotingbing commented on the issue:
https://github.com/apache/spark/pull/20029
It seems each time when connect to thrift server through beeline, the
`SessionState.start(state)` will be called two times. one is in
`HiveSessionImpl:open` , another
Github user zuotingbing commented on the issue:
https://github.com/apache/spark/pull/19989
Could you please to check this PR or find out how to correct it? It seems a
critical bug. Thanks! @cloud-fan @rxin
Github user zuotingbing commented on the issue:
https://github.com/apache/spark/pull/20029
Thanks @srowen , so whom could i ping to make sure this change has no side
effects?
---
-
To unsubscribe, e-mail: reviews
GitHub user zuotingbing opened a pull request:
https://github.com/apache/spark/pull/20029
[SPARK-22793][SQL]Memory leak in Spark Thrift Server
# What changes were proposed in this pull request?
1. Start HiveThriftServer2.
2. Connect to thriftserver through beeline.
3
GitHub user zuotingbing opened a pull request:
https://github.com/apache/spark/pull/20025
[SPARK-22837][SQL]Session timeout checker does not work in SessionManager.
## What changes were proposed in this pull request?
Currently we do not call the `super.init(hiveConf
Github user zuotingbing commented on the issue:
https://github.com/apache/spark/pull/19989
we can find the cache size of FileSystem `deleteOnExit` will keep
increasing.
[
` will be called two times. one is in
`HiveSessionImpl:open` , **another
GitHub user zuotingbing opened a pull request:
https://github.com/apache/spark/pull/19989
[SPARK-22793][SQL]Memory leak in Spark Thrift Server
## What changes were proposed in this pull request?
1. Start HiveThriftServer2.
2. Connect to thriftserver through beeline
Github user zuotingbing commented on the issue:
https://github.com/apache/spark/pull/19841
Thanks srowen.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h
Github user zuotingbing commented on the issue:
https://github.com/apache/spark/pull/19841
It would be great if someone help to merge it to master, Thanks!
@gatorsmile @srowen @HyukjinKwon @wangyum
Github user zuotingbing commented on the issue:
https://github.com/apache/spark/pull/19841
please review it again, thanks all.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user zuotingbing commented on the issue:
https://github.com/apache/spark/pull/19841
I extract a separate function but it has too many parameters. Could i
extract several separate functions
Github user zuotingbing commented on the issue:
https://github.com/apache/spark/pull/19841
Yes i am also confused why the diff is so big but it reported by git
originally.
---
-
To unsubscribe, e-mail: reviews
GitHub user zuotingbing opened a pull request:
https://github.com/apache/spark/pull/19841
[SPARK-22642][SQL] the createdTempDir will not be deleted if an exception
occurs
## What changes were proposed in this pull request?
We found staging directories
Github user zuotingbing commented on the issue:
https://github.com/apache/spark/pull/19398
Got it, thanks all.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail
Github user zuotingbing closed the pull request at:
https://github.com/apache/spark/pull/19398
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h
Github user zuotingbing commented on the issue:
https://github.com/apache/spark/pull/19398
Do you know what is the reason to choice to manually create? What about
changing it to create it automatically? Thanks
GitHub user zuotingbing opened a pull request:
https://github.com/apache/spark/pull/19398
[SPARK-22174][CORE]Support to automatically create the directory where the
event logs go (`spark.eventLog.dir`)
## What changes were proposed in this pull request?
`2017-09-30 09
Github user zuotingbing commented on the issue:
https://github.com/apache/spark/pull/19300
if we find some mistakes likes this , shouldn't we to report it at first?
---
-
To unsubscribe, e-mail: reviews-unsubscr
Github user zuotingbing commented on the issue:
https://github.com/apache/spark/pull/19300
@cloud-fan @gatorsmile Could you please help to review this and merge it
to master? Thanks.
---
-
To unsubscribe, e-mail
Github user zuotingbing commented on the issue:
https://github.com/apache/spark/pull/19300
> or don't bother and close this
I have not found other typos and sorry i'm not agree with you. Since this
spelling mistake affects the API doc in official website, we should
Github user zuotingbing closed the pull request at:
https://github.com/apache/spark/pull/19275
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h
Github user zuotingbing commented on a diff in the pull request:
https://github.com/apache/spark/pull/19277#discussion_r140137420
--- Diff:
core/src/main/scala/org/apache/spark/scheduler/EventLoggingListener.scala ---
@@ -351,14 +351,14 @@ private[spark] object
GitHub user zuotingbing opened a pull request:
https://github.com/apache/spark/pull/19300
[SPARK-22082][SparkR]Spelling mistake: choosen in API doc of R.
## What changes were proposed in this pull request?
"choosen" should be "chosen" in AP
Github user zuotingbing commented on a diff in the pull request:
https://github.com/apache/spark/pull/19277#discussion_r139902219
--- Diff:
core/src/main/scala/org/apache/spark/scheduler/EventLoggingListener.scala ---
@@ -351,11 +351,11 @@ private[spark] object
Github user zuotingbing commented on the issue:
https://github.com/apache/spark/pull/19277
@jerryshao if this line `new BufferedInputStream(fs.open(log))` throws
exception, it mean we do not need to close the object of BufferedInputStream
because it new failed
Github user zuotingbing commented on the issue:
https://github.com/apache/spark/pull/19277
I am not sure. If we move this line `val in = new
BufferedInputStream(fs.open(log))` into try~catch, we should define `var in:
BufferedInputStream = null` before, and use `catch { case e
Github user zuotingbing commented on a diff in the pull request:
https://github.com/apache/spark/pull/19277#discussion_r139878136
--- Diff:
core/src/main/scala/org/apache/spark/scheduler/EventLoggingListener.scala ---
@@ -351,11 +351,11 @@ private[spark] object
Github user zuotingbing commented on a diff in the pull request:
https://github.com/apache/spark/pull/19277#discussion_r139665504
--- Diff:
core/src/main/scala/org/apache/spark/scheduler/EventLoggingListener.scala ---
@@ -347,15 +347,14 @@ private[spark] object
GitHub user zuotingbing opened a pull request:
https://github.com/apache/spark/pull/19277
[SPARK-22058][COREthe BufferedInputStream will not be closed if an
exception occurs.
## What changes were proposed in this pull request?
EventLoggingListener use `val in = new
GitHub user zuotingbing opened a pull request:
https://github.com/apache/spark/pull/19275
[SPARK-22057][CORE]Provide to update the applications in the HistoryPage
which the event log files have not been existed.
## What changes were proposed in this pull request
Github user zuotingbing closed the pull request at:
https://github.com/apache/spark/pull/19034
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature
Github user zuotingbing commented on the issue:
https://github.com/apache/spark/pull/19034
@srowen @vanzin could you please take a look at this? Thanks.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project
Github user zuotingbing commented on the issue:
https://github.com/apache/spark/pull/19034
Hmm, this problem is common , i think some helpful tool scripts is to be
expected to make spark more robust, and are useful for users. ï¼ï¼
Similarly In Hive, there is a tool script
GitHub user zuotingbing opened a pull request:
https://github.com/apache/spark/pull/19034
[SPARK-21821][CORE]Support to force kill the CoarseGrainedExecutorBackend
process which is likely be orphaned.
## What changes were proposed in this pull request?
In out
Github user zuotingbing commented on the issue:
https://github.com/apache/spark/pull/18811
ok, have done. Thanks @srowen .
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user zuotingbing commented on a diff in the pull request:
https://github.com/apache/spark/pull/18816#discussion_r131037188
--- Diff:
sql/hive-thriftserver/src/main/scala/org/apache/spark/sql/hive/thriftserver/SparkSQLCLIDriver.scala
---
@@ -272,7 +272,7 @@ private[hive
GitHub user zuotingbing opened a pull request:
https://github.com/apache/spark/pull/18816
[SPARK-21611][SQL]Error class name for log in several classes.
## What changes were proposed in this pull request?
Error class name for log in several classes. such as:
`2017-08-02
Github user zuotingbing commented on the issue:
https://github.com/apache/spark/pull/18811
Thanks @srowen , will do.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled
GitHub user zuotingbing opened a pull request:
https://github.com/apache/spark/pull/18812
[SPARK-21606][SQL]HiveThriftServer2 catches OOMs on request threads
## What changes were proposed in this pull request?
Refer to Hive, ThriftCLIService methods such as ExecuteStatement
Github user zuotingbing closed the pull request at:
https://github.com/apache/spark/pull/18619
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature
GitHub user zuotingbing opened a pull request:
https://github.com/apache/spark/pull/18811
[Spark-21604][SQL]Error class name for log, and if the object extends
Logging, i suggest to remove the var LOG which is useless.
## What changes were proposed in this pull request
Github user zuotingbing commented on the issue:
https://github.com/apache/spark/pull/18619
Yes this PR is a trivial Improvement for me. i will closed it if this has
side-effects exactly. Thanks.
---
If your project is set up for it, you can reply to this email and have your
reply
Github user zuotingbing commented on the issue:
https://github.com/apache/spark/pull/18619
I just use `mvn checkstyle:check` ,which contains output files with
`basedir`, to check whether the `basedir` is correct with this PR.
Could you please specify the affect exactly as you
Github user zuotingbing commented on the issue:
https://github.com/apache/spark/pull/18619
@srowen Thanks for your time. I am not sure the open issue, which is
mentioned in the note, whether has bean resolved. as the note said
> setting a value for this parame
GitHub user zuotingbing opened a pull request:
https://github.com/apache/spark/pull/18619
[SPARK-21397][BUILD]Maven shade plugin adding dependency-reduced-pom.xml to
â¦
â¦base directory
## What changes were proposed in this pull request?
1. run `./build/mvn
Github user zuotingbing commented on the issue:
https://github.com/apache/spark/pull/18285
Thanks @vanzin !
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes
GitHub user zuotingbing opened a pull request:
https://github.com/apache/spark/pull/18285
[SPARK-20338]Spaces in spark.eventLog.dir are not correctly handled
## What changes were proposed in this pull request?
âspark.eventLog.dirâ supports with space characters
Github user zuotingbing commented on a diff in the pull request:
https://github.com/apache/spark/pull/18158#discussion_r119535585
--- Diff: core/src/test/scala/org/apache/spark/util/UtilsSuite.scala ---
@@ -473,7 +474,7 @@ class UtilsSuite extends SparkFunSuite
1 - 100 of 159 matches
Mail list logo
