[GitHub] [spark] HeartSaVioR commented on issue #23747: [SPARK-26848][SQL] Introduce new option to Kafka source: offset by timestamp (starting/ending)

HeartSaVioR commented on issue #23747: [SPARK-26848][SQL] Introduce new option to Kafka source: offset by timestamp (starting/ending) URL: https://github.com/apache/spark/pull/23747#issuecomment-531647803 Rebased with master as we merged #22282

[GitHub] [spark] HeartSaVioR commented on a change in pull request #23747: [SPARK-26848][SQL] Introduce new option to Kafka source: offset by timestamp (starting/ending)

HeartSaVioR commented on a change in pull request #23747: [SPARK-26848][SQL] Introduce new option to Kafka source: offset by timestamp (starting/ending) URL: https://github.com/apache/spark/pull/23747#discussion_r324515259 ## File path:

[GitHub] [spark] MichaelChirico commented on issue #24888: [SPARK-28040][SPARK-28070][R] Write type object s3

MichaelChirico commented on issue #24888: [SPARK-28040][SPARK-28070][R] Write type object s3 URL: https://github.com/apache/spark/pull/24888#issuecomment-531647608 I didn't see anything earlier. Of course on re-build when I pipe to a log to `grep`... the build succeeds 樂 ```

[GitHub] [spark] srowen commented on a change in pull request #25799: [SPARK-28483][FOLLOW-UP] Fix flaky test in BarrierTaskContextSuite

srowen commented on a change in pull request #25799: [SPARK-28483][FOLLOW-UP] Fix flaky test in BarrierTaskContextSuite URL: https://github.com/apache/spark/pull/25799#discussion_r324515180 ## File path: core/src/test/scala/org/apache/spark/scheduler/BarrierTaskContextSuite.scala

[GitHub] [spark] HeartSaVioR commented on a change in pull request #23747: [SPARK-26848][SQL] Introduce new option to Kafka source: offset by timestamp (starting/ending)

HeartSaVioR commented on a change in pull request #23747: [SPARK-26848][SQL] Introduce new option to Kafka source: offset by timestamp (starting/ending) URL: https://github.com/apache/spark/pull/23747#discussion_r324515092 ## File path:

[GitHub] [spark] HyukjinKwon commented on a change in pull request #25782: [SPARK-29074][SQL] Optimize `date_format` for foldable `fmt`

HyukjinKwon commented on a change in pull request #25782: [SPARK-29074][SQL] Optimize `date_format` for foldable `fmt` URL: https://github.com/apache/spark/pull/25782#discussion_r324488922 ## File path:

[GitHub] [spark] HyukjinKwon commented on issue #25782: [SPARK-29074][SQL] Optimize `date_format` for foldable `fmt`

HyukjinKwon commented on issue #25782: [SPARK-29074][SQL] Optimize `date_format` for foldable `fmt` URL: https://github.com/apache/spark/pull/25782#issuecomment-531643617 LGTM if tests pass. This is an automated message from

[GitHub] [spark] WeichenXu123 commented on issue #25799: [SPARK-28483][FOLLOW-UP] Fix flaky test in BarrierTaskContextSuite

WeichenXu123 commented on issue #25799: [SPARK-28483][FOLLOW-UP] Fix flaky test in BarrierTaskContextSuite URL: https://github.com/apache/spark/pull/25799#issuecomment-531642823 Jenkins, retest this please. This is an

[GitHub] [spark] dongjoon-hyun commented on issue #25800: [SPARK-28932][BUILD][FOLLOWUP] Switch to scala-library compile dependency for JDK11

dongjoon-hyun commented on issue #25800: [SPARK-28932][BUILD][FOLLOWUP] Switch to scala-library compile dependency for JDK11 URL: https://github.com/apache/spark/pull/25800#issuecomment-531641905 Thank you again, @viirya !

[GitHub] [spark] viirya commented on issue #25800: [SPARK-28932][BUILD][FOLLOWUP] Switch to scala-library compile dependency for JDK11

viirya commented on issue #25800: [SPARK-28932][BUILD][FOLLOWUP] Switch to scala-library compile dependency for JDK11 URL: https://github.com/apache/spark/pull/25800#issuecomment-531641152 Verified. ``` mvn clean install -pl common/network-common -DskipTests ... [INFO] ---

[GitHub] [spark] dongjoon-hyun commented on issue #25798: [SPARK-29046][SQL][2.4] Fix NPE in SQLConf.get when active SparkContext is stopping

dongjoon-hyun commented on issue #25798: [SPARK-29046][SQL][2.4] Fix NPE in SQLConf.get when active SparkContext is stopping URL: https://github.com/apache/spark/pull/25798#issuecomment-531641079 Thank you for making a backporting, too. Yes. Amplab Jenkins has been down Today.

[GitHub] [spark] felixcheung commented on issue #24888: [SPARK-28040][SPARK-28070][R] Write type object s3

felixcheung commented on issue #24888: [SPARK-28040][SPARK-28070][R] Write type object s3 URL: https://github.com/apache/spark/pull/24888#issuecomment-531638807 Anything in the build log about: [INFO] Spark Project ML Library ... FAILURE [01:26 min]

[GitHub] [spark] HeartSaVioR commented on issue #25798: [SPARK-29046][SQL][2.4] Fix NPE in SQLConf.get when active SparkContext is stopping

HeartSaVioR commented on issue #25798: [SPARK-29046][SQL][2.4] Fix NPE in SQLConf.get when active SparkContext is stopping URL: https://github.com/apache/spark/pull/25798#issuecomment-531638191 Looks like CI build doesn't have been triggered recently.

[GitHub] [spark] dongjoon-hyun commented on issue #25800: [SPARK-28932][BUILD][FOLLOWUP] Switch to scala-library compile dependency for JDK11

dongjoon-hyun commented on issue #25800: [SPARK-28932][BUILD][FOLLOWUP] Switch to scala-library compile dependency for JDK11 URL: https://github.com/apache/spark/pull/25800#issuecomment-531636551 cc @viirya and @srowen . Could you review this once more? Sorry about making this

[GitHub] [spark] wangyum commented on issue #25797: [SPARK-29043][Core] Improve the concurrent performance of History Server

wangyum commented on issue #25797: [SPARK-29043][Core] Improve the concurrent performance of History Server URL: https://github.com/apache/spark/pull/25797#issuecomment-531636539 ok to test This is an automated message from

[GitHub] [spark] dongjoon-hyun opened a new pull request #25800: [SPARK-28932][BUILD][FOLLOWUP] Switch to scala-library compile dependency for JDK11

dongjoon-hyun opened a new pull request #25800: [SPARK-28932][BUILD][FOLLOWUP] Switch to scala-library compile dependency for JDK11 URL: https://github.com/apache/spark/pull/25800 ### What changes were proposed in this pull request? ### Why are the changes needed?

[GitHub] [spark] dongjoon-hyun edited a comment on issue #25638: [SPARK-28932][BUILD] Add scala-library test dependency to network-common module for JDK11

dongjoon-hyun edited a comment on issue #25638: [SPARK-28932][BUILD] Add scala-library test dependency to network-common module for JDK11 URL: https://github.com/apache/spark/pull/25638#issuecomment-531635273 Hi, @srowen and @viirya . Very sorry. During testing the last commit, it seems

[GitHub] [spark] dongjoon-hyun commented on issue #25638: [SPARK-28932][BUILD] Add scala-library test dependency to network-common module for JDK11

dongjoon-hyun commented on issue #25638: [SPARK-28932][BUILD] Add scala-library test dependency to network-common module for JDK11 URL: https://github.com/apache/spark/pull/25638#issuecomment-531635273 Hi, @srowen and @viirya . Very sorry. During testing the last commit, it seems that

[GitHub] [spark] sharangk commented on issue #25606: [SPARK-28814][SQL][DOC] Document SET/RESET in SQL Reference

sharangk commented on issue #25606: [SPARK-28814][SQL][DOC] Document SET/RESET in SQL Reference URL: https://github.com/apache/spark/pull/25606#issuecomment-531634459 @dilipbiswal , Thanks for the review. Resubmitted the PR with updates. Please have a look. Thanks !!!

[GitHub] [spark] sharangk commented on issue #25595: [SPARK-28792][SQL][DOC] Document CREATE DATABASE statement in SQL Reference.

sharangk commented on issue #25595: [SPARK-28792][SQL][DOC] Document CREATE DATABASE statement in SQL Reference. URL: https://github.com/apache/spark/pull/25595#issuecomment-531634436 @dilipbiswal , Thanks for the review. Resubmitted the PR with updates. Please have a look. Thanks !!!

[GitHub] [spark] WeichenXu123 opened a new pull request #25799: [SPARK-28483][FOLLOW-UP] Fix flaky test in BarrierTaskSuite

WeichenXu123 opened a new pull request #25799: [SPARK-28483][FOLLOW-UP] Fix flaky test in BarrierTaskSuite URL: https://github.com/apache/spark/pull/25799 ### What changes were proposed in this pull request? I fix the test "barrier task killed" which is flaky: split

[GitHub] [spark] kiszk commented on a change in pull request #25717: [SPARK-29013][SQL] Structurally equivalent subexpression elimination

kiszk commented on a change in pull request #25717: [SPARK-29013][SQL] Structurally equivalent subexpression elimination URL: https://github.com/apache/spark/pull/25717#discussion_r324503543 ## File path:

[GitHub] [spark] xuanyuanking commented on a change in pull request #25620: [SPARK-25341][Core] Support rolling back a shuffle map stage and re-generate the shuffle files

xuanyuanking commented on a change in pull request #25620: [SPARK-25341][Core] Support rolling back a shuffle map stage and re-generate the shuffle files URL: https://github.com/apache/spark/pull/25620#discussion_r324503207 ## File path:

[GitHub] [spark] kiszk commented on a change in pull request #25717: [SPARK-29013][SQL] Structurally equivalent subexpression elimination

kiszk commented on a change in pull request #25717: [SPARK-29013][SQL] Structurally equivalent subexpression elimination URL: https://github.com/apache/spark/pull/25717#discussion_r324503037 ## File path:

[GitHub] [spark] beliefer commented on issue #25416: [SPARK-28330][SQL] Support ANSI SQL: result offset clause in query expression

beliefer commented on issue #25416: [SPARK-28330][SQL] Support ANSI SQL: result offset clause in query expression URL: https://github.com/apache/spark/pull/25416#issuecomment-531631376 Retest this please. This is an

[GitHub] [spark] turboFei commented on a change in pull request #25797: [SPARK-29043][Core] Improve the concurrent performance of History Server

turboFei commented on a change in pull request #25797: [SPARK-29043][Core] Improve the concurrent performance of History Server URL: https://github.com/apache/spark/pull/25797#discussion_r324502916 ## File path:

[GitHub] [spark] turboFei commented on a change in pull request #25797: [SPARK-29043][Core] Improve the concurrent performance of History Server

turboFei commented on a change in pull request #25797: [SPARK-29043][Core] Improve the concurrent performance of History Server URL: https://github.com/apache/spark/pull/25797#discussion_r324502803 ## File path:

[GitHub] [spark] turboFei commented on a change in pull request #25797: [SPARK-29043][Core] Improve the concurrent performance of History Server

turboFei commented on a change in pull request #25797: [SPARK-29043][Core] Improve the concurrent performance of History Server URL: https://github.com/apache/spark/pull/25797#discussion_r324502898 ## File path:

[GitHub] [spark] cloud-fan commented on issue #25747: [SPARK-29039][SQL] centralize the catalog and table lookup logic

cloud-fan commented on issue #25747: [SPARK-29039][SQL] centralize the catalog and table lookup logic URL: https://github.com/apache/spark/pull/25747#issuecomment-531630599 Hi @rdblue , thanks for taking a look! What I want to get consensus on is the categories of v1/v2 table resolution I

[GitHub] [spark] kiszk commented on a change in pull request #25717: [SPARK-29013][SQL] Structurally equivalent subexpression elimination

kiszk commented on a change in pull request #25717: [SPARK-29013][SQL] Structurally equivalent subexpression elimination URL: https://github.com/apache/spark/pull/25717#discussion_r324501856 ## File path:

[GitHub] [spark] cloud-fan commented on issue #25626: [SPARK-28892][SQL] Add UPDATE support for DataSource V2

cloud-fan commented on issue #25626: [SPARK-28892][SQL] Add UPDATE support for DataSource V2 URL: https://github.com/apache/spark/pull/25626#issuecomment-531629992 Another benefit I can think of: with the parser rule and the UPDATE statement added, at least users can add catalyst rules to

[GitHub] [spark] cloud-fan commented on issue #25626: [SPARK-28892][SQL] Add UPDATE support for DataSource V2

cloud-fan commented on issue #25626: [SPARK-28892][SQL] Add UPDATE support for DataSource V2 URL: https://github.com/apache/spark/pull/25626#issuecomment-531629583 retest this please This is an automated message from the

[GitHub] [spark] MichaelChirico commented on a change in pull request #24888: [SPARK-28040][SPARK-28070][R] Write type object s3

MichaelChirico commented on a change in pull request #24888: [SPARK-28040][SPARK-28070][R] Write type object s3 URL: https://github.com/apache/spark/pull/24888#discussion_r324500771 ## File path: R/pkg/R/serialize.R ## @@ -36,192 +36,246 @@ # nolint end getSerdeType

[GitHub] [spark] JkSelf commented on a change in pull request #25295: [SPARK-28560][SQL] Optimize shuffle reader to local shuffle reader when smj converted to bhj in adaptive execution

JkSelf commented on a change in pull request #25295: [SPARK-28560][SQL] Optimize shuffle reader to local shuffle reader when smj converted to bhj in adaptive execution URL: https://github.com/apache/spark/pull/25295#discussion_r324500192 ## File path:

[GitHub] [spark] beliefer commented on issue #25001: [SPARK-28083][SQL] Support LIKE ... ESCAPE syntax

beliefer commented on issue #25001: [SPARK-28083][SQL] Support LIKE ... ESCAPE syntax URL: https://github.com/apache/spark/pull/25001#issuecomment-531628175 @dongjoon-hyun @HyukjinKwon Could you have time to follow up this PR ?

[GitHub] [spark] HeartSaVioR edited a comment on issue #25792: [SPARK-23539][SS][FOLLOWUP][TESTS] Add UT to ensure existing query doesn't break with default conf of includeHeaders

HeartSaVioR edited a comment on issue #25792: [SPARK-23539][SS][FOLLOWUP][TESTS] Add UT to ensure existing query doesn't break with default conf of includeHeaders URL: https://github.com/apache/spark/pull/25792#issuecomment-531535795 Actually the other intention was let test fail when

[GitHub] [spark] AngersZhuuuu commented on issue #25775: [WIP][SPARK-29015][SQL][test-hadoop3.2] fix bug for jdk11, can't use 'add jar' jar's class as create table serde

AngersZh commented on issue #25775: [WIP][SPARK-29015][SQL][test-hadoop3.2] fix bug for jdk11, can't use 'add jar' jar's class as create table serde URL: https://github.com/apache/spark/pull/25775#issuecomment-531627763 gentle ping @dongjoon-hyun can you help yo trigger again for jdk8

[GitHub] [spark] beliefer commented on a change in pull request #25416: [SPARK-28330][SQL] Support ANSI SQL: result offset clause in query expression

beliefer commented on a change in pull request #25416: [SPARK-28330][SQL] Support ANSI SQL: result offset clause in query expression URL: https://github.com/apache/spark/pull/25416#discussion_r324499438 ## File path:

[GitHub] [spark] kiszk commented on a change in pull request #25728: [SPARK-29020][WIP][SQL] Improving array_sort behaviour

kiszk commented on a change in pull request #25728: [SPARK-29020][WIP][SQL] Improving array_sort behaviour URL: https://github.com/apache/spark/pull/25728#discussion_r324499435 ## File path: sql/core/src/test/scala/org/apache/spark/sql/DataFrameFunctionsSuite.scala ## @@

[GitHub] [spark] kiszk commented on a change in pull request #25728: [SPARK-29020][WIP][SQL] Improving array_sort behaviour

kiszk commented on a change in pull request #25728: [SPARK-29020][WIP][SQL] Improving array_sort behaviour URL: https://github.com/apache/spark/pull/25728#discussion_r324499400 ## File path: sql/core/src/test/scala/org/apache/spark/sql/DataFrameFunctionsSuite.scala ## @@

[GitHub] [spark] dongjinleekr commented on a change in pull request #25792: [SPARK-23539][SS][FOLLOWUP][TESTS] Add UT to ensure existing query doesn't break with default conf of includeHeaders

dongjinleekr commented on a change in pull request #25792: [SPARK-23539][SS][FOLLOWUP][TESTS] Add UT to ensure existing query doesn't break with default conf of includeHeaders URL: https://github.com/apache/spark/pull/25792#discussion_r324499294 ## File path:

[GitHub] [spark] kiszk commented on issue #25728: [SPARK-29020][WIP][SQL] Improving array_sort behaviour

kiszk commented on issue #25728: [SPARK-29020][WIP][SQL] Improving array_sort behaviour URL: https://github.com/apache/spark/pull/25728#issuecomment-531627005 @Gschiavon @ueshin Sorry for confusing you. What I wanted to say is @ueshin 's comment.

[GitHub] [spark] HyukjinKwon commented on issue #22570: [SPARK-25553][BUILD] Add EmptyInterpolatedStringChecker to scalastyle config

HyukjinKwon commented on issue #22570: [SPARK-25553][BUILD] Add EmptyInterpolatedStringChecker to scalastyle config URL: https://github.com/apache/spark/pull/22570#issuecomment-531626835 > another option in my mind is applying this rule in master and branch-2.4 together. It will reduce

[GitHub] [spark] kiszk commented on a change in pull request #25766: [SPARK-29061][SQL] Prints bytecode statistics in debugCodegen

kiszk commented on a change in pull request #25766: [SPARK-29061][SQL] Prints bytecode statistics in debugCodegen URL: https://github.com/apache/spark/pull/25766#discussion_r324498462 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/debug/package.scala

[GitHub] [spark] sujith71955 edited a comment on issue #25720: [SPARK-28930][SQL] Last Access Time value shall display 'UNKNOWN' in all clients

sujith71955 edited a comment on issue #25720: [SPARK-28930][SQL] Last Access Time value shall display 'UNKNOWN' in all clients URL: https://github.com/apache/spark/pull/25720#issuecomment-531625336 > I still don't quite understand it, are you trying to fix a cosmetic issue or a bug? I

[GitHub] [spark] sujith71955 edited a comment on issue #25720: [SPARK-28930][SQL] Last Access Time value shall display 'UNKNOWN' in all clients

sujith71955 edited a comment on issue #25720: [SPARK-28930][SQL] Last Access Time value shall display 'UNKNOWN' in all clients URL: https://github.com/apache/spark/pull/25720#issuecomment-531625336 > I still don't quite understand it, are you trying to fix a cosmetic issue or a bug? I

[GitHub] [spark] sujith71955 commented on issue #25720: [SPARK-28930][SQL] Last Access Time value shall display 'UNKNOWN' in all clients

sujith71955 commented on issue #25720: [SPARK-28930][SQL] Last Access Time value shall display 'UNKNOWN' in all clients URL: https://github.com/apache/spark/pull/25720#issuecomment-531625336 > I still don't quite understand it, are you trying to fix a cosmetic issue or a bug? I don't see

[GitHub] [spark] turboFei commented on issue #25795: [SPARK-29037][Core] Spark gives duplicate result when an application was killed

turboFei commented on issue #25795: [SPARK-29037][Core] Spark gives duplicate result when an application was killed URL: https://github.com/apache/spark/pull/25795#issuecomment-531624809 @dongjoon-hyun Thanks for your suggestion. I have moved the UT to sql/core module. And I

[GitHub] [spark] HeartSaVioR edited a comment on issue #25790: [SPARK-29046][SQL] Fix NPE in SQLConf.get when active SparkContext is stopping

HeartSaVioR edited a comment on issue #25790: [SPARK-29046][SQL] Fix NPE in SQLConf.get when active SparkContext is stopping URL: https://github.com/apache/spark/pull/25790#issuecomment-531618439 FYI: I've raised the patch for #25798 against branch-2.4 as the branch I've originally found

[GitHub] [spark] HeartSaVioR commented on issue #25790: [SPARK-29046][SQL] Fix NPE in SQLConf.get when active SparkContext is stopping

HeartSaVioR commented on issue #25790: [SPARK-29046][SQL] Fix NPE in SQLConf.get when active SparkContext is stopping URL: https://github.com/apache/spark/pull/25790#issuecomment-531618439 FYI: I've raised the patch for #25798 against branch-2.4 as the branch I've originally found the

[GitHub] [spark] HeartSaVioR opened a new pull request #25798: [SPARK-29046][SQL][2.4] Fix NPE in SQLConf.get when active SparkContext is stopping

HeartSaVioR opened a new pull request #25798: [SPARK-29046][SQL][2.4] Fix NPE in SQLConf.get when active SparkContext is stopping URL: https://github.com/apache/spark/pull/25798 ### What changes were proposed in this pull request? This patch fixes the bug regarding NPE in

[GitHub] [spark] HeartSaVioR commented on a change in pull request #25797: [SPARK-29043][Core] Improve the concurrent performance of History Server

HeartSaVioR commented on a change in pull request #25797: [SPARK-29043][Core] Improve the concurrent performance of History Server URL: https://github.com/apache/spark/pull/25797#discussion_r324490819 ## File path:

[GitHub] [spark] HeartSaVioR commented on a change in pull request #25797: [SPARK-29043][Core] Improve the concurrent performance of History Server

HeartSaVioR commented on a change in pull request #25797: [SPARK-29043][Core] Improve the concurrent performance of History Server URL: https://github.com/apache/spark/pull/25797#discussion_r324490443 ## File path:

[GitHub] [spark] HeartSaVioR commented on a change in pull request #25797: [SPARK-29043][Core] Improve the concurrent performance of History Server

HeartSaVioR commented on a change in pull request #25797: [SPARK-29043][Core] Improve the concurrent performance of History Server URL: https://github.com/apache/spark/pull/25797#discussion_r324490965 ## File path:

[GitHub] [spark] HyukjinKwon commented on issue #25759: [SPARK-19147][CORE] Avoid task to throw NPE caused by TransportClientFactory.createClient after executor stop

HyukjinKwon commented on issue #25759: [SPARK-19147][CORE] Avoid task to throw NPE caused by TransportClientFactory.createClient after executor stop URL: https://github.com/apache/spark/pull/25759#issuecomment-531615618 @colinmjj, also please fill other items in PR description.

[GitHub] [spark] HyukjinKwon commented on a change in pull request #25761: [SPARK-29029][SQL] - Using AttributeMap in PhysicalOperation.collectProjectsAndFilters

HyukjinKwon commented on a change in pull request #25761: [SPARK-29029][SQL] - Using AttributeMap in PhysicalOperation.collectProjectsAndFilters URL: https://github.com/apache/spark/pull/25761#discussion_r324490313 ## File path:

[GitHub] [spark] HyukjinKwon commented on issue #25761: [SPARK-29029][SQL] - Using AttributeMap in PhysicalOperation.collectProjectsAndFilters

HyukjinKwon commented on issue #25761: [SPARK-29029][SQL] - Using AttributeMap in PhysicalOperation.collectProjectsAndFilters URL: https://github.com/apache/spark/pull/25761#issuecomment-531615250 @nikitagkonda, can you keep the PR template,

[GitHub] [spark] HyukjinKwon closed pull request #25774: [SPARK-29069][SQL] ResolveInsertInto should not do table lookup

HyukjinKwon closed pull request #25774: [SPARK-29069][SQL] ResolveInsertInto should not do table lookup URL: https://github.com/apache/spark/pull/25774 This is an automated message from the Apache Git Service. To respond to

[GitHub] [spark] HyukjinKwon commented on issue #25774: [SPARK-29069][SQL] ResolveInsertInto should not do table lookup

HyukjinKwon commented on issue #25774: [SPARK-29069][SQL] ResolveInsertInto should not do table lookup URL: https://github.com/apache/spark/pull/25774#issuecomment-531614819 Merged to master. This is an automated message

[GitHub] [spark] HyukjinKwon commented on a change in pull request #25777: [SPARK-29070][CORE] Allow SparkLauncher to return full spark-submit c…

HyukjinKwon commented on a change in pull request #25777: [SPARK-29070][CORE] Allow SparkLauncher to return full spark-submit c… URL: https://github.com/apache/spark/pull/25777#discussion_r324489377 ## File path: launcher/src/main/java/org/apache/spark/launcher/SparkLauncher.java

[GitHub] [spark] HyukjinKwon commented on a change in pull request #25782: [SPARK-29074][SQL] Optimize `date_format` for foldable `fmt`

HyukjinKwon commented on a change in pull request #25782: [SPARK-29074][SQL] Optimize `date_format` for foldable `fmt` URL: https://github.com/apache/spark/pull/25782#discussion_r324488922 ## File path:

[GitHub] [spark] HyukjinKwon commented on a change in pull request #25782: [SPARK-29074][SQL] Optimize `date_format` for foldable `fmt`

HyukjinKwon commented on a change in pull request #25782: [SPARK-29074][SQL] Optimize `date_format` for foldable `fmt` URL: https://github.com/apache/spark/pull/25782#discussion_r324488539 ## File path:

[GitHub] [spark] HyukjinKwon commented on issue #25795: [SPARK-29037][Core] Spark gives duplicate result when an application was killed

HyukjinKwon commented on issue #25795: [SPARK-29037][Core] Spark gives duplicate result when an application was killed URL: https://github.com/apache/spark/pull/25795#issuecomment-531610582 retest this please This is an

[GitHub] [spark] HyukjinKwon commented on issue #25797: [SPARK-29043][Core] Improve the concurrent performance of History Server

HyukjinKwon commented on issue #25797: [SPARK-29043][Core] Improve the concurrent performance of History Server URL: https://github.com/apache/spark/pull/25797#issuecomment-531610458 cc @vanzin too This is an automated

[GitHub] [spark] HyukjinKwon commented on issue #25797: [SPARK-29043][Core] Improve the concurrent performance of History Server

HyukjinKwon commented on issue #25797: [SPARK-29043][Core] Improve the concurrent performance of History Server URL: https://github.com/apache/spark/pull/25797#issuecomment-531610453 ok to test This is an automated message

[GitHub] [spark] HyukjinKwon commented on a change in pull request #25728: [SPARK-29020][WIP][SQL] Improving array_sort behaviour

HyukjinKwon commented on a change in pull request #25728: [SPARK-29020][WIP][SQL] Improving array_sort behaviour URL: https://github.com/apache/spark/pull/25728#discussion_r324487219 ## File path:

[GitHub] [spark] HeartSaVioR commented on issue #25790: [SPARK-29046][SQL] Fix NPE in SQLConf.get when active SparkContext is stopping

HeartSaVioR commented on issue #25790: [SPARK-29046][SQL] Fix NPE in SQLConf.get when active SparkContext is stopping URL: https://github.com/apache/spark/pull/25790#issuecomment-531609591 Thanks all for reviewing and merging again. :)

[GitHub] [spark] HyukjinKwon edited a comment on issue #25757: [SPARK-29052][DOCS][ML][PYTHON][CORE][R][SQL][SS] Create a Migration Guide tap in Spark documentation

HyukjinKwon edited a comment on issue #25757: [SPARK-29052][DOCS][ML][PYTHON][CORE][R][SQL][SS] Create a Migration Guide tap in Spark documentation URL: https://github.com/apache/spark/pull/25757#issuecomment-531608470 Thanks @srowen @dongjoon-hyun @viirya. I was on vacation so the

[GitHub] [spark] HyukjinKwon commented on issue #25757: [SPARK-29052][DOCS][ML][PYTHON][CORE][R][SQL][SS] Create a Migration Guide tap in Spark documentation

HyukjinKwon commented on issue #25757: [SPARK-29052][DOCS][ML][PYTHON][CORE][R][SQL][SS] Create a Migration Guide tap in Spark documentation URL: https://github.com/apache/spark/pull/25757#issuecomment-531608470 Thanks @dongjoon-hyun @viirya. I was on vacation so the reaction was late.

[GitHub] [spark] viirya commented on a change in pull request #25789: [SPARK-28927][ML] Show warning when input data to ALS is indeterminate

viirya commented on a change in pull request #25789: [SPARK-28927][ML] Show warning when input data to ALS is indeterminate URL: https://github.com/apache/spark/pull/25789#discussion_r324486310 ## File path: R/pkg/R/mllib_recommendation.R ## @@ -82,6 +82,12 @@

[GitHub] [spark] srowen commented on a change in pull request #25789: [SPARK-28927][ML] Show warning when input data to ALS is indeterminate

srowen commented on a change in pull request #25789: [SPARK-28927][ML] Show warning when input data to ALS is indeterminate URL: https://github.com/apache/spark/pull/25789#discussion_r324485669 ## File path: R/pkg/R/mllib_recommendation.R ## @@ -82,6 +82,12 @@

[GitHub] [spark] turboFei commented on issue #25797: [SPARK-29043][Core] Improve the concurrent performance of History Server

turboFei commented on issue #25797: [SPARK-29043][Core] Improve the concurrent performance of History Server URL: https://github.com/apache/spark/pull/25797#issuecomment-531606700 retest please This is an automated message

[GitHub] [spark] viirya commented on a change in pull request #25745: [SPARK-29033][SQL][WIP] Always use UnsafeRow-based version of CreateNamedStruct

viirya commented on a change in pull request #25745: [SPARK-29033][SQL][WIP] Always use UnsafeRow-based version of CreateNamedStruct URL: https://github.com/apache/spark/pull/25745#discussion_r324481472 ## File path:

[GitHub] [spark] joshrosen-stripe commented on issue #25745: [SPARK-29033][SQL][WIP] Always use UnsafeRow-based version of CreateNamedStruct

joshrosen-stripe commented on issue #25745: [SPARK-29033][SQL][WIP] Always use UnsafeRow-based version of CreateNamedStruct URL: https://github.com/apache/spark/pull/25745#issuecomment-531599373 While we're at it, there could be significant wins from eliminating the `if

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #25795: [SPARK-29037][Core] Spark gives duplicate result when an application was killed

dongjoon-hyun commented on a change in pull request #25795: [SPARK-29037][Core] Spark gives duplicate result when an application was killed URL: https://github.com/apache/spark/pull/25795#discussion_r324480846 ## File path:

[GitHub] [spark] dongjoon-hyun commented on issue #25745: [SPARK-29033][SQL][WIP] Always use UnsafeRow-based version of CreateNamedStruct

dongjoon-hyun commented on issue #25745: [SPARK-29033][SQL][WIP] Always use UnsafeRow-based version of CreateNamedStruct URL: https://github.com/apache/spark/pull/25745#issuecomment-531597617 Retest this please. This is an

[GitHub] [spark] dongjoon-hyun commented on issue #22570: [SPARK-25553][BUILD] Add EmptyInterpolatedStringChecker to scalastyle config

dongjoon-hyun commented on issue #22570: [SPARK-25553][BUILD] Add EmptyInterpolatedStringChecker to scalastyle config URL: https://github.com/apache/spark/pull/22570#issuecomment-531597327 By default, if there is no comment updates in this PR in next a few days, please close this,

[GitHub] [spark] dongjoon-hyun commented on issue #22570: [SPARK-25553][BUILD] Add EmptyInterpolatedStringChecker to scalastyle config

dongjoon-hyun commented on issue #22570: [SPARK-25553][BUILD] Add EmptyInterpolatedStringChecker to scalastyle config URL: https://github.com/apache/spark/pull/22570#issuecomment-531597158 Yep. I agree that closing is one of the option, @HyukjinKwon and @srowen . And, another option in my

[GitHub] [spark] AmplabJenkins removed a comment on issue #25797: [SPARK-29043][Core] Improve the concurrent performance of History Server

AmplabJenkins removed a comment on issue #25797: [SPARK-29043][Core] Improve the concurrent performance of History Server URL: https://github.com/apache/spark/pull/25797#issuecomment-531576634 Can one of the admins verify this patch?

[GitHub] [spark] dongjoon-hyun commented on issue #25797: [SPARK-29043][Core] Improve the concurrent performance of History Server

dongjoon-hyun commented on issue #25797: [SPARK-29043][Core] Improve the concurrent performance of History Server URL: https://github.com/apache/spark/pull/25797#issuecomment-531596395 ok to test cc @wangyum This is

[GitHub] [spark] viirya commented on a change in pull request #25789: [SPARK-28927][ML] Show warning when input data to ALS is indeterminate

viirya commented on a change in pull request #25789: [SPARK-28927][ML] Show warning when input data to ALS is indeterminate URL: https://github.com/apache/spark/pull/25789#discussion_r324478452 ## File path: R/pkg/R/mllib_recommendation.R ## @@ -82,6 +82,10 @@

[GitHub] [spark] Gschiavon commented on issue #25728: [SPARK-29020][WIP][SQL] Improving array_sort behaviour

Gschiavon commented on issue #25728: [SPARK-29020][WIP][SQL] Improving array_sort behaviour URL: https://github.com/apache/spark/pull/25728#issuecomment-531593895 I've added some changes to add array_sort as HOF. I've followed https://prestodb.github.io/docs/current/functions/array.html

[GitHub] [spark] SparkQA removed a comment on issue #25690: [SPARK-27831][FOLLOW-UP][SQL][TEST][test-maven] Move Hive test jars to local file

SparkQA removed a comment on issue #25690: [SPARK-27831][FOLLOW-UP][SQL][TEST][test-maven] Move Hive test jars to local file URL: https://github.com/apache/spark/pull/25690#issuecomment-531589508 **[Test build #4863 has

[GitHub] [spark] SparkQA commented on issue #25690: [SPARK-27831][FOLLOW-UP][SQL][TEST][test-maven] Move Hive test jars to local file

SparkQA commented on issue #25690: [SPARK-27831][FOLLOW-UP][SQL][TEST][test-maven] Move Hive test jars to local file URL: https://github.com/apache/spark/pull/25690#issuecomment-531593203 **[Test build #4863 has

[GitHub] [spark] srowen commented on a change in pull request #25789: [SPARK-28927][ML] Show warning when input data to ALS is indeterminate

srowen commented on a change in pull request #25789: [SPARK-28927][ML] Show warning when input data to ALS is indeterminate URL: https://github.com/apache/spark/pull/25789#discussion_r324476392 ## File path: R/pkg/R/mllib_recommendation.R ## @@ -82,6 +82,10 @@

[GitHub] [spark] viirya commented on a change in pull request #25789: [SPARK-28927][ML] Show warning when input data to ALS is indeterminate

viirya commented on a change in pull request #25789: [SPARK-28927][ML] Show warning when input data to ALS is indeterminate URL: https://github.com/apache/spark/pull/25789#discussion_r324476149 ## File path: R/pkg/R/mllib_recommendation.R ## @@ -82,6 +82,10 @@

[GitHub] [spark] SparkQA commented on issue #25690: [SPARK-27831][FOLLOW-UP][SQL][TEST][test-maven] Move Hive test jars to local file

SparkQA commented on issue #25690: [SPARK-27831][FOLLOW-UP][SQL][TEST][test-maven] Move Hive test jars to local file URL: https://github.com/apache/spark/pull/25690#issuecomment-531589637 **[Test build #4864 has

[GitHub] [spark] SparkQA commented on issue #25690: [SPARK-27831][FOLLOW-UP][SQL][TEST][test-maven] Move Hive test jars to local file

SparkQA commented on issue #25690: [SPARK-27831][FOLLOW-UP][SQL][TEST][test-maven] Move Hive test jars to local file URL: https://github.com/apache/spark/pull/25690#issuecomment-531589508 **[Test build #4863 has

[GitHub] [spark] srowen commented on a change in pull request #25789: [SPARK-28927][ML] Show warning when input data to ALS is indeterminate

srowen commented on a change in pull request #25789: [SPARK-28927][ML] Show warning when input data to ALS is indeterminate URL: https://github.com/apache/spark/pull/25789#discussion_r324475932 ## File path: R/pkg/R/mllib_recommendation.R ## @@ -82,6 +82,10 @@

[GitHub] [spark] srowen commented on issue #22570: [SPARK-25553][BUILD] Add EmptyInterpolatedStringChecker to scalastyle config

srowen commented on issue #22570: [SPARK-25553][BUILD] Add EmptyInterpolatedStringChecker to scalastyle config URL: https://github.com/apache/spark/pull/22570#issuecomment-531589388 I'm OK dropping this on the grounds that it makes backports harder, even though this is for Spark 3. The

[GitHub] [spark] srowen commented on a change in pull request #25689: [SPARK-28972][DOCS] Describe unit for `spark.memory.offHeap.size`

srowen commented on a change in pull request #25689: [SPARK-28972][DOCS] Describe unit for `spark.memory.offHeap.size` URL: https://github.com/apache/spark/pull/25689#discussion_r324475775 ## File path: docs/configuration.md ## @@ -866,7 +866,9 @@ Apart from these, the

[GitHub] [spark] srowen commented on a change in pull request #25689: [SPARK-28972][DOCS] Describe unit for `spark.memory.offHeap.size`

srowen commented on a change in pull request #25689: [SPARK-28972][DOCS] Describe unit for `spark.memory.offHeap.size` URL: https://github.com/apache/spark/pull/25689#discussion_r324475755 ## File path: core/src/main/scala/org/apache/spark/internal/config/package.scala ##

[GitHub] [spark] srowen commented on a change in pull request #25689: [SPARK-28972][DOCS] Describe unit for `spark.memory.offHeap.size`

srowen commented on a change in pull request #25689: [SPARK-28972][DOCS] Describe unit for `spark.memory.offHeap.size` URL: https://github.com/apache/spark/pull/25689#discussion_r324475786 ## File path: docs/configuration.md ## @@ -1384,10 +1388,12 @@ Apart from these,

[GitHub] [spark] srowen commented on issue #25720: [SPARK-28930][SQL] Last Access Time value shall display 'UNKNOWN' in all clients

srowen commented on issue #25720: [SPARK-28930][SQL] Last Access Time value shall display 'UNKNOWN' in all clients URL: https://github.com/apache/spark/pull/25720#issuecomment-531588662 I still don't quite understand it, are you trying to fix a cosmetic issue or a bug? I don't see a test

[GitHub] [spark] dongjoon-hyun closed pull request #25757: [SPARK-29052][DOCS][ML][PYTHON][CORE][R][SQL][SS] Create a Migration Guide tap in Spark documentation

dongjoon-hyun closed pull request #25757: [SPARK-29052][DOCS][ML][PYTHON][CORE][R][SQL][SS] Create a Migration Guide tap in Spark documentation URL: https://github.com/apache/spark/pull/25757 This is an automated message

[GitHub] [spark] dongjoon-hyun edited a comment on issue #25757: [SPARK-29052][DOCS][ML][PYTHON][CORE][R][SQL][SS] Create a Migration Guide tap in Spark documentation

dongjoon-hyun edited a comment on issue #25757: [SPARK-29052][DOCS][ML][PYTHON][CORE][R][SQL][SS] Create a Migration Guide tap in Spark documentation URL: https://github.com/apache/spark/pull/25757#issuecomment-531587406 Merged to master. cc @jiangxb1987 since he is a release manager

[GitHub] [spark] dongjoon-hyun commented on issue #25757: [SPARK-29052][DOCS][ML][PYTHON][CORE][R][SQL][SS] Create a Migration Guide tap in Spark documentation

dongjoon-hyun commented on issue #25757: [SPARK-29052][DOCS][ML][PYTHON][CORE][R][SQL][SS] Create a Migration Guide tap in Spark documentation URL: https://github.com/apache/spark/pull/25757#issuecomment-531587406 Merged to master. cc @jiangxb1987 since he is a release manager for

[GitHub] [spark] dongjoon-hyun closed pull request #25758: [SPARK-28856][FOLLOW-UP][SQL][TEST] Add the `namespaces` keyword to TableIdentifierParserSuite

dongjoon-hyun closed pull request #25758: [SPARK-28856][FOLLOW-UP][SQL][TEST] Add the `namespaces` keyword to TableIdentifierParserSuite URL: https://github.com/apache/spark/pull/25758 This is an automated message from the

[GitHub] [spark] dongjoon-hyun closed pull request #25790: [SPARK-29046][SQL] Fix NPE in SQLConf.get when active SparkContext is stopping

dongjoon-hyun closed pull request #25790: [SPARK-29046][SQL] Fix NPE in SQLConf.get when active SparkContext is stopping URL: https://github.com/apache/spark/pull/25790 This is an automated message from the Apache Git

[GitHub] [spark] viirya commented on a change in pull request #25789: [SPARK-28927][ML] Show warning when input data to ALS is indeterminate

viirya commented on a change in pull request #25789: [SPARK-28927][ML] Show warning when input data to ALS is indeterminate URL: https://github.com/apache/spark/pull/25789#discussion_r324474279 ## File path: R/pkg/R/mllib_recommendation.R ## @@ -82,6 +82,10 @@

  1   2   3   4   5   >