[jira] [Updated] (SPARK-42090) Introduce sasl retry count in RetryingBlockTransferor

2023-01-16 Thread Ted Yu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42090?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated SPARK-42090: --- Description: Previously a boolean variable, saslTimeoutSeen, was used in RetryingBlockTransferor. However,

[jira] [Created] (SPARK-42090) Introduce sasl retry count in RetryingBlockTransferor

2023-01-16 Thread Ted Yu (Jira)
Ted Yu created SPARK-42090: -- Summary: Introduce sasl retry count in RetryingBlockTransferor Key: SPARK-42090 URL: https://issues.apache.org/jira/browse/SPARK-42090 Project: Spark Issue Type: Bug

[jira] [Created] (SPARK-41853) Use Map in place of SortedMap for ErrorClassesJsonReader

2023-01-02 Thread Ted Yu (Jira)
Ted Yu created SPARK-41853: -- Summary: Use Map in place of SortedMap for ErrorClassesJsonReader Key: SPARK-41853 URL: https://issues.apache.org/jira/browse/SPARK-41853 Project: Spark Issue Type:

[jira] [Created] (SPARK-41705) Move generate_protos.sh to dev/

2022-12-25 Thread Ted Yu (Jira)
Ted Yu created SPARK-41705: -- Summary: Move generate_protos.sh to dev/ Key: SPARK-41705 URL: https://issues.apache.org/jira/browse/SPARK-41705 Project: Spark Issue Type: Task Components:

[jira] [Created] (SPARK-41419) [K8S] Decrement PVC_COUNTER when the pod deletion happens

2022-12-06 Thread Ted Yu (Jira)
Ted Yu created SPARK-41419: -- Summary: [K8S] Decrement PVC_COUNTER when the pod deletion happens Key: SPARK-41419 URL: https://issues.apache.org/jira/browse/SPARK-41419 Project: Spark Issue Type:

[jira] [Resolved] (SPARK-41274) Bump Kubernetes Client Version to 6.2.0

2022-11-26 Thread Ted Yu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41274?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu resolved SPARK-41274. Resolution: Duplicate This is dup of commit 02a2242a45062755bf7e20805958d5bdf1f5ed74 > Bump Kubernetes

[jira] [Created] (SPARK-41274) Bump Kubernetes Client Version to 6.2.0

2022-11-26 Thread Ted Yu (Jira)
Ted Yu created SPARK-41274: -- Summary: Bump Kubernetes Client Version to 6.2.0 Key: SPARK-41274 URL: https://issues.apache.org/jira/browse/SPARK-41274 Project: Spark Issue Type: Improvement

[jira] [Created] (SPARK-41197) Upgrade Kafka version to 3.3 release

2022-11-18 Thread Ted Yu (Jira)
Ted Yu created SPARK-41197: -- Summary: Upgrade Kafka version to 3.3 release Key: SPARK-41197 URL: https://issues.apache.org/jira/browse/SPARK-41197 Project: Spark Issue Type: Improvement

[jira] [Updated] (SPARK-40508) Treat unknown partitioning as UnknownPartitioning

2022-09-20 Thread Ted Yu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated SPARK-40508: --- Description: When running spark application against spark 3.3, I see the following : {code}

[jira] [Created] (SPARK-40508) Treat unknown partitioning as UnknownPartitioning

2022-09-20 Thread Ted Yu (Jira)
Ted Yu created SPARK-40508: -- Summary: Treat unknown partitioning as UnknownPartitioning Key: SPARK-40508 URL: https://issues.apache.org/jira/browse/SPARK-40508 Project: Spark Issue Type: Bug

[jira] [Created] (SPARK-39769) Rename trait Unevaluable

2022-07-13 Thread Ted Yu (Jira)
Ted Yu created SPARK-39769: -- Summary: Rename trait Unevaluable Key: SPARK-39769 URL: https://issues.apache.org/jira/browse/SPARK-39769 Project: Spark Issue Type: Bug Components: Spark

[jira] [Updated] (SPARK-38998) Call to MetricsSystem#getServletHandlers() may take place before MetricsSystem becomes running

2022-04-22 Thread Ted Yu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38998?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated SPARK-38998: --- Description: Sometimes the following exception is observed: {code} java.lang.IllegalArgumentException:

[jira] [Updated] (SPARK-38998) Call to MetricsSystem#getServletHandlers() may take place before MetricsSystem becomes running

2022-04-22 Thread Ted Yu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38998?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated SPARK-38998: --- Description: Sometimes the following exception is observed: {code} java.lang.IllegalArgumentException:

[jira] [Created] (SPARK-38998) Call to MetricsSystem#getServletHandlers() may take place before MetricsSystem becomes running

2022-04-22 Thread Ted Yu (Jira)
Ted Yu created SPARK-38998: -- Summary: Call to MetricsSystem#getServletHandlers() may take place before MetricsSystem becomes running Key: SPARK-38998 URL: https://issues.apache.org/jira/browse/SPARK-38998

[jira] [Commented] (SPARK-34532) IntervalUtils.add() may result in 'long overflow'

2021-02-24 Thread Ted Yu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34532?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17290646#comment-17290646 ] Ted Yu commented on SPARK-34532: Included the test command and some more information in the description.

[jira] [Updated] (SPARK-34532) IntervalUtils.add() may result in 'long overflow'

2021-02-24 Thread Ted Yu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34532?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated SPARK-34532: --- Description: I noticed the following when running test suite: build/sbt "sql/testOnly *SQLQueryTestSuite"

[jira] [Created] (SPARK-34532) IntervalUtils.add() may result in 'long overflow'

2021-02-24 Thread Ted Yu (Jira)
Ted Yu created SPARK-34532: -- Summary: IntervalUtils.add() may result in 'long overflow' Key: SPARK-34532 URL: https://issues.apache.org/jira/browse/SPARK-34532 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-34476) Duplicate referenceNames are given for ambiguousReferences

2021-02-19 Thread Ted Yu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17287403#comment-17287403 ] Ted Yu commented on SPARK-34476: The basic jsonb test is here:

[jira] [Updated] (SPARK-34476) Duplicate referenceNames are given for ambiguousReferences

2021-02-19 Thread Ted Yu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34476?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated SPARK-34476: --- Description: When running test with Spark extension that converts custom function to json path expression,

[jira] [Created] (SPARK-34476) Duplicate referenceNames are given for ambiguousReferences

2021-02-19 Thread Ted Yu (Jira)
Ted Yu created SPARK-34476: -- Summary: Duplicate referenceNames are given for ambiguousReferences Key: SPARK-34476 URL: https://issues.apache.org/jira/browse/SPARK-34476 Project: Spark Issue Type:

[jira] [Commented] (SPARK-34017) Pass json column information via pruneColumns()

2021-01-05 Thread Ted Yu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34017?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17259423#comment-17259423 ] Ted Yu commented on SPARK-34017: For PushDownUtils#pruneColumns, I am experimenting with the following:

[jira] [Updated] (SPARK-34017) Pass json column information via pruneColumns()

2021-01-05 Thread Ted Yu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34017?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated SPARK-34017: --- Description: Currently PushDownUtils#pruneColumns only passes root fields to

[jira] [Created] (SPARK-34017) Pass json column information via pruneColumns()

2021-01-05 Thread Ted Yu (Jira)
Ted Yu created SPARK-34017: -- Summary: Pass json column information via pruneColumns() Key: SPARK-34017 URL: https://issues.apache.org/jira/browse/SPARK-34017 Project: Spark Issue Type: Improvement

[jira] [Resolved] (SPARK-33997) Running bin/spark-sql gives NoSuchMethodError

2021-01-04 Thread Ted Yu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33997?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu resolved SPARK-33997. Resolution: Cannot Reproduce Rebuilt Spark locally and the error was gone. > Running bin/spark-sql gives

[jira] [Created] (SPARK-33997) Running bin/spark-sql gives NoSuchMethodError

2021-01-04 Thread Ted Yu (Jira)
Ted Yu created SPARK-33997: -- Summary: Running bin/spark-sql gives NoSuchMethodError Key: SPARK-33997 URL: https://issues.apache.org/jira/browse/SPARK-33997 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-33915) Allow json expression to be pushable column

2021-01-02 Thread Ted Yu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33915?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17257647#comment-17257647 ] Ted Yu commented on SPARK-33915: Here is sample code for capturing the column and fields in downstream

[jira] [Issue Comment Deleted] (SPARK-33915) Allow json expression to be pushable column

2021-01-02 Thread Ted Yu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33915?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated SPARK-33915: --- Comment: was deleted (was: Opened https://github.com/apache/spark/pull/30984) > Allow json expression to

[jira] [Created] (SPARK-33953) readImages test fails due to NoClassDefFoundError for ImageTypeSpecifier

2020-12-31 Thread Ted Yu (Jira)
Ted Yu created SPARK-33953: -- Summary: readImages test fails due to NoClassDefFoundError for ImageTypeSpecifier Key: SPARK-33953 URL: https://issues.apache.org/jira/browse/SPARK-33953 Project: Spark

[jira] [Commented] (SPARK-33915) Allow json expression to be pushable column

2020-12-31 Thread Ted Yu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33915?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17257025#comment-17257025 ] Ted Yu commented on SPARK-33915: Opened https://github.com/apache/spark/pull/30984 > Allow json

[jira] [Comment Edited] (SPARK-33915) Allow json expression to be pushable column

2020-12-31 Thread Ted Yu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33915?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17255389#comment-17255389 ] Ted Yu edited comment on SPARK-33915 at 12/31/20, 3:16 PM: --- Here is the plan

[jira] [Comment Edited] (SPARK-33915) Allow json expression to be pushable column

2020-12-29 Thread Ted Yu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33915?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17255389#comment-17255389 ] Ted Yu edited comment on SPARK-33915 at 12/29/20, 6:51 PM: --- Here is the plan

[jira] [Commented] (SPARK-33915) Allow json expression to be pushable column

2020-12-28 Thread Ted Yu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33915?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17255631#comment-17255631 ] Ted Yu commented on SPARK-33915: [~codingcat] [~XuanYuan][~viirya][~Alex Herman] Can you provide your

[jira] [Commented] (SPARK-33915) Allow json expression to be pushable column

2020-12-27 Thread Ted Yu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33915?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17255389#comment-17255389 ] Ted Yu commented on SPARK-33915: Here is the plan prior to predicate pushdown: {code} 2020-12-26

[jira] [Comment Edited] (SPARK-33915) Allow json expression to be pushable column

2020-12-25 Thread Ted Yu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33915?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17254944#comment-17254944 ] Ted Yu edited comment on SPARK-33915 at 12/26/20, 4:14 AM: --- I have

[jira] [Comment Edited] (SPARK-33915) Allow json expression to be pushable column

2020-12-25 Thread Ted Yu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33915?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17254944#comment-17254944 ] Ted Yu edited comment on SPARK-33915 at 12/26/20, 4:14 AM: --- I have

[jira] [Commented] (SPARK-33915) Allow json expression to be pushable column

2020-12-25 Thread Ted Yu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33915?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17254944#comment-17254944 ] Ted Yu commented on SPARK-33915: I have experimented with two patches which allow json expression to be

[jira] [Updated] (SPARK-33915) Allow json expression to be pushable column

2020-12-25 Thread Ted Yu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33915?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated SPARK-33915: --- Description: Currently PushableColumnBase provides no support for json / jsonb expression. Example of json

[jira] [Created] (SPARK-33915) Allow json expression to be pushable column

2020-12-25 Thread Ted Yu (Jira)
Ted Yu created SPARK-33915: -- Summary: Allow json expression to be pushable column Key: SPARK-33915 URL: https://issues.apache.org/jira/browse/SPARK-33915 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-25954) Upgrade to Kafka 2.1.0

2018-11-06 Thread Ted Yu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16677152#comment-16677152 ] Ted Yu commented on SPARK-25954: Looking at Kafka thread, message from Satish indicated there may be

[jira] [Created] (SPARK-25201) Synchronization performed on AtomicReference in LevelDB class

2018-08-22 Thread Ted Yu (JIRA)
Ted Yu created SPARK-25201: -- Summary: Synchronization performed on AtomicReference in LevelDB class Key: SPARK-25201 URL: https://issues.apache.org/jira/browse/SPARK-25201 Project: Spark Issue

[jira] [Commented] (SPARK-23636) [SPARK 2.2] | Kafka Consumer | KafkaUtils.createRDD throws Exception - java.util.ConcurrentModificationException: KafkaConsumer is not safe for multi-threaded access

2018-06-28 Thread Ted Yu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23636?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16527204#comment-16527204 ] Ted Yu commented on SPARK-23636: It seems in KafkaDataConsumer#close : {code} def close(): Unit =

[jira] [Commented] (SPARK-18057) Update structured streaming kafka from 0.10.0.1 to 1.1.0

2018-06-03 Thread Ted Yu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18057?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16499535#comment-16499535 ] Ted Yu commented on SPARK-18057: I have created a PR, shown above. Kafka 2.0.0 is used since that would

[jira] [Commented] (SPARK-18057) Update structured streaming kafka from 0.10.0.1 to 1.1.0

2018-05-31 Thread Ted Yu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18057?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16497070#comment-16497070 ] Ted Yu commented on SPARK-18057: I tend to agree with Cody. Just wondering if other people would accept

[jira] [Commented] (SPARK-18057) Update structured streaming kafka from 0.10.0.1 to 1.1.0

2018-05-30 Thread Ted Yu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18057?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16495580#comment-16495580 ] Ted Yu commented on SPARK-18057: There are compilation errors in KafkaTestUtils.scala against Kafka

[jira] [Commented] (SPARK-21569) Internal Spark class needs to be kryo-registered

2018-05-11 Thread Ted Yu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21569?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16472295#comment-16472295 ] Ted Yu commented on SPARK-21569: What would be workaround ? Thanks > Internal Spark class needs to be

[jira] [Commented] (SPARK-23714) Add metrics for cached KafkaConsumer

2018-03-27 Thread Ted Yu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23714?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16416138#comment-16416138 ] Ted Yu commented on SPARK-23714: [~tdas]: What do you think ? Thanks > Add metrics for cached

[jira] [Commented] (SPARK-23714) Add metrics for cached KafkaConsumer

2018-03-17 Thread Ted Yu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23714?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16403792#comment-16403792 ] Ted Yu commented on SPARK-23714: Ryan reminded me of the possibility of using the existing metrics

[jira] [Commented] (SPARK-23714) Add metrics for cached KafkaConsumer

2018-03-16 Thread Ted Yu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23714?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16403112#comment-16403112 ] Ted Yu commented on SPARK-23714: [~tdas] made the following suggestion: apache commons pool does expose

[jira] [Created] (SPARK-23714) Add metrics for cached KafkaConsumer

2018-03-16 Thread Ted Yu (JIRA)
Ted Yu created SPARK-23714: -- Summary: Add metrics for cached KafkaConsumer Key: SPARK-23714 URL: https://issues.apache.org/jira/browse/SPARK-23714 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-23347) Introduce buffer between Java data stream and gzip stream

2018-02-06 Thread Ted Yu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23347?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16354939#comment-16354939 ] Ted Yu commented on SPARK-23347: {code} public final byte[] serialize(Object o) throws Exception {

[jira] [Commented] (SPARK-23347) Introduce buffer between Java data stream and gzip stream

2018-02-06 Thread Ted Yu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23347?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16354907#comment-16354907 ] Ted Yu commented on SPARK-23347: See JDK 1.8 code: {code} class DeflaterOutputStream { public void

[jira] [Created] (SPARK-23347) Introduce buffer between Java data stream and gzip stream

2018-02-06 Thread Ted Yu (JIRA)
Ted Yu created SPARK-23347: -- Summary: Introduce buffer between Java data stream and gzip stream Key: SPARK-23347 URL: https://issues.apache.org/jira/browse/SPARK-23347 Project: Spark Issue Type:

[jira] [Updated] (SPARK-10561) Provide tooling for auto-generating Spark SQL reference manual

2016-10-27 Thread Ted Yu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10561?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated SPARK-10561: --- Labels: tool (was: ) > Provide tooling for auto-generating Spark SQL reference manual >

[jira] [Commented] (SPARK-10868) monotonicallyIncreasingId() supports offset for indexing

2016-08-17 Thread Ted Yu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10868?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15425607#comment-15425607 ] Ted Yu commented on SPARK-10868: bq. this makes sense What was the reasoning behind the initial

[jira] [Commented] (SPARK-10868) monotonicallyIncreasingId() supports offset for indexing

2016-08-09 Thread Ted Yu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10868?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15414164#comment-15414164 ] Ted Yu commented on SPARK-10868: I can pick up this one if Martin hasn't started working on it. >

[jira] [Created] (SPARK-15273) YarnSparkHadoopUtil#getOutOfMemoryErrorArgument should respect OnOutOfMemoryError parameter given by user

2016-05-11 Thread Ted Yu (JIRA)
Ted Yu created SPARK-15273: -- Summary: YarnSparkHadoopUtil#getOutOfMemoryErrorArgument should respect OnOutOfMemoryError parameter given by user Key: SPARK-15273 URL: https://issues.apache.org/jira/browse/SPARK-15273

[jira] [Created] (SPARK-15003) Use ConcurrentHashMap in place of HashMap for NewAccumulator.originals

2016-04-29 Thread Ted Yu (JIRA)
Ted Yu created SPARK-15003: -- Summary: Use ConcurrentHashMap in place of HashMap for NewAccumulator.originals Key: SPARK-15003 URL: https://issues.apache.org/jira/browse/SPARK-15003 Project: Spark

[jira] [Created] (SPARK-14448) Improvements to ColumnVector

2016-04-06 Thread Ted Yu (JIRA)
Ted Yu created SPARK-14448: -- Summary: Improvements to ColumnVector Key: SPARK-14448 URL: https://issues.apache.org/jira/browse/SPARK-14448 Project: Spark Issue Type: Improvement

[jira] [Created] (SPARK-13663) Upgrade Snappy Java to 1.1.2.1

2016-03-03 Thread Ted Yu (JIRA)
Ted Yu created SPARK-13663: -- Summary: Upgrade Snappy Java to 1.1.2.1 Key: SPARK-13663 URL: https://issues.apache.org/jira/browse/SPARK-13663 Project: Spark Issue Type: Improvement

[jira] [Updated] (SPARK-10561) Provide tooling for auto-generating Spark SQL reference manual

2016-02-21 Thread Ted Yu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10561?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated SPARK-10561: --- Description: Here is the discussion thread: http://search-hadoop.com/m/q3RTtcD20F1o62xE Richard Hillegas

[jira] [Updated] (SPARK-10561) Provide tooling for auto-generating Spark SQL reference manual

2016-02-08 Thread Ted Yu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10561?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated SPARK-10561: --- Description: Here is the discussion thread: http://search-hadoop.com/m/q3RTtcD20F1o62xE Richard Hillegas

[jira] [Commented] (SPARK-13180) Protect against SessionState being null when accessing HiveClientImpl#conf

2016-02-08 Thread Ted Yu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15137674#comment-15137674 ] Ted Yu commented on SPARK-13180: I wonder if we should provide better error message when NPE happens -

[jira] [Commented] (SPARK-13120) Shade protobuf-java

2016-02-07 Thread Ted Yu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15136320#comment-15136320 ] Ted Yu commented on SPARK-13120: I do see the advantage of shading protobuf-java which would make user

[jira] [Updated] (SPARK-13120) Shade protobuf-java

2016-02-07 Thread Ted Yu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13120?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated SPARK-13120: --- Description: https://groups.google.com/forum/#!topic/protobuf/wAqvtPLBsE8 PB2 and PB3 are wire compatible,

[jira] [Commented] (SPARK-13120) Shade protobuf-java

2016-02-07 Thread Ted Yu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15136383#comment-15136383 ] Ted Yu commented on SPARK-13120: I don't think this JIRA is limited to the scenario from the thread

[jira] [Commented] (SPARK-13171) Update promise & future to Promise and Future as the old ones are deprecated

2016-02-06 Thread Ted Yu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13171?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15136006#comment-15136006 ] Ted Yu commented on SPARK-13171:

[jira] [Commented] (SPARK-13171) Update promise & future to Promise and Future as the old ones are deprecated

2016-02-06 Thread Ted Yu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13171?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15136004#comment-15136004 ] Ted Yu commented on SPARK-13171: No, I have not. bq. Could it be that Hive does some funky reflection

[jira] [Commented] (SPARK-13171) Update promise & future to Promise and Future as the old ones are deprecated

2016-02-06 Thread Ted Yu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13171?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15135972#comment-15135972 ] Ted Yu commented on SPARK-13171: The above Jenkins jobs were running Scala 2.11 > Update promise &

[jira] [Commented] (SPARK-13171) Update promise & future to Promise and Future as the old ones are deprecated

2016-02-06 Thread Ted Yu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13171?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15135784#comment-15135784 ] Ted Yu commented on SPARK-13171: Turns out not that trivial. For build against hadoop 2.4 , some test

[jira] [Updated] (SPARK-13180) Protect against SessionState being null when accessing HiveClientImpl#conf

2016-02-05 Thread Ted Yu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13180?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated SPARK-13180: --- Attachment: spark-13180-util.patch Patch with HiveConfUtil.scala For record. > Protect against

[jira] [Commented] (SPARK-13204) Replace use of mutable.SynchronizedMap with ConcurrentHashMap

2016-02-05 Thread Ted Yu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13204?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15134006#comment-15134006 ] Ted Yu commented on SPARK-13204: Just noticed that parent JIRA was marked trivial. Should have done the

[jira] [Created] (SPARK-13204) Replace use of mutable.SynchronizedMap with ConcurrentHashMap

2016-02-04 Thread Ted Yu (JIRA)
Ted Yu created SPARK-13204: -- Summary: Replace use of mutable.SynchronizedMap with ConcurrentHashMap Key: SPARK-13204 URL: https://issues.apache.org/jira/browse/SPARK-13204 Project: Spark Issue

[jira] [Created] (SPARK-13203) Add scalastyle rule banning use of mutable.SynchronizedBuffer

2016-02-04 Thread Ted Yu (JIRA)
Ted Yu created SPARK-13203: -- Summary: Add scalastyle rule banning use of mutable.SynchronizedBuffer Key: SPARK-13203 URL: https://issues.apache.org/jira/browse/SPARK-13203 Project: Spark Issue

[jira] [Created] (SPARK-13199) Upgrade apache httpclient version to the latest 4.5 for security

2016-02-04 Thread Ted Yu (JIRA)
Ted Yu created SPARK-13199: -- Summary: Upgrade apache httpclient version to the latest 4.5 for security Key: SPARK-13199 URL: https://issues.apache.org/jira/browse/SPARK-13199 Project: Spark Issue

[jira] [Created] (SPARK-13180) Protect against SessionState being null when accessing HiveClientImpl#conf

2016-02-03 Thread Ted Yu (JIRA)
Ted Yu created SPARK-13180: -- Summary: Protect against SessionState being null when accessing HiveClientImpl#conf Key: SPARK-13180 URL: https://issues.apache.org/jira/browse/SPARK-13180 Project: Spark

[jira] [Commented] (SPARK-13120) Shade protobuf-java

2016-02-02 Thread Ted Yu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15128759#comment-15128759 ] Ted Yu commented on SPARK-13120: https://groups.google.com/forum/#!topic/protobuf/wAqvtPLBsE8 PB2 and

[jira] [Created] (SPARK-13120) Shade protobuf-java

2016-02-01 Thread Ted Yu (JIRA)
Ted Yu created SPARK-13120: -- Summary: Shade protobuf-java Key: SPARK-13120 URL: https://issues.apache.org/jira/browse/SPARK-13120 Project: Spark Issue Type: Improvement Components: Build

[jira] [Resolved] (SPARK-13084) Utilize @SerialVersionUID to avoid local class incompatibility

2016-02-01 Thread Ted Yu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13084?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu resolved SPARK-13084. Resolution: Won't Fix > Utilize @SerialVersionUID to avoid local class incompatibility >

[jira] [Created] (SPARK-13084) Utilize @SerialVersionUID to avoid local class incompatibility

2016-01-29 Thread Ted Yu (JIRA)
Ted Yu created SPARK-13084: -- Summary: Utilize @SerialVersionUID to avoid local class incompatibility Key: SPARK-13084 URL: https://issues.apache.org/jira/browse/SPARK-13084 Project: Spark Issue

[jira] [Updated] (SPARK-13084) Utilize @SerialVersionUID to avoid local class incompatibility

2016-01-29 Thread Ted Yu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13084?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated SPARK-13084: --- Description: Here is related thread:

[jira] [Updated] (SPARK-13084) Utilize @SerialVersionUID to avoid local class incompatibility

2016-01-29 Thread Ted Yu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13084?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated SPARK-13084: --- Component/s: Spark Core > Utilize @SerialVersionUID to avoid local class incompatibility >

[jira] [Resolved] (SPARK-13022) Shade jackson core

2016-01-27 Thread Ted Yu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13022?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu resolved SPARK-13022. Resolution: Won't Fix > Shade jackson core > -- > > Key: SPARK-13022 >

[jira] [Created] (SPARK-13022) Shade jackson core

2016-01-26 Thread Ted Yu (JIRA)
Ted Yu created SPARK-13022: -- Summary: Shade jackson core Key: SPARK-13022 URL: https://issues.apache.org/jira/browse/SPARK-13022 Project: Spark Issue Type: Improvement Components: Deploy

[jira] [Created] (SPARK-12778) Use of Java Unsafe should take endianness into account

2016-01-12 Thread Ted Yu (JIRA)
Ted Yu created SPARK-12778: -- Summary: Use of Java Unsafe should take endianness into account Key: SPARK-12778 URL: https://issues.apache.org/jira/browse/SPARK-12778 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-12778) Use of Java Unsafe should take endianness into account

2016-01-12 Thread Ted Yu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12778?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15094078#comment-15094078 ] Ted Yu commented on SPARK-12778: I did perform a search in JIRA for Unsafe related Spark JIRAs but didn't

[jira] [Updated] (SPARK-12778) Use of Java Unsafe should take endianness into account

2016-01-12 Thread Ted Yu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12778?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated SPARK-12778: --- Component/s: Input/Output > Use of Java Unsafe should take endianness into account >

[jira] [Comment Edited] (SPARK-12778) Use of Java Unsafe should take endianness into account

2016-01-12 Thread Ted Yu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12778?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15094078#comment-15094078 ] Ted Yu edited comment on SPARK-12778 at 1/12/16 3:33 PM: - I did perform a search

[jira] [Updated] (SPARK-12778) Use of Java Unsafe should take endianness into account

2016-01-12 Thread Ted Yu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12778?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated SPARK-12778: --- Description: In Platform.java, methods of Java Unsafe are called directly without considering endianness.

[jira] [Updated] (SPARK-12778) Use of Java Unsafe should take endianness into account

2016-01-12 Thread Ted Yu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12778?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated SPARK-12778: --- Description: In Platform.java, methods of Java Unsafe are called directly without considering endianness.

[jira] [Commented] (SPARK-12778) Use of Java Unsafe should take endianness into account

2016-01-12 Thread Ted Yu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12778?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15094090#comment-15094090 ] Ted Yu commented on SPARK-12778: I have seen SPARK-12555 but it didn't identify where in the codebase the

[jira] [Resolved] (SPARK-4066) Make whether maven builds fails on scalastyle violation configurable

2016-01-07 Thread Ted Yu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4066?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu resolved SPARK-4066. --- Resolution: Later Haven't heard of much feedback so far. Resolving for now. > Make whether maven builds

[jira] [Resolved] (SPARK-12527) Add private val after @transient for kinesis-asl module

2015-12-26 Thread Ted Yu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12527?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu resolved SPARK-12527. Resolution: Duplicate Dup of SPARK-1252 > Add private val after @transient for kinesis-asl module >

[jira] [Issue Comment Deleted] (SPARK-12527) Add private val after @transient for kinesis-asl module

2015-12-25 Thread Ted Yu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12527?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated SPARK-12527: --- Comment: was deleted (was: https://github.com/apache/spark/pull/10482) > Add private val after @transient

[jira] [Created] (SPARK-12527) Add private val after @transient for kinesis-asl module

2015-12-25 Thread Ted Yu (JIRA)
Ted Yu created SPARK-12527: -- Summary: Add private val after @transient for kinesis-asl module Key: SPARK-12527 URL: https://issues.apache.org/jira/browse/SPARK-12527 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-12527) Add private val after @transient for kinesis-asl module

2015-12-25 Thread Ted Yu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12527?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated SPARK-12527: --- Description: In SBT build using Scala 2.11, the following warnings were reported which resulted in build

[jira] [Commented] (SPARK-12527) Add private val after @transient for kinesis-asl module

2015-12-25 Thread Ted Yu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12527?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15071818#comment-15071818 ] Ted Yu commented on SPARK-12527: https://github.com/apache/spark/pull/10482 > Add private val after

[jira] [Created] (SPARK-12365) Use ShutdownHookManager where Runtime.getRuntime.addShutdownHook() is called

2015-12-16 Thread Ted Yu (JIRA)
Ted Yu created SPARK-12365: -- Summary: Use ShutdownHookManager where Runtime.getRuntime.addShutdownHook() is called Key: SPARK-12365 URL: https://issues.apache.org/jira/browse/SPARK-12365 Project: Spark

[jira] [Created] (SPARK-12181) Check Cached unaligned-access capability before using Unsafe

2015-12-07 Thread Ted Yu (JIRA)
Ted Yu created SPARK-12181: -- Summary: Check Cached unaligned-access capability before using Unsafe Key: SPARK-12181 URL: https://issues.apache.org/jira/browse/SPARK-12181 Project: Spark Issue

[jira] [Commented] (SPARK-12181) Check Cached unaligned-access capability before using Unsafe

2015-12-07 Thread Ted Yu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15045309#comment-15045309 ] Ted Yu commented on SPARK-12181: We can add method to Platform which performs check on the architecture.

[jira] [Updated] (SPARK-12181) Check Cached unaligned-access capability before using Unsafe

2015-12-07 Thread Ted Yu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12181?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated SPARK-12181: --- Attachment: spark-12181.txt > Check Cached unaligned-access capability before using Unsafe >

  1   2   3   >