[jira] [Resolved] (SPARK-29095) add extractInstances

2019-09-24 Thread Sean Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29095?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-29095. --- Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 25802

[jira] [Assigned] (SPARK-29095) add extractInstances

2019-09-24 Thread Sean Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29095?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen reassigned SPARK-29095: - Assignee: zhengruifeng > add extractInstances > > > Key:

[jira] [Assigned] (SPARK-26848) Introduce new option to Kafka source - specify timestamp to start and end offset

2019-09-23 Thread Sean Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-26848?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen reassigned SPARK-26848: - Assignee: Jungtaek Lim > Introduce new option to Kafka source - specify timestamp to start and

[jira] [Resolved] (SPARK-26848) Introduce new option to Kafka source - specify timestamp to start and end offset

2019-09-23 Thread Sean Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-26848?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-26848. --- Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 23747

[jira] [Updated] (SPARK-29053) Sort does not work on some columns

2019-09-23 Thread Sean Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29053?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-29053: -- Fix Version/s: 2.4.5 > Sort does not work on some columns > -- > >

[jira] [Commented] (SPARK-29204) Remove `Spark Release` Jenkins tab and its four jobs

2019-09-22 Thread Sean Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29204?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16935432#comment-16935432 ] Sean Owen commented on SPARK-29204: --- Is this just a matter of deleting the view? I have permissions to

[jira] [Resolved] (SPARK-29121) Support Dot Product for Vectors

2019-09-21 Thread Sean Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29121?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-29121. --- Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 25818

[jira] [Assigned] (SPARK-29121) Support Dot Product for Vectors

2019-09-21 Thread Sean Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29121?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen reassigned SPARK-29121: - Assignee: Patrick Pisciuneri > Support Dot Product for Vectors >

[jira] [Resolved] (SPARK-29053) Sort does not work on some columns

2019-09-21 Thread Sean Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29053?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-29053. --- Fix Version/s: 3.0.0 Assignee: Aman Omer Resolution: Fixed Resolved by

[jira] [Updated] (SPARK-19147) netty throw NPE

2019-09-21 Thread Sean Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-19147?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-19147: -- Issue Type: Improvement (was: Bug) Priority: Minor (was: Major) > netty throw NPE >

[jira] [Resolved] (SPARK-19147) netty throw NPE

2019-09-21 Thread Sean Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-19147?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-19147. --- Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 25759

[jira] [Assigned] (SPARK-19147) netty throw NPE

2019-09-21 Thread Sean Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-19147?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen reassigned SPARK-19147: - Assignee: Colin Ma > netty throw NPE > --- > > Key: SPARK-19147 >

[jira] [Assigned] (SPARK-29144) Binarizer handle sparse vectors incorrectly with negative threshold

2019-09-20 Thread Sean Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29144?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen reassigned SPARK-29144: - Assignee: zhengruifeng > Binarizer handle sparse vectors incorrectly with negative threshold >

[jira] [Resolved] (SPARK-29144) Binarizer handle sparse vectors incorrectly with negative threshold

2019-09-20 Thread Sean Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29144?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-29144. --- Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 25829

[jira] [Commented] (SPARK-28772) Upgrade breeze to 1.0

2019-09-20 Thread Sean Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28772?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16934806#comment-16934806 ] Sean Owen commented on SPARK-28772: --- Wait, no, I've made a mistake, misread this. We do have to do

[jira] [Updated] (SPARK-28772) Upgrade breeze to 1.0

2019-09-20 Thread Sean Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28772?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-28772: -- Parent: SPARK-25075 Issue Type: Sub-task (was: Task) > Upgrade breeze to 1.0 >

[jira] [Updated] (SPARK-28772) Upgrade breeze to 1.0

2019-09-20 Thread Sean Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28772?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-28772: -- Parent: (was: SPARK-25075) Issue Type: Task (was: Sub-task) > Upgrade breeze to 1.0 >

[jira] [Commented] (SPARK-28772) Upgrade breeze to 1.0

2019-09-20 Thread Sean Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28772?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16934804#comment-16934804 ] Sean Owen commented on SPARK-28772: --- We should do this, and I'll open a PR, but it is not required for

[jira] [Commented] (SPARK-29129) Test failure: org.apache.spark.sql.hive.JavaDataFrameSuite (hadoop-2.7/JDK 11 combination)

2019-09-20 Thread Sean Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29129?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16934581#comment-16934581 ] Sean Owen commented on SPARK-29129: --- Oh, heh, we have a Hadoop 2 + JDK 11 build? I haven't been paying

[jira] [Updated] (SPARK-29082) Spark driver cannot start with only delegation tokens

2019-09-20 Thread Sean Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29082?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-29082: -- Fix Version/s: (was: 3.0.0) > Spark driver cannot start with only delegation tokens >

[jira] [Reopened] (SPARK-29082) Spark driver cannot start with only delegation tokens

2019-09-20 Thread Sean Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29082?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen reopened SPARK-29082: --- Reopened because we had to revert it for now > Spark driver cannot start with only delegation tokens >

[jira] [Resolved] (SPARK-26338) Use scala-xml explicitly

2019-09-20 Thread Sean Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-26338?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-26338. --- Resolution: Invalid Reopen if there is any detail here > Use scala-xml explicitly >

[jira] [Commented] (SPARK-28900) Test Pyspark, SparkR on JDK 11 with run-tests

2019-09-19 Thread Sean Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28900?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16933973#comment-16933973 ] Sean Owen commented on SPARK-28900: --- The meaning of JIRAs and umbrellas is already kind of nebulous,

[jira] [Resolved] (SPARK-29172) Fix some exception issue of explain commands

2019-09-19 Thread Sean Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29172?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-29172. --- Resolution: Not A Problem > Fix some exception issue of explain commands >

[jira] [Commented] (SPARK-28900) Test Pyspark, SparkR on JDK 11 with run-tests

2019-09-19 Thread Sean Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28900?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16933892#comment-16933892 ] Sean Owen commented on SPARK-28900: --- Meh, I think it's kind of important to declare JDK 11 support

[jira] [Commented] (SPARK-29183) Upgrade JDK 11 Installation to 11.0.4

2019-09-19 Thread Sean Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29183?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16933885#comment-16933885 ] Sean Owen commented on SPARK-29183: --- I fully support Shane doing all the work to get this done! >

[jira] [Commented] (SPARK-24417) Build and Run Spark on JDK11

2019-09-19 Thread Sean Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-24417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16933700#comment-16933700 ] Sean Owen commented on SPARK-24417: --- I don't care at all, myself, as only a minor contributor. The

[jira] [Resolved] (SPARK-28985) Pyspark ClassificationModel and RegressionModel support column setters/getters/predict

2019-09-19 Thread Sean Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28985?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-28985. --- Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 25776

[jira] [Assigned] (SPARK-28985) Pyspark ClassificationModel and RegressionModel support column setters/getters/predict

2019-09-19 Thread Sean Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28985?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen reassigned SPARK-28985: - Assignee: Huaxin Gao > Pyspark ClassificationModel and RegressionModel support column >

[jira] [Resolved] (SPARK-29118) Avoid redundant computation in GMM.transform && GLR.transform

2019-09-18 Thread Sean Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29118?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-29118. --- Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 25815

[jira] [Assigned] (SPARK-29118) Avoid redundant computation in GMM.transform && GLR.transform

2019-09-18 Thread Sean Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29118?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen reassigned SPARK-29118: - Assignee: zhengruifeng > Avoid redundant computation in GMM.transform && GLR.transform >

[jira] [Resolved] (SPARK-28927) Improve error for ArrayIndexOutOfBoundsException and Not-stable AUC metrics in ALS for datasets with 12 billion instances

2019-09-18 Thread Sean Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28927?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-28927. --- Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 25789

[jira] [Updated] (SPARK-28927) Improve error for ArrayIndexOutOfBoundsException and Not-stable AUC metrics in ALS for datasets with 12 billion instances

2019-09-18 Thread Sean Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28927?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-28927: -- Issue Type: Improvement (was: Bug) Priority: Minor (was: Major) Summary: Improve error

[jira] [Resolved] (SPARK-28842) Cleanup the formatting/trailing spaces in resource-managers/kubernetes/integration-tests/README.md

2019-09-18 Thread Sean Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28842?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-28842. --- Resolution: Not A Problem > Cleanup the formatting/trailing spaces in >

[jira] [Resolved] (SPARK-28972) [Spark] spark.memory.offHeap.size description require to update in document

2019-09-18 Thread Sean Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28972?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-28972. --- Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 25689

[jira] [Updated] (SPARK-28972) [Spark] spark.memory.offHeap.size description require to update in document

2019-09-18 Thread Sean Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28972?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-28972: -- Issue Type: Improvement (was: Bug) > [Spark] spark.memory.offHeap.size description require to update

[jira] [Assigned] (SPARK-28972) [Spark] spark.memory.offHeap.size description require to update in document

2019-09-18 Thread Sean Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28972?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen reassigned SPARK-28972: - Assignee: pavithra ramachandran > [Spark] spark.memory.offHeap.size description require to

[jira] [Assigned] (SPARK-28799) Document TRUNCATE TABLE in SQL Reference.

2019-09-18 Thread Sean Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28799?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen reassigned SPARK-28799: - Assignee: pavithra ramachandran > Document TRUNCATE TABLE in SQL Reference. >

[jira] [Updated] (SPARK-28799) Document TRUNCATE TABLE in SQL Reference.

2019-09-18 Thread Sean Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28799?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-28799: -- Priority: Minor (was: Major) > Document TRUNCATE TABLE in SQL Reference. >

[jira] [Resolved] (SPARK-28799) Document TRUNCATE TABLE in SQL Reference.

2019-09-18 Thread Sean Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28799?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-28799. --- Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 25557

[jira] [Commented] (SPARK-29051) Spark Application UI search is not working for some fields

2019-09-17 Thread Sean Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16931530#comment-16931530 ] Sean Owen commented on SPARK-29051: --- I don't know, I think it's because the text is formatted in

[jira] [Updated] (SPARK-28929) Spark Logging level should be INFO instead of Debug in Executor Plugin API[SPARK-24918]

2019-09-17 Thread Sean Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28929?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-28929: -- Priority: Trivial (was: Minor) > Spark Logging level should be INFO instead of Debug in Executor

[jira] [Updated] (SPARK-28264) Revisiting Python / pandas UDF

2019-09-17 Thread Sean Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28264?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-28264: -- Priority: Critical (was: Major) > Revisiting Python / pandas UDF > -- >

[jira] [Resolved] (SPARK-26565) modify dev/create-release/release-build.sh to let jenkins build packages w/o publishing

2019-09-16 Thread Sean Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-26565?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-26565. --- Resolution: Not A Problem > modify dev/create-release/release-build.sh to let jenkins build

[jira] [Resolved] (SPARK-22381) Add StringParam that supports valid options

2019-09-16 Thread Sean Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-22381?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-22381. --- Resolution: Won't Fix > Add StringParam that supports valid options >

[jira] [Resolved] (SPARK-10408) Autoencoder

2019-09-16 Thread Sean Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-10408?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-10408. --- Resolution: Won't Fix > Autoencoder > --- > > Key: SPARK-10408 >

[jira] [Resolved] (SPARK-22111) OnlineLDAOptimizer should filter out empty documents beforehand

2019-09-16 Thread Sean Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-22111?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-22111. --- Resolution: Won't Fix > OnlineLDAOptimizer should filter out empty documents beforehand >

[jira] [Resolved] (SPARK-24806) Brush up generated code so that JDK Java compilers can handle it

2019-09-16 Thread Sean Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-24806?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-24806. --- Resolution: Won't Fix > Brush up generated code so that JDK Java compilers can handle it >

[jira] [Resolved] (SPARK-23694) The staging directory should under hive.exec.stagingdir if we set hive.exec.stagingdir but not under the table directory

2019-09-16 Thread Sean Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-23694?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-23694. --- Resolution: Won't Fix > The staging directory should under hive.exec.stagingdir if we set >

[jira] [Resolved] (SPARK-24671) DataFrame length using a dunder/magic method in PySpark

2019-09-16 Thread Sean Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-24671?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-24671. --- Resolution: Won't Fix > DataFrame length using a dunder/magic method in PySpark >

[jira] [Resolved] (SPARK-19184) Improve numerical stability for method tallSkinnyQR.

2019-09-16 Thread Sean Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-19184?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-19184. --- Resolution: Won't Fix > Improve numerical stability for method tallSkinnyQR. >

[jira] [Resolved] (SPARK-26524) If the application directory fails to be created on the SPARK_WORKER_DIR on some woker nodes (for example, bad disk or disk has no capacity), the application executor w

2019-09-16 Thread Sean Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-26524?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-26524. --- Resolution: Won't Fix > If the application directory fails to be created on the SPARK_WORKER_DIR on

[jira] [Resolved] (SPARK-23539) Add support for Kafka headers in Structured Streaming

2019-09-13 Thread Sean Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-23539?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-23539. --- Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 22282

[jira] [Assigned] (SPARK-23539) Add support for Kafka headers in Structured Streaming

2019-09-13 Thread Sean Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-23539?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen reassigned SPARK-23539: - Assignee: Lee Dongjin > Add support for Kafka headers in Structured Streaming >

[jira] [Assigned] (SPARK-28969) OneVsRestModel in the py side should not set WeightCol and Classifier

2019-09-13 Thread Sean Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28969?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen reassigned SPARK-28969: - Assignee: Huaxin Gao > OneVsRestModel in the py side should not set WeightCol and Classifier >

[jira] [Resolved] (SPARK-28969) OneVsRestModel in the py side should not set WeightCol and Classifier

2019-09-13 Thread Sean Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28969?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-28969. --- Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 25715

[jira] [Commented] (SPARK-22796) Add multiple column support to PySpark QuantileDiscretizer

2019-09-12 Thread Sean Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-22796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16928541#comment-16928541 ] Sean Owen commented on SPARK-22796: --- Yes, but you can see that it was reverted. There are pointers to

[jira] [Resolved] (SPARK-26731) remove EOLed spark jobs from jenkins

2019-09-12 Thread Sean Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-26731?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-26731. --- Target Version/s: (was: 3.0.0) Resolution: Duplicate > remove EOLed spark jobs from

[jira] [Commented] (SPARK-29066) Remove old Jenkins jobs for EOL versions or obsolete combinations

2019-09-12 Thread Sean Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29066?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16928539#comment-16928539 ] Sean Owen commented on SPARK-29066: --- Sounds good. This is the same as

[jira] [Updated] (SPARK-29050) Fix typo in some docs

2019-09-11 Thread Sean Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29050?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-29050: -- Issue Type: Improvement (was: Bug) Priority: Trivial (was: Major) This can't be considered a

[jira] [Updated] (SPARK-28987) DiskBlockManager#createTempShuffleBlock should skip directory which is read-only

2019-09-11 Thread Sean Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28987?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-28987: -- Priority: Minor (was: Major) > DiskBlockManager#createTempShuffleBlock should skip directory which

[jira] [Resolved] (SPARK-28987) DiskBlockManager#createTempShuffleBlock should skip directory which is read-only

2019-09-11 Thread Sean Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28987?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-28987. --- Resolution: Won't Fix > DiskBlockManager#createTempShuffleBlock should skip directory which is >

[jira] [Resolved] (SPARK-28906) `bin/spark-submit --version` shows incorrect info

2019-09-11 Thread Sean Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28906?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-28906. --- Fix Version/s: 3.0.0 2.4.5 Resolution: Fixed Issue resolved by pull

[jira] [Assigned] (SPARK-28906) `bin/spark-submit --version` shows incorrect info

2019-09-11 Thread Sean Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28906?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen reassigned SPARK-28906: - Assignee: Kazuaki Ishizaki > `bin/spark-submit --version` shows incorrect info >

[jira] [Resolved] (SPARK-29028) Add links to IBM Cloud Object Storage connector in cloud-integration.md

2019-09-10 Thread Sean Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29028?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-29028. --- Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 25737

[jira] [Assigned] (SPARK-29028) Add links to IBM Cloud Object Storage connector in cloud-integration.md

2019-09-10 Thread Sean Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29028?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen reassigned SPARK-29028: - Assignee: Dilip Biswal > Add links to IBM Cloud Object Storage connector in

[jira] [Updated] (SPARK-28657) Fix currentContext Instance failed sometimes

2019-09-09 Thread Sean Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28657?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-28657: -- Fix Version/s: (was: 2.4.5) > Fix currentContext Instance failed sometimes >

[jira] [Resolved] (SPARK-28657) Fix currentContext Instance failed sometimes

2019-09-09 Thread Sean Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28657?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-28657. --- Fix Version/s: 3.0.0 2.4.5 Resolution: Fixed Issue resolved by pull

[jira] [Assigned] (SPARK-28657) Fix currentContext Instance failed sometimes

2019-09-09 Thread Sean Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28657?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen reassigned SPARK-28657: - Assignee: hong dongdong > Fix currentContext Instance failed sometimes >

[jira] [Updated] (SPARK-29011) Upgrade netty-all to 4.1.39-Final

2019-09-09 Thread Sean Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29011?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-29011: -- Fix Version/s: 2.4.5 > Upgrade netty-all to 4.1.39-Final > - > >

[jira] [Updated] (SPARK-14098) Generate Java code to build CachedColumnarBatch and get values from CachedColumnarBatch when DataFrame.cache() is called

2019-09-09 Thread Sean Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-14098?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-14098: -- Labels: release-notes (was: releasenotes) > Generate Java code to build CachedColumnarBatch and get

[jira] [Resolved] (SPARK-14643) Remove overloaded methods which become ambiguous in Scala 2.12

2019-09-09 Thread Sean Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-14643?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-14643. --- Target Version/s: (was: 3.0.0) Resolution: Won't Fix Reopen if anyone has different

[jira] [Resolved] (SPARK-14098) Generate Java code to build CachedColumnarBatch and get values from CachedColumnarBatch when DataFrame.cache() is called

2019-09-09 Thread Sean Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-14098?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-14098. --- Target Version/s: (was: 3.0.0) Resolution: Done > Generate Java code to build

[jira] [Resolved] (SPARK-16275) Implement all the Hive fallback functions

2019-09-09 Thread Sean Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-16275?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-16275. --- Target Version/s: (was: 3.0.0) Resolution: Done > Implement all the Hive fallback

[jira] [Resolved] (SPARK-16390) Dataset API improvements

2019-09-09 Thread Sean Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-16390?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-16390. --- Target Version/s: (was: 3.0.0) Resolution: Done > Dataset API improvements >

[jira] [Resolved] (SPARK-18455) General support for correlated subquery processing

2019-09-09 Thread Sean Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-18455?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-18455. --- Target Version/s: (was: 3.0.0) Resolution: Done Looks like the subtasks are done? >

[jira] [Updated] (SPARK-22236) CSV I/O: does not respect RFC 4180

2019-09-09 Thread Sean Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-22236?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-22236: -- Target Version/s: (was: 3.0.0) > CSV I/O: does not respect RFC 4180 >

[jira] [Resolved] (SPARK-25261) Standardize the default units of spark.driver|executor.memory

2019-09-09 Thread Sean Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-25261?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-25261. --- Fix Version/s: 2.4.0 Target Version/s: (was: 3.0.0) Assignee: huangtengfei

[jira] [Updated] (SPARK-25157) Streaming of image files from directory

2019-09-09 Thread Sean Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-25157?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-25157: -- Target Version/s: (was: 3.0.0) > Streaming of image files from directory >

[jira] [Resolved] (SPARK-25783) Spark shell fails because of jline incompatibility

2019-09-09 Thread Sean Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-25783?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-25783. --- Target Version/s: (was: 3.0.0) Resolution: Duplicate > Spark shell fails because of

[jira] [Updated] (SPARK-26247) SPIP - ML Model Extension for no-Spark MLLib Online Serving

2019-09-09 Thread Sean Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-26247?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-26247: -- Target Version/s: (was: 3.0.0) > SPIP - ML Model Extension for no-Spark MLLib Online Serving >

[jira] [Updated] (SPARK-28227) Spark can’t support TRANSFORM with aggregation

2019-09-09 Thread Sean Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28227?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-28227: -- Target Version/s: (was: 3.0.0) > Spark can’t support TRANSFORM with aggregation >

[jira] [Updated] (SPARK-29020) Unifying behaviour between array_sort and sort_array

2019-09-09 Thread Sean Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29020?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-29020: -- Target Version/s: (was: 2.4.5, 3.0.0) Labels: (was: functions sql) > Unifying

[jira] [Resolved] (SPARK-28980) Remove most remaining deprecated items since <= 2.2.0 for 3.0

2019-09-09 Thread Sean Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28980?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-28980. --- Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 25684

[jira] [Updated] (SPARK-28980) Remove most remaining deprecated items since <= 2.2.0 for 3.0

2019-09-09 Thread Sean Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28980?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-28980: -- Description: Following on https://issues.apache.org/jira/browse/SPARK-25908 I'd like to propose

[jira] [Updated] (SPARK-28969) OneVsRestModel in the py side should not set WeightCol and Classifier

2019-09-09 Thread Sean Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28969?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-28969: -- Docs Text: The setClassifier method in Pyspark's OneVsRestModel has been removed in 3.0 for parity

[jira] [Resolved] (SPARK-28340) Noisy exceptions when tasks are killed: "DiskBlockObjectWriter: Uncaught exception while reverting partial writes to file: java.nio.channels.ClosedByInterruptException"

2019-09-09 Thread Sean Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28340?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-28340. --- Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 25674

[jira] [Assigned] (SPARK-28340) Noisy exceptions when tasks are killed: "DiskBlockObjectWriter: Uncaught exception while reverting partial writes to file: java.nio.channels.ClosedByInterruptException"

2019-09-09 Thread Sean Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28340?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen reassigned SPARK-28340: - Assignee: Colin Ma > Noisy exceptions when tasks are killed: "DiskBlockObjectWriter: Uncaught

[jira] [Resolved] (SPARK-28884) [Spark]Spark driver cores is showing 0 instead of 1 in UI for cluster mode deployment

2019-09-09 Thread Sean Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28884?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-28884. --- Resolution: Not A Problem > [Spark]Spark driver cores is showing 0 instead of 1 in UI for cluster

[jira] [Resolved] (SPARK-28886) Kubernetes DepsTestsSuite fails on OSX with minikube 1.3.1 due to formatting

2019-09-08 Thread Sean Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28886?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-28886. --- Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 25599

[jira] [Resolved] (SPARK-28953) Integration tests fail due to malformed URL

2019-09-08 Thread Sean Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28953?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-28953. --- Resolution: Duplicate > Integration tests fail due to malformed URL >

[jira] [Resolved] (SPARK-27420) KinesisInputDStream should expose a way to configure CloudWatch metrics

2019-09-08 Thread Sean Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-27420?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-27420. --- Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 24651

[jira] [Assigned] (SPARK-27420) KinesisInputDStream should expose a way to configure CloudWatch metrics

2019-09-08 Thread Sean Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-27420?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen reassigned SPARK-27420: - Assignee: Kengo Seki > KinesisInputDStream should expose a way to configure CloudWatch metrics

[jira] [Updated] (SPARK-28942) [Spark][WEB UI]Spark in local mode hostname display localhost in the Host Column of Task Summary Page

2019-09-08 Thread Sean Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28942?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-28942: -- Issue Type: Improvement (was: Bug) > [Spark][WEB UI]Spark in local mode hostname display localhost

[jira] [Resolved] (SPARK-28942) [Spark][WEB UI]Spark in local mode hostname display localhost in the Host Column of Task Summary Page

2019-09-08 Thread Sean Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28942?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-28942. --- Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 25645

[jira] [Assigned] (SPARK-28942) [Spark][WEB UI]Spark in local mode hostname display localhost in the Host Column of Task Summary Page

2019-09-08 Thread Sean Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28942?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen reassigned SPARK-28942: - Assignee: Shivu Sondur > [Spark][WEB UI]Spark in local mode hostname display localhost in the

[jira] [Resolved] (SPARK-28967) ConcurrentModificationException is thrown from EventLoggingListener

2019-09-06 Thread Sean Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28967?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-28967. --- Fix Version/s: 3.0.0 Assignee: Jungtaek Lim Resolution: Fixed Resolved by

[jira] [Updated] (SPARK-28980) Remove most remaining deprecated items since <= 2.2.0 for 3.0

2019-09-05 Thread Sean Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28980?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-28980: -- Description: Following on https://issues.apache.org/jira/browse/SPARK-25908 I'd like to propose

[jira] [Commented] (SPARK-28981) Missing library for reading/writing Snappy-compressed files

2019-09-05 Thread Sean Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28981?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16923682#comment-16923682 ] Sean Owen commented on SPARK-28981: --- (Really we could say it's a duplicate of

[jira] [Updated] (SPARK-28977) JDBC Dataframe Reader Doc Doesn't Match JDBC Data Source Page

2019-09-05 Thread Sean Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28977?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-28977: -- Fix Version/s: 2.4.5 > JDBC Dataframe Reader Doc Doesn't Match JDBC Data Source Page >

  1   2   3   4   5   6   7   8   9   10   >