[jira] [Created] (SPARK-21189) Handle unknown error codes in Jenkins rather then leaving incomplete comment in PRs

2017-06-23 Thread Hyukjin Kwon (JIRA)
Hyukjin Kwon created SPARK-21189: Summary: Handle unknown error codes in Jenkins rather then leaving incomplete comment in PRs Key: SPARK-21189 URL: https://issues.apache.org/jira/browse/SPARK-21189 P

[jira] [Assigned] (SPARK-21189) Handle unknown error codes in Jenkins rather then leaving incomplete comment in PRs

2017-06-23 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21189?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21189: Assignee: (was: Apache Spark) > Handle unknown error codes in Jenkins rather then leav

[jira] [Assigned] (SPARK-21189) Handle unknown error codes in Jenkins rather then leaving incomplete comment in PRs

2017-06-23 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21189?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21189: Assignee: Apache Spark > Handle unknown error codes in Jenkins rather then leaving incompl

[jira] [Commented] (SPARK-21189) Handle unknown error codes in Jenkins rather then leaving incomplete comment in PRs

2017-06-23 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21189?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16060501#comment-16060501 ] Apache Spark commented on SPARK-21189: -- User 'HyukjinKwon' has created a pull reques

[jira] [Updated] (SPARK-21176) Master UI hangs with spark.ui.reverseProxy=true if the master node has many CPUs

2017-06-23 Thread Ingo Schuster (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21176?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ingo Schuster updated SPARK-21176: -- Description: In reverse proxy mode, Sparks exhausts the Jetty thread pool if the master node h

[jira] [Updated] (SPARK-21176) Master UI hangs with spark.ui.reverseProxy=true if the master node has many CPUs

2017-06-23 Thread Ingo Schuster (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21176?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ingo Schuster updated SPARK-21176: -- Description: In reverse proxy mode, Sparks exhausts the Jetty thread pool if the master node h

[jira] [Updated] (SPARK-21176) Master UI hangs with spark.ui.reverseProxy=true if the master node has many CPUs

2017-06-23 Thread Ingo Schuster (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21176?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ingo Schuster updated SPARK-21176: -- Description: In reverse proxy mode, Sparks exhausts the Jetty thread pool if the master node h

[jira] [Commented] (SPARK-21187) Complete support for remaining Spark data types in Arrow Converters

2017-06-23 Thread Bryan Cutler (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21187?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16060510#comment-16060510 ] Bryan Cutler commented on SPARK-21187: -- Pandas only supports flat columns, I'm not s

[jira] [Resolved] (SPARK-21145) Restarted queries reuse same StateStoreProvider, causing multiple concurrent tasks to update same StateStore

2017-06-23 Thread Tathagata Das (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21145?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das resolved SPARK-21145. --- Resolution: Fixed Fix Version/s: 2.3.0 Issue resolved by pull request 18355 [https://g

[jira] [Created] (SPARK-21190) SPIP: Vectorized UDFs for Python

2017-06-23 Thread Reynold Xin (JIRA)
Reynold Xin created SPARK-21190: --- Summary: SPIP: Vectorized UDFs for Python Key: SPARK-21190 URL: https://issues.apache.org/jira/browse/SPARK-21190 Project: Spark Issue Type: New Feature

[jira] [Updated] (SPARK-21190) SPIP: Vectorized UDFs for Python

2017-06-23 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21190?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-21190: Description: *Background and Motivation* Python is one of the most popular programming languages

[jira] [Assigned] (SPARK-21190) SPIP: Vectorized UDFs for Python

2017-06-23 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21190?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin reassigned SPARK-21190: --- Assignee: Reynold Xin > SPIP: Vectorized UDFs for Python >

[jira] [Updated] (SPARK-21190) SPIP: Vectorized UDFs in Python

2017-06-23 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21190?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-21190: Summary: SPIP: Vectorized UDFs in Python (was: SPIP: Vectorized UDFs for Python) > SPIP: Vectoriz

[jira] [Updated] (SPARK-21190) SPIP: Vectorized UDFs in Python

2017-06-23 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21190?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-21190: Description: *Background and Motivation* Python is one of the most popular programming languages am

[jira] [Commented] (SPARK-14220) Build and test Spark against Scala 2.12

2017-06-23 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14220?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16060545#comment-16060545 ] Sean Owen commented on SPARK-14220: --- I have a branch locally for 2.12. That much isn't

[jira] [Commented] (SPARK-14220) Build and test Spark against Scala 2.12

2017-06-23 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14220?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16060549#comment-16060549 ] Reynold Xin commented on SPARK-14220: - Making it build isn't that much work, but gett

[jira] [Commented] (SPARK-21176) Master UI hangs with spark.ui.reverseProxy=true if the master node has many CPUs

2017-06-23 Thread Ingo Schuster (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16060553#comment-16060553 ] Ingo Schuster commented on SPARK-21176: --- Best way I can see to control the number o

[jira] [Resolved] (SPARK-21171) Speculate task scheduling block dirve handle normal task when a job task number more than one hundred thousand

2017-06-23 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21171?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-21171. --- Resolution: Invalid > Speculate task scheduling block dirve handle normal task when a job task > num

[jira] [Commented] (SPARK-21183) Unable to return Google BigQuery INTEGER data type into Spark via google BigQuery JDBC driver: java.sql.SQLDataException: [Simba][JDBC](10140) Error converting value t

2017-06-23 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21183?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16060557#comment-16060557 ] Sean Owen commented on SPARK-21183: --- I'm not clear this is a Spark problem. What's the

[jira] [Commented] (SPARK-19726) Faild to insert null timestamp value to mysql using spark jdbc

2017-06-23 Thread wangshuangshuang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19726?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16060558#comment-16060558 ] wangshuangshuang commented on SPARK-19726: -- How to reappear this issue via testc

[jira] [Commented] (SPARK-19726) Faild to insert null timestamp value to mysql using spark jdbc

2017-06-23 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19726?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16060562#comment-16060562 ] Sean Owen commented on SPARK-19726: --- I'm not sure this is a Spark problem. You're tryin

[jira] [Resolved] (SPARK-21185) Spurious errors in unidoc causing PRs to fail

2017-06-23 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21185?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-21185. --- Resolution: Duplicate Yes, the underlying issue is that javadoc 8 is stricter. We have a few issues

[jira] [Commented] (SPARK-21152) Use level 3 BLAS operations in LogisticAggregator

2017-06-23 Thread Yanbo Liang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21152?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16060603#comment-16060603 ] Yanbo Liang commented on SPARK-21152: - [~sethah] This is an interesting topic, thanks

[jira] [Created] (SPARK-21191) DataFrame Row StructType check duplicate name

2017-06-23 Thread darion yaphet (JIRA)
darion yaphet created SPARK-21191: - Summary: DataFrame Row StructType check duplicate name Key: SPARK-21191 URL: https://issues.apache.org/jira/browse/SPARK-21191 Project: Spark Issue Type: B

[jira] [Commented] (SPARK-21191) DataFrame Row StructType check duplicate name

2017-06-23 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16060629#comment-16060629 ] Sean Owen commented on SPARK-21191: --- What is the issue? You have the cold duplicate nam

[jira] [Assigned] (SPARK-21191) DataFrame Row StructType check duplicate name

2017-06-23 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21191?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21191: Assignee: Apache Spark > DataFrame Row StructType check duplicate name > -

[jira] [Assigned] (SPARK-21191) DataFrame Row StructType check duplicate name

2017-06-23 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21191?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21191: Assignee: (was: Apache Spark) > DataFrame Row StructType check duplicate name > --

[jira] [Commented] (SPARK-21191) DataFrame Row StructType check duplicate name

2017-06-23 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16060689#comment-16060689 ] Apache Spark commented on SPARK-21191: -- User 'darionyaphet' has created a pull reque

[jira] [Created] (SPARK-21192) Preserve State Store provider class configuration across restarts

2017-06-23 Thread Tathagata Das (JIRA)
Tathagata Das created SPARK-21192: - Summary: Preserve State Store provider class configuration across restarts Key: SPARK-21192 URL: https://issues.apache.org/jira/browse/SPARK-21192 Project: Spark

[jira] [Updated] (SPARK-21192) Preserve State Store provider class configuration across StreamingQuery restarts

2017-06-23 Thread Tathagata Das (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21192?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das updated SPARK-21192: -- Summary: Preserve State Store provider class configuration across StreamingQuery restarts (was

[jira] [Assigned] (SPARK-21192) Preserve State Store provider class configuration across StreamingQuery restarts

2017-06-23 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21192?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21192: Assignee: Apache Spark (was: Tathagata Das) > Preserve State Store provider class configu

[jira] [Assigned] (SPARK-21192) Preserve State Store provider class configuration across StreamingQuery restarts

2017-06-23 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21192?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21192: Assignee: Tathagata Das (was: Apache Spark) > Preserve State Store provider class configu

[jira] [Commented] (SPARK-21192) Preserve State Store provider class configuration across StreamingQuery restarts

2017-06-23 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16060699#comment-16060699 ] Apache Spark commented on SPARK-21192: -- User 'tdas' has created a pull request for t

[jira] [Assigned] (SPARK-21192) Preserve State Store provider class configuration across StreamingQuery restarts

2017-06-23 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21192?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21192: Assignee: Apache Spark (was: Tathagata Das) > Preserve State Store provider class configu

[jira] [Created] (SPARK-21193) Specify Pandas version in setup.py

2017-06-23 Thread Hyukjin Kwon (JIRA)
Hyukjin Kwon created SPARK-21193: Summary: Specify Pandas version in setup.py Key: SPARK-21193 URL: https://issues.apache.org/jira/browse/SPARK-21193 Project: Spark Issue Type: Improvement

[jira] [Assigned] (SPARK-21193) Specify Pandas version in setup.py

2017-06-23 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21193?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21193: Assignee: Apache Spark > Specify Pandas version in setup.py >

[jira] [Commented] (SPARK-21193) Specify Pandas version in setup.py

2017-06-23 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21193?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16060719#comment-16060719 ] Apache Spark commented on SPARK-21193: -- User 'HyukjinKwon' has created a pull reques

[jira] [Assigned] (SPARK-21193) Specify Pandas version in setup.py

2017-06-23 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21193?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21193: Assignee: (was: Apache Spark) > Specify Pandas version in setup.py > -

[jira] [Commented] (SPARK-21191) DataFrame Row StructType check duplicate name

2017-06-23 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16060724#comment-16060724 ] Takeshi Yamamuro commented on SPARK-21191: -- I think this is a design: https://is

[jira] [Resolved] (SPARK-21191) DataFrame Row StructType check duplicate name

2017-06-23 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21191?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-21191. --- Resolution: Duplicate > DataFrame Row StructType check duplicate name > -

[jira] [Commented] (SPARK-19726) Faild to insert null timestamp value to mysql using spark jdbc

2017-06-23 Thread Chang chen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19726?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16060739#comment-16060739 ] Chang chen commented on SPARK-19726: @srowen, As we investigated, Spark translate nul

[jira] [Commented] (SPARK-21184) QuantileSummaries implementation is wrong and QuantileSummariesSuite fails with larger n

2017-06-23 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21184?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16060741#comment-16060741 ] Sean Owen commented on SPARK-21184: --- [~timhunter] [~clockfly] do you have opinions on t

[jira] [Assigned] (SPARK-18004) DataFrame filter Predicate push-down fails for Oracle Timestamp type columns

2017-06-23 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18004?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18004: Assignee: (was: Apache Spark) > DataFrame filter Predicate push-down fails for Oracle

[jira] [Assigned] (SPARK-18004) DataFrame filter Predicate push-down fails for Oracle Timestamp type columns

2017-06-23 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18004?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18004: Assignee: Apache Spark > DataFrame filter Predicate push-down fails for Oracle Timestamp t

[jira] [Commented] (SPARK-18004) DataFrame filter Predicate push-down fails for Oracle Timestamp type columns

2017-06-23 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16060795#comment-16060795 ] Apache Spark commented on SPARK-18004: -- User 'SharpRay' has created a pull request f

[jira] [Assigned] (SPARK-21047) Add test suites for complicated cases in ColumnarBatchSuite

2017-06-23 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21047?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-21047: --- Assignee: jin xing > Add test suites for complicated cases in ColumnarBatchSuite > -

[jira] [Resolved] (SPARK-21047) Add test suites for complicated cases in ColumnarBatchSuite

2017-06-23 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21047?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-21047. - Resolution: Fixed Fix Version/s: 2.3.0 Issue resolved by pull request 18327 [https://githu

[jira] [Resolved] (SPARK-21165) Fail to write into partitioned hive table due to attribute reference not working with cast on partition column

2017-06-23 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21165?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-21165. - Resolution: Fixed Fix Version/s: 2.2.0 Issue resolved by pull request 18386 [https://githu

[jira] [Assigned] (SPARK-21115) If the cores left is less than the coresPerExecutor,the cores left will not be allocated, so it should not to check in every schedule

2017-06-23 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21115?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-21115: --- Assignee: eaton > If the cores left is less than the coresPerExecutor,the cores left will no

[jira] [Resolved] (SPARK-21115) If the cores left is less than the coresPerExecutor,the cores left will not be allocated, so it should not to check in every schedule

2017-06-23 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21115?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-21115. - Resolution: Fixed Fix Version/s: 2.3.0 Issue resolved by pull request 18322 [https://githu

[jira] [Created] (SPARK-21194) Fail the putNullmethod when containsNull=false.

2017-06-23 Thread jin xing (JIRA)
jin xing created SPARK-21194: Summary: Fail the putNullmethod when containsNull=false. Key: SPARK-21194 URL: https://issues.apache.org/jira/browse/SPARK-21194 Project: Spark Issue Type: Bug

[jira] [Assigned] (SPARK-21194) Fail the putNullmethod when containsNull=false.

2017-06-23 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21194?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21194: Assignee: Apache Spark > Fail the putNullmethod when containsNull=false. > ---

[jira] [Assigned] (SPARK-21194) Fail the putNullmethod when containsNull=false.

2017-06-23 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21194?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21194: Assignee: (was: Apache Spark) > Fail the putNullmethod when containsNull=false. >

[jira] [Commented] (SPARK-21194) Fail the putNullmethod when containsNull=false.

2017-06-23 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21194?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16060919#comment-16060919 ] Apache Spark commented on SPARK-21194: -- User 'jinxing64' has created a pull request

[jira] [Assigned] (SPARK-21193) Specify Pandas version in setup.py

2017-06-23 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21193?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-21193: --- Assignee: Hyukjin Kwon > Specify Pandas version in setup.py > --

[jira] [Resolved] (SPARK-21193) Specify Pandas version in setup.py

2017-06-23 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21193?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-21193. - Resolution: Fixed Fix Version/s: 2.3.0 Issue resolved by pull request 18403 [https://githu

[jira] [Created] (SPARK-21195) MetricSystem should pick up dynamically registered metrics in sources

2017-06-23 Thread Robert Kruszewski (JIRA)
Robert Kruszewski created SPARK-21195: - Summary: MetricSystem should pick up dynamically registered metrics in sources Key: SPARK-21195 URL: https://issues.apache.org/jira/browse/SPARK-21195 Proje

[jira] [Commented] (SPARK-21195) MetricSystem should pick up dynamically registered metrics in sources

2017-06-23 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21195?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16060939#comment-16060939 ] Apache Spark commented on SPARK-21195: -- User 'robert3005' has created a pull request

[jira] [Assigned] (SPARK-21195) MetricSystem should pick up dynamically registered metrics in sources

2017-06-23 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21195?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21195: Assignee: Apache Spark > MetricSystem should pick up dynamically registered metrics in sou

[jira] [Assigned] (SPARK-21195) MetricSystem should pick up dynamically registered metrics in sources

2017-06-23 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21195?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21195: Assignee: (was: Apache Spark) > MetricSystem should pick up dynamically registered met

[jira] [Commented] (SPARK-18004) DataFrame filter Predicate push-down fails for Oracle Timestamp type columns

2017-06-23 Thread Rui Zha (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16060963#comment-16060963 ] Rui Zha commented on SPARK-18004: - The PR 18404 is closed. I will resend the PR later. >

[jira] [Updated] (SPARK-20952) ParquetFileFormat should forward TaskContext to its forkjoinpool

2017-06-23 Thread Robert Kruszewski (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20952?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robert Kruszewski updated SPARK-20952: -- Summary: ParquetFileFormat should forward TaskContext to its forkjoinpool (was: TaskCo

[jira] [Commented] (SPARK-21181) Suppress memory leak errors reported by netty

2017-06-23 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16060970#comment-16060970 ] Apache Spark commented on SPARK-21181: -- User 'dhruve' has created a pull request for

[jira] [Commented] (SPARK-20952) ParquetFileFormat should forward TaskContext to its forkjoinpool

2017-06-23 Thread Robert Kruszewski (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20952?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16060972#comment-16060972 ] Robert Kruszewski commented on SPARK-20952: --- [~zsxwing] updated the issue title

[jira] [Commented] (SPARK-20307) SparkR: pass on setHandleInvalid to spark.mllib functions that use StringIndexer

2017-06-23 Thread Joseph Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20307?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16060982#comment-16060982 ] Joseph Wang commented on SPARK-20307: - Hello, I am new to the discussion. I come acro

[jira] [Commented] (SPARK-21152) Use level 3 BLAS operations in LogisticAggregator

2017-06-23 Thread Seth Hendrickson (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21152?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16061158#comment-16061158 ] Seth Hendrickson commented on SPARK-21152: -- [~yanboliang] I can do performance t

[jira] [Commented] (SPARK-20307) SparkR: pass on setHandleInvalid to spark.mllib functions that use StringIndexer

2017-06-23 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20307?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16061159#comment-16061159 ] Felix Cheung commented on SPARK-20307: -- Not much update as far as I know. Would you

[jira] [Commented] (SPARK-20307) SparkR: pass on setHandleInvalid to spark.mllib functions that use StringIndexer

2017-06-23 Thread Joseph Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20307?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16061161#comment-16061161 ] Joseph Wang commented on SPARK-20307: - Hi Felix, the problem for me is how to get int

[jira] [Resolved] (SPARK-21144) Unexpected results when the data schema and partition schema have the duplicate columns

2017-06-23 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21144?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-21144. - Resolution: Fixed Assignee: Takeshi Yamamuro Fix Version/s: 2.2.0 > Unexpected results wh

[jira] [Updated] (SPARK-21164) Remove isTableSample from Sample and isGenerated from Alias and AttributeReference

2017-06-23 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-21164: Summary: Remove isTableSample from Sample and isGenerated from Alias and AttributeReference (was: Remove i

[jira] [Updated] (SPARK-21164) Remove isTableSample from Sample and isGenerated from Alias and AttributeReference

2017-06-23 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-21164: Description: isTableSample and isGenerated were introduced for SQL Generation respectively by #11148 and #

[jira] [Updated] (SPARK-21164) Remove isTableSample from Sample and isGenerated from Alias and AttributeReference

2017-06-23 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-21164: Description: isTableSample and isGenerated were introduced for SQL Generation respectively by PR 11148 and

[jira] [Resolved] (SPARK-21149) Add job description API for R

2017-06-23 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21149?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung resolved SPARK-21149. -- Resolution: Fixed Assignee: Hyukjin Kwon Fix Version/s: 2.3.0 Targe

[jira] [Commented] (SPARK-21155) Add (? running tasks) into Spark UI progress

2017-06-23 Thread Eric Vandenberg (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16061225#comment-16061225 ] Eric Vandenberg commented on SPARK-21155: - Added screen shot with skipped tasks f

[jira] [Resolved] (SPARK-21180) Remove conf from stats functions since now we have conf in LogicalPlan

2017-06-23 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21180?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-21180. - Resolution: Fixed Assignee: Zhenhua Wang Fix Version/s: 2.3.0 > Remove conf from stats fu

[jira] [Resolved] (SPARK-21181) Suppress memory leak errors reported by netty

2017-06-23 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21181?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-21181. Resolution: Fixed Assignee: Dhruve Ashar Fix Version/s: 2.3.0

[jira] [Commented] (SPARK-20555) Incorrect handling of Oracle's decimal types via JDBC

2017-06-23 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16061304#comment-16061304 ] Apache Spark commented on SPARK-20555: -- User 'gatorsmile' has created a pull request

[jira] [Resolved] (SPARK-21192) Preserve State Store provider class configuration across StreamingQuery restarts

2017-06-23 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21192?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu resolved SPARK-21192. -- Resolution: Fixed Assignee: Tathagata Das (was: Apache Spark) Fix Version/s: 2.

[jira] [Resolved] (SPARK-20417) Move error reporting for subquery from Analyzer to CheckAnalysis

2017-06-23 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20417?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-20417. - Resolution: Fixed Assignee: Dilip Biswal Fix Version/s: 2.3.0 > Move error reporting for

[jira] [Created] (SPARK-21196) Split codegen info of query plan into sequence

2017-06-23 Thread Gengliang Wang (JIRA)
Gengliang Wang created SPARK-21196: -- Summary: Split codegen info of query plan into sequence Key: SPARK-21196 URL: https://issues.apache.org/jira/browse/SPARK-21196 Project: Spark Issue Type

[jira] [Commented] (SPARK-21196) Split codegen info of query plan into sequence

2017-06-23 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21196?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16061364#comment-16061364 ] Apache Spark commented on SPARK-21196: -- User 'gengliangwang' has created a pull requ

[jira] [Assigned] (SPARK-21196) Split codegen info of query plan into sequence

2017-06-23 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21196?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21196: Assignee: (was: Apache Spark) > Split codegen info of query plan into sequence > -

[jira] [Assigned] (SPARK-21196) Split codegen info of query plan into sequence

2017-06-23 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21196?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21196: Assignee: Apache Spark > Split codegen info of query plan into sequence >

[jira] [Updated] (SPARK-21110) Structs should be usable in inequality filters

2017-06-23 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21110?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-21110: - Target Version/s: 2.3.0 > Structs should be usable in inequality filters > --

[jira] [Commented] (SPARK-21110) Structs should be usable in inequality filters

2017-06-23 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21110?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16061398#comment-16061398 ] Michael Armbrust commented on SPARK-21110: -- It seems if you can call {{min}} and

[jira] [Reopened] (SPARK-21137) Spark reads many small files slowly

2017-06-23 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21137?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen reopened SPARK-21137: --- > Spark reads many small files slowly > --- > > Key: SPAR

[jira] [Updated] (SPARK-21137) Spark reads many small files slowly

2017-06-23 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21137?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-21137: -- Affects Version/s: (was: 2.2.1) 2.1.1 Priority: Minor (was: Maj

[jira] [Updated] (SPARK-21190) SPIP: Vectorized UDFs in Python

2017-06-23 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21190?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-21190: Description: *Background and Motivation* Python is one of the most popular programming languages am

[jira] [Closed] (SPARK-20817) Benchmark.getProcessorName() returns "Unknown processor" on ppc and 390 platforms

2017-06-23 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20817?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin closed SPARK-20817. --- Resolution: Won't Fix See github discussions. > Benchmark.getProcessorName() returns "Unknown proce

[jira] [Updated] (SPARK-21190) SPIP: Vectorized UDFs in Python

2017-06-23 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21190?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-21190: Description: *Background and Motivation* Python is one of the most popular programming languages am

[jira] [Commented] (SPARK-20307) SparkR: pass on setHandleInvalid to spark.mllib functions that use StringIndexer

2017-06-23 Thread Miao Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20307?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16061525#comment-16061525 ] Miao Wang commented on SPARK-20307: --- [~Monday0927!] I am working on it now. Thanks! Mia

[jira] [Commented] (SPARK-20307) SparkR: pass on setHandleInvalid to spark.mllib functions that use StringIndexer

2017-06-23 Thread Joseph Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20307?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16061532#comment-16061532 ] Joseph Wang commented on SPARK-20307: - Thanks a lot. Let me know if you have done tha

[jira] [Resolved] (SPARK-21164) Remove isTableSample from Sample and isGenerated from Alias and AttributeReference

2017-06-23 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-21164. - Resolution: Fixed Fix Version/s: 2.3.0 > Remove isTableSample from Sample and isGenerated from Ali

[jira] [Commented] (SPARK-21157) Report Total Memory Used by Spark Executors

2017-06-23 Thread Jose Soltren (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21157?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16061582#comment-16061582 ] Jose Soltren commented on SPARK-21157: -- Hello Marcelo - It's true, the design doc do

[jira] [Commented] (SPARK-20971) Purge the metadata log for FileStreamSource

2017-06-23 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20971?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16061596#comment-16061596 ] Apache Spark commented on SPARK-20971: -- User 'CodingCat' has created a pull request

[jira] [Assigned] (SPARK-20971) Purge the metadata log for FileStreamSource

2017-06-23 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20971?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20971: Assignee: Apache Spark > Purge the metadata log for FileStreamSource > ---

[jira] [Assigned] (SPARK-20971) Purge the metadata log for FileStreamSource

2017-06-23 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20971?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20971: Assignee: (was: Apache Spark) > Purge the metadata log for FileStreamSource >

[jira] [Created] (SPARK-21197) Tricky use cases makes dead application struggle for a long duration

2017-06-23 Thread Nan Zhu (JIRA)
Nan Zhu created SPARK-21197: --- Summary: Tricky use cases makes dead application struggle for a long duration Key: SPARK-21197 URL: https://issues.apache.org/jira/browse/SPARK-21197 Project: Spark I

[jira] [Updated] (SPARK-21197) Tricky use case makes dead application struggle for a long duration

2017-06-23 Thread Nan Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21197?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nan Zhu updated SPARK-21197: Summary: Tricky use case makes dead application struggle for a long duration (was: Tricky use cases makes

[jira] [Created] (SPARK-21198) SparkSession catalog is terribly slow

2017-06-23 Thread Saif Addin (JIRA)
Saif Addin created SPARK-21198: -- Summary: SparkSession catalog is terribly slow Key: SPARK-21198 URL: https://issues.apache.org/jira/browse/SPARK-21198 Project: Spark Issue Type: Bug C

  1   2   >