[jira] [Assigned] (SPARK-21923) Avoid calling reserveUnrollMemoryForThisTask for every record

2017-09-18 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21923?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-21923: --- Assignee: Xianyang Liu > Avoid calling reserveUnrollMemoryForThisTask for every record > ---

[jira] [Resolved] (SPARK-21923) Avoid calling reserveUnrollMemoryForThisTask for every record

2017-09-18 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21923?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-21923. - Resolution: Fixed Fix Version/s: 2.3.0 Issue resolved by pull request 19135 [https://githu

[jira] [Commented] (SPARK-22047) HiveExternalCatalogVersionsSuite is Flaky on Jenkins

2017-09-18 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22047?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16171079#comment-16171079 ] Wenchen Fan commented on SPARK-22047: - Let's wait for a few days to see if my fix wor

[jira] [Assigned] (SPARK-21428) CliSessionState never be recognized because of IsolatedClientLoader

2017-09-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21428?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21428: Assignee: Apache Spark (was: Kent Yao) > CliSessionState never be recognized because of I

[jira] [Commented] (SPARK-21428) CliSessionState never be recognized because of IsolatedClientLoader

2017-09-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21428?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16171077#comment-16171077 ] Apache Spark commented on SPARK-21428: -- User 'cloud-fan' has created a pull request

[jira] [Assigned] (SPARK-21428) CliSessionState never be recognized because of IsolatedClientLoader

2017-09-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21428?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21428: Assignee: Apache Spark (was: Kent Yao) > CliSessionState never be recognized because of I

[jira] [Assigned] (SPARK-21428) CliSessionState never be recognized because of IsolatedClientLoader

2017-09-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21428?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21428: Assignee: Kent Yao (was: Apache Spark) > CliSessionState never be recognized because of I

[jira] [Reopened] (SPARK-21428) CliSessionState never be recognized because of IsolatedClientLoader

2017-09-18 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21428?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reopened SPARK-21428: - > CliSessionState never be recognized because of IsolatedClientLoader > -

[jira] [Updated] (SPARK-22054) Allow release managers to inject their keys

2017-09-18 Thread holdenk (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22054?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] holdenk updated SPARK-22054: Fix Version/s: (was: 2.2.1) (was: 2.3.0) > Allow release managers to inject thei

[jira] [Updated] (SPARK-22054) Allow release managers to inject their keys

2017-09-18 Thread holdenk (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22054?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] holdenk updated SPARK-22054: Target Version/s: 2.2.1, 2.3.0 > Allow release managers to inject their keys >

[jira] [Created] (SPARK-22055) Port release scripts

2017-09-18 Thread holdenk (JIRA)
holdenk created SPARK-22055: --- Summary: Port release scripts Key: SPARK-22055 URL: https://issues.apache.org/jira/browse/SPARK-22055 Project: Spark Issue Type: Bug Components: Build Af

[jira] [Created] (SPARK-22054) Allow release managers to inject their keys

2017-09-18 Thread holdenk (JIRA)
holdenk created SPARK-22054: --- Summary: Allow release managers to inject their keys Key: SPARK-22054 URL: https://issues.apache.org/jira/browse/SPARK-22054 Project: Spark Issue Type: Bug C

[jira] [Commented] (SPARK-12823) Cannot create UDF with StructType input

2017-09-18 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16171019#comment-16171019 ] Takeshi Yamamuro commented on SPARK-12823: -- As wenchen suggested, why don't you

[jira] [Commented] (SPARK-20466) HadoopRDD#addLocalConfiguration throws NPE

2017-09-18 Thread liyunzhang_intel (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20466?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16171009#comment-16171009 ] liyunzhang_intel commented on SPARK-20466: -- [~stakiar]: this exception happene

[jira] [Assigned] (SPARK-22030) GraphiteSink fails to re-connect to Graphite instances behind an ELB or any other auto-scaled LB

2017-09-18 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22030?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao reassigned SPARK-22030: --- Assignee: Alex Mikhailau > GraphiteSink fails to re-connect to Graphite instances behind an

[jira] [Updated] (SPARK-22030) GraphiteSink fails to re-connect to Graphite instances behind an ELB or any other auto-scaled LB

2017-09-18 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22030?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao updated SPARK-22030: Priority: Minor (was: Critical) > GraphiteSink fails to re-connect to Graphite instances behind an

[jira] [Resolved] (SPARK-22030) GraphiteSink fails to re-connect to Graphite instances behind an ELB or any other auto-scaled LB

2017-09-18 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22030?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao resolved SPARK-22030. - Resolution: Fixed Fix Version/s: 2.3.0 Issue resolved by pull request 19210 [https://githu

[jira] [Assigned] (SPARK-21842) Support Kerberos ticket renewal and creation in Mesos

2017-09-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21842?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21842: Assignee: Apache Spark > Support Kerberos ticket renewal and creation in Mesos >

[jira] [Assigned] (SPARK-21842) Support Kerberos ticket renewal and creation in Mesos

2017-09-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21842?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21842: Assignee: (was: Apache Spark) > Support Kerberos ticket renewal and creation in Mesos

[jira] [Commented] (SPARK-21842) Support Kerberos ticket renewal and creation in Mesos

2017-09-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21842?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16170932#comment-16170932 ] Apache Spark commented on SPARK-21842: -- User 'ArtRand' has created a pull request fo

[jira] [Commented] (SPARK-22053) Implement stream-stream inner join in Append mode

2017-09-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22053?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16170906#comment-16170906 ] Apache Spark commented on SPARK-22053: -- User 'tdas' has created a pull request for t

[jira] [Assigned] (SPARK-22053) Implement stream-stream inner join in Append mode

2017-09-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22053?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22053: Assignee: Apache Spark (was: Tathagata Das) > Implement stream-stream inner join in Appen

[jira] [Assigned] (SPARK-22053) Implement stream-stream inner join in Append mode

2017-09-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22053?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22053: Assignee: Tathagata Das (was: Apache Spark) > Implement stream-stream inner join in Appen

[jira] [Updated] (SPARK-22053) Implement stream-stream inner join in Append mode

2017-09-18 Thread Tathagata Das (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22053?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das updated SPARK-22053: -- Description: Stream-stream inner join is traditionally implemented using a two-way symmetric h

[jira] [Commented] (SPARK-10413) ML models should support prediction on single instances

2017-09-18 Thread Derek Kaknes (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10413?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16170887#comment-16170887 ] Derek Kaknes commented on SPARK-10413: -- Is anyone still working on this? This would

[jira] [Updated] (SPARK-22053) Implement stream-stream inner join in Append mode

2017-09-18 Thread Tathagata Das (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22053?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das updated SPARK-22053: -- Summary: Implement stream-stream inner join in Append mode (was: Implement stream-stream inner

[jira] [Updated] (SPARK-22053) Implement stream-stream inner join

2017-09-18 Thread Tathagata Das (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22053?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das updated SPARK-22053: -- Description: Stream-stream inner join is traditionally implemented using a two-way symmetric h

[jira] [Updated] (SPARK-22053) Implement stream-stream inner join in Append mode

2017-09-18 Thread Tathagata Das (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22053?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das updated SPARK-22053: -- Description: Stream-stream inner join is traditionally implemented using a two-way symmetric h

[jira] [Updated] (SPARK-22053) Implement stream-stream inner join

2017-09-18 Thread Tathagata Das (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22053?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das updated SPARK-22053: -- Description: Stream-stream inner join is traditionally implemented using a two-way symmetric h

[jira] [Updated] (SPARK-22053) Implement stream-stream inner join

2017-09-18 Thread Tathagata Das (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22053?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das updated SPARK-22053: -- Description: Stream-stream inner join is traditionally implemented using a two-way symmetric h

[jira] [Created] (SPARK-22053) Implement stream-stream inner join

2017-09-18 Thread Tathagata Das (JIRA)
Tathagata Das created SPARK-22053: - Summary: Implement stream-stream inner join Key: SPARK-22053 URL: https://issues.apache.org/jira/browse/SPARK-22053 Project: Spark Issue Type: Sub-task

[jira] [Updated] (SPARK-18791) Stream-Stream Joins

2017-09-18 Thread Tathagata Das (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18791?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das updated SPARK-18791: -- Description: Stream stream join is a much requested, but missing feature in Structured Streami

[jira] [Assigned] (SPARK-18791) Stream-Stream Joins

2017-09-18 Thread Tathagata Das (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18791?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das reassigned SPARK-18791: - Assignee: Tathagata Das > Stream-Stream Joins > --- > >

[jira] [Updated] (SPARK-18791) Stream-Stream Joins

2017-09-18 Thread Tathagata Das (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18791?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das updated SPARK-18791: -- Description: Stream stream join is a much requested, but missing feature in Structured Streami

[jira] [Updated] (SPARK-18791) Stream-Stream Joins

2017-09-18 Thread Tathagata Das (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18791?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das updated SPARK-18791: -- Description: Stream stream join is a much requested, but missing feature in Structured Streamin

[jira] [Updated] (SPARK-18791) Stream-Stream Joins

2017-09-18 Thread Tathagata Das (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18791?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das updated SPARK-18791: -- Description: Stream stream join is a much requested, but missing feature in Structured Streamin

[jira] [Commented] (SPARK-20466) HadoopRDD#addLocalConfiguration throws NPE

2017-09-18 Thread Sahil Takiar (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20466?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16170759#comment-16170759 ] Sahil Takiar commented on SPARK-20466: -- I just hit this issue in Hive-on-Spark when

[jira] [Commented] (SPARK-21770) ProbabilisticClassificationModel: Improve normalization of all-zero raw predictions

2017-09-18 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21770?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16170734#comment-16170734 ] Joseph K. Bradley commented on SPARK-21770: --- Update from PR discussion: The new

[jira] [Commented] (SPARK-21809) Change Stage Page to use datatables to support sorting columns and searching

2017-09-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16170707#comment-16170707 ] Apache Spark commented on SPARK-21809: -- User 'pgandhi999' has created a pull request

[jira] [Updated] (SPARK-14878) Support Trim characters in the string trim function

2017-09-18 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14878?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-14878: Fix Version/s: 2.3.0 > Support Trim characters in the string trim function > --

[jira] [Resolved] (SPARK-14878) Support Trim characters in the string trim function

2017-09-18 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14878?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-14878. - Resolution: Fixed Assignee: kevin yu > Support Trim characters in the string trim function > --

[jira] [Comment Edited] (SPARK-12823) Cannot create UDF with StructType input

2017-09-18 Thread Simeon H.K. Fitch (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16170540#comment-16170540 ] Simeon H.K. Fitch edited comment on SPARK-12823 at 9/18/17 7:33 PM: ---

[jira] [Commented] (SPARK-12823) Cannot create UDF with StructType input

2017-09-18 Thread Simeon H.K. Fitch (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16170540#comment-16170540 ] Simeon H.K. Fitch commented on SPARK-12823: --- My suspicion is that the bug is [

[jira] [Updated] (SPARK-22052) Incorrect Metric assigned in MetricsReporter.scala

2017-09-18 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-22052: -- Flags: (was: Patch) Target Version/s: (was: 2.3.0) Labels: (was: easy

[jira] [Comment Edited] (SPARK-12823) Cannot create UDF with StructType input

2017-09-18 Thread Simeon H.K. Fitch (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16170511#comment-16170511 ] Simeon H.K. Fitch edited comment on SPARK-12823 at 9/18/17 7:06 PM: ---

[jira] [Commented] (SPARK-12823) Cannot create UDF with StructType input

2017-09-18 Thread Simeon H.K. Fitch (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16170511#comment-16170511 ] Simeon H.K. Fitch commented on SPARK-12823: --- Unfortunately, using a `TypedColum

[jira] [Updated] (SPARK-22052) Incorrect Metric assigned in MetricsReporter.scala

2017-09-18 Thread Jason Taaffe (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Taaffe updated SPARK-22052: - Description: The wrong metric is being sent in MetricsReporter.scala The current implementation

[jira] [Assigned] (SPARK-22052) Incorrect Metric reported in MetricsReporter.scala

2017-09-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22052: Assignee: Apache Spark > Incorrect Metric reported in MetricsReporter.scala >

[jira] [Assigned] (SPARK-22052) Incorrect Metric reported in MetricsReporter.scala

2017-09-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22052: Assignee: (was: Apache Spark) > Incorrect Metric reported in MetricsReporter.scala > -

[jira] [Commented] (SPARK-22052) Incorrect Metric reported in MetricsReporter.scala

2017-09-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16170399#comment-16170399 ] Apache Spark commented on SPARK-22052: -- User 'Taaffy' has created a pull request for

[jira] [Issue Comment Deleted] (SPARK-22052) Incorrect Metric reported in MetricsReporter.scala

2017-09-18 Thread Jason Taaffe (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Taaffe updated SPARK-22052: - Comment: was deleted (was: Please check the attached file "Processed Rows Per Second". After alte

[jira] [Updated] (SPARK-22052) Incorrect Metric reported in MetricsReporter.scala

2017-09-18 Thread Jason Taaffe (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Taaffe updated SPARK-22052: - Description: The wrong metric is being sent in MetricsReporter.scala The current implementation

[jira] [Updated] (SPARK-22052) Incorrect Metric reported in MetricsReporter.scala

2017-09-18 Thread Jason Taaffe (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Taaffe updated SPARK-22052: - Target Version/s: 2.3.0 (was: 2.2.0, 2.3.0) > Incorrect Metric reported in MetricsReporter.scala

[jira] [Updated] (SPARK-22052) Incorrect Metric reported in MetricsReporter.scala

2017-09-18 Thread Jason Taaffe (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Taaffe updated SPARK-22052: - Remaining Estimate: (was: 1h) Original Estimate: (was: 1h) > Incorrect Metric report

[jira] [Comment Edited] (SPARK-22052) Incorrect Metric reported in MetricsReporter.scala

2017-09-18 Thread Jason Taaffe (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16170365#comment-16170365 ] Jason Taaffe edited comment on SPARK-22052 at 9/18/17 5:32 PM:

[jira] [Updated] (SPARK-22052) Incorrect Metric reported in MetricsReporter.scala

2017-09-18 Thread Jason Taaffe (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Taaffe updated SPARK-22052: - Attachment: Processed Rows Per Second.png After altering the code the correct values are displaye

[jira] [Created] (SPARK-22052) Incorrect Metric reported in MetricsReporter.scala

2017-09-18 Thread Jason Taaffe (JIRA)
Jason Taaffe created SPARK-22052: Summary: Incorrect Metric reported in MetricsReporter.scala Key: SPARK-22052 URL: https://issues.apache.org/jira/browse/SPARK-22052 Project: Spark Issue Type

[jira] [Commented] (SPARK-22026) data source v2 write path

2017-09-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22026?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16170349#comment-16170349 ] Apache Spark commented on SPARK-22026: -- User 'cloud-fan' has created a pull request

[jira] [Assigned] (SPARK-22026) data source v2 write path

2017-09-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22026?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22026: Assignee: Apache Spark (was: Wenchen Fan) > data source v2 write path > -

[jira] [Assigned] (SPARK-22026) data source v2 write path

2017-09-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22026?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22026: Assignee: Wenchen Fan (was: Apache Spark) > data source v2 write path > -

[jira] [Commented] (SPARK-22000) org.codehaus.commons.compiler.CompileException: toString method is not declared

2017-09-18 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22000?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16170319#comment-16170319 ] Kazuaki Ishizaki commented on SPARK-22000: -- If there is no sample code, it may t

[jira] [Commented] (SPARK-20628) Keep track of nodes which are going to be shut down & avoid scheduling new tasks

2017-09-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20628?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16170276#comment-16170276 ] Apache Spark commented on SPARK-20628: -- User 'juanrh' has created a pull request for

[jira] [Commented] (SPARK-22049) Confusing behavior of from_utc_timestamp and to_utc_timestamp

2017-09-18 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22049?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16170205#comment-16170205 ] Sean Owen commented on SPARK-22049: --- Questions should normally go to the mailing list,

[jira] [Resolved] (SPARK-22003) vectorized reader does not work with UDF when the column is array

2017-09-18 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22003?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-22003. - Resolution: Fixed Assignee: Feng Liu Fix Version/s: 2.3.0 > vectorized reader does not wo

[jira] [Commented] (SPARK-22033) BufferHolder size checks should account for the specific VM array size limitations

2017-09-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22033?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16170121#comment-16170121 ] Apache Spark commented on SPARK-22033: -- User 'srowen' has created a pull request for

[jira] [Updated] (SPARK-22033) BufferHolder, other size checks should account for the specific VM array size limitations

2017-09-18 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22033?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-22033: -- Summary: BufferHolder, other size checks should account for the specific VM array size limitations (wa

[jira] [Assigned] (SPARK-22033) BufferHolder size checks should account for the specific VM array size limitations

2017-09-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22033?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22033: Assignee: (was: Apache Spark) > BufferHolder size checks should account for the specif

[jira] [Assigned] (SPARK-22033) BufferHolder size checks should account for the specific VM array size limitations

2017-09-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22033?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22033: Assignee: Apache Spark > BufferHolder size checks should account for the specific VM array

[jira] [Comment Edited] (SPARK-22033) BufferHolder size checks should account for the specific VM array size limitations

2017-09-18 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22033?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16170109#comment-16170109 ] Takeshi Yamamuro edited comment on SPARK-22033 at 9/18/17 2:57 PM:

[jira] [Commented] (SPARK-22033) BufferHolder size checks should account for the specific VM array size limitations

2017-09-18 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22033?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16170109#comment-16170109 ] Takeshi Yamamuro commented on SPARK-22033: -- just a head-up: this is probably rel

[jira] [Commented] (SPARK-22028) spark-submit trips over environment variables

2017-09-18 Thread Franz Wimmer (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22028?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16170101#comment-16170101 ] Franz Wimmer commented on SPARK-22028: -- I temporarily circumvented the problem by ca

[jira] [Created] (SPARK-22051) Explicit control of number of partitions after dataframe operations (join, order...)

2017-09-18 Thread Fernando Pereira (JIRA)
Fernando Pereira created SPARK-22051: Summary: Explicit control of number of partitions after dataframe operations (join, order...) Key: SPARK-22051 URL: https://issues.apache.org/jira/browse/SPARK-22051

[jira] [Commented] (SPARK-22047) HiveExternalCatalogVersionsSuite is Flaky on Jenkins

2017-09-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22047?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16169927#comment-16169927 ] Apache Spark commented on SPARK-22047: -- User 'cloud-fan' has created a pull request

[jira] [Assigned] (SPARK-22050) Allow BlockUpdated events to be optionally logged to the event log

2017-09-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22050?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22050: Assignee: (was: Apache Spark) > Allow BlockUpdated events to be optionally logged to t

[jira] [Commented] (SPARK-22050) Allow BlockUpdated events to be optionally logged to the event log

2017-09-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22050?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16169919#comment-16169919 ] Apache Spark commented on SPARK-22050: -- User 'michaelmior' has created a pull reques

[jira] [Assigned] (SPARK-22050) Allow BlockUpdated events to be optionally logged to the event log

2017-09-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22050?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22050: Assignee: Apache Spark > Allow BlockUpdated events to be optionally logged to the event lo

[jira] [Created] (SPARK-22050) Allow BlockUpdated events to be optionally logged to the event log

2017-09-18 Thread Michael Mior (JIRA)
Michael Mior created SPARK-22050: Summary: Allow BlockUpdated events to be optionally logged to the event log Key: SPARK-22050 URL: https://issues.apache.org/jira/browse/SPARK-22050 Project: Spark

[jira] [Updated] (SPARK-22049) Confusing behavior of from_utc_timestamp and to_utc_timestamp

2017-09-18 Thread Felipe Olmos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22049?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felipe Olmos updated SPARK-22049: - Description: Hello everyone, I am confused about the behavior of the functions {{from_utc_timest

[jira] [Updated] (SPARK-22049) Confusing behavior of from_utc_timestamp and to_utc_timestamp

2017-09-18 Thread Felipe Olmos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22049?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felipe Olmos updated SPARK-22049: - Description: Hello everyone, I am confused about the behavior of the functions {{from_utc_timest

[jira] [Updated] (SPARK-22049) Confusing behavior of from_utc_timestamp and to_utc_timestamp

2017-09-18 Thread Felipe Olmos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22049?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felipe Olmos updated SPARK-22049: - Description: Hello everyone, I am confused about the behavior of the functions {{from_utc_timest

[jira] [Issue Comment Deleted] (SPARK-21067) Thrift Server - CTAS fail with Unable to move source

2017-09-18 Thread xinzhang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21067?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] xinzhang updated SPARK-21067: - Comment: was deleted (was: I try to solve it by coding the source code by myself. It is too complex to m

[jira] [Updated] (SPARK-22049) Confusing behavior of from_utc_timestamp and to_utc_timestamp

2017-09-18 Thread Felipe Olmos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22049?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felipe Olmos updated SPARK-22049: - Description: Hello everyone, I am confused about the behavior of the functions {{from_utc_timest

[jira] [Created] (SPARK-22049) Confusing behavior of from_utc_timestamp and to_utc_timestamp

2017-09-18 Thread Felipe Olmos (JIRA)
Felipe Olmos created SPARK-22049: Summary: Confusing behavior of from_utc_timestamp and to_utc_timestamp Key: SPARK-22049 URL: https://issues.apache.org/jira/browse/SPARK-22049 Project: Spark

[jira] [Commented] (SPARK-21067) Thrift Server - CTAS fail with Unable to move source

2017-09-18 Thread xinzhang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21067?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16169797#comment-16169797 ] xinzhang commented on SPARK-21067: -- i try to solve it by coding the source code by mysel

[jira] [Comment Edited] (SPARK-21067) Thrift Server - CTAS fail with Unable to move source

2017-09-18 Thread xinzhang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21067?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16169797#comment-16169797 ] xinzhang edited comment on SPARK-21067 at 9/18/17 9:35 AM: --- I t

[jira] [Updated] (SPARK-22048) Show id, runId, batch in Description column in SQL tab for streaming queries (as in Jobs)

2017-09-18 Thread Jacek Laskowski (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22048?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jacek Laskowski updated SPARK-22048: Attachment: webui-jobs-description.png webui-sql-description.png > Show id,

[jira] [Created] (SPARK-22048) Show id, runId, batch in Description column in SQL tab for streaming queries (as in Jobs)

2017-09-18 Thread Jacek Laskowski (JIRA)
Jacek Laskowski created SPARK-22048: --- Summary: Show id, runId, batch in Description column in SQL tab for streaming queries (as in Jobs) Key: SPARK-22048 URL: https://issues.apache.org/jira/browse/SPARK-22048

[jira] [Assigned] (SPARK-22047) HiveExternalCatalogVersionsSuite is Flaky on Jenkins

2017-09-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22047?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22047: Assignee: Apache Spark > HiveExternalCatalogVersionsSuite is Flaky on Jenkins > --

[jira] [Assigned] (SPARK-22047) HiveExternalCatalogVersionsSuite is Flaky on Jenkins

2017-09-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22047?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22047: Assignee: (was: Apache Spark) > HiveExternalCatalogVersionsSuite is Flaky on Jenkins >

[jira] [Commented] (SPARK-22047) HiveExternalCatalogVersionsSuite is Flaky on Jenkins

2017-09-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22047?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16169754#comment-16169754 ] Apache Spark commented on SPARK-22047: -- User 'cloud-fan' has created a pull request

[jira] [Commented] (SPARK-22047) HiveExternalCatalogVersionsSuite is Flaky on Jenkins

2017-09-18 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22047?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16169743#comment-16169743 ] Wenchen Fan commented on SPARK-22047: - let me send a PR to ignore this test suite tem

[jira] [Commented] (SPARK-22047) HiveExternalCatalogVersionsSuite is Flaky on Jenkins

2017-09-18 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22047?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16169725#comment-16169725 ] Sean Owen commented on SPARK-22047: --- [~cloud_fan] I think this is failing builds on mas

[jira] [Created] (SPARK-22047) HiveExternalCatalogVersionsSuite is Flaky on Jenkins

2017-09-18 Thread Armin Braun (JIRA)
Armin Braun created SPARK-22047: --- Summary: HiveExternalCatalogVersionsSuite is Flaky on Jenkins Key: SPARK-22047 URL: https://issues.apache.org/jira/browse/SPARK-22047 Project: Spark Issue Type

[jira] [Commented] (SPARK-22028) spark-submit trips over environment variables

2017-09-18 Thread Franz Wimmer (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22028?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16169684#comment-16169684 ] Franz Wimmer commented on SPARK-22028: -- That's what bothers me, too... I will look f

[jira] [Commented] (SPARK-22028) spark-submit trips over environment variables

2017-09-18 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22028?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16169668#comment-16169668 ] Sean Owen commented on SPARK-22028: --- Wouldn't this mean the JVM doesn't work on Windows

[jira] [Updated] (SPARK-22046) Streaming State cannot be scalable

2017-09-18 Thread danny mor (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22046?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] danny mor updated SPARK-22046: -- Description: State cannot be distributed on the cluster. When the {color:#59afe1}StateStoreRDD{color}'s

[jira] [Updated] (SPARK-22046) Streaming State cannot be scalable

2017-09-18 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22046?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-22046: -- Issue Type: Improvement (was: Bug) > Streaming State cannot be scalable >

[jira] [Created] (SPARK-22046) Streaming State cannot be scalable

2017-09-18 Thread danny mor (JIRA)
danny mor created SPARK-22046: - Summary: Streaming State cannot be scalable Key: SPARK-22046 URL: https://issues.apache.org/jira/browse/SPARK-22046 Project: Spark Issue Type: Bug Compon

[jira] [Commented] (SPARK-22028) spark-submit trips over environment variables

2017-09-18 Thread Franz Wimmer (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22028?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16169663#comment-16169663 ] Franz Wimmer commented on SPARK-22028: -- Even if I could unset this - which I cannot