[jira] [Assigned] (SPARK-19145) Timestamp to String casting is slowing the query significantly

[ https://issues.apache.org/jira/browse/SPARK-19145?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19145: Assignee: (was: Apache Spark) > Timestamp to String casting is slowing the query

[jira] [Assigned] (SPARK-19145) Timestamp to String casting is slowing the query significantly

[ https://issues.apache.org/jira/browse/SPARK-19145?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19145: Assignee: Apache Spark > Timestamp to String casting is slowing the query significantly >

[jira] [Commented] (SPARK-19145) Timestamp to String casting is slowing the query significantly

[ https://issues.apache.org/jira/browse/SPARK-19145?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15896842#comment-15896842 ] Apache Spark commented on SPARK-19145: -- User 'tanejagagan' has created a pull request for this

[jira] [Assigned] (SPARK-19832) DynamicPartitionWriteTask should escape the partition name

[ https://issues.apache.org/jira/browse/SPARK-19832?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19832: Assignee: (was: Apache Spark) > DynamicPartitionWriteTask should escape the partition

[jira] [Commented] (SPARK-19832) DynamicPartitionWriteTask should escape the partition name

[ https://issues.apache.org/jira/browse/SPARK-19832?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15896813#comment-15896813 ] Apache Spark commented on SPARK-19832: -- User 'windpiger' has created a pull request for this issue:

[jira] [Assigned] (SPARK-19832) DynamicPartitionWriteTask should escape the partition name

[ https://issues.apache.org/jira/browse/SPARK-19832?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19832: Assignee: Apache Spark > DynamicPartitionWriteTask should escape the partition name >

[jira] [Created] (SPARK-19832) DynamicPartitionWriteTask should escape the partition name

Song Jun created SPARK-19832: Summary: DynamicPartitionWriteTask should escape the partition name Key: SPARK-19832 URL: https://issues.apache.org/jira/browse/SPARK-19832 Project: Spark Issue

[jira] [Commented] (SPARK-19008) Avoid boxing/unboxing overhead of calling a lambda with primitive type from Dataset program

[ https://issues.apache.org/jira/browse/SPARK-19008?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15896808#comment-15896808 ] Apache Spark commented on SPARK-19008: -- User 'kiszk' has created a pull request for this issue:

[jira] [Assigned] (SPARK-19008) Avoid boxing/unboxing overhead of calling a lambda with primitive type from Dataset program

[ https://issues.apache.org/jira/browse/SPARK-19008?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19008: Assignee: (was: Apache Spark) > Avoid boxing/unboxing overhead of calling a lambda

[jira] [Assigned] (SPARK-19008) Avoid boxing/unboxing overhead of calling a lambda with primitive type from Dataset program

[ https://issues.apache.org/jira/browse/SPARK-19008?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19008: Assignee: Apache Spark > Avoid boxing/unboxing overhead of calling a lambda with

[jira] [Updated] (SPARK-19831) Sending the heartbeat master from worker maybe blocked by other rpc messages

[ https://issues.apache.org/jira/browse/SPARK-19831?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] hustfxj updated SPARK-19831: Description: Cleaning the application may cost much time at worker, then it will block that the worker

[jira] [Updated] (SPARK-19831) Sending the heartbeat master from worker maybe blocked by other rpc messages

[ https://issues.apache.org/jira/browse/SPARK-19831?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] hustfxj updated SPARK-19831: Description: Cleaning the application may cost much time at worker, then it will block that the worker

[jira] [Updated] (SPARK-19831) Sending the heartbeat master from worker maybe blocked by other rpc messages

[ https://issues.apache.org/jira/browse/SPARK-19831?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] hustfxj updated SPARK-19831: Description: Cleaning the application may cost much time at worker, then it will block that the worker

[jira] [Updated] (SPARK-19831) Sending the heartbeat master from worker maybe blocked by other rpc messages

[ https://issues.apache.org/jira/browse/SPARK-19831?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] hustfxj updated SPARK-19831: Description: Cleaning the application may cost much time at worker, then it will block that the worker

[jira] [Updated] (SPARK-19831) Sending the heartbeat master from worker maybe blocked by other rpc messages

[ https://issues.apache.org/jira/browse/SPARK-19831?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] hustfxj updated SPARK-19831: Summary: Sending the heartbeat master from worker maybe blocked by other rpc messages (was: Sending the

[jira] [Created] (SPARK-19831) Sending the heartbeat to master maybe blocked by other rpc messages

hustfxj created SPARK-19831: --- Summary: Sending the heartbeat to master maybe blocked by other rpc messages Key: SPARK-19831 URL: https://issues.apache.org/jira/browse/SPARK-19831 Project: Spark

[jira] [Commented] (SPARK-19578) Poor pyspark performance + incorrect UI input-size metrics

[ https://issues.apache.org/jira/browse/SPARK-19578?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15896751#comment-15896751 ] holdenk commented on SPARK-19578: - [~nchammas] That sounds like a pretty good summary from my point of

[jira] [Commented] (SPARK-19067) mapGroupsWithState - arbitrary stateful operations with Structured Streaming (similar to DStream.mapWithState)

[ https://issues.apache.org/jira/browse/SPARK-19067?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15896741#comment-15896741 ] Tathagata Das commented on SPARK-19067: --- Hey [~amitsela] Apologies for not noticing this comment

[jira] [Assigned] (SPARK-19830) Add parseTableSchema API to ParserInterface

[ https://issues.apache.org/jira/browse/SPARK-19830?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19830: Assignee: Apache Spark (was: Xiao Li) > Add parseTableSchema API to ParserInterface >

[jira] [Assigned] (SPARK-19830) Add parseTableSchema API to ParserInterface

[ https://issues.apache.org/jira/browse/SPARK-19830?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19830: Assignee: Xiao Li (was: Apache Spark) > Add parseTableSchema API to ParserInterface >

[jira] [Commented] (SPARK-19830) Add parseTableSchema API to ParserInterface

[ https://issues.apache.org/jira/browse/SPARK-19830?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15896736#comment-15896736 ] Apache Spark commented on SPARK-19830: -- User 'gatorsmile' has created a pull request for this issue:

[jira] [Updated] (SPARK-19830) Add parseTableSchema API to ParserInterface

[ https://issues.apache.org/jira/browse/SPARK-19830?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-19830: Description: Specifying the table schema in DDL formats is needed for different scenarios. For example,

[jira] [Created] (SPARK-19830) Add parseTableSchema API to ParserInterface

Xiao Li created SPARK-19830: --- Summary: Add parseTableSchema API to ParserInterface Key: SPARK-19830 URL: https://issues.apache.org/jira/browse/SPARK-19830 Project: Spark Issue Type: Improvement

[jira] [Closed] (SPARK-19815) Not orderable should be applied to right key instead of left key

[ https://issues.apache.org/jira/browse/SPARK-19815?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li closed SPARK-19815. --- Resolution: Won't Fix > Not orderable should be applied to right key instead of left key >

[jira] [Updated] (SPARK-19829) The log about driver should support rolling like executor

[ https://issues.apache.org/jira/browse/SPARK-19829?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] hustfxj updated SPARK-19829: Description: We should rollback the log of the driver , or the log maybe large！！！

[jira] [Created] (SPARK-19829) The log about driver should support rolling like executor

hustfxj created SPARK-19829: --- Summary: The log about driver should support rolling like executor Key: SPARK-19829 URL: https://issues.apache.org/jira/browse/SPARK-19829 Project: Spark Issue Type:

[jira] [Commented] (SPARK-19825) spark.ml R API for FPGrowth

[ https://issues.apache.org/jira/browse/SPARK-19825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15896665#comment-15896665 ] Apache Spark commented on SPARK-19825: -- User 'zero323' has created a pull request for this issue:

[jira] [Assigned] (SPARK-19825) spark.ml R API for FPGrowth

[ https://issues.apache.org/jira/browse/SPARK-19825?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19825: Assignee: Apache Spark > spark.ml R API for FPGrowth > --- > >

[jira] [Assigned] (SPARK-19825) spark.ml R API for FPGrowth

[ https://issues.apache.org/jira/browse/SPARK-19825?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19825: Assignee: (was: Apache Spark) > spark.ml R API for FPGrowth >

[jira] [Commented] (SPARK-18085) Better History Server scalability for many / large applications

[ https://issues.apache.org/jira/browse/SPARK-18085?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15896652#comment-15896652 ] DjvuLee commented on SPARK-18085: - "A separate jar file" means we generate a new jar file for the history

[jira] [Resolved] (SPARK-19822) CheckpointSuite.testCheckpointedOperation: should not check checkpointFilesOfLatestTime by the PATH string.

[ https://issues.apache.org/jira/browse/SPARK-19822?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu resolved SPARK-19822. -- Resolution: Fixed Assignee: Genmao Yu Fix Version/s: 2.2.0

[jira] [Assigned] (SPARK-19701) the `in` operator in pyspark is broken

[ https://issues.apache.org/jira/browse/SPARK-19701?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-19701: --- Assignee: Hyukjin Kwon > the `in` operator in pyspark is broken >

[jira] [Resolved] (SPARK-19701) the `in` operator in pyspark is broken

[ https://issues.apache.org/jira/browse/SPARK-19701?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-19701. - Resolution: Fixed Fix Version/s: 2.2.0 Issue resolved by pull request 17160

[jira] [Resolved] (SPARK-19535) ALSModel recommendAll analogs

[ https://issues.apache.org/jira/browse/SPARK-19535?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley resolved SPARK-19535. --- Resolution: Fixed Fix Version/s: 2.2.0 Issue resolved by pull request 17090

[jira] [Updated] (SPARK-19828) R to support JSON array in column from_json

[ https://issues.apache.org/jira/browse/SPARK-19828?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung updated SPARK-19828: - Summary: R to support JSON array in column from_json (was: R support JSON array in column

[jira] [Commented] (SPARK-19828) R support JSON array in column from_json

[ https://issues.apache.org/jira/browse/SPARK-19828?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15896622#comment-15896622 ] Felix Cheung commented on SPARK-19828: -- see SPARK-19595 > R support JSON array in column from_json

[jira] [Created] (SPARK-19828) R support JSON array in column from_json

Felix Cheung created SPARK-19828: Summary: R support JSON array in column from_json Key: SPARK-19828 URL: https://issues.apache.org/jira/browse/SPARK-19828 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-18085) Better History Server scalability for many / large applications

[ https://issues.apache.org/jira/browse/SPARK-18085?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15896619#comment-15896619 ] Marcelo Vanzin commented on SPARK-18085: Not sure what you mean by "a separate jar file". There's

[jira] [Updated] (SPARK-19765) UNCACHE TABLE should also un-cache all cached plans that refer to this table

[ https://issues.apache.org/jira/browse/SPARK-19765?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan updated SPARK-19765: Summary: UNCACHE TABLE should also un-cache all cached plans that refer to this table (was:

[jira] [Commented] (SPARK-19656) Can't load custom type from avro file to RDD with newAPIHadoopFile

[ https://issues.apache.org/jira/browse/SPARK-19656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15896590#comment-15896590 ] Nira Amit commented on SPARK-19656: --- I will not, but please consider documenting the correct way to

[jira] [Closed] (SPARK-19656) Can't load custom type from avro file to RDD with newAPIHadoopFile

[ https://issues.apache.org/jira/browse/SPARK-19656?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen closed SPARK-19656. - > Can't load custom type from avro file to RDD with newAPIHadoopFile >

[jira] [Resolved] (SPARK-19595) from_json produces only a single row when input is a json array

[ https://issues.apache.org/jira/browse/SPARK-19595?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Burak Yavuz resolved SPARK-19595. - Resolution: Fixed Fix Version/s: 2.2.0 Resolved by

[jira] [Assigned] (SPARK-19595) from_json produces only a single row when input is a json array

[ https://issues.apache.org/jira/browse/SPARK-19595?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Burak Yavuz reassigned SPARK-19595: --- Assignee: Hyukjin Kwon > from_json produces only a single row when input is a json array >

[jira] [Reopened] (SPARK-19656) Can't load custom type from avro file to RDD with newAPIHadoopFile

[ https://issues.apache.org/jira/browse/SPARK-19656?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen reopened SPARK-19656: --- > Can't load custom type from avro file to RDD with newAPIHadoopFile >

[jira] [Resolved] (SPARK-19656) Can't load custom type from avro file to RDD with newAPIHadoopFile

[ https://issues.apache.org/jira/browse/SPARK-19656?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-19656. --- Resolution: Not A Problem > Can't load custom type from avro file to RDD with newAPIHadoopFile >

[jira] [Resolved] (SPARK-19656) Can't load custom type from avro file to RDD with newAPIHadoopFile

[ https://issues.apache.org/jira/browse/SPARK-19656?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-19656. --- Resolution: Fixed I do not see anything surprising given your description. Please don't reopen

[jira] [Closed] (SPARK-19656) Can't load custom type from avro file to RDD with newAPIHadoopFile

[ https://issues.apache.org/jira/browse/SPARK-19656?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen closed SPARK-19656. - > Can't load custom type from avro file to RDD with newAPIHadoopFile >

[jira] [Commented] (SPARK-19656) Can't load custom type from avro file to RDD with newAPIHadoopFile

[ https://issues.apache.org/jira/browse/SPARK-19656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15896581#comment-15896581 ] Nira Amit commented on SPARK-19656: --- I found a problem in my schema and managed to load my custom type.

[jira] [Updated] (SPARK-19656) Can't load custom type from avro file to RDD with newAPIHadoopFile

[ https://issues.apache.org/jira/browse/SPARK-19656?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nira Amit updated SPARK-19656: -- Attachment: (was: datum2.png) > Can't load custom type from avro file to RDD with newAPIHadoopFile

[jira] [Issue Comment Deleted] (SPARK-19656) Can't load custom type from avro file to RDD with newAPIHadoopFile

[ https://issues.apache.org/jira/browse/SPARK-19656?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nira Amit updated SPARK-19656: -- Comment: was deleted (was: {code} public static class ABgoEventAvroReader extends

[jira] [Updated] (SPARK-19656) Can't load custom type from avro file to RDD with newAPIHadoopFile

[ https://issues.apache.org/jira/browse/SPARK-19656?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nira Amit updated SPARK-19656: -- Attachment: (was: datum.png) > Can't load custom type from avro file to RDD with newAPIHadoopFile

[jira] [Issue Comment Deleted] (SPARK-19656) Can't load custom type from avro file to RDD with newAPIHadoopFile

[ https://issues.apache.org/jira/browse/SPARK-19656?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nira Amit updated SPARK-19656: -- Comment: was deleted (was: [~srowen] Will you at least consider the possibility that I'm on to a real

[jira] [Resolved] (SPARK-19795) R should support column functions to_json, from_json

[ https://issues.apache.org/jira/browse/SPARK-19795?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung resolved SPARK-19795. -- Resolution: Fixed Fix Version/s: 2.2.0 Target Version/s: 2.2.0 > R should

[jira] [Updated] (SPARK-19827) spark.ml R API for PIC

[ https://issues.apache.org/jira/browse/SPARK-19827?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung updated SPARK-19827: - Shepherd: Felix Cheung > spark.ml R API for PIC > -- > >

[jira] [Updated] (SPARK-19825) spark.ml R API for FPGrowth

[ https://issues.apache.org/jira/browse/SPARK-19825?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung updated SPARK-19825: - Shepherd: Felix Cheung > spark.ml R API for FPGrowth > --- > >

[jira] [Created] (SPARK-19827) spark.ml R API for PIC

Felix Cheung created SPARK-19827: Summary: spark.ml R API for PIC Key: SPARK-19827 URL: https://issues.apache.org/jira/browse/SPARK-19827 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-19826) spark.ml Python API for PIC

Felix Cheung created SPARK-19826: Summary: spark.ml Python API for PIC Key: SPARK-19826 URL: https://issues.apache.org/jira/browse/SPARK-19826 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-19825) spark.ml R API for FPGrowth

Felix Cheung created SPARK-19825: Summary: spark.ml R API for FPGrowth Key: SPARK-19825 URL: https://issues.apache.org/jira/browse/SPARK-19825 Project: Spark Issue Type: Sub-task

[jira] [Updated] (SPARK-19656) Can't load custom type from avro file to RDD with newAPIHadoopFile

[ https://issues.apache.org/jira/browse/SPARK-19656?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nira Amit updated SPARK-19656: -- Attachment: datum2.png > Can't load custom type from avro file to RDD with newAPIHadoopFile >

[jira] [Updated] (SPARK-19656) Can't load custom type from avro file to RDD with newAPIHadoopFile

[ https://issues.apache.org/jira/browse/SPARK-19656?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nira Amit updated SPARK-19656: -- Attachment: datum.png {code} public static class ABgoEventAvroReader extends

[jira] [Reopened] (SPARK-19656) Can't load custom type from avro file to RDD with newAPIHadoopFile

[ https://issues.apache.org/jira/browse/SPARK-19656?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nira Amit reopened SPARK-19656: --- [~srowen] Will you at least consider the possibility that I'm on to a real problem here? I may be wrong

[jira] [Commented] (SPARK-14083) Analyze JVM bytecode and turn closures into Catalyst expressions

[ https://issues.apache.org/jira/browse/SPARK-14083?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15896452#comment-15896452 ] Kazuaki Ishizaki commented on SPARK-14083: -- Does anyone go forward with this? If not, I will

[jira] [Commented] (SPARK-19656) Can't load custom type from avro file to RDD with newAPIHadoopFile

[ https://issues.apache.org/jira/browse/SPARK-19656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15896343#comment-15896343 ] Sean Owen commented on SPARK-19656: --- It accepts it because you tell it that's what the InputFormat will

[jira] [Closed] (SPARK-19656) Can't load custom type from avro file to RDD with newAPIHadoopFile

[ https://issues.apache.org/jira/browse/SPARK-19656?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen closed SPARK-19656. - > Can't load custom type from avro file to RDD with newAPIHadoopFile >

[jira] [Commented] (SPARK-19656) Can't load custom type from avro file to RDD with newAPIHadoopFile

[ https://issues.apache.org/jira/browse/SPARK-19656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15896314#comment-15896314 ] Nira Amit commented on SPARK-19656: --- And by the way, "what is in the file" is bytes. The question is

[jira] [Assigned] (SPARK-19714) Bucketizer Bug Regarding Handling Unbucketed Inputs

[ https://issues.apache.org/jira/browse/SPARK-19714?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19714: Assignee: Apache Spark > Bucketizer Bug Regarding Handling Unbucketed Inputs >

[jira] [Commented] (SPARK-19714) Bucketizer Bug Regarding Handling Unbucketed Inputs

[ https://issues.apache.org/jira/browse/SPARK-19714?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15896290#comment-15896290 ] Apache Spark commented on SPARK-19714: -- User 'wojtek-szymanski' has created a pull request for this

[jira] [Assigned] (SPARK-19714) Bucketizer Bug Regarding Handling Unbucketed Inputs

[ https://issues.apache.org/jira/browse/SPARK-19714?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19714: Assignee: (was: Apache Spark) > Bucketizer Bug Regarding Handling Unbucketed Inputs >

[jira] [Comment Edited] (SPARK-19656) Can't load custom type from avro file to RDD with newAPIHadoopFile

[ https://issues.apache.org/jira/browse/SPARK-19656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15896286#comment-15896286 ] Nira Amit edited comment on SPARK-19656 at 3/5/17 2:55 PM: --- But then why does

[jira] [Comment Edited] (SPARK-19656) Can't load custom type from avro file to RDD with newAPIHadoopFile

[ https://issues.apache.org/jira/browse/SPARK-19656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15896286#comment-15896286 ] Nira Amit edited comment on SPARK-19656 at 3/5/17 2:56 PM: --- But then why does

[jira] [Commented] (SPARK-19656) Can't load custom type from avro file to RDD with newAPIHadoopFile

[ https://issues.apache.org/jira/browse/SPARK-19656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15896286#comment-15896286 ] Nira Amit commented on SPARK-19656: --- But then why does the compiler accepts what newAPIHadoopFile

[jira] [Commented] (SPARK-19656) Can't load custom type from avro file to RDD with newAPIHadoopFile

[ https://issues.apache.org/jira/browse/SPARK-19656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15896282#comment-15896282 ] Sean Owen commented on SPARK-19656: --- Well, at the least, I'd suggest posting a more compilable example.

[jira] [Commented] (SPARK-19656) Can't load custom type from avro file to RDD with newAPIHadoopFile

[ https://issues.apache.org/jira/browse/SPARK-19656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15896277#comment-15896277 ] Nira Amit commented on SPARK-19656: --- The only reason my code sample doesn't compile is because it

[jira] [Commented] (SPARK-19656) Can't load custom type from avro file to RDD with newAPIHadoopFile

[ https://issues.apache.org/jira/browse/SPARK-19656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15896272#comment-15896272 ] Sean Owen commented on SPARK-19656: --- PS I should be concrete about why I think the original code

[jira] [Commented] (SPARK-19656) Can't load custom type from avro file to RDD with newAPIHadoopFile

[ https://issues.apache.org/jira/browse/SPARK-19656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15896268#comment-15896268 ] Sean Owen commented on SPARK-19656: --- Yes, I just tried to compile your code example above, and it

[jira] [Commented] (SPARK-19656) Can't load custom type from avro file to RDD with newAPIHadoopFile

[ https://issues.apache.org/jira/browse/SPARK-19656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15896266#comment-15896266 ] Nira Amit commented on SPARK-19656: --- [~sowen] I have been trying this for weeks every way I could

[jira] [Commented] (SPARK-19656) Can't load custom type from avro file to RDD with newAPIHadoopFile

[ https://issues.apache.org/jira/browse/SPARK-19656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15896262#comment-15896262 ] Nira Amit commented on SPARK-19656: --- Thanks Eric, but my question is about RDDs. Is it correct that it

[jira] [Commented] (SPARK-16102) Use Record API from Univocity rather than current data cast API.

[ https://issues.apache.org/jira/browse/SPARK-16102?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15896247#comment-15896247 ] Hyukjin Kwon commented on SPARK-16102: -- This takes longer than I thought. Let me update this soon.

[jira] [Resolved] (SPARK-19254) Support Seq, Map, and Struct in functions.lit

[ https://issues.apache.org/jira/browse/SPARK-19254?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell resolved SPARK-19254. --- Resolution: Fixed Assignee: Takeshi Yamamuro Fix Version/s: 2.2.0 >

[jira] [Comment Edited] (SPARK-19656) Can't load custom type from avro file to RDD with newAPIHadoopFile

[ https://issues.apache.org/jira/browse/SPARK-19656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15896194#comment-15896194 ] Eric Maynard edited comment on SPARK-19656 at 3/5/17 11:51 AM: --- Here is a

[jira] [Commented] (SPARK-19656) Can't load custom type from avro file to RDD with newAPIHadoopFile

[ https://issues.apache.org/jira/browse/SPARK-19656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15896194#comment-15896194 ] Eric Maynard commented on SPARK-19656: -- Here is a complete working example in Java:

[jira] [Commented] (SPARK-19824) Standalone master JSON not showing cores for running applications

[ https://issues.apache.org/jira/browse/SPARK-19824?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15896193#comment-15896193 ] Sean Owen commented on SPARK-19824: --- I guess it doesn't show "memory per executor" either? That came up

[jira] [Resolved] (SPARK-19805) Log the row type when query result dose not match

[ https://issues.apache.org/jira/browse/SPARK-19805?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell resolved SPARK-19805. --- Resolution: Fixed Assignee: Genmao Yu Fix Version/s: 2.2.0 > Log the

[jira] [Commented] (SPARK-19823) Support Gang Distribution of Task

[ https://issues.apache.org/jira/browse/SPARK-19823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15896192#comment-15896192 ] Sean Owen commented on SPARK-19823: --- I don't think that's quite right. Task assignment takes into

[jira] [Commented] (SPARK-6407) Streaming ALS for Collaborative Filtering

[ https://issues.apache.org/jira/browse/SPARK-6407?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15896190#comment-15896190 ] Sean Owen commented on SPARK-6407: -- I did some work on this, but it's not a paper or anything, just some

[jira] [Commented] (SPARK-19656) Can't load custom type from avro file to RDD with newAPIHadoopFile

[ https://issues.apache.org/jira/browse/SPARK-19656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15896186#comment-15896186 ] Sean Owen commented on SPARK-19656: --- I guess I mean, have you really tried it? it doesn't result in a

[jira] [Commented] (SPARK-19656) Can't load custom type from avro file to RDD with newAPIHadoopFile

[ https://issues.apache.org/jira/browse/SPARK-19656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15896179#comment-15896179 ] Nira Amit commented on SPARK-19656: --- Yes, I did, and answered him that it gives a compilation error in

[jira] [Comment Edited] (SPARK-6407) Streaming ALS for Collaborative Filtering

[ https://issues.apache.org/jira/browse/SPARK-6407?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15896177#comment-15896177 ] Daniel Li edited comment on SPARK-6407 at 3/5/17 10:57 AM: --- {quote} In practice

[jira] [Commented] (SPARK-6407) Streaming ALS for Collaborative Filtering

[ https://issues.apache.org/jira/browse/SPARK-6407?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15896177#comment-15896177 ] Daniel Li commented on SPARK-6407: -- bq. In practice fold-in works fine. Folding in a day or so of updates

[jira] [Commented] (SPARK-19656) Can't load custom type from avro file to RDD with newAPIHadoopFile

[ https://issues.apache.org/jira/browse/SPARK-19656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15896174#comment-15896174 ] Sean Owen commented on SPARK-19656: --- Have you tried Eric's suggestion? asInstanceOf is just casting in

[jira] [Commented] (SPARK-19656) Can't load custom type from avro file to RDD with newAPIHadoopFile

[ https://issues.apache.org/jira/browse/SPARK-19656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15896172#comment-15896172 ] Nira Amit commented on SPARK-19656: --- But if this is not possible to do in Java then it IS an actionable

[jira] [Commented] (SPARK-19656) Can't load custom type from avro file to RDD with newAPIHadoopFile

[ https://issues.apache.org/jira/browse/SPARK-19656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15896169#comment-15896169 ] Sean Owen commented on SPARK-19656: --- Mostly, it is that questions should go to the mailing list. I

[jira] [Commented] (SPARK-19656) Can't load custom type from avro file to RDD with newAPIHadoopFile

[ https://issues.apache.org/jira/browse/SPARK-19656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15896166#comment-15896166 ] Nira Amit commented on SPARK-19656: --- [~emaynard] There is no "asInstanceOf" method in the Java API. And

[jira] [Resolved] (SPARK-19656) Can't load custom type from avro file to RDD with newAPIHadoopFile

[ https://issues.apache.org/jira/browse/SPARK-19656?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-19656. --- Resolution: Not A Problem > Can't load custom type from avro file to RDD with newAPIHadoopFile >

[jira] [Assigned] (SPARK-19792) In the Master Page,the column named “Memory per Node” ,I think it is not all right

[ https://issues.apache.org/jira/browse/SPARK-19792?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen reassigned SPARK-19792: - Assignee: liuxian > In the Master Page,the column named “Memory per Node” ,I think it is not

[jira] [Resolved] (SPARK-19792) In the Master Page,the column named “Memory per Node” ,I think it is not all right

[ https://issues.apache.org/jira/browse/SPARK-19792?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-19792. --- Resolution: Fixed Fix Version/s: 2.2.0 Issue resolved by pull request 17132

[jira] [Resolved] (SPARK-19713) saveAsTable

[ https://issues.apache.org/jira/browse/SPARK-19713?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-19713. --- Resolution: Not A Problem > saveAsTable > --- > > Key: SPARK-19713 >

[jira] [Commented] (SPARK-6407) Streaming ALS for Collaborative Filtering

[ https://issues.apache.org/jira/browse/SPARK-6407?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15896154#comment-15896154 ] Sean Owen commented on SPARK-6407: -- Computing one or two iterations per update -- as in every time

[jira] [Updated] (SPARK-19737) New analysis rule for reporting unregistered functions without relying on relation resolution

[ https://issues.apache.org/jira/browse/SPARK-19737?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-19737: --- Description: Let's consider the following simple SQL query that reference an undefined function

[jira] [Updated] (SPARK-19737) New analysis rule for reporting unregistered functions without relying on relation resolution

[ https://issues.apache.org/jira/browse/SPARK-19737?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-19737: --- Description: Let's consider the following simple SQL query that reference an undefined function

  1   2   >