[jira] [Created] (SPARK-16700) StructType doesn't accept Python dicts anymore

2016-07-24 Thread Sylvain Zimmer (JIRA)
Sylvain Zimmer created SPARK-16700: -- Summary: StructType doesn't accept Python dicts anymore Key: SPARK-16700 URL: https://issues.apache.org/jira/browse/SPARK-16700 Project: Spark Issue Type

[jira] [Updated] (SPARK-16700) StructType doesn't accept Python dicts anymore

2016-07-24 Thread Sylvain Zimmer (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16700?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sylvain Zimmer updated SPARK-16700: --- Component/s: PySpark > StructType doesn't accept Python dicts anymore > -

[jira] [Updated] (SPARK-16700) StructType doesn't accept Python dicts anymore

2016-07-24 Thread Sylvain Zimmer (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16700?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sylvain Zimmer updated SPARK-16700: --- Description: Hello, I found this issue while testing my codebase with 2.0.0-rc5 StructType

[jira] [Updated] (SPARK-16700) StructType doesn't accept Python dicts anymore

2016-07-24 Thread Sylvain Zimmer (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16700?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sylvain Zimmer updated SPARK-16700: --- Component/s: (was: Spark Core) > StructType doesn't accept Python dicts anymore > ---

[jira] [Commented] (SPARK-16700) StructType doesn't accept Python dicts anymore

2016-07-25 Thread Sylvain Zimmer (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16700?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15392200#comment-15392200 ] Sylvain Zimmer commented on SPARK-16700: I dug into this a bit more: {{_verify_

[jira] [Created] (SPARK-16740) joins.LongToUnsafeRowMap crashes with NegativeArraySizeException

2016-07-26 Thread Sylvain Zimmer (JIRA)
Sylvain Zimmer created SPARK-16740: -- Summary: joins.LongToUnsafeRowMap crashes with NegativeArraySizeException Key: SPARK-16740 URL: https://issues.apache.org/jira/browse/SPARK-16740 Project: Spark

[jira] [Commented] (SPARK-16740) joins.LongToUnsafeRowMap crashes with NegativeArraySizeException

2016-07-26 Thread Sylvain Zimmer (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15394437#comment-15394437 ] Sylvain Zimmer commented on SPARK-16740: I'm not a Scala expert but from a quick

[jira] [Comment Edited] (SPARK-16740) joins.LongToUnsafeRowMap crashes with NegativeArraySizeException

2016-07-26 Thread Sylvain Zimmer (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15394437#comment-15394437 ] Sylvain Zimmer edited comment on SPARK-16740 at 7/26/16 7:54 PM: --

[jira] [Commented] (SPARK-16740) joins.LongToUnsafeRowMap crashes with NegativeArraySizeException

2016-07-26 Thread Sylvain Zimmer (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15394628#comment-15394628 ] Sylvain Zimmer commented on SPARK-16740: Thanks! I just did. Let me know if that'

[jira] [Commented] (SPARK-16740) joins.LongToUnsafeRowMap crashes with NegativeArraySizeException

2016-07-26 Thread Sylvain Zimmer (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15394644#comment-15394644 ] Sylvain Zimmer commented on SPARK-16740: OK! Looks like that would be [~davies] :

[jira] [Created] (SPARK-16802) joins.LongToUnsafeRowMap crashes with ArrayIndexOutOfBoundsException

2016-07-29 Thread Sylvain Zimmer (JIRA)
Sylvain Zimmer created SPARK-16802: -- Summary: joins.LongToUnsafeRowMap crashes with ArrayIndexOutOfBoundsException Key: SPARK-16802 URL: https://issues.apache.org/jira/browse/SPARK-16802 Project: Spa

[jira] [Updated] (SPARK-16802) joins.LongToUnsafeRowMap crashes with ArrayIndexOutOfBoundsException

2016-07-29 Thread Sylvain Zimmer (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16802?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sylvain Zimmer updated SPARK-16802: --- Description: Hello! This is a little similar to [SPARK-16740|https://issues.apache.org/jira

[jira] [Updated] (SPARK-16802) joins.LongToUnsafeRowMap crashes with ArrayIndexOutOfBoundsException

2016-07-29 Thread Sylvain Zimmer (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16802?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sylvain Zimmer updated SPARK-16802: --- Description: Hello! This is a little similar to [SPARK-16740|https://issues.apache.org/jira

[jira] [Created] (SPARK-16807) Optimize some ABS() statements

2016-07-29 Thread Sylvain Zimmer (JIRA)
Sylvain Zimmer created SPARK-16807: -- Summary: Optimize some ABS() statements Key: SPARK-16807 URL: https://issues.apache.org/jira/browse/SPARK-16807 Project: Spark Issue Type: Improvement

[jira] [Updated] (SPARK-16807) Optimize some ABS() statements

2016-07-29 Thread Sylvain Zimmer (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16807?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sylvain Zimmer updated SPARK-16807: --- Description: I'm not a Catalyst expert, but I think some use cases for the ABS() function co

[jira] [Updated] (SPARK-16807) Optimize some ABS() statements

2016-07-29 Thread Sylvain Zimmer (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16807?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sylvain Zimmer updated SPARK-16807: --- Priority: Minor (was: Major) > Optimize some ABS() statements >

[jira] [Commented] (SPARK-16802) joins.LongToUnsafeRowMap crashes with ArrayIndexOutOfBoundsException

2016-07-29 Thread Sylvain Zimmer (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16802?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15400260#comment-15400260 ] Sylvain Zimmer commented on SPARK-16802: Maybe useful for others: an ugly workaro

[jira] [Created] (SPARK-16826) java.util.Hashtable limits the throughput of PARSE_URL()

2016-07-31 Thread Sylvain Zimmer (JIRA)
Sylvain Zimmer created SPARK-16826: -- Summary: java.util.Hashtable limits the throughput of PARSE_URL() Key: SPARK-16826 URL: https://issues.apache.org/jira/browse/SPARK-16826 Project: Spark

[jira] [Updated] (SPARK-16826) java.util.Hashtable limits the throughput of PARSE_URL()

2016-07-31 Thread Sylvain Zimmer (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16826?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sylvain Zimmer updated SPARK-16826: --- Description: Hello! I'm using {{c4.8xlarge}} instances on EC2 with 36 cores and doing lots o

[jira] [Updated] (SPARK-16826) java.util.Hashtable limits the throughput of PARSE_URL()

2016-07-31 Thread Sylvain Zimmer (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16826?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sylvain Zimmer updated SPARK-16826: --- Description: Hello! I'm using {{c4.8xlarge}} instances on EC2 with 36 cores and doing lots o

[jira] [Updated] (SPARK-16826) java.util.Hashtable limits the throughput of PARSE_URL()

2016-07-31 Thread Sylvain Zimmer (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16826?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sylvain Zimmer updated SPARK-16826: --- Description: Hello! I'm using {{c4.8xlarge}} instances on EC2 with 36 cores and doing lots o

[jira] [Commented] (SPARK-16826) java.util.Hashtable limits the throughput of PARSE_URL()

2016-08-01 Thread Sylvain Zimmer (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16826?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15403165#comment-15403165 ] Sylvain Zimmer commented on SPARK-16826: [~srowen] thanks for the pointers! I'm

[jira] [Comment Edited] (SPARK-16826) java.util.Hashtable limits the throughput of PARSE_URL()

2016-08-01 Thread Sylvain Zimmer (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16826?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15403165#comment-15403165 ] Sylvain Zimmer edited comment on SPARK-16826 at 8/2/16 1:15 AM: ---

[jira] [Commented] (SPARK-16826) java.util.Hashtable limits the throughput of PARSE_URL()

2016-08-01 Thread Sylvain Zimmer (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16826?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15403236#comment-15403236 ] Sylvain Zimmer commented on SPARK-16826: Sorry I can't be more helpful on the Jav

[jira] [Commented] (SPARK-16826) java.util.Hashtable limits the throughput of PARSE_URL()

2016-08-01 Thread Sylvain Zimmer (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16826?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15403262#comment-15403262 ] Sylvain Zimmer commented on SPARK-16826: [~srowen] what about this? https://gith

[jira] [Commented] (SPARK-16700) StructType doesn't accept Python dicts anymore

2016-08-03 Thread Sylvain Zimmer (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16700?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15406754#comment-15406754 ] Sylvain Zimmer commented on SPARK-16700: The verifySchema flag works great, and t