[jira] [Commented] (SPARK-16700) StructType doesn't accept Python dicts anymore

2016-08-03 Thread Sylvain Zimmer (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16700?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15406754#comment-15406754 ] Sylvain Zimmer commented on SPARK-16700: The verifySchema flag works great, and the {{dict}}

[jira] [Commented] (SPARK-16826) java.util.Hashtable limits the throughput of PARSE_URL()

2016-08-01 Thread Sylvain Zimmer (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16826?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15403262#comment-15403262 ] Sylvain Zimmer commented on SPARK-16826: [~srowen] what about this?

[jira] [Commented] (SPARK-16826) java.util.Hashtable limits the throughput of PARSE_URL()

2016-08-01 Thread Sylvain Zimmer (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16826?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15403236#comment-15403236 ] Sylvain Zimmer commented on SPARK-16826: Sorry I can't be more helpful on the Java side... But I

[jira] [Comment Edited] (SPARK-16826) java.util.Hashtable limits the throughput of PARSE_URL()

2016-08-01 Thread Sylvain Zimmer (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16826?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15403165#comment-15403165 ] Sylvain Zimmer edited comment on SPARK-16826 at 8/2/16 1:15 AM: [~srowen]

[jira] [Commented] (SPARK-16826) java.util.Hashtable limits the throughput of PARSE_URL()

2016-08-01 Thread Sylvain Zimmer (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16826?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15403165#comment-15403165 ] Sylvain Zimmer commented on SPARK-16826: [~srowen] thanks for the pointers! I'm parsing every

[jira] [Updated] (SPARK-16826) java.util.Hashtable limits the throughput of PARSE_URL()

2016-07-31 Thread Sylvain Zimmer (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16826?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sylvain Zimmer updated SPARK-16826: --- Description: Hello! I'm using {{c4.8xlarge}} instances on EC2 with 36 cores and doing lots

[jira] [Updated] (SPARK-16826) java.util.Hashtable limits the throughput of PARSE_URL()

2016-07-31 Thread Sylvain Zimmer (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16826?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sylvain Zimmer updated SPARK-16826: --- Description: Hello! I'm using {{c4.8xlarge}} instances on EC2 with 36 cores and doing lots

[jira] [Updated] (SPARK-16826) java.util.Hashtable limits the throughput of PARSE_URL()

2016-07-31 Thread Sylvain Zimmer (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16826?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sylvain Zimmer updated SPARK-16826: --- Description: Hello! I'm using {{c4.8xlarge}} instances on EC2 with 36 cores and doing lots

[jira] [Created] (SPARK-16826) java.util.Hashtable limits the throughput of PARSE_URL()

2016-07-31 Thread Sylvain Zimmer (JIRA)
Sylvain Zimmer created SPARK-16826: -- Summary: java.util.Hashtable limits the throughput of PARSE_URL() Key: SPARK-16826 URL: https://issues.apache.org/jira/browse/SPARK-16826 Project: Spark

[jira] [Commented] (SPARK-16802) joins.LongToUnsafeRowMap crashes with ArrayIndexOutOfBoundsException

2016-07-29 Thread Sylvain Zimmer (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16802?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15400260#comment-15400260 ] Sylvain Zimmer commented on SPARK-16802: Maybe useful for others: an ugly workaround to avoid

[jira] [Updated] (SPARK-16807) Optimize some ABS() statements

2016-07-29 Thread Sylvain Zimmer (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16807?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sylvain Zimmer updated SPARK-16807: --- Priority: Minor (was: Major) > Optimize some ABS() statements >

[jira] [Updated] (SPARK-16807) Optimize some ABS() statements

2016-07-29 Thread Sylvain Zimmer (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16807?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sylvain Zimmer updated SPARK-16807: --- Description: I'm not a Catalyst expert, but I think some use cases for the ABS() function

[jira] [Created] (SPARK-16807) Optimize some ABS() statements

2016-07-29 Thread Sylvain Zimmer (JIRA)
Sylvain Zimmer created SPARK-16807: -- Summary: Optimize some ABS() statements Key: SPARK-16807 URL: https://issues.apache.org/jira/browse/SPARK-16807 Project: Spark Issue Type: Improvement

[jira] [Updated] (SPARK-16802) joins.LongToUnsafeRowMap crashes with ArrayIndexOutOfBoundsException

2016-07-29 Thread Sylvain Zimmer (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16802?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sylvain Zimmer updated SPARK-16802: --- Description: Hello! This is a little similar to

[jira] [Updated] (SPARK-16802) joins.LongToUnsafeRowMap crashes with ArrayIndexOutOfBoundsException

2016-07-29 Thread Sylvain Zimmer (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16802?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sylvain Zimmer updated SPARK-16802: --- Description: Hello! This is a little similar to

[jira] [Created] (SPARK-16802) joins.LongToUnsafeRowMap crashes with ArrayIndexOutOfBoundsException

2016-07-29 Thread Sylvain Zimmer (JIRA)
Sylvain Zimmer created SPARK-16802: -- Summary: joins.LongToUnsafeRowMap crashes with ArrayIndexOutOfBoundsException Key: SPARK-16802 URL: https://issues.apache.org/jira/browse/SPARK-16802 Project:

[jira] [Commented] (SPARK-16740) joins.LongToUnsafeRowMap crashes with NegativeArraySizeException

2016-07-26 Thread Sylvain Zimmer (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15394644#comment-15394644 ] Sylvain Zimmer commented on SPARK-16740: OK! Looks like that would be [~davies] :-) >

[jira] [Commented] (SPARK-16740) joins.LongToUnsafeRowMap crashes with NegativeArraySizeException

2016-07-26 Thread Sylvain Zimmer (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15394628#comment-15394628 ] Sylvain Zimmer commented on SPARK-16740: Thanks! I just did. Let me know if that's okay. >

[jira] [Comment Edited] (SPARK-16740) joins.LongToUnsafeRowMap crashes with NegativeArraySizeException

2016-07-26 Thread Sylvain Zimmer (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15394437#comment-15394437 ] Sylvain Zimmer edited comment on SPARK-16740 at 7/26/16 7:54 PM: - I'm not

[jira] [Commented] (SPARK-16740) joins.LongToUnsafeRowMap crashes with NegativeArraySizeException

2016-07-26 Thread Sylvain Zimmer (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15394437#comment-15394437 ] Sylvain Zimmer commented on SPARK-16740: I'm not a Scala expert but from a quick review of the

[jira] [Created] (SPARK-16740) joins.LongToUnsafeRowMap crashes with NegativeArraySizeException

2016-07-26 Thread Sylvain Zimmer (JIRA)
Sylvain Zimmer created SPARK-16740: -- Summary: joins.LongToUnsafeRowMap crashes with NegativeArraySizeException Key: SPARK-16740 URL: https://issues.apache.org/jira/browse/SPARK-16740 Project: Spark

[jira] [Commented] (SPARK-16700) StructType doesn't accept Python dicts anymore

2016-07-25 Thread Sylvain Zimmer (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16700?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15392200#comment-15392200 ] Sylvain Zimmer commented on SPARK-16700: I dug into this a bit more: {{_verify_type({},

[jira] [Updated] (SPARK-16700) StructType doesn't accept Python dicts anymore

2016-07-24 Thread Sylvain Zimmer (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16700?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sylvain Zimmer updated SPARK-16700: --- Component/s: (was: Spark Core) > StructType doesn't accept Python dicts anymore >

[jira] [Updated] (SPARK-16700) StructType doesn't accept Python dicts anymore

2016-07-24 Thread Sylvain Zimmer (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16700?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sylvain Zimmer updated SPARK-16700: --- Description: Hello, I found this issue while testing my codebase with 2.0.0-rc5 StructType

[jira] [Updated] (SPARK-16700) StructType doesn't accept Python dicts anymore

2016-07-24 Thread Sylvain Zimmer (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16700?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sylvain Zimmer updated SPARK-16700: --- Component/s: PySpark > StructType doesn't accept Python dicts anymore >

[jira] [Created] (SPARK-16700) StructType doesn't accept Python dicts anymore

2016-07-24 Thread Sylvain Zimmer (JIRA)
Sylvain Zimmer created SPARK-16700: -- Summary: StructType doesn't accept Python dicts anymore Key: SPARK-16700 URL: https://issues.apache.org/jira/browse/SPARK-16700 Project: Spark Issue