Error when Applying schema to a dictionary with a Tuple as key

2014-12-16 Thread sahanbull
Hi Guys, Im running a spark cluster in AWS with Spark 1.1.0 in EC2 I am trying to convert a an RDD with tuple (u'string', int , {(int, int): int, (int, int): int}) to a schema rdd using the schema: fields = [StructField('field1',StringType(),True),

Re: Error when Applying schema to a dictionary with a Tuple as key

2014-12-16 Thread Davies Liu
It's a bug, could you file a JIRA for this? thanks! On Tue, Dec 16, 2014 at 5:49 AM, sahanbull sa...@skimlinks.com wrote: Hi Guys, Im running a spark cluster in AWS with Spark 1.1.0 in EC2 I am trying to convert a an RDD with tuple (u'string', int , {(int, int): int, (int, int): int})

Re: Error when Applying schema to a dictionary with a Tuple as key

2014-12-16 Thread Davies Liu
I had created https://issues.apache.org/jira/browse/SPARK-4866, it will be fixed by https://github.com/apache/spark/pull/3714. Thank you for reporting this. Davies On Tue, Dec 16, 2014 at 12:44 PM, Davies Liu dav...@databricks.com wrote: It's a bug, could you file a JIRA for this? thanks! On