Re: Unsupported Catalyst types in Parquet

2014-12-30 Thread Alessandro Baretta
@spark.apache.org *Subject:* Re: Unsupported Catalyst types in Parquet Yeah, I saw those. The problem is that #3822 truncates timestamps that include nanoseconds. On Mon, Dec 29, 2014 at 5:14 PM, Alessandro Baretta alexbare...@gmail.com wrote: Michael, Actually, Adrian Wang already

Re: Unsupported Catalyst types in Parquet

2014-12-30 Thread Alessandro Baretta
there. Thanks, Daoyuan *From:* Michael Armbrust [mailto:mich...@databricks.com] *Sent:* Tuesday, December 30, 2014 10:43 AM *To:* Alessandro Baretta *Cc:* Wang, Daoyuan; dev@spark.apache.org *Subject:* Re: Unsupported Catalyst types in Parquet Yeah, I saw those. The problem is that #3822

Re: Unsupported Catalyst types in Parquet

2014-12-30 Thread Alessandro Baretta
there. Thanks, Daoyuan *From:* Michael Armbrust [mailto:mich...@databricks.com] *Sent:* Tuesday, December 30, 2014 10:43 AM *To:* Alessandro Baretta *Cc:* Wang, Daoyuan; dev@spark.apache.org *Subject:* Re: Unsupported Catalyst types in Parquet Yeah, I saw those. The problem

Re: Unsupported Catalyst types in Parquet

2014-12-30 Thread Alessandro Baretta
SQLContext, so that we can put more flags there. Thanks, Daoyuan *From:* Michael Armbrust [mailto:mich...@databricks.com] *Sent:* Tuesday, December 30, 2014 10:43 AM *To:* Alessandro Baretta *Cc:* Wang, Daoyuan; dev@spark.apache.org *Subject:* Re: Unsupported Catalyst types in Parquet

RE: Unsupported Catalyst types in Parquet

2014-12-29 Thread Wang, Daoyuan
: Saturday, December 27, 2014 2:47 PM To: dev@spark.apache.org; Michael Armbrust Subject: Unsupported Catalyst types in Parquet Michael, I'm having trouble storing my SchemaRDDs in Parquet format with SparkSQL, due to my RDDs having having DateType and DecimalType fields. What would it take to add

RE: Unsupported Catalyst types in Parquet

2014-12-29 Thread Alessandro Baretta
] Sent: Saturday, December 27, 2014 2:47 PM To: dev@spark.apache.org; Michael Armbrust Subject: Unsupported Catalyst types in Parquet Michael, I'm having trouble storing my SchemaRDDs in Parquet format with SparkSQL, due to my RDDs having having DateType and DecimalType fields. What would

Re: Unsupported Catalyst types in Parquet

2014-12-29 Thread Michael Armbrust
: Unsupported Catalyst types in Parquet Michael, I'm having trouble storing my SchemaRDDs in Parquet format with SparkSQL, due to my RDDs having having DateType and DecimalType fields. What would it take to add Parquet support for these Catalyst? Are there any other Catalyst types for which

Re: Unsupported Catalyst types in Parquet

2014-12-29 Thread Alessandro Baretta
Catalyst types in Parquet Michael, I'm having trouble storing my SchemaRDDs in Parquet format with SparkSQL, due to my RDDs having having DateType and DecimalType fields. What would it take to add Parquet support for these Catalyst? Are there any other Catalyst types for which there is no Catalyst

RE: Unsupported Catalyst types in Parquet

2014-12-29 Thread Wang, Daoyuan
10:43 AM To: Alessandro Baretta Cc: Wang, Daoyuan; dev@spark.apache.org Subject: Re: Unsupported Catalyst types in Parquet Yeah, I saw those. The problem is that #3822 truncates timestamps that include nanoseconds. On Mon, Dec 29, 2014 at 5:14 PM, Alessandro Baretta alexbare

Unsupported Catalyst types in Parquet

2014-12-26 Thread Alessandro Baretta
Michael, I'm having trouble storing my SchemaRDDs in Parquet format with SparkSQL, due to my RDDs having having DateType and DecimalType fields. What would it take to add Parquet support for these Catalyst? Are there any other Catalyst types for which there is no Catalyst support? Alex