We upgraded to version 1.5.0 and this is no longer an issue. Also thanks Julien for your suggestions.
On Thu, Jul 31, 2014 at 5:26 PM, Issac Buenrostro <buenros...@ooyala.com> wrote: > Hello, > > We are using com.twitter.parquet-thrift 1.2.5-cdh5.0.0-beta-2 for writing > data into HDFS. Recently, we tried modifying our thrift schema by adding a > single optional field. According to > http://diwakergupta.github.io/thrift-missing-guide/#_versioning_compatibility > this change should not affect compatibility. > > However, when using the new schema to decode old files, we get the > following error: > > [error] (run-main-0) parquet.io.InvalidRecordException: state not found in > message ParquetSchema { > at parquet.schema.GroupType.getFieldIndex(GroupType.java:104) > at parquet.schema.GroupType.getType(GroupType.java:136) > at parquet.schema.GroupType.checkGroupContains(GroupType.java:273) > at parquet.schema.MessageType.checkContains(MessageType.java:126) > at parquet.hadoop.api.ReadSupport.getSchemaForRead(ReadSupport.java:55) > at > parquet.hadoop.thrift.ThriftReadSupport.init(ThriftReadSupport.java:116) > at parquet.hadoop.ParquetReader.<init>(ParquetReader.java:107) > at parquet.hadoop.ParquetReader.<init>(ParquetReader.java:69) > at parquet.thrift.ThriftParquetReader.<init>(ThriftParquetReader.java:70) > ... > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:606) > > The code we are using is: > > val reader = new ThriftParquetReader[ExampleClass](conf, path) > reader.read > > Is parquet-thrift supposed to offer the same backwards compatibility as > thrift? Are we doing something wrong? > > Thank you, > Issac > > -- > -- > *Issac Buenrostro* > Software Engineer | > buenros...@ooyala.com | (617) 997-3350 > www.ooyala.com | blog <http://www.ooyala.com/blog> | @ooyala > <http://www.twitter.com/ooyala> > -- -- *Issac Buenrostro* Software Engineer | buenros...@ooyala.com | (617) 997-3350 www.ooyala.com | blog <http://www.ooyala.com/blog> | @ooyala <http://www.twitter.com/ooyala>
