Try loading the data in schema one, and storing it/transforming it in schema two in a new location?
Then try to load it, and see what it is. This may not be feasible if your data is large, but worth a try to help debug? Russell Jurney http://datasyndrome.com On Oct 5, 2012, at 7:48 PM, Runping Qi <[email protected]> wrote: > Hi, > > I have the following problem to solve: > > I have data in HDFS written out using AvroStorage with schema1. > I have schema2 which is different than schema1 but they are compatible. > Now, I want to use AvroStorage to load the data and convert to schema2. > How can I achieve that? > > I tried something like: > > data = load 'my_data_dir' using AvroStorage('schema', SCHEMA2_STRING); > describe data; > > The schema for the loaded-in data is still in schema1, not in schema2. > > Any help is highly appreciated. > > Thanks, > > Runping Qi > A9.COM
