Thanks for all the help! I will try both ways and let you all know what worked best.
From: Sean Busbey [mailto:bus...@cloudera.com] Sent: Wednesday, July 09, 2014 3:41 AM To: user@avro apache. org Subject: Re: COnverting CSV files to avro and back to text You could consider using the KiteSDK[1]. It uses Avro schemas to describe datasets, can infer an avro schema from CSV headers, and includes tools for importing from CSV to Avro for storage. There's a tool demo that can walk you through getting things into avro and displaying a plain text version[2]. The current MapReduce support is a first pass, but is enough to start playing on[3]. Unfortunately, I don't think it has a demo walkthrough yet. The next release is supposed to be more user friendly. [1]: http://kitesdk.org/docs/current/kite-data/guide.html [2]: http://kitesdk.org/docs/current/usingkiteclicreatedataset.html [3]: http://kitesdk.org/docs/current/apidocs/org/kitesdk/data/mapreduce/DatasetKeyInputFormat.html On Mon, Jul 7, 2014 at 4:13 PM, Bhuvana Bellala <bhuvana.bell...@cormac-corp.com<mailto:bhuvana.bell...@cormac-corp.com>> wrote: Hi, I want convert a CSV file into avro format so that I can compress data and do some map reduce things in Hadoop. And then I want to convert the avro data back to text. Are there any tools to do this? I know I could use hive to do this but I want to know if there are any other ways to do this? Thank! [Cormac_logo] Bhuvana Bellala| Intern ________________________________ 13921 Park Center Road, Suite 180, Herndon VA 20171| CMMI Maturity Level 3 Rated| GSA Schedule 70| SBA SDB, 8(a)| www.cormac-corp.com<http://www.cormac-corp.com/> This email and any files transmitted with it are confidential and intended solely for the use of the individual or entity to whom they are addressed. If you are not the named addressee you should not disseminate, distribute or copy this e-mail. Please notify the sender immediately by e-mail if you have received this e-mail by mistake and delete this e-mail from your system. If you are not the intended recipient you are notified that disclosing, copying, distributing or taking any action in reliance on the contents of this information is strictly prohibited. -- Sean
