Ok, I'll create a Hive jira for it then. I can't find a Hadoop one for adding custom line separators in the TextInputFormat either, if there is none I'll create that too.
/Johan Zheng Shao wrote: > Yes this is a known issue. "lines terminated by" is not supported yet > because the text input format do not allow configurable line > separators yet. > > > Zheng > > > On 2/24/09, Johan Oskarsson <[email protected]> wrote: >> I've been trying to use a text file with the field separator \001 and >> line separator \002\n in Hive, similar to what's described here >> http://wiki.apache.org/hadoop/Hive/LanguageManual/DDL. >> >> I've set the "lines terminated by" to \002 but when selecting data the >> last column still includes that character, if the last col is a string. >> If it's an int the value fails to parse and is left as null. >> >> Is this a known issue? I can't find a ticket for it. >> I assume the TextInputFormat takes care of the \n so that I only need to >> use \002 as the termination. >> >> Example create queries >> >> create table artistsong (export_date bigint, artist_id int, song_id int) >> ROW FORMAT DELIMITED FIELDS TERMINATED BY '\001' lines terminated by '\002' >> >> and this one: >> >> create table artist (export_date bigint, artist_id int, name string, >> is_actual_artist int, view_url string) ROW FORMAT DELIMITED FIELDS >> TERMINATED BY '\001' lines terminated by '\002' >> >> /Johan >> >
