Hi,
My opinion is that it's a very reasonable requirement for control
characters to be escaped in JSON, on any other format for that matter.
Drill uses new-line as a record separator in JSON and other formats (CSV
for example) and it should not have to know that it's inside quotation
marks or not.
After a brief googling session it seems that all good JSON serializers do
this and that the problem is mainly due to the content.
I know this does not help much but I think you are better off with "fixing"
the content rather than looking for tools that deal with it.
Regards,
-Stefan
On Wed, Jan 6, 2016 at 8:26 AM, <masahide.mi...@brother.co.jp> wrote:
> Hi, Stefan
>
> > is it possible that your JSON serializer is not escaping characters that
> should be escaped?
> No. I confirmed my JSON, but it isn't.
> More simply, I tested like bellow.
>
> Query:
> select * from dfs.root.`test.json`
> where a like 'test%'
>
> Target JSON:
> {"a": "test value"}
>
> => Returns this record.
>
> Target JSON:
> I edited same file and insert '\n ' to value.
>
> {"a": "test \n value"}
>
> => No result found.
> # This is valid JSON, I believe.... isn't it?
>
> Target JSON:
> Same file with copied row and edited.
>
> {"a": "test value"}
> {"a": "test \n value"}
>
> => Returns 1 record ("test value")
>
> Thank you.
>
> --
> Miura, Masahide
>
>
> -----Original Message-----
> From: Stefán Baxter [mailto:ste...@activitystream.com]
> Sent: Wednesday, January 06, 2016 4:54 PM
> To: user
> Subject: Re: Does drill recognize new line correctly?
>
> Hi,
>
> I'm not the right person to give you an answer from the Drill perspective
> but is it possible that your JSON serializer is not escaping characters
> that should be escaped?
>
> please see:
>
> -
>
> http://stackoverflow.com/questions/4253367/how-to-escape-a-json-string-containing-newline-characters-using-javascript
> -
>
> http://stackoverflow.com/questions/5506000/javascript-json-stringify-doesnt-escap
>
> Regards,
> -Stefan
>
>
>
> On Wed, Jan 6, 2016 at 4:10 AM, <masahide.mi...@brother.co.jp> wrote:
>
> > Happy new year!
> > We, Japanese like new year's greeting ;-)
> >
> > There's two issues in this message.
> >
> > First, CSV file which has value includes new line.
> > The other, JSON file which has value includes new line.
> >
> > 1) CSV
> > Doesn't drill recognize CSV which has some columns including new line?
> > For example, CSV file exported from MS-Excel.
> >
> > I tried some patterns. Quoting column, escaping by \ (like \[LF]),
> > replacing \r or \n...
> > But all of those are not good for me.
> >
> > By the way, new lines are approved in CSV columns by RFC, you know.
> > * https://tools.ietf.org/html/rfc4180
> > Then I would like to parse such CSV though I know it is
> > informational definition.
> >
> > 2) JSON
> > It can't query correctly to JSON with records include new line.
> >
> > JSON:
> > { "key": "test record with \n newline" }
> >
> > Query:
> > select * from dfs.`test.json` where key like 'test%'
> >
> > Result:
> > No result found
> >
> > It doesn't compare value correctly if it includes new line, I think.
> >
> > Do you know how to use new lines in values as expected?
> >
> > Thank you.
> >
> > --
> > Miura, Masahide
> >
> >
>
