I have transformed my field with talend to be like this "yyyy-MM-ddTHH:mm:ssZ" and that went good ! Doesn't ES accept another format? other than the one above?
On Tue, Apr 14, 2015 at 5:05 PM, <[email protected]> wrote: > Hi, > > I'm not sure if it will fix your problem but you could try the type > TIMESTAMP for your sendingTime field. > TIMESTAMP type is equivalent to the date elasticsearch type according to > http://www.elastic.co/guide/en/elasticsearch/hadoop/current/hive.html > > Le mardi 14 avril 2015 11:26:49 UTC+2, BEN SALEM Omar a écrit : >> >> It's " 'es.mapping.names' = 'sendingTime:@timestamp');" >> not 'es.mapping.names' = 'date:@timestamp'); >> >> On Tuesday, April 14, 2015 at 11:19:13 AM UTC+2, BEN SALEM Omar wrote: >>> >>> Hi, >>> >>> I have trouble with the timestamp format required for elasticsearch. >>> >>> I have data in my hadoop that I want to load into ES. >>> My date field was first as this : "EEE, dd MMM yyyy HH:mm:ss Z" >>> >>> I've transformed that with talend like this : >>> >>> >>> <https://lh3.googleusercontent.com/-rzoBkk29LMM/VSzY7w4kSFI/AAAAAAAAAHA/Uq3XtZEdsyI/s1600/Capture.PNG> >>> >>> >>> >>> Now if I try to load my data into ES, it keeps on mapping it as a string >>> and not as a timestamp as it should be ! >>> >>> This is what I've done : >>> >>> 1)CREATE TABLE xx ( subject STRING, author string, sendingTime string) >>> >>> ROW FORMAT DELIMITED FIELDS TERMINATED BY '\t'; >>> >>> >>> LOAD DATA INPATH '/user/hive/out.csv' OVERWRITE INTO TABLE mails; >>> >>> >>> 2)CREATE EXTERNAL TABLE es_xx (subject string,author string, sendingTime >>> string) >>> >>> STORED BY 'org.elasticsearch.hadoop.hive.EsStorageHandler' >>> >>> TBLPROPERTIES('es.resource' = 'xx/xx', >>> >>> 'es.nodes'='xx', >>> >>> 'es.mapping.names' = 'date:@timestamp'); >>> >>> >>> >>> INSERT OVERWRITE TABLE es_xx SELECT s.subject, s.author, s.sendingTime >>> from xx s; >>> >>> And this how it was mapped : >>> >>> {"xx":{"mappings":{"xx":{"properties":{"author":{"type":"string"},"sendingtime":{"type":"string"},"subject":{"type":"string"}}}}}} >>> >>> >>> >>> In another example I've transformed my date format like this >>> yyyy-MM-ddTHH:mm:ssZ and that went good ! >>> >>> Is that the only format that ES accept? If so, that have to change ! >>> >>> >>> What should I do to make ES recognize my date field as a timestamp? >>> >>> If I do a static mapping and transform the sendingtime field into a >>> timestamp, would it work? or the current format has to change? >>> >>> >>> Hope for a quick response, thanks you all, >>> >>> best regards, >>> >>> Omar, >>> >>> -- > You received this message because you are subscribed to a topic in the > Google Groups "elasticsearch" group. > To unsubscribe from this topic, visit > https://groups.google.com/d/topic/elasticsearch/HAarVd-hPhA/unsubscribe. > To unsubscribe from this group and all its topics, send an email to > [email protected]. > To view this discussion on the web visit > https://groups.google.com/d/msgid/elasticsearch/938af0c2-3a03-4f48-94be-21bb2db744a4%40googlegroups.com > <https://groups.google.com/d/msgid/elasticsearch/938af0c2-3a03-4f48-94be-21bb2db744a4%40googlegroups.com?utm_medium=email&utm_source=footer> > . > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAGgBrpzd5daDj6thAx0LfXX5V21kBqpc55VrX0ObCcO3B%2BazTw%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
