No, we send json twitter data but in flume channel ı saw content type word for each tweet. Is it normal ? How can ı send just tweets json without any content type. I took tweets json from GNIP company.
Thanks iPhone'umdan gönderildi 2 Mar 2016 tarihinde 10:56 saatinde, Gonzalo Herreros <[email protected]<mailto:[email protected]>> şunları yazdı: Could it be that you are serializing avro instead of json? On 2 March 2016 at 08:25, Baris Akgun (Garanti Teknoloji) <[email protected]<mailto:[email protected]>> wrote: Hi, When I send json data to flume with using http post, flume adds Co**ntent-Typeapplication/json** for each json post. In my http post java code, I give the content-type with using **con.setRequestProperty("Content-Type", "application/json");** function. I am using blob handler. **In flume conf file** *tier1.sources.source1.type = org.apache.flume.source.http.HTTPSource tier1.sources.source1.handler = org.apache.flume.sink.solr.morphline.BlobHandler* In flume channel, flume adds content type for each post as you see. After HDFS sink, The content type word causes a problem when ı try to parse json with spark sql or hive serDe. **The flume channel log data** *^LContent-Typeapplication/jsonú{"id":"+ag:_ea_ch.++i++e_.c- ^LContentTypeapplication/json‘{"id":"tag:search.twitter.com<http://search.twitter.com>,2005:642913165047648* Is there any idea for that problem? Thank a lot. Barış Akgün Analitik Veri Ambarı ve Büyük Veri Yönetimi Uzman Tel : Dahili : Faks : Bu mesaj ve ekleri, mesajda gonderildigi belirtilen kisi/kisilere ozeldir ve gizlidir. Bu mesajin muhatabi olmamaniza ragmen tarafiniza ulasmis olmasi halinde mesaj iceriginin gizliligi ve bu gizlilik yukumlulugune uyulmasi zorunlulugu tarafiniz icin de soz konusudur. Mesaj ve eklerinde yer alan bilgilerin dogrulugu ve guncelligi konusunda gonderenin ya da sirketimizin herhangi bir sorumlulugu bulunmamaktadir. Sirketimiz mesajin ve bilgilerinin size degisiklige ugrayarak veya gec ulasmasindan, butunlugunun ve gizliliginin korunamamasindan, virus icermesinden ve bilgisayar sisteminize verebilecegi herhangi bir zarardan sorumlu tutulamaz. This message and attachments are confidential and intended solely for the individual(s) stated in this message. If you received this message although you are not the addressee, you are responsible to keep the message confidential. The sender has no responsibility for the accuracy or correctness of the information in the message and its attachments. Our company shall have no liability for any changes or late receiving, loss of integrity and confidentiality, viruses and any damages caused in anyway to your computer system. Bu mesaj ve ekleri, mesajda gonderildigi belirtilen kisi/kisilere ozeldir ve gizlidir. Bu mesajin muhatabi olmamaniza ragmen tarafiniza ulasmis olmasi halinde mesaj iceriginin gizliligi ve bu gizlilik yukumlulugune uyulmasi zorunlulugu tarafiniz icin de soz konusudur. Mesaj ve eklerinde yer alan bilgilerin dogrulugu ve guncelligi konusunda gonderenin ya da sirketimizin herhangi bir sorumlulugu bulunmamaktadir. Sirketimiz mesajin ve bilgilerinin size degisiklige ugrayarak veya gec ulasmasindan, butunlugunun ve gizliliginin korunamamasindan, virus icermesinden ve bilgisayar sisteminize verebilecegi herhangi bir zarardan sorumlu tutulamaz. This message and attachments are confidential and intended solely for the individual(s) stated in this message. If you received this message although you are not the addressee, you are responsible to keep the message confidential. The sender has no responsibility for the accuracy or correctness of the information in the message and its attachments. Our company shall have no liability for any changes or late receiving, loss of integrity and confidentiality, viruses and any damages caused in anyway to your computer system.
