Yes, that too :) Also if its a time based data, you will not be able to use kibana's date filtering etc - because it lacks the @timestamp field. Basically, the XDCR elasticsearch plugin was built around the XDCR / Couchbase realm and not around Elasticsearch's. Unfortunately this means many ES features are unavailable / hard to use, e.g. https://github.com/couchbaselabs/elasticsearch-transport-couchbase/issues/63
https://github.com/couchbaselabs/elasticsearch-transport-couchbase/issues/64 I can help fixing this on the XDCR plugin if you'd like - ping me privately and we can work something out (or I can convince you to avoid using the XDCR replication) -- Itamar Syn-Hershko http://code972.com | @synhershko <https://twitter.com/synhershko> Freelance Developer & Consultant Lucene.NET committer and PMC member On Thu, Feb 12, 2015 at 5:18 PM, Nadav Hashimshony <nad...@gmail.com> wrote: > ok, ill try. > > this "envelope document", is it something i need to be concerned about > when I'm querying via Kibana? > > On Thursday, February 12, 2015 at 5:14:39 PM UTC+2, Itamar Syn-Hershko > wrote: >> >> Yes. Just make sure the template reflects the actual document structure - >> as I said XDCR wraps your document in an envelope document >> >> -- >> >> Itamar Syn-Hershko >> http://code972.com | @synhershko <https://twitter.com/synhershko> >> Freelance Developer & Consultant >> Lucene.NET committer and PMC member >> >> On Thu, Feb 12, 2015 at 5:12 PM, Nadav Hashimshony <nad...@gmail.com> >> wrote: >> >>> ok, just to be clear. >>> >>> the steps i did was as followed: >>> 1. create the index with the mapping. >>> 2. define the XDCR to replicate my bucket with the index in ES. >>> 3. insert data to couchbase. >>> 4. try to query with kibana >>> >>> What you suggest is to Add another BEFORE step 1: >>> 0. create a template to include my mapping. >>> 1. crate the index in ES >>> and so on... >>> >>> did i get it right? >>> >>> Thanks. >>> Nadav. >>> >>> >>> On Thursday, February 12, 2015 at 5:04:24 PM UTC+2, Itamar Syn-Hershko >>> wrote: >>>> >>>> Like I said, you need the mapping to "catch" before the XDCR plugin >>>> begins the replication - so you need to put a template with this mapping >>>> that will override XDCR's >>>> >>>> -- >>>> >>>> Itamar Syn-Hershko >>>> http://code972.com | @synhershko <https://twitter.com/synhershko> >>>> Freelance Developer & Consultant >>>> Lucene.NET committer and PMC member >>>> >>>> On Thu, Feb 12, 2015 at 4:59 PM, Nadav Hashimshony <nad...@gmail.com> >>>> wrote: >>>> >>>>> Thanks you for the response >>>>> >>>>> i am using mapping, i created the following index >>>>> PUT /storage/files/_mapping >>>>> { >>>>> "files": { >>>>> "properties": { >>>>> "file": { >>>>> "type": "attachment", >>>>> "path": "full", >>>>> "fields": { >>>>> "content_type": { >>>>> "type": "string", >>>>> "store": true >>>>> } >>>>> } >>>>> } >>>>> } >>>>> } >>>>> } >>>>> >>>>> when i insert data via ES and query it, all is fine. >>>>> the problem is when data is inserted to Couchbase.. >>>>> >>>>> Nadav >>>>> >>>>> On Thursday, February 12, 2015 at 4:03:01 PM UTC+2, Itamar Syn-Hershko >>>>> wrote: >>>>>> >>>>>> The XDCR plugin indexes the data using an envelope document. Long >>>>>> story short, make sure you use the latest XDCR plugin as older ones are >>>>>> missing lots of important functions, and use templates and dynamic >>>>>> templates with proper field paths for this to work correctly >>>>>> >>>>>> http://code972.com/blog/2015/02/80-elasticsearch-one-tip-a-d >>>>>> ay-managing-index-mappings-like-a-pro >>>>>> http://code972.com/blog/2015/02/81-elasticsearch-one-tip-a-d >>>>>> ay-using-dynamic-templates-to-avoid-rigorous-mappings >>>>>> >>>>>> -- >>>>>> >>>>>> Itamar Syn-Hershko >>>>>> http://code972.com | @synhershko <https://twitter.com/synhershko> >>>>>> Freelance Developer & Consultant >>>>>> Lucene.NET committer and PMC member >>>>>> >>>>>> On Thu, Feb 12, 2015 at 3:59 PM, Nadav Hashimshony <nad...@gmail.com> >>>>>> wrote: >>>>>> >>>>>>> Hi, >>>>>>> >>>>>>> I'm new to the group, hope ill find what i need and share my >>>>>>> experience as i go along.. >>>>>>> >>>>>>> im using ES with the attachment-plugin in order to store and search >>>>>>> files. >>>>>>> when i set the mapping right and insert the file data in a Base64 >>>>>>> manner I'm able to query my data via Kibana. >>>>>>> >>>>>>> my problem is this. >>>>>>> >>>>>>> if i create the index + mapping in ES, then insert the data to >>>>>>> Couchbase and use XDRC to replicate it to ES, i can't query the Data >>>>>>> with >>>>>>> Kibana. >>>>>>> it looks like the mapping of the index created in ES doesn't index >>>>>>> well the data it gets from Couchbase. >>>>>>> >>>>>>> has anyone encounter such an issue? >>>>>>> >>>>>>> Thanks You >>>>>>> >>>>>>> Nadav. >>>>>>> >>>>>>> -- >>>>>>> You received this message because you are subscribed to the Google >>>>>>> Groups "elasticsearch" group. >>>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>>> send an email to elasticsearc...@googlegroups.com. >>>>>>> To view this discussion on the web visit >>>>>>> https://groups.google.com/d/msgid/elasticsearch/8092eaf5-0ef >>>>>>> 8-4249-8e5d-acff8281a81a%40googlegroups.com >>>>>>> <https://groups.google.com/d/msgid/elasticsearch/8092eaf5-0ef8-4249-8e5d-acff8281a81a%40googlegroups.com?utm_medium=email&utm_source=footer> >>>>>>> . >>>>>>> For more options, visit https://groups.google.com/d/optout. >>>>>>> >>>>>> >>>>>> -- >>>>> You received this message because you are subscribed to the Google >>>>> Groups "elasticsearch" group. >>>>> To unsubscribe from this group and stop receiving emails from it, send >>>>> an email to elasticsearc...@googlegroups.com. >>>>> To view this discussion on the web visit https://groups.google.com/d/ >>>>> msgid/elasticsearch/1d9c8ce5-116f-40cc-a5e3-6ebe47191850%40goo >>>>> glegroups.com >>>>> <https://groups.google.com/d/msgid/elasticsearch/1d9c8ce5-116f-40cc-a5e3-6ebe47191850%40googlegroups.com?utm_medium=email&utm_source=footer> >>>>> . >>>>> >>>>> For more options, visit https://groups.google.com/d/optout. >>>>> >>>> >>>> -- >>> You received this message because you are subscribed to the Google >>> Groups "elasticsearch" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to elasticsearc...@googlegroups.com. >>> To view this discussion on the web visit https://groups.google.com/d/ >>> msgid/elasticsearch/8628ef07-2e10-423a-9de0-13ebaa37a0e8% >>> 40googlegroups.com >>> <https://groups.google.com/d/msgid/elasticsearch/8628ef07-2e10-423a-9de0-13ebaa37a0e8%40googlegroups.com?utm_medium=email&utm_source=footer> >>> . >>> >>> For more options, visit https://groups.google.com/d/optout. >>> >> >> -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to elasticsearch+unsubscr...@googlegroups.com. > To view this discussion on the web visit > https://groups.google.com/d/msgid/elasticsearch/96f30e99-30dc-4af2-a51b-b12b0dee9e53%40googlegroups.com > <https://groups.google.com/d/msgid/elasticsearch/96f30e99-30dc-4af2-a51b-b12b0dee9e53%40googlegroups.com?utm_medium=email&utm_source=footer> > . > > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAHTr4Zt%3Drs%2BnOhoHP2UPM5ZmV%2BWQet_8FrmiGHFpf_Ykokdxyw%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
