My tongue-in-cheek comment about how to tell veteran programmers from novices:
Novice: “My code compiled fine, it should run perfectly” Veteran: “That can’t be right, my code appeared to succeed the first time I ran it.”... > On Oct 21, 2019, at 12:54 PM, Vincenzo D'Amore <v.dam...@gmail.com> wrote: > > Hi all, > > at last I've tried to write my own payload function, and it has worked > (pretty well) immediately! > Well, to be honest I'm a little puzzled, so I have applied immediately the > first rule: always beware when your code works at first run. > > Now I'm testing it. > > On Mon, Oct 21, 2019 at 4:37 PM Erick Erickson <erickerick...@gmail.com> > wrote: > >> Don’t go to thousands of fields, it’s usually a bad idea for that many >> fields. >> >> If all you’re doing is returning the data for display, have you considered >> indexing, perhaps in a separate field, a string token for each? Something >> like >> store1_USD_125.00 >> ? >> >> Then have the UI split that apart in a pleasing manner. >> >> Doesn’t work if you want to, say, sort by price though. >> >> Erick >> >>> On Oct 21, 2019, at 10:19 AM, Alexandre Rafalovitch <arafa...@gmail.com> >> wrote: >>> >>> I remember several years ago a discussion/blog post about a similar >>> problem. The author went through a lot of thinking and decided that >>> the best way to deal with a similar problem was to have Solr documents >>> represent different level of abstraction, more granular. >>> >>> IIRC, the equivalent for your example would be to represent 'pricing' >>> (or even pricing/availability) as the document, not a 'product'. You >>> may need to duplicate product field values for that to work. But that >>> - apparently - allowed them to represent a lot of concepts (like >>> upcoming discounts, etc) easier by just adding availability dates, etc >>> into that final lower-level record. And, of course, allowed to update >>> individual store/product price by changing one record per time without >>> having to invalidate the cache for all other stores. >>> >>> Regards, >>> Alex. >>> >>> >>> On Mon, 21 Oct 2019 at 09:59, Vincenzo D'Amore <v.dam...@gmail.com> >> wrote: >>>> >>>> Hi Erick, >>>> >>>> thanks for getting back to me. We started to use payloads because we >> have >>>> the classical per-store pricing problem. >>>> Thousands of stores across and different prices. >>>> Then we found the payloads very useful started to use it for many >> reasons, >>>> like enabling/disabling the product for such store, save the stock >>>> availability, or save the other info like buy/sell price, discount >> rates, >>>> and so on. >>>> All those information are numbers, but stores can also be in different >>>> countries, I mean would be useful also have the currency and other >>>> attributes related to the store. >>>> >>>> Thinking about an alternative for payloads maybe I could use the dynamic >>>> fields, well, I know it is ugly. >>>> >>>> Consider this hypothetical case where I have two field payload : >>>> >>>> payloadPrice: [ >>>> "store1|125.0", >>>> "store2|220.0", >>>> "store3|225.0" >>>> ] >>>> >>>> payloadCurrency: [ >>>> "store1|USD", >>>> "store2|EUR", >>>> "store3|GBP" >>>> ] >>>> >>>> with dynamic fields I could have different fields for each document. >>>> >>>> currency_store1_s: "USD" >>>> currency_store2_s: "EUR" >>>> currency_store3_s: "GBP" >>>> >>>> But how many dynamic fields like this can I have? more than thousands? >>>> >>>> Again, I've just started to look at solr-ocrhighlighting github project >> you >>>> suggested. >>>> Those seems have written their own payload object type where store ocr >>>> highlighting information. >>>> It seems interesting, I'll take a look immediately. >>>> >>>> Thanks again for your time. >>>> >>>> Best regards, >>>> Vincenzo >>>> >>>> >>>> On Mon, Oct 21, 2019 at 2:55 PM Erick Erickson <erickerick...@gmail.com >>> >>>> wrote: >>>> >>>>> This is one of those situations where I know a client did it, but >> didn’t >>>>> see the code myself. >>>>> >>>>> So I can’t help much. >>>>> >>>>> Perhaps a good question at this point, though, is “why do you want to >> add >>>>> string payloads anyway”? >>>>> >>>>> This isn’t the client, but it might give you some pointers: >>>>> >>>>> >>>>> >> https://github.com/dbmdz/solr-ocrpayload-plugin/blob/master/src/main/java/de/digitalcollections/solr/plugin/components/ocrhighlighting/OcrHighlighting.java >>>>> >>>>> Best, >>>>> Erick >>>>> >>>>>> On Oct 21, 2019, at 6:37 AM, Vincenzo D'Amore <v.dam...@gmail.com> >>>>> wrote: >>>>>> >>>>>> Hi Erick, >>>>>> >>>>>> It seems I've reached a dead-point, or at least it seems looking at >> the >>>>>> code, it seems I can't easily add a custom decoder: >>>>>> >>>>>> Looking at PayloadUtils class there is getPayloadDecoder method >> invoked >>>>> to >>>>>> return the PayloadDecoder : >>>>>> >>>>>> public static PayloadDecoder getPayloadDecoder(FieldType fieldType) { >>>>>> PayloadDecoder decoder = null; >>>>>> >>>>>> String encoder = getPayloadEncoder(fieldType); >>>>>> >>>>>> if ("integer".equals(encoder)) { >>>>>> decoder = (BytesRef payload) -> payload == null ? 1 : >>>>>> PayloadHelper.decodeInt(payload.bytes, payload.offset); >>>>>> } >>>>>> if ("float".equals(encoder)) { >>>>>> decoder = (BytesRef payload) -> payload == null ? 1 : >>>>>> PayloadHelper.decodeFloat(payload.bytes, payload.offset); >>>>>> } >>>>>> // encoder could be "identity" at this point, in the case of >>>>>> DelimitedTokenFilterFactory encoder="identity" >>>>>> >>>>>> // TODO: support pluggable payload decoders? >>>>>> >>>>>> return decoder; >>>>>> } >>>>>> >>>>>> Any advice to work around this situation? >>>>>> >>>>>> >>>>>> On Mon, Oct 21, 2019 at 1:51 AM Erick Erickson < >> erickerick...@gmail.com> >>>>>> wrote: >>>>>> >>>>>>> You’d need to write one. Payloads are generally intended to hold >>>>> numerics >>>>>>> you can then use in a function query to factor into the score… >>>>>>> >>>>>>> Best, >>>>>>> Erick >>>>>>> >>>>>>>> On Oct 20, 2019, at 4:57 PM, Vincenzo D'Amore <v.dam...@gmail.com> >>>>>>> wrote: >>>>>>>> >>>>>>>> Sorry, I just realized that I was wrong in how I'm using the payload >>>>>>>> function. >>>>>>>> Give that the payload function only handles a numeric (integer or >>>>> float) >>>>>>>> payload, could you suggest me an alternative function that handles >>>>>>> strings? >>>>>>>> If not, should I write one? >>>>>>>> >>>>>>>> On Sun, Oct 20, 2019 at 10:43 PM Vincenzo D'Amore < >> v.dam...@gmail.com> >>>>>>>> wrote: >>>>>>>> >>>>>>>>> Hi all, >>>>>>>>> >>>>>>>>> I'm trying to understand what I did wrong with a payload query that >>>>>>>>> returns >>>>>>>>> >>>>>>>>> error: { >>>>>>>>> metadata: [ "error-class", "org.apache.solr.common.SolrException", >>>>>>>>> "root-error-class", "org.apache.solr.common.SolrException" ], >>>>>>>>> msg: "No payload decoder found for field: colorCode", >>>>>>>>> code: 400 >>>>>>>>> } >>>>>>>>> >>>>>>>>> I have reduced my problem in a little sample to show what happens >> to >>>>> me. >>>>>>>>> Basically I have a document with a couple of payload fields one >>>>>>>>> delimited_payloads_string and one delimited_payloads_integer >>>>>>>>> >>>>>>>>> { >>>>>>>>> field_dps: "key|data", >>>>>>>>> field_dpi: "key|1", >>>>>>>>> } >>>>>>>>> >>>>>>>>> When I execute this query solr returns as expected the payload for >> the >>>>>>> key >>>>>>>>> >>>>>>>>> q=*:*&fl=payload(field_dpi,key) >>>>>>>>> >>>>>>>>> { >>>>>>>>> payload(field_dpi,key): 1 >>>>>>>>> } >>>>>>>>> >>>>>>>>> But for the strings there have to be something of different to do, >>>>>>> because >>>>>>>>> I'm unable receive the payload value back. Executing this query, >> as in >>>>>>> the >>>>>>>>> short introduction of this post, I receive an error. >>>>>>>>> >>>>>>>>> ?q=*:*&fl=payload(field_dps,key) >>>>>>>>> >>>>>>>>> error: { >>>>>>>>> metadata: [ "error-class", "org.apache.solr.common.SolrException", >>>>>>>>> "root-error-class", "org.apache.solr.common.SolrException" ], >>>>>>>>> msg: "No payload decoder found for field: colorCode", >>>>>>>>> code: 400 >>>>>>>>> } >>>>>>>>> >>>>>>>>> Am I doing something wrong? How can I read strings payload data? >>>>>>>>> >>>>>>>>> Thanks in advance for your time, >>>>>>>>> Vincenzo >>>>>>>>> >>>>>>>>> -- >>>>>>>>> Vincenzo D'Amore >>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>>> -- >>>>>>>> Vincenzo D'Amore >>>>>>> >>>>>>> >>>>>> >>>>>> -- >>>>>> Vincenzo D'Amore >>>>> >>>>> >>>> >>>> -- >>>> Vincenzo D'Amore >> >> > > -- > Vincenzo D'Amore
