Thank you. Yes, we have complete control over both apps.
On Wednesday, July 8, 2015 at 8:58:53 PM UTC-4, Francis Avila wrote: > > Who is saving these strings, and who is reading them? Do you have complete > control over both apps, or does one of them need to be aligned with the > other? > > If the Java app is the baseline, you need to know the exact details of the > format of the data it saves. Just knowing "it's JSON" is not enough, > because there may be other data types (e.g. dates) that don't have native > JSON representations. (The go-to JSON de/encoder for clojure is cheshire: > https://github.com/dakrone/cheshire ) > > I took a quick look at Jedis and the easy, default way of using it is with > strings. It will encode strings to UTF-8 before sending to Redis and decode > from UTF-8 on read. You can set raw byte arrays too (which will not be > altered in any way before sending), but it's not clear to me how it can > read out raw byte arrays. (I'm sure there's a way, but it's not immediately > obvious.) > > As for the escaped quotes, you may be using pr or prn to print, or maybe > you are using pr-str to produce the string representation. I can't be sure. > > On Wednesday, July 8, 2015 at 5:31:27 PM UTC-5, gingers...@gmail.com > wrote: >> >> And I have another stupid question. Using the above code, I am sometimes >> getting strings in Redis that have escaped quotation marks, like this: >> >> " \"transaction-id\" : \" 1ec47c2e-21ee-427c-841c-80a0f89f55d7 \" >> \"debrief\" : \" Susan Hilly at Citi called to get a quotation for >> discounted weekly car rental for approximately 3 cars per week, or 150 >> rentals annually. \" " >> >> Why is that happening? >> >> >> >> On Wednesday, July 8, 2015 at 5:38:20 PM UTC-4, gingers...@gmail.com >> wrote: >>> >>> Francis Avila, >>> >>> Thank you for your response. The Java app is using Jedis and the Clojure >>> app is using Carmine. I'm wondering if you can suggest what you think would >>> be the easiest way to allow these 2 apps to understand each other's strings? >>> >>> You were correct about how unsafe the above code was. I tested it for >>> less than 15 minutes and ran into the fact that a \n newline made a mess of >>> everything. >>> >>> >>> >>> On Wednesday, July 8, 2015 at 5:15:35 PM UTC-4, Francis Avila wrote: >>>> >>>> You are running into Carmine's automatic nippy serialization. >>>> https://github.com/ptaoussanis/carmine#serialization >>>> >>>> Redis only stores byte arrays (what it calls "strings"). Carmine uses >>>> the nippy library (the meaning of "NPY" in your byte stream) to represent >>>> rich types compactly as bytes. https://github.com/ptaoussanis/nippy >>>> >>>> If you give Carmine a byte array to store, it will store it directly >>>> without nippy-encoding it. E.g. (.getBytes "{}" "UTF-8") >>>> >>>> BTW your document-as-string example is extremely unsafe: how will you >>>> reliably read this message out again? e.g. what if the 'debrief' string >>>> contains a single quote? Use a proper serialization format. >>>> >>>> So the key is to have both your Clojure and Java app store *bytes* in >>>> Redis using the same serialization. You can store anything you want >>>> (nippy, >>>> utf-8-encoded json, fressian, bson, utf-8 xml, utf-16 java strings, >>>> whatever) as long as it's bytes and it's read and written the same way in >>>> all your apps. >>>> >>>> The Redis library your Java app is using may have its own automatic >>>> de/serialization, too. You need to find out what it's doing and either >>>> work >>>> with this or turn it off, just like with Carmine. >>>> >>>> Nippy unfortunately does not have a Java API out of the box: >>>> https://github.com/ptaoussanis/nippy/issues/66 >>>> >>>> >>>> On Wednesday, July 8, 2015 at 3:35:49 PM UTC-5, gingers...@gmail.com >>>> wrote: >>>>> >>>>> >>>>> I am not sure if this is a Clojure question or a Java question. I >>>>> don't know Java, so I could use whatever help folks can offer. >>>>> >>>>> "primitive string" here means what I can write when I am at the >>>>> terminal. >>>>> >>>>> We have 2 apps, one in Clojure, one in Java. They talk to each other >>>>> via Redis. I know the Java app can read stuff out of Redis, using our >>>>> "transaction-id", if I use the terminal and open up "redis-clj" and write >>>>> a >>>>> string directly from the terminal. But I have this Clojure code, which >>>>> depends on Peter Taoussanis's Carmine library: >>>>> (defn worker [document] >>>>> {:pre [(string? (:transaction-id document))]} >>>>> (let [transaction-id (:transaction-id document) >>>>> document-as-string (str "{'transaction-id' : '" transaction-id >>>>> "', 'debrief' : '" (:debrief document) "'}" ) >>>>> redis-connection {:pool {} :spec {:host "127.0.0.1" :port 6379 >>>>> }}] >>>>> (timbre/log :trace " message we will send to NLP " >>>>> document-as-string) >>>>> (carmine/wcar redis-connection (carmine/set transaction-id >>>>> document)) >>>>> (loop [document-in-redis (carmine/wcar redis-connection >>>>> (carmine/get transaction-id))] >>>>> >>>>> (if-not (.contains (first document-in-redis) "processed") >>>>> (recur (carmine/wcar redis-connection (carmine/get >>>>> transaction-id))) >>>>> (do >>>>> (carmine/wcar redis-connection (carmine/del transaction-id)) >>>>> document-in-redis))))) >>>>> >>>>> This line in particular, I have tried doing this several ways: >>>>> >>>>> document-as-string (str "{'transaction-id' : '" transaction-id >>>>> "', 'debrief' : '" (:debrief document) "'}" ) >>>>> >>>>> In Redis, I expect to see: >>>>> >>>>> {'transaction-id' : '42e574e7-3b80-424a-b9ff-01072f1e0358', 'debrief' >>>>> : 'Smeek Hallie of Withers, Smeg, Harrington and Norvig responded to our >>>>> proposal and said his company is read to move forward. The rate of $400 >>>>> per >>>>> ton of shredded paper was acceptable to them, and they shred about 2 tons >>>>> of documents every month. $96,000 in potential revenue annually. I will >>>>> meet with him tomorrow and we will sign the contract.'} >>>>> >>>>> But if I then launch redis-cli, I see: >>>>> >>>>> >>>>> 127.0.0.1:6379> keys * >>>>> 1) "42e574e7-3b80-424a-b9ff-01072f1e0358" >>>>> >>>>> 127.0.0.1:6379> get "42e574e7-3b80-424a-b9ff-01072f1e0358" >>>>> "\x00>NPY\b\x00\x00\x01\xfc\xf1\xfe\x1b\x00\x00\x00\nj\nip-addressi\x0e165.254.84.238j\x05tokeni$46b87d64-cff3-4b8b-895c-e089ac59544dj\x0bapi-versioni\x02v1j\x0etransaction-idi$42e574e7-3b80-424a-b9ff-01072f1e0358j\adebrief\r\x00\x00\x01YSmeek >>>>> >>>>> Hallie of Withers, Smeg, Harrington and Norvig responded to our proposal >>>>> and said his company is rea-\x00\xf1\x06move forward. The >>>>> raty\x00\xf0\x0c$400 per ton of shredded pa\x16\x00\xf1\bwas acceptable >>>>> to >>>>> them,q\x00Bthey0\x00\x80 about 2E\x00\x10sF\x00\xf1Ldocuments every >>>>> month. >>>>> $96,000 in potential revenue annually. I will meet with him >>>>> tomorrow{\x00\"we#\x00@sign\x92\x00\xa0 contract." >>>>> >>>>> >>>>> I don't know what all of those extra characters are. The Java app is >>>>> not picking this item up, so I assume the Java app is not seeing this as >>>>> a >>>>> string. I expected this to look the same as if I had written this at the >>>>> terminal: >>>>> >>>>> {'transaction-id' : '42e574e7-3b80-424a-b9ff-01072f1e0358', 'debrief' >>>>> : 'Smeek Hallie of Withers, Smeg, Harrington and Norvig responded to our >>>>> proposal and said his company is read to move forward. The rate of $400 >>>>> per >>>>> ton of shredded paper was acceptable to them, and they shred about 2 tons >>>>> of documents every month. $96,000 in potential revenue annually. I will >>>>> meet with him tomorrow and we will sign the contract.'} >>>>> >>>>> I assume it is easy to get a string into a format that can be >>>>> understood by both a Clojure app and a Java app. I don't care what format >>>>> that is, but it needs to be consistent. >>>>> >>>>> Can anyone make suggestions about what I can do to make sure the >>>>> Clojure app and the Java app both write to Redis using a format that the >>>>> other will understand? In particular, both apps need to see the >>>>> "'transaction-id". >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups "Clojure" group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
