And I have another stupid question. Using the above code, I am sometimes getting strings in Redis that have escaped quotation marks, like this:
" \"transaction-id\" : \" 1ec47c2e-21ee-427c-841c-80a0f89f55d7 \" \"debrief\" : \" Susan Hilly at Citi called to get a quotation for discounted weekly car rental for approximately 3 cars per week, or 150 rentals annually. \" " Why is that happening? On Wednesday, July 8, 2015 at 5:38:20 PM UTC-4, gingers...@gmail.com wrote: > > Francis Avila, > > Thank you for your response. The Java app is using Jedis and the Clojure > app is using Carmine. I'm wondering if you can suggest what you think would > be the easiest way to allow these 2 apps to understand each other's strings? > > You were correct about how unsafe the above code was. I tested it for less > than 15 minutes and ran into the fact that a \n newline made a mess of > everything. > > > > On Wednesday, July 8, 2015 at 5:15:35 PM UTC-4, Francis Avila wrote: >> >> You are running into Carmine's automatic nippy serialization. >> https://github.com/ptaoussanis/carmine#serialization >> >> Redis only stores byte arrays (what it calls "strings"). Carmine uses the >> nippy library (the meaning of "NPY" in your byte stream) to represent rich >> types compactly as bytes. https://github.com/ptaoussanis/nippy >> >> If you give Carmine a byte array to store, it will store it directly >> without nippy-encoding it. E.g. (.getBytes "{}" "UTF-8") >> >> BTW your document-as-string example is extremely unsafe: how will you >> reliably read this message out again? e.g. what if the 'debrief' string >> contains a single quote? Use a proper serialization format. >> >> So the key is to have both your Clojure and Java app store *bytes* in >> Redis using the same serialization. You can store anything you want (nippy, >> utf-8-encoded json, fressian, bson, utf-8 xml, utf-16 java strings, >> whatever) as long as it's bytes and it's read and written the same way in >> all your apps. >> >> The Redis library your Java app is using may have its own automatic >> de/serialization, too. You need to find out what it's doing and either work >> with this or turn it off, just like with Carmine. >> >> Nippy unfortunately does not have a Java API out of the box: >> https://github.com/ptaoussanis/nippy/issues/66 >> >> >> On Wednesday, July 8, 2015 at 3:35:49 PM UTC-5, gingers...@gmail.com >> wrote: >>> >>> >>> I am not sure if this is a Clojure question or a Java question. I don't >>> know Java, so I could use whatever help folks can offer. >>> >>> "primitive string" here means what I can write when I am at the terminal. >>> >>> We have 2 apps, one in Clojure, one in Java. They talk to each other via >>> Redis. I know the Java app can read stuff out of Redis, using our >>> "transaction-id", if I use the terminal and open up "redis-clj" and write a >>> string directly from the terminal. But I have this Clojure code, which >>> depends on Peter Taoussanis's Carmine library: >>> (defn worker [document] >>> {:pre [(string? (:transaction-id document))]} >>> (let [transaction-id (:transaction-id document) >>> document-as-string (str "{'transaction-id' : '" transaction-id >>> "', 'debrief' : '" (:debrief document) "'}" ) >>> redis-connection {:pool {} :spec {:host "127.0.0.1" :port 6379 >>> }}] >>> (timbre/log :trace " message we will send to NLP " >>> document-as-string) >>> (carmine/wcar redis-connection (carmine/set transaction-id document)) >>> (loop [document-in-redis (carmine/wcar redis-connection (carmine/get >>> transaction-id))] >>> >>> (if-not (.contains (first document-in-redis) "processed") >>> (recur (carmine/wcar redis-connection (carmine/get >>> transaction-id))) >>> (do >>> (carmine/wcar redis-connection (carmine/del transaction-id)) >>> document-in-redis))))) >>> >>> This line in particular, I have tried doing this several ways: >>> >>> document-as-string (str "{'transaction-id' : '" transaction-id >>> "', 'debrief' : '" (:debrief document) "'}" ) >>> >>> In Redis, I expect to see: >>> >>> {'transaction-id' : '42e574e7-3b80-424a-b9ff-01072f1e0358', 'debrief' : >>> 'Smeek Hallie of Withers, Smeg, Harrington and Norvig responded to our >>> proposal and said his company is read to move forward. The rate of $400 per >>> ton of shredded paper was acceptable to them, and they shred about 2 tons >>> of documents every month. $96,000 in potential revenue annually. I will >>> meet with him tomorrow and we will sign the contract.'} >>> >>> But if I then launch redis-cli, I see: >>> >>> >>> 127.0.0.1:6379> keys * >>> 1) "42e574e7-3b80-424a-b9ff-01072f1e0358" >>> >>> 127.0.0.1:6379> get "42e574e7-3b80-424a-b9ff-01072f1e0358" >>> "\x00>NPY\b\x00\x00\x01\xfc\xf1\xfe\x1b\x00\x00\x00\nj\nip-addressi\x0e165.254.84.238j\x05tokeni$46b87d64-cff3-4b8b-895c-e089ac59544dj\x0bapi-versioni\x02v1j\x0etransaction-idi$42e574e7-3b80-424a-b9ff-01072f1e0358j\adebrief\r\x00\x00\x01YSmeek >>> >>> Hallie of Withers, Smeg, Harrington and Norvig responded to our proposal >>> and said his company is rea-\x00\xf1\x06move forward. The >>> raty\x00\xf0\x0c$400 per ton of shredded pa\x16\x00\xf1\bwas acceptable to >>> them,q\x00Bthey0\x00\x80 about 2E\x00\x10sF\x00\xf1Ldocuments every month. >>> $96,000 in potential revenue annually. I will meet with him >>> tomorrow{\x00\"we#\x00@sign\x92\x00\xa0 contract." >>> >>> >>> I don't know what all of those extra characters are. The Java app is not >>> picking this item up, so I assume the Java app is not seeing this as a >>> string. I expected this to look the same as if I had written this at the >>> terminal: >>> >>> {'transaction-id' : '42e574e7-3b80-424a-b9ff-01072f1e0358', 'debrief' : >>> 'Smeek Hallie of Withers, Smeg, Harrington and Norvig responded to our >>> proposal and said his company is read to move forward. The rate of $400 per >>> ton of shredded paper was acceptable to them, and they shred about 2 tons >>> of documents every month. $96,000 in potential revenue annually. I will >>> meet with him tomorrow and we will sign the contract.'} >>> >>> I assume it is easy to get a string into a format that can be understood >>> by both a Clojure app and a Java app. I don't care what format that is, but >>> it needs to be consistent. >>> >>> Can anyone make suggestions about what I can do to make sure the Clojure >>> app and the Java app both write to Redis using a format that the other will >>> understand? In particular, both apps need to see the "'transaction-id". >>> >>> >>> >>> >>> >>> >>> >>> -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups "Clojure" group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
