Francis Avila, Thank you for your response. The Java app is using Jedis and the Clojure app is using Carmine. I'm wondering if you can suggest what you think would be the easiest way to allow these 2 apps to understand each other's strings?
You were correct about how unsafe the above code was. I tested it for less than 15 minutes and ran into the fact that a \n newline made a mess of everything. On Wednesday, July 8, 2015 at 5:15:35 PM UTC-4, Francis Avila wrote: > > You are running into Carmine's automatic nippy serialization. > https://github.com/ptaoussanis/carmine#serialization > > Redis only stores byte arrays (what it calls "strings"). Carmine uses the > nippy library (the meaning of "NPY" in your byte stream) to represent rich > types compactly as bytes. https://github.com/ptaoussanis/nippy > > If you give Carmine a byte array to store, it will store it directly > without nippy-encoding it. E.g. (.getBytes "{}" "UTF-8") > > BTW your document-as-string example is extremely unsafe: how will you > reliably read this message out again? e.g. what if the 'debrief' string > contains a single quote? Use a proper serialization format. > > So the key is to have both your Clojure and Java app store *bytes* in > Redis using the same serialization. You can store anything you want (nippy, > utf-8-encoded json, fressian, bson, utf-8 xml, utf-16 java strings, > whatever) as long as it's bytes and it's read and written the same way in > all your apps. > > The Redis library your Java app is using may have its own automatic > de/serialization, too. You need to find out what it's doing and either work > with this or turn it off, just like with Carmine. > > Nippy unfortunately does not have a Java API out of the box: > https://github.com/ptaoussanis/nippy/issues/66 > > > On Wednesday, July 8, 2015 at 3:35:49 PM UTC-5, gingers...@gmail.com > wrote: >> >> >> I am not sure if this is a Clojure question or a Java question. I don't >> know Java, so I could use whatever help folks can offer. >> >> "primitive string" here means what I can write when I am at the terminal. >> >> We have 2 apps, one in Clojure, one in Java. They talk to each other via >> Redis. I know the Java app can read stuff out of Redis, using our >> "transaction-id", if I use the terminal and open up "redis-clj" and write a >> string directly from the terminal. But I have this Clojure code, which >> depends on Peter Taoussanis's Carmine library: >> (defn worker [document] >> {:pre [(string? (:transaction-id document))]} >> (let [transaction-id (:transaction-id document) >> document-as-string (str "{'transaction-id' : '" transaction-id >> "', 'debrief' : '" (:debrief document) "'}" ) >> redis-connection {:pool {} :spec {:host "127.0.0.1" :port 6379 }}] >> (timbre/log :trace " message we will send to NLP " >> document-as-string) >> (carmine/wcar redis-connection (carmine/set transaction-id document)) >> (loop [document-in-redis (carmine/wcar redis-connection (carmine/get >> transaction-id))] >> >> (if-not (.contains (first document-in-redis) "processed") >> (recur (carmine/wcar redis-connection (carmine/get >> transaction-id))) >> (do >> (carmine/wcar redis-connection (carmine/del transaction-id)) >> document-in-redis))))) >> >> This line in particular, I have tried doing this several ways: >> >> document-as-string (str "{'transaction-id' : '" transaction-id >> "', 'debrief' : '" (:debrief document) "'}" ) >> >> In Redis, I expect to see: >> >> {'transaction-id' : '42e574e7-3b80-424a-b9ff-01072f1e0358', 'debrief' : >> 'Smeek Hallie of Withers, Smeg, Harrington and Norvig responded to our >> proposal and said his company is read to move forward. The rate of $400 per >> ton of shredded paper was acceptable to them, and they shred about 2 tons >> of documents every month. $96,000 in potential revenue annually. I will >> meet with him tomorrow and we will sign the contract.'} >> >> But if I then launch redis-cli, I see: >> >> >> 127.0.0.1:6379> keys * >> 1) "42e574e7-3b80-424a-b9ff-01072f1e0358" >> >> 127.0.0.1:6379> get "42e574e7-3b80-424a-b9ff-01072f1e0358" >> "\x00>NPY\b\x00\x00\x01\xfc\xf1\xfe\x1b\x00\x00\x00\nj\nip-addressi\x0e165.254.84.238j\x05tokeni$46b87d64-cff3-4b8b-895c-e089ac59544dj\x0bapi-versioni\x02v1j\x0etransaction-idi$42e574e7-3b80-424a-b9ff-01072f1e0358j\adebrief\r\x00\x00\x01YSmeek >> >> Hallie of Withers, Smeg, Harrington and Norvig responded to our proposal >> and said his company is rea-\x00\xf1\x06move forward. The >> raty\x00\xf0\x0c$400 per ton of shredded pa\x16\x00\xf1\bwas acceptable to >> them,q\x00Bthey0\x00\x80 about 2E\x00\x10sF\x00\xf1Ldocuments every month. >> $96,000 in potential revenue annually. I will meet with him >> tomorrow{\x00\"we#\x00@sign\x92\x00\xa0 contract." >> >> >> I don't know what all of those extra characters are. The Java app is not >> picking this item up, so I assume the Java app is not seeing this as a >> string. I expected this to look the same as if I had written this at the >> terminal: >> >> {'transaction-id' : '42e574e7-3b80-424a-b9ff-01072f1e0358', 'debrief' : >> 'Smeek Hallie of Withers, Smeg, Harrington and Norvig responded to our >> proposal and said his company is read to move forward. The rate of $400 per >> ton of shredded paper was acceptable to them, and they shred about 2 tons >> of documents every month. $96,000 in potential revenue annually. I will >> meet with him tomorrow and we will sign the contract.'} >> >> I assume it is easy to get a string into a format that can be understood >> by both a Clojure app and a Java app. I don't care what format that is, but >> it needs to be consistent. >> >> Can anyone make suggestions about what I can do to make sure the Clojure >> app and the Java app both write to Redis using a format that the other will >> understand? In particular, both apps need to see the "'transaction-id". >> >> >> >> >> >> >> >> -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups "Clojure" group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
