We considered that but the problem is that would be the only thing we use that cache for. We also considered putting it in the Cassandra database but that seemed wasteful for what is very transient data.
-- Robert On Thursday, November 30, 2017 at 11:45:11 AM UTC-6, Rob Crawford wrote: > > Maybe a shared cache would be a better way to deal with it? > > On Thursday, November 30, 2017 at 12:41:28 PM UTC-5, kraythe wrote: >> >> Yeah the problem is it cant be broken up, its sort of all or nothing. I >> cant be too specific beyond it being a parametric matrix. Believe me I >> would prefer not shipping it around. >> >> On Thursday, November 30, 2017 at 9:11:51 AM UTC-6, Patrik Nordwall wrote: >>> >>> Sending 10 MB messages is indeed recipie for trouble. >>> >>> I’d suggest to split the 10 MB into 100 smaller messages and collect, >>> reassembe, them on the receiving side. >>> >>> Perhaps you are fine with loosing such messages since they are >>> periodically published again. Otherwise you could use DistributedData to >>> share them. >>> >>> /Patrik >>> tors 30 nov. 2017 kl. 14:35 skrev kraythe <kra...@gmail.com>: >>> >>>> Greetings, >>>> >>>> We have an actor structure where there is a cluster sharded actor that >>>> calculates a parametric matrix of about 7 meg and has to distribute it to >>>> 5 >>>> other nodes for consumption. First of all there are the following >>>> constraints. >>>> >>>> 1. The matrix has to be generated in one place. >>>> 2. The matrix is needed on all nodes to handle user load. >>>> 3. The matrix will be periodically generated again to handle changing >>>> variables and then sent off to all nodes to handle user load. >>>> 4. Saving the matrix in a database is probably not viable as it merely >>>> shifts the networking load and the database would get very large very >>>> fast. >>>> The Database only saves the input params. >>>> >>>> We changed the akka max message size to 10 meg to accomplish this but >>>> that feels a bit odd, but we didn't see another choice. Normally this >>>> works >>>> fine even though passing 10 meg messages around a DistributedPubSub seems >>>> odd to me. However on startup the system has to stat up 2000 of these all >>>> at once. As a result the sharding coordinators scream at us about buffered >>>> messages. It eventually calms down and life resumes but I would love to be >>>> able to do this without the bloodbath in the logs. >>>> >>>> Can someone recommend an alternative strategy for handling the >>>> distribution of the parametric matrix that gets it to every node but >>>> doesn't cause shard coordinator complaint bloodbath? >>>> >>>> Thanks in advance. >>>> Robert SImmons Jr. MSc. >>>> >>>> -- >>>> >>>>>>>>>> Read the docs: http://akka.io/docs/ >>>> >>>>>>>>>> Check the FAQ: >>>> http://doc.akka.io/docs/akka/current/additional/faq.html >>>> >>>>>>>>>> Search the archives: >>>> https://groups.google.com/group/akka-user >>>> --- >>>> You received this message because you are subscribed to the Google >>>> Groups "Akka User List" group. >>>> To unsubscribe from this group and stop receiving emails from it, send >>>> an email to akka-user+...@googlegroups.com. >>>> To post to this group, send email to akka...@googlegroups.com. >>>> Visit this group at https://groups.google.com/group/akka-user. >>>> For more options, visit https://groups.google.com/d/optout. >>>> >>> -- >>>>>>>>>> Read the docs: http://akka.io/docs/ >>>>>>>>>> Check the FAQ: >>>>>>>>>> http://doc.akka.io/docs/akka/current/additional/faq.html >>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user --- You received this message because you are subscribed to the Google Groups "Akka User List" group. To unsubscribe from this group and stop receiving emails from it, send an email to akka-user+unsubscr...@googlegroups.com. To post to this group, send email to akka-user@googlegroups.com. Visit this group at https://groups.google.com/group/akka-user. For more options, visit https://groups.google.com/d/optout.
