Hello beachers, I am reaching out to the brains trust for a solution to an age-old problem. I have two "identical" AWS EC2 servers streaming tweets from Twitter and putting the results of a filter into an end process via AWS SQS. The data should effectively be identical although the timing is not guaranteed to be. What's the most effective and efficient way to get only one copy of the data to the end process? I am leaning towards writing each event (or a hash) into a unique-keyed database table to detect and remove duplicates but I am looking for alternative ideas.
Anyone? -- You received this message because you are subscribed to the Silicon Beach Australia mailing list. Vist http://siliconbeachaustralia.org for more Forum rules 1) No lurkers! It is expected that you introduce yourself. 2) No jobs postings. You can use http://siliconbeachaustralia.org/jobs To post to this group, send email to [email protected] To unsubscribe from this group, send email to [email protected] For more options, visit this group at http://groups.google.com/group/silicon-beach-australia?hl=en?hl=en
