Re: Storing tweets For WC2014

2014-05-16 Thread Aman Tandon
I haven't tried situation this this but as per your requirements, you can make the schema for defining all those fields required by you like, date, location, etc you can also configure the faceting form solrconfig.xml if you want the same for every request. You should give it a try by allocating

Re: Storing tweets For WC2014

2014-05-16 Thread Michael Della Bitta
Some of the data providers for Twitter offer a search API. Depending on what you're doing, you might not even need to host this yourself. My company does do search and analytics over tweets, but by the time we end up indexing them, we've winnowed down the initial set to 10% of what we've

Re: Storing tweets For WC2014

2014-05-16 Thread Alexandre Rafalovitch
That's a lot of tweets. There is an article talking about smaller scale lessons, might be still useful: http://ricston.com/blog/guerrilla-search-solr-run-3-million-documents-search-15month-machine/ Regards, Alex. Personal website: http://www.outerthoughts.com/ Current project:

Storing tweets For WC2014

2014-05-15 Thread Cool Techi
Hi, We have a requirement from one of our customers to provide search and analytics on the upcoming Soccer World cup, given the sheer volume of tweet's that would be generated at such an event I cannot imagine what would be required to store this in solr. It would be great if there can be some