Check out this datastax article
http://www.datastax.com/documentation/cassandra/2.0/cassandra/tools/toolsBulkloader_t.html
And code examples can be found here
https://github.com/PatrickCallaghan/datastax-bulkloader-writer-example
You can write a writer in scala or Java which will convert csv et into ss
tables and then use sstableloader to load direct into Cassandra
K
--
Keith Sterling
Head of Software
E: keith.sterl...@first-utility.com
P: +44 7771 597 630
W: first-utility.com
A: Opus 40 Business Park,
Haywood Road, Warwick CV34 5AH
On Sat, Dec 27, 2014 at 1:11 PM, Jack Krupansky
wrote:
> Sorry, but you are still not being clear. In particular, "website data" has
> no common, defined meaning. You'll need to use some standard, defined
> terminology or specific examples so that we can have some idea what you are
> referring to.
> The blog post you cited is referring to the Twitter API, presumably to read
> tweets. Okay, fine, but you'll have to be more specific about what you want
> to do with them. Yes, Cassandra is primarily focus on structured data, but
> you can of course store unstructured and semi-structured data as blobs,
> JSON strings, map columns, etc.
> Please describe in a little more detail what problem you are trying to
> solve.
> I mean, "website data" might mean any data (in any format) stored at a web
> URL, which might be a web "page", a data file linked by a web page, or...
> it could be a REST API like Twitter). Or it could be... whatever. Cassandra
> is basically a storage engine - it can store anything. There are a wide
> variety of tools that can be used to "ingest" data from the infinite
> variety of "sources" for data. But you'll need to state more specifically
> what you are actually tring to accomplish.
> Also, "large data" could be... anything, like "Big Data". So more
> specificity is needed.
> Alternatively, you could hire a consultant to help guide you through the
> "application analysis" process to determine your "application
> requirements", and then you could simply post your application
> requirements, or at least a concise summary or relevant excerpt.
> -- Jack Krupansky
> -- Jack Krupansky
> On Sat, Dec 27, 2014 at 1:48 AM, Joanne Contact
> wrote:
>> Thank you. I did not express clearly on my question.
>>
>> I wonder if there is sample code to load any website data to Cassandra?
>>
>> Say, this webpage http://datatomix.com/?p=84 seems to use Python, tweepy,
>> to use twitter API to get data in json format and then load data into
>> Cassandra.
>>
>> So it seems tweepy is special for twitter API. Is there a code for any
>> website?
>> Btw I am not familiar with Python yet. So the answer may not be limited to
>> Python.
>>
>> Thanks!
>>
>> On Fri, Dec 26, 2014 at 12:46 PM, Keith Sterling <
>> keith.sterl...@first-utility.com> wrote:
>>
>>> Take a look at sstableloader. We use it to load 30+m rows into Cassandra
>>>
>>> Datastax documentation is a good staty
>>>
>>> --
>>> *Keith Sterling*
>>> *Head of Software*
>>>
>>> *E:* keith.sterl...@first-utility.com
>>> *P:* +44 7771 597 630
>>> *W:* first-utility.com <http://www.first-utility.com/>
>>> *A:* Opus 40 Business Park,
>>> Haywood Road, Warwick CV34 5AH
>>>
>>>
>>>
>>> On Fri, Dec 26, 2014 at 7:59 PM, Joanne Contact
>>> wrote:
>>>
>>>> Hello I am new. Did not seem to find the answer after a brief
>>>> research. Please help.
>>>>
>>>> Thanks!
>>>>
>>>> J
>>>>
>>>>
>>>
>>
