Re: HdfsSpout

2016-02-09 Thread K Zharas
Yes, I have faced above mentioned problems after following that. Please, reread my question. On Wed, Feb 10, 2016 at 1:37 AM, Artem Ervits wrote: > does this help? > https://github.com/apache/storm/tree/master/external/storm-hdfs > > On Tue, Feb 9, 2016 at 1:44 AM, K Zharas wrote:

Re: HdfsSpout

2016-02-08 Thread K Zharas
That is all about "HdfsBolt", not "HdfsSpout" On Tue, Feb 9, 2016 at 7:09 AM, Artem Ervits wrote: > Here is some info > http://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.3.4/bk_storm-user-guide/content/writing-data-with-storm-hdfs-connector.html > On Feb 8, 20

HdfsSpout

2016-02-08 Thread K Zharas
Hi. 1) How can I set "HdfsSpout" so that it will emit a tuple every X seconds? Is it done by "hdfsspout.commit.sec = 30"? 2) I have only one file which has 300+ lines. After processing all the line it does not move it to "done" directory, and it still has ".lock" & ".inprogress". Also, i

Re: Storm + HDFS

2016-02-04 Thread K Zharas
storm-hdfs dependency in your project: > > > org.apache.storm > storm-hdfs > 1.0.0-SNAPSHOT > > > You can find more information on using the spout and other HDFS components > here: > > > https://github.com/apache/storm/tree/1.x-branch/external/storm-hdfs#hdf

Re: Storm + HDFS

2016-02-03 Thread K Zharas
make it available for development. > > From: K Zharas > Reply-To: "user@storm.apache.org" > Date: Wednesday, February 3, 2016 at 11:41 AM > To: "user@storm.apache.org" > Subject: Re: Storm + HDFS > > Yes, looks like it is. But, I have added dependenc

Re: Storm + HDFS

2016-02-03 Thread K Zharas
jar with the proper storm maven > dependency). > > Cheers, > Nick > > On Wed, Feb 3, 2016 at 2:31 PM, K Zharas wrote: > >> It throws and error that packages does not exist. I have also tried >> changing org.apache to backtype, still got an error but only for >

Re: Storm + HDFS

2016-02-03 Thread K Zharas
wrote: > Storm does provide HdfsSpout and HdfsBolt already. Just use those, > instead of writing your own spout/bolt: > > https://github.com/apache/storm/tree/master/external/storm-hdfs > > -Matthias > > > On 02/03/2016 12:34 PM, K Zharas wrote: > > Can anyone hel

Re: Storm + HDFS

2016-02-03 Thread K Zharas
eredReader br = new BufferedReader(new InputStreamReader(fs.open(pt))); String line = br.readLine(); while (line != null){ System.out.println(line); line=br.readLine(); _collector.emit(new Values(line)); } } On Tue, Feb 2, 2016 at 1:19 PM, K Zharas wr

Storm + HDFS

2016-02-01 Thread K Zharas
Hi. I have a project I'm currently working on. The idea is to implement "scikit-learn" into Storm and integrate it with HDFS. I've already implemented "scikit-learn". But, currently I'm using a text file to read and write. However, I need to use HDFS, but finding it hard to integrate with HDFS.

Storm + sklearn

2016-01-02 Thread K Zharas
Hi. After running storm-start's topologies, I decided to create my own topology by implementing sklearn. However, I get errors as I expected. My questions are 1) Am I in the right path of implementing sklearn into Storm? How to fix this topology? 2) Do I have to run "mvn clean install -DskipTests=

Re: WordCountTopology Error

2016-01-02 Thread K Zharas
It works using bin/storm as suggested by Matthias J. Sax <http://stackoverflow.com/users/4953079/matthias-j-sax>. On Sun, Jan 3, 2016 at 2:12 PM, K Zharas wrote: > There is no storm.py in multilang/resource by default. Should I copy > storm.py into multilang/resource? I haven't

Re: WordCountTopology Error

2016-01-02 Thread K Zharas
ultilang/resource so that splitsentence.py > can import it. Did you change folder structure or delete storm.py or > something else? > 在2016年01月03日 03:14,K Zharas 写道: > > Hi. > > I have been following a storm-starter guide, but still faced a problem > with WordCountTopol

WordCountTopology Error

2016-01-02 Thread K Zharas
Hi. I have been following a storm-starter guide, but still faced a problem with WordCountTopology. However, my ExclamationTopology runs successfully. When I run WordCountTopology, it gives me an error below --java.lang.RuntimeException: backtype.storm.multilang.NoOutputException: Pipe --to subpr