Chandresh,
I think that the suggestion of Roman is preferable, we can use the
following code :
Let me know if you have problems
/** Start a simulated topology */
public void startTopology(StormStreamer stormStreamer) {
MkClusterParam mkClusterParam = new MkClusterParam();
mkClusterParam.setSupervisors(4);
Config daemonConf = new Config();
daemonConf.put(Config.STORM_LOCAL_MODE_ZMQ, false);
mkClusterParam.setDaemonConf(daemonConf);
Testing.withSimulatedTimeLocalCluster(mkClusterParam, new TestJob()
{
@Override
public void run(ILocalCluster cluster) throws
IOException {
TopologyBuilder builder = new TopologyBuilder();
builder.setSpout("testSpout", new StormSpout());
builder.setBolt("bolt", stormStreamer)
.shuffleGrouping("spout");
StormTopology topology = builder.createTopology();
MockedSources mockedSources = new MockedSources();
//Our spout will be processing this values.
mockedSources.addMockData("testSpout",new
Values(stormStream.getKeyValMap()));
// prepare the config
Config conf = new Config();
conf.setNumWorkers(2);
CompleteTopologyParam completeTopologyParam = new
CompleteTopologyParam();
completeTopologyParam.setMockedSources(mockedSources);
completeTopologyParam.setStormConf(conf);
Map result = Testing.completeTopology(cluster,
topology, completeTopologyParam);
}
}
);
}
2015-09-28 6:11 GMT+02:00 chandresh pancholi <[email protected]
>:
> Agree,
> Will check for some other solution and update!!
>
> On Mon, Sep 28, 2015 at 9:12 AM, Vishal Garg <[email protected]> wrote:
>
> > Chandresh,
> >
> > Generally sleep in tests is not a good idea. Will try to look into your
> > test this week, see if we can avoid it.
> > Meanwhile you can look at how storm does its own unit testing if you
> could
> > find one?
> > Vishal
> >
> > Sent from my iPhone
> >
> > > On Sep 27, 2015, at 8:23 PM, Roman <[email protected]> wrote:
> > >
> > > I also have concerns about this sleep().Storm has testing facilities
> you
> > can use for your test. How about Testing.withSimulatedTimeLocalCluster()?
> > > Roman
> > >
> > >
> > >
> > > On Monday, September 28, 2015 5:39 AM, Dmitriy Setrakyan <
> > [email protected]> wrote:
> > >
> > >
> > > On Sun, Sep 27, 2015 at 1:36 PM, chandresh pancholi <
> > > [email protected]> wrote:
> > >
> > >> Gian,
> > >> I have checked the code. It looks good.
> > >> Just one little change make it 20 sec.
> > >>
> > >> /* Topology will run for 10sec */
> > >> Utils.sleep(20000);
> > >
> > > Hm... Can someone explain what is this sleep() for?
> > >
> > >
> > >>
> > >>
> > >>
> > >> On Sun, Sep 27, 2015 at 4:30 PM, Gianfranco Murador <
> > >> [email protected]> wrote:
> > >>
> > >>> Chandresh, we're ready to do a pull request. Look here:
> > >>> https://github.com/murador/ignite/tree/IGNITE-429/modules/storm
> > >>>
> > >>> 2015-09-27 8:49 GMT+02:00 chandresh pancholi <
> > >>> [email protected]
> > >>>> :
> > >>>
> > >>>> Here is the log file.
> > >>>>
> > >>>> Thanks
> > >>>>
> > >>>> On Sat, Sep 26, 2015 at 10:40 PM, Gianfranco Murador <
> > >>>> [email protected]> wrote:
> > >>>>
> > >>>>> Chan,
> > >>>>> I can run the test succefully, but sometimes I get this error:
> > >>>>> Caused by: class org.apache.ignite.spi.IgniteSpiException: Failed
> to
> > >> add
> > >>>>> node to topology because remote node is configured to use loopback
> > >>>>> address,
> > >>>>> but local node is not (consider changing 'localAddress'
> configuration
> > >>>>> parameter).
> > >>>>>
> > >>>>> can you send the log file?
> > >>>>> Regards, Gianfranco
> > >>>>>
> > >>>>>
> > >>>>>
> > >>>>> 2015-09-26 17:13 GMT+02:00 chandresh pancholi <
> > >>>>> [email protected]>:
> > >>>>>
> > >>>>>> Gian,
> > >>>>>> Its still giving "Ouch! Argument cannot be null: streamer"
> > >>>>>>
> > >>>>>> On Sat, Sep 26, 2015 at 7:24 PM, Gianfranco Murador <
> > >>>>>> [email protected]> wrote:
> > >>>>>>
> > >>>>>>> Chan,
> > >>>>>>> I've fixed some issue, please take a look here:
> > >>
> >
> https://github.com/murador/ignite/blob/IGNITE-429/modules/storm/src/test/java/org/apache/ignite/stream/storm/StormIgniteStreamerSelfTest.java
> > >>>>>>>
> > >>>>>>> 2015-09-26 8:29 GMT+02:00 chandresh pancholi <
> > >>>>>>> [email protected]
> > >>>>>>>> :
> > >>>>>>>
> > >>>>>>>> Gian,
> > >>>>>>>>
> > >>>>>>>> I have followed instructions given by you. I instantiate
> > >>>>>>> IgniteDateStreamer
> > >>>>>>>> and ignite in prepare() method of bolt. but getting "*Cache
> > >>> doesn't
> > >>>>>>>> exist*".
> > >>>>>>>>
> > >>>>>>>> Code snippet from stormStreamer class in prepare() method
> > >>>>>>>>
> > >>>>>>>> StormStreamer<String, String, String> stormStreamer = null;
> > >>>>>>>>
> > >>>>>>>> Ignite ignite = Ignition.start();
> > >>>>>>>>
> > >>>>>>>> System.out.println(" ignite ====== "+ignite.toString());
> > >>>>>>>>
> > >>>>>>>> try(IgniteDataStreamer<String, String> stmr =
> > >>>>>>>> ignite.dataStreamer("cache-0")) {
> > >>>>>>>>
> > >>>>>>>> stormStreamer = new StormStreamer<>();
> > >>>>>>>>
> > >>>>>>>> IgniteCache<String, String> cache = ignite.cache("cache-0");
> > >>>>>>>>
> > >>>>>>>> stmr.allowOverwrite(true);
> > >>>>>>>>
> > >>>>>>>> /* Set ignite instance */
> > >>>>>>>> stormStreamer.setIgnite(ignite);
> > >>>>>>>>
> > >>>>>>>> /* Set streamer instance */
> > >>>>>>>> stormStreamer.setStreamer(stmr);
> > >>>>>>>>
> > >>>>>>>> /* set thread count */
> > >>>>>>>> setThreads(5);
> > >>>>>>>>
> > >>>>>>>> }
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>>> On Thu, Sep 24, 2015 at 4:19 PM, Anton Vinogradov <
> > >>>>>>>> [email protected]>
> > >>>>>>>> wrote:
> > >>>>>>>>
> > >>>>>>>>> Chandresh,
> > >>>>>>>>> As far as understand IBolt implementation should setup all
> > >>>>> external
> > >>>>>>>>> connections at .prepare() method.
> > >>>>>>>>> So, better way is to get existing Ignite instance or create
> > >> new
> > >>> at
> > >>>>>>>>> .prepare()
> > >>>>>>>>> method.
> > >>>>>>>>>
> > >>>>>>>>>
> > >>>>>>>>> On Thu, Sep 24, 2015 at 10:55 AM, Gianfranco Murador <
> > >>>>>>>>> [email protected]> wrote:
> > >>>>>>>>>
> > >>>>>>>>>> Chandresh,
> > >>>>>>>>>> I will launch the unit test later this week , if the problem
> > >>>>>>> persists.
> > >>>>>>>> I
> > >>>>>>>>>> think that you should instantiate Ignite through the start()
> > >>>>> method
> > >>>>>>> of
> > >>>>>>>>>> Ignition class
> > >>>>>>>>>> and retrieve the Ignite cache specifying a name for the
> > >>> default
> > >>>>>>> cache.
> > >>>>>>>>> You
> > >>>>>>>>>> should provide a name also for data streamer, not passing
> > >>> null.
> > >>>>>>>>>> Could you do this and run again the unit test ?
> > >>>>>>>>>> Thank you,
> > >>>>>>>>>> Regards,
> > >>>>>>>>>> Gianfranco
> > >>>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>>> 2015-09-24 5:40 GMT+02:00 chandresh pancholi <
> > >>>>>>>>>> [email protected]
> > >>>>>>>>>>> :
> > >>>>>>>>>>
> > >>>>>>>>>>> Anton,
> > >>>>>>>>>>>
> > >>>>>>>>>>> Clone this repo
> > >>> https://github.com/chandresh-pancholi/ignite
> > >>>>> and
> > >>>>>>> run
> > >>>>>>>>>> this
> > >>>>>>>>>>> class
> > >>
> >
> https://github.com/chandresh-pancholi/ignite/blob/master/modules/storm/src/test/java/org/apache/ignite/stream/storm/IgniteStormStreamerSelfTestSuite.java
> > >>>>>>>>>>>
> > >>>>>>>>>>> You will get the Error saying "Oouch,Argument is Null" for
> > >>>>>>>>> getStreamer().
> > >>>>>>>>>>>
> > >>>>>>>>>>> On Wed, Sep 23, 2015 at 9:46 PM, Anton Vinogradov <
> > >>>>>>>>>>> [email protected]>
> > >>>>>>>>>>> wrote:
> > >>>>>>>>>>>
> > >>>>>>>>>>>> Chandresh,
> > >>>>>>>>>>>> I'm trying to analize problem.
> > >>>>>>>>>>>> Could you please provide more details and explain
> > >>>>> step-by-step
> > >>>>>>> how
> > >>>>>>>>> can
> > >>>>>>>>>> I
> > >>>>>>>>>>>> reproduce problem?
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> On Tue, Sep 22, 2015 at 11:06 PM, chandresh pancholi <
> > >>>>>>>>>>>> [email protected]> wrote:
> > >>>>>>>>>>>>
> > >>>>>>>>>>>>> Not yet.
> > >>>>>>>>>>>>> Still doing brainstorming why StormStreamer object is
> > >>>>> getting
> > >>>>>>>> NULLL
> > >>>>>>>>>>>>> in-spite setting up ignite and streamer instances.
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>> On Tue, Sep 22, 2015 at 7:22 PM, Vishal Garg <
> > >>>>>>> [email protected]
> > >>>>>>>>>
> > >>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>>> Did you figure it out? Something related to your
> > >> storm
> > >>>>>>>>> integration?
> > >>>>>>>>>>>>>> Vishal
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>> Sent from my iPhone
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>> On Sep 22, 2015, at 4:50 AM, chandresh pancholi <
> > >>>>>>>>>>>>>>> [email protected]> wrote:
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>> Problem here is with Storm. Its making streamer
> > >>> object
> > >>>>>> null
> > >>>>>>>>> once
> > >>>>>>>>>> it
> > >>>>>>>>>>>>> goes
> > >>>>>>>>>>>>>>> into submit topology.
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>> On Mon, Sep 21, 2015 at 7:13 PM, Lalit Kumar Jha <
> > >>>>>>>>>>>> [email protected]
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>> Hi Chandresh,
> > >>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>> See test suite class here, its based on
> > >> annotations
> > >>
> >
> https://github.com/sylentprayer/ignite/blob/ignite-530/modules/twitter/src/test/java/org/apache/ignite/stream/twitter/IgniteTwitterStreamerTestSuite.java
> > >>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>> No code required in class body.
> > >>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>> On Mon, Sep 21, 2015 at 6:24 PM, chandresh
> > >>> pancholi <
> > >>>>>>>>>>>>>>>> [email protected]> wrote:
> > >>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>> Hi Gian/Vishal/Lalit,
> > >>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>> You guys have worked on Streamer integration
> > >> with
> > >>>>>>> Ignite. I
> > >>>>>>>>> am
> > >>>>>>>>>>>>> working
> > >>>>>>>>>>>>>> on
> > >>>>>>>>>>>>>>>>> Storm-ignite integration.
> > >>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>> I am setting up setStreamer() and setIgnite() in
> > >>>>> Test
> > >>>>>>> class
> > >>>>>>>>> and
> > >>>>>>>>>>>>> sending
> > >>>>>>>>>>>>>>>>> this object to setBolt() method.
> > >>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>> When i try to read via gettreamer() and
> > >>> getIgnite()
> > >>>>> i
> > >>>>>> am
> > >>>>>>>>>> getting
> > >>>>>>>>>>>>> NULL.
> > >>>>>>>>>>>>>> I
> > >>>>>>>>>>>>>>>> am
> > >>>>>>>>>>>>>>>>> trying to find out why is it happening but No
> > >>> Luck.
> > >>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>> If any one of you find some time and look into
> > >> the
> > >>>>> code
> > >>>>>>> and
> > >>>>>>>>>> help
> > >>>>>>>>>>> me
> > >>>>>>>>>>>>> to
> > >>>>>>>>>>>>>>>>> resolve this. It would be great.
> > >>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>> Git :
> > >>> https://github.com/chandresh-pancholi/ignite
> > >>>>>>>>>>>>>>>>> Module : storm
> > >>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>> --
> > >>>>>>>>>>>>>>>>> Chandresh Pancholi
> > >>>>>>>>>>>>>>>>> Senior Software Engineer
> > >>>>>>>>>>>>>>>>> Flipkart.com
> > >>>>>>>>>>>>>>>>> Email-id:[email protected]
> > >>>>>>>>>>>>>>>>> Contact:08951803660
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>> --
> > >>>>>>>>>>>>>>> Chandresh Pancholi
> > >>>>>>>>>>>>>>> Senior Software Engineer
> > >>>>>>>>>>>>>>> Flipkart.com
> > >>>>>>>>>>>>>>> Email-id:[email protected]
> > >>>>>>>>>>>>>>> Contact:08951803660
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>> --
> > >>>>>>>>>>>>> Chandresh Pancholi
> > >>>>>>>>>>>>> Senior Software Engineer
> > >>>>>>>>>>>>> Flipkart.com
> > >>>>>>>>>>>>> Email-id:[email protected]
> > >>>>>>>>>>>>> Contact:08951803660
> > >>>>>>>>>>>
> > >>>>>>>>>>>
> > >>>>>>>>>>>
> > >>>>>>>>>>> --
> > >>>>>>>>>>> Chandresh Pancholi
> > >>>>>>>>>>> Senior Software Engineer
> > >>>>>>>>>>> Flipkart.com
> > >>>>>>>>>>> Email-id:[email protected]
> > >>>>>>>>>>> Contact:08951803660
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>>> --
> > >>>>>>>> Chandresh Pancholi
> > >>>>>>>> Senior Software Engineer
> > >>>>>>>> Flipkart.com
> > >>>>>>>> Email-id:[email protected]
> > >>>>>>>> Contact:08951803660
> > >>>>>>
> > >>>>>>
> > >>>>>>
> > >>>>>> --
> > >>>>>> Chandresh Pancholi
> > >>>>>> Senior Software Engineer
> > >>>>>> Flipkart.com
> > >>>>>> Email-id:[email protected]
> > >>>>>> Contact:08951803660
> > >>>>
> > >>>>
> > >>>>
> > >>>> --
> > >>>> Chandresh Pancholi
> > >>>> Senior Software Engineer
> > >>>> Flipkart.com
> > >>>> Email-id:[email protected]
> > >>>> Contact:08951803660
> > >>
> > >>
> > >>
> > >> --
> > >> Chandresh Pancholi
> > >> Senior Software Engineer
> > >> Flipkart.com
> > >> Email-id:[email protected]
> > >> Contact:08951803660
> > >
> > >
> >
>
>
>
> --
> Chandresh Pancholi
> Senior Software Engineer
> Flipkart.com
> Email-id:[email protected]
> Contact:08951803660
>
