Re: Spring bolts
Hi, can you explain why it won't work? We use Breeze to read messages from a Redis queue and have completely defined our beans in Spring. Please create an issue in the github repo so we can help you further. 2014/1/5 Michal Singer mic...@leadspace.com Hi, I checked this out. this project does give a way to run spring beans inside a bolt or a spout. But if I want for example to define a spout which is a JMS listener of activemq and is configured in spring it won’t work. I will need to define a regular spout which reads from a spring bean which is a jms listener. This will be problematic as to a point of failure since I will not read directly from jms queue (Activemq) but instead from an internal queue which will be somhow loaded from the AMQ. Thanks *From:* Eugene [mailto:edvor...@gmail.com] *Sent:* Wednesday, December 25, 2013 10:11 PM *To:* user@storm.incubator.apache.org *Subject:* Re: Spring bolts Did you look into this project: https://github.com/internet-research-network/breeze/ I did not try it yet, but it may give you what are you looking for. Merry Christmas! On Wed, Dec 25, 2013 at 10:54 AM, Michal Singer mic...@leadspace.com wrote: Thanks, Merry Christmas to you! *From:* Michael Rose [mailto:mich...@fullcontact.com] *Sent:* Wednesday, December 25, 2013 5:48 PM *To:* user@storm.incubator.apache.org *Subject:* RE: Spring bolts Yes, you'll need a Spring context in prepare. Given you have multiple bolts per JVM, its worth ensuring only one creates it in prepare then shares that context. We do this with Guice injectors and double checked locks. Each bolt uses the singleton injector to inject its members. I imagine Spring has a similar concept once you have a context. Life cycle of bolts is quite strange in Storm given they're made before deployment and serialized. There's quite a few gotchas. Bolt constructors can't be trusted, thus prepare. There may be a spring storm example out there somewhere. Merry Christmas! On Dec 25, 2013 8:17 AM, Michal Singer mic...@leadspace.com wrote: I am not sure I understand. Spring beans are defined in the spring configuration files. How can I inject them in the members. What I thought to do is that the bolts will not be spring beans and in the prepare method I will initialize the spring context. This way, the bolts will call other spring beans which are not bolts and initialized in spring. But of course this is a very limited solution. *From:* Michael Rose [mailto:mich...@fullcontact.com] *Sent:* Wednesday, December 25, 2013 5:06 PM *To:* user@storm.incubator.apache.org *Subject:* Re: Spring bolts Make a base spring bolt, in your prepare method inject the members. That's the best I've come up with, as prepare happens server side whereas topology config and static initializers happen at deploy time client side. On Dec 25, 2013 7:51 AM, Michal Singer mic...@leadspace.com wrote: Hi, I am trying to understand how to use beans in spring as bolts/spouts. If I have the definition in spring which is initialized once the bolt or spout is initialized. But when creating a topology I need to do: new Bolt()…. And cannot get it from spring. So what is the right way to do this? Thanks, Michal -- Eugene Dvorkin Software Engineer WebMD email: edvor...@gmail.com phone: 646-250-9649 eugenedvorkin.com Connect with me on: LinkedIn http://www.linkedin.com/in/eugenedvorkin Twitterhttp://twitter.com/edvorkin
RE: Spring bolts
Ok. I opened an issue. thanks *From:* Bakker, Jethro [mailto:j.bak...@flusso.nl] *Sent:* Monday, January 06, 2014 2:23 PM *To:* user@storm.incubator.apache.org *Subject:* Re: Spring bolts Hi, can you explain why it won't work? We use Breeze to read messages from a Redis queue and have completely defined our beans in Spring. Please create an issue in the github repo so we can help you further. 2014/1/5 Michal Singer mic...@leadspace.com Hi, I checked this out. this project does give a way to run spring beans inside a bolt or a spout. But if I want for example to define a spout which is a JMS listener of activemq and is configured in spring it won’t work. I will need to define a regular spout which reads from a spring bean which is a jms listener. This will be problematic as to a point of failure since I will not read directly from jms queue (Activemq) but instead from an internal queue which will be somhow loaded from the AMQ. Thanks *From:* Eugene [mailto:edvor...@gmail.com] *Sent:* Wednesday, December 25, 2013 10:11 PM *To:* user@storm.incubator.apache.org *Subject:* Re: Spring bolts Did you look into this project: https://github.com/internet-research-network/breeze/ I did not try it yet, but it may give you what are you looking for. Merry Christmas! On Wed, Dec 25, 2013 at 10:54 AM, Michal Singer mic...@leadspace.com wrote: Thanks, Merry Christmas to you! *From:* Michael Rose [mailto:mich...@fullcontact.com] *Sent:* Wednesday, December 25, 2013 5:48 PM *To:* user@storm.incubator.apache.org *Subject:* RE: Spring bolts Yes, you'll need a Spring context in prepare. Given you have multiple bolts per JVM, its worth ensuring only one creates it in prepare then shares that context. We do this with Guice injectors and double checked locks. Each bolt uses the singleton injector to inject its members. I imagine Spring has a similar concept once you have a context. Life cycle of bolts is quite strange in Storm given they're made before deployment and serialized. There's quite a few gotchas. Bolt constructors can't be trusted, thus prepare. There may be a spring storm example out there somewhere. Merry Christmas! On Dec 25, 2013 8:17 AM, Michal Singer mic...@leadspace.com wrote: I am not sure I understand. Spring beans are defined in the spring configuration files. How can I inject them in the members. What I thought to do is that the bolts will not be spring beans and in the prepare method I will initialize the spring context. This way, the bolts will call other spring beans which are not bolts and initialized in spring. But of course this is a very limited solution. *From:* Michael Rose [mailto:mich...@fullcontact.com] *Sent:* Wednesday, December 25, 2013 5:06 PM *To:* user@storm.incubator.apache.org *Subject:* Re: Spring bolts Make a base spring bolt, in your prepare method inject the members. That's the best I've come up with, as prepare happens server side whereas topology config and static initializers happen at deploy time client side. On Dec 25, 2013 7:51 AM, Michal Singer mic...@leadspace.com wrote: Hi, I am trying to understand how to use beans in spring as bolts/spouts. If I have the definition in spring which is initialized once the bolt or spout is initialized. But when creating a topology I need to do: new Bolt()…. And cannot get it from spring. So what is the right way to do this? Thanks, Michal -- Eugene Dvorkin Software Engineer WebMD email: edvor...@gmail.com phone: 646-250-9649 eugenedvorkin.com Connect with me on: LinkedIn http://www.linkedin.com/in/eugenedvorkin Twitterhttp://twitter.com/edvorkin
Re: Spring bolts
Did you raise an issue in https://github.com/internet-research-network/breeze/ ? 2014/1/6 Michal Singer mic...@leadspace.com Ok. I opened an issue. thanks *From:* Bakker, Jethro [mailto:j.bak...@flusso.nl] *Sent:* Monday, January 06, 2014 2:23 PM *To:* user@storm.incubator.apache.org *Subject:* Re: Spring bolts Hi, can you explain why it won't work? We use Breeze to read messages from a Redis queue and have completely defined our beans in Spring. Please create an issue in the github repo so we can help you further. 2014/1/5 Michal Singer mic...@leadspace.com Hi, I checked this out. this project does give a way to run spring beans inside a bolt or a spout. But if I want for example to define a spout which is a JMS listener of activemq and is configured in spring it won’t work. I will need to define a regular spout which reads from a spring bean which is a jms listener. This will be problematic as to a point of failure since I will not read directly from jms queue (Activemq) but instead from an internal queue which will be somhow loaded from the AMQ. Thanks *From:* Eugene [mailto:edvor...@gmail.com] *Sent:* Wednesday, December 25, 2013 10:11 PM *To:* user@storm.incubator.apache.org *Subject:* Re: Spring bolts Did you look into this project: https://github.com/internet-research-network/breeze/ I did not try it yet, but it may give you what are you looking for. Merry Christmas! On Wed, Dec 25, 2013 at 10:54 AM, Michal Singer mic...@leadspace.com wrote: Thanks, Merry Christmas to you! *From:* Michael Rose [mailto:mich...@fullcontact.com] *Sent:* Wednesday, December 25, 2013 5:48 PM *To:* user@storm.incubator.apache.org *Subject:* RE: Spring bolts Yes, you'll need a Spring context in prepare. Given you have multiple bolts per JVM, its worth ensuring only one creates it in prepare then shares that context. We do this with Guice injectors and double checked locks. Each bolt uses the singleton injector to inject its members. I imagine Spring has a similar concept once you have a context. Life cycle of bolts is quite strange in Storm given they're made before deployment and serialized. There's quite a few gotchas. Bolt constructors can't be trusted, thus prepare. There may be a spring storm example out there somewhere. Merry Christmas! On Dec 25, 2013 8:17 AM, Michal Singer mic...@leadspace.com wrote: I am not sure I understand. Spring beans are defined in the spring configuration files. How can I inject them in the members. What I thought to do is that the bolts will not be spring beans and in the prepare method I will initialize the spring context. This way, the bolts will call other spring beans which are not bolts and initialized in spring. But of course this is a very limited solution. *From:* Michael Rose [mailto:mich...@fullcontact.com] *Sent:* Wednesday, December 25, 2013 5:06 PM *To:* user@storm.incubator.apache.org *Subject:* Re: Spring bolts Make a base spring bolt, in your prepare method inject the members. That's the best I've come up with, as prepare happens server side whereas topology config and static initializers happen at deploy time client side. On Dec 25, 2013 7:51 AM, Michal Singer mic...@leadspace.com wrote: Hi, I am trying to understand how to use beans in spring as bolts/spouts. If I have the definition in spring which is initialized once the bolt or spout is initialized. But when creating a topology I need to do: new Bolt()…. And cannot get it from spring. So what is the right way to do this? Thanks, Michal -- Eugene Dvorkin Software Engineer WebMD email: edvor...@gmail.com phone: 646-250-9649 eugenedvorkin.com Connect with me on: LinkedIn http://www.linkedin.com/in/eugenedvorkin Twitterhttp://twitter.com/edvorkin
RE: Spring bolts
Now I did. Thanks. *From:* Bakker, Jethro [mailto:j.bak...@flusso.nl] *Sent:* Monday, January 06, 2014 3:25 PM *To:* user@storm.incubator.apache.org *Subject:* Re: Spring bolts Did you raise an issue in https://github.com/internet-research-network/breeze/ ? 2014/1/6 Michal Singer mic...@leadspace.com Ok. I opened an issue. thanks *From:* Bakker, Jethro [mailto:j.bak...@flusso.nl] *Sent:* Monday, January 06, 2014 2:23 PM *To:* user@storm.incubator.apache.org *Subject:* Re: Spring bolts Hi, can you explain why it won't work? We use Breeze to read messages from a Redis queue and have completely defined our beans in Spring. Please create an issue in the github repo so we can help you further. 2014/1/5 Michal Singer mic...@leadspace.com Hi, I checked this out. this project does give a way to run spring beans inside a bolt or a spout. But if I want for example to define a spout which is a JMS listener of activemq and is configured in spring it won’t work. I will need to define a regular spout which reads from a spring bean which is a jms listener. This will be problematic as to a point of failure since I will not read directly from jms queue (Activemq) but instead from an internal queue which will be somhow loaded from the AMQ. Thanks *From:* Eugene [mailto:edvor...@gmail.com] *Sent:* Wednesday, December 25, 2013 10:11 PM *To:* user@storm.incubator.apache.org *Subject:* Re: Spring bolts Did you look into this project: https://github.com/internet-research-network/breeze/ I did not try it yet, but it may give you what are you looking for. Merry Christmas! On Wed, Dec 25, 2013 at 10:54 AM, Michal Singer mic...@leadspace.com wrote: Thanks, Merry Christmas to you! *From:* Michael Rose [mailto:mich...@fullcontact.com] *Sent:* Wednesday, December 25, 2013 5:48 PM *To:* user@storm.incubator.apache.org *Subject:* RE: Spring bolts Yes, you'll need a Spring context in prepare. Given you have multiple bolts per JVM, its worth ensuring only one creates it in prepare then shares that context. We do this with Guice injectors and double checked locks. Each bolt uses the singleton injector to inject its members. I imagine Spring has a similar concept once you have a context. Life cycle of bolts is quite strange in Storm given they're made before deployment and serialized. There's quite a few gotchas. Bolt constructors can't be trusted, thus prepare. There may be a spring storm example out there somewhere. Merry Christmas! On Dec 25, 2013 8:17 AM, Michal Singer mic...@leadspace.com wrote: I am not sure I understand. Spring beans are defined in the spring configuration files. How can I inject them in the members. What I thought to do is that the bolts will not be spring beans and in the prepare method I will initialize the spring context. This way, the bolts will call other spring beans which are not bolts and initialized in spring. But of course this is a very limited solution. *From:* Michael Rose [mailto:mich...@fullcontact.com] *Sent:* Wednesday, December 25, 2013 5:06 PM *To:* user@storm.incubator.apache.org *Subject:* Re: Spring bolts Make a base spring bolt, in your prepare method inject the members. That's the best I've come up with, as prepare happens server side whereas topology config and static initializers happen at deploy time client side. On Dec 25, 2013 7:51 AM, Michal Singer mic...@leadspace.com wrote: Hi, I am trying to understand how to use beans in spring as bolts/spouts. If I have the definition in spring which is initialized once the bolt or spout is initialized. But when creating a topology I need to do: new Bolt()…. And cannot get it from spring. So what is the right way to do this? Thanks, Michal -- Eugene Dvorkin Software Engineer WebMD email: edvor...@gmail.com phone: 646-250-9649 eugenedvorkin.com Connect with me on: LinkedIn http://www.linkedin.com/in/eugenedvorkin Twitterhttp://twitter.com/edvorkin
RE: Spring bolts
Hi, I checked this out. this project does give a way to run spring beans inside a bolt or a spout. But if I want for example to define a spout which is a JMS listener of activemq and is configured in spring it won’t work. I will need to define a regular spout which reads from a spring bean which is a jms listener. This will be problematic as to a point of failure since I will not read directly from jms queue (Activemq) but instead from an internal queue which will be somhow loaded from the AMQ. Thanks *From:* Eugene [mailto:edvor...@gmail.com] *Sent:* Wednesday, December 25, 2013 10:11 PM *To:* user@storm.incubator.apache.org *Subject:* Re: Spring bolts Did you look into this project: https://github.com/internet-research-network/breeze/ I did not try it yet, but it may give you what are you looking for. Merry Christmas! On Wed, Dec 25, 2013 at 10:54 AM, Michal Singer mic...@leadspace.com wrote: Thanks, Merry Christmas to you! *From:* Michael Rose [mailto:mich...@fullcontact.com] *Sent:* Wednesday, December 25, 2013 5:48 PM *To:* user@storm.incubator.apache.org *Subject:* RE: Spring bolts Yes, you'll need a Spring context in prepare. Given you have multiple bolts per JVM, its worth ensuring only one creates it in prepare then shares that context. We do this with Guice injectors and double checked locks. Each bolt uses the singleton injector to inject its members. I imagine Spring has a similar concept once you have a context. Life cycle of bolts is quite strange in Storm given they're made before deployment and serialized. There's quite a few gotchas. Bolt constructors can't be trusted, thus prepare. There may be a spring storm example out there somewhere. Merry Christmas! On Dec 25, 2013 8:17 AM, Michal Singer mic...@leadspace.com wrote: I am not sure I understand. Spring beans are defined in the spring configuration files. How can I inject them in the members. What I thought to do is that the bolts will not be spring beans and in the prepare method I will initialize the spring context. This way, the bolts will call other spring beans which are not bolts and initialized in spring. But of course this is a very limited solution. *From:* Michael Rose [mailto:mich...@fullcontact.com] *Sent:* Wednesday, December 25, 2013 5:06 PM *To:* user@storm.incubator.apache.org *Subject:* Re: Spring bolts Make a base spring bolt, in your prepare method inject the members. That's the best I've come up with, as prepare happens server side whereas topology config and static initializers happen at deploy time client side. On Dec 25, 2013 7:51 AM, Michal Singer mic...@leadspace.com wrote: Hi, I am trying to understand how to use beans in spring as bolts/spouts. If I have the definition in spring which is initialized once the bolt or spout is initialized. But when creating a topology I need to do: new Bolt()…. And cannot get it from spring. So what is the right way to do this? Thanks, Michal -- Eugene Dvorkin Software Engineer WebMD email: edvor...@gmail.com phone: 646-250-9649 eugenedvorkin.com Connect with me on: LinkedIn http://www.linkedin.com/in/eugenedvorkin Twitterhttp://twitter.com/edvorkin
Spring bolts
Hi, I am trying to understand how to use beans in spring as bolts/spouts. If I have the definition in spring which is initialized once the bolt or spout is initialized. But when creating a topology I need to do: new Bolt()…. And cannot get it from spring. So what is the right way to do this? Thanks, Michal
Re: Spring bolts
Make a base spring bolt, in your prepare method inject the members. That's the best I've come up with, as prepare happens server side whereas topology config and static initializers happen at deploy time client side. On Dec 25, 2013 7:51 AM, Michal Singer mic...@leadspace.com wrote: Hi, I am trying to understand how to use beans in spring as bolts/spouts. If I have the definition in spring which is initialized once the bolt or spout is initialized. But when creating a topology I need to do: new Bolt()…. And cannot get it from spring. So what is the right way to do this? Thanks, Michal
RE: Spring bolts
I am not sure I understand. Spring beans are defined in the spring configuration files. How can I inject them in the members. What I thought to do is that the bolts will not be spring beans and in the prepare method I will initialize the spring context. This way, the bolts will call other spring beans which are not bolts and initialized in spring. But of course this is a very limited solution. *From:* Michael Rose [mailto:mich...@fullcontact.com] *Sent:* Wednesday, December 25, 2013 5:06 PM *To:* user@storm.incubator.apache.org *Subject:* Re: Spring bolts Make a base spring bolt, in your prepare method inject the members. That's the best I've come up with, as prepare happens server side whereas topology config and static initializers happen at deploy time client side. On Dec 25, 2013 7:51 AM, Michal Singer mic...@leadspace.com wrote: Hi, I am trying to understand how to use beans in spring as bolts/spouts. If I have the definition in spring which is initialized once the bolt or spout is initialized. But when creating a topology I need to do: new Bolt()…. And cannot get it from spring. So what is the right way to do this? Thanks, Michal
RE: Spring bolts
Yes, you'll need a Spring context in prepare. Given you have multiple bolts per JVM, its worth ensuring only one creates it in prepare then shares that context. We do this with Guice injectors and double checked locks. Each bolt uses the singleton injector to inject its members. I imagine Spring has a similar concept once you have a context. Life cycle of bolts is quite strange in Storm given they're made before deployment and serialized. There's quite a few gotchas. Bolt constructors can't be trusted, thus prepare. There may be a spring storm example out there somewhere. Merry Christmas! On Dec 25, 2013 8:17 AM, Michal Singer mic...@leadspace.com wrote: I am not sure I understand. Spring beans are defined in the spring configuration files. How can I inject them in the members. What I thought to do is that the bolts will not be spring beans and in the prepare method I will initialize the spring context. This way, the bolts will call other spring beans which are not bolts and initialized in spring. But of course this is a very limited solution. *From:* Michael Rose [mailto:mich...@fullcontact.com] *Sent:* Wednesday, December 25, 2013 5:06 PM *To:* user@storm.incubator.apache.org *Subject:* Re: Spring bolts Make a base spring bolt, in your prepare method inject the members. That's the best I've come up with, as prepare happens server side whereas topology config and static initializers happen at deploy time client side. On Dec 25, 2013 7:51 AM, Michal Singer mic...@leadspace.com wrote: Hi, I am trying to understand how to use beans in spring as bolts/spouts. If I have the definition in spring which is initialized once the bolt or spout is initialized. But when creating a topology I need to do: new Bolt()…. And cannot get it from spring. So what is the right way to do this? Thanks, Michal
RE: Spring bolts
Thanks, Merry Christmas to you! *From:* Michael Rose [mailto:mich...@fullcontact.com] *Sent:* Wednesday, December 25, 2013 5:48 PM *To:* user@storm.incubator.apache.org *Subject:* RE: Spring bolts Yes, you'll need a Spring context in prepare. Given you have multiple bolts per JVM, its worth ensuring only one creates it in prepare then shares that context. We do this with Guice injectors and double checked locks. Each bolt uses the singleton injector to inject its members. I imagine Spring has a similar concept once you have a context. Life cycle of bolts is quite strange in Storm given they're made before deployment and serialized. There's quite a few gotchas. Bolt constructors can't be trusted, thus prepare. There may be a spring storm example out there somewhere. Merry Christmas! On Dec 25, 2013 8:17 AM, Michal Singer mic...@leadspace.com wrote: I am not sure I understand. Spring beans are defined in the spring configuration files. How can I inject them in the members. What I thought to do is that the bolts will not be spring beans and in the prepare method I will initialize the spring context. This way, the bolts will call other spring beans which are not bolts and initialized in spring. But of course this is a very limited solution. *From:* Michael Rose [mailto:mich...@fullcontact.com] *Sent:* Wednesday, December 25, 2013 5:06 PM *To:* user@storm.incubator.apache.org *Subject:* Re: Spring bolts Make a base spring bolt, in your prepare method inject the members. That's the best I've come up with, as prepare happens server side whereas topology config and static initializers happen at deploy time client side. On Dec 25, 2013 7:51 AM, Michal Singer mic...@leadspace.com wrote: Hi, I am trying to understand how to use beans in spring as bolts/spouts. If I have the definition in spring which is initialized once the bolt or spout is initialized. But when creating a topology I need to do: new Bolt()…. And cannot get it from spring. So what is the right way to do this? Thanks, Michal
