Hi Jungtaek, For adding jars and maven at submission, you have used the word Submitter. Is Submitter the person running storm jar command or is Submitter the java code that actually submits it to Nimbus? Also, I did not quite understand the --jars option. If you could please elaborate a little on that, that will be great
Thanks Priyank On 8/2/16, 7:05 AM, "Jungtaek Lim" <[email protected]> wrote: >Ah, Satish you got the point. I meant copied version of files in >supervisor, but itself can be isolated. >I didn't think about removing blobs, and it seems not easy to do. > >Jungtaek Lim (HeartSaVIoR) > > >2016년 8월 2일 (화) 오후 7:35, Satish Duggana <[email protected]>님이 작성: > >> Hi Jungtaek, >> With the current proposal, are we removing blob store files referred by a >> topology when it is killed? >> >> Thanks, >> Satish. >> >> On Tue, Aug 2, 2016 at 3:50 PM, Jungtaek Lim <[email protected]> wrote: >> >> > Hi Satish, >> > >> > Thanks for reviewing and share your idea. >> > >> > Yes this is shared dependencies vs isolated dependencies. >> > If we name file of dependency to contain group name, artifact name, and >> > version, that can be shared. >> > One downside of this approach is storage space since we don't know when >> > it's safe to delete without additional care, but I'm curious that disk >> > fills up due to dependency blob jar files in normal situation. >> > So I think we're OK to do this but I would like to see others opinions. >> > >> > Btw, I'm designing details based on proposal. Will update to this thread >> if >> > there're not covered things with initial design. >> > >> > Thanks, >> > Jungtaek Lim (HeartSaVioR) >> > >> > 2016년 8월 2일 (화) 오후 6:58, Satish Duggana <[email protected]>님이 작성: >> > >> > > Hi Jungtaek, >> > > Proposal looks good to me. Good that we are not going with other >> > > alternative using mutable classloader etc. >> > > >> > > Good to have the mentioned config in proposal to add those jars before >> or >> > > after storm core/libs. There is a property Config. >> > > TOPOLOGY_CLASSPATH_BEGINNING which is to have that value as initial >> > > classpath and that should continue to be working as expected even with >> > the >> > > new configuration. >> > > >> > > One enhancement which we may want to add to the existing proposal. >> > > When --packages are used, storm submitter can upload those dependencies >> > in >> > > blob store with a defined naming convention so that same set of >> packages >> > > are not uploaded again and they can be used again for other topologies >> if >> > > they use same package. >> > > >> > > Thanks, >> > > Satish. >> > > >> > > >> > > On Tue, Aug 2, 2016 at 7:25 AM, Jungtaek Lim <[email protected]> >> wrote: >> > > >> > > > Hi dev, >> > > > >> > > > This is proposal review thread for submitting topology with adding >> jars >> > > and >> > > > maven artifacts. This is also following up discussion thread for >> > > > [DISCUSSION] >> > > > Policy of resolving dependencies for non storm-core modules.[1] >> > > > >> > > > I've written design doc which also describes motivation on this. >> > > > >> > > > >> > > >> > >> https://cwiki.apache.org/confluence/display/STORM/A.+Design+doc%3A+adding+jars+and+maven+artifacts+at+submission >> > > > >> > > > Please review this and comment to "this thread" instead of wiki page >> so >> > > > that all devs can be notified for the update. >> > > > >> > > > Thanks, >> > > > Jungtaek Lim (HeartSaVioR) >> > > > >> > > > [1] >> > > > >> > > > >> > > >> > >> http://mail-archives.apache.org/mod_mbox/storm-dev/201607.mbox/%3CCAF5108jByyJLTKrV_P4fS=dj8rsr_o5oubzqbviscggsc1c...@mail.gmail.com%3E >> > > > >> > > >> > >>
