Hi Jungtaek, With the current proposal, are we removing blob store files referred by a topology when it is killed?
Thanks, Satish. On Tue, Aug 2, 2016 at 3:50 PM, Jungtaek Lim <[email protected]> wrote: > Hi Satish, > > Thanks for reviewing and share your idea. > > Yes this is shared dependencies vs isolated dependencies. > If we name file of dependency to contain group name, artifact name, and > version, that can be shared. > One downside of this approach is storage space since we don't know when > it's safe to delete without additional care, but I'm curious that disk > fills up due to dependency blob jar files in normal situation. > So I think we're OK to do this but I would like to see others opinions. > > Btw, I'm designing details based on proposal. Will update to this thread if > there're not covered things with initial design. > > Thanks, > Jungtaek Lim (HeartSaVioR) > > 2016년 8월 2일 (화) 오후 6:58, Satish Duggana <[email protected]>님이 작성: > > > Hi Jungtaek, > > Proposal looks good to me. Good that we are not going with other > > alternative using mutable classloader etc. > > > > Good to have the mentioned config in proposal to add those jars before or > > after storm core/libs. There is a property Config. > > TOPOLOGY_CLASSPATH_BEGINNING which is to have that value as initial > > classpath and that should continue to be working as expected even with > the > > new configuration. > > > > One enhancement which we may want to add to the existing proposal. > > When --packages are used, storm submitter can upload those dependencies > in > > blob store with a defined naming convention so that same set of packages > > are not uploaded again and they can be used again for other topologies if > > they use same package. > > > > Thanks, > > Satish. > > > > > > On Tue, Aug 2, 2016 at 7:25 AM, Jungtaek Lim <[email protected]> wrote: > > > > > Hi dev, > > > > > > This is proposal review thread for submitting topology with adding jars > > and > > > maven artifacts. This is also following up discussion thread for > > > [DISCUSSION] > > > Policy of resolving dependencies for non storm-core modules.[1] > > > > > > I've written design doc which also describes motivation on this. > > > > > > > > > https://cwiki.apache.org/confluence/display/STORM/A.+Design+doc%3A+adding+jars+and+maven+artifacts+at+submission > > > > > > Please review this and comment to "this thread" instead of wiki page so > > > that all devs can be notified for the update. > > > > > > Thanks, > > > Jungtaek Lim (HeartSaVioR) > > > > > > [1] > > > > > > > > > http://mail-archives.apache.org/mod_mbox/storm-dev/201607.mbox/%3CCAF5108jByyJLTKrV_P4fS=dj8rsr_o5oubzqbviscggsc1c...@mail.gmail.com%3E > > > > > >
