+1 to moving non-conncectors to top level. I think we should keep stom-kafka-monitor under external or connectors(after renaming).
Jungtaek, just to clarify on what you said regarding storm core referencing storm-kafka-monitor. Like you said its just calling the script from ui jvm. There is no dependency in terms of class files needed to run the script from ui. The script itself adds a –cp argument and all it needs is storm-kafka-monitor jar in classpath. As far as packaging the script is concerned we can do what Satish suggested. i.e. move it to storm-kafka-monitor in source and while packaging put it under bin. Reiterating to make sure I am not mis-understanding anything. On 3/24/17, 9:14 AM, "Harsha Chintalapani" <[email protected]> wrote: +1 on moving non-connectors to top-level like sql and storm-perf. Regarding storm-kafka-monitor we can move this into "util" folder or keep in the external. -Harsha On Fri, Mar 24, 2017 at 2:23 AM Satish Duggana <[email protected]> wrote: > storm-kafka-monitor is not a connector by itself but it is related to kafka > connectors. So, any utility related to that connector should be part of > that connector module(can be a submodule) instead of a top level module. > core/ui uses this utility referring directly in a hacky way, which we may > want to fix later. storm-kafka-monitor script exists in bin directory which > can be moved to storm-kafka-monitor module and the same script can be > packaged as part of storm/bin directory while packaging the distribution. > > Thanks, > ~Satish. > > On Fri, Mar 24, 2017 at 1:07 PM, Jungtaek Lim <[email protected]> wrote: > > > storm-kafka-monitor is referred by storm-core, though it's referenced via > > executing command. Yes it's a bit odd to place it as top directory, but > > it's not a connector for that reason too. Neither is ideal for me, so > > ironically, either is fine. > > > > - Jungtaek Lim (HeartSaVioR) > > > > 2017년 3월 24일 (금) 오후 4:19, Satish Duggana <[email protected]>님이 > 작성: > > > > > +1 except for storm-kafka-monitor module as this utility is more about > > > querying topic/partition offsets of kafka spouts in a topology. Do not > we > > > want to push this module into connectors/kafka as a submodule along > with > > > other submodules including old/new kafka spout modules? > > > > > > Thanks, > > > Satish. > > > > > > > > > On Fri, Mar 24, 2017 at 12:10 PM, Arun Iyer <[email protected]> > > wrote: > > > > > > > +1 > > > > > > > > Makes sense to move the non-connectors to top level and keep only the > > > > connectors under “connectors” folder. > > > > > > > > > > > > On 3/24/17, 12:00 PM, "Jungtaek Lim" <[email protected]> wrote: > > > > > > > > >(Sent this yesterday but can't find this from storm-dev mbox... > > sending > > > it > > > > >again) > > > > > > > > > >Hi dev, > > > > > > > > > >I'd like to start discussion regarding moving non-connectors modules > > out > > > > of > > > > >external, maybe top directory. > > > > > > > > > >"external" directory has non-connectors (SQL, Flux, > > storm-kafka-monitor, > > > > >storm-submit-tools), and except Flux, others should be placed to the > > > > binary > > > > >dist. since Storm itself (not from user topology) needs to refer > them. > > > > > > > > > >They're actually tied to the core of Storm, so I feel that it would > be > > > > >better to treat them (including Flux) as non-external, maybe same > > level > > > as > > > > >storm-core. > > > > >(I'm not sure what "external" actually means for Storm project btw.) > > > > > > > > > >In addition, after doing that I'd like to change the directory name > > > > >"external" to "connector" or so, so that the name could be > > > self-describing > > > > >and we can only place connectors to that directory. > > > > >(I know it would be painful for already opened pull requests, so no > > > strong > > > > >opinion regarding this.) > > > > > > > > > >Looking forward to your opinion! > > > > > > > > > >Thanks, > > > > >Jungtaek Lim (HeartSaVioR) > > > > > > > > > > > > > >
