Cassandra, Jan, et al., Looks like I cannot create a SIP. There is a Jira already, but perhaps I will create a new Jira that answers these two very questions:
My wish for the SIP is to be very clear on exactly how the UI is proposed shipped and whether any manual steps are needed to enable/install. Also try to clarify whether you propose a big-bang replace or whether the old and new UI will need to co-exists. @ Yeikel - The UI will not introduce changes to the APIs. I think we ought to consider as a community modifying some APIs is a learning I have garnered from this work. The UI will be maintained, for sure. Thanks, Marcus On Tue, Apr 14, 2020 at 7:35 AM yeikel valdes <em...@yeikel.com> wrote: > And whether they can coexist at all. From the previous emails it seemed > to me this change will introduce changes to the existing APIS and I that > means that compatibility will be broken. I am also not sure if it we will > maintain the existing UI after that. > > > ---- On Tue, 14 Apr 2020 09:51:23 -0400 * jan....@cominvent.com > <jan....@cominvent.com> * wrote ---- > > Yes, please go through SIP before inviting us to review the actual PR. > And once you re-work the SIP page based on feedback and converge towards > consensus, I’d recommend starting a formal VOTE thread for the SIP, as > outlined in the SIP page. That way people can have a chance to speak up on > the overall design so that does not become a topic once again in the PR. > > My wish for the SIP is to be very clear on exactly how the UI is proposed > shipped and whether any manual steps are needed to enable/install. Also try > to clarify whether you propose a big-bang replace or whether the old and > new UI will need to co-exists. > > Jan > > 14. apr. 2020 kl. 15:29 skrev Cassandra Targett <casstarg...@gmail.com>: > > Marcus, > > A couple thoughts… > > First, sorry you’re having problems with Confluence. I suspect the issue > is permissions. There are only two groups allowed to add pages to the SOLR > space, “lucene” and “lucene-pmc”. I believe these correspond to ASF LDAP > groups, which would mean they include committers and PMC members only. We > can grant you individual permission to add/edit pages, however; we’ve done > this for a handful of others. I could do this for you, just ping me > off-thread so I can confirm your username. > > I don’t know what happened to the pages you tried to create. You can try > to see if they are hidden somehow, maybe in your profile’s “Recently worked > on” section ( > https://cwiki.apache.org/confluence/dashboard.action#recently-worked). > When you’re on that page, the menu on the left also shows a “Saved for > later” page that maybe has them. > > At any rate, I wouldn’t suggest putting all the design > decisions/discussion into a PR. That’s what we traditionally used Jira for. > We decided to use SIPs to make navigating design discussions in Jira easier > - decisions would get lost in comments - and to forestall someone doing > possibly wasted work until they have some degree of community agreement for > what they hope to do. If you put it in a PR, by its very nature those > decisions would already be made and that could lead to significant amounts > of rework. > > The SIP process is to write up the SIP and then file a Jira issue, and you > can’t file a PR without a Jira, so you’ll need a Jira issue for this work > anyway. If you really can’t write a SIP, then the fallback is Jira, not a > PR. > > Re: the API issues - those should also go into Jira. I can’t see your list > so far since it requires permission, but I’ll just say I would not > recommend dropping your whole doc into a single Jira. They’ll need to be > broken out into separate ones (and it’s pretty likely issues already exist > for at least some of the things). I know that sounds like a PITA, but we > don’t track issues in personal Google docs, we use Jira. And if some of the > items are possibly controversial, then individual items for each one will > allow us to work through the controversies and not stall progress on things > that are not problematic (if anyone is so inclined to work on those). > > Hopefully you don’t mind the unsolicited advice on this, just trying to > help you understand some of our ways of doing things. > > Cassandra > On Apr 14, 2020, 3:18 AM -0500, Marcus Eagan <marcusea...@gmail.com>, > wrote: > > Mike and Gus, > > I tried to share my SIP after writing it, and then it disappeared. I also > tried to write some and save it, but then it disappeared. I also tried to > write it outside the wiki and paste it again. > > I'm not sure what's going on, but I will try again tomorrow I suppose. If > I fail again, my SIP will probably come in the form of a Pull Request, with > videos, screenshots, a lit of todos, and helpful documentation so that > people can get started easily. > > From this project, I have also uncovered what I would consider some > serious issues with the Solr API and some other issues that I dig through > Jira for before I bother the community. I will do my best to document them > all and open tickets. Some times I rage quit, walk away, come back and > forget. I will try to fix some of them, but some of them that I have looked > into look like worm cans. It might be good if a few people here are > prepared to explore fixing the API in a few areas, but I'm not sure about > the best way to go about doing that with angering lots of people. *Seeking > advice on how to approach the API challenges.* I don't just want to start > complaining about things, nor do I want to take on everything. A small > group from a variety of different organizations to discuss some of these > challenges might be most helpful. On the flip side, I suspect things will > improve in other areas as a result. > > You can view progress here: > https://drive.google.com/file/d/1NgO34DRp1llMp3EwJQcKBn2LM4r5PD39/view?usp=sharing > > I've mostly managed to get all the requests sorted, and rendering of the > data (mostly). i didn't get much time today as I was very busy at work. And > tomorrow I get my quarantine checkup and will probably be sleepy early. But > by the end of next weekend, I should have finished everything but the query > editor and all the other associated collections screens which I will bucket > under collections. That current dropdown menu is less intuitive than > optimal but I understand. > > Soliciting feedback now because once I open the PR I am hoping we have > agreed on as many things as possible. If no one suggest query view designs, > I will mock some up and share them. Again, I repeat, not a designer. But I > did take couple classes, and have built products from start to finish > mostly on an island so I will be able to manage. Looking for feedback that > the community might be getting from its clients. Obviously, I cannot > accommodate all the requests, but if something is recurring, I want to > support the users to support the adoption of this new UI. > > As if I needed to say it, and many of you probably suspected this was > coming: Erick this is a lot of work. Holy shit. And not just because I'm a > product manager. It would be challenging for anyone. This project, like all > project, has some skeletons. From my perspective, that's gfreat because > there are lots of things to improve. Also, the project is still pretty > amazing. > > Thanks again for your help everyone, > > Marcus > > > On Mon, Apr 13, 2020 at 9:55 AM Marcus Eagan <marcusea...@gmail.com> > wrote: > > Gus, > > SIP sounds good. I will share. > > Marcus > > > > On Mon, Apr 13, 2020 at 09:13 Gus Heck <gus.h...@gmail.com> wrote: > > Maybe start collecting Design and Design choices in a SIP? This discussion > has been good and there seems to be consensus that we want a new UI, we > want it to be a package and we want the package to be available by default > and well tested. "Package" seems to imply that it can be added or removed > or replaced or an alternative UI installed along side of it. If we got all > of those things done this would be amazingly awesome :) > > Another thing that would be valuable is a good doc that explains "how to > edit and maintain the UI", written for an audience that is experienced in > SW dev but not UI development (probably including some basics around > framework chosen). This could be in a README or in the "dev docs" that has > been mentioned elsewhere. > > The SIP would be a great place to elaborate on technology choices & supply > a link to things like the video :) > > On Mon, Apr 13, 2020 at 10:35 AM Mike Drob <md...@apache.org> wrote: > > Hi Marcus, > > The mailing list strips attachments for some folks, can you upload the > video somewhere else and link to it for us poor unfortunate souls? Thanks > for your work! Excited to see the progress as it happens. > > Mike > > On Mon, Apr 13, 2020 at 5:30 AM Marcus Eagan <marcusea...@gmail.com> > wrote: > > In general, I asked for some degree of trouble when I volunteered for this > work. Don't beat me too hard. My primary goal is to achieve three things: > > 1) Improve security when using Solr Admin UI by removing EOL, unsupported > code. > 2) Make it easier or more welcoming for new developers to try the project > and even become contributors in all areas of the project because the UI > looks and functions as slightly more contemporary. > 3) Give back more substantially to a community from which I have received > so much with a testable and perrty UI. > > I've added another (4) which is contribute to help make the package > manager a first-class citizen in the minds of many Solr users around the > world via the UI package. I will need some help from someone in this list > on deploying this UI with a jar in Gradle if we want to offer an > alternative option to install the UI in Solr 9. I've attached where I'm at. > It's a nights and weekends project, but I will always be available for > bug fixes or discussions, unless i'm in a meeting or reading a book 😇 > > I won't solicit a ton of feedback prior to the the first PR, which I will > leave open for a few weeks or even a couple months while I put some > lipstick on it and improve performance of the application > > *<<<<<<< Over the weekend read lots of documentation, wrote a bunch of > code when it wasn't holidays, built the services, and stumbled through the > logic of rendering all these data points so you can watch the attached > video if you want to check it out. >>>>>>>>* > > . There's definitely some areas where I didn't do the TypeScript thing > because I'm still trying to grok it a bit.The two areas of the that I am > looking to somewhat overhaul in potentially controversial ways are the > *queries > page* and actual flow of the collections experience, which at the moment > are sort of linked, yet disconnected at the same time. The Collections page > tab today is really an Alias page. It won't have its own tab in the new > application is the plan, unless someone can give me a good reason. Almost > everything else will stay the same. > > For that reason, I'd like to solicit feedback if anyone has any examples > or ideas they'd like to share, I would greatly appreciate it. I'm somewhat > far along with the Admin UI as of now. Short weekend because of holiday > activities and general quarantine craziness, but I'm maybe 25-35% of the > way to completion, depending on how much care I devote to the query > screen. > > Even though there are some big problems with Angular — some major ones — I > think this was the right way to go for many reasons. I'm about a quarter as > fast in Angular as I am in React, but this is the right decision for the > long haul. I can elaborate if anyone really cares. Most importantly, this > app will be a lot easier to maintain. > > But now I have a handle on TypeScript. Many optimizations and improvements > coming to Solr as a result. TypeScript can be pretty opinionated about APIs > for someone thinking that they are back in the wild wild west. > > Sending big air hugs, > Marcus > > On Fri, Apr 10, 2020 at 16:20 Noble Paul <noble.p...@gmail.com> wrote: > > +1 to David Smiley > > On Sat, Apr 11, 2020, 8:12 AM Marcus Eagan <marcusea...@gmail.com> wrote: > > I don't see admin UI as non-core. I think that an application UI for > end-users of an application consumes Solr non-core. I have to resign from > arguing, though. > > I don't consider myself a UI expert. I can do the work. > > > On Fri, Apr 10, 2020 at 11:42 AM Ishan Chattopadhyaya < > ichattopadhy...@gmail.com> wrote: > > David, you capture my thoughts well. > > Having a UI as a package gives users more choice and gives our users more > flexibility. > 1. Users would be able to use a latter version of the UI with an older > version of Solr, or vice versa. > 2. Users should be able to install multiple types of UI, from different > publishers, at once. > 3. Contributors should be able to contribute to the UI more easily, since > collaboration can be less bureaucratic. Experts like Marcus won't need to > depend on preoccupied committers like us. > 4. A UI (not the default one) can use libraries that aren't even Apache > 2.0 compatible. > 5. We can setup and use UI test frameworks for test automation (selenium > etc), that would be challenging to setup and maintain with ASF Jenkins. > > List goes on.. > > Whether the package is a first party or third party can be a separate > discussion. There should be an extremely easy and well defined way (support > in the script itself) to start Solr with the packaged UI enabled. > > In any case, I don't think it is conducive to let UI code be part of the > Solr's core codebase, where it currently is. The reason is, we can't fix > bugs if we break something. We don't have automated testing either to know > whether or not we broke anything. > > Every healthy project has a rich plugin ecosystem, and such non core > improvements should be delivered via packages. > > On Fri, 10 Apr, 2020, 8:33 pm Marcus Eagan, <marcusea...@gmail.com> wrote: > > I agree with you. 😃 > > On Fri, Apr 10, 2020 at 06:22 David Smiley <david.w.smi...@gmail.com> > wrote: > > I disagree that "package management frameworks are for loading > non-essential features or features not enabled by default". > > I don't think the proposal of the UI being a "package" (in the new package > system) implies that the UI (or _any_ package) is not a highly valuable > package that is so highly valuable that we want it installed by default. > Noble and I were brainstorming on some ideas where even much of Solr's > internal instances of plugin interfaces (e.g. query parsers, etc.) might > even be a new "core" package or some-such. The value in putting much of > Solr in a package, 1st party, is separation of concerns. and better > classpath management. > > I think it's "essential" that a UI ship with Solr by default -- meaning, > without the user having to take any additional steps whatsoever. As Jan > said it's been this way a long time. > > ~ David Smiley > Apache Lucene/Solr Search Developer > http://www.linkedin.com/in/davidwsmiley > > > On Fri, Apr 10, 2020 at 1:35 AM Marcus Eagan <marcusea...@gmail.com> > wrote: > > I think package management frameworks are for loading non-essential > features or features not enabled by default. On essentialness, experts > should not decide what is essential based on how they use a system. They > should consider the community of users. Regarding UI, it is and should be > enabled by default. Only a few use cases prefer it to be disabled and some > of those are because of its current state. They would like to use it in an > updated form. > > What is the technical rationale that outweighs the needs and behaviors of > our users to strip the user interface out of Solr? > > Thank you Noble and everyone else, > Marcus > > > On Thu, Apr 9, 2020 at 19:06 Noble Paul <noble.p...@gmail.com> wrote: > > My 2 cents (again) > > if packages are disabled by default , how will UI work? > > We can make an exception for this one and enable only this by default > > Do we test the UI and certify it? > > The UI package can be shipped along with Solr distro ,like the million > other jars that we ship with Solr today and every version of Solr can > be certified for a certain version of the UI package. We should have > sanity tests to ensure that the given version of UI works well with > Solr. "My commit has broken the UI and it's not my problem" should not > be a valid excuse. The UI sanity tests should pass as a part of the > tests. > > Is the UI important? > > Yes, the admin UI is the face of Solr for may users. People always > assumed it existed & they depend on it. > > The current admin UI has fallen behind. If the new UI effort delivers > on the promise, this is a great opportunity to get rid of that old > baggage & make Solr codebase even slimmer > > On Fri, Apr 10, 2020 at 6:06 AM Jan Høydahl <jan....@cominvent.com> wrote: > > > > Solr has always had an admin UI and if anyone wants to propose it should > not, please start another thread or vote about that, and do not divert this > thread which is about how to improve and future proof the Admin UI. > > > > I believe the Admin UI should be strengthened and enhanced, not removed. > It can perfectly well be an official and even default on part of every > release, perfectly in sync. Whether it is in core as today or a package or > a stand-alone process or a new webapp, are then really what we discuss here. > > > > Perhaps after people have voiced their opinions in this thread, a SIP > can be crafted with a concrete plan. We can then have a vote on the SIP. > > > > Jan Høydahl > > > > > 9. apr. 2020 kl. 20:06 skrev Cassandra Targett <casstarg...@gmail.com > >: > > > > > > > > > Thanks for your message, Gus. You touched on things I was thinking > this morning as I caught up to the thread, and had started to draft a > message about. > > > > > > I feel like there is an assumption underlying some of our discussion > about packages that says a feature or whatever has to either part of our > core codebase or 100% maintained by someone “outside” the community (by > which I mean someone who is not a committer and/or operates completely > independently from the rest of the project activities). > > > > > > But it’s important to remember there’s nothing inherent in the package > concept that says a package can't be wholly > maintained/distributed/supported by the Lucene PMC and our community of > committers and contributors. It’s not uncommon for software to have a base > package of the core software and also plugins that most users would > consider “essential”, maintained by the same people making the core, but > which are added after the base install. There would be details to work out > in terms of how we manage that, but none of those are technically > impossible. I don’t think we only have a binary choice (3rd party package > or part of Solr core). > > > > > > In fact, I’d go so far as to say I suspect the only way packages are > going to see any real traction is if we take the lead in creating and > maintaining a few to show the way forward for everyone else who may be > interested in doing the same. The package concept introduces an idea of a > Solr ecosystem that has not really existed to date and like all new > ecosystems (communities), it needs some degree of nurturing to grow or it > will not take off. > > > > > > To bring it back around to the UI, though, I agree we need to decide: > is a UI important? I would argue that it is. I regularly talk to users > whose Solr knowledge and experience are quite advanced yet who still rely > entirely on the UI to carry out basic tasks. It’s just easier - less to > remember, don’t need to look up a command, etc. Persistent problems with > performance of the UI in large clusters and gaping holes in functionality > are deeply frustrating to those users. > > > > > > Because it is important to our users, even if any new UI ends up > living outside our community, it will need our explicit approval in some > form or we’re going to hear complaints until the end of time that Solr > doesn’t have a UI (or worse, we “took away” the UI). Without our ongoing > endorsement and blessing it will just be another toy maintained by > “somebody” (no offense, Marcus) until he is forced to abandon it due to > lack of user interest and/or lack of personal time to do all the heavy > lifting by himself. > > > > > > Cassandra > > >> On Apr 9, 2020, 11:32 AM -0500, Erick Erickson < > erickerick...@gmail.com>, wrote: > > >> Gus: > > >> > > >> Very thoughtful post. I think you raise an _extremely_ important > point about “how critical is the UI?” And by extension other packages. If > they’re critical to Solr, the question of how to keep them in sync becomes > paramount. > > >> > > >> I agree that the admin UI is important, if we have a mechanism to > insure it’s kept in sync with the release that would be near the to of my > list. > > >> > > >> Best, > > >> Erick > > >> > > >>> On Apr 9, 2020, at 12:06 PM, Gus Heck <gus.h...@gmail.com> wrote: > > >>> > > >>> In my view this brings us up against a bit of an existential > question. How do we ensure quality of packages that are key to Solr? I'm > sure that there are folks who don't find the UI very useful, but it's > important to others. The rationale that "elastic keeps their's separate" > has to be tempered by the actual real differences between Elastic and Solr. > Elastic has a corporate sponsor, a coordinated road map, and explicitly > ensures that all the bits that are maintained separately work together (so > that they don't have excessive support costs or bad first experiences that > impact their bottom line etc.). > > >>> > > >>> Solr is in a different place however, and we need to carefully > examine the question of whether something that works for Elastic works for > us. Lucidworks and several other large companies do spend a lot of money on > developers that contribute to Solr, but there is no organization around > multiple components that MUST work together. Another example of this is > Solr and Lucene, and our defense against a lack of coordination of > components in that case has been to unify them into a single release > package. > > >>> > > >>> So IMHO we need to answer 2 questions: > > >>> 1.) Do we consider the UI important. If I'm alone or in a minority > in feeling it's important, then so be it and it doesn't really matter what > we do. (maybe a vote?) > > >>> 2.) If we make it "a package" how do we ensure that important > packages such as the UI are never broken by a new release. > > >>> > > >>> IMHO I don't thing we should tolerate a situation where things we > consider important are broken frequently. > > >>> > > >>> For my part obviously my answer to 1 is "yes" :). As for 2, one > thing that comes to mind is what the Ant project did (may still do?) with > (and my memory from 15 years ago is foggy here, so forgive me if something > I say is not quite perfect) the GUMP build server that ran ant builds for a > bunch of different projects that depended on ant to ensure early detections > of changes that would break existing projects. If we have a good UI test > suite and commit to that being part of the release build package that might > be a solution. I honestly don't actually care where it lives, but I do > think it hurts us if it becomes broken and unusable, or hard to install. > > >>> > > >>> My worry is that "Solr developers are not UI developers" is really > code-words for "I want to be able to break it and let others clean up the > mess, because I'm not a UI developer". I have this worry with respect to > all "packages", but I may be missing info from discussions about the > package system, which initiated during a very busy period for me so > background links that I should have read are welcome if I've got something > wrong here :) I went looking for a SIP but didn't see it... I have found a > google doc linked from SOLR-13661 > > >>> > > >>> Finally to a detail about one of the above suggestions the option to > automatically download and install the UI could be good, but that then > requires that packages be available from somewhere that never goes away > like maven central, or that Apache commits to hosting a repository server > indefinitely, but again that's surely been discussed WRT packages > already... Using Github in such a way is subject to being broken > arbitrarily when Github decides to restrict things for cost reasons (ask > Bower about that one WRT rate limiting...) or the "repository" has to be > something local and therefore must be included part of the distribution... > at which point it's still a thing we distribute and since we're > distributing it and we probably don't mean to distribute broken stuff we > still need UI developers... > > >>> > > >>> Also, I thought the package loading stuff was supposed to be > disabled by default for security, that seems to conflict with or at least > complicate the notion of easily installing as a package. > > >>> > > >>> So "package" is a good for modularizing code, or for 3rd party > (possibly paid) plugins (I have a client that might find that interesting > in the future) but we have to ensure that it doesn't lead to a lack of > maintenance for things that are critical. > > >>> > > >>> Incidentally though I've said I favor Angular CLI, (significantly > because I've got some start on learning it) it also occurs to me that > perhaps anything "modern" is a difficulty because those things all have a > learning curve, and maximizing accessibility and ease of modifications for > folks not steeped in UI development might be our priority (different from > the priorities a commercial site would have). The flip side argument is > that with a popular framework, it would be easier for UI focused folks to > contribute... but will they? and does that leave us perennially rewriting > the UI in whatever is popular? (maybe that's ok?) I think in all our > decisions here we need to be very careful to distinguish how our needs may > differ in unusual ways from the needs of commercial web development. > > >>> > > >>> -Gus > > >>> > > >>> On Thu, Apr 9, 2020 at 8:14 AM Erick Erickson < > erickerick...@gmail.com> wrote: > > >>> Marcus: > > >>> > > >>> re-reading the thread, it looks to me like the consensus from Noble > and Ishan and Jan is that as long as the new, nifty UI is a separate > package, go ahead and knock yourself out ;). The objection is to making it > part of the Solr code base… We’ll all be thrilled with if we can rip the > current admin UI out ;) > > >>> > > >>> That said, I suspect it’ll be one of the tighter packages. It’d be > super-cool if we could run the UI tests on Jenkins say once a day just to > keep it up to date. > > >>> > > >>> The admin UI has always been somewhat awkwardly bolted on the side > of Solr, it’d be great to have it have a more elegant architecture. > > >>> > > >>> The other exciting thing would be that clients could then use the > package code as something they can incorporate/fork/whatever. Practically > every client I’ve worked with at large installations has rolled their own > dashboard. If they could use a package as a starting point, it’d be welcome. > > >>> > > >>> Best, > > >>> Erick > > >>> > > >>>> On Apr 9, 2020, at 3:07 AM, Marcus Eagan <marcusea...@gmail.com> > wrote: > > >>>> > > >>>> Hey Noble, > > >>>> > > >>>> -1 is a definitive, so I want to clarify that you are saying you do > not wish to remove the EOL front end and replace it with another one in the > longer term? > > >>>> > > >>>> I hear you! As a product manager in my day job, my primary goal is > to find features to cut! I spend a lot of time thinking about non-essential > vs used heavily vs causes more problems than it's worth. I can tell you > from watching the many people in the field at Lucidworks, there are a lot > of people who know quite a bit about Solr, but rely on the Admin UI heavily > because they feel comfortable there. Those people in effect help us stay > employed despite never contributing or being capable of contributing to > Solr. So hear me out. I've got a proposal: > > >>>> > > >>>> To start, I can work on this app as an optional package for your > awesome new package manager. It will be the second one I've worked on in my > evenings and weekends btw. The first was a package validator that I hope to > eventually open source, but its complexity and lack of popularity because > it is security ;( will likely make it the second one I open source/finish. > I'm also collaborating with a couple members of the Lucidworks security > team on that one, but I have built the basics already for them to build > upon. > > >>>> > > >>>> Back to the new UI discussion and my update that I promised. > > >>>> > > >>>> My update was going to be that after evaluating the projects Jan > posted, the most recent project that Jan listed created a pretty good base > to build on. After lots of auditing of the packages and a bit of > refactoring because the UI world moves fast, I was able to get it to > transpile and run again (as I'm sure it previously did) and from (2290 > vulns): > > >>>> <image.png> > > >>>> no npm fix doesn't magically fix, I wish it did > > >>>> > > >>>> to (2 low sev, non-productions vulns): > > >>>> <image.png> > > >>>> > > >>>> These two issues will not affect production and really only the > unit tests. Besides, I plan to remove them before we get to a stage even > that matters. I've also started investigating the level of effort for me to > get it to feature parity with the current app. The preliminary answer is > not very much compared to other work I've done in shorter time — working > with a jerk for a boss (years ago, don't worry Hatcher). I'm building a > couple of the missing features as we speak. From the beginning, it will > have infinitely more test coverage and will be a lot more approachable to > contemporary UI developers. It will also make the Solr experience for new > developers simpler. The major design changes that I have been thinking > about would be to the cloud view and the query view. Both of those are > important, the first to more experienced users, and the second to less > advanced users though occasionally an advanced debugger or demo presenter > in my experience. > > >>>> > > >>>> In the end Noble, this is about making Solr more approachable to > new users not experts like you. The growth of Solr adoption only benefits > you, so I would ask you to revisit your -1 at some point in the near future > when you see the progress and breadth of improvement. We have had customers > complain about the Admin UI and the community has even complained about it. > I think this is the right thing to do. If you still consider effectively > upgrading the current Admin UI as feature creep, I can revisit the package > manager compromise or move the efforts elsewhere. I respect your position. > A search service is nothing without a strong and diverse set of skills and > capabilities behind it and making it accessible to everyone who needs it. > > >>>> > > >>>> For those who care, here's my 4 Node cluster of tech products > running locally. > > >>>> > > >>>> <image.png> > > >>>> > > >>>> The shoulders of the homie that put that scaffold together are > broad! Props to him. I will volunteer to put together extensive docs for JS > devs who want to contribute and make it better once we get it to a place > where it replaces and improves upon the current option. I'll even sponsor > some prizes for college kids or people recently out of work to get cranking > on this bad boy. > > >>>> > > >>>> Thank you Noble and everyone else, > > >>>> > > >>>> Marcus > > >>>> > > >>>> On Wed, Apr 8, 2020 at 11:20 PM Noble Paul <noble.p...@gmail.com> > wrote: > > >>>> Solr developers are not UI experts. We are a search service and > such a service should have nice clean APIs + documentation. Is a UI useful > ? yes. The last thing we want today is another complex component in Solr > codebase that nobody understands or cannot maintain. > > >>>> > > >>>> So, Solr UI can be hosted outside our codebase and we can have an > option to install UI from that remote repo > > >>>> > > >>>> something like "bin/solr install-gui" > > >>>> > > >>>> I'm -1 on anymore feature creep. > > >>>> > > >>>> --Noble > > >>>> > > >>>> > > >>>> On Thu, Apr 9, 2020 at 12:22 PM Marcus Eagan <marcusea...@gmail.com> > wrote: > > >>>> Thanks again Gus. > > >>>> > > >>>> Lots of people indeed misuse REST so we could go on and on about > whether requests are stateless or not in another thread. Let's spare the > group. > > >>>> > > >>>> I think most everyone on this channel would be in agreement with > you on separate app. I'll be opening a new ticket and a PR that will > document a few things to make it easy for UI devs who know little to no > Java how to get started. > > >>>> > > >>>> Ishan, there's some significant UI expertise in the team. Erickson > finds his way to open every cookie jar. Erik Hatcher wrote the first > version of Blacklight. I've seen Pugh do lots of work on Quepid's UI. Jan > and Kevin have done a lot of work, and so have many others. The list goes > on, and *likes to work on UI* is a different discussion. > > >>>> > > >>>> Beyond committers because I'm not a committer, I have UI expertise > that I can polish off and improve for the sake of my interest and > commitment to the community and I like to do it. I've also led UI teams. I > can help to steward the effort overall and keep things up to date up to the > point where I need to ask one of the committers to help me get changes > merged. I'll probably even hire a developer to work on it once we are to > that point. ;-) > > >>>> > > >>>> Expertise is not something that should block us but motivate us to > expand this community and/or our own skillsets long term. > > >>>> > > >>>> Thank you both and everyone else, > > >>>> > > >>>> Marcus > > >>>> > > >>>> On Wed, Apr 8, 2020, 10:21 AM Gus Heck <gus.h...@gmail.com> wrote: > > >>>> While running it in an external node does ensure separability, I > don't think it does a good job of addressing my other point of not needing > to manage a 3rd server. It's still a server if it's started by java, and > one still has to ensure it exists, and it will be extra hard to figure out > how to configure it if started by Solr. > > >>>> > > >>>> I'm strongly in favor of us having a UI from my perspective as a > consultant it makes discovery of things like their startup parameters and > directories and such very easy (just go to front page of the admin screen), > and it's so much easier to get a customer with security concerns and strict > controls on who can access what (think banks, military, etc) to share a web > session where they drive the UI than to get direct access to machines. > It'll be a lot slower and much lower service to be making people wait while > I craft curl statements to paste into the web session (and then fix the > inevitable typos, or detect when they missed the last char of what I > pasted, etc...). > > >>>> > > >>>> I definitely against Solr spawning some other server (node or > otherwise) on it's own and thereby requiring additional system > dependencies, or creating a second process that needs to be configured and > properly secured. To me that's even worse than requiring the UI to run > outside of Solr. We have a perfectly good web container already, and > furthermore there's a much greater likelihood that maintainers will be > facile with java/j2ee than anything else (IMHO). It's great if the > framework we choose uses little or no JSP/Servlet and is modernized with a > 100% javascript, templated etc. front end, but the back end should be > java/jetty because we've got lots of java folks. > > >>>> > > >>>> If the back end matters deeply then you're not really programming > to MVC/REST style... > > >>>> > > >>>> So there's another $0.02 :) and if you're not careful I'll give you > an entire nickle's worth of ways people misuse/misunderstand the term REST > :) > > >>>> > > >>>> -Gus > > >>>> > > >>>> On Tue, Apr 7, 2020 at 9:06 PM Marcus Eagan <marcusea...@gmail.com> > wrote: > > >>>> Gus, > > >>>> > > >>>> Your $.02 are worth a lot more than $.02 USD, so thank you. > > >>>> > > >>>> By separate app, I think I mean to endorse managed by a Node.js > process started by NPM. I don’t think that conflicts with what you have > proposed. The NPM command should be issued by Java || or Bash but I don’t > think it would add significant overhead. Also, seems like on CI and or > precommit hooks front end could be sizzled in parallel without adding much > overhead. > > >>>> > > >>>> As for the front end framework, the most important things to > consider in my view are simplicity and maintainability. We need to do a > thorough analysis on the ecosystem and issues like the size of a React > project vs Angular project vs Vue project, but React and Vue certainly have > the velocity and the hearts if the front end community more than Angular. > React is MIT license now and for the foreseeable > > >>>> future thanks to the power and reach of its developers. > > >>>> > > >>>> <gus.h...@gmail.com> wrote: > > >>>> +1 for Angular CLI / Typescript since I've fiddled with this in a > minor way recently, Also MIT license is super friendly. > > >>>> > > >>>> > > >>>> As a disenfranchised volunteer to the project, I also assume voters > on specific choices like frameworks will be helping build in some respect > at some point now or in the future. Is that a fair or misguided assumption? > > >>>> > > >>>> Marcus > > >>>> > > >>>> On Tue, Apr 7, 2020 at 17:15 Gus Heck <gus.h...@gmail.com> wrote: > > >>>> +1 for Angular CLI / Typescript since I've fiddled with this in a > minor way recently, Also MIT license is super friendly. > > >>>> > > >>>> Separate App - hmm... that's got some attraction, but also gives my > stomach some churning when I think about solr now requiring management of 3 > different servers (solr, something to serve UI and zookeeper). Adding more > infrastructure gives me pause with respect to all the smaller > installations. I've had several small self funded startup clients and a few > clients with existing initial installs that they are outgrowing in places > where procuring new machines and new software is a 6-12 mo endeavor and > both types seem to squirm when I make suggestions such as running zookeeper > separately, (let alone 3 of them). I think separate looks good for medium > to large folks or very large companies that **already have** a solr expert > on hand, but hurts the small clients and the departments in large orgs that > got started with insufficient advice/expertise, so maybe > > >>>> > > >>>> - The UI should be installed by default > > >>>> - it should be easy to remove it, or start with it disabled > > >>>> - it should be self contained and separately downloadable. > > >>>> > > >>>> My recent fiddling included figuring out how to make angular CLI > play nice in a J2ee war file structure seen here: > https://github.com/nsoft/ns-login > > >>>> > > >>>> By play nice I mean, > > >>>> - build creates a war file that "just works" when installed > > >>>> - Angluar CLI commands work > > >>>> - Angular serve command works (for auto-reloading ui changes, > running on port 4200; note the use of proxy to allow it to talk to an > already running web container) > > >>>> > > >>>> My $0.02, > > >>>> > > >>>> -Gus > > >>>> > > >>>> On Mon, Apr 6, 2020 at 11:03 AM Jörn Franke <jornfra...@gmail.com> > wrote: > > >>>> I think standalone would be very useful. > > >>>> I propose Angular with Typescript - it fits to a more data centric > approach with data types etc. > > >>>> Maybe even two types of UIs - Admin UI and a simple Search UI. > > >>>> > > >>>> > > >>>>> Am 06.04.2020 um 16:53 schrieb Jan Høydahl <jan....@cominvent.com > >: > > >>>>> > > >>>>> Thanks for kickstarting this and bringing some fresh blood and > enthusiasm :) > > >>>>> > > >>>>> Looks like others have had similar wish for a standalone Solr > Admin App, here’s a quick GitHub search for inspiration: > > >>>>> > > >>>>> https://github.com/savantly-net/solr-admin (Angular, nice > screenshots, 1y old) > > >>>>> https://github.com/kezhenxu94/yasa (vuejs, impressive > screenshots, 2y old) > > >>>>> https://github.com/thereactleague/galaxy (React, no screenshots, > 4y old) > > >>>>> > > >>>>> They all seem abandoned but perhaps a new official effort could > bring their developers in as contributors again? > > >>>>> > > >>>>>> the people who work on the Admin UI do not need to be expected to > know the Java workflow, necessarily. This reality widens the net for who > can contribute. > > >>>>> > > >>>>> Agree. Frontend devs have been a shortage in this project, and if > we can make it easier to attract UI committers who feel at home and > productive with the UI code, that would be a win. On the other hand, if we > expect that the UI will be maintained by regular Java committers, then > anything that makes it easier for them/us to contribute is also a win, like > perhaps strongly-typed. > > >>>>> > > >>>>> Again, thanks Marcus for reviving this topic. Let us all try not > to be overly ambitious here or shoot the initiative down with bikeshedding. > It is far more important to fuel the energy and momentum and get something > built than to remain stuck :) > > >>>>> > > >>>>> Jan > > >>>>> > > >>>>> > > >>>>>> 6. apr. 2020 kl. 13:47 skrev Marcus Eagan <m...@marcuseagan.com>: > > >>>>>> > > >>>>>> Coming back to these existential questions from my phone: > > >>>>>> > > >>>>>> > > >>>>>> > > >>>>>> > > >>>>>> > > >>>>>> > > >>>>>> > > >>>>>> > > >>>>>> > > >>>>>> > > >>>>>> Jan Høydahl > > >>>>>> Added 1 hour ago > > >>>>>> There are many opinions around admin UI. So I think the best > place to start would be a new mail-thread in dev@ to discuss the way > forward. Before we start a major re-work, we should probably ask ourselves > a few existential questions: > > >>>>>> • Should we turn Amin UI into a standalone app instead of > embedded in Solr? > > >>>>>> > > >>>>>> I think it should be a standalone app. There are many advantages > gained from a separation of such concerns. Some of the ones include, the > people who work on the Admin UI do not need to be expected to know the Java > workflow, necessarily. This reality widens the net for who can contribute. > > >>>>>> > > >>>>>> Testing becomes a lot easier because JS developers are accustomed > to building tests for static assets and self-contained node apps. They > generally know less about testing a bit of JS within a massive Java > project. The test could also run independently for changes that only affect > the front end. Adding test coverage without adding time to tests sounds > awesome. > > >>>>>> > > >>>>>> There are quite a few tickets over the years that have seemed to > suggest that people want more fine-grained control over the Solr admin UI > overall. Two recent tickets discussed topics like running a Solr Admin app > on only one node and disabling it al together for whatever reason. See: > https://issues.apache.org/jira/browse/SOLR-14014. > > >>>>>> > > >>>>>> • What UI framework? Guess anything is better than current EOL, > but will largely depend on who is willing to do the job! > > >>>>>> I’m happy to take this on (and willing to follow through on > completing in my nights and weekends), but I am mostly framework agnostic. > My stronge preference would be React, provided the license is kosher. There > was one blip of “practically unusable for most orgs” a couple years back, > but Facebook made it right really soon after. However, I’m flexible. > Angular (not JS) and Vue are also great. I would recommend we consider > Typescript also because of the size of project and number of strongly-typed > devs on this mailing list. My only reservation with TypeScript, though it > may not apply in this case, is that the supersets of JS have changed a lot > more than the frameworks. While CoffeeScript was an unnecessary layer of > abstraction from my limited perspective, TypeScript might make JS more > embraceable to a list of Java hackers. > > >>>>>> > > >>>>>> • Current UI has no test coverage, can we do better with the new > UI? > > >>>>>> > > >>>>>> It’s imperative.React, Angular, and Vue each make it easy to > include tests. > > >>>>>> > > >>>>>> > > >>>>>> > > >>>>>> > https://issues.apache.org/jira/browse/SOLR-12276?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17076204#comment-17076204 > > >>>>>> > > >>>>>> > > >>>>>> > > >>>>> > > >>>> > > >>>> > > >>>> -- > > >>>> http://www.needhamsoftware.com (work) > > >>>> http://www.the111shift.com (play) > > >>>> -- > > >>>> Marcus Eagan > > >>>> > > >>>> > > >>>> > > >>>> -- > > >>>> http://www.needhamsoftware.com (work) > > >>>> http://www.the111shift.com (play) > > >>>> > > >>>> > > >>>> -- > > >>>> ----------------------------------------------------- > > >>>> Noble Paul > > >>>> > > >>>> > > >>>> -- > > >>>> Marcus Eagan > > >>>> > > >>> > > >>> > > >>> --------------------------------------------------------------------- > > >>> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org > > >>> For additional commands, e-mail: dev-h...@lucene.apache.org > > >>> > > >>> > > >>> > > >>> -- > > >>> http://www.needhamsoftware.com (work) > > >>> http://www.the111shift.com (play) > > >> > > >> > > >> --------------------------------------------------------------------- > > >> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org > > >> For additional commands, e-mail: dev-h...@lucene.apache.org > > >> > > > > --------------------------------------------------------------------- > > To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org > > For additional commands, e-mail: dev-h...@lucene.apache.org > > > > > -- > ----------------------------------------------------- > Noble Paul > > --------------------------------------------------------------------- > To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org > For additional commands, e-mail: dev-h...@lucene.apache.org > > -- > Marcus Eagan > > -- > Marcus Eagan > > > > -- > Marcus Eagan > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org > For additional commands, e-mail: dev-h...@lucene.apache.org > > > > -- > http://www.needhamsoftware.com (work) > http://www.the111shift.com (play) > > -- > Marcus Eagan > > > > -- > Marcus Eagan > > > > -- Marcus Eagan
