The advantage of hosting this outside are * You can have ui experts as committers instead of back end devs like us * You can iterate fast * Users can update to the new version of UI without reinstalling Solr or even restarting their nodes * People can even downgrade to an older version if their current version is buggy * You can even add advanced features to it w/o going through the bureaucracy of Solr
On Thu, Apr 9, 2020, 6:40 PM Noble Paul <[email protected]> wrote: > @Marcus Eagan > > Is the current UI bad? totally yes. > Is a better UI welcome ? 100% > Is your proposal better ? 100% > > My only concern is adding a huge amount of code into our already > bloated codebase > > We have multiple options that can work > > a) We already add so many jars in our distro. Can this be added as a > dependency? totally? > b) Make it a package where , I can use a command to install it. > > To make this work, you can start it as a github project and add us as > collaborators (if needed). You may also add UI experts to that project > to get it built faster. > you can iterate faster too. > > think of us doing a simple command such as > > bin/solr package deploy awesome-solr-gui:0.1.1 > > I think there are some missing pieces in the package system to make > this work. But we can plug those holes once we identify them > > > > I'm OK with any of these solution it it helps us get rid of the bad UI > that we have today. I can pitch in and help you in any which way you > want > > > > On Thu, Apr 9, 2020 at 5:30 PM Ishan Chattopadhyaya > <[email protected]> wrote: > > > > I think a UI should be a package, and I'm willing to help make it > possible such that package manager is capable of doing so. Also, I see no > reason why this UI needs to be in Solr codebase. > > > > I totally agree that Solr users need a better UI. But, I'm -1 to adding > that in Solr core codebase. > > > > I'm willing to consider having this UI as a first party package, > separate from Solr distribution. > > > > By the way, the UI looks great! I fully appreciate your efforts. I think > this UI will be much better placed in your GitHub account under your > stewardship and Solr documents/guides could add a hint on how to let Solr > start with your UI. > > > > > > On Thu, 9 Apr, 2020, 12:38 pm Marcus Eagan, <[email protected]> > wrote: > >> > >> Hey Noble, > >> > >> -1 is a definitive, so I want to clarify that you are saying you do not > wish to remove the EOL front end and replace it with another one in the > longer term? > >> > >> I hear you! As a product manager in my day job, my primary goal is to > find features to cut! I spend a lot of time thinking about non-essential vs > used heavily vs causes more problems than it's worth. I can tell you from > watching the many people in the field at Lucidworks, there are a lot of > people who know quite a bit about Solr, but rely on the Admin UI heavily > because they feel comfortable there. Those people in effect help us stay > employed despite never contributing or being capable of contributing to > Solr. So hear me out. I've got a proposal: > >> > >> To start, I can work on this app as an optional package for your > awesome new package manager. It will be the second one I've worked on in my > evenings and weekends btw. The first was a package validator that I hope to > eventually open source, but its complexity and lack of popularity because > it is security ;( will likely make it the second one I open source/finish. > I'm also collaborating with a couple members of the Lucidworks security > team on that one, but I have built the basics already for them to build > upon. > >> > >> Back to the new UI discussion and my update that I promised. > >> > >> My update was going to be that after evaluating the projects Jan > posted, the most recent project that Jan listed created a pretty good base > to build on. After lots of auditing of the packages and a bit of > refactoring because the UI world moves fast, I was able to get it to > transpile and run again (as I'm sure it previously did) and from (2290 > vulns): > >> > >> no npm fix doesn't magically fix, I wish it did > >> > >> to (2 low sev, non-productions vulns): > >> > >> > >> These two issues will not affect production and really only the unit > tests. Besides, I plan to remove them before we get to a stage even that > matters. I've also started investigating the level of effort for me to get > it to feature parity with the current app. The preliminary answer is not > very much compared to other work I've done in shorter time — working with a > jerk for a boss (years ago, don't worry Hatcher). I'm building a couple of > the missing features as we speak. From the beginning, it will have > infinitely more test coverage and will be a lot more approachable to > contemporary UI developers. It will also make the Solr experience for new > developers simpler. The major design changes that I have been thinking > about would be to the cloud view and the query view. Both of those are > important, the first to more experienced users, and the second to less > advanced users though occasionally an advanced debugger or demo presenter > in my experience. > >> > >> In the end Noble, this is about making Solr more approachable to new > users not experts like you. The growth of Solr adoption only benefits you, > so I would ask you to revisit your -1 at some point in the near future when > you see the progress and breadth of improvement. We have had customers > complain about the Admin UI and the community has even complained about it. > I think this is the right thing to do. If you still consider effectively > upgrading the current Admin UI as feature creep, I can revisit the package > manager compromise or move the efforts elsewhere. I respect your position. > A search service is nothing without a strong and diverse set of skills and > capabilities behind it and making it accessible to everyone who needs it. > >> > >> For those who care, here's my 4 Node cluster of tech products running > locally. > >> > >> > >> > >> The shoulders of the homie that put that scaffold together are broad! > Props to him. I will volunteer to put together extensive docs for JS devs > who want to contribute and make it better once we get it to a place where > it replaces and improves upon the current option. I'll even sponsor some > prizes for college kids or people recently out of work to get cranking on > this bad boy. > >> > >> Thank you Noble and everyone else, > >> > >> Marcus > >> > >> On Wed, Apr 8, 2020 at 11:20 PM Noble Paul <[email protected]> > wrote: > >>> > >>> Solr developers are not UI experts. We are a search service and such a > service should have nice clean APIs + documentation. Is a UI useful ? yes. > The last thing we want today is another complex component in Solr codebase > that nobody understands or cannot maintain. > >>> > >>> So, Solr UI can be hosted outside our codebase and we can have an > option to install UI from that remote repo > >>> > >>> something like "bin/solr install-gui" > >>> > >>> I'm -1 on anymore feature creep. > >>> > >>> --Noble > >>> > >>> > >>> On Thu, Apr 9, 2020 at 12:22 PM Marcus Eagan <[email protected]> > wrote: > >>>> > >>>> Thanks again Gus. > >>>> > >>>> Lots of people indeed misuse REST so we could go on and on about > whether requests are stateless or not in another thread. Let's spare the > group. > >>>> > >>>> I think most everyone on this channel would be in agreement with you > on separate app. I'll be opening a new ticket and a PR that will document a > few things to make it easy for UI devs who know little to no Java how to > get started. > >>>> > >>>> Ishan, there's some significant UI expertise in the team. Erickson > finds his way to open every cookie jar. Erik Hatcher wrote the first > version of Blacklight. I've seen Pugh do lots of work on Quepid's UI. Jan > and Kevin have done a lot of work, and so have many others. The list goes > on, and *likes to work on UI* is a different discussion. > >>>> > >>>> Beyond committers because I'm not a committer, I have UI expertise > that I can polish off and improve for the sake of my interest and > commitment to the community and I like to do it. I've also led UI teams. I > can help to steward the effort overall and keep things up to date up to the > point where I need to ask one of the committers to help me get changes > merged. I'll probably even hire a developer to work on it once we are to > that point. ;-) > >>>> > >>>> Expertise is not something that should block us but motivate us to > expand this community and/or our own skillsets long term. > >>>> > >>>> Thank you both and everyone else, > >>>> > >>>> Marcus > >>>> > >>>> On Wed, Apr 8, 2020, 10:21 AM Gus Heck <[email protected]> wrote: > >>>>> > >>>>> While running it in an external node does ensure separability, I > don't think it does a good job of addressing my other point of not needing > to manage a 3rd server. It's still a server if it's started by java, and > one still has to ensure it exists, and it will be extra hard to figure out > how to configure it if started by Solr. > >>>>> > >>>>> I'm strongly in favor of us having a UI from my perspective as a > consultant it makes discovery of things like their startup parameters and > directories and such very easy (just go to front page of the admin screen), > and it's so much easier to get a customer with security concerns and strict > controls on who can access what (think banks, military, etc) to share a web > session where they drive the UI than to get direct access to machines. > It'll be a lot slower and much lower service to be making people wait while > I craft curl statements to paste into the web session (and then fix the > inevitable typos, or detect when they missed the last char of what I > pasted, etc...). > >>>>> > >>>>> I definitely against Solr spawning some other server (node or > otherwise) on it's own and thereby requiring additional system > dependencies, or creating a second process that needs to be configured and > properly secured. To me that's even worse than requiring the UI to run > outside of Solr. We have a perfectly good web container already, and > furthermore there's a much greater likelihood that maintainers will be > facile with java/j2ee than anything else (IMHO). It's great if the > framework we choose uses little or no JSP/Servlet and is modernized with a > 100% javascript, templated etc. front end, but the back end should be > java/jetty because we've got lots of java folks. > >>>>> > >>>>> If the back end matters deeply then you're not really programming to > MVC/REST style... > >>>>> > >>>>> So there's another $0.02 :) and if you're not careful I'll give you > an entire nickle's worth of ways people misuse/misunderstand the term REST > :) > >>>>> > >>>>> -Gus > >>>>> > >>>>> On Tue, Apr 7, 2020 at 9:06 PM Marcus Eagan <[email protected]> > wrote: > >>>>>> > >>>>>> Gus, > >>>>>> > >>>>>> Your $.02 are worth a lot more than $.02 USD, so thank you. > >>>>>> > >>>>>> By separate app, I think I mean to endorse managed by a Node.js > process started by NPM. I don’t think that conflicts with what you have > proposed. The NPM command should be issued by Java || or Bash but I don’t > think it would add significant overhead. Also, seems like on CI and or > precommit hooks front end could be sizzled in parallel without adding much > overhead. > >>>>>> > >>>>>> As for the front end framework, the most important things to > consider in my view are simplicity and maintainability. We need to do a > thorough analysis on the ecosystem and issues like the size of a React > project vs Angular project vs Vue project, but React and Vue certainly have > the velocity and the hearts if the front end community more than Angular. > React is MIT license now and for the foreseeable > >>>>>> future thanks to the power and reach of its developers. > >>>>>> > >>>>>> <[email protected]> wrote: > >>>>>>> > >>>>>>> +1 for Angular CLI / Typescript since I've fiddled with this in a > minor way recently, Also MIT license is super friendly. > >>>>>>> > >>>>>> > >>>>>> As a disenfranchised volunteer to the project, I also assume voters > on specific choices like frameworks will be helping build in some respect > at some point now or in the future. Is that a fair or misguided assumption? > >>>>>> > >>>>>> Marcus > >>>>>> > >>>>>> On Tue, Apr 7, 2020 at 17:15 Gus Heck <[email protected]> wrote: > >>>>>>> > >>>>>>> +1 for Angular CLI / Typescript since I've fiddled with this in a > minor way recently, Also MIT license is super friendly. > >>>>>>> > >>>>>>> Separate App - hmm... that's got some attraction, but also gives > my stomach some churning when I think about solr now requiring management > of 3 different servers (solr, something to serve UI and zookeeper). Adding > more infrastructure gives me pause with respect to all the smaller > installations. I've had several small self funded startup clients and a few > clients with existing initial installs that they are outgrowing in places > where procuring new machines and new software is a 6-12 mo endeavor and > both types seem to squirm when I make suggestions such as running zookeeper > separately, (let alone 3 of them). I think separate looks good for medium > to large folks or very large companies that **already have** a solr expert > on hand, but hurts the small clients and the departments in large orgs that > got started with insufficient advice/expertise, so maybe > >>>>>>> > >>>>>>> - The UI should be installed by default > >>>>>>> - it should be easy to remove it, or start with it disabled > >>>>>>> - it should be self contained and separately downloadable. > >>>>>>> > >>>>>>> My recent fiddling included figuring out how to make angular CLI > play nice in a J2ee war file structure seen here: > https://github.com/nsoft/ns-login > >>>>>>> > >>>>>>> By play nice I mean, > >>>>>>> - build creates a war file that "just works" when installed > >>>>>>> - Angluar CLI commands work > >>>>>>> - Angular serve command works (for auto-reloading ui changes, > running on port 4200; note the use of proxy to allow it to talk to an > already running web container) > >>>>>>> > >>>>>>> My $0.02, > >>>>>>> > >>>>>>> -Gus > >>>>>>> > >>>>>>> On Mon, Apr 6, 2020 at 11:03 AM Jörn Franke <[email protected]> > wrote: > >>>>>>>> > >>>>>>>> I think standalone would be very useful. > >>>>>>>> I propose Angular with Typescript - it fits to a more data > centric approach with data types etc. > >>>>>>>> Maybe even two types of UIs - Admin UI and a simple Search UI. > >>>>>>>> > >>>>>>>> > >>>>>>>> Am 06.04.2020 um 16:53 schrieb Jan Høydahl <[email protected] > >: > >>>>>>>> > >>>>>>>> Thanks for kickstarting this and bringing some fresh blood and > enthusiasm :) > >>>>>>>> > >>>>>>>> Looks like others have had similar wish for a standalone Solr > Admin App, here’s a quick GitHub search for inspiration: > >>>>>>>> > >>>>>>>> https://github.com/savantly-net/solr-admin (Angular, nice > screenshots, 1y old) > >>>>>>>> https://github.com/kezhenxu94/yasa (vuejs, impressive > screenshots, 2y old) > >>>>>>>> https://github.com/thereactleague/galaxy (React, no > screenshots, 4y old) > >>>>>>>> > >>>>>>>> They all seem abandoned but perhaps a new official effort could > bring their developers in as contributors again? > >>>>>>>> > >>>>>>>> the people who work on the Admin UI do not need to be expected > to know the Java workflow, necessarily. This reality widens the net for who > can contribute. > >>>>>>>> > >>>>>>>> > >>>>>>>> Agree. Frontend devs have been a shortage in this project, and if > we can make it easier to attract UI committers who feel at home and > productive with the UI code, that would be a win. On the other hand, if we > expect that the UI will be maintained by regular Java committers, then > anything that makes it easier for them/us to contribute is also a win, like > perhaps strongly-typed. > >>>>>>>> > >>>>>>>> Again, thanks Marcus for reviving this topic. Let us all try not > to be overly ambitious here or shoot the initiative down with bikeshedding. > It is far more important to fuel the energy and momentum and get something > built than to remain stuck :) > >>>>>>>> > >>>>>>>> Jan > >>>>>>>> > >>>>>>>> > >>>>>>>> 6. apr. 2020 kl. 13:47 skrev Marcus Eagan <[email protected]>: > >>>>>>>> > >>>>>>>> Coming back to these existential questions from my phone: > >>>>>>>> > >>>>>>>> > >>>>>>>> > >>>>>>>> > >>>>>>>> > >>>>>>>> > >>>>>>>> > >>>>>>>> > >>>>>>>> > >>>>>>>> Jan Høydahl > >>>>>>>> Added 1 hour ago > >>>>>>>> There are many opinions around admin UI. So I think the best > place to start would be a new mail-thread in dev@ to discuss the way > forward. Before we start a major re-work, we should probably ask ourselves > a few existential questions: > >>>>>>>> > >>>>>>>> Should we turn Amin UI into a standalone app instead of embedded > in Solr? > >>>>>>>> > >>>>>>>> > >>>>>>>> I think it should be a standalone app. There are many advantages > gained from a separation of such concerns. Some of the ones include, the > people who work on the Admin UI do not need to be expected to know the Java > workflow, necessarily. This reality widens the net for who can contribute. > >>>>>>>> > >>>>>>>> Testing becomes a lot easier because JS developers are accustomed > to building tests for static assets and self-contained node apps. They > generally know less about testing a bit of JS within a massive Java > project. The test could also run independently for changes that only > affect the front end. Adding test coverage without adding time to tests > sounds awesome. > >>>>>>>> > >>>>>>>> There are quite a few tickets over the years that have seemed to > suggest that people want more fine-grained control over the Solr admin UI > overall. Two recent tickets discussed topics like running a Solr Admin app > on only one node and disabling it al together for whatever reason. See: > https://issues.apache.org/jira/browse/SOLR-14014. > >>>>>>>> > >>>>>>>> What UI framework? Guess anything is better than current EOL, but > will largely depend on who is willing to do the job! > >>>>>>>> > >>>>>>>> I’m happy to take this on (and willing to follow through on > completing in my nights and weekends), but I am mostly framework agnostic. > My stronge preference would be React, provided the license is kosher. There > was one blip of “practically unusable for most orgs” a couple years back, > but Facebook made it right really soon after. However, I’m flexible. > Angular (not JS) and Vue are also great. I would recommend we consider > Typescript also because of the size of project and number of strongly-typed > devs on this mailing list. My only reservation with TypeScript, though it > may not apply in this case, is that the supersets of JS have changed a lot > more than the frameworks. While CoffeeScript was an unnecessary layer of > abstraction from my limited perspective, TypeScript might make JS more > embraceable to a list of Java hackers. > >>>>>>>> > >>>>>>>> Current UI has no test coverage, can we do better with the new UI? > >>>>>>>> > >>>>>>>> > >>>>>>>> It’s imperative.React, Angular, and Vue each make it easy to > include tests. > >>>>>>>> > >>>>>>>> > >>>>>>>> > >>>>>>>> > https://issues.apache.org/jira/browse/SOLR-12276?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17076204#comment-17076204 > >>>>>>>> > >>>>>>>> > >>>>>>>> > >>>>>>>> > >>>>>>> > >>>>>>> > >>>>>>> -- > >>>>>>> http://www.needhamsoftware.com (work) > >>>>>>> http://www.the111shift.com (play) > >>>>>> > >>>>>> -- > >>>>>> Marcus Eagan > >>>>>> > >>>>> > >>>>> > >>>>> -- > >>>>> http://www.needhamsoftware.com (work) > >>>>> http://www.the111shift.com (play) > >>> > >>> > >>> > >>> -- > >>> ----------------------------------------------------- > >>> Noble Paul > >> > >> > >> > >> -- > >> Marcus Eagan > >> > > > -- > ----------------------------------------------------- > Noble Paul >
