And whether they can coexist at all. From the previous emails it seemed to me this change will introduce changes to the existing APIS and I that means that compatibility will be broken. I am also not sure if it we will maintain the existing UI after that.
---- On Tue, 14 Apr 2020 09:51:23 -0400 [email protected] wrote ---- Yes, please go through SIP before inviting us to review the actual PR. And once you re-work the SIP page based on feedback and converge towards consensus, I’d recommend starting a formal VOTE thread for the SIP, as outlined in the SIP page. That way people can have a chance to speak up on the overall design so that does not become a topic once again in the PR. My wish for the SIP is to be very clear on exactly how the UI is proposed shipped and whether any manual steps are needed to enable/install. Also try to clarify whether you propose a big-bang replace or whether the old and new UI will need to co-exists. Jan 14. apr. 2020 kl. 15:29 skrev Cassandra Targett <[email protected]>: Marcus, A couple thoughts… First, sorry you’re having problems with Confluence. I suspect the issue is permissions. There are only two groups allowed to add pages to the SOLR space, “lucene” and “lucene-pmc”. I believe these correspond to ASF LDAP groups, which would mean they include committers and PMC members only. We can grant you individual permission to add/edit pages, however; we’ve done this for a handful of others. I could do this for you, just ping me off-thread so I can confirm your username. I don’t know what happened to the pages you tried to create. You can try to see if they are hidden somehow, maybe in your profile’s “Recently worked on” section (https://cwiki.apache.org/confluence/dashboard.action#recently-worked). When you’re on that page, the menu on the left also shows a “Saved for later” page that maybe has them. At any rate, I wouldn’t suggest putting all the design decisions/discussion into a PR. That’s what we traditionally used Jira for. We decided to use SIPs to make navigating design discussions in Jira easier - decisions would get lost in comments - and to forestall someone doing possibly wasted work until they have some degree of community agreement for what they hope to do. If you put it in a PR, by its very nature those decisions would already be made and that could lead to significant amounts of rework. The SIP process is to write up the SIP and then file a Jira issue, and you can’t file a PR without a Jira, so you’ll need a Jira issue for this work anyway. If you really can’t write a SIP, then the fallback is Jira, not a PR. Re: the API issues - those should also go into Jira. I can’t see your list so far since it requires permission, but I’ll just say I would not recommend dropping your whole doc into a single Jira. They’ll need to be broken out into separate ones (and it’s pretty likely issues already exist for at least some of the things). I know that sounds like a PITA, but we don’t track issues in personal Google docs, we use Jira. And if some of the items are possibly controversial, then individual items for each one will allow us to work through the controversies and not stall progress on things that are not problematic (if anyone is so inclined to work on those). Hopefully you don’t mind the unsolicited advice on this, just trying to help you understand some of our ways of doing things. Cassandra On Apr 14, 2020, 3:18 AM -0500, Marcus Eagan <[email protected]>, wrote: Mike and Gus, I tried to share my SIP after writing it, and then it disappeared. I also tried to write some and save it, but then it disappeared. I also tried to write it outside the wiki and paste it again. I'm not sure what's going on, but I will try again tomorrow I suppose. If I fail again, my SIP will probably come in the form of a Pull Request, with videos, screenshots, a lit of todos, and helpful documentation so that people can get started easily. >From this project, I have also uncovered what I would consider some serious >issues with the Solr API and some other issues that I dig through Jira for >before I bother the community. I will do my best to document them all and open >tickets. Some times I rage quit, walk away, come back and forget. I will try >to fix some of them, but some of them that I have looked into look like worm >cans. It might be good if a few people here are prepared to explore fixing the >API in a few areas, but I'm not sure about the best way to go about doing that >with angering lots of people. Seeking advice on how to approach the API >challenges. I don't just want to start complaining about things, nor do I want >to take on everything. A small group from a variety of different organizations >to discuss some of these challenges might be most helpful. On the flip side, I >suspect things will improve in other areas as a result. You can view progress here: https://drive.google.com/file/d/1NgO34DRp1llMp3EwJQcKBn2LM4r5PD39/view?usp=sharing I've mostly managed to get all the requests sorted, and rendering of the data (mostly). i didn't get much time today as I was very busy at work. And tomorrow I get my quarantine checkup and will probably be sleepy early. But by the end of next weekend, I should have finished everything but the query editor and all the other associated collections screens which I will bucket under collections. That current dropdown menu is less intuitive than optimal but I understand. Soliciting feedback now because once I open the PR I am hoping we have agreed on as many things as possible. If no one suggest query view designs, I will mock some up and share them. Again, I repeat, not a designer. But I did take couple classes, and have built products from start to finish mostly on an island so I will be able to manage. Looking for feedback that the community might be getting from its clients. Obviously, I cannot accommodate all the requests, but if something is recurring, I want to support the users to support the adoption of this new UI. As if I needed to say it, and many of you probably suspected this was coming: Erick this is a lot of work. Holy shit. And not just because I'm a product manager. It would be challenging for anyone. This project, like all project, has some skeletons. From my perspective, that's gfreat because there are lots of things to improve. Also, the project is still pretty amazing. Thanks again for your help everyone, Marcus On Mon, Apr 13, 2020 at 9:55 AM Marcus Eagan <[email protected]> wrote: Gus, SIP sounds good. I will share. Marcus On Mon, Apr 13, 2020 at 09:13 Gus Heck <[email protected]> wrote: Maybe start collecting Design and Design choices in a SIP? This discussion has been good and there seems to be consensus that we want a new UI, we want it to be a package and we want the package to be available by default and well tested. "Package" seems to imply that it can be added or removed or replaced or an alternative UI installed along side of it. If we got all of those things done this would be amazingly awesome :) Another thing that would be valuable is a good doc that explains "how to edit and maintain the UI", written for an audience that is experienced in SW dev but not UI development (probably including some basics around framework chosen). This could be in a README or in the "dev docs" that has been mentioned elsewhere. The SIP would be a great place to elaborate on technology choices & supply a link to things like the video :) On Mon, Apr 13, 2020 at 10:35 AM Mike Drob <[email protected]> wrote: Hi Marcus, The mailing list strips attachments for some folks, can you upload the video somewhere else and link to it for us poor unfortunate souls? Thanks for your work! Excited to see the progress as it happens. Mike On Mon, Apr 13, 2020 at 5:30 AM Marcus Eagan <[email protected]> wrote: In general, I asked for some degree of trouble when I volunteered for this work. Don't beat me too hard. My primary goal is to achieve three things: 1) Improve security when using Solr Admin UI by removing EOL, unsupported code. 2) Make it easier or more welcoming for new developers to try the project and even become contributors in all areas of the project because the UI looks and functions as slightly more contemporary. 3) Give back more substantially to a community from which I have received so much with a testable and perrty UI. I've added another (4) which is contribute to help make the package manager a first-class citizen in the minds of many Solr users around the world via the UI package. I will need some help from someone in this list on deploying this UI with a jar in Gradle if we want to offer an alternative option to install the UI in Solr 9. I've attached where I'm at. It's a nights and weekends project, but I will always be available for bug fixes or discussions, unless i'm in a meeting or reading a book 😇 I won't solicit a ton of feedback prior to the the first PR, which I will leave open for a few weeks or even a couple months while I put some lipstick on it and improve performance of the application <<<<<<< Over the weekend read lots of documentation, wrote a bunch of code when it wasn't holidays, built the services, and stumbled through the logic of rendering all these data points so you can watch the attached video if you want to check it out. >>>>>>>> . There's definitely some areas where I didn't do the TypeScript thing because I'm still trying to grok it a bit.The two areas of the that I am looking to somewhat overhaul in potentially controversial ways are the queries page and actual flow of the collections experience, which at the moment are sort of linked, yet disconnected at the same time. The Collections page tab today is really an Alias page. It won't have its own tab in the new application is the plan, unless someone can give me a good reason. Almost everything else will stay the same. For that reason, I'd like to solicit feedback if anyone has any examples or ideas they'd like to share, I would greatly appreciate it. I'm somewhat far along with the Admin UI as of now. Short weekend because of holiday activities and general quarantine craziness, but I'm maybe 25-35% of the way to completion, depending on how much care I devote to the query screen. Even though there are some big problems with Angular — some major ones — I think this was the right way to go for many reasons. I'm about a quarter as fast in Angular as I am in React, but this is the right decision for the long haul. I can elaborate if anyone really cares. Most importantly, this app will be a lot easier to maintain. But now I have a handle on TypeScript. Many optimizations and improvements coming to Solr as a result. TypeScript can be pretty opinionated about APIs for someone thinking that they are back in the wild wild west. Sending big air hugs, Marcus On Fri, Apr 10, 2020 at 16:20 Noble Paul <[email protected]> wrote: +1 to David Smiley On Sat, Apr 11, 2020, 8:12 AM Marcus Eagan <[email protected]> wrote: I don't see admin UI as non-core. I think that an application UI for end-users of an application consumes Solr non-core. I have to resign from arguing, though. I don't consider myself a UI expert. I can do the work. On Fri, Apr 10, 2020 at 11:42 AM Ishan Chattopadhyaya <[email protected]> wrote: David, you capture my thoughts well. Having a UI as a package gives users more choice and gives our users more flexibility. 1. Users would be able to use a latter version of the UI with an older version of Solr, or vice versa. 2. Users should be able to install multiple types of UI, from different publishers, at once. 3. Contributors should be able to contribute to the UI more easily, since collaboration can be less bureaucratic. Experts like Marcus won't need to depend on preoccupied committers like us. 4. A UI (not the default one) can use libraries that aren't even Apache 2.0 compatible. 5. We can setup and use UI test frameworks for test automation (selenium etc), that would be challenging to setup and maintain with ASF Jenkins. List goes on.. Whether the package is a first party or third party can be a separate discussion. There should be an extremely easy and well defined way (support in the script itself) to start Solr with the packaged UI enabled. In any case, I don't think it is conducive to let UI code be part of the Solr's core codebase, where it currently is. The reason is, we can't fix bugs if we break something. We don't have automated testing either to know whether or not we broke anything. Every healthy project has a rich plugin ecosystem, and such non core improvements should be delivered via packages. On Fri, 10 Apr, 2020, 8:33 pm Marcus Eagan, <[email protected]> wrote: I agree with you. 😃 On Fri, Apr 10, 2020 at 06:22 David Smiley <[email protected]> wrote: I disagree that "package management frameworks are for loading non-essential features or features not enabled by default". I don't think the proposal of the UI being a "package" (in the new package system) implies that the UI (or _any_ package) is not a highly valuable package that is so highly valuable that we want it installed by default. Noble and I were brainstorming on some ideas where even much of Solr's internal instances of plugin interfaces (e.g. query parsers, etc.) might even be a new "core" package or some-such. The value in putting much of Solr in a package, 1st party, is separation of concerns. and better classpath management. I think it's "essential" that a UI ship with Solr by default -- meaning, without the user having to take any additional steps whatsoever. As Jan said it's been this way a long time. ~ David Smiley Apache Lucene/Solr Search Developer http://www.linkedin.com/in/davidwsmiley On Fri, Apr 10, 2020 at 1:35 AM Marcus Eagan <[email protected]> wrote: I think package management frameworks are for loading non-essential features or features not enabled by default. On essentialness, experts should not decide what is essential based on how they use a system. They should consider the community of users. Regarding UI, it is and should be enabled by default. Only a few use cases prefer it to be disabled and some of those are because of its current state. They would like to use it in an updated form. What is the technical rationale that outweighs the needs and behaviors of our users to strip the user interface out of Solr? Thank you Noble and everyone else, Marcus On Thu, Apr 9, 2020 at 19:06 Noble Paul <[email protected]> wrote: My 2 cents (again) if packages are disabled by default , how will UI work? We can make an exception for this one and enable only this by default Do we test the UI and certify it? The UI package can be shipped along with Solr distro ,like the million other jars that we ship with Solr today and every version of Solr can be certified for a certain version of the UI package. We should have sanity tests to ensure that the given version of UI works well with Solr. "My commit has broken the UI and it's not my problem" should not be a valid excuse. The UI sanity tests should pass as a part of the tests. Is the UI important? Yes, the admin UI is the face of Solr for may users. People always assumed it existed & they depend on it. The current admin UI has fallen behind. If the new UI effort delivers on the promise, this is a great opportunity to get rid of that old baggage & make Solr codebase even slimmer On Fri, Apr 10, 2020 at 6:06 AM Jan Høydahl <[email protected]> wrote: > > Solr has always had an admin UI and if anyone wants to propose it should not, > please start another thread or vote about that, and do not divert this thread > which is about how to improve and future proof the Admin UI. > > I believe the Admin UI should be strengthened and enhanced, not removed. It > can perfectly well be an official and even default on part of every release, > perfectly in sync. Whether it is in core as today or a package or a > stand-alone process or a new webapp, are then really what we discuss here. > > Perhaps after people have voiced their opinions in this thread, a SIP can be > crafted with a concrete plan. We can then have a vote on the SIP. > > Jan Høydahl > > > 9. apr. 2020 kl. 20:06 skrev Cassandra Targett <[email protected]>: > > > > > > Thanks for your message, Gus. You touched on things I was thinking this > > morning as I caught up to the thread, and had started to draft a message > > about. > > > > I feel like there is an assumption underlying some of our discussion about > > packages that says a feature or whatever has to either part of our core > > codebase or 100% maintained by someone “outside” the community (by which I > > mean someone who is not a committer and/or operates completely > > independently from the rest of the project activities). > > > > But it’s important to remember there’s nothing inherent in the package > > concept that says a package can't be wholly > > maintained/distributed/supported by the Lucene PMC and our community of > > committers and contributors. It’s not uncommon for software to have a base > > package of the core software and also plugins that most users would > > consider “essential”, maintained by the same people making the core, but > > which are added after the base install. There would be details to work out > > in terms of how we manage that, but none of those are technically > > impossible. I don’t think we only have a binary choice (3rd party package > > or part of Solr core). > > > > In fact, I’d go so far as to say I suspect the only way packages are going > > to see any real traction is if we take the lead in creating and maintaining > > a few to show the way forward for everyone else who may be interested in > > doing the same. The package concept introduces an idea of a Solr ecosystem > > that has not really existed to date and like all new ecosystems > > (communities), it needs some degree of nurturing to grow or it will not > > take off. > > > > To bring it back around to the UI, though, I agree we need to decide: is a > > UI important? I would argue that it is. I regularly talk to users whose > > Solr knowledge and experience are quite advanced yet who still rely > > entirely on the UI to carry out basic tasks. It’s just easier - less to > > remember, don’t need to look up a command, etc. Persistent problems with > > performance of the UI in large clusters and gaping holes in functionality > > are deeply frustrating to those users. > > > > Because it is important to our users, even if any new UI ends up living > > outside our community, it will need our explicit approval in some form or > > we’re going to hear complaints until the end of time that Solr doesn’t have > > a UI (or worse, we “took away” the UI). Without our ongoing endorsement and > > blessing it will just be another toy maintained by “somebody” (no offense, > > Marcus) until he is forced to abandon it due to lack of user interest > > and/or lack of personal time to do all the heavy lifting by himself. > > > > Cassandra > >> On Apr 9, 2020, 11:32 AM -0500, Erick Erickson <[email protected]>, > >> wrote: > >> Gus: > >> > >> Very thoughtful post. I think you raise an _extremely_ important point > >> about “how critical is the UI?” And by extension other packages. If > >> they’re critical to Solr, the question of how to keep them in sync becomes > >> paramount. > >> > >> I agree that the admin UI is important, if we have a mechanism to insure > >> it’s kept in sync with the release that would be near the to of my list. > >> > >> Best, > >> Erick > >> > >>> On Apr 9, 2020, at 12:06 PM, Gus Heck <[email protected]> wrote: > >>> > >>> In my view this brings us up against a bit of an existential question. > >>> How do we ensure quality of packages that are key to Solr? I'm sure that > >>> there are folks who don't find the UI very useful, but it's important to > >>> others. The rationale that "elastic keeps their's separate" has to be > >>> tempered by the actual real differences between Elastic and Solr. Elastic > >>> has a corporate sponsor, a coordinated road map, and explicitly ensures > >>> that all the bits that are maintained separately work together (so that > >>> they don't have excessive support costs or bad first experiences that > >>> impact their bottom line etc.). > >>> > >>> Solr is in a different place however, and we need to carefully examine > >>> the question of whether something that works for Elastic works for us. > >>> Lucidworks and several other large companies do spend a lot of money on > >>> developers that contribute to Solr, but there is no organization around > >>> multiple components that MUST work together. Another example of this is > >>> Solr and Lucene, and our defense against a lack of coordination of > >>> components in that case has been to unify them into a single release > >>> package. > >>> > >>> So IMHO we need to answer 2 questions: > >>> 1.) Do we consider the UI important. If I'm alone or in a minority in > >>> feeling it's important, then so be it and it doesn't really matter what > >>> we do. (maybe a vote?) > >>> 2.) If we make it "a package" how do we ensure that important packages > >>> such as the UI are never broken by a new release. > >>> > >>> IMHO I don't thing we should tolerate a situation where things we > >>> consider important are broken frequently. > >>> > >>> For my part obviously my answer to 1 is "yes" :). As for 2, one thing > >>> that comes to mind is what the Ant project did (may still do?) with (and > >>> my memory from 15 years ago is foggy here, so forgive me if something I > >>> say is not quite perfect) the GUMP build server that ran ant builds for a > >>> bunch of different projects that depended on ant to ensure early > >>> detections of changes that would break existing projects. If we have a > >>> good UI test suite and commit to that being part of the release build > >>> package that might be a solution. I honestly don't actually care where it > >>> lives, but I do think it hurts us if it becomes broken and unusable, or > >>> hard to install. > >>> > >>> My worry is that "Solr developers are not UI developers" is really > >>> code-words for "I want to be able to break it and let others clean up the > >>> mess, because I'm not a UI developer". I have this worry with respect to > >>> all "packages", but I may be missing info from discussions about the > >>> package system, which initiated during a very busy period for me so > >>> background links that I should have read are welcome if I've got > >>> something wrong here :) I went looking for a SIP but didn't see it... I > >>> have found a google doc linked from SOLR-13661 > >>> > >>> Finally to a detail about one of the above suggestions the option to > >>> automatically download and install the UI could be good, but that then > >>> requires that packages be available from somewhere that never goes away > >>> like maven central, or that Apache commits to hosting a repository server > >>> indefinitely, but again that's surely been discussed WRT packages > >>> already... Using Github in such a way is subject to being broken > >>> arbitrarily when Github decides to restrict things for cost reasons (ask > >>> Bower about that one WRT rate limiting...) or the "repository" has to be > >>> something local and therefore must be included part of the > >>> distribution... at which point it's still a thing we distribute and since > >>> we're distributing it and we probably don't mean to distribute broken > >>> stuff we still need UI developers... > >>> > >>> Also, I thought the package loading stuff was supposed to be disabled by > >>> default for security, that seems to conflict with or at least complicate > >>> the notion of easily installing as a package. > >>> > >>> So "package" is a good for modularizing code, or for 3rd party (possibly > >>> paid) plugins (I have a client that might find that interesting in the > >>> future) but we have to ensure that it doesn't lead to a lack of > >>> maintenance for things that are critical. > >>> > >>> Incidentally though I've said I favor Angular CLI, (significantly because > >>> I've got some start on learning it) it also occurs to me that perhaps > >>> anything "modern" is a difficulty because those things all have a > >>> learning curve, and maximizing accessibility and ease of modifications > >>> for folks not steeped in UI development might be our priority (different > >>> from the priorities a commercial site would have). The flip side argument > >>> is that with a popular framework, it would be easier for UI focused folks > >>> to contribute... but will they? and does that leave us perennially > >>> rewriting the UI in whatever is popular? (maybe that's ok?) I think in > >>> all our decisions here we need to be very careful to distinguish how our > >>> needs may differ in unusual ways from the needs of commercial web > >>> development. > >>> > >>> -Gus > >>> > >>> On Thu, Apr 9, 2020 at 8:14 AM Erick Erickson <[email protected]> > >>> wrote: > >>> Marcus: > >>> > >>> re-reading the thread, it looks to me like the consensus from Noble and > >>> Ishan and Jan is that as long as the new, nifty UI is a separate package, > >>> go ahead and knock yourself out ;). The objection is to making it part of > >>> the Solr code base… We’ll all be thrilled with if we can rip the current > >>> admin UI out ;) > >>> > >>> That said, I suspect it’ll be one of the tighter packages. It’d be > >>> super-cool if we could run the UI tests on Jenkins say once a day just to > >>> keep it up to date. > >>> > >>> The admin UI has always been somewhat awkwardly bolted on the side of > >>> Solr, it’d be great to have it have a more elegant architecture. > >>> > >>> The other exciting thing would be that clients could then use the package > >>> code as something they can incorporate/fork/whatever. Practically every > >>> client I’ve worked with at large installations has rolled their own > >>> dashboard. If they could use a package as a starting point, it’d be > >>> welcome. > >>> > >>> Best, > >>> Erick > >>> > >>>> On Apr 9, 2020, at 3:07 AM, Marcus Eagan <[email protected]> wrote: > >>>> > >>>> Hey Noble, > >>>> > >>>> -1 is a definitive, so I want to clarify that you are saying you do not > >>>> wish to remove the EOL front end and replace it with another one in the > >>>> longer term? > >>>> > >>>> I hear you! As a product manager in my day job, my primary goal is to > >>>> find features to cut! I spend a lot of time thinking about non-essential > >>>> vs used heavily vs causes more problems than it's worth. I can tell you > >>>> from watching the many people in the field at Lucidworks, there are a > >>>> lot of people who know quite a bit about Solr, but rely on the Admin UI > >>>> heavily because they feel comfortable there. Those people in effect help > >>>> us stay employed despite never contributing or being capable of > >>>> contributing to Solr. So hear me out. I've got a proposal: > >>>> > >>>> To start, I can work on this app as an optional package for your awesome > >>>> new package manager. It will be the second one I've worked on in my > >>>> evenings and weekends btw. The first was a package validator that I hope > >>>> to eventually open source, but its complexity and lack of popularity > >>>> because it is security ;( will likely make it the second one I open > >>>> source/finish. I'm also collaborating with a couple members of the > >>>> Lucidworks security team on that one, but I have built the basics > >>>> already for them to build upon. > >>>> > >>>> Back to the new UI discussion and my update that I promised. > >>>> > >>>> My update was going to be that after evaluating the projects Jan posted, > >>>> the most recent project that Jan listed created a pretty good base to > >>>> build on. After lots of auditing of the packages and a bit of > >>>> refactoring because the UI world moves fast, I was able to get it to > >>>> transpile and run again (as I'm sure it previously did) and from (2290 > >>>> vulns): > >>>> <image.png> > >>>> no npm fix doesn't magically fix, I wish it did > >>>> > >>>> to (2 low sev, non-productions vulns): > >>>> <image.png> > >>>> > >>>> These two issues will not affect production and really only the unit > >>>> tests. Besides, I plan to remove them before we get to a stage even that > >>>> matters. I've also started investigating the level of effort for me to > >>>> get it to feature parity with the current app. The preliminary answer is > >>>> not very much compared to other work I've done in shorter time — working > >>>> with a jerk for a boss (years ago, don't worry Hatcher). I'm building a > >>>> couple of the missing features as we speak. From the beginning, it will > >>>> have infinitely more test coverage and will be a lot more approachable > >>>> to contemporary UI developers. It will also make the Solr experience for > >>>> new developers simpler. The major design changes that I have been > >>>> thinking about would be to the cloud view and the query view. Both of > >>>> those are important, the first to more experienced users, and the second > >>>> to less advanced users though occasionally an advanced debugger or demo > >>>> presenter in my experience. > >>>> > >>>> In the end Noble, this is about making Solr more approachable to new > >>>> users not experts like you. The growth of Solr adoption only benefits > >>>> you, so I would ask you to revisit your -1 at some point in the near > >>>> future when you see the progress and breadth of improvement. We have had > >>>> customers complain about the Admin UI and the community has even > >>>> complained about it. I think this is the right thing to do. If you still > >>>> consider effectively upgrading the current Admin UI as feature creep, I > >>>> can revisit the package manager compromise or move the efforts > >>>> elsewhere. I respect your position. A search service is nothing without > >>>> a strong and diverse set of skills and capabilities behind it and making > >>>> it accessible to everyone who needs it. > >>>> > >>>> For those who care, here's my 4 Node cluster of tech products running > >>>> locally. > >>>> > >>>> <image.png> > >>>> > >>>> The shoulders of the homie that put that scaffold together are broad! > >>>> Props to him. I will volunteer to put together extensive docs for JS > >>>> devs who want to contribute and make it better once we get it to a place > >>>> where it replaces and improves upon the current option. I'll even > >>>> sponsor some prizes for college kids or people recently out of work to > >>>> get cranking on this bad boy. > >>>> > >>>> Thank you Noble and everyone else, > >>>> > >>>> Marcus > >>>> > >>>> On Wed, Apr 8, 2020 at 11:20 PM Noble Paul <[email protected]> wrote: > >>>> Solr developers are not UI experts. We are a search service and such a > >>>> service should have nice clean APIs + documentation. Is a UI useful ? > >>>> yes. The last thing we want today is another complex component in Solr > >>>> codebase that nobody understands or cannot maintain. > >>>> > >>>> So, Solr UI can be hosted outside our codebase and we can have an option > >>>> to install UI from that remote repo > >>>> > >>>> something like "bin/solr install-gui" > >>>> > >>>> I'm -1 on anymore feature creep. > >>>> > >>>> --Noble > >>>> > >>>> > >>>> On Thu, Apr 9, 2020 at 12:22 PM Marcus Eagan <[email protected]> > >>>> wrote: > >>>> Thanks again Gus. > >>>> > >>>> Lots of people indeed misuse REST so we could go on and on about whether > >>>> requests are stateless or not in another thread. Let's spare the group. > >>>> > >>>> I think most everyone on this channel would be in agreement with you on > >>>> separate app. I'll be opening a new ticket and a PR that will document a > >>>> few things to make it easy for UI devs who know little to no Java how to > >>>> get started. > >>>> > >>>> Ishan, there's some significant UI expertise in the team. Erickson finds > >>>> his way to open every cookie jar. Erik Hatcher wrote the first version > >>>> of Blacklight. I've seen Pugh do lots of work on Quepid's UI. Jan and > >>>> Kevin have done a lot of work, and so have many others. The list goes > >>>> on, and *likes to work on UI* is a different discussion. > >>>> > >>>> Beyond committers because I'm not a committer, I have UI expertise that > >>>> I can polish off and improve for the sake of my interest and commitment > >>>> to the community and I like to do it. I've also led UI teams. I can help > >>>> to steward the effort overall and keep things up to date up to the point > >>>> where I need to ask one of the committers to help me get changes merged. > >>>> I'll probably even hire a developer to work on it once we are to that > >>>> point. ;-) > >>>> > >>>> Expertise is not something that should block us but motivate us to > >>>> expand this community and/or our own skillsets long term. > >>>> > >>>> Thank you both and everyone else, > >>>> > >>>> Marcus > >>>> > >>>> On Wed, Apr 8, 2020, 10:21 AM Gus Heck <[email protected]> wrote: > >>>> While running it in an external node does ensure separability, I don't > >>>> think it does a good job of addressing my other point of not needing to > >>>> manage a 3rd server. It's still a server if it's started by java, and > >>>> one still has to ensure it exists, and it will be extra hard to figure > >>>> out how to configure it if started by Solr. > >>>> > >>>> I'm strongly in favor of us having a UI from my perspective as a > >>>> consultant it makes discovery of things like their startup parameters > >>>> and directories and such very easy (just go to front page of the admin > >>>> screen), and it's so much easier to get a customer with security > >>>> concerns and strict controls on who can access what (think banks, > >>>> military, etc) to share a web session where they drive the UI than to > >>>> get direct access to machines. It'll be a lot slower and much lower > >>>> service to be making people wait while I craft curl statements to paste > >>>> into the web session (and then fix the inevitable typos, or detect when > >>>> they missed the last char of what I pasted, etc...). > >>>> > >>>> I definitely against Solr spawning some other server (node or otherwise) > >>>> on it's own and thereby requiring additional system dependencies, or > >>>> creating a second process that needs to be configured and properly > >>>> secured. To me that's even worse than requiring the UI to run outside of > >>>> Solr. We have a perfectly good web container already, and furthermore > >>>> there's a much greater likelihood that maintainers will be facile with > >>>> java/j2ee than anything else (IMHO). It's great if the framework we > >>>> choose uses little or no JSP/Servlet and is modernized with a 100% > >>>> javascript, templated etc. front end, but the back end should be > >>>> java/jetty because we've got lots of java folks. > >>>> > >>>> If the back end matters deeply then you're not really programming to > >>>> MVC/REST style... > >>>> > >>>> So there's another $0.02 :) and if you're not careful I'll give you an > >>>> entire nickle's worth of ways people misuse/misunderstand the term REST > >>>> :) > >>>> > >>>> -Gus > >>>> > >>>> On Tue, Apr 7, 2020 at 9:06 PM Marcus Eagan <[email protected]> > >>>> wrote: > >>>> Gus, > >>>> > >>>> Your $.02 are worth a lot more than $.02 USD, so thank you. > >>>> > >>>> By separate app, I think I mean to endorse managed by a Node.js process > >>>> started by NPM. I don’t think that conflicts with what you have > >>>> proposed. The NPM command should be issued by Java || or Bash but I > >>>> don’t think it would add significant overhead. Also, seems like on CI > >>>> and or precommit hooks front end could be sizzled in parallel without > >>>> adding much overhead. > >>>> > >>>> As for the front end framework, the most important things to consider in > >>>> my view are simplicity and maintainability. We need to do a thorough > >>>> analysis on the ecosystem and issues like the size of a React project vs > >>>> Angular project vs Vue project, but React and Vue certainly have the > >>>> velocity and the hearts if the front end community more than Angular. > >>>> React is MIT license now and for the foreseeable > >>>> future thanks to the power and reach of its developers. > >>>> > >>>> <[email protected]> wrote: > >>>> +1 for Angular CLI / Typescript since I've fiddled with this in a minor > >>>> way recently, Also MIT license is super friendly. > >>>> > >>>> > >>>> As a disenfranchised volunteer to the project, I also assume voters on > >>>> specific choices like frameworks will be helping build in some respect > >>>> at some point now or in the future. Is that a fair or misguided > >>>> assumption? > >>>> > >>>> Marcus > >>>> > >>>> On Tue, Apr 7, 2020 at 17:15 Gus Heck <[email protected]> wrote: > >>>> +1 for Angular CLI / Typescript since I've fiddled with this in a minor > >>>> way recently, Also MIT license is super friendly. > >>>> > >>>> Separate App - hmm... that's got some attraction, but also gives my > >>>> stomach some churning when I think about solr now requiring management > >>>> of 3 different servers (solr, something to serve UI and zookeeper). > >>>> Adding more infrastructure gives me pause with respect to all the > >>>> smaller installations. I've had several small self funded startup > >>>> clients and a few clients with existing initial installs that they are > >>>> outgrowing in places where procuring new machines and new software is a > >>>> 6-12 mo endeavor and both types seem to squirm when I make suggestions > >>>> such as running zookeeper separately, (let alone 3 of them). I think > >>>> separate looks good for medium to large folks or very large companies > >>>> that **already have** a solr expert on hand, but hurts the small clients > >>>> and the departments in large orgs that got started with insufficient > >>>> advice/expertise, so maybe > >>>> > >>>> - The UI should be installed by default > >>>> - it should be easy to remove it, or start with it disabled > >>>> - it should be self contained and separately downloadable. > >>>> > >>>> My recent fiddling included figuring out how to make angular CLI play > >>>> nice in a J2ee war file structure seen here: > >>>> https://github.com/nsoft/ns-login > >>>> > >>>> By play nice I mean, > >>>> - build creates a war file that "just works" when installed > >>>> - Angluar CLI commands work > >>>> - Angular serve command works (for auto-reloading ui changes, running on > >>>> port 4200; note the use of proxy to allow it to talk to an already > >>>> running web container) > >>>> > >>>> My $0.02, > >>>> > >>>> -Gus > >>>> > >>>> On Mon, Apr 6, 2020 at 11:03 AM Jörn Franke <[email protected]> wrote: > >>>> I think standalone would be very useful. > >>>> I propose Angular with Typescript - it fits to a more data centric > >>>> approach with data types etc. > >>>> Maybe even two types of UIs - Admin UI and a simple Search UI. > >>>> > >>>> > >>>>> Am 06.04.2020 um 16:53 schrieb Jan Høydahl <[email protected]>: > >>>>> > >>>>> Thanks for kickstarting this and bringing some fresh blood and > >>>>> enthusiasm :) > >>>>> > >>>>> Looks like others have had similar wish for a standalone Solr Admin > >>>>> App, here’s a quick GitHub search for inspiration: > >>>>> > >>>>> https://github.com/savantly-net/solr-admin (Angular, nice screenshots, > >>>>> 1y old) > >>>>> https://github.com/kezhenxu94/yasa (vuejs, impressive screenshots, 2y > >>>>> old) > >>>>> https://github.com/thereactleague/galaxy (React, no screenshots, 4y old) > >>>>> > >>>>> They all seem abandoned but perhaps a new official effort could bring > >>>>> their developers in as contributors again? > >>>>> > >>>>>> the people who work on the Admin UI do not need to be expected to know > >>>>>> the Java workflow, necessarily. This reality widens the net for who > >>>>>> can contribute. > >>>>> > >>>>> Agree. Frontend devs have been a shortage in this project, and if we > >>>>> can make it easier to attract UI committers who feel at home and > >>>>> productive with the UI code, that would be a win. On the other hand, if > >>>>> we expect that the UI will be maintained by regular Java committers, > >>>>> then anything that makes it easier for them/us to contribute is also a > >>>>> win, like perhaps strongly-typed. > >>>>> > >>>>> Again, thanks Marcus for reviving this topic. Let us all try not to be > >>>>> overly ambitious here or shoot the initiative down with bikeshedding. > >>>>> It is far more important to fuel the energy and momentum and get > >>>>> something built than to remain stuck :) > >>>>> > >>>>> Jan > >>>>> > >>>>> > >>>>>> 6. apr. 2020 kl. 13:47 skrev Marcus Eagan <[email protected]>: > >>>>>> > >>>>>> Coming back to these existential questions from my phone: > >>>>>> > >>>>>> > >>>>>> > >>>>>> > >>>>>> > >>>>>> > >>>>>> > >>>>>> > >>>>>> > >>>>>> > >>>>>> Jan Høydahl > >>>>>> Added 1 hour ago > >>>>>> There are many opinions around admin UI. So I think the best place to > >>>>>> start would be a new mail-thread in dev@ to discuss the way forward. > >>>>>> Before we start a major re-work, we should probably ask ourselves a > >>>>>> few existential questions: > >>>>>> • Should we turn Amin UI into a standalone app instead of embedded in > >>>>>> Solr? > >>>>>> > >>>>>> I think it should be a standalone app. There are many advantages > >>>>>> gained from a separation of such concerns. Some of the ones include, > >>>>>> the people who work on the Admin UI do not need to be expected to know > >>>>>> the Java workflow, necessarily. This reality widens the net for who > >>>>>> can contribute. > >>>>>> > >>>>>> Testing becomes a lot easier because JS developers are accustomed to > >>>>>> building tests for static assets and self-contained node apps. They > >>>>>> generally know less about testing a bit of JS within a massive Java > >>>>>> project. The test could also run independently for changes that only > >>>>>> affect the front end. Adding test coverage without adding time to > >>>>>> tests sounds awesome. > >>>>>> > >>>>>> There are quite a few tickets over the years that have seemed to > >>>>>> suggest that people want more fine-grained control over the Solr admin > >>>>>> UI overall. Two recent tickets discussed topics like running a Solr > >>>>>> Admin app on only one node and disabling it al together for whatever > >>>>>> reason. See: https://issues.apache.org/jira/browse/SOLR-14014. > >>>>>> > >>>>>> • What UI framework? Guess anything is better than current EOL, but > >>>>>> will largely depend on who is willing to do the job! > >>>>>> I’m happy to take this on (and willing to follow through on completing > >>>>>> in my nights and weekends), but I am mostly framework agnostic. My > >>>>>> stronge preference would be React, provided the license is kosher. > >>>>>> There was one blip of “practically unusable for most orgs” a couple > >>>>>> years back, but Facebook made it right really soon after. However, I’m > >>>>>> flexible. Angular (not JS) and Vue are also great. I would recommend > >>>>>> we consider Typescript also because of the size of project and number > >>>>>> of strongly-typed devs on this mailing list. My only reservation with > >>>>>> TypeScript, though it may not apply in this case, is that the > >>>>>> supersets of JS have changed a lot more than the frameworks. While > >>>>>> CoffeeScript was an unnecessary layer of abstraction from my limited > >>>>>> perspective, TypeScript might make JS more embraceable to a list of > >>>>>> Java hackers. > >>>>>> > >>>>>> • Current UI has no test coverage, can we do better with the new UI? > >>>>>> > >>>>>> It’s imperative.React, Angular, and Vue each make it easy to include > >>>>>> tests. > >>>>>> > >>>>>> > >>>>>> > >>>>>> https://issues.apache.org/jira/browse/SOLR-12276?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17076204#comment-17076204 > >>>>>> > >>>>>> > >>>>>> > >>>>> > >>>> > >>>> > >>>> -- > >>>> http://www.needhamsoftware.com (work) > >>>> http://www.the111shift.com (play) > >>>> -- > >>>> Marcus Eagan > >>>> > >>>> > >>>> > >>>> -- > >>>> http://www.needhamsoftware.com (work) > >>>> http://www.the111shift.com (play) > >>>> > >>>> > >>>> -- > >>>> ----------------------------------------------------- > >>>> Noble Paul > >>>> > >>>> > >>>> -- > >>>> Marcus Eagan > >>>> > >>> > >>> > >>> --------------------------------------------------------------------- > >>> To unsubscribe, e-mail: [email protected] > >>> For additional commands, e-mail: [email protected] > >>> > >>> > >>> > >>> -- > >>> http://www.needhamsoftware.com (work) > >>> http://www.the111shift.com (play) > >> > >> > >> --------------------------------------------------------------------- > >> To unsubscribe, e-mail: [email protected] > >> For additional commands, e-mail: [email protected] > >> > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [email protected] > For additional commands, e-mail: [email protected] > -- ----------------------------------------------------- Noble Paul --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected] -- Marcus Eagan -- Marcus Eagan -- Marcus Eagan --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected] -- http://www.needhamsoftware.com (work) http://www.the111shift.com (play) -- Marcus Eagan -- Marcus Eagan
