Hey all,
I've got an updated stab at a JAX-RS integration using Jersey:
https://github.com/apache/solr/pull/975. That PR solves most of the
concerns mentioned above (authc/authz integration, per-core API
registration). It's still definitely in "draft" territory: logging
needs cleaned up, it needs javadocs and tests, etc. But it should be
complete enough to give a realistic sense of the pros/cons.
Some notes on the implementation:
* Jersey invocation happens in the same ServletFilter as everything
else (SolrDispatchFilter).
** I initially hoped to put it in its own Servlet, but gave up when I
had to copy big chunks of SDF and V2HttpCall anyways. Gus is right
that we should really break SDF down into smaller composable filters,
but I didn't have the time (or knowledge, really) to jump into that
all here.
* JAX-RS resources are registered in an "Application" and then
looked-up and invoked by a "ApplicationHandler".
** ApplicationHandler is analogous to Solr's "ApiBag" class, and it
has the same lifecycle/span (both on CoreContainer and on individual
SolrCore instances)
* Authorization is done via Jersey's "Request Filter" functionality,
see SolrRequestAuthorizer.
* Jersey supports a few response "mime types" natively, but offers an
interface ("MessageBodyWriter") to add others. This PR tests that out
by adding a javabin implementation in "JavabinWriter". It's def not
perfect but should serve as a PoC.
* For testing purposes, I implemented one "admin" API
(ListConfigSetsAPI) and one "core" API
(SomeCoreHandler/SomeCoreResource) to be Jersey-based. Planning to
add a few more in short order, but there's at least something to test
out-of-the-box (e.g. curl http://localhost:8983/api/cluster/configs)
Overall, while it definitely wasn't trivial, in my opinion the
integration code wasn't as bad as it might've been. And especially
with some of the OpenAPI tie-in benefits in mind, JAX-RS is worth
pursuing.
If no one sees any grounds to veto, my next-steps would probably be to
file a JIRA and start hardening the spike and filling gaps (plugin
framework support, etc.) with a mind to commit down the road once
those are cleared away.
Excited to hear what folks think!
Best,
Jason
On Tue, Aug 9, 2022 at 5:03 PM Colvin Cowie <colvin.cowie....@gmail.com> wrote:
>
> Yes that's true, if you only want to generate clients and not the server
> then you can definitely have an easier time of things.
>
> On Tue, 9 Aug 2022 at 14:21, Jason Gerlowski <gerlowsk...@gmail.com> wrote:
>
> > Thanks for the insight Colvin! Spec-driving-the-code sounds appealing
> > in a lot of ways (I'd love to have defaults handled uniformly across
> > Solr, for instance), but I think Jan's right that it's probably
> > infeasible for Solr.
> >
> > And "point taken" about expecting some issues with the tooling!
> >
> > Best,
> >
> > Jason
> >
> > On Tue, Aug 9, 2022 at 6:10 AM Jan Høydahl <jan....@cominvent.com> wrote:
> > >
> > > I have recent experience from a SpringBoot / Kotlin application where we
> > use code-first annotation driven OpenAPI definition. I think that will be
> > the only feasible way for Solr.
> > >
> > > The sweet thing about that approach is that the OpenAPI definitions and
> > documentation is all done inline with the code using @Tag and @Operaton
> > annotations, and there is no drift. You can then generate the OpenAPI spec
> > JSON from code to do (client) code generation. Swagger UI comes for free in
> > the app itself with a simple dependency.
> > >
> > > Jan
> > >
> > > > 9. aug. 2022 kl. 11:45 skrev Colvin Cowie <colvin.cowie....@gmail.com
> > >:
> > > >
> > > > I spent the first few months of this year adopting OpenAPI 3 spec in
> > our
> > > > product, which has a fairly large API.
> > > >
> > > > We previously generated a Swagger 1.0 spec from our Java code, now
> > we're
> > > > generating our POJOs and JAX-RS endpoints from the OA3 spec.
> > > >
> > > > Aside from needing to get off outdated Swagger 1 tools, we were
> > motivated
> > > > to go spec-first to close the gap between our API designs "on paper"
> > and
> > > > the actual implementation, be more rigorous with our API definitions,
> > and
> > > > to (hopefully) speed up / simplify development in the future.
> > > >
> > > > Migration for us was complicated by the fact that our existing
> > codebase was
> > > > inconsistent when it came to things like default initialization of
> > fields
> > > > in POJOs, use of primitives/Objects, and List vs Collection etc. While
> > the
> > > > code generator is consistent in what it produces (naturally). Our
> > codebase
> > > > is also fairly tightly coupled between transport classes and model
> > classes
> > > > as we eschewed the use of models for a lot of it. We did modify the
> > > > templates and add extra x- vendor attributes to our spec to control the
> > > > generation of certain things where we needed to though, so we were
> > able to
> > > > maintain some of our existing idiosyncrasies when necessary.
> > > >
> > > > We were also bitten by some bugs/inconsistencies in the OA3 tooling we
> > used
> > > > - even though OA3 is a specification, everyone seems to have a slightly
> > > > different interpretation or level of support for some parts of it.
> > > > The main pain point for us is around support for polymorphic types /
> > > > inheritance (
> > > >
> > https://swagger.io/docs/specification/data-models/oneof-anyof-allof-not/),
> > > > where modelling them one way works for one viewer but not another, or
> > > > doesn't work for code generators, or api-diffing and so on.
> > > >
> > > > We use
> > > >
> > > > - https://www.apicur.io/apicurito to edit the spec
> > > > - https://github.com/OpenAPITools/openapi-generator to generate
> > Java code
> > > > - https://github.com/rapi-doc/RapiDoc as our viewer, we like
> > > > https://redocly.com/redoc too
> > > > - https://github.com/OpenAPITools/openapi-diff to check for
> > (breaking)
> > > > changes
> > > >
> > > > Getting to where we are was quite painful - it would definitely be a
> > lot
> > > > easier to adopt a spec-first approach in a brand new project - but now
> > that
> > > > we're here it was probably worth it for the consistency and API-first
> > view
> > > > it brings.
> > > >
> > > > So I hope I don't put you off trying it, but just be aware that some
> > things
> > > > won't work as well as you might expect them to and you might find
> > yourself
> > > > contributing to some of these OpenAPI projects to get things to work
> > the
> > > > way you need them to :)
> > > >
> > > > Colvin
> > > >
> > > >
> > > > On Tue, 9 Aug 2022 at 08:30, Jan Høydahl <jan....@cominvent.com>
> > wrote:
> > > >
> > > >> Makes sense to explore at this point to figure out best order of
> > > >> development. And with the door open to slightly adapt some URL
> > patterns in
> > > >> v2, should it be required, it's less likely that roadblocks would
> > derail
> > > >> the entire effort.
> > > >>
> > > >> Jan
> > > >>
> > > >>> 8. aug. 2022 kl. 18:10 skrev Jason Gerlowski <gerlowsk...@gmail.com
> > >:
> > > >>>
> > > >>> Hey all,
> > > >>>
> > > >>> I spent some time last week digging into OpenAPI and spiking out how
> > > >>> it could be integrated into Solr. I came away very impressed. It
> > > >>> opens a lot of really cool doors: auto-generated clients and docs,
> > > >>> "Swagger UI", backcompat/api-breakage detection, etc. There's a LOT
> > > >>> to gain.
> > > >>>
> > > >>> Another takeaway from my spike though was that OpenAPI is much more
> > > >>> "do-able" in JAX-RS projects. It's possible to create
> > > >>> annotation-drive OpenAPI specs without JAX-RS, but it requires more
> > > >>> explicit (and duplicative) documentation of each API's inputs and
> > > >>> ouputs, which probably isn't maintainable in a project as large as
> > > >>> ours.
> > > >>>
> > > >>> With that in mind, I'm planning on returning to the JAX-RS spike Eric
> > > >>> and I did months back, and updating it to cover some of the issues
> > > >>> this thread brought to light. In particular: security integration
> > and
> > > >>> serving collection/core-specific APIs. If that pans out, we can
> > > >>> figure out next steps from there. (A SIP? A JIRA?)
> > > >>>
> > > >>> Before I start that effort though, I figured it'd be worth
> > > >>> double-checking that there are no -1's/vetos on the idea
> > sight-unseen?
> > > >>> If so, let me know!
> > > >>>
> > > >>> Best,
> > > >>>
> > > >>> Jason
> > > >>>
> > > >>>
> > > >>> On Fri, Jan 7, 2022 at 5:36 PM Eric Pugh
> > > >>> <ep...@opensourceconnections.com> wrote:
> > > >>>>
> > > >>>> I wanted to share that our hack day did happen, but it went a bit
> > > >> sideways as we spent the first half of the day experimenting with how
> > to
> > > >> support CORS in Solr. So API related, but not JAX-RS API specific.
> > The
> > > >> second half of the day got consumed by $dayjob.
> > > >>>>
> > > >>>> We’re going to pick it up again next month, and dig into trying out
> > how
> > > >> existing Solr security would work.
> > > >>>>
> > > >>>> https://github.com/gerlowskija/solr/tree/cors_stuff if you want to
> > see.
> > > >>>>
> > > >>>> Eric
> > > >>>>
> > > >>>>
> > > >>>> On Dec 9, 2021, at 10:06 AM, Eric Pugh <
> > ep...@opensourceconnections.com>
> > > >> wrote:
> > > >>>>
> > > >>>> Thank everyone for the input, it’s been a productive conversation!
> > > >>>>
> > > >>>> Jason and I are planning on another hack day Jan 7th to take some of
> > > >> the feedback, and work more on how our spike can help meet some
> > > >> concerns/show promise, so we’ll report back then!
> > > >>>>
> > > >>>> We’re planning on zooming during US East Coast hours, and I’ll drop
> > the
> > > >> Zoom invite in the ASF Slack for anyone who wants to join in and say
> > hi!
> > > >>>>
> > > >>>> Eric
> > > >>>>
> > > >>>>
> > > >>>>
> > > >>>> On Dec 7, 2021, at 3:47 PM, Mark Miller <markrmil...@gmail.com>
> > wrote:
> > > >>>>
> > > >>>> Two cents from the peanut gallery:
> > > >>>>
> > > >>>> I’ve looked at this before. My opinion:
> > > >>>>
> > > >>>> Our stuff was a just terrible, take your pick on the api version.
> > > >> Reasons are numerous.
> > > >>>>
> > > >>>> Custom end points is an anti feature. Even worse for cloud.
> > > >>>>
> > > >>>> JAX-RS looked ridiculously sensible.
> > > >>>>
> > > >>>>
> > > >>>> --
> > > >>>> - MRM
> > > >>>>
> > > >>>>
> > > >>>> _______________________
> > > >>>> Eric Pugh | Founder & CEO | OpenSource Connections, LLC |
> > 434.466.1467
> > > >> | http://www.opensourceconnections.com | My Free/Busy
> > > >>>> Co-Author: Apache Solr Enterprise Search Server, 3rd Ed
> > > >>>> This e-mail and all contents, including attachments, is considered
> > to
> > > >> be Company Confidential unless explicitly stated otherwise,
> > regardless of
> > > >> whether attachments are marked as such.
> > > >>>>
> > > >>>>
> > > >>>> _______________________
> > > >>>> Eric Pugh | Founder & CEO | OpenSource Connections, LLC |
> > 434.466.1467
> > > >> | http://www.opensourceconnections.com | My Free/Busy
> > > >>>> Co-Author: Apache Solr Enterprise Search Server, 3rd Ed
> > > >>>> This e-mail and all contents, including attachments, is considered
> > to
> > > >> be Company Confidential unless explicitly stated otherwise,
> > regardless of
> > > >> whether attachments are marked as such.
> > > >>>>
> > > >>>
> > > >>> ---------------------------------------------------------------------
> > > >>> To unsubscribe, e-mail: dev-unsubscr...@solr.apache.org
> > > >>> For additional commands, e-mail: dev-h...@solr.apache.org
> > > >>>
> > > >>
> > > >>
> > > >> ---------------------------------------------------------------------
> > > >> To unsubscribe, e-mail: dev-unsubscr...@solr.apache.org
> > > >> For additional commands, e-mail: dev-h...@solr.apache.org
> > > >>
> > > >>
> > >
> > >
> > > ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: dev-unsubscr...@solr.apache.org
> > > For additional commands, e-mail: dev-h...@solr.apache.org
> > >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscr...@solr.apache.org
> > For additional commands, e-mail: dev-h...@solr.apache.org
> >
> >
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscr...@solr.apache.org
For additional commands, e-mail: dev-h...@solr.apache.org
