On Tue, 23 Feb 2021 at 16:32, Jody Garnett <jody.garn...@gmail.com> wrote:
> Yeah they were intended as documentation and were more complete than what > was previously available. > > Gabe is it possible to throw out the 2.0 yaml and roll out your 3.0 > version as a replacement? > I'll have some allocated time this or next week to donate the project as a community module. The port of 2.0 to 3.0 API definitions is incomplete for a number of reasons: - The object model of request and responses in the 2.0 version don't match what the REST API actually expects and spits out - The migration is painful because it often requires debugging to know exactly what the server does - For the bits I've migrated, only cared about the json representation, stripping off the XML and HTML declarations and examples. If it was up to me, I'd keep it that way. > That is does it meet the documentation goal? > Once the migration is finished, it would not only meet the documentation goal but also allow to generate clients in different languages. Yet the mismatch between API responses and some expected request body schemas makes the generated clients still painful to use. My opinion is still that completing the v2 to v3 migration would make it so much easier to have a v2 rest API with clean contracts and the ability to generate working clients in about any language, which could alleviate the concern Andrea mentioned about having to maintain both API versions. I'd certainly prefer to freeze development on v1 and start out v2 with an api-first approach, it could start being a community module / separate service and see where it goes from there. > -- > Jody Garnett > > > On Wed, 10 Feb 2021 at 10:08, Marc Le Bihan <mlebiha...@gmail.com> wrote: > >> I see : you interfaced or skipped Geoserver flawed 2.0 yaml and produced >> a working OAS 3.0 version instead. >> That's an impressive work ! >> >> Well, I have to think. >> Knowing that yaml (Swagger 2.0) descriptions of Geoserver config REST >> services are wrong, it disturbs me to see them remaining online, >> making people believe that they can trust them either for documentation >> or generation, and that isn't the case. It's like you offer SOAP >> web-services with a wrong WSDL file coming along... >> >> I would like to see what happens if Geoserver REST config server attempts >> to generate itself the json description (or yaml one) that depict itsellf >> the best, using springdoc-openapi-maven-plugin:generate. >> >> >> On 10/02/2021 17:50, Gabriel Roldan wrote: >> >> On Wed, 10 Feb 2021 at 12:21, Marc LE BIHAN <mlebiha...@gmail.com> wrote: >> >>> Oh, Ok. Sorry my english isn't so good and I hadn't understood all. >>> >> np >> >>> An OpenAPI 3 version of geoserver REST contracts already exist and is >>> debugged ? >>> It's a very good news for me and there's absolutely no need to continue >>> using Swagger V2 yaml, then : you're right. >>> But where are these OAS 3 files ? I'd better use them immediately. >>> >> >> >> https://github.com/camptocamp/geoserver-rest-openapi/tree/master/openapi/1.0.0 >> >> Note the catalog object model is in catalog.yml, and workspaces.yml uses >> it. The code generation configured in that project's poms takes care of it. >> Either use it as it (the client it generates) or use it as a guideline. >> The integration tests could be of help >> https://github.com/camptocamp/geoserver-rest-openapi/tree/master/java-client/src/test/java/org/geoserver/restconfig/client >> >> Cheers, >> >> >>> >>> Regards, >>> >>> Marc. >>> >>> Le mer. 10 févr. 2021 à 15:30, Gabriel Roldan <gabriel.rol...@gmail.com> >>> a écrit : >>> >>>> >>>> >>>> On Wed, 10 Feb 2021 at 07:57, Marc LE BIHAN <mlebiha...@gmail.com> >>>> wrote: >>>> >>>>> Hello, >>>>> >>>>> Your tool shows the need to have a way to automate installation(s) of >>>>> Geoserver sometimes. And I have it too. >>>>> curl statements aren't enough for this task, and OpenAPI is a >>>>> convenient way to drive its installation. Using a client API to override >>>>> it >>>>> like you do and offer additional help is useful (I have to create >>>>> internally mine too !). But the beginning is that OpenAPI calls have to >>>>> work, and there's some corrections to do in yaml files provided, for that. >>>>> >>>>> Currently, in [GEOS-9886] issue (that I've renamed to focus only on >>>>> workspaces.yaml content), I've addressed three flaws : >>>>> - One in the input content sent to postWorkspaces, that doesn't send >>>>> the JSON the server expects. >>>>> - Another in the output of geoserver/rest/workspaces (= list all the >>>>> workspaces) that should expect an array incoming in return of a call, >>>>> instead of an object. >>>>> - Another in the output of geoserver/rest/workspaces/{workspaceName} >>>>> (= a single workspace) that should expect a "workspace:" incoming instead >>>>> of an anonymous object. >>>>> >>>>> >>>> If you want to fix the swagger 2 files go for it. I was just pointing >>>> out there're those OAS3 ones with the errors you mention already fixed if >>>> you want to use them instead (i.e. given swagger 2 is superseded by OAS3). >>>> >>>> >>>>> I still have to test deleteWorkspace method, and then the >>>>> workspaces.yaml file will be corrected. >>>>> >>>>> I came too far with my idea of a V2 rest yaml files describing the >>>>> Geoserver REST services, using annotations on server classes to generate >>>>> these yaml files and ensure their accuracy, >>>>> but the problem is established : if on that first workspaces.yaml file >>>>> I've found three bugs, the other might have the same amount and, one by >>>>> one, depending on my needs, I will have to check and correct them. >>>>> >>>>> Regards, >>>>> >>>>> Marc. >>>>> >>>>> Le mer. 10 févr. 2021 à 01:58, Gabriel Roldan < >>>>> gabriel.rol...@gmail.com> a écrit : >>>>> >>>>>> Hey, I just saw this, sorry for being late to the party >>>>>> >>>>>> There's a lot going on in this discussion, so I'll try to be succinct. >>>>>> >>>>>> First and foremost, I am generating a Java client from the OpenAPI >>>>>> documents. Not the swagger 2 ones provided as documentation, but OAS3 >>>>>> ones >>>>>> created using those as reference. >>>>>> >>>>>> Here's a project I just made public a couple of days ago: >>>>>> https://github.com/camptocamp/geoserver-rest-openapi >>>>>> >>>>>> At camptocamp we're using it in production for over a year in an >>>>>> internal project, but now I moved it as a standalone project to be used >>>>>> by >>>>>> the OSS project geOrchestra. >>>>>> >>>>>> It only implements the minimum I needed for that internal project so >>>>>> far. That is, working with workspaces, datastores, feature types, layers, >>>>>> and styles. And probably not even all of it. >>>>>> >>>>>> Also, it focuses on JSON representation only. >>>>>> >>>>>> On the bright side, it's a standalone library, uses the OpenAPI maven >>>>>> code generator, and runs integration tests against a GeoServer docker >>>>>> container managed by the maven life cycle. >>>>>> >>>>>> CI builds are set up using github workflows: >>>>>> https://github.com/camptocamp/geoserver-rest-openapi/actions >>>>>> >>>>>> --- >>>>>> <rant> >>>>>> That said, writing the OAS3 files and the generated client wrapper >>>>>> code is hard, often having to debug GeoServer itself to figure out what's >>>>>> going on in the server to be able of mimicking it on the api definition. >>>>>> There are several inconsistencies, not only in the outdated/incomplete >>>>>> swagger2 files, but in the server itself, most of which I think are due >>>>>> to >>>>>> using XStream to generate the JSON representations, like a single object >>>>>> instead of an array when an array is expected but the result contains a >>>>>> single element, excessive wrapping of objects, and several other >>>>>> annoyances >>>>>> I didn't care to document properly at the time, but believe me, I thought >>>>>> having a working set of OAS3 specs would be a nice first step towards >>>>>> defining a v2 api with clearly defined api contracts and code-generated >>>>>> server stubs. Achieving that would of course require significant >>>>>> resources >>>>>> so it most probably will die in the wish-list. >>>>>> >>>>>> But by all means, feel free to check whether that project is of use >>>>>> to you and to contribute to it. Given enough community interest, we could >>>>>> even manage to land it as part of geoserver's codebase. >>>>>> >>>>>> This Andrea's comment is of most interest and I think I know how to >>>>>> address it >>>>>> >>>>>> > Third, if you want to have reliable yamls that you can generate >>>>>> clients for, make a plan for it, write a GSIP >>>>>> <https://docs.geoserver.org/stable/en/developer/policies/gsip.html>, >>>>>> > resource it fully, and then implement it. Do it in a way that the >>>>>> system is self sustaining (*every deviation * >>>>>> *> gets a test failure*) and you might not need to be involved long >>>>>> term all that much (but plan for some). >>>>>> >>>>>> Since the OAS3 api object model (catalog info objects, etc) mirrors >>>>>> GeoServer Catalog object model, I've experience using mapstruct to create >>>>>> code-generated object model mappers, and to make the code generation fail >>>>>> if something changes, which is a good way to ensure both stay in sync. >>>>>> That'd be of special interest in case a v2 api would be defined, but I >>>>>> had >>>>>> the intention of creating those mappers nonetheless at least for the sake >>>>>> of keeping the models in sync. >>>>>> >>>>>> As a final note, if I had the resources, I'd very much would like to >>>>>> implement such new api version, but would definitely do so as a >>>>>> microservice in the context of this project >>>>>> https://github.com/camptocamp/geoserver-microservices >>>>>> </rant> >>>>>> >>>>>> Hope that helps, >>>>>> >>>>>> Gabriel >>>>>> >>>>>> On Mon, 8 Feb 2021 at 04:49, Andrea Aime < >>>>>> andrea.a...@geo-solutions.it> wrote: >>>>>> >>>>>>> On Sun, Feb 7, 2021 at 10:00 PM Marc Le Bihan <mlebiha...@gmail.com> >>>>>>> wrote: >>>>>>> >>>>>>>> For that, I think the best thing to do would be to annotate server >>>>>>>> classes and object to allow Swagger/Open Api to produce their exact >>>>>>>> declarations of services in a new yaml file. >>>>>>>> But this new yaml file and Swagger-UI documentation should be in >>>>>>>> another URL, let say /geoserver/rest/v2/... to ensure that existing >>>>>>>> code >>>>>>>> that rely on version 1 of the API can continue to use >>>>>>>> /geoserver/rest/... >>>>>>>> without trouble . >>>>>>>> (but this is only the beginning of a solution...) >>>>>>>> >>>>>>> >>>>>>> I don't see the need of a v2, you're not changing the existing REST >>>>>>> API implementation and the resource representations no? Just its >>>>>>> description. >>>>>>> Nobody is using the yamls to generate clients right now, anyways. >>>>>>> >>>>>>> Also, in general, versioning will likely not work, the API changes >>>>>>> little by little as new needs pop up, we'd be at "v200" (exaggerating of >>>>>>> course) if we considered every >>>>>>> model change happened so far, and every new resource addition or new >>>>>>> request parameter. >>>>>>> >>>>>>> What we typically try to do, is to preserve backwards compatibility, >>>>>>> that is, a client written for a previous release should keep on working >>>>>>> with the >>>>>>> new one (new fields are optional), new parameters in resource >>>>>>> requests are optional. >>>>>>> >>>>>>> As said in my previous mail, the yamls are just documentation at the >>>>>>> moment: I don't know why you explain that a client can be generated >>>>>>> from YAML files if they are correctly set up (we know that very >>>>>>> well), as I told you already, right now it's not a goal, due to the >>>>>>> limited >>>>>>> staffing of the project. >>>>>>> The yaml files have been hand written once, and then mostly left >>>>>>> there to rot. >>>>>>> >>>>>>> If you want to join in the effort to make the yaml files >>>>>>> maintainable, and provide resources for both its initial setup and long >>>>>>> term maintenance, then we can make >>>>>>> "yaml as code generation support" a project goal too. Otherwise >>>>>>> there is nothing to discuss, it's not a matter of "right or wrong", >>>>>>> it's a matter of having the man hours to do an initial version of >>>>>>> it, and then look after it in the long term. >>>>>>> Existing developers are busy up to their eyeballs and more, there is >>>>>>> no amount of reasoning that will change that. >>>>>>> >>>>>>> >>>>>>> >>>>>>>> My question is : >>>>>>>> Do you want this file to be integrated in the restconfig project >>>>>>>> with a test if I can create one that is convincing, >>>>>>>> or do you think it's too much, too early, and I better keep these >>>>>>>> corrected file for myself, for my own use only ? >>>>>>>> >>>>>>> >>>>>>> First step, treat it as documentation and simply issue a fix for the >>>>>>> yaml. >>>>>>> >>>>>>> Second step, if you can write a test that generates a client, and >>>>>>> can use it in an interaction with a GeoServer, >>>>>>> in a fully automated way, that is also welcomed (you might need to >>>>>>> setup a GitHub action build too, if the >>>>>>> test needs a live GeoServer to work against, otherwise no one will >>>>>>> run those tests). >>>>>>> >>>>>>> Third, if you want to have reliable yamls that you can generate >>>>>>> clients for, make a plan for it, write a GSIP >>>>>>> <https://docs.geoserver.org/stable/en/developer/policies/gsip.html>, >>>>>>> resource it fully, and then implement it. Do it in a way that the >>>>>>> system is self sustaining (every deviation >>>>>>> gets a test failure) and you might not need to be involved long term >>>>>>> all that much (but plan for some). >>>>>>> >>>>>>> >>>>>>> Regards, Andrea Aime >>>>>>> >>>>>>> == GeoServer Professional Services from the experts! Visit >>>>>>> http://goo.gl/it488V for more information. == Ing. Andrea Aime >>>>>>> @geowolf Technical Lead GeoSolutions S.A.S. Via di Montramito 3/A 55054 >>>>>>> Massarosa (LU) phone: +39 0584 962313 fax: +39 0584 1660272 mob: +39 339 >>>>>>> 8844549 http://www.geo-solutions.it >>>>>>> http://twitter.com/geosolutions_it >>>>>>> ------------------------------------------------------- *Con >>>>>>> riferimento alla normativa sul trattamento dei dati personali (Reg. UE >>>>>>> 2016/679 - Regolamento generale sulla protezione dei dati “GDPR”), si >>>>>>> precisa che ogni circostanza inerente alla presente email (il suo >>>>>>> contenuto, gli eventuali allegati, etc.) è un dato la cui conoscenza è >>>>>>> riservata al/i solo/i destinatario/i indicati dallo scrivente. Se il >>>>>>> messaggio Le è giunto per errore, è tenuta/o a cancellarlo, ogni altra >>>>>>> operazione è illecita. Le sarei comunque grato se potesse darmene >>>>>>> notizia. >>>>>>> This email is intended only for the person or entity to which it is >>>>>>> addressed and may contain information that is privileged, confidential >>>>>>> or >>>>>>> otherwise protected from disclosure. We remind that - as provided by >>>>>>> European Regulation 2016/679 “GDPR” - copying, dissemination or use of >>>>>>> this >>>>>>> e-mail or the information herein by anyone other than the intended >>>>>>> recipient is prohibited. If you have received this email by mistake, >>>>>>> please >>>>>>> notify us immediately by telephone or e-mail.* >>>>>>> _______________________________________________ >>>>>>> Geoserver-devel mailing list >>>>>>> Geoserver-devel@lists.sourceforge.net >>>>>>> https://lists.sourceforge.net/lists/listinfo/geoserver-devel >>>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> Gabriel Roldán >>>>>> >>>>> >>>> >>>> -- >>>> Gabriel Roldán >>>> >>> >> >> -- >> Gabriel Roldán >> >> >> _______________________________________________ >> Geoserver-devel mailing list >> Geoserver-devel@lists.sourceforge.net >> https://lists.sourceforge.net/lists/listinfo/geoserver-devel >> > -- Gabriel Roldán
_______________________________________________ Geoserver-devel mailing list Geoserver-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/geoserver-devel
