Hi Antonio, First of all, it is nice to see such good progress here. I will fork the repo at github and will give it a try very soon because I'm starting to need it for a real use case, so I will be able to provide user feedback very soon. Let me make some comments over your email:
On Fri, Jun 5, 2015 at 6:48 PM, Antonio David Pérez Morales < adperezmora...@gmail.com> wrote: > Hi devs and all > > As part of the development of the Atlassian Confluence connector for > Manifold, I have created a repository [1] on my GitHub account > Moreover I have developed and pushed the first version of the Confluence > repository connector on a branch called 'feature/repository-connector'. > This first version of the repository connector allows to crawl all the > Confluence pages contained in the spaces (only pages), with the possibility > to filter by a space. > At this moment, only one space or all of them can be configured to be > crawled, but I will continue improving the connector adding more features > like configuring more than one space or others you may see interesting to > be added. > Do we know something about pages metadata? How is the content retrieved, in a raw format or is it possible to get also formatted content? > The ACLs of the documents crawled are the space it belongs to, so that it > is aligned with the proposal for the Authority connector starting at Space > level and then going more fine grain if necessary. > There are no tests included at this moment, because I'm working on them. > Ok, perfect. Probably it is too soon for having tests, but it would be great to include them progressively while you are developing the connector. > > To see if the proposed Authority connector could be developed in a good > way, I have done a quick proof of concept with the logic of it and I was > able to get the spaces which the user has permission to access. So I can > confirm that at space level, we can add permissions to the documents > crawled in order to filter them later on the search engine being used. > > One important thing is that the new Confluence REST Api does not include > any endpoints for permissions yet, so legacy API's have to be used for that > while Confluence developers port completely the legacy methods to the new > api. > Let's hope they include them for the second part of the project and if not we can always use the legacy API and update it after GSoC as a normal contribution > > I will continue improving the repository connector and thinking new > features to be added, but if you think some feature is interesting or good > to have, please let me know and I will take a look in order to include it. > > As stated in the proposal, the Authority Connector is planned for the > second part of the project, but I started to work a bit on it to make sure > we can have a general working version and then iteratively improving it. > Ok, let's focus on the repository connector then until midterm evaluation. Next step would be probably to include more filtering options and check what kind of metadata can be extracted from the pages. I will check the code, but the kind of this we should be thinking now are for example page's URLs to be indexed, seeding strategy and so on. > > As always, comments and suggestions are more than welcome. > Nothing else from my side. Well done so far! > > Regards > > > [1] https://github.com/adperezmorales/confluence-manifold-connector/ > > 2015-05-26 17:10 GMT+02:00 Antonio David Pérez Morales < > adperezmora...@gmail.com>: > > > Hi all > > > > During the bonding period and these days I have been taking a look and > > familiarizing with Confluence API, > > doing some tests using CURL before start the implementation of the > > repository connector which is the first step as stated in the proposal. > > > > I have deployed a local instance of Confluence as well, so that I can do > > the development and tests using that instance. > > > > As stated in the proposal, Confluence is migrating its old APIs (rpc-xml, > > rpc-json) to the new REST API, so all the methods are not migrated yet. > > For getting the changes, fields and content of the documents there won't > > be any problem, but for permissions I have to check more in deep if the > new > > REST API already support it. > > If not, we will have to do a mix using the methods provided by the > > rpc-json api for that, and update it when the REST API contains all the > > methods. > > > > After the first tests, there is no easy way to retrieve the user > > permissions because they are tied to documents and/or spaces. So in order > > to retrieve the user permissions, > > documentId or SpaceId and user have to be provided. I proposed two > > approaches to tackle this, one not so efficient, making many requests to > > Confluence and the other developing a Confluence plugin to get them > > (because at least at Java API level it is possible, but don't know yet > what > > kind of permissions it returns) > > > > So I think, for that part, we can start using (trying) permissions at > > Space level and then try to go finer at document level. > > These problems are mainly related to the second part of the project > > (Authority Connector) but I think it is interesting to put here some > > results after the first overall tests I have performed. > > > > Regards > > >
