Hi Karl, Alfresco : yeah its the problem here. I am still trying to get something working here.
Solr: Yes i have installed MCF Solr plugin , put jar in SOlr , updated Solr schema with 4 additional fields & updated solrconfig.xml for MCF request handler. I do have authenticated user which is querying the Solr for search results and no unauthenticated user will see the search page. By getting authenticated user into solr query means adding user & his permissions into solr query using a filter or fields? Regards. On Wed, Jun 11, 2014 at 4:19 PM, Karl Wright <[email protected]> wrote: > Hi Lalit, > > About Alfresco: do you see any user security information in this record? > I don't. Without that iinformation, I don't see how security can be done. > Perhaps there's another way to get at it? > > About Solr: Have you installed the appropriate ManifoldCF Solr Plugin > into your solr instance yet? You drop down a jar, and then you need to > include one of the plugin filtering classes in your query parsing or > component processing chains in Solr. You will also need a way of getting > an authenticated user into your Solr query so that the plugin can see it. > > Karl > > > > On Wed, Jun 11, 2014 at 11:08 AM, lalit jangra <[email protected]> > wrote: > >> Sure Karl, >> >> I have invoked the REST based webscript @ >> http://localhost:8080/alfresco/service/api/people/admin & below are >> results in json. >> >> { >> "url": "\/alfresco\/service\/api\/person\/admin", >> "userName": "admin", >> "enabled": true, >> "firstName": "Administrator", >> "lastName": "", >> "jobtitle": null, >> "organization": null, >> "organizationId": "", >> "location": null, >> "telephone": null, >> "mobile": null, >> "email": "[email protected]", >> "companyaddress1": null, >> "companyaddress2": null, >> "companyaddress3": null, >> "companypostcode": null, >> "companytelephone": null, >> "companyfax": null, >> "companyemail": null, >> "skype": null, >> "instantmsg": null, >> "userStatus": null, >> "userStatusTime": null, >> "googleusername": null, >> "quota": -1, >> "sizeCurrent": 0, >> "emailFeedDisabled": false, >> "persondescription": null >> , >> "capabilities": >> { >> "isMutable": >> true >> ,"isGuest": >> false >> ,"isAdmin": >> true >> } >> } >> >> >> Also i am able to index ACL for Sharepoint & Shared Drive into solr, can you >> guide me how can i use them while searching content from these both >> repositories? >> >> >> Do i need to add another filter to my query for same? If so what should be >> the name of filter? >> >> Regards. >> >> >> >> >> On Wed, Jun 11, 2014 at 3:34 PM, Karl Wright <[email protected]> wrote: >> >>> Hi Lalit, >>> >>> Looking at more up-to-date documentation here: >>> http://docs.alfresco.com/4.2/references/RESTful-PersonPersonGet.html >>> >>> It would be great if you could try this operation with a known user >>> against an Alfresco implementation, and see what you get back in the user >>> JSON. I think you could take these steps: >>> >>> (1) Use a browser session to log into your alfresco instance UI >>> (2) Construct the described URL above in the same browser's URL field, >>> and fire it off >>> (3) Send me the resulting JSON >>> >>> Thanks! >>> Karl >>> >>> Karl >>> >>> >>> On Wed, Jun 11, 2014 at 6:22 AM, Karl Wright <[email protected]> wrote: >>> >>>> Looking at the REST api, it seems like it is not a true stateless REST >>>> api. Apparently, there is a session cookie required (?). For example, >>>> see: >>>> >>>> >>>> http://docs.alfresco.com/4.0/references/RESTful-RepositoryLoginticketGet.html >>>> >>>> That will (somewhat) complicate things; it's decidedly non-standard. >>>> Also, I don't see any way still to get access tokens given a user: >>>> >>>> http://docs.alfresco.com/4.0/references/RESTful-Person.html >>>> >>>> Do you see any way to do this? >>>> >>>> Karl >>>> >>>> >>>> >>>> On Wed, Jun 11, 2014 at 5:24 AM, lalit jangra <[email protected] >>>> > wrote: >>>> >>>>> Thanks Karl, >>>>> >>>>> Sadly this confirms that neither CMIS nor Alfresco connectors support >>>>> ACL indexing and storage. I checked into Alfresco connector code but >>>>> nothing has been mentioned about ACL indexing & storage. >>>>> >>>>> Next alfresco does support REST based API very well and infact REST is >>>>> used in Alfresco all over the place. So we can definitely write some >>>>> better >>>>> stuff here. For documentation around REST usage in alfresco, please refer >>>>> to below URLs. Latest version of alfresco is 4.2 and a lot has been >>>>> changed >>>>> into it for almost everything to make things simpler and efficient. >>>>> >>>>> http://docs.alfresco.com/4.0/concepts/API-intro-4.html >>>>> http://wiki.alfresco.com/wiki/Repository_RESTful_API_Reference >>>>> >>>>> These pretty much cover features provided by REST into alfresco and we >>>>> can definitely start from here & do let me know for any more >>>>> documentation. >>>>> Every alfresco instance supports REST based processing so testing REST is >>>>> not a tedious task. >>>>> >>>>> In the meantime i am checking GitHub to find if something is already >>>>> available. >>>>> >>>>> Regards. >>>>> >>>>> >>>>> On Wed, Jun 11, 2014 at 9:50 AM, Karl Wright <[email protected]> >>>>> wrote: >>>>> >>>>>> Hi Lalit, >>>>>> >>>>>> The best way to start is to describe the Alfresco server you are >>>>>> trying to crawl. What version? It matters a lot, as you will read >>>>>> below. >>>>>> >>>>>> The Alfresco connector was originally submitted by Piergiorgio Lucidi >>>>>> under the direction of SourceSense. My understanding is that he had no >>>>>> trouble getting access tokens with the connector, but unfortunately on >>>>>> the >>>>>> other end (mapping users to access tokens) the original Alfresco API's >>>>>> didn't do this. Since then, I believe, Alfresco has completely revamped >>>>>> their API's, and they have a REST-style API available to do the job (or >>>>>> so >>>>>> I am told). A company called Zaizi did some work on it and was supposed >>>>>> to >>>>>> contribute the updated connector, but for two releases that hasn't >>>>>> happened. But it is possible that the connector is in GitHub somewhere? >>>>>> >>>>>> In short, Alfresco is a bit of a mess, and I would very much like to >>>>>> get it repaired to a point where it is usable fully. >>>>>> >>>>>> If your instance has the REST API, and you can provide me with the >>>>>> REST API documentation for your Alfresco instance, I am happy to set up a >>>>>> branch to build an Alfresco REST connector from scratch (provided it >>>>>> looks >>>>>> like everything works the way it is supposed to). Building a connector >>>>>> in >>>>>> this way will take usually a couple of weeks, and you MUST have access to >>>>>> the instance you are trying to crawl, and be willing to test the >>>>>> connector >>>>>> against it and reiterate. There is a chance we'd fail, but with the >>>>>> documentation available in advance, the chances of that would be low. >>>>>> >>>>>> Thoughts? >>>>>> Karl >>>>>> >>>>>> >>>>>> On Wed, Jun 11, 2014 at 1:58 AM, lalit jangra < >>>>>> [email protected]> wrote: >>>>>> >>>>>>> Thanks Karl, >>>>>>> >>>>>>> So its a show stopper now. >>>>>>> >>>>>>> As a fallback mechanism, i am looking for alfresco only connector >>>>>>> for ACL storing mechanism but can you confirm if alfresco specific >>>>>>> connector supports this feature or not. >>>>>>> >>>>>>> And finally if no all the ways, what would be the optimum way to >>>>>>> start implementing the same. >>>>>>> >>>>>>> Regards. >>>>>>> >>>>>>> >>>>>>> On Wed, Jun 11, 2014 at 12:47 AM, Karl Wright <[email protected]> >>>>>>> wrote: >>>>>>> >>>>>>>> Hi Lalit, >>>>>>>> >>>>>>>> CMIS does not give a way for a user to query for ACLs, so >>>>>>>> repository document security is not supported for that connector. >>>>>>>> Documents indexed by CMIS are thus "wide open" and will not be >>>>>>>> restricted >>>>>>>> from being searchable by anybody. >>>>>>>> >>>>>>>> This is, unfortunately, a limitation of CMIS -- at least, CMIS at >>>>>>>> the time the connector was implemented. Feel free to submit patches >>>>>>>> to add >>>>>>>> security to the connector if the spec has evolved to the point where >>>>>>>> it is >>>>>>>> possible. >>>>>>>> >>>>>>>> Thanks, >>>>>>>> Karl >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> On Tue, Jun 10, 2014 at 6:38 PM, lalit jangra < >>>>>>>> [email protected]> wrote: >>>>>>>> >>>>>>>>> Thanks Karl, >>>>>>>>> >>>>>>>>> As per your suggestions, i am able to see ACLs into solr index (I >>>>>>>>> made stored="true" for ACLs in schema.xml) as below. I can see >>>>>>>>> permissions >>>>>>>>> for Sharepoint as well as shared drive but for CMIS, i am not able to >>>>>>>>> see >>>>>>>>> any permissions apart from default stored. Am i missing anything in >>>>>>>>> CMIS? >>>>>>>>> >>>>>>>>> *Sharepoint*: >>>>>>>>> >>>>>>>>> >>>>>>>>> "allow_token_share": [ >>>>>>>>> >>>>>>>>> "__nosecurity__" >>>>>>>>> >>>>>>>>> ], >>>>>>>>> >>>>>>>>> "deny_token_share": [ >>>>>>>>> >>>>>>>>> "__nosecurity__" >>>>>>>>> >>>>>>>>> ] >>>>>>>>> >>>>>>>>> }, >>>>>>>>> >>>>>>>>> { >>>>>>>>> >>>>>>>>> "content_name": "Alfresco-in-an-Hour.pdf" >>>>>>>>> >>>>>>>>> "deny_token_document": [ >>>>>>>>> >>>>>>>>> "SP+Group:DEAD_AUTHORITY" >>>>>>>>> >>>>>>>>> ], >>>>>>>>> >>>>>>>>> "allow_token_document": [ >>>>>>>>> >>>>>>>>> "SP+Group:GTest+lalit+Portal+Visitors", >>>>>>>>> >>>>>>>>> "SP+Group:GTest+lalit+Portal+Owners", >>>>>>>>> >>>>>>>>> "SP+Group:GRestricted+Readers", >>>>>>>>> >>>>>>>>> "SP+Group:GTest+lalit+Administrators", >>>>>>>>> >>>>>>>>> "SP+Group:GTest+lalit+Portal+Members", >>>>>>>>> >>>>>>>>> "SP+Group:Uc%3A0%28.s%7Ctrue", >>>>>>>>> >>>>>>>>> "SP+Group:GHierarchy+Managers", >>>>>>>>> >>>>>>>>> "SP+Group:GApprovers", >>>>>>>>> >>>>>>>>> "SP+Group:GViewers", >>>>>>>>> >>>>>>>>> "SP+Group:GDesigners" >>>>>>>>> >>>>>>>>> ], >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> *Share Drive:* >>>>>>>>> >>>>>>>>> { >>>>>>>>> >>>>>>>>> "deny_token_share": [ >>>>>>>>> >>>>>>>>> "AD+Group:DEAD_AUTHORITY" >>>>>>>>> >>>>>>>>> ], >>>>>>>>> >>>>>>>>> "content_name": "hello.txt", >>>>>>>>> >>>>>>>>> "content_modifier": "lalitjangra", >>>>>>>>> >>>>>>>>> "deny_token_document": [ >>>>>>>>> >>>>>>>>> "AD+Group:DEAD_AUTHORITY" >>>>>>>>> >>>>>>>>> ], >>>>>>>>> >>>>>>>>> "id": "file://///SDD/lalit/manifoldtest/hekko.txt", >>>>>>>>> >>>>>>>>> "allow_token_document": [ >>>>>>>>> >>>>>>>>> "AD+Group:S-1-5-18", >>>>>>>>> >>>>>>>>> "AD+Group:S-1-5-21-2630432783-15384281-2988178474-12088", >>>>>>>>> >>>>>>>>> "AD+Group:S-1-5-21-2630432783-15384281-2988178474-12147", >>>>>>>>> >>>>>>>>> "AD+Group:S-1-5-21-2630432783-15384281-2988178474-12148", >>>>>>>>> >>>>>>>>> "AD+Group:S-1-5-21-2630432783-15384281-2988178474-12149", >>>>>>>>> >>>>>>>>> "AD+Group:S-1-5-21-2630432783-15384281-2988178474-12150", >>>>>>>>> >>>>>>>>> "AD+Group:S-1-5-21-2630432783-15384281-2988178474-12217", >>>>>>>>> >>>>>>>>> "AD+Group:S-1-5-21-2630432783-15384281-2988178474-15154", >>>>>>>>> >>>>>>>>> "AD+Group:S-1-5-21-2630432783-15384281-2988178474-8005", >>>>>>>>> >>>>>>>>> "AD+Group:S-1-5-32-544" >>>>>>>>> >>>>>>>>> ], >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> "allow_token_share": [ >>>>>>>>> >>>>>>>>> "AD+Group:S-1-1-0", >>>>>>>>> >>>>>>>>> "AD+Group:S-1-5-32-544" >>>>>>>>> >>>>>>>>> ], >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> *CMIS *: >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> "allow_token_share": [ >>>>>>>>> >>>>>>>>> "__nosecurity__" >>>>>>>>> >>>>>>>>> ], >>>>>>>>> >>>>>>>>> "deny_token_document": [ >>>>>>>>> >>>>>>>>> "__nosecurity__" >>>>>>>>> >>>>>>>>> ], >>>>>>>>> >>>>>>>>> "deny_token_share": [ >>>>>>>>> >>>>>>>>> "__nosecurity__" >>>>>>>>> >>>>>>>>> ], >>>>>>>>> >>>>>>>>> "allow_token_document": [ >>>>>>>>> >>>>>>>>> "__nosecurity__" >>>>>>>>> >>>>>>>>> ] >>>>>>>>> >>>>>>>>> >>>>>>>>> Regards. >>>>>>>>> >>>>>>>>> >>>>>>>>> On Tue, Jun 10, 2014 at 5:13 PM, Karl Wright <[email protected]> >>>>>>>>> wrote: >>>>>>>>> >>>>>>>>>> Hi Lalit, >>>>>>>>>> >>>>>>>>>> You always use the Active Directory authority for Windows shared >>>>>>>>>> drive authorization. But you do not for SharePoint; you typically >>>>>>>>>> use >>>>>>>>>> SharePoint/Native and SharePoint/AD. You therefore should have a >>>>>>>>>> second >>>>>>>>>> authority group for SharePoint that is distinct from the one for >>>>>>>>>> Windows. >>>>>>>>>> >>>>>>>>>> Each access token is qualified with the name of the ManifoldCF >>>>>>>>>> authority group, so there is never any chance that they will >>>>>>>>>> collide. So >>>>>>>>>> it is perfectly fine to have multiple authority groups in a single >>>>>>>>>> installation, in fact we'd expect you to. >>>>>>>>>> >>>>>>>>>> As for the Solr plugin, you can either download it here: >>>>>>>>>> >>>>>>>>>> http://manifoldcf.apache.org/en_US/download.html >>>>>>>>>> >>>>>>>>>> ... or you will find that it is there in the bin distribution >>>>>>>>>> already, under the "integration" directory. Please have a look and >>>>>>>>>> read >>>>>>>>>> the README. >>>>>>>>>> >>>>>>>>>> Karl >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On Tue, Jun 10, 2014 at 11:41 AM, lalit jangra < >>>>>>>>>> [email protected]> wrote: >>>>>>>>>> >>>>>>>>>>> Thanks Karl, >>>>>>>>>>> >>>>>>>>>>> I am having two content repositories based on active directory >>>>>>>>>>> authentication : SharePoint 2010 and Windows Share Drive, so i am >>>>>>>>>>> using >>>>>>>>>>> active directory as authority type in authority connection. All my >>>>>>>>>>> connections are working fine as well as job is running good but i >>>>>>>>>>> am still >>>>>>>>>>> not able to see any ACL information in solr. >>>>>>>>>>> >>>>>>>>>>> Do i need to enable any configuration in solr to see the same? >>>>>>>>>>> >>>>>>>>>>> Also by Solr Plugin, is it a jar or connector which needs to be >>>>>>>>>>> placed inside solr application server or is it that i need to write >>>>>>>>>>> custom >>>>>>>>>>> code to make search query permission aware? >>>>>>>>>>> >>>>>>>>>>> Can you please guide? >>>>>>>>>>> >>>>>>>>>>> Regards. >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> On Tue, Jun 10, 2014 at 1:59 PM, Karl Wright <[email protected] >>>>>>>>>>> > wrote: >>>>>>>>>>> >>>>>>>>>>>> Hi Lalit, >>>>>>>>>>>> >>>>>>>>>>>> (1) You need first to specify "SharePoint Native" as the >>>>>>>>>>>> authority type in your SharePoint repository connection. You'd >>>>>>>>>>>> use "Active >>>>>>>>>>>> directory" as the authority type only if you were using the "Active >>>>>>>>>>>> directory" authority. To be precise: >>>>>>>>>>>> >>>>>>>>>>>> - If you are using "SharePoint/Native", "SharePoint/AD", or any >>>>>>>>>>>> combination of these for your authority group, use "SharePoint >>>>>>>>>>>> native" >>>>>>>>>>>> authority type >>>>>>>>>>>> - If you are using "Active Directory", use "Active directory" >>>>>>>>>>>> as your authority type >>>>>>>>>>>> >>>>>>>>>>>> (2) You should see acl information get posted to Solr if you >>>>>>>>>>>> have everything configured right. >>>>>>>>>>>> (3) On the Solr side, you need to install and configure the >>>>>>>>>>>> appropriate Solr plugin. Each plugin comes with a README, which >>>>>>>>>>>> describes >>>>>>>>>>>> how to set up the schema on Solr to support security. >>>>>>>>>>>> >>>>>>>>>>>> Thanks! >>>>>>>>>>>> Karl >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> On Tue, Jun 10, 2014 at 8:51 AM, lalit jangra < >>>>>>>>>>>> [email protected]> wrote: >>>>>>>>>>>> >>>>>>>>>>>>> Hello, >>>>>>>>>>>>> >>>>>>>>>>>>> I am working on Apache MCF 1.5.1 indexing Sharepoint 2010 >>>>>>>>>>>>> repository storing index in Solr 4.6. >>>>>>>>>>>>> >>>>>>>>>>>>> For permissions, i followed below steps. >>>>>>>>>>>>> >>>>>>>>>>>>> 1. Created a new authority group called "SharePoint Group" in >>>>>>>>>>>>> MCF. >>>>>>>>>>>>> 2. Created a new User Mapping connection called "SharePoint >>>>>>>>>>>>> User Mapping" using authority group "SharePoint Group" in step 1. >>>>>>>>>>>>> For user >>>>>>>>>>>>> mapping, i used "Regular Expression" mapping and used default >>>>>>>>>>>>> expressions.When i save it, it says "Connection Working". >>>>>>>>>>>>> 3. Created a new Authority connection called "SharePoint >>>>>>>>>>>>> Authority Connection" and using "SharePoint Group"as in step1 as >>>>>>>>>>>>> authority >>>>>>>>>>>>> group type, using "SharePoint User Mapping" as prerequisite, >>>>>>>>>>>>> using my own >>>>>>>>>>>>> domain controller and other related details. When i save it, it >>>>>>>>>>>>> says >>>>>>>>>>>>> "Connection Working". >>>>>>>>>>>>> >>>>>>>>>>>>> Next i have created a new repository connection for SharePoint >>>>>>>>>>>>> called "SharePoint Connection" selecting authority group called >>>>>>>>>>>>> "SharePoint >>>>>>>>>>>>> Group" from pull down list , providing all relevant details for >>>>>>>>>>>>> server , >>>>>>>>>>>>> and choosing authority type as "Active Directory". On saving, it >>>>>>>>>>>>> worked >>>>>>>>>>>>> fine without any error. >>>>>>>>>>>>> >>>>>>>>>>>>> Finally i created a new SharePoint job providing following >>>>>>>>>>>>> details. >>>>>>>>>>>>> 1. Connections : repository connection as "SharePoint >>>>>>>>>>>>> Connection" and output connection as "Solr Connection". >>>>>>>>>>>>> 2. Choosing paths for document library, shared documents, >>>>>>>>>>>>> announcements & lists. >>>>>>>>>>>>> 3. Selecting all metadata in metadata tab selecting all >>>>>>>>>>>>> metadata. >>>>>>>>>>>>> 4. Mapped required metadata fields to solr schema fields. >>>>>>>>>>>>> 4. Enabled the security. >>>>>>>>>>>>> >>>>>>>>>>>>> Now i started the job and i can see SharePoint content getting >>>>>>>>>>>>> indexed and stored in solr. I can verify it using solr admin >>>>>>>>>>>>> query console. >>>>>>>>>>>>> >>>>>>>>>>>>> But i am not seeing any additional ACL information. How can i >>>>>>>>>>>>> verify that ACL & permission information is also stored in Solr. >>>>>>>>>>>>> Do i need >>>>>>>>>>>>> to change Solr configurations also? >>>>>>>>>>>>> >>>>>>>>>>>>> Please help. >>>>>>>>>>>>> >>>>>>>>>>>>> Regards, >>>>>>>>>>>>> Lalit Jangra. >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> -- >>>>>>>>>>> Regards, >>>>>>>>>>> Lalit Jangra. >>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> -- >>>>>>>>> Regards, >>>>>>>>> Lalit Jangra. >>>>>>>>> >>>>>>>> >>>>>>>> >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> Regards, >>>>>>> Lalit Jangra. >>>>>>> >>>>>> >>>>>> >>>>> >>>>> >>>>> -- >>>>> Regards, >>>>> Lalit Jangra. >>>>> >>>> >>>> >>> >> >> >> -- >> Regards, >> Lalit Jangra. >> > > -- Regards, Lalit Jangra.
