I found a solution to create the right configuration for the CMIS Repository Connector in the integration test code, and now it works ;)
Chemistry guys are supporting me about an issue that I have found in the OpenCMIS InMemory Repository that I would like to use in the integration test for Manifold. Now I'm finishing the integration test implementation using the public Alfresco CMIS server. Then I can start to test this new version of the InMemory Repository with the bugfix provided by Jens. And I have to follow some useful suggestions provided by Florian. I'll let you know soon about all these tasks. Piergiorgio ---------- Forwarded message ---------- From: Jens Hübel <jhue...@opentext.com> Date: 2011/8/4 Subject: RE: CMIS and Lucene To: d...@chemistry.apache.org Hi Piergiorgio, you are right. I could reproduce the problem. There is a bug in the query implementation of the InMemory server if you query for predefined properties like cmis:name, cmis:createdBy, etc. :-( I have fixed this, so hopefully this works for you now if you take the latest build. I have created CMIS-413 for this ( https://issues.apache.org/jira/browse/CMIS-414). Please reopen if you still have issues. One more note: Your test code is quite fragile if you use a fixed name for your test document/folder. This implementation does not allow multiple objects with the same name in one folder. This means unless you restart the server your code will work only once. I recommend you using a random UUID as name or something like that. In the default configuration the InMemory creates a tree of document and folders by default. So it might not be necessary to create objects at all (use the Workbench to take a look at that). If you really need this reproducible behavior there is an option to run the server using the local binding in the same Java VM. Then you can restart the server with each test. The InMemory JUnit tests use this mechanism. I do not recommend this however, because you bypass the whole protocol layer for AtomPub/SOAP. This leaves many issues undetected a client may see in a real connection later (and of course you lose the option to switch to another CMIS server just by changing configuration). Jens -----Original Message----- From: Piergiorgio Lucidi [mailto:piergiorgioluc...@gmail.com] Sent: Mittwoch, 3. August 2011 14:48 To: d...@chemistry.apache.org Subject: Re: CMIS and Lucene Hi Jens, here the code of my integration test that is used to create some content in the InMemory Repository, the OpenCMIS server is running because it is bootstrapped by Jetty / Maven. The CMIS Repository Connector works fine with Alfresco 3.4d Community, but I would like to have the OpenCMIS server in the test suite. In the following snippet I started to initialize the test environment with a new folder with a new content, here I don't have any problem, it works with the InMemory Repository: private Session getCmisClientSession(){ > // default factory implementation > SessionFactory factory = SessionFactoryImpl.newInstance(); > Map<String, String> parameters = new HashMap<String, String>(); > // user credentials > parameters.put(SessionParameter.USER, "dummyuser"); > parameters.put(SessionParameter.PASSWORD, "dummysecret"); > // connection settings > parameters.put(SessionParameter.ATOMPUB_URL, CMIS_ENDPOINT_TEST_SERVER > ); > parameters.put(SessionParameter.BINDING_TYPE, BindingType.ATOMPUB > .value()); > // create session > return factory.getRepositories(parameters).get(0).createSession(); > } > @Before > public void createTestArea() > throws Exception > { > try > { > Session session = getCmisClientSession(); > //creating a new folder > Folder root = session.getRootFolder(); > Map<String, Object> folderProperties = new HashMap<String, > Object>(); > folderProperties.put(PropertyIds.OBJECT_TYPE_ID, "cmis:folder"); > folderProperties.put(PropertyIds.NAME, "testdata"); > > Folder newFolder = root.createFolder(folderProperties); > //create a new content in the folder > String name = "testdata1.txt"; > // properties > // (minimal set: name and object type id) > Map<String, Object> contentProperties = new HashMap<String, > Object>(); > contentProperties.put(PropertyIds.OBJECT_TYPE_ID, "cmis:document"); > contentProperties.put(PropertyIds.NAME, name); > > // content > byte[] content = "CMIS Testdata One".getBytes(); > InputStream stream = new ByteArrayInputStream(content); > ContentStream contentStream = new ContentStreamImpl(name, newBigInteger(content), > "text/plain", stream); > > // create a major version > Document newContent1 = newFolder.createDocument(contentProperties, > contentStream, null); But if I try to search the new content in the InMemory Repository, in the same way I implemented in the CMIS Repository Connector: ItemIterable<QueryResult> results = session.query("SELECT * FROM > cmis:folder WHERE cmis:name='testdata'", false); > for (QueryResult result : results) { > String id = result.getPropertyValueById(PropertyIds.OBJECT_ID); > } It returns the following exception: org.apache.chemistry.opencmis.commons.exceptions.CmisRuntimeException: null > at > org.apache.chemistry.opencmis.client.bindings.spi.atompub.AbstractAtomPubService.convertStatusCode( > AbstractAtomPubService.java:450) > at > org.apache.chemistry.opencmis.client.bindings.spi.atompub.AbstractAtomPubService.post( > AbstractAtomPubService.java:568) > at > org.apache.chemistry.opencmis.client.bindings.spi.atompub.DiscoveryServiceImpl.query( > DiscoveryServiceImpl.java:141) > at org.apache.chemistry.opencmis.client.runtime.SessionImpl$3.fetchPage( > SessionImpl.java:557) > at > org.apache.chemistry.opencmis.client.runtime.util.AbstractIterator.getCurrentPage( > AbstractIterator.java:132) > at > org.apache.chemistry.opencmis.client.runtime.util.CollectionIterator.hasNext( > CollectionIterator.java:48) > at org.apache.manifoldcf.cmis_tests.APISanityTest.createTestArea( > APISanityTest.java:139) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at sun.reflect.NativeMethodAccessorImpl.invoke( > NativeMethodAccessorImpl.java:39) > at sun.reflect.DelegatingMethodAccessorImpl.invoke( > DelegatingMethodAccessorImpl.java:25) > at java.lang.reflect.Method.invoke(Method.java:597) > at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall( > FrameworkMethod.java:44) > at org.junit.internal.runners.model.ReflectiveCallable.run( > ReflectiveCallable.java:15) > at org.junit.runners.model.FrameworkMethod.invokeExplosively( > FrameworkMethod.java:41) > at org.junit.internal.runners.statements.RunBefores.evaluate( > RunBefores.java:27) > at org.junit.internal.runners.statements.RunAfters.evaluate( > RunAfters.java:31) > at org.junit.runners.BlockJUnit4ClassRunner.runNotIgnored( > BlockJUnit4ClassRunner.java:79) > at org.junit.runners.BlockJUnit4ClassRunner.runChild( > BlockJUnit4ClassRunner.java:71) > at org.junit.runners.BlockJUnit4ClassRunner.runChild( > BlockJUnit4ClassRunner.java:49) > at org.junit.runners.ParentRunner$3.run(ParentRunner.java:193) > at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:52) > at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:191) > at org.junit.runners.ParentRunner.access$000(ParentRunner.java:42) > at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:184) > at org.junit.runners.ParentRunner.run(ParentRunner.java:236) > at org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference.run( > JUnit4TestReference.java:49) > at org.eclipse.jdt.internal.junit.runner.TestExecution.run( > TestExecution.java:38) > at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests( > RemoteTestRunner.java:467) > at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests( > RemoteTestRunner.java:683) > at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run( > RemoteTestRunner.java:390) > at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main( > RemoteTestRunner.java:197) Let me know if you have any hints! Thank you again for your support. Regards, Piergiorgio 2011/8/3 Jens Hübel <jhue...@opentext.com> > Hi Pergiorio, > > do you have a code piece that I can take a look at? What is your client > API? Do you connect to Chemistry client API (chemistry-opencmis-client-api)? > Or perhaps you can grab the query sent from the log file? > > Thanks Jens > > > -----Original Message----- > From: Piergiorgio Lucidi [mailto:piergiorgioluc...@gmail.com] > Sent: Mittwoch, 3. August 2011 13:22 > To: d...@chemistry.apache.org > Subject: Re: CMIS and Lucene > > Hi Jens, > > I contributed to the Apache ManifoldCF (ex Lucene Connector Framework) > project implementing the CMIS Connector using OpenCMIS 0.4.0. > > Now I'm working on integration test trying to use the OpenCMIS InMemory > Repository that is available from Apache Chemistry ;) > > I have some problems because now I'm finishing my test implementation but > it > seems that I can't execute queries on the InMemory Repository, it returns a > null string in the convertStatusCode method. > > The CMIS Repository Connector that I implemented for ManifoldCF has a job > parameter that is the CMIS Query, this parameter is used by agents to > select > all the contents that needs to be indexed on the Output Connector (for > example Apache Solr). > Without a query feature exposed by the InMemory Repository I can't test the > connector in the right way. > > I tried to use the latest version of the InMemory Repository > (0.5.0-SNAPSHOT) but with the same result. > > Have you got any ideas to solve this problem? > Thank you for your support. > > Regards, > Piergiorgio > > > 2011/8/3 Jens Hübel <jhue...@opentext.com> > > > Hi Chemistries, > > > > > > > > is anyone of us already involved in the Lucene project? > > > > > > > > Reading this here > > http://search-lucene.com/m/wI38e1K3BWJ&subj=Re+CMIS+Connector+Tests > > > > > > > > I think it would make sense to give them an introduction to what we have. > > > > > > > > if I understand this correctly one of the options they discuss is to > create > > an InMemory mock server. It does not make much sense to duplicate what we > > already have (and if we need getContentChanges() for that I am happy to > > implement this). > > > > > > > > Jens > > > > > > > > > > > -- > Piergiorgio Lucidi > Web: http://about.me/piergiorgiolucidi > -- Piergiorgio Lucidi Web: http://about.me/piergiorgiolucidi -- Piergiorgio Lucidi Web: http://about.me/piergiorgiolucidi 2011/8/3 Piergiorgio Lucidi <piergiorgioluc...@gmail.com> > Yes, this is a very quick way to test my configuration code for the > integration test ;) > I'm going to fix this part, I saw what I need in the JSON retrieved by a > GET call against the Manifold API service. > > I'll let you know tomorrow an update about this. > > Anyway we have a problem with the OpenCMIS InMemory server. It seems that > there is a problem during the execution of CMIS queries, I notified the > problem to the Chemistry guys (Jens Hubel and Florian Muller) and they are > trying to reproduce the issue to solve our problem. > > I'm going on developing the code using the public Alfresco CMIS server > exposed at the following address: > http://cmis.alfresco.com > > Piergiorgio > > > 2011/8/3 Karl Wright <daddy...@gmail.com> > >> Another good way to see exactly what you need to do is to call the API >> to get configuration information for an existing connection. Then, >> use the toXML() method to convert to XML, or the toJSON() to get it as >> JSON. Either way you will see the structure. BTW, ManifoldCF in >> Action Chapter 3 covers this in great detail as well. >> >> Karl >> >> On Wed, Aug 3, 2011 at 12:50 PM, Karl Wright <daddy...@gmail.com> wrote: >> > The ConfigParams class is, I believe, derived from the Configuration >> > class. So, you can create a ConfigParams object instead of a >> > Configuration object if you want to use the API in the manner you >> > describe. >> > >> > The reason your commented-out code doesn't work is because the >> > setParameter() method isn't doing quite what you are expecting. It's >> > creating a node named "_PARAMETER_" with a "name" attribute and a >> > value area, and you are creating nodes named by the parameter name. >> > >> > Karl >> > >> > On Wed, Aug 3, 2011 at 12:41 PM, Piergiorgio Lucidi >> > <piergiorgioluc...@gmail.com> wrote: >> >> I'm trying to implement tests but I found a problem to set all the >> needed >> >> parameters to the CMIS Repository Connector that needs: an username, a >> >> password and the endpoint (url). >> >> >> >> I need to know how to create the configuration nodes for the connector, >> in >> >> the connector code I managed the configuration parameters in this way >> in the >> >> processConfigurationPost method: >> >> >> >> public String processConfigurationPost(IThreadContext threadContext, >> >>> IPostParameters variableContext, ConfigParams parameters) >> >>> throws ManifoldCFException { >> >>> String username = >> variableContext.getParameter(CONFIG_PARAM_USERNAME); >> >>> if (StringUtils.isNotEmpty(username)) >> >>> parameters.setParameter(CONFIG_PARAM_USERNAME, username); >> >>> String password = >> variableContext.getParameter(CONFIG_PARAM_PASSWORD); >> >>> if (StringUtils.isNotEmpty(password)) >> >>> parameters.setParameter(CONFIG_PARAM_PASSWORD, password); >> >>> String endpoint = >> variableContext.getParameter(CONFIG_PARAM_ENDPOINT); >> >>> if (StringUtils.isNotEmpty(endpoint) && endpoint.length() > 0) >> >>> parameters.setParameter(CONFIG_PARAM_ENDPOINT, endpoint); >> >>> String repositoryId = variableContext >> >>> .getParameter(CONFIG_PARAM_REPOSITORY_ID); >> >>> if (StringUtils.isNotEmpty(repositoryId)) >> >>> parameters.setParameter(CONFIG_PARAM_REPOSITORY_ID, >> repositoryId); >> >>> return null; >> >>> } >> >> >> >> >> >> Now I have to setup the same parameters inside my test class >> APISanityTest >> >> that doesn't like the following snippet, it works only if CMIS >> parameters >> >> are commented as the following: >> >> >> >> @Test >> >>> public void sanityCheck() >> >>> throws Exception >> >>> { >> >>> try >> >>> { >> >>> // Hey, we were able to install the file system connector etc. >> >>> // Now, create a local test job and run it. >> >>> IThreadContext tc = ThreadContextFactory.make(); >> >>> int i; >> >>> IJobManager jobManager = JobManagerFactory.make(tc); >> >>> // Create a basic file system connection, and save it. >> >>> ConfigurationNode connectionObject; >> >>> ConfigurationNode child; >> >>> Configuration requestObject; >> >>> Configuration result; >> >>> >> >>> connectionObject = new >> ConfigurationNode("repositoryconnection"); >> >>> >> >>> child = new ConfigurationNode("name"); >> >>> child.setValue("CMIS Connection"); >> >>> >> connectionObject.addChild(connectionObject.getChildCount(),child); >> >>> >> >>> child = new ConfigurationNode("class_name"); >> >>> child.setValue( >> >>> >> "org.apache.manifoldcf.crawler.connectors.cmis.CmisRepositoryConnector"); >> >>> >> connectionObject.addChild(connectionObject.getChildCount(),child); >> >>> >> >>> child = new ConfigurationNode("description"); >> >>> child.setValue("CMIS Connection"); >> >>> >> connectionObject.addChild(connectionObject.getChildCount(),child); >> >>> child = new ConfigurationNode("max_connections"); >> >>> child.setValue("10"); >> >>> >> connectionObject.addChild(connectionObject.getChildCount(),child); >> >>> >> >>> //setting the CMIS specific parameters >> >>> // child = new ConfigurationNode("username"); >> >>> // child.setValue(CMIS_USERNAME); >> >>> // >> connectionObject.addChild(connectionObject.getChildCount(),child); >> >>> // >> >>> // child = new ConfigurationNode("password"); >> >>> // child.setValue(CMIS_PASSWORD); >> >>> // >> connectionObject.addChild(connectionObject.getChildCount(),child); >> >>> // >> >>> // child = new ConfigurationNode("endpoint"); >> >>> // child.setValue(CMIS_ENDPOINT_TEST_SERVER); >> >>> // >> connectionObject.addChild(connectionObject.getChildCount(),child); >> >>> requestObject = new Configuration(); >> >>> requestObject.addChild(0,connectionObject); >> >>> >> >>> result = performAPIPutOperationViaNodes( >> >>> "repositoryconnections/CMIS%20Connection",201,requestObject); >> >> >> >> >> >> How can I set the username, password and endpoint for the CMIS >> Repository >> >> Connector parameters in this test class? >> >> >> >> Thank you. >> >> >> >> Piergiorgio >> >> >> >> >> >> 2011/8/2 Karl Wright <daddy...@gmail.com> >> >> >> >>> Thanks for the status report. I hope to see your patch soon! >> >>> >> >>> Also, FWIW, once the documentation is also done I'd like to consider >> >>> solidifying the 0.3 release. It's got a lot of good stuff in it and I >> >>> think as soon as we've finished off the new CMIS connector in all >> >>> dimensions we should go ahead. Thoughts, anyone? >> >>> >> >>> Karl >> >>> >> >>> >> >>> On Tue, Aug 2, 2011 at 5:00 AM, Piergiorgio Lucidi >> >>> <piergiorgioluc...@gmail.com> wrote: >> >>> > Yesterday I started to work on end-to-end integration test for the >> CMIS >> >>> > Connector and now I have a full running OpenCMIS test server >> integrated >> >>> with >> >>> > the ManifoldCF Maven build process. >> >>> > >> >>> > Now I have to implement: >> >>> > - a setup method to create the test documents in the CMIS server >> >>> > - a null output connector using the ManifoldCF api >> >>> > - tests using the ManifoldCF api to create a mock configuration >> against >> >>> the >> >>> > test CMIS server >> >>> > >> >>> > I'll let you know when it works. >> >>> > >> >>> > Regards, >> >>> > Piergiorgio >> >>> > >> >>> > 2011/7/29 Piergiorgio Lucidi <piergiorgioluc...@gmail.com> >> >>> > >> >>> >> Hi Karl, >> >>> >> >> >>> >> thank you for the details and as soon as I finish a first version >> of >> >>> >> integration and/or unit test I will create a new ticket in the CMIS >> >>> >> Component to release the patch. >> >>> >> >> >>> >> I hope to release this new patch soon. >> >>> >> I'll let you know during the next week. >> >>> >> >> >>> >> Piergiorgio >> >>> >> >> >>> >> >> >>> >> 2011/7/28 Karl Wright <daddy...@gmail.com> >> >>> >> >> >>> >>> The tests/filesystem/src/test/java area are end-to-end tests >> >>> >>> principally designed to test the filesystem connector. (They also >> >>> >>> involve other connectors because, obviously, an end-to-end test is >> not >> >>> >>> going going to work without them. This is why they are at the >> root >> >>> >>> level.) >> >>> >>> >> >>> >>> There are also unit tests you can use as models under >> >>> >>> connectors/filesystem/connector/src/test/.... These tests use >> only >> >>> >>> the filesystem connector, no others. >> >>> >>> >> >>> >>> If you would be so kind as to create a ticket to cover your test >> work, >> >>> >>> that would be great. >> >>> >>> >> >>> >>> Also, another workitem I've been meaning to get to but haven't >> >>> >>> involves the conversion of a python-based HTML browser and >> Javascript >> >>> >>> engine I wrote for MetaCarta into a Java equivalent. This would >> allow >> >>> >>> direct testing of the UI components of the framework and >> connectors. >> >>> >>> The python sources are in: >> >>> >>> >> >>> >>> legacy-tests/VirtualBrowser.py >> >>> >>> and >> >>> >>> legacy-tests/Javascript.py >> >>> >>> >> >>> >>> You may not have time to get that far for a while, but I thought >> you'd >> >>> >>> want to know where my thoughts were headed. ;-) >> >>> >>> >> >>> >>> Karl >> >>> >>> >> >>> >>> On Thu, Jul 28, 2011 at 2:03 PM, Piergiorgio Lucidi >> >>> >>> <piergiorgioluc...@gmail.com> wrote: >> >>> >>> > I'm starting to think a way to implement tests for the CMIS >> >>> Connector, >> >>> >>> > probably I can use the Apache Chemistry (OpenCMIS) local binding >> to >> >>> >>> create a >> >>> >>> > mock of a CMIS Server (an InMemory repository), but I have to >> check >> >>> this >> >>> >>> > possibility because I'm not sure that all the features are now >> >>> >>> implemented. >> >>> >>> > >> >>> >>> > Anyway I saw some test classes for the filesystem connector at >> the >> >>> >>> following >> >>> >>> > package: >> >>> >>> > >> >>> >>> > tests/filesystem/src/test/java >> >>> >>> > >> >>> >>> > Could you please confirm that these classes could be taken as an >> >>> example >> >>> >>> for >> >>> >>> > tests? >> >>> >>> > Or do I need to take a look at other classes? >> >>> >>> > >> >>> >>> > Thank you. >> >>> >>> > Piergiorgio >> >>> >>> > >> >>> >>> > -- >> >>> >>> > Piergiorgio Lucidi >> >>> >>> > Web: http://about.me/piergiorgiolucidi >> >>> >>> > >> >>> >>> >> >>> >> >> >>> >> >> >>> >> >> >>> >> -- >> >>> >> Piergiorgio Lucidi >> >>> >> Web: http://about.me/piergiorgiolucidi >> >>> >> >> >>> >> >> >>> > >> >>> > >> >>> > -- >> >>> > Piergiorgio Lucidi >> >>> > Web: http://about.me/piergiorgiolucidi >> >>> > >> >>> >> >> >> >> >> >> >> >> -- >> >> Piergiorgio Lucidi >> >> Web: http://about.me/piergiorgiolucidi >> >> >> > >> > > > > -- > Piergiorgio Lucidi > Web: http://about.me/piergiorgiolucidi > > -- Piergiorgio Lucidi Web: http://about.me/piergiorgiolucidi
