date:20180919

[jira] [Commented] (CONNECTORS-1532) Moving a file outside of the job's Paths is not the same as deleting it

2018-09-19 Thread James Thomas (JIRA)



[ 
https://issues.apache.org/jira/browse/CONNECTORS-1532?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16621531#comment-16621531
 ] 

James Thomas commented on CONNECTORS-1532:
--

{quote}I may have found the reason you see this behavior, though. If the folder 
affinity is versioned information, and I believe it is, then the seeding query 
will pick up the last version of the document that was in the right folder. 
That's because the seeding query uses the chronicle_id, which is really a 
specific document version:
{quote}
Thanks, is this consistent with my observation that re-seeding explicitly (from 
the Job's View page) forces the change to be recognised?

Is there some logging I can provide or other experiment I can do that will help 
you to narrow this down further or confirm your thoughts?

Also, this may be an unrelated question, and I can take it to a different 
ticket if you like, but why does the File System connector zero these files 
rather than deleting them from disk?

 

> Moving a file outside of the job's Paths is not the same as deleting it
> ---
>
> Key: CONNECTORS-1532
> URL: https://issues.apache.org/jira/browse/CONNECTORS-1532
> Project: ManifoldCF
>  Issue Type: Bug
>  Components: Documentum connector
>Affects Versions: ManifoldCF 2.10
> Environment: Manifold 2.10 patched for #1512, #1517
>Reporter: James Thomas
>Assignee: Karl Wright
>Priority: Major
> Attachments: 2018-09-19_1758.png
>
>
> If I have a MF job which is connecting a specific folder, F, in Documentum to 
> a File System output then:
> 1. deleting files in Documentum shows them as zero size in the file system
> 2. moving files out of F does not remove them or zero them in the file system
> Note that moving a file from another folder (which the job is not looking at) 
> to F has the same effect as adding it to F by e.g. importing it in DM or 
> POSTing it to DM via the REST interface.
> Intuitively, I expect that moving a file out of the "view" of the Documentum 
> connector would have the same effect on the File System as deleting it. (My 
> model here is of MF synchronising content between the Paths (DM) and the 
> Output Path (File System) that I have specified in the job.)
> Starting point, I have run the MF job to fetch a bunch of files from a folder 
> - call it F - in DM (i.e. I have configured Paths in the job to be F). This 
> is what 'ls -l' on the file system looks like:
> {code:java}
> -rw-r--r--. 1 root i2e  12541 Sep 19 07:21 
> drl?versionLabel=CURRENT=09018000f7c0
> -rw-r--r--. 1 root i2e 26 Sep 19 07:21 
> drl?versionLabel=CURRENT=09018000f7be
> -rw-r--r--. 1 root i2e  85772 Sep 19 07:21 
> drl?versionLabel=CURRENT=09018000f7c7
> -rw-r--r--. 1 root i2e   8790 Sep 19 07:21 
> drl?versionLabel=CURRENT=09018000f7c2
> -rw-r--r--. 1 root i2e 101888 Sep 19 07:21 
> drl?versionLabel=CURRENT=09018000f7c3
> -rw-r--r--. 1 root i2e  32783 Sep 19 07:21 
> drl?versionLabel=CURRENT=09018000f7c4
> -rw-r--r--. 1 root i2e  23040 Sep 19 07:22 
> drl?versionLabel=CURRENT=09018000f7c1
> -rw-r--r--. 1 root i2e  26112 Sep 19 07:22 
> drl?versionLabel=CURRENT=09018000f7bf{code}
> In DM, I delete one of the files in F and it shows as zero size, and the 
> modification date has changed:
> {code:java}
> -rw-r--r--. 1 root i2e  12541 Sep 19 07:21 
> drl?versionLabel=CURRENT=09018000f7c0
> -rw-r--r--. 1 root i2e 26 Sep 19 07:21 
> drl?versionLabel=CURRENT=09018000f7be
> -rw-r--r--. 1 root i2e   8790 Sep 19 07:21 
> drl?versionLabel=CURRENT=09018000f7c2
> -rw-r--r--. 1 root i2e 101888 Sep 19 07:21 
> drl?versionLabel=CURRENT=09018000f7c3
> -rw-r--r--. 1 root i2e  32783 Sep 19 07:21 
> drl?versionLabel=CURRENT=09018000f7c4
> -rw-r--r--. 1 root i2e  23040 Sep 19 07:22 
> drl?versionLabel=CURRENT=09018000f7c1
> -rw-r--r--. 1 root i2e  26112 Sep 19 07:22 
> drl?versionLabel=CURRENT=09018000f7bf
> -rw-r--r--. 1 root i2e  0 Sep 19 07:23 
> drl?versionLabel=CURRENT=09018000f7c7{code}
> In DM, I move a file from F to another folder. (Right click, add to 
> clipboard, go to new folder, Edit> Move here). 
> The file shows as modified (07:25), but is still apparently in F (i.e. in the 
> Path my MF job is looking at):
> {code:java}
> -rw-r--r--. 1 root i2e  12541 Sep 19 07:21 
> drl?versionLabel=CURRENT=09018000f7c0
> -rw-r--r--. 1 root i2e 26 Sep 19 07:21 
> drl?versionLabel=CURRENT=09018000f7be
> -rw-r--r--. 1 root i2e   8790 Sep 19 07:21 
> drl?versionLabel=CURRENT=09018000f7c2
> -rw-r--r--. 1 root i2e 101888 Sep 19 07:21 
> drl?versionLabel=CURRENT=09018000f7c3
> -rw-r--r--. 1 root i2e  23040 Sep 19 07:22 
> drl?versionLabel=CURRENT=09018000f7c1
> -rw-r--r--. 1 root i2e  26112 Sep 19 07:22

[VOTE] Release ManifoldCF 2.11, RC1

2018-09-19 Thread Karl Wright

Please vote on whether to release ManifoldCF 2.11, RC1.  This release
contains a number of fixes/improvements/additions, described in the
CHANGES.txt file.  In addition, it includes Tika 1.19, which has a number
of fixes for classpath issues specifically requested by ManifoldCF.

This fixes a SolrJ related problem with the Solr Connector found in RC1.
All tests pass.

The release artifact can be found at:

https://dist.apache.org/repos/dist/dev/manifoldcf/apache-manifoldcf-2.11

There is also a tag at:

https://svn.apache.org/repos/asf/manifoldcf/tags/release-2.11-RC1

Thanks again,
Karl Wright

Build failed in Jenkins: ManifoldCF-ant #658

2018-09-19 Thread Apache Jenkins Server

See 

--
[...truncated 9.91 KB...]
at 
org.tmatesoft.svn.core.internal.wc.SVNErrorManager.error(SVNErrorManager.java:70)
at 
org.tmatesoft.svn.core.internal.wc.SVNErrorManager.error(SVNErrorManager.java:57)
at 
org.tmatesoft.svn.core.internal.db.SVNSqlJetDb.createSqlJetError(SVNSqlJetDb.java:195)
at 
org.tmatesoft.svn.core.internal.db.SVNSqlJetDb.beginTransaction(SVNSqlJetDb.java:211)
at 
org.tmatesoft.svn.core.internal.db.SVNSqlJetDb.runTransaction(SVNSqlJetDb.java:257)
at 
org.tmatesoft.svn.core.internal.db.SVNSqlJetDb.runTransaction(SVNSqlJetDb.java:252)
at 
org.tmatesoft.svn.core.internal.wc17.db.SVNWCDb.obtainWCLock(SVNWCDb.java:5867)
at 
org.tmatesoft.svn.core.internal.wc17.SVNWCContext.acquireWriteLock(SVNWCContext.java:1652)
at 
org.tmatesoft.svn.core.internal.wc2.ng.SvnNgAbstractUpdate.update(SvnNgAbstractUpdate.java:110)
at 
org.tmatesoft.svn.core.internal.wc2.ng.SvnNgUpdate.run(SvnNgUpdate.java:40)
at 
org.tmatesoft.svn.core.internal.wc2.ng.SvnNgUpdate.run(SvnNgUpdate.java:18)
at 
org.tmatesoft.svn.core.internal.wc2.ng.SvnNgOperationRunner.run(SvnNgOperationRunner.java:20)
at 
org.tmatesoft.svn.core.internal.wc2.SvnOperationRunner.run(SvnOperationRunner.java:21)
at 
org.tmatesoft.svn.core.wc2.SvnOperationFactory.run(SvnOperationFactory.java:1239)
at org.tmatesoft.svn.core.wc2.SvnOperation.run(SvnOperation.java:294)
at 
org.tmatesoft.svn.core.wc.SVNUpdateClient.doUpdate(SVNUpdateClient.java:311)
at 
org.tmatesoft.svn.core.wc.SVNUpdateClient.doUpdate(SVNUpdateClient.java:291)
at 
org.tmatesoft.svn.core.wc.SVNUpdateClient.doUpdate(SVNUpdateClient.java:387)
at 
hudson.scm.subversion.UpdateUpdater$TaskImpl.perform(UpdateUpdater.java:161)
at 
hudson.scm.subversion.WorkspaceUpdater$UpdateTask.delegateTo(WorkspaceUpdater.java:168)
at 
hudson.scm.SubversionSCM$CheckOutTask.perform(SubversionSCM.java:1041)
at hudson.scm.SubversionSCM$CheckOutTask.invoke(SubversionSCM.java:1017)
at hudson.scm.SubversionSCM$CheckOutTask.invoke(SubversionSCM.java:990)
at hudson.FilePath$FileCallableWrapper.call(FilePath.java:2918)
at hudson.remoting.UserRequest.perform(UserRequest.java:212)
at hudson.remoting.UserRequest.perform(UserRequest.java:54)
at hudson.remoting.Request$2.run(Request.java:369)
at 
hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.tmatesoft.sqljet.core.SqlJetException: FULL: error code is FULL
at 
org.tmatesoft.sqljet.core.internal.pager.SqlJetPager.openJournal(SqlJetPager.java:3044)
at 
org.tmatesoft.sqljet.core.internal.pager.SqlJetPager.begin(SqlJetPager.java:2775)
at 
org.tmatesoft.sqljet.core.internal.btree.SqlJetBtree.beginTrans(SqlJetBtree.java:931)
at 
org.tmatesoft.sqljet.core.table.engine.SqlJetEngine.doBeginTransaction(SqlJetEngine.java:561)
at 
org.tmatesoft.sqljet.core.table.engine.SqlJetEngine.access$100(SqlJetEngine.java:55)
at 
org.tmatesoft.sqljet.core.table.engine.SqlJetEngine$9.runSynchronized(SqlJetEngine.java:475)
at 
org.tmatesoft.sqljet.core.table.engine.SqlJetEngine.runSynchronized(SqlJetEngine.java:217)
at 
org.tmatesoft.sqljet.core.table.engine.SqlJetEngine.beginTransaction(SqlJetEngine.java:471)
at 
org.tmatesoft.svn.core.internal.db.SVNSqlJetDb.beginTransaction(SVNSqlJetDb.java:206)
... 28 more
ERROR: Subversion update failed
org.tmatesoft.sqljet.core.SqlJetException: FULL: error code is FULL
at 
org.tmatesoft.sqljet.core.internal.pager.SqlJetPager.openJournal(SqlJetPager.java:3044)
at 
org.tmatesoft.sqljet.core.internal.pager.SqlJetPager.begin(SqlJetPager.java:2775)
at 
org.tmatesoft.sqljet.core.internal.btree.SqlJetBtree.beginTrans(SqlJetBtree.java:931)
at 
org.tmatesoft.sqljet.core.table.engine.SqlJetEngine.doBeginTransaction(SqlJetEngine.java:561)
at 
org.tmatesoft.sqljet.core.table.engine.SqlJetEngine.access$100(SqlJetEngine.java:55)
at 
org.tmatesoft.sqljet.core.table.engine.SqlJetEngine$9.runSynchronized(SqlJetEngine.java:475)
at 
org.tmatesoft.sqljet.core.table.engine.SqlJetEngine.runSynchronized(SqlJetEngine.java:217)
at 
org.tmatesoft.sqljet.core.table.engine.SqlJetEngine.beginTransaction(SqlJetEngine.java:471)
at 
org.tmatesoft.svn.core.internal.db.SVNSqlJetDb.beginTransaction(SVNSqlJetDb.java:206)
Caused: org.tmatesoft.svn.core.SVNException:

[CANCEL][VOTE] Release Apache ManifoldCF 2.11, RC0

2018-09-19 Thread Karl Wright

Tests fail; specifically, Solr tests:

FATAL 2018-09-19T17:36:09,580 (Document delete thread '4') - Error tossed:
This Should not happen
java.lang.RuntimeException: This Should not happen
at
org.apache.solr.client.solrj.impl.BinaryRequestWriter.getContentStreams(BinaryRequestWriter.java:67)
~[solr-solrj-7.4.0.jar:7.4.0 9060ac689c270b02143f375de0348b7f626adebc -
jpountz - 2018-06-18 16:55:14]
at
org.apache.manifoldcf.agents.output.solr.ModifiedHttpSolrClient.createMethod(ModifiedHttpSolrClient.java:109)
~[classes/:?]
at
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:253)
~[solr-solrj-7.4.0.jar:7.4.0 9060ac689c270b02143f375de0348b7f626adebc -
jpountz - 2018-06-18 16:55:14]
at
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
~[solr-solrj-7.4.0.jar:7.4.0 9060ac689c270b02143f375de0348b7f626adebc -
jpountz - 2018-06-18 16:55:14]
at
org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
~[solr-solrj-7.4.0.jar:7.4.0 9060ac689c270b02143f375de0348b7f626adebc -
jpountz - 2018-06-18 16:55:14]
at
org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:211)
~[solr-solrj-7.4.0.jar:7.4.0 9060ac689c270b02143f375de0348b7f626adebc -
jpountz - 2018-06-18 16:55:14]
at
org.apache.manifoldcf.agents.output.solr.HttpPoster$DeleteThread.run(HttpPoster.java:1364)
~[classes/:?]

We did update to the latest SolrJ 7.4 release.  It looks like something is
broken there.
Karl

On Wed, Sep 19, 2018 at 10:58 AM Karl Wright  wrote:

> Please vote on whether to release ManifoldCF 2.11, RC0.  This release
> contains a number of fixes/improvements/additions, described in the
> CHANGES.txt file.  In addition, it includes Tika 1.19, which has a number
> of fixes for classpath issues specifically requested by ManifoldCF.
>
> The release artifact can be found at:
>
> https://dist.apache.org/repos/dist/dev/manifoldcf/apache-manifoldcf-2.11
>
> There is also a tag at:
>
> https://svn.apache.org/repos/asf/manifoldcf/tags/release-2.11-RC0
>
> Thanks again,
> Karl Wright
>
>

[jira] [Commented] (CONNECTORS-1532) Moving a file outside of the job's Paths is not the same as deleting it

2018-09-19 Thread Karl Wright (JIRA)



[ 
https://issues.apache.org/jira/browse/CONNECTORS-1532?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16621019#comment-16621019
 ] 

Karl Wright commented on CONNECTORS-1532:
-

As I suspected, there is no code difference in the framework between MODEL_ADD 
and MODEL_ADD_CHANGE:

{code}
./build/crawler-ui/java/org/apache/jsp/editjob_jsp.java:  int model = 
IRepositoryConnector.MODEL_ADD_CHANGE_DELETE;
./build/crawler-ui/java/org/apache/jsp/editjob_jsp.java:if (model != -1 && 
model != IRepositoryConnector.MODEL_ADD_CHANGE_DELETE && model != 
IRepositoryConnector.MODEL_CHAINED_ADD_CHANGE_DELETE)
./build/crawler-ui/java/org/apache/jsp/viewjob_jsp.java:if (model != -1 && 
model != IRepositoryConnector.MODEL_ADD_CHANGE_DELETE)
./pull-agent/src/main/java/org/apache/manifoldcf/crawler/interfaces/IRepositoryConnector.java:*
 is the most restrictive that is still accurate.  For example, if 
MODEL_ADD_CHANGE_DELETE applies, you would
./pull-agent/src/main/java/org/apache/manifoldcf/crawler/interfaces/IRepositoryConnector.java:*
 return that value rather than MODEL_ADD.
./pull-agent/src/main/java/org/apache/manifoldcf/crawler/interfaces/IRepositoryConnector.java:
  public static final int
MODEL_ADD = 1;
./pull-agent/src/main/java/org/apache/manifoldcf/crawler/interfaces/IRepositoryConnector.java:
  public static final int
MODEL_ADD_CHANGE = 2;
./pull-agent/src/main/java/org/apache/manifoldcf/crawler/interfaces/IRepositoryConnector.java:
  public static final int
MODEL_ADD_CHANGE_DELETE = 3;
./pull-agent/src/main/java/org/apache/manifoldcf/crawler/interfaces/IRepositoryConnector.java:
  /** Like MODEL_ADD, except considering document discovery */
./pull-agent/src/main/java/org/apache/manifoldcf/crawler/interfaces/IRepositoryConnector.java:
  /** Like MODEL_ADD_CHANGE, except considering document discovery */
./pull-agent/src/main/java/org/apache/manifoldcf/crawler/interfaces/IRepositoryConnector.java:
  /** Like MODEL_ADD_CHANGE_DELETE, except considering document discovery */
./pull-agent/src/main/java/org/apache/manifoldcf/crawler/jobs/JobManager.java:  
  // (1) If the connector has MODEL_ADD_CHANGE_DELETE, then
./pull-agent/src/main/java/org/apache/manifoldcf/crawler/jobs/JobManager.java:  
  if (connectorModel == IRepositoryConnector.MODEL_ADD_CHANGE_DELETE)
{code}

I may have found the reason you see this behavior, though.  If the folder 
affinity is versioned information, and I believe it is, then the seeding query 
will pick up the last version of the document that was in the right folder.  
That's because the seeding query uses the chronicle_id, which is really a 
specific document version:

{code}
  String strDQLstart = "select for READ distinct i_chronicle_id from ";
{code}

I wouldn't know the DQL for checking to be sure that the particular version of 
the document was the last one, unfortunately.


> Moving a file outside of the job's Paths is not the same as deleting it
> ---
>
> Key: CONNECTORS-1532
> URL: https://issues.apache.org/jira/browse/CONNECTORS-1532
> Project: ManifoldCF
>  Issue Type: Bug
>  Components: Documentum connector
>Affects Versions: ManifoldCF 2.10
> Environment: Manifold 2.10 patched for #1512, #1517
>Reporter: James Thomas
>Assignee: Karl Wright
>Priority: Major
> Attachments: 2018-09-19_1758.png
>
>
> If I have a MF job which is connecting a specific folder, F, in Documentum to 
> a File System output then:
> 1. deleting files in Documentum shows them as zero size in the file system
> 2. moving files out of F does not remove them or zero them in the file system
> Note that moving a file from another folder (which the job is not looking at) 
> to F has the same effect as adding it to F by e.g. importing it in DM or 
> POSTing it to DM via the REST interface.
> Intuitively, I expect that moving a file out of the "view" of the Documentum 
> connector would have the same effect on the File System as deleting it. (My 
> model here is of MF synchronising content between the Paths (DM) and the 
> Output Path (File System) that I have specified in the job.)
> Starting point, I have run the MF job to fetch a bunch of files from a folder 
> - call it F - in DM (i.e. I have configured Paths in the job to be F). This 
> is what 'ls -l' on the file system looks like:
> {code:java}
> -rw-r--r--. 1 root i2e  12541 Sep 19 07:21 
> drl?versionLabel=CURRENT=09018000f7c0
> -rw-r--r--. 1 root i2e 26 Sep 19 07:21 
> drl?versionLabel=CURRENT=09018000f7be
> -rw-r--r--. 1 root i2e  85772 Sep 19 07:21 
> drl?versionLabel=CURRENT=09018000f7c7
> -rw-r--r--. 1 root i2e   8790 Sep 19 07:21 
> drl?versionLabel=CURRENT=09018000f7c2
> -rw-r--r--. 1 root i2e 101888 Sep 19 07:21 
> drl?versionLabel=CURRENT=09018000f7c3
>

[jira] [Commented] (CONNECTORS-1532) Moving a file outside of the job's Paths is not the same as deleting it

2018-09-19 Thread Karl Wright (JIRA)



[ 
https://issues.apache.org/jira/browse/CONNECTORS-1532?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16621006#comment-16621006
 ] 

Karl Wright commented on CONNECTORS-1532:
-

Whenever you click "Start" on such jobs, seeding is done.  Documents are 
located based on their folder affinity and added to the queue.  The documents 
that were added are then processed.  At the end of the job, when document 
processing is complete, documents that were not discovered at seeding time are 
removed from the index.

There are numerous integration tests in ManifoldCF that verify this basic 
sequence.  The Documentum connector has almost nothing to do with this process 
unless your operation of "moving the document out of the folder" is not in fact 
removing the document from whatever folder structure metadata Documentum 
maintains, and the document is still discoverable with the same DQL query as 
before.  The final deletion is a function of the framework, not of any of the 
connectors.

The only difference I know of is that the Documentum Connector uses MODEL_ADD, 
which is not very common in connectors, while the tests use connectors that use 
MODEL_ADD_CHANGE.  This should not have any effect on the cleanup cycle but I 
can trivially confirm that that is true.

If I cannot find any difference, the next step would be for me to modify the 
FileSystem connector (used in the tests extensively) to use MODEL_ADD instead.  
Then I can see if the tests still pass.  If they do, then I'd be forced to 
conclude that you aren't changing the Documentum folder in the way you think 
you are.


> Moving a file outside of the job's Paths is not the same as deleting it
> ---
>
> Key: CONNECTORS-1532
> URL: https://issues.apache.org/jira/browse/CONNECTORS-1532
> Project: ManifoldCF
>  Issue Type: Bug
>  Components: Documentum connector
>Affects Versions: ManifoldCF 2.10
> Environment: Manifold 2.10 patched for #1512, #1517
>Reporter: James Thomas
>Assignee: Karl Wright
>Priority: Major
> Attachments: 2018-09-19_1758.png
>
>
> If I have a MF job which is connecting a specific folder, F, in Documentum to 
> a File System output then:
> 1. deleting files in Documentum shows them as zero size in the file system
> 2. moving files out of F does not remove them or zero them in the file system
> Note that moving a file from another folder (which the job is not looking at) 
> to F has the same effect as adding it to F by e.g. importing it in DM or 
> POSTing it to DM via the REST interface.
> Intuitively, I expect that moving a file out of the "view" of the Documentum 
> connector would have the same effect on the File System as deleting it. (My 
> model here is of MF synchronising content between the Paths (DM) and the 
> Output Path (File System) that I have specified in the job.)
> Starting point, I have run the MF job to fetch a bunch of files from a folder 
> - call it F - in DM (i.e. I have configured Paths in the job to be F). This 
> is what 'ls -l' on the file system looks like:
> {code:java}
> -rw-r--r--. 1 root i2e  12541 Sep 19 07:21 
> drl?versionLabel=CURRENT=09018000f7c0
> -rw-r--r--. 1 root i2e 26 Sep 19 07:21 
> drl?versionLabel=CURRENT=09018000f7be
> -rw-r--r--. 1 root i2e  85772 Sep 19 07:21 
> drl?versionLabel=CURRENT=09018000f7c7
> -rw-r--r--. 1 root i2e   8790 Sep 19 07:21 
> drl?versionLabel=CURRENT=09018000f7c2
> -rw-r--r--. 1 root i2e 101888 Sep 19 07:21 
> drl?versionLabel=CURRENT=09018000f7c3
> -rw-r--r--. 1 root i2e  32783 Sep 19 07:21 
> drl?versionLabel=CURRENT=09018000f7c4
> -rw-r--r--. 1 root i2e  23040 Sep 19 07:22 
> drl?versionLabel=CURRENT=09018000f7c1
> -rw-r--r--. 1 root i2e  26112 Sep 19 07:22 
> drl?versionLabel=CURRENT=09018000f7bf{code}
> In DM, I delete one of the files in F and it shows as zero size, and the 
> modification date has changed:
> {code:java}
> -rw-r--r--. 1 root i2e  12541 Sep 19 07:21 
> drl?versionLabel=CURRENT=09018000f7c0
> -rw-r--r--. 1 root i2e 26 Sep 19 07:21 
> drl?versionLabel=CURRENT=09018000f7be
> -rw-r--r--. 1 root i2e   8790 Sep 19 07:21 
> drl?versionLabel=CURRENT=09018000f7c2
> -rw-r--r--. 1 root i2e 101888 Sep 19 07:21 
> drl?versionLabel=CURRENT=09018000f7c3
> -rw-r--r--. 1 root i2e  32783 Sep 19 07:21 
> drl?versionLabel=CURRENT=09018000f7c4
> -rw-r--r--. 1 root i2e  23040 Sep 19 07:22 
> drl?versionLabel=CURRENT=09018000f7c1
> -rw-r--r--. 1 root i2e  26112 Sep 19 07:22 
> drl?versionLabel=CURRENT=09018000f7bf
> -rw-r--r--. 1 root i2e  0 Sep 19 07:23 
> drl?versionLabel=CURRENT=09018000f7c7{code}
> In DM, I move a file from F to another folder. (Right click, add to 
> clipboard, go to new folder, Edit> Move here). 
> The file shows as

[jira] [Commented] (CONNECTORS-1532) Moving a file outside of the job's Paths is not the same as deleting it

2018-09-19 Thread James Thomas (JIRA)



[ 
https://issues.apache.org/jira/browse/CONNECTORS-1532?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16620854#comment-16620854
 ] 

James Thomas commented on CONNECTORS-1532:
--

It's a "scan every document once" job, and I'm running by clicking Start.

!2018-09-19_1758.png!

> Moving a file outside of the job's Paths is not the same as deleting it
> ---
>
> Key: CONNECTORS-1532
> URL: https://issues.apache.org/jira/browse/CONNECTORS-1532
> Project: ManifoldCF
>  Issue Type: Bug
>  Components: Documentum connector
>Affects Versions: ManifoldCF 2.10
> Environment: Manifold 2.10 patched for #1512, #1517
>Reporter: James Thomas
>Assignee: Karl Wright
>Priority: Major
> Attachments: 2018-09-19_1758.png
>
>
> If I have a MF job which is connecting a specific folder, F, in Documentum to 
> a File System output then:
> 1. deleting files in Documentum shows them as zero size in the file system
> 2. moving files out of F does not remove them or zero them in the file system
> Note that moving a file from another folder (which the job is not looking at) 
> to F has the same effect as adding it to F by e.g. importing it in DM or 
> POSTing it to DM via the REST interface.
> Intuitively, I expect that moving a file out of the "view" of the Documentum 
> connector would have the same effect on the File System as deleting it. (My 
> model here is of MF synchronising content between the Paths (DM) and the 
> Output Path (File System) that I have specified in the job.)
> Starting point, I have run the MF job to fetch a bunch of files from a folder 
> - call it F - in DM (i.e. I have configured Paths in the job to be F). This 
> is what 'ls -l' on the file system looks like:
> {code:java}
> -rw-r--r--. 1 root i2e  12541 Sep 19 07:21 
> drl?versionLabel=CURRENT=09018000f7c0
> -rw-r--r--. 1 root i2e 26 Sep 19 07:21 
> drl?versionLabel=CURRENT=09018000f7be
> -rw-r--r--. 1 root i2e  85772 Sep 19 07:21 
> drl?versionLabel=CURRENT=09018000f7c7
> -rw-r--r--. 1 root i2e   8790 Sep 19 07:21 
> drl?versionLabel=CURRENT=09018000f7c2
> -rw-r--r--. 1 root i2e 101888 Sep 19 07:21 
> drl?versionLabel=CURRENT=09018000f7c3
> -rw-r--r--. 1 root i2e  32783 Sep 19 07:21 
> drl?versionLabel=CURRENT=09018000f7c4
> -rw-r--r--. 1 root i2e  23040 Sep 19 07:22 
> drl?versionLabel=CURRENT=09018000f7c1
> -rw-r--r--. 1 root i2e  26112 Sep 19 07:22 
> drl?versionLabel=CURRENT=09018000f7bf{code}
> In DM, I delete one of the files in F and it shows as zero size, and the 
> modification date has changed:
> {code:java}
> -rw-r--r--. 1 root i2e  12541 Sep 19 07:21 
> drl?versionLabel=CURRENT=09018000f7c0
> -rw-r--r--. 1 root i2e 26 Sep 19 07:21 
> drl?versionLabel=CURRENT=09018000f7be
> -rw-r--r--. 1 root i2e   8790 Sep 19 07:21 
> drl?versionLabel=CURRENT=09018000f7c2
> -rw-r--r--. 1 root i2e 101888 Sep 19 07:21 
> drl?versionLabel=CURRENT=09018000f7c3
> -rw-r--r--. 1 root i2e  32783 Sep 19 07:21 
> drl?versionLabel=CURRENT=09018000f7c4
> -rw-r--r--. 1 root i2e  23040 Sep 19 07:22 
> drl?versionLabel=CURRENT=09018000f7c1
> -rw-r--r--. 1 root i2e  26112 Sep 19 07:22 
> drl?versionLabel=CURRENT=09018000f7bf
> -rw-r--r--. 1 root i2e  0 Sep 19 07:23 
> drl?versionLabel=CURRENT=09018000f7c7{code}
> In DM, I move a file from F to another folder. (Right click, add to 
> clipboard, go to new folder, Edit> Move here). 
> The file shows as modified (07:25), but is still apparently in F (i.e. in the 
> Path my MF job is looking at):
> {code:java}
> -rw-r--r--. 1 root i2e  12541 Sep 19 07:21 
> drl?versionLabel=CURRENT=09018000f7c0
> -rw-r--r--. 1 root i2e 26 Sep 19 07:21 
> drl?versionLabel=CURRENT=09018000f7be
> -rw-r--r--. 1 root i2e   8790 Sep 19 07:21 
> drl?versionLabel=CURRENT=09018000f7c2
> -rw-r--r--. 1 root i2e 101888 Sep 19 07:21 
> drl?versionLabel=CURRENT=09018000f7c3
> -rw-r--r--. 1 root i2e  23040 Sep 19 07:22 
> drl?versionLabel=CURRENT=09018000f7c1
> -rw-r--r--. 1 root i2e  26112 Sep 19 07:22 
> drl?versionLabel=CURRENT=09018000f7bf
> -rw-r--r--. 1 root i2e  0 Sep 19 07:23 
> drl?versionLabel=CURRENT=09018000f7c7
> -rw-r--r--. 1 root i2e  32783 Sep 19 07:25 
> drl?versionLabel=CURRENT=09018000f7c4{code}
> In DM, I move a file from another folder to F and it shows up with the 
> timestamp of the move (07:28):
> {code:java}
> -rw-r--r--. 1 root i2e  12541 Sep 19 07:21 
> drl?versionLabel=CURRENT=09018000f7c0
> -rw-r--r--. 1 root i2e 26 Sep 19 07:21 
> drl?versionLabel=CURRENT=09018000f7be
> -rw-r--r--. 1 root i2e   8790 Sep 19 07:21 
> drl?versionLabel=CURRENT=09018000f7c2
> -rw-r--r--. 1 root i2e 101888 Sep 19 07:21 
> drl?versionLabel=CURRENT=09018000f7c3
>

[jira] [Updated] (CONNECTORS-1532) Moving a file outside of the job's Paths is not the same as deleting it

2018-09-19 Thread James Thomas (JIRA)



 [ 
https://issues.apache.org/jira/browse/CONNECTORS-1532?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

James Thomas updated CONNECTORS-1532:
-
Attachment: 2018-09-19_1758.png

> Moving a file outside of the job's Paths is not the same as deleting it
> ---
>
> Key: CONNECTORS-1532
> URL: https://issues.apache.org/jira/browse/CONNECTORS-1532
> Project: ManifoldCF
>  Issue Type: Bug
>  Components: Documentum connector
>Affects Versions: ManifoldCF 2.10
> Environment: Manifold 2.10 patched for #1512, #1517
>Reporter: James Thomas
>Assignee: Karl Wright
>Priority: Major
> Attachments: 2018-09-19_1758.png
>
>
> If I have a MF job which is connecting a specific folder, F, in Documentum to 
> a File System output then:
> 1. deleting files in Documentum shows them as zero size in the file system
> 2. moving files out of F does not remove them or zero them in the file system
> Note that moving a file from another folder (which the job is not looking at) 
> to F has the same effect as adding it to F by e.g. importing it in DM or 
> POSTing it to DM via the REST interface.
> Intuitively, I expect that moving a file out of the "view" of the Documentum 
> connector would have the same effect on the File System as deleting it. (My 
> model here is of MF synchronising content between the Paths (DM) and the 
> Output Path (File System) that I have specified in the job.)
> Starting point, I have run the MF job to fetch a bunch of files from a folder 
> - call it F - in DM (i.e. I have configured Paths in the job to be F). This 
> is what 'ls -l' on the file system looks like:
> {code:java}
> -rw-r--r--. 1 root i2e  12541 Sep 19 07:21 
> drl?versionLabel=CURRENT=09018000f7c0
> -rw-r--r--. 1 root i2e 26 Sep 19 07:21 
> drl?versionLabel=CURRENT=09018000f7be
> -rw-r--r--. 1 root i2e  85772 Sep 19 07:21 
> drl?versionLabel=CURRENT=09018000f7c7
> -rw-r--r--. 1 root i2e   8790 Sep 19 07:21 
> drl?versionLabel=CURRENT=09018000f7c2
> -rw-r--r--. 1 root i2e 101888 Sep 19 07:21 
> drl?versionLabel=CURRENT=09018000f7c3
> -rw-r--r--. 1 root i2e  32783 Sep 19 07:21 
> drl?versionLabel=CURRENT=09018000f7c4
> -rw-r--r--. 1 root i2e  23040 Sep 19 07:22 
> drl?versionLabel=CURRENT=09018000f7c1
> -rw-r--r--. 1 root i2e  26112 Sep 19 07:22 
> drl?versionLabel=CURRENT=09018000f7bf{code}
> In DM, I delete one of the files in F and it shows as zero size, and the 
> modification date has changed:
> {code:java}
> -rw-r--r--. 1 root i2e  12541 Sep 19 07:21 
> drl?versionLabel=CURRENT=09018000f7c0
> -rw-r--r--. 1 root i2e 26 Sep 19 07:21 
> drl?versionLabel=CURRENT=09018000f7be
> -rw-r--r--. 1 root i2e   8790 Sep 19 07:21 
> drl?versionLabel=CURRENT=09018000f7c2
> -rw-r--r--. 1 root i2e 101888 Sep 19 07:21 
> drl?versionLabel=CURRENT=09018000f7c3
> -rw-r--r--. 1 root i2e  32783 Sep 19 07:21 
> drl?versionLabel=CURRENT=09018000f7c4
> -rw-r--r--. 1 root i2e  23040 Sep 19 07:22 
> drl?versionLabel=CURRENT=09018000f7c1
> -rw-r--r--. 1 root i2e  26112 Sep 19 07:22 
> drl?versionLabel=CURRENT=09018000f7bf
> -rw-r--r--. 1 root i2e  0 Sep 19 07:23 
> drl?versionLabel=CURRENT=09018000f7c7{code}
> In DM, I move a file from F to another folder. (Right click, add to 
> clipboard, go to new folder, Edit> Move here). 
> The file shows as modified (07:25), but is still apparently in F (i.e. in the 
> Path my MF job is looking at):
> {code:java}
> -rw-r--r--. 1 root i2e  12541 Sep 19 07:21 
> drl?versionLabel=CURRENT=09018000f7c0
> -rw-r--r--. 1 root i2e 26 Sep 19 07:21 
> drl?versionLabel=CURRENT=09018000f7be
> -rw-r--r--. 1 root i2e   8790 Sep 19 07:21 
> drl?versionLabel=CURRENT=09018000f7c2
> -rw-r--r--. 1 root i2e 101888 Sep 19 07:21 
> drl?versionLabel=CURRENT=09018000f7c3
> -rw-r--r--. 1 root i2e  23040 Sep 19 07:22 
> drl?versionLabel=CURRENT=09018000f7c1
> -rw-r--r--. 1 root i2e  26112 Sep 19 07:22 
> drl?versionLabel=CURRENT=09018000f7bf
> -rw-r--r--. 1 root i2e  0 Sep 19 07:23 
> drl?versionLabel=CURRENT=09018000f7c7
> -rw-r--r--. 1 root i2e  32783 Sep 19 07:25 
> drl?versionLabel=CURRENT=09018000f7c4{code}
> In DM, I move a file from another folder to F and it shows up with the 
> timestamp of the move (07:28):
> {code:java}
> -rw-r--r--. 1 root i2e  12541 Sep 19 07:21 
> drl?versionLabel=CURRENT=09018000f7c0
> -rw-r--r--. 1 root i2e 26 Sep 19 07:21 
> drl?versionLabel=CURRENT=09018000f7be
> -rw-r--r--. 1 root i2e   8790 Sep 19 07:21 
> drl?versionLabel=CURRENT=09018000f7c2
> -rw-r--r--. 1 root i2e 101888 Sep 19 07:21 
> drl?versionLabel=CURRENT=09018000f7c3
> -rw-r--r--. 1 root i2e  23040 Sep 19 07:22 
> drl?versionLabel=CURRENT=09018000f7c1
> -rw-r--r--. 1 root

[jira] [Commented] (CONNECTORS-1532) Moving a file outside of the job's Paths is not the same as deleting it

2018-09-19 Thread James Thomas (JIRA)



[ 
https://issues.apache.org/jira/browse/CONNECTORS-1532?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16620830#comment-16620830
 ] 

James Thomas commented on CONNECTORS-1532:
--

Having not touched anything since I reported the cut-down experiment, I re-ran 
the job (I am not sure this is what you mean by a complete job?)

The File System view did not change, and the Simple History Report has a new 
entry:
{code:java}
-rw-r--r--. 1 root i2e 26 Sep 19 17:46 
drl?versionLabel=CURRENT=09018000f99a{code}
 
||Start Time||Activity||Identifier||Result Code||Bytes||Time||Result 
Description||
|09-19-2018 16:30:47.179|fetch|09018000f99a|OK|26|26| |

I then reset seeding on the job and re-ran it.

There is no additional History, but this time the file is zeroed:

 
{code:java}
-rw-r--r--. 1 root i2e 0 Sep 19 18:36 
drl?versionLabel=CURRENT=09018000f99a{code}
 

 

So, from my empirical data, I think that a move in DM requires reseeding before 
the output is synchronised with the repository? Does that fit with what you 
were saying?

You mentioned that continuous jobs would not delete - do you mean that they 
won't delete until the job ends, or until the next reseed, if a reseed interval 
is specified?

This may be an unrelated question, and I can take it to a different ticket if 
you like, but why does the File System connector zero files rather than 
deleting them from disk?

 

 

 

> Moving a file outside of the job's Paths is not the same as deleting it
> ---
>
> Key: CONNECTORS-1532
> URL: https://issues.apache.org/jira/browse/CONNECTORS-1532
> Project: ManifoldCF
>  Issue Type: Bug
>  Components: Documentum connector
>Affects Versions: ManifoldCF 2.10
> Environment: Manifold 2.10 patched for #1512, #1517
>Reporter: James Thomas
>Assignee: Karl Wright
>Priority: Major
>
> If I have a MF job which is connecting a specific folder, F, in Documentum to 
> a File System output then:
> 1. deleting files in Documentum shows them as zero size in the file system
> 2. moving files out of F does not remove them or zero them in the file system
> Note that moving a file from another folder (which the job is not looking at) 
> to F has the same effect as adding it to F by e.g. importing it in DM or 
> POSTing it to DM via the REST interface.
> Intuitively, I expect that moving a file out of the "view" of the Documentum 
> connector would have the same effect on the File System as deleting it. (My 
> model here is of MF synchronising content between the Paths (DM) and the 
> Output Path (File System) that I have specified in the job.)
> Starting point, I have run the MF job to fetch a bunch of files from a folder 
> - call it F - in DM (i.e. I have configured Paths in the job to be F). This 
> is what 'ls -l' on the file system looks like:
> {code:java}
> -rw-r--r--. 1 root i2e  12541 Sep 19 07:21 
> drl?versionLabel=CURRENT=09018000f7c0
> -rw-r--r--. 1 root i2e 26 Sep 19 07:21 
> drl?versionLabel=CURRENT=09018000f7be
> -rw-r--r--. 1 root i2e  85772 Sep 19 07:21 
> drl?versionLabel=CURRENT=09018000f7c7
> -rw-r--r--. 1 root i2e   8790 Sep 19 07:21 
> drl?versionLabel=CURRENT=09018000f7c2
> -rw-r--r--. 1 root i2e 101888 Sep 19 07:21 
> drl?versionLabel=CURRENT=09018000f7c3
> -rw-r--r--. 1 root i2e  32783 Sep 19 07:21 
> drl?versionLabel=CURRENT=09018000f7c4
> -rw-r--r--. 1 root i2e  23040 Sep 19 07:22 
> drl?versionLabel=CURRENT=09018000f7c1
> -rw-r--r--. 1 root i2e  26112 Sep 19 07:22 
> drl?versionLabel=CURRENT=09018000f7bf{code}
> In DM, I delete one of the files in F and it shows as zero size, and the 
> modification date has changed:
> {code:java}
> -rw-r--r--. 1 root i2e  12541 Sep 19 07:21 
> drl?versionLabel=CURRENT=09018000f7c0
> -rw-r--r--. 1 root i2e 26 Sep 19 07:21 
> drl?versionLabel=CURRENT=09018000f7be
> -rw-r--r--. 1 root i2e   8790 Sep 19 07:21 
> drl?versionLabel=CURRENT=09018000f7c2
> -rw-r--r--. 1 root i2e 101888 Sep 19 07:21 
> drl?versionLabel=CURRENT=09018000f7c3
> -rw-r--r--. 1 root i2e  32783 Sep 19 07:21 
> drl?versionLabel=CURRENT=09018000f7c4
> -rw-r--r--. 1 root i2e  23040 Sep 19 07:22 
> drl?versionLabel=CURRENT=09018000f7c1
> -rw-r--r--. 1 root i2e  26112 Sep 19 07:22 
> drl?versionLabel=CURRENT=09018000f7bf
> -rw-r--r--. 1 root i2e  0 Sep 19 07:23 
> drl?versionLabel=CURRENT=09018000f7c7{code}
> In DM, I move a file from F to another folder. (Right click, add to 
> clipboard, go to new folder, Edit> Move here). 
> The file shows as modified (07:25), but is still apparently in F (i.e. in the 
> Path my MF job is looking at):
> {code:java}
> -rw-r--r--. 1 root i2e  12541 Sep 19 07:21 
> drl?versionLabel=CURRENT=09018000f7c0
> -rw-r--r--. 1 root i2e 26 Sep 19

[jira] [Commented] (CONNECTORS-1532) Moving a file outside of the job's Paths is not the same as deleting it

2018-09-19 Thread Karl Wright (JIRA)



[ 
https://issues.apache.org/jira/browse/CONNECTORS-1532?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16620795#comment-16620795
 ] 

Karl Wright commented on CONNECTORS-1532:
-

Ok, I've reviewed the documentum connector code.

Folder specifications for a job are used only during seeding.  Once a document 
has been discovered (at seeding time), it will remain in the queue until it is 
unable to be fetched, at which point it is removed.  Since the folder 
specification is not used during processing, documents cannot be removed merely 
because their folder changed.

The document *will* be removed when a complete job run is done (not a 
continuous job, and not a "minimal" run), because the document will be noted as 
not having been seeded or discovered and then, at the end of the run, it will 
be deleted.


> Moving a file outside of the job's Paths is not the same as deleting it
> ---
>
> Key: CONNECTORS-1532
> URL: https://issues.apache.org/jira/browse/CONNECTORS-1532
> Project: ManifoldCF
>  Issue Type: Bug
>  Components: Documentum connector
>Affects Versions: ManifoldCF 2.10
> Environment: Manifold 2.10 patched for #1512, #1517
>Reporter: James Thomas
>Assignee: Karl Wright
>Priority: Major
>
> If I have a MF job which is connecting a specific folder, F, in Documentum to 
> a File System output then:
> 1. deleting files in Documentum shows them as zero size in the file system
> 2. moving files out of F does not remove them or zero them in the file system
> Note that moving a file from another folder (which the job is not looking at) 
> to F has the same effect as adding it to F by e.g. importing it in DM or 
> POSTing it to DM via the REST interface.
> Intuitively, I expect that moving a file out of the "view" of the Documentum 
> connector would have the same effect on the File System as deleting it. (My 
> model here is of MF synchronising content between the Paths (DM) and the 
> Output Path (File System) that I have specified in the job.)
> Starting point, I have run the MF job to fetch a bunch of files from a folder 
> - call it F - in DM (i.e. I have configured Paths in the job to be F). This 
> is what 'ls -l' on the file system looks like:
> {code:java}
> -rw-r--r--. 1 root i2e  12541 Sep 19 07:21 
> drl?versionLabel=CURRENT=09018000f7c0
> -rw-r--r--. 1 root i2e 26 Sep 19 07:21 
> drl?versionLabel=CURRENT=09018000f7be
> -rw-r--r--. 1 root i2e  85772 Sep 19 07:21 
> drl?versionLabel=CURRENT=09018000f7c7
> -rw-r--r--. 1 root i2e   8790 Sep 19 07:21 
> drl?versionLabel=CURRENT=09018000f7c2
> -rw-r--r--. 1 root i2e 101888 Sep 19 07:21 
> drl?versionLabel=CURRENT=09018000f7c3
> -rw-r--r--. 1 root i2e  32783 Sep 19 07:21 
> drl?versionLabel=CURRENT=09018000f7c4
> -rw-r--r--. 1 root i2e  23040 Sep 19 07:22 
> drl?versionLabel=CURRENT=09018000f7c1
> -rw-r--r--. 1 root i2e  26112 Sep 19 07:22 
> drl?versionLabel=CURRENT=09018000f7bf{code}
> In DM, I delete one of the files in F and it shows as zero size, and the 
> modification date has changed:
> {code:java}
> -rw-r--r--. 1 root i2e  12541 Sep 19 07:21 
> drl?versionLabel=CURRENT=09018000f7c0
> -rw-r--r--. 1 root i2e 26 Sep 19 07:21 
> drl?versionLabel=CURRENT=09018000f7be
> -rw-r--r--. 1 root i2e   8790 Sep 19 07:21 
> drl?versionLabel=CURRENT=09018000f7c2
> -rw-r--r--. 1 root i2e 101888 Sep 19 07:21 
> drl?versionLabel=CURRENT=09018000f7c3
> -rw-r--r--. 1 root i2e  32783 Sep 19 07:21 
> drl?versionLabel=CURRENT=09018000f7c4
> -rw-r--r--. 1 root i2e  23040 Sep 19 07:22 
> drl?versionLabel=CURRENT=09018000f7c1
> -rw-r--r--. 1 root i2e  26112 Sep 19 07:22 
> drl?versionLabel=CURRENT=09018000f7bf
> -rw-r--r--. 1 root i2e  0 Sep 19 07:23 
> drl?versionLabel=CURRENT=09018000f7c7{code}
> In DM, I move a file from F to another folder. (Right click, add to 
> clipboard, go to new folder, Edit> Move here). 
> The file shows as modified (07:25), but is still apparently in F (i.e. in the 
> Path my MF job is looking at):
> {code:java}
> -rw-r--r--. 1 root i2e  12541 Sep 19 07:21 
> drl?versionLabel=CURRENT=09018000f7c0
> -rw-r--r--. 1 root i2e 26 Sep 19 07:21 
> drl?versionLabel=CURRENT=09018000f7be
> -rw-r--r--. 1 root i2e   8790 Sep 19 07:21 
> drl?versionLabel=CURRENT=09018000f7c2
> -rw-r--r--. 1 root i2e 101888 Sep 19 07:21 
> drl?versionLabel=CURRENT=09018000f7c3
> -rw-r--r--. 1 root i2e  23040 Sep 19 07:22 
> drl?versionLabel=CURRENT=09018000f7c1
> -rw-r--r--. 1 root i2e  26112 Sep 19 07:22 
> drl?versionLabel=CURRENT=09018000f7bf
> -rw-r--r--. 1 root i2e  0 Sep 19 07:23 
> drl?versionLabel=CURRENT=09018000f7c7
> -rw-r--r--. 1 root i2e  32783 Sep 19 07:25 
> drl?versionLabel=CURRENT=09018000f7c4{code}

[jira] [Commented] (CONNECTORS-1532) Moving a file outside of the job's Paths is not the same as deleting it

2018-09-19 Thread James Thomas (JIRA)



[ 
https://issues.apache.org/jira/browse/CONNECTORS-1532?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16620779#comment-16620779
 ] 

James Thomas commented on CONNECTORS-1532:
--

[~kwri...@metacarta.com], here's a simple experiment cut down from the original 
which I hope is what you're after.
 * In DM I created a folder F containing a single file X
 * In MF I configured a job to look at F and output to File System
 * In MF I ran the job
 * On the File System I ran 'ls -l'  and found F
 * In DM I moved X to a folder outside of F (Right click, add to clipboard, go 
to new folder, Edit> Move here).
 * In MF I ran the job again
 * On the File System I ran 'ls' again
 * In MF I ran Simple History Report

(In the data below, note that the time stamps are two hours different because 
of the time zone setup I have currently to work around #1521.)

 

Two calls to 'ls':

 
{code:java}
-rw-r--r--. 1 root i2e 26 Sep 19 17:44 
drl?versionLabel=CURRENT=09018000f99a
-rw-r--r--. 1 root i2e 26 Sep 19 17:46 
drl?versionLabel=CURRENT=09018000f99a
{code}
 

Simple History Report for object 09018000f99a:
||Start Time||Activity||Identifier||Result Code||Bytes||Time||Result 
Description||
|09-19-2018 15:46:04.940|fetch|09018000f99a|OK|26|284| |
|09-19-2018 15:44:30.228|fetch|09018000f99a|OK|26|34| |

 

> Moving a file outside of the job's Paths is not the same as deleting it
> ---
>
> Key: CONNECTORS-1532
> URL: https://issues.apache.org/jira/browse/CONNECTORS-1532
> Project: ManifoldCF
>  Issue Type: Bug
>  Components: Documentum connector
>Affects Versions: ManifoldCF 2.10
> Environment: Manifold 2.10 patched for #1512, #1517
>Reporter: James Thomas
>Assignee: Karl Wright
>Priority: Major
>
> If I have a MF job which is connecting a specific folder, F, in Documentum to 
> a File System output then:
> 1. deleting files in Documentum shows them as zero size in the file system
> 2. moving files out of F does not remove them or zero them in the file system
> Note that moving a file from another folder (which the job is not looking at) 
> to F has the same effect as adding it to F by e.g. importing it in DM or 
> POSTing it to DM via the REST interface.
> Intuitively, I expect that moving a file out of the "view" of the Documentum 
> connector would have the same effect on the File System as deleting it. (My 
> model here is of MF synchronising content between the Paths (DM) and the 
> Output Path (File System) that I have specified in the job.)
> Starting point, I have run the MF job to fetch a bunch of files from a folder 
> - call it F - in DM (i.e. I have configured Paths in the job to be F). This 
> is what 'ls -l' on the file system looks like:
> {code:java}
> -rw-r--r--. 1 root i2e  12541 Sep 19 07:21 
> drl?versionLabel=CURRENT=09018000f7c0
> -rw-r--r--. 1 root i2e 26 Sep 19 07:21 
> drl?versionLabel=CURRENT=09018000f7be
> -rw-r--r--. 1 root i2e  85772 Sep 19 07:21 
> drl?versionLabel=CURRENT=09018000f7c7
> -rw-r--r--. 1 root i2e   8790 Sep 19 07:21 
> drl?versionLabel=CURRENT=09018000f7c2
> -rw-r--r--. 1 root i2e 101888 Sep 19 07:21 
> drl?versionLabel=CURRENT=09018000f7c3
> -rw-r--r--. 1 root i2e  32783 Sep 19 07:21 
> drl?versionLabel=CURRENT=09018000f7c4
> -rw-r--r--. 1 root i2e  23040 Sep 19 07:22 
> drl?versionLabel=CURRENT=09018000f7c1
> -rw-r--r--. 1 root i2e  26112 Sep 19 07:22 
> drl?versionLabel=CURRENT=09018000f7bf{code}
> In DM, I delete one of the files in F and it shows as zero size, and the 
> modification date has changed:
> {code:java}
> -rw-r--r--. 1 root i2e  12541 Sep 19 07:21 
> drl?versionLabel=CURRENT=09018000f7c0
> -rw-r--r--. 1 root i2e 26 Sep 19 07:21 
> drl?versionLabel=CURRENT=09018000f7be
> -rw-r--r--. 1 root i2e   8790 Sep 19 07:21 
> drl?versionLabel=CURRENT=09018000f7c2
> -rw-r--r--. 1 root i2e 101888 Sep 19 07:21 
> drl?versionLabel=CURRENT=09018000f7c3
> -rw-r--r--. 1 root i2e  32783 Sep 19 07:21 
> drl?versionLabel=CURRENT=09018000f7c4
> -rw-r--r--. 1 root i2e  23040 Sep 19 07:22 
> drl?versionLabel=CURRENT=09018000f7c1
> -rw-r--r--. 1 root i2e  26112 Sep 19 07:22 
> drl?versionLabel=CURRENT=09018000f7bf
> -rw-r--r--. 1 root i2e  0 Sep 19 07:23 
> drl?versionLabel=CURRENT=09018000f7c7{code}
> In DM, I move a file from F to another folder. (Right click, add to 
> clipboard, go to new folder, Edit> Move here). 
> The file shows as modified (07:25), but is still apparently in F (i.e. in the 
> Path my MF job is looking at):
> {code:java}
> -rw-r--r--. 1 root i2e  12541 Sep 19 07:21 
> drl?versionLabel=CURRENT=09018000f7c0
> -rw-r--r--. 1 root i2e 26 Sep 19 07:21 
> drl?versionLabel=CURRENT=09018000f7be
> -rw-r--r--. 1 root i2e   8790 Sep

[VOTE] Release Apache ManifoldCF 2.11, RC0

2018-09-19 Thread Karl Wright

Please vote on whether to release ManifoldCF 2.11, RC0.  This release
contains a number of fixes/improvements/additions, described in the
CHANGES.txt file.  In addition, it includes Tika 1.19, which has a number
of fixes for classpath issues specifically requested by ManifoldCF.

The release artifact can be found at:

https://dist.apache.org/repos/dist/dev/manifoldcf/apache-manifoldcf-2.11

There is also a tag at:

https://svn.apache.org/repos/asf/manifoldcf/tags/release-2.11-RC0

Thanks again,
Karl Wright

Re: Release status of 2.11

2018-09-19 Thread Karl Wright

Hi all,

Tika 1.19 work has been done, and we're ready to start the ManifoldCF 2.11
release cycle.
It was not trivial though, so I would ask people to try things out.  I'll
be creating a 2.11 release branch shortly.

Karl


On Wed, Sep 19, 2018 at 4:17 AM Karl Wright  wrote:

> Thanks!!
>
> I'll get going on updating ManifoldCF to use it.
>
> Karl
>
> On Wed, Sep 19, 2018 at 2:35 AM Piergiorgio Lucidi 
> wrote:
>
>> It seems that Tika 1.1.9 was released yesterday:
>>
>> https://lists.apache.org/thread.html/8642d60a7e8b9e01a669c50928e23a39c8e984effb77e75f5994135e@%3Cdev.tika.apache.org%3E
>>
>>
>>
>> Il giorno ven 7 set 2018 alle ore 14:11 Karl Wright 
>> ha
>> scritto:
>>
>> > We're still waiting for the next version of Tika to be released and
>> > integrated into trunk before attempting our release.  For some reason
>> this
>> > has not yet been completed, but I think it is due by the middle of
>> > September.
>> >
>> > Perhaps we should formally synchronize MCF releases with Tika going
>> > forward.
>> >
>> > Karl
>> >
>>
>>
>> --
>> Piergiorgio
>>
>

[jira] [Commented] (CONNECTORS-1532) Moving a file outside of the job's Paths is not the same as deleting it

2018-09-19 Thread Karl Wright (JIRA)



[ 
https://issues.apache.org/jira/browse/CONNECTORS-1532?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16620411#comment-16620411
 ] 

Karl Wright commented on CONNECTORS-1532:
-

It's not clear that this behavior has anything to do with the Documentum 
connector.  Can you look at the Simple History for the file in question and 
follow what it does there?  The file output connector may have problems in this 
area.


> Moving a file outside of the job's Paths is not the same as deleting it
> ---
>
> Key: CONNECTORS-1532
> URL: https://issues.apache.org/jira/browse/CONNECTORS-1532
> Project: ManifoldCF
>  Issue Type: Bug
>  Components: Documentum connector
>Affects Versions: ManifoldCF 2.10
> Environment: Manifold 2.10 patched for #1512, #1517
>Reporter: James Thomas
>Assignee: Karl Wright
>Priority: Major
>
> If I have a MF job which is connecting a specific folder, F, in Documentum to 
> a File System output then:
> 1. deleting files in Documentum shows them as zero size in the file system
> 2. moving files out of F does not remove them or zero them in the file system
> Note that moving a file from another folder (which the job is not looking at) 
> to F has the same effect as adding it to F by e.g. importing it in DM or 
> POSTing it to DM via the REST interface.
> Intuitively, I expect that moving a file out of the "view" of the Documentum 
> connector would have the same effect on the File System as deleting it. (My 
> model here is of MF synchronising content between the Paths (DM) and the 
> Output Path (File System) that I have specified in the job.)
> Starting point, I have run the MF job to fetch a bunch of files from a folder 
> - call it F - in DM (i.e. I have configured Paths in the job to be F). This 
> is what 'ls -l' on the file system looks like:
> {code:java}
> -rw-r--r--. 1 root i2e  12541 Sep 19 07:21 
> drl?versionLabel=CURRENT=09018000f7c0
> -rw-r--r--. 1 root i2e 26 Sep 19 07:21 
> drl?versionLabel=CURRENT=09018000f7be
> -rw-r--r--. 1 root i2e  85772 Sep 19 07:21 
> drl?versionLabel=CURRENT=09018000f7c7
> -rw-r--r--. 1 root i2e   8790 Sep 19 07:21 
> drl?versionLabel=CURRENT=09018000f7c2
> -rw-r--r--. 1 root i2e 101888 Sep 19 07:21 
> drl?versionLabel=CURRENT=09018000f7c3
> -rw-r--r--. 1 root i2e  32783 Sep 19 07:21 
> drl?versionLabel=CURRENT=09018000f7c4
> -rw-r--r--. 1 root i2e  23040 Sep 19 07:22 
> drl?versionLabel=CURRENT=09018000f7c1
> -rw-r--r--. 1 root i2e  26112 Sep 19 07:22 
> drl?versionLabel=CURRENT=09018000f7bf{code}
> In DM, I delete one of the files in F and it shows as zero size, and the 
> modification date has changed:
> {code:java}
> -rw-r--r--. 1 root i2e  12541 Sep 19 07:21 
> drl?versionLabel=CURRENT=09018000f7c0
> -rw-r--r--. 1 root i2e 26 Sep 19 07:21 
> drl?versionLabel=CURRENT=09018000f7be
> -rw-r--r--. 1 root i2e   8790 Sep 19 07:21 
> drl?versionLabel=CURRENT=09018000f7c2
> -rw-r--r--. 1 root i2e 101888 Sep 19 07:21 
> drl?versionLabel=CURRENT=09018000f7c3
> -rw-r--r--. 1 root i2e  32783 Sep 19 07:21 
> drl?versionLabel=CURRENT=09018000f7c4
> -rw-r--r--. 1 root i2e  23040 Sep 19 07:22 
> drl?versionLabel=CURRENT=09018000f7c1
> -rw-r--r--. 1 root i2e  26112 Sep 19 07:22 
> drl?versionLabel=CURRENT=09018000f7bf
> -rw-r--r--. 1 root i2e  0 Sep 19 07:23 
> drl?versionLabel=CURRENT=09018000f7c7{code}
> In DM, I move a file from F to another folder. (Right click, add to 
> clipboard, go to new folder, Edit> Move here). 
> The file shows as modified (07:25), but is still apparently in F (i.e. in the 
> Path my MF job is looking at):
> {code:java}
> -rw-r--r--. 1 root i2e  12541 Sep 19 07:21 
> drl?versionLabel=CURRENT=09018000f7c0
> -rw-r--r--. 1 root i2e 26 Sep 19 07:21 
> drl?versionLabel=CURRENT=09018000f7be
> -rw-r--r--. 1 root i2e   8790 Sep 19 07:21 
> drl?versionLabel=CURRENT=09018000f7c2
> -rw-r--r--. 1 root i2e 101888 Sep 19 07:21 
> drl?versionLabel=CURRENT=09018000f7c3
> -rw-r--r--. 1 root i2e  23040 Sep 19 07:22 
> drl?versionLabel=CURRENT=09018000f7c1
> -rw-r--r--. 1 root i2e  26112 Sep 19 07:22 
> drl?versionLabel=CURRENT=09018000f7bf
> -rw-r--r--. 1 root i2e  0 Sep 19 07:23 
> drl?versionLabel=CURRENT=09018000f7c7
> -rw-r--r--. 1 root i2e  32783 Sep 19 07:25 
> drl?versionLabel=CURRENT=09018000f7c4{code}
> In DM, I move a file from another folder to F and it shows up with the 
> timestamp of the move (07:28):
> {code:java}
> -rw-r--r--. 1 root i2e  12541 Sep 19 07:21 
> drl?versionLabel=CURRENT=09018000f7c0
> -rw-r--r--. 1 root i2e 26 Sep 19 07:21 
> drl?versionLabel=CURRENT=09018000f7be
> -rw-r--r--. 1 root i2e   8790 Sep 19 07:21 
> drl?versionLabel=CURRENT=09018000f7c2
>

[jira] [Assigned] (CONNECTORS-1532) Moving a file outside of the job's Paths is not the same as deleting it

2018-09-19 Thread Karl Wright (JIRA)



 [ 
https://issues.apache.org/jira/browse/CONNECTORS-1532?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Karl Wright reassigned CONNECTORS-1532:
---

Assignee: Karl Wright

> Moving a file outside of the job's Paths is not the same as deleting it
> ---
>
> Key: CONNECTORS-1532
> URL: https://issues.apache.org/jira/browse/CONNECTORS-1532
> Project: ManifoldCF
>  Issue Type: Bug
>  Components: Documentum connector
>Affects Versions: ManifoldCF 2.10
> Environment: Manifold 2.10 patched for #1512, #1517
>Reporter: James Thomas
>Assignee: Karl Wright
>Priority: Major
>
> If I have a MF job which is connecting a specific folder, F, in Documentum to 
> a File System output then:
> 1. deleting files in Documentum shows them as zero size in the file system
> 2. moving files out of F does not remove them or zero them in the file system
> Note that moving a file from another folder (which the job is not looking at) 
> to F has the same effect as adding it to F by e.g. importing it in DM or 
> POSTing it to DM via the REST interface.
> Intuitively, I expect that moving a file out of the "view" of the Documentum 
> connector would have the same effect on the File System as deleting it. (My 
> model here is of MF synchronising content between the Paths (DM) and the 
> Output Path (File System) that I have specified in the job.)
> Starting point, I have run the MF job to fetch a bunch of files from a folder 
> - call it F - in DM (i.e. I have configured Paths in the job to be F). This 
> is what 'ls -l' on the file system looks like:
> {code:java}
> -rw-r--r--. 1 root i2e  12541 Sep 19 07:21 
> drl?versionLabel=CURRENT=09018000f7c0
> -rw-r--r--. 1 root i2e 26 Sep 19 07:21 
> drl?versionLabel=CURRENT=09018000f7be
> -rw-r--r--. 1 root i2e  85772 Sep 19 07:21 
> drl?versionLabel=CURRENT=09018000f7c7
> -rw-r--r--. 1 root i2e   8790 Sep 19 07:21 
> drl?versionLabel=CURRENT=09018000f7c2
> -rw-r--r--. 1 root i2e 101888 Sep 19 07:21 
> drl?versionLabel=CURRENT=09018000f7c3
> -rw-r--r--. 1 root i2e  32783 Sep 19 07:21 
> drl?versionLabel=CURRENT=09018000f7c4
> -rw-r--r--. 1 root i2e  23040 Sep 19 07:22 
> drl?versionLabel=CURRENT=09018000f7c1
> -rw-r--r--. 1 root i2e  26112 Sep 19 07:22 
> drl?versionLabel=CURRENT=09018000f7bf{code}
> In DM, I delete one of the files in F and it shows as zero size, and the 
> modification date has changed:
> {code:java}
> -rw-r--r--. 1 root i2e  12541 Sep 19 07:21 
> drl?versionLabel=CURRENT=09018000f7c0
> -rw-r--r--. 1 root i2e 26 Sep 19 07:21 
> drl?versionLabel=CURRENT=09018000f7be
> -rw-r--r--. 1 root i2e   8790 Sep 19 07:21 
> drl?versionLabel=CURRENT=09018000f7c2
> -rw-r--r--. 1 root i2e 101888 Sep 19 07:21 
> drl?versionLabel=CURRENT=09018000f7c3
> -rw-r--r--. 1 root i2e  32783 Sep 19 07:21 
> drl?versionLabel=CURRENT=09018000f7c4
> -rw-r--r--. 1 root i2e  23040 Sep 19 07:22 
> drl?versionLabel=CURRENT=09018000f7c1
> -rw-r--r--. 1 root i2e  26112 Sep 19 07:22 
> drl?versionLabel=CURRENT=09018000f7bf
> -rw-r--r--. 1 root i2e  0 Sep 19 07:23 
> drl?versionLabel=CURRENT=09018000f7c7{code}
> In DM, I move a file from F to another folder. (Right click, add to 
> clipboard, go to new folder, Edit> Move here). 
> The file shows as modified (07:25), but is still apparently in F (i.e. in the 
> Path my MF job is looking at):
> {code:java}
> -rw-r--r--. 1 root i2e  12541 Sep 19 07:21 
> drl?versionLabel=CURRENT=09018000f7c0
> -rw-r--r--. 1 root i2e 26 Sep 19 07:21 
> drl?versionLabel=CURRENT=09018000f7be
> -rw-r--r--. 1 root i2e   8790 Sep 19 07:21 
> drl?versionLabel=CURRENT=09018000f7c2
> -rw-r--r--. 1 root i2e 101888 Sep 19 07:21 
> drl?versionLabel=CURRENT=09018000f7c3
> -rw-r--r--. 1 root i2e  23040 Sep 19 07:22 
> drl?versionLabel=CURRENT=09018000f7c1
> -rw-r--r--. 1 root i2e  26112 Sep 19 07:22 
> drl?versionLabel=CURRENT=09018000f7bf
> -rw-r--r--. 1 root i2e  0 Sep 19 07:23 
> drl?versionLabel=CURRENT=09018000f7c7
> -rw-r--r--. 1 root i2e  32783 Sep 19 07:25 
> drl?versionLabel=CURRENT=09018000f7c4{code}
> In DM, I move a file from another folder to F and it shows up with the 
> timestamp of the move (07:28):
> {code:java}
> -rw-r--r--. 1 root i2e  12541 Sep 19 07:21 
> drl?versionLabel=CURRENT=09018000f7c0
> -rw-r--r--. 1 root i2e 26 Sep 19 07:21 
> drl?versionLabel=CURRENT=09018000f7be
> -rw-r--r--. 1 root i2e   8790 Sep 19 07:21 
> drl?versionLabel=CURRENT=09018000f7c2
> -rw-r--r--. 1 root i2e 101888 Sep 19 07:21 
> drl?versionLabel=CURRENT=09018000f7c3
> -rw-r--r--. 1 root i2e  23040 Sep 19 07:22 
> drl?versionLabel=CURRENT=09018000f7c1
> -rw-r--r--. 1 root i2e  26112 Sep 19 07:22 
>

[jira] [Updated] (CONNECTORS-1532) Moving a file outside of the job's Paths is not the same as deleting it

2018-09-19 Thread James Thomas (JIRA)



 [ 
https://issues.apache.org/jira/browse/CONNECTORS-1532?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

James Thomas updated CONNECTORS-1532:
-
Summary: Moving a file outside of the job's Paths is not the same as 
deleting it  (was: Moving a file outside of the jobs Paths is not the same as 
deleting it)

> Moving a file outside of the job's Paths is not the same as deleting it
> ---
>
> Key: CONNECTORS-1532
> URL: https://issues.apache.org/jira/browse/CONNECTORS-1532
> Project: ManifoldCF
>  Issue Type: Bug
>  Components: Documentum connector
>Affects Versions: ManifoldCF 2.10
> Environment: Manifold 2.10 patched for #1512, #1517
>Reporter: James Thomas
>Priority: Major
>
> If I have a MF job which is connecting a specific folder, F, in Documentum to 
> a File System output then:
> 1. deleting files in Documentum shows them as zero size in the file system
> 2. moving files out of F does not remove them or zero them in the file system
> Note that moving a file from another folder (which the job is not looking at) 
> to F has the same effect as adding it to F by e.g. importing it in DM or 
> POSTing it to DM via the REST interface.
> Intuitively, I expect that moving a file out of the "view" of the Documentum 
> connector would have the same effect on the File System as deleting it. (My 
> model here is of MF synchronising content between the Paths (DM) and the 
> Output Path (File System) that I have specified in the job.)
> Starting point, I have run the MF job to fetch a bunch of files from a folder 
> - call it F - in DM (i.e. I have configured Paths in the job to be F). This 
> is what 'ls -l' on the file system looks like:
> {code:java}
> -rw-r--r--. 1 root i2e  12541 Sep 19 07:21 
> drl?versionLabel=CURRENT=09018000f7c0
> -rw-r--r--. 1 root i2e 26 Sep 19 07:21 
> drl?versionLabel=CURRENT=09018000f7be
> -rw-r--r--. 1 root i2e  85772 Sep 19 07:21 
> drl?versionLabel=CURRENT=09018000f7c7
> -rw-r--r--. 1 root i2e   8790 Sep 19 07:21 
> drl?versionLabel=CURRENT=09018000f7c2
> -rw-r--r--. 1 root i2e 101888 Sep 19 07:21 
> drl?versionLabel=CURRENT=09018000f7c3
> -rw-r--r--. 1 root i2e  32783 Sep 19 07:21 
> drl?versionLabel=CURRENT=09018000f7c4
> -rw-r--r--. 1 root i2e  23040 Sep 19 07:22 
> drl?versionLabel=CURRENT=09018000f7c1
> -rw-r--r--. 1 root i2e  26112 Sep 19 07:22 
> drl?versionLabel=CURRENT=09018000f7bf{code}
> In DM, I delete one of the files in F and it shows as zero size, and the 
> modification date has changed:
> {code:java}
> -rw-r--r--. 1 root i2e  12541 Sep 19 07:21 
> drl?versionLabel=CURRENT=09018000f7c0
> -rw-r--r--. 1 root i2e 26 Sep 19 07:21 
> drl?versionLabel=CURRENT=09018000f7be
> -rw-r--r--. 1 root i2e   8790 Sep 19 07:21 
> drl?versionLabel=CURRENT=09018000f7c2
> -rw-r--r--. 1 root i2e 101888 Sep 19 07:21 
> drl?versionLabel=CURRENT=09018000f7c3
> -rw-r--r--. 1 root i2e  32783 Sep 19 07:21 
> drl?versionLabel=CURRENT=09018000f7c4
> -rw-r--r--. 1 root i2e  23040 Sep 19 07:22 
> drl?versionLabel=CURRENT=09018000f7c1
> -rw-r--r--. 1 root i2e  26112 Sep 19 07:22 
> drl?versionLabel=CURRENT=09018000f7bf
> -rw-r--r--. 1 root i2e  0 Sep 19 07:23 
> drl?versionLabel=CURRENT=09018000f7c7{code}
> In DM, I move a file from F to another folder. (Right click, add to 
> clipboard, go to new folder, Edit> Move here). 
> The file shows as modified (07:25), but is still apparently in F (i.e. in the 
> Path my MF job is looking at):
> {code:java}
> -rw-r--r--. 1 root i2e  12541 Sep 19 07:21 
> drl?versionLabel=CURRENT=09018000f7c0
> -rw-r--r--. 1 root i2e 26 Sep 19 07:21 
> drl?versionLabel=CURRENT=09018000f7be
> -rw-r--r--. 1 root i2e   8790 Sep 19 07:21 
> drl?versionLabel=CURRENT=09018000f7c2
> -rw-r--r--. 1 root i2e 101888 Sep 19 07:21 
> drl?versionLabel=CURRENT=09018000f7c3
> -rw-r--r--. 1 root i2e  23040 Sep 19 07:22 
> drl?versionLabel=CURRENT=09018000f7c1
> -rw-r--r--. 1 root i2e  26112 Sep 19 07:22 
> drl?versionLabel=CURRENT=09018000f7bf
> -rw-r--r--. 1 root i2e  0 Sep 19 07:23 
> drl?versionLabel=CURRENT=09018000f7c7
> -rw-r--r--. 1 root i2e  32783 Sep 19 07:25 
> drl?versionLabel=CURRENT=09018000f7c4{code}
> In DM, I move a file from another folder to F and it shows up with the 
> timestamp of the move (07:28):
> {code:java}
> -rw-r--r--. 1 root i2e  12541 Sep 19 07:21 
> drl?versionLabel=CURRENT=09018000f7c0
> -rw-r--r--. 1 root i2e 26 Sep 19 07:21 
> drl?versionLabel=CURRENT=09018000f7be
> -rw-r--r--. 1 root i2e   8790 Sep 19 07:21 
> drl?versionLabel=CURRENT=09018000f7c2
> -rw-r--r--. 1 root i2e 101888 Sep 19 07:21 
> drl?versionLabel=CURRENT=09018000f7c3
> -rw-r--r--. 1 root i2e  23040 Sep 19 07:22 
>

[jira] [Created] (CONNECTORS-1532) Moving a file outside of the jobs Paths is not the same as deleting it

2018-09-19 Thread James Thomas (JIRA)

James Thomas created CONNECTORS-1532:


 Summary: Moving a file outside of the jobs Paths is not the same 
as deleting it
 Key: CONNECTORS-1532
 URL: https://issues.apache.org/jira/browse/CONNECTORS-1532
 Project: ManifoldCF
  Issue Type: Bug
  Components: Documentum connector
Affects Versions: ManifoldCF 2.10
 Environment: Manifold 2.10 patched for #1512, #1517
Reporter: James Thomas


If I have a MF job which is connecting a specific folder, F, in Documentum to a 
File System output then:

1. deleting files in Documentum shows them as zero size in the file system
2. moving files out of F does not remove them or zero them in the file system

Note that moving a file from another folder (which the job is not looking at) 
to F has the same effect as adding it to F by e.g. importing it in DM or 
POSTing it to DM via the REST interface.

Intuitively, I expect that moving a file out of the "view" of the Documentum 
connector would have the same effect on the File System as deleting it. (My 
model here is of MF synchronising content between the Paths (DM) and the Output 
Path (File System) that I have specified in the job.)

Starting point, I have run the MF job to fetch a bunch of files from a folder - 
call it F - in DM (i.e. I have configured Paths in the job to be F). This is 
what 'ls -l' on the file system looks like:


{code:java}
-rw-r--r--. 1 root i2e  12541 Sep 19 07:21 
drl?versionLabel=CURRENT=09018000f7c0
-rw-r--r--. 1 root i2e 26 Sep 19 07:21 
drl?versionLabel=CURRENT=09018000f7be
-rw-r--r--. 1 root i2e  85772 Sep 19 07:21 
drl?versionLabel=CURRENT=09018000f7c7
-rw-r--r--. 1 root i2e   8790 Sep 19 07:21 
drl?versionLabel=CURRENT=09018000f7c2
-rw-r--r--. 1 root i2e 101888 Sep 19 07:21 
drl?versionLabel=CURRENT=09018000f7c3
-rw-r--r--. 1 root i2e  32783 Sep 19 07:21 
drl?versionLabel=CURRENT=09018000f7c4
-rw-r--r--. 1 root i2e  23040 Sep 19 07:22 
drl?versionLabel=CURRENT=09018000f7c1
-rw-r--r--. 1 root i2e  26112 Sep 19 07:22 
drl?versionLabel=CURRENT=09018000f7bf{code}

In DM, I delete one of the files in F and it shows as zero size, and the 
modification date has changed:


{code:java}
-rw-r--r--. 1 root i2e  12541 Sep 19 07:21 
drl?versionLabel=CURRENT=09018000f7c0
-rw-r--r--. 1 root i2e 26 Sep 19 07:21 
drl?versionLabel=CURRENT=09018000f7be
-rw-r--r--. 1 root i2e   8790 Sep 19 07:21 
drl?versionLabel=CURRENT=09018000f7c2
-rw-r--r--. 1 root i2e 101888 Sep 19 07:21 
drl?versionLabel=CURRENT=09018000f7c3
-rw-r--r--. 1 root i2e  32783 Sep 19 07:21 
drl?versionLabel=CURRENT=09018000f7c4
-rw-r--r--. 1 root i2e  23040 Sep 19 07:22 
drl?versionLabel=CURRENT=09018000f7c1
-rw-r--r--. 1 root i2e  26112 Sep 19 07:22 
drl?versionLabel=CURRENT=09018000f7bf
-rw-r--r--. 1 root i2e  0 Sep 19 07:23 
drl?versionLabel=CURRENT=09018000f7c7{code}

In DM, I move a file from F to another folder. (Right click, add to clipboard, 
go to new folder, Edit> Move here). 

The file shows as modified (07:25), but is still apparently in F (i.e. in the 
Path my MF job is looking at):


{code:java}
-rw-r--r--. 1 root i2e  12541 Sep 19 07:21 
drl?versionLabel=CURRENT=09018000f7c0
-rw-r--r--. 1 root i2e 26 Sep 19 07:21 
drl?versionLabel=CURRENT=09018000f7be
-rw-r--r--. 1 root i2e   8790 Sep 19 07:21 
drl?versionLabel=CURRENT=09018000f7c2
-rw-r--r--. 1 root i2e 101888 Sep 19 07:21 
drl?versionLabel=CURRENT=09018000f7c3
-rw-r--r--. 1 root i2e  23040 Sep 19 07:22 
drl?versionLabel=CURRENT=09018000f7c1
-rw-r--r--. 1 root i2e  26112 Sep 19 07:22 
drl?versionLabel=CURRENT=09018000f7bf
-rw-r--r--. 1 root i2e  0 Sep 19 07:23 
drl?versionLabel=CURRENT=09018000f7c7
-rw-r--r--. 1 root i2e  32783 Sep 19 07:25 
drl?versionLabel=CURRENT=09018000f7c4{code}

In DM, I move a file from another folder to F and it shows up with the 
timestamp of the move (07:28):


{code:java}
-rw-r--r--. 1 root i2e  12541 Sep 19 07:21 
drl?versionLabel=CURRENT=09018000f7c0
-rw-r--r--. 1 root i2e 26 Sep 19 07:21 
drl?versionLabel=CURRENT=09018000f7be
-rw-r--r--. 1 root i2e   8790 Sep 19 07:21 
drl?versionLabel=CURRENT=09018000f7c2
-rw-r--r--. 1 root i2e 101888 Sep 19 07:21 
drl?versionLabel=CURRENT=09018000f7c3
-rw-r--r--. 1 root i2e  23040 Sep 19 07:22 
drl?versionLabel=CURRENT=09018000f7c1
-rw-r--r--. 1 root i2e  26112 Sep 19 07:22 
drl?versionLabel=CURRENT=09018000f7bf
-rw-r--r--. 1 root i2e  0 Sep 19 07:23 
drl?versionLabel=CURRENT=09018000f7c7
-rw-r--r--. 1 root i2e  32783 Sep 19 07:25 
drl?versionLabel=CURRENT=09018000f7c4
-rw-r--r--. 1 root i2e 191513 Sep 19 07:28 
drl?versionLabel=CURRENT=0901800045b9{code}

But if I immediately move it out in DM then, again, the timestamp (07:30) 
alters but the file apparently remains:


{code:java}
-rw-r--r--. 1 root i2e  12541 Sep

Re: Release status of 2.11

2018-09-19 Thread Karl Wright

Thanks!!

I'll get going on updating ManifoldCF to use it.

Karl

On Wed, Sep 19, 2018 at 2:35 AM Piergiorgio Lucidi 
wrote:

> It seems that Tika 1.1.9 was released yesterday:
>
> https://lists.apache.org/thread.html/8642d60a7e8b9e01a669c50928e23a39c8e984effb77e75f5994135e@%3Cdev.tika.apache.org%3E
>
>
>
> Il giorno ven 7 set 2018 alle ore 14:11 Karl Wright 
> ha
> scritto:
>
> > We're still waiting for the next version of Tika to be released and
> > integrated into trunk before attempting our release.  For some reason
> this
> > has not yet been completed, but I think it is due by the middle of
> > September.
> >
> > Perhaps we should formally synchronize MCF releases with Tika going
> > forward.
> >
> > Karl
> >
>
>
> --
> Piergiorgio
>

Re: Release status of 2.11

2018-09-19 Thread Piergiorgio Lucidi

It seems that Tika 1.1.9 was released yesterday:
https://lists.apache.org/thread.html/8642d60a7e8b9e01a669c50928e23a39c8e984effb77e75f5994135e@%3Cdev.tika.apache.org%3E



Il giorno ven 7 set 2018 alle ore 14:11 Karl Wright  ha
scritto:

> We're still waiting for the next version of Tika to be released and
> integrated into trunk before attempting our release.  For some reason this
> has not yet been completed, but I think it is due by the middle of
> September.
>
> Perhaps we should formally synchronize MCF releases with Tika going
> forward.
>
> Karl
>


-- 
Piergiorgio

[jira] [Commented] (CONNECTORS-1532) Moving a file outside of the job's Paths is not the same as deleting it

[VOTE] Release ManifoldCF 2.11, RC1

Build failed in Jenkins: ManifoldCF-ant #658

[CANCEL][VOTE] Release Apache ManifoldCF 2.11, RC0

[jira] [Commented] (CONNECTORS-1532) Moving a file outside of the job's Paths is not the same as deleting it

[jira] [Commented] (CONNECTORS-1532) Moving a file outside of the job's Paths is not the same as deleting it

[jira] [Commented] (CONNECTORS-1532) Moving a file outside of the job's Paths is not the same as deleting it

[jira] [Updated] (CONNECTORS-1532) Moving a file outside of the job's Paths is not the same as deleting it

[jira] [Commented] (CONNECTORS-1532) Moving a file outside of the job's Paths is not the same as deleting it

[jira] [Commented] (CONNECTORS-1532) Moving a file outside of the job's Paths is not the same as deleting it

[jira] [Commented] (CONNECTORS-1532) Moving a file outside of the job's Paths is not the same as deleting it

[VOTE] Release Apache ManifoldCF 2.11, RC0

Re: Release status of 2.11

[jira] [Commented] (CONNECTORS-1532) Moving a file outside of the job's Paths is not the same as deleting it

[jira] [Assigned] (CONNECTORS-1532) Moving a file outside of the job's Paths is not the same as deleting it

[jira] [Updated] (CONNECTORS-1532) Moving a file outside of the job's Paths is not the same as deleting it

[jira] [Created] (CONNECTORS-1532) Moving a file outside of the jobs Paths is not the same as deleting it

Re: Release status of 2.11

Re: Release status of 2.11

19 matches

Site Navigation

Mail list logo

Footer information