[jira] [Issue Comment Edited] (CONNECTORS-347) MySQL HopCount test sometimes fails

2012-04-20 Thread Karl Wright (Issue Comment Edited) (JIRA) 

[ 
https://issues.apache.org/jira/browse/CONNECTORS-347?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13258202#comment-13258202
 ] 

Karl Wright edited comment on CONNECTORS-347 at 4/20/12 12:58 PM:
--

Here's a postmortem for a run that succeeded:

{code}
mysql select parentidhash,distance from hopcount;
+--+--+
| parentidhash | distance |
+--+--+
| 5432CFFCE5729BF1F3E695FFCF83B08E415694C6 |0 |
| 718564BC3D0C94B0F88B36B7FA2FC915F665A5E2 |3 |
| 1D9DB3181447F82677FB9AB91C56F33B1A2C75B0 |2 |
| B622AAB99E7BFF749E2C93E01009C3E02F90693C |2 |
| 9F6F59234CC36DF341184FAEC934BEEE516E1E60 |1 |
| CD31925B98C4B481F6523030A1C7C9D9DF0CC080 |1 |
| 1B26DBA4653B26B3151433EBB4C1CDCB8358C1F2 |1 |
+--+--+
7 rows in set (0.00 sec)

mysql select parentidhash,childidhash from intrinsiclink;
+--+--+
| parentidhash | childidhash  |
+--+--+
| 1B26DBA4653B26B3151433EBB4C1CDCB8358C1F2 | 
5432CFFCE5729BF1F3E695FFCF83B08E415694C6 |
| 1D9DB3181447F82677FB9AB91C56F33B1A2C75B0 | 
1B26DBA4653B26B3151433EBB4C1CDCB8358C1F2 |
| 5432CFFCE5729BF1F3E695FFCF83B08E415694C6 |  |
| 718564BC3D0C94B0F88B36B7FA2FC915F665A5E2 | 
1D9DB3181447F82677FB9AB91C56F33B1A2C75B0 |
| 9F6F59234CC36DF341184FAEC934BEEE516E1E60 | 
5432CFFCE5729BF1F3E695FFCF83B08E415694C6 |
| B622AAB99E7BFF749E2C93E01009C3E02F90693C | 
1B26DBA4653B26B3151433EBB4C1CDCB8358C1F2 |
| CD31925B98C4B481F6523030A1C7C9D9DF0CC080 | 
5432CFFCE5729BF1F3E695FFCF83B08E415694C6 |
+--+--+
7 rows in set (0.00 sec)

mysql select dochash,docid from jobqueue;
+--++
| dochash  | docid  
  |
+--++
| 5432CFFCE5729BF1F3E695FFCF83B08E415694C6 | 
C:\wip\mcf\trunk\tests\filesystem\test-mysql-output\testdata   |
| B622AAB99E7BFF749E2C93E01009C3E02F90693C | 
C:\wip\mcf\trunk\tests\filesystem\test-mysql-output\testdata\testdir\test3.txt |
| 1D9DB3181447F82677FB9AB91C56F33B1A2C75B0 | 
C:\wip\mcf\trunk\tests\filesystem\test-mysql-output\testdata\testdir\seconddir |
| CD31925B98C4B481F6523030A1C7C9D9DF0CC080 | 
C:\wip\mcf\trunk\tests\filesystem\test-mysql-output\testdata\test1.txt |
| 9F6F59234CC36DF341184FAEC934BEEE516E1E60 | 
C:\wip\mcf\trunk\tests\filesystem\test-mysql-output\testdata\test2.txt |
| 1B26DBA4653B26B3151433EBB4C1CDCB8358C1F2 | 
C:\wip\mcf\trunk\tests\filesystem\test-mysql-output\testdata\testdir   |
+--++
6 rows in set (0.00 sec)
{code}

Even in this one, the jobqueue table is inconsistent with the hopcount and 
intrinsiclink tables.  They should all be consistent.



  was (Author: kwri...@metacarta.com):
Here's a postmortem for a run that succeeded:

{code}
mysql select parentidhash,distance from hopcount;
+--+--+
| parentidhash | distance |
+--+--+
| 5432CFFCE5729BF1F3E695FFCF83B08E415694C6 |0 |
| 718564BC3D0C94B0F88B36B7FA2FC915F665A5E2 |3 |
| 1D9DB3181447F82677FB9AB91C56F33B1A2C75B0 |2 |
| B622AAB99E7BFF749E2C93E01009C3E02F90693C |2 |
| 9F6F59234CC36DF341184FAEC934BEEE516E1E60 |1 |
| CD31925B98C4B481F6523030A1C7C9D9DF0CC080 |1 |
| 1B26DBA4653B26B3151433EBB4C1CDCB8358C1F2 |1 |
+--+--+
7 rows in set (0.00 sec)

mysql select parentidhash,childidhash from intrinsiclink;
+--+--+
| parentidhash | childidhash  |
+--+--+
| 1B26DBA4653B26B3151433EBB4C1CDCB8358C1F2 | 
5432CFFCE5729BF1F3E695FFCF83B08E415694C6 |
| 1D9DB3181447F82677FB9AB91C56F33B1A2C75B0 | 
1B26DBA4653B26B3151433EBB4C1CDCB8358C1F2 |
| 5432CFFCE5729BF1F3E695FFCF83B08E415694C6 |  |
| 718564BC3D0C94B0F88B36B7FA2FC915F665A5E2 | 
1D9DB3181447F82677FB9AB91C56F33B1A2C75B0 |
| 9F6F59234CC36DF341184FAEC934BEEE516E1E60 |

[jira] [Issue Comment Edited] (CONNECTORS-347) MySQL HopCount test sometimes fails

2012-04-20 Thread Karl Wright (Issue Comment Edited) (JIRA) 

[ 
https://issues.apache.org/jira/browse/CONNECTORS-347?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13258237#comment-13258237
 ] 

Karl Wright edited comment on CONNECTORS-347 at 4/20/12 1:27 PM:
-

A postgresql run is as follows:

{code}
testdb=# select dochash,docid from jobqueue;
 dochash  |   docid
--+-
 EE6C0ED002CE9303183FEFEF111D6598203F6295 | 
C:\wip\mcf\trunk\tests\filesystem\test-postgresql-output\testdata
 E7DE0B8D3F153CECFD3C57B4D83E59073EC4DD60 | 
C:\wip\mcf\trunk\tests\filesystem\test-postgresql-output\testdata\testdir
 181B9C9FB7F62D24461967391B73FF7CB0B14CD9 | 
C:\wip\mcf\trunk\tests\filesystem\test-postgresql-output\testdata\test2.txt
 24081AC0B03B2394F38D3EB78A3BEDB6CCF1D981 | 
C:\wip\mcf\trunk\tests\filesystem\test-postgresql-output\testdata\test1.txt
 5AA0DB8B2B7D267FAD5E3E0E8926996DEFC4C666 | 
C:\wip\mcf\trunk\tests\filesystem\test-postgresql-output\testdata\testdir\seconddir
 4ED8AFCC917B604C1EE4C83458DB9B7265967F40 | 
C:\wip\mcf\trunk\tests\filesystem\test-postgresql-output\testdata\testdir\test3.txt
(6 rows)


testdb=# select parentidhash, childidhash from intrinsiclink;
   parentidhash   |   childidhash

--+--
 EE6C0ED002CE9303183FEFEF111D6598203F6295 |
 181B9C9FB7F62D24461967391B73FF7CB0B14CD9 | 
EE6C0ED002CE9303183FEFEF111D6598203F6295
 24081AC0B03B2394F38D3EB78A3BEDB6CCF1D981 | 
EE6C0ED002CE9303183FEFEF111D6598203F6295
 E7DE0B8D3F153CECFD3C57B4D83E59073EC4DD60 | 
EE6C0ED002CE9303183FEFEF111D6598203F6295
 4ED8AFCC917B604C1EE4C83458DB9B7265967F40 | 
E7DE0B8D3F153CECFD3C57B4D83E59073EC4DD60
 5AA0DB8B2B7D267FAD5E3E0E8926996DEFC4C666 | 
E7DE0B8D3F153CECFD3C57B4D83E59073EC4DD60
 7013D738BB77ED47763449997384046B20B77832 | 
5AA0DB8B2B7D267FAD5E3E0E8926996DEFC4C666
(7 rows)


testdb=# select parentidhash, distance from hopcount;
   parentidhash   | distance
--+--
 EE6C0ED002CE9303183FEFEF111D6598203F6295 |0
 24081AC0B03B2394F38D3EB78A3BEDB6CCF1D981 |1
 181B9C9FB7F62D24461967391B73FF7CB0B14CD9 |1
 E7DE0B8D3F153CECFD3C57B4D83E59073EC4DD60 |1
 5AA0DB8B2B7D267FAD5E3E0E8926996DEFC4C666 |2
 4ED8AFCC917B604C1EE4C83458DB9B7265967F40 |2
 7013D738BB77ED47763449997384046B20B77832 |3
(7 rows)


testdb=#
{code}


  was (Author: kwri...@metacarta.com):
A postgresql run is as follows:

{code}
testdb=# select dochash,docid from jobqueue;
 dochash  |
   docid
--+-

 EE6C0ED002CE9303183FEFEF111D6598203F6295 | C:\wip\mcf\trunk\tests\filesystem\te
st-postgresql-output\testdata
 E7DE0B8D3F153CECFD3C57B4D83E59073EC4DD60 | C:\wip\mcf\trunk\tests\filesystem\te
st-postgresql-output\testdata\testdir
 181B9C9FB7F62D24461967391B73FF7CB0B14CD9 | C:\wip\mcf\trunk\tests\filesystem\te
st-postgresql-output\testdata\test2.txt
 24081AC0B03B2394F38D3EB78A3BEDB6CCF1D981 | C:\wip\mcf\trunk\tests\filesystem\te
st-postgresql-output\testdata\test1.txt
 5AA0DB8B2B7D267FAD5E3E0E8926996DEFC4C666 | C:\wip\mcf\trunk\tests\filesystem\te
st-postgresql-output\testdata\testdir\seconddir
 4ED8AFCC917B604C1EE4C83458DB9B7265967F40 | C:\wip\mcf\trunk\tests\filesystem\te
st-postgresql-output\testdata\testdir\test3.txt
(6 rows)


testdb=# select parentidhash, childidhash from intrinsiclink;
   parentidhash   |   childidhash

--+-
-
 EE6C0ED002CE9303183FEFEF111D6598203F6295 |
 181B9C9FB7F62D24461967391B73FF7CB0B14CD9 | EE6C0ED002CE9303183FEFEF111D6598203F
6295
 24081AC0B03B2394F38D3EB78A3BEDB6CCF1D981 | EE6C0ED002CE9303183FEFEF111D6598203F
6295
 E7DE0B8D3F153CECFD3C57B4D83E59073EC4DD60 | EE6C0ED002CE9303183FEFEF111D6598203F
6295
 4ED8AFCC917B604C1EE4C83458DB9B7265967F40 | E7DE0B8D3F153CECFD3C57B4D83E59073EC4
DD60
 5AA0DB8B2B7D267FAD5E3E0E8926996DEFC4C666 | E7DE0B8D3F153CECFD3C57B4D83E59073EC4
DD60
 7013D738BB77ED47763449997384046B20B77832 | 5AA0DB8B2B7D267FAD5E3E0E8926996DEFC4
C666
(7 rows)


testdb=# select parentidhash, distance from hopcount;
   parentidhash   | distance
--+--
 EE6C0ED002CE9303183FEFEF111D6598203F6295 |0
 24081AC0B03B2394F38D3EB78A3BEDB6CCF1D981 |1
 181B9C9FB7F62D24461967391B73FF7CB0B14CD9 |1
 E7DE0B8D3F153CECFD3C57B4D83E59073EC4DD60 |1
 5AA0DB8B2B7D267FAD5E3E0E8926996DEFC4C666 |2
 4ED8AFCC917B604C1EE4C83458DB9B7265967F40

[jira] [Issue Comment Edited] (CONNECTORS-347) MySQL HopCount test sometimes fails

2012-04-19 Thread Karl Wright (Issue Comment Edited) (JIRA) 

[ 
https://issues.apache.org/jira/browse/CONNECTORS-347?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13257899#comment-13257899
 ] 

Karl Wright edited comment on CONNECTORS-347 at 4/19/12 11:23 PM:
--

Just saw it again; clearly it is real.
My working hypothesis is that the counts returned by count(x) in the SQL 
statements are in fact not synchronized with the transactions, so maybe the 
count can run behind.  The context in which this happens is an initial crawl 
of six documents, with a hopcount filter.  If the filter is not working the 
count would be 7; with a working filter, it's 6.  But how can we then get 4?


  was (Author: kwri...@metacarta.com):
Just saw it again; clearly it is real.
  
 MySQL HopCount test sometimes fails
 ---

 Key: CONNECTORS-347
 URL: https://issues.apache.org/jira/browse/CONNECTORS-347
 Project: ManifoldCF
  Issue Type: Bug
  Components: Framework core
Affects Versions: ManifoldCF 0.5
Reporter: Karl Wright
Assignee: Karl Wright
 Fix For: ManifoldCF next


 The HopcountMySQLIT test sometimes fails complaining that it is expecting to 
 see 6 documents but only sees 4.
 {code}
 [junit] org.apache.manifoldcf.core.interfaces.ManifoldCFException: Wrong 
 number of documents processed - expected 6, saw 4
 [junit] at 
 org.apache.manifoldcf.filesystem_tests.HopcountMySQLIT.hopcountCheck(HopcountMySQLIT.java:152)
 {code}
 This does not happen on every run.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Issue Comment Edited] (CONNECTORS-347) MySQL HopCount test sometimes fails

2012-04-19 Thread Karl Wright (Issue Comment Edited) (JIRA) 

[ 
https://issues.apache.org/jira/browse/CONNECTORS-347?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13257899#comment-13257899
 ] 

Karl Wright edited comment on CONNECTORS-347 at 4/20/12 1:42 AM:
-

Just saw it again; clearly it is real.
My working hypothesis is that the counts returned by count() in the SQL 
statements are in fact not synchronized with the transactions, so maybe the 
count can run behind.  The context in which this happens is an initial crawl 
of six documents, with a hopcount filter.  If the filter is not working the 
count would be 7; with a working filter, it's 6.  But how can we then get 4?


  was (Author: kwri...@metacarta.com):
Just saw it again; clearly it is real.
My working hypothesis is that the counts returned by count(x) in the SQL 
statements are in fact not synchronized with the transactions, so maybe the 
count can run behind.  The context in which this happens is an initial crawl 
of six documents, with a hopcount filter.  If the filter is not working the 
count would be 7; with a working filter, it's 6.  But how can we then get 4?

  
 MySQL HopCount test sometimes fails
 ---

 Key: CONNECTORS-347
 URL: https://issues.apache.org/jira/browse/CONNECTORS-347
 Project: ManifoldCF
  Issue Type: Bug
  Components: Framework core
Affects Versions: ManifoldCF 0.5, ManifoldCF 0.6
Reporter: Karl Wright
Assignee: Karl Wright
 Fix For: ManifoldCF 0.6


 The HopcountMySQLIT test sometimes fails complaining that it is expecting to 
 see 6 documents but only sees 4.
 {code}
 [junit] org.apache.manifoldcf.core.interfaces.ManifoldCFException: Wrong 
 number of documents processed - expected 6, saw 4
 [junit] at 
 org.apache.manifoldcf.filesystem_tests.HopcountMySQLIT.hopcountCheck(HopcountMySQLIT.java:152)
 {code}
 This does not happen on every run.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira