Re: [VOTE] Release Apache ManifoldCF 1.0, RC5
On 28.09.12 13.31, Erlend Garåsen wrote: OK, I will give you a stack trace in the beginning of next week. Do you still need the stack trace? If you do, I need to adjust the log level and/or change the source code in order to print it out. I'm still a little bit worried about how MCF deals with 500 server errors since the job I started last Friday is still running. It retries and retries the three documents I previously mentioned. Is it really a normal behaviour that MCF retries the same document every fourth second after the last attempt and continues do do this (perhaps) thousand times? MCF has probably retried these documents in four days now. I doubt this is normal behaviour. The job should end in the middle of the day on Saturday, and now it's Tuesday. I will test the latest RC after these issues have been clarified. Erlend -- Erlend Garåsen Center for Information Technology Services University of Oslo P.O. Box 1086 Blindern, N-0317 OSLO, Norway Ph: (+47) 22840193, Fax: (+47) 22852970, Mobile: (+47) 91380968, VIP: 31050
Re: [VOTE] Release Apache ManifoldCF 1.0, RC5
Karl, you wrote: I was able to reproduce the exception here using your URL. It is indeed a bug in how it handles the 500 error. OK, then I guess that the StringIndexOutOfBoundsException *was* related to the 500 server issue (It is not clear at all that it is related to the 500 error you described before, but it could be.). To clarify another thing: These three documents are fetched over and over again every fourth second (in four days). I was mentioning this in case we had another issue. I'm just trying to clarify this before I deploy RC7 as I wrote. Anyway, I will deploy RC7 now and start my job once more. Erlend On 02.10.12 11.03, Karl Wright wrote: No stack trace needed. If you read the rest of the mail, you will note that I was able to reproduce the issue using the URL you had provided. There have been two RC's since; we are on RC7 now. Karl On Tue, Oct 2, 2012 at 4:38 AM, Erlend Garåsen e.f.gara...@usit.uio.no wrote: On 28.09.12 13.31, Erlend Garåsen wrote: OK, I will give you a stack trace in the beginning of next week. Do you still need the stack trace? If you do, I need to adjust the log level and/or change the source code in order to print it out. I'm still a little bit worried about how MCF deals with 500 server errors since the job I started last Friday is still running. It retries and retries the three documents I previously mentioned. Is it really a normal behaviour that MCF retries the same document every fourth second after the last attempt and continues do do this (perhaps) thousand times? MCF has probably retried these documents in four days now. I doubt this is normal behaviour. The job should end in the middle of the day on Saturday, and now it's Tuesday. I will test the latest RC after these issues have been clarified. Erlend -- Erlend Garåsen Center for Information Technology Services University of Oslo P.O. Box 1086 Blindern, N-0317 OSLO, Norway Ph: (+47) 22840193, Fax: (+47) 22852970, Mobile: (+47) 91380968, VIP: 31050 -- Erlend Garåsen Center for Information Technology Services University of Oslo P.O. Box 1086 Blindern, N-0317 OSLO, Norway Ph: (+47) 22840193, Fax: (+47) 22852970, Mobile: (+47) 91380968, VIP: 31050
Re: [VOTE] Release Apache ManifoldCF 1.0, RC5
I'm trying to start a crawl before I have to run to the airport. I just discovered that MCF recrawls the same host over and over again when it returns result code 500: 09-28-2012 11:40:11.024 fetch http://foreninger.uio.no/go/oslo_open_2012_no.php 500 It's just not this document, but several others returning the same HTTP result code. Meanwhile, the following is filling up my log: FATAL 2012-09-28 11:42:32,112 (Worker thread '29') - Error tossed: String index out of range: -1 java.lang.StringIndexOutOfBoundsException: String index out of range: -1 I'm pretty sure they are related to each other. I will end this job before I leave because I'm afraid that MCF will try to fetch these documents over and over again during this weekend. Erlend On 28.09.12 09.58, Karl Wright wrote: Please vote +1 to release ManifoldCF 1.0, RC5. The release artifact can be found at: http://people.apache.org/~kwright/apache-manifoldcf-1.0 There is also an SVN tag at: https://svn.apache.org/repos/asf/manifoldcf/tags/release-1.0-RC5 Fixes since RC4: CONNECTORS-545 Fixes since RC3: CONNECTORS-544 -- Erlend Garåsen Center for Information Technology Services University of Oslo P.O. Box 1086 Blindern, N-0317 OSLO, Norway Ph: (+47) 22840193, Fax: (+47) 22852970, Mobile: (+47) 91380968, VIP: 31050
Re: [VOTE] Release Apache ManifoldCF 1.0, RC5
Hi Erlend, This is expected behavior. ManifoldCF is designed to retry on errors that mean potential server problems for a period of time and then give up. The logic here is that 500 may well mean the server is down but will be rebooted or whatever. It retries every N minutes until M minutes have elapsed. For a 500 error, I believe it's every 5 minutes for either 6 or 12 hours. Karl On Fri, Sep 28, 2012 at 5:49 AM, Erlend Garåsen e.f.gara...@usit.uio.no wrote: I'm trying to start a crawl before I have to run to the airport. I just discovered that MCF recrawls the same host over and over again when it returns result code 500: 09-28-2012 11:40:11.024 fetch http://foreninger.uio.no/go/oslo_open_2012_no.php 500 It's just not this document, but several others returning the same HTTP result code. Meanwhile, the following is filling up my log: FATAL 2012-09-28 11:42:32,112 (Worker thread '29') - Error tossed: String index out of range: -1 java.lang.StringIndexOutOfBoundsException: String index out of range: -1 I'm pretty sure they are related to each other. I will end this job before I leave because I'm afraid that MCF will try to fetch these documents over and over again during this weekend. Erlend On 28.09.12 09.58, Karl Wright wrote: Please vote +1 to release ManifoldCF 1.0, RC5. The release artifact can be found at: http://people.apache.org/~kwright/apache-manifoldcf-1.0 There is also an SVN tag at: https://svn.apache.org/repos/asf/manifoldcf/tags/release-1.0-RC5 Fixes since RC4: CONNECTORS-545 Fixes since RC3: CONNECTORS-544 -- Erlend Garåsen Center for Information Technology Services University of Oslo P.O. Box 1086 Blindern, N-0317 OSLO, Norway Ph: (+47) 22840193, Fax: (+47) 22852970, Mobile: (+47) 91380968, VIP: 31050
Re: [VOTE] Release Apache ManifoldCF 1.0, RC5
Meanwhile, the following is filling up my log: FATAL 2012-09-28 11:42:32,112 (Worker thread '29') - Error tossed: String index out of range: -1 java.lang.StringIndexOutOfBoundsException: String index out of range: -1 This is indeed a problem I agree we should fix, but in order to do that I need a stack trace. It is not clear at all that it is related to the 500 error you described before, but it could be. I will create a ticket for it though. Karl On Fri, Sep 28, 2012 at 5:49 AM, Erlend Garåsen e.f.gara...@usit.uio.no wrote: I'm trying to start a crawl before I have to run to the airport. I just discovered that MCF recrawls the same host over and over again when it returns result code 500: 09-28-2012 11:40:11.024 fetch http://foreninger.uio.no/go/oslo_open_2012_no.php 500 It's just not this document, but several others returning the same HTTP result code. Meanwhile, the following is filling up my log: FATAL 2012-09-28 11:42:32,112 (Worker thread '29') - Error tossed: String index out of range: -1 java.lang.StringIndexOutOfBoundsException: String index out of range: -1 I'm pretty sure they are related to each other. I will end this job before I leave because I'm afraid that MCF will try to fetch these documents over and over again during this weekend. Erlend On 28.09.12 09.58, Karl Wright wrote: Please vote +1 to release ManifoldCF 1.0, RC5. The release artifact can be found at: http://people.apache.org/~kwright/apache-manifoldcf-1.0 There is also an SVN tag at: https://svn.apache.org/repos/asf/manifoldcf/tags/release-1.0-RC5 Fixes since RC4: CONNECTORS-545 Fixes since RC3: CONNECTORS-544 -- Erlend Garåsen Center for Information Technology Services University of Oslo P.O. Box 1086 Blindern, N-0317 OSLO, Norway Ph: (+47) 22840193, Fax: (+47) 22852970, Mobile: (+47) 91380968, VIP: 31050
Re: [VOTE] Release Apache ManifoldCF 1.0, RC5
OK, I will give you a stack trace in the beginning of next week. I will start the crawler once more and check the results when I'm back and change my vote then if it is ok. Erlend On 28.09.12 13.26, Karl Wright wrote: Meanwhile, the following is filling up my log: FATAL 2012-09-28 11:42:32,112 (Worker thread '29') - Error tossed: String index out of range: -1 java.lang.StringIndexOutOfBoundsException: String index out of range: -1 This is indeed a problem I agree we should fix, but in order to do that I need a stack trace. It is not clear at all that it is related to the 500 error you described before, but it could be. I will create a ticket for it though. Karl On Fri, Sep 28, 2012 at 5:49 AM, Erlend Garåsen e.f.gara...@usit.uio.no wrote: I'm trying to start a crawl before I have to run to the airport. I just discovered that MCF recrawls the same host over and over again when it returns result code 500: 09-28-2012 11:40:11.024 fetch http://foreninger.uio.no/go/oslo_open_2012_no.php 500 It's just not this document, but several others returning the same HTTP result code. Meanwhile, the following is filling up my log: FATAL 2012-09-28 11:42:32,112 (Worker thread '29') - Error tossed: String index out of range: -1 java.lang.StringIndexOutOfBoundsException: String index out of range: -1 I'm pretty sure they are related to each other. I will end this job before I leave because I'm afraid that MCF will try to fetch these documents over and over again during this weekend. Erlend On 28.09.12 09.58, Karl Wright wrote: Please vote +1 to release ManifoldCF 1.0, RC5. The release artifact can be found at: http://people.apache.org/~kwright/apache-manifoldcf-1.0 There is also an SVN tag at: https://svn.apache.org/repos/asf/manifoldcf/tags/release-1.0-RC5 Fixes since RC4: CONNECTORS-545 Fixes since RC3: CONNECTORS-544 -- Erlend Garåsen Center for Information Technology Services University of Oslo P.O. Box 1086 Blindern, N-0317 OSLO, Norway Ph: (+47) 22840193, Fax: (+47) 22852970, Mobile: (+47) 91380968, VIP: 31050 -- Erlend Garåsen Center for Information Technology Services University of Oslo P.O. Box 1086 Blindern, N-0317 OSLO, Norway Ph: (+47) 22840193, Fax: (+47) 22852970, Mobile: (+47) 91380968, VIP: 31050
Re: [VOTE] Release Apache ManifoldCF 1.0, RC5
CONNECTORS-547 (index out of bounds) CONNECTORS-548 (cannot build with maven) Karl On Fri, Sep 28, 2012 at 7:26 AM, Karl Wright daddy...@gmail.com wrote: Meanwhile, the following is filling up my log: FATAL 2012-09-28 11:42:32,112 (Worker thread '29') - Error tossed: String index out of range: -1 java.lang.StringIndexOutOfBoundsException: String index out of range: -1 This is indeed a problem I agree we should fix, but in order to do that I need a stack trace. It is not clear at all that it is related to the 500 error you described before, but it could be. I will create a ticket for it though. Karl On Fri, Sep 28, 2012 at 5:49 AM, Erlend Garåsen e.f.gara...@usit.uio.no wrote: I'm trying to start a crawl before I have to run to the airport. I just discovered that MCF recrawls the same host over and over again when it returns result code 500: 09-28-2012 11:40:11.024 fetch http://foreninger.uio.no/go/oslo_open_2012_no.php 500 It's just not this document, but several others returning the same HTTP result code. Meanwhile, the following is filling up my log: FATAL 2012-09-28 11:42:32,112 (Worker thread '29') - Error tossed: String index out of range: -1 java.lang.StringIndexOutOfBoundsException: String index out of range: -1 I'm pretty sure they are related to each other. I will end this job before I leave because I'm afraid that MCF will try to fetch these documents over and over again during this weekend. Erlend On 28.09.12 09.58, Karl Wright wrote: Please vote +1 to release ManifoldCF 1.0, RC5. The release artifact can be found at: http://people.apache.org/~kwright/apache-manifoldcf-1.0 There is also an SVN tag at: https://svn.apache.org/repos/asf/manifoldcf/tags/release-1.0-RC5 Fixes since RC4: CONNECTORS-545 Fixes since RC3: CONNECTORS-544 -- Erlend Garåsen Center for Information Technology Services University of Oslo P.O. Box 1086 Blindern, N-0317 OSLO, Norway Ph: (+47) 22840193, Fax: (+47) 22852970, Mobile: (+47) 91380968, VIP: 31050
