Re: Incomplete URL requests resulting in Excceptions
Hi, we have a similar problem... we are hosting approximately 300 websites that is using our tapestry application to which all the content are red from the database and build up on the fly. We also gets a lot of 'ghost' exceptions when search engine spiders and robots try to access our application. Our application sends us a error email every time an exception occurs in the application and that means at least around a 100 emails a day. I also noticed that we get a lot of pageNotFoundException for page names that do not exists in our application name space... is this normal? How do you prevent the search engines to do this? Thanks in advance for any help, Jacob -- View this message in context: http://www.nabble.com/Incomplete-URL-requests-resulting-in-Excceptions-tf3567555.html#a9984430 Sent from the Tapestry - User mailing list archive at Nabble.com. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Incomplete URL requests resulting in Excceptions
I'm not as sure about all of the methods for preventing search engine spidering on certain parts of your app but I have built lots of application / etc monitoring software in the past and you almost always want these kinds of alerts to go through some kind of intelligent filter that prevents 60 emails coming out about the exact same non-issue . On 4/13/07, jake123 [EMAIL PROTECTED] wrote: Hi, we have a similar problem... we are hosting approximately 300 websites that is using our tapestry application to which all the content are red from the database and build up on the fly. We also gets a lot of 'ghost' exceptions when search engine spiders and robots try to access our application. Our application sends us a error email every time an exception occurs in the application and that means at least around a 100 emails a day. I also noticed that we get a lot of pageNotFoundException for page names that do not exists in our application name space... is this normal? How do you prevent the search engines to do this? Thanks in advance for any help, Jacob -- View this message in context: http://www.nabble.com/Incomplete-URL-requests-resulting-in-Excceptions-tf3567555.html#a9984430 Sent from the Tapestry - User mailing list archive at Nabble.com. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] -- Jesse Kuhnert Tapestry/Dojo team member/developer Open source based consulting work centered around dojo/tapestry/tacos/hivemind. http://blog.opencomponentry.com
Re: Incomplete URL requests resulting in Excceptions
So this is happening for you to? If this is the normal behavior for the search engines then I guess we can filter the results, but we only want to do this if this is the normal thing... otherwise wants to fix it the 'right' way.. Thanks again, Jacob -- View this message in context: http://www.nabble.com/Incomplete-URL-requests-resulting-in-Excceptions-tf3567555.html#a9986177 Sent from the Tapestry - User mailing list archive at Nabble.com. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Incomplete URL requests resulting in Excceptions
Not all of them are going to be well behaved web spiders. Exploit scanners tend to hit specific URL suffixes to feed in their exploit code (looking for vulnerable phpbb, phpnuke, etc), and they don't respond to robots.txt ;) It clogs up traditional Apache error logs as well. I would suggest simply filtering the error emails. jake123 wrote: Hi, we have a similar problem... we are hosting approximately 300 websites that is using our tapestry application to which all the content are red from the database and build up on the fly. We also gets a lot of 'ghost' exceptions when search engine spiders and robots try to access our application. Our application sends us a error email every time an exception occurs in the application and that means at least around a 100 emails a day. I also noticed that we get a lot of pageNotFoundException for page names that do not exists in our application name space... is this normal? How do you prevent the search engines to do this? Thanks in advance for any help, Jacob - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
