Hi Raj, I can't seem to find an issue crawling that site, but maybe your parser is hanging. It is usually the case when 'hanging' threads are detected. You can also increase -Dmapred.task.timeout=, it controls how long it waits before giving up on hanging threads.
Also check your logs, there can be a hint in there, such as a GC issue, or whatever. Regards, Markus Op wo 1 jan 2025 om 15:26 schreef Raj Chidara <[email protected]>: > Hi > > I have problem in crawling and fetching this site > https://www.titck.gov.tr/ . It is either crawling same page again and > again and some times I get an error that Thread #0 hung while processing > https://www.titck.gov.tr/. Can you please help me out. > > Thanks and Regards > > Raj Chidara > > > > > Global Locations: > > USA | UK | India | Singapore | Japan > > *ISO 9001, 27001, 13485 Compliant > > www.DDIsmart.com > > About Us | Awards | Blog | News | Contact Us > > > > > > > > DISCLAIMER: This message is intended solely for the use of the individual > or entity to which it is addressed. If you are not the intended recipient, > you should not use, copy, alter, or disclose the contents of this message. > All information or opinions expressed in this message and/or any > attachments are those of the author and are not necessarily those of the > group companies. > > > > >
