Hi all,
I have a question regarding the Web connector. I want to crawl a website
which responses with compressed HTTP. The returned HTTP response header
is content-encoding:gzip.
Is Manifold able to handle such compressed websites? If I try to crawl
the website there is a 404 displayed in the history. But from the
machine the site is accessable and if I try to crawl a non compressed
website, all works fine. So the 404 is maybe missleading.
Can I set the accepted-encoding header for the Web connector repo to
avoid getting compressed content?
Or is there any other solution?
Regards
Sebastian
