[google-appengine] Call to URLFetch failed with application error 5 for url x

[Flav Ible]

*Background:*

I am using google app engine and am having a weird bug in my site crawler.

I have a backend that will automatically crawl a site every night. This is 
instigated by a task pushed to a pushQueue due to time limits in php.

*Problem:*

When I manually run the script that creates the task, the task completes as 
expected with no errors. However when cron launches the task I get the 
following error.

Call to URLFetch failed with application error 5 for url x

Code:

function url_get_contents ($Url) {
    global $retry;
    try {
        if (!function_exists('curl_init')){ 
            die('CURL is not installed!');
        }
        $ch = curl_init();
        curl_setopt($ch, CURLOPT_URL, $Url);
        curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
        $output = curl_exec($ch);
        curl_close($ch);
        return $output;
    } catch (Exception $e) {
        syslog(LOG_INFO, 'Caught exception: ',  $e->getMessage());
        if($retry > 0){
            $retry -= 1;
            return url_get_contents($Url);
        }
        else{
            return null;
        }
    }}

Thanks to syslog I can see that the $url is fine which is driving me crazy 
as it works when the exact same script is launched manually not through 
cron.

How can I fix this?

Thanks in advance.

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at http://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/677e26b9-79b0-4de6-adff-585fb679fa9f%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.