Re: Problem with curl_multi C API

[Etienne Bougoin]

Le 18 mars 10 à 14:33, Etienne Bougoin a écrit :

Hello,

I have a problem with curl multi API on several platforms.

I try to fetch map tiles from open street map and I use curl_multi API to fetch them faster.

My problem is when one or more request return 404 error, the other request doen't work and images are not downloaded correctly.

I use this code to generate easy_handles:

  // Initialize curl handle
  CURL * lEasyHandle;
  lEasyHandle = curl_easy_init();

  if (!lEasyHandle)
  {

itkExceptionMacro( <<"Tile Map IO : Curl easy handle init error.");

  }

  // Create file
  FILE* lOutputFile = fopen(tileInfo.filename.c_str(), "w");

  if (lOutputFile == NULL)
  {
    itkExceptionMacro(<<"TileMap read : bad file name.");
  }

  // Add file to vector
  m_ListFiles.push_back(lOutputFile);

  // Param easy handle
  curl_easy_setopt(lEasyHandle, CURLOPT_USERAGENT, m_Browser.data());

curl_easy_setopt(lEasyHandle, CURLOPT_URL, m_ListURLs.back().data()); curl_easy_setopt(lEasyHandle, CURLOPT_WRITEDATA, m_ListFiles.back());

  // Add easy handle to multi handle
  curl_multi_add_handle(m_MultiHandle, lEasyHandle);

  // Add hanle to vector
  m_ListCurlHandles.push_back(lEasyHandle);

And this one to perform requests:

  // Configure multi handle - set the maximum connections
  curl_multi_setopt(m_MultiHandle, CURLMOPT_MAXCONNECTS, 10);
  curl_multi_setopt(m_MultiHandle, CURLMOPT_PIPELINING, 0);

    // Perform
 int lStillRunning;

  while(

CURLM_CALL_MULTI_PERFORM == curl_multi_perform(m_MultiHandle, &lStillRunning)

  );

  // Now get that URL
  while(lStillRunning)
  {
    struct timeval timeout;
    int rc; // Return code

    fd_set fdread;
    fd_set fdwrite;
    fd_set fdexcep;
    int maxfd;

    FD_ZERO(&fdread);
    FD_ZERO(&fdwrite);
    FD_ZERO(&fdexcep);

    /* set a suitable timeout to play around with */
    timeout.tv_sec = 0;
    timeout.tv_usec = 1;

    /* get file descriptors from the transfers */

curl_multi_fdset(m_MultiHandle, &fdread, &fdwrite, &fdexcep, &maxfd);

    rc = select(maxfd+1, &fdread, &fdwrite, &fdexcep, &timeout);

    switch(rc) {
    case -1:
      /* select error */
      break;
    case 0:
      /* timeout */
    default:
      /* timeout or readable/writable sockets */
      while(

CURLM_CALL_MULTI_PERFORM == curl_multi_perform(m_MultiHandle, &lStillRunning)

      );
      break;
    }
  }

  int remaining_msgs = 1;
  int error = 0;
  CURLMsg *msg;
  while (remaining_msgs)
  {
    msg = curl_multi_info_read(m_MultiHandle, &remaining_msgs);
    if (msg != NULL)
    {
      if (CURLE_OK != msg->data.result)
        error = 1;
    }
  }

Can you help me to find where I do something wrong?

Many thanks

Etienne
-------------------------------------------------------------------
List admin: http://cool.haxx.se/list/listinfo/curl-library
Etiquette:  http://curl.haxx.se/mail/etiquette.html

I forgot to specify on wich platforms I make the test:

- Mac OSX 10.5.8
        - curl v7.16.3
        - curl v7.19.4
        - curl v7.20.0

- CENT OS
        - curl v7.15.5

- Debian Lenny
        - curl v7.18.2

But it works fine on this platform:

- Ubuntu v8.04
        - curl v7.18.0

Etienne


-------------------------------------------------------------------
List admin: http://cool.haxx.se/list/listinfo/curl-library
Etiquette:  http://curl.haxx.se/mail/etiquette.html