Re: [pacman-dev] [PATCH] Remove AC_FUNC_MALLOC check.

2016-08-30 Thread Allan McRae
On 31/08/16 11:13, Alastair Hughes wrote:
> We weren't supplying the rpl_malloc function needed if this failed, and
> didn't check for realloc, so just remove.
> 
> Signed-off-by: Alastair Hughes 
> ---
> 
> The build will currently fail if cross compiling or if
> ac_cv_func_malloc_0_nonnull=no is set, because we don't supply a rpl_malloc
> function anywhere. As we don't use AC_FUNC_REALLOC either, just assume that we
> will never encounter a buggy system and remove the check.
> 
> The other option appears to be to provide a malloc definition somewhere, but 
> in
> that case it appears that we should also check for realloc.
> 
> I'm unsure which one should be used - opinions?

We are not providing a malloc definition...   So this patch is fine.

Allan


Re: [pacman-dev] [PATCH] Replace CURLOPT_PROGRESSFUNCTION with CURLOPT_XFERINFOFUNCTION

2016-08-30 Thread Allan McRae
On 31/08/16 06:33, ivy.fos...@gmail.com wrote:
> From: Ivy Foster 
> 
> Curl 7.32.0 added CURLOPT_XFERINFOFUNCTION, which deprecates
> CURLOPT_PROGRESSFUNCTION and means less casting doubles to size_ts for
> alpm. This change has no user-facing nor frontend-facing effects.
> 

OK.   I don't think pacman is used anywhere where curl older than August
2013 is an issue...

Allan


[pacman-dev] [PATCH] Remove AC_FUNC_MALLOC check.

2016-08-30 Thread Alastair Hughes
We weren't supplying the rpl_malloc function needed if this failed, and
didn't check for realloc, so just remove.

Signed-off-by: Alastair Hughes 
---

The build will currently fail if cross compiling or if
ac_cv_func_malloc_0_nonnull=no is set, because we don't supply a rpl_malloc
function anywhere. As we don't use AC_FUNC_REALLOC either, just assume that we
will never encounter a buggy system and remove the check.

The other option appears to be to provide a malloc definition somewhere, but in
that case it appears that we should also check for realloc.

I'm unsure which one should be used - opinions?

 configure.ac | 1 -
 1 file changed, 1 deletion(-)

diff --git a/configure.ac b/configure.ac
index dd4ac04..f2d3b75 100644
--- a/configure.ac
+++ b/configure.ac
@@ -308,7 +308,6 @@ PATH_MAX_DEFINED
 AC_FUNC_FORK
 AC_FUNC_GETMNTENT
 AC_FUNC_LSTAT_FOLLOWS_SLASHED_SYMLINK
-AC_FUNC_MALLOC
 AC_FUNC_MKTIME
 AC_FUNC_STRCOLL
 AC_CHECK_FUNCS([dup2 getcwd getmntinfo gettimeofday memmove memset \
-- 
2.9.3


[pacman-dev] [PATCH v2] Replace CURLOPT_PROGRESSFUNCTION with

2016-08-30 Thread ivy . foster
On Tue, Aug 30, 2016 at 02:54:31PM -0500, Dave Reisner  
wrote:
> From: Ivy Foster 
> > Curl 7.32.0 added CURLOPT_XFERINFOFUNCTION

> Making this change would require that you update configure.ac to bump
> the minimum version for libcurl (which is currently 7.19.4). FWIW 7.32.0
> was released in August 2013.

Ah, okay, that didn't even occur to me (clearly). Thanks for the
feedback. This version of the patch includes the updated. configure.ac
check.

iff


[pacman-dev] [PATCH] Replace CURLOPT_PROGRESSFUNCTION with CURLOPT_XFERINFOFUNCTION

2016-08-30 Thread ivy . foster
From: Ivy Foster 

Curl 7.32.0 added CURLOPT_XFERINFOFUNCTION, which deprecates
CURLOPT_PROGRESSFUNCTION and means less casting doubles to size_ts for
alpm. This change has no user-facing nor frontend-facing effects.

Signed-off-by: Ivy Foster 
---
 configure.ac|  4 ++--
 lib/libalpm/dload.c | 18 +-
 2 files changed, 11 insertions(+), 11 deletions(-)

diff --git a/configure.ac b/configure.ac
index 51288e7..d4a1c9f 100644
--- a/configure.ac
+++ b/configure.ac
@@ -232,10 +232,10 @@ AM_CONDITIONAL(HAVE_LIBSSL, [test "$have_openssl" = 
"yes"])
 # Check for libcurl
 have_libcurl=no
 if test "x$with_libcurl" != "xno"; then
-   PKG_CHECK_MODULES(LIBCURL, [libcurl >= 7.19.4],
+   PKG_CHECK_MODULES(LIBCURL, [libcurl >= 7.32.0],
[AC_DEFINE(HAVE_LIBCURL, 1, [Define if libcurl is available]) 
have_libcurl=yes], have_libcurl=no)
if test "x$have_libcurl" = xno -a "x$with_libcurl" = xyes; then
-   AC_MSG_ERROR([*** libcurl >= 7.19.4 is required for internal 
downloader support])
+   AC_MSG_ERROR([*** libcurl >= 7.32.0 is required for internal 
downloader support])
fi
 fi
 AM_CONDITIONAL(HAVE_LIBCURL, [test "$have_libcurl" = "yes"])
diff --git a/lib/libalpm/dload.c b/lib/libalpm/dload.c
index f4e6a27..dc57c92 100644
--- a/lib/libalpm/dload.c
+++ b/lib/libalpm/dload.c
@@ -90,8 +90,8 @@ static void inthandler(int UNUSED signum)
dload_interrupted = ABORT_SIGINT;
 }
 
-static int dload_progress_cb(void *file, double dltotal, double dlnow,
-   double UNUSED ultotal, double UNUSED ulnow)
+static int dload_progress_cb(void *file, curl_off_t dltotal, curl_off_t dlnow,
+   curl_off_t UNUSED ultotal, curl_off_t UNUSED ulnow)
 {
struct dload_payload *payload = (struct dload_payload *)file;
off_t current_size, total_size;
@@ -106,7 +106,7 @@ static int dload_progress_cb(void *file, double dltotal, 
double dlnow,
return 1;
}
 
-   current_size = payload->initial_size + (off_t)dlnow;
+   current_size = payload->initial_size + dlnow;
 
/* is our filesize still under any set limit? */
if(payload->max_size && current_size > payload->max_size) {
@@ -119,9 +119,9 @@ static int dload_progress_cb(void *file, double dltotal, 
double dlnow,
return 0;
}
 
-   total_size = payload->initial_size + (off_t)dltotal;
+   total_size = payload->initial_size + dltotal;
 
-   if(DOUBLE_EQ(dltotal, 0.0) || payload->prevprogress == total_size) {
+   if(dltotal == 0 || payload->prevprogress == total_size) {
return 0;
}
 
@@ -134,7 +134,7 @@ static int dload_progress_cb(void *file, double dltotal, 
double dlnow,
 * x {x>0}, x: download complete
 * x {x>0, x 0}: download progress, expected total is known 
*/
if(current_size == total_size) {
-   payload->handle->dlcb(payload->remote_name, (off_t)dlnow, 
(off_t)dltotal);
+   payload->handle->dlcb(payload->remote_name, dlnow, dltotal);
} else if(!payload->prevprogress) {
payload->handle->dlcb(payload->remote_name, 0, -1);
} else if(payload->prevprogress == current_size) {
@@ -142,7 +142,7 @@ static int dload_progress_cb(void *file, double dltotal, 
double dlnow,
} else {
/* do NOT include initial_size since it wasn't part of the package's
 * download_size (nor included in the total download size callback) */
-   payload->handle->dlcb(payload->remote_name, (off_t)dlnow, 
(off_t)dltotal);
+   payload->handle->dlcb(payload->remote_name, dlnow, dltotal);
}
 
payload->prevprogress = current_size;
@@ -303,8 +303,8 @@ static void curl_set_handle_opts(struct dload_payload 
*payload,
curl_easy_setopt(curl, CURLOPT_FILETIME, 1L);
curl_easy_setopt(curl, CURLOPT_NOPROGRESS, 0L);
curl_easy_setopt(curl, CURLOPT_FOLLOWLOCATION, 1L);
-   curl_easy_setopt(curl, CURLOPT_PROGRESSFUNCTION, dload_progress_cb);
-   curl_easy_setopt(curl, CURLOPT_PROGRESSDATA, (void *)payload);
+   curl_easy_setopt(curl, CURLOPT_XFERINFOFUNCTION, dload_progress_cb);
+   curl_easy_setopt(curl, CURLOPT_XFERINFODATA, (void *)payload);
curl_easy_setopt(curl, CURLOPT_LOW_SPEED_LIMIT, 1L);
curl_easy_setopt(curl, CURLOPT_LOW_SPEED_TIME, 10L);
curl_easy_setopt(curl, CURLOPT_HEADERFUNCTION, dload_parseheader_cb);
-- 
2.9.3


Re: [pacman-dev] [PATCH v2] Make DUFLAGS be overrideable during configure

2016-08-30 Thread Alastair Hughes
On 8/30/16, Allan McRae  wrote:
> On 20/08/16 06:52, Alastair Hughes wrote:
>> Not all du implementations on linux accept --apparent-size, so let the
>> user configure the arguments passed to du if required.
>>
>> This fixes FS#47943.
>>
>> Signed-off-by: Alastair Hughes 
>> ---
>
> Why this approach over the previous style of:
>
> DUFLAGS="${DUFLAGS- -sk}"
>
> ?

I decided that the prior approach was messy due to the new default in
the case statement. I thought that it seemed less messy and more
consistent to use the case statement to set a default and then use
that only if DUFLAGS was not set.

Looking at this again, I realize that I've moved some things around
more than they should be. I apologize for the messiness :( - I'll try
to be more careful. Thanks for looking at this!

Alastair Hughes


Re: [pacman-dev] [PATCH] Replace CURLOPT_PROGRESSFUNCTION with CURLOPT_XFERINFOFUNCTION

2016-08-30 Thread Dave Reisner
On Tue, Aug 30, 2016 at 02:54:31PM -0500, ivy.fos...@gmail.com wrote:
> From: Ivy Foster 
> 
> Curl 7.32.0 added CURLOPT_XFERINFOFUNCTION, which deprecates
> CURLOPT_PROGRESSFUNCTION and means less casting doubles to size_ts for
> alpm. This change has no user-facing nor frontend-facing effects.

Making this change would require that you update configure.ac to bump
the minimum version for libcurl (which is currently 7.19.4). FWIW 7.32.0
was released in August 2013.

> Signed-off-by: Ivy Foster 
> ---
>  lib/libalpm/dload.c | 18 +-
>  1 file changed, 9 insertions(+), 9 deletions(-)
> 
> diff --git a/lib/libalpm/dload.c b/lib/libalpm/dload.c
> index f4e6a27..dc57c92 100644
> --- a/lib/libalpm/dload.c
> +++ b/lib/libalpm/dload.c
> @@ -90,8 +90,8 @@ static void inthandler(int UNUSED signum)
>   dload_interrupted = ABORT_SIGINT;
>  }
>  
> -static int dload_progress_cb(void *file, double dltotal, double dlnow,
> - double UNUSED ultotal, double UNUSED ulnow)
> +static int dload_progress_cb(void *file, curl_off_t dltotal, curl_off_t 
> dlnow,
> + curl_off_t UNUSED ultotal, curl_off_t UNUSED ulnow)
>  {
>   struct dload_payload *payload = (struct dload_payload *)file;
>   off_t current_size, total_size;
> @@ -106,7 +106,7 @@ static int dload_progress_cb(void *file, double dltotal, 
> double dlnow,
>   return 1;
>   }
>  
> - current_size = payload->initial_size + (off_t)dlnow;
> + current_size = payload->initial_size + dlnow;
>  
>   /* is our filesize still under any set limit? */
>   if(payload->max_size && current_size > payload->max_size) {
> @@ -119,9 +119,9 @@ static int dload_progress_cb(void *file, double dltotal, 
> double dlnow,
>   return 0;
>   }
>  
> - total_size = payload->initial_size + (off_t)dltotal;
> + total_size = payload->initial_size + dltotal;
>  
> - if(DOUBLE_EQ(dltotal, 0.0) || payload->prevprogress == total_size) {
> + if(dltotal == 0 || payload->prevprogress == total_size) {
>   return 0;
>   }
>  
> @@ -134,7 +134,7 @@ static int dload_progress_cb(void *file, double dltotal, 
> double dlnow,
>* x {x>0}, x: download complete
>* x {x>0, x 0}: download progress, expected total is known 
> */
>   if(current_size == total_size) {
> - payload->handle->dlcb(payload->remote_name, (off_t)dlnow, 
> (off_t)dltotal);
> + payload->handle->dlcb(payload->remote_name, dlnow, dltotal);
>   } else if(!payload->prevprogress) {
>   payload->handle->dlcb(payload->remote_name, 0, -1);
>   } else if(payload->prevprogress == current_size) {
> @@ -142,7 +142,7 @@ static int dload_progress_cb(void *file, double dltotal, 
> double dlnow,
>   } else {
>   /* do NOT include initial_size since it wasn't part of the package's
>* download_size (nor included in the total download size callback) */
> - payload->handle->dlcb(payload->remote_name, (off_t)dlnow, 
> (off_t)dltotal);
> + payload->handle->dlcb(payload->remote_name, dlnow, dltotal);
>   }
>  
>   payload->prevprogress = current_size;
> @@ -303,8 +303,8 @@ static void curl_set_handle_opts(struct dload_payload 
> *payload,
>   curl_easy_setopt(curl, CURLOPT_FILETIME, 1L);
>   curl_easy_setopt(curl, CURLOPT_NOPROGRESS, 0L);
>   curl_easy_setopt(curl, CURLOPT_FOLLOWLOCATION, 1L);
> - curl_easy_setopt(curl, CURLOPT_PROGRESSFUNCTION, dload_progress_cb);
> - curl_easy_setopt(curl, CURLOPT_PROGRESSDATA, (void *)payload);
> + curl_easy_setopt(curl, CURLOPT_XFERINFOFUNCTION, dload_progress_cb);
> + curl_easy_setopt(curl, CURLOPT_XFERINFODATA, (void *)payload);
>   curl_easy_setopt(curl, CURLOPT_LOW_SPEED_LIMIT, 1L);
>   curl_easy_setopt(curl, CURLOPT_LOW_SPEED_TIME, 10L);
>   curl_easy_setopt(curl, CURLOPT_HEADERFUNCTION, dload_parseheader_cb);
> -- 
> 2.9.3


[pacman-dev] [PATCH] Replace CURLOPT_PROGRESSFUNCTION with CURLOPT_XFERINFOFUNCTION

2016-08-30 Thread ivy . foster
From: Ivy Foster 

Curl 7.32.0 added CURLOPT_XFERINFOFUNCTION, which deprecates
CURLOPT_PROGRESSFUNCTION and means less casting doubles to size_ts for
alpm. This change has no user-facing nor frontend-facing effects.

Signed-off-by: Ivy Foster 
---
 lib/libalpm/dload.c | 18 +-
 1 file changed, 9 insertions(+), 9 deletions(-)

diff --git a/lib/libalpm/dload.c b/lib/libalpm/dload.c
index f4e6a27..dc57c92 100644
--- a/lib/libalpm/dload.c
+++ b/lib/libalpm/dload.c
@@ -90,8 +90,8 @@ static void inthandler(int UNUSED signum)
dload_interrupted = ABORT_SIGINT;
 }
 
-static int dload_progress_cb(void *file, double dltotal, double dlnow,
-   double UNUSED ultotal, double UNUSED ulnow)
+static int dload_progress_cb(void *file, curl_off_t dltotal, curl_off_t dlnow,
+   curl_off_t UNUSED ultotal, curl_off_t UNUSED ulnow)
 {
struct dload_payload *payload = (struct dload_payload *)file;
off_t current_size, total_size;
@@ -106,7 +106,7 @@ static int dload_progress_cb(void *file, double dltotal, 
double dlnow,
return 1;
}
 
-   current_size = payload->initial_size + (off_t)dlnow;
+   current_size = payload->initial_size + dlnow;
 
/* is our filesize still under any set limit? */
if(payload->max_size && current_size > payload->max_size) {
@@ -119,9 +119,9 @@ static int dload_progress_cb(void *file, double dltotal, 
double dlnow,
return 0;
}
 
-   total_size = payload->initial_size + (off_t)dltotal;
+   total_size = payload->initial_size + dltotal;
 
-   if(DOUBLE_EQ(dltotal, 0.0) || payload->prevprogress == total_size) {
+   if(dltotal == 0 || payload->prevprogress == total_size) {
return 0;
}
 
@@ -134,7 +134,7 @@ static int dload_progress_cb(void *file, double dltotal, 
double dlnow,
 * x {x>0}, x: download complete
 * x {x>0, x 0}: download progress, expected total is known 
*/
if(current_size == total_size) {
-   payload->handle->dlcb(payload->remote_name, (off_t)dlnow, 
(off_t)dltotal);
+   payload->handle->dlcb(payload->remote_name, dlnow, dltotal);
} else if(!payload->prevprogress) {
payload->handle->dlcb(payload->remote_name, 0, -1);
} else if(payload->prevprogress == current_size) {
@@ -142,7 +142,7 @@ static int dload_progress_cb(void *file, double dltotal, 
double dlnow,
} else {
/* do NOT include initial_size since it wasn't part of the package's
 * download_size (nor included in the total download size callback) */
-   payload->handle->dlcb(payload->remote_name, (off_t)dlnow, 
(off_t)dltotal);
+   payload->handle->dlcb(payload->remote_name, dlnow, dltotal);
}
 
payload->prevprogress = current_size;
@@ -303,8 +303,8 @@ static void curl_set_handle_opts(struct dload_payload 
*payload,
curl_easy_setopt(curl, CURLOPT_FILETIME, 1L);
curl_easy_setopt(curl, CURLOPT_NOPROGRESS, 0L);
curl_easy_setopt(curl, CURLOPT_FOLLOWLOCATION, 1L);
-   curl_easy_setopt(curl, CURLOPT_PROGRESSFUNCTION, dload_progress_cb);
-   curl_easy_setopt(curl, CURLOPT_PROGRESSDATA, (void *)payload);
+   curl_easy_setopt(curl, CURLOPT_XFERINFOFUNCTION, dload_progress_cb);
+   curl_easy_setopt(curl, CURLOPT_XFERINFODATA, (void *)payload);
curl_easy_setopt(curl, CURLOPT_LOW_SPEED_LIMIT, 1L);
curl_easy_setopt(curl, CURLOPT_LOW_SPEED_TIME, 10L);
curl_easy_setopt(curl, CURLOPT_HEADERFUNCTION, dload_parseheader_cb);
-- 
2.9.3


[pacman-dev] [PATCH 1/1] add command line options for libcurl's "low speed" timeout

2016-08-30 Thread Christian Hesse
From: Christian Hesse 

Signed-off-by: Christian Hesse 
---
 doc/pacman.8.txt| 13 +
 src/pacman/conf.h   |  4 +++-
 src/pacman/pacman.c | 14 ++
 3 files changed, 30 insertions(+), 1 deletion(-)

diff --git a/doc/pacman.8.txt b/doc/pacman.8.txt
index 231e0bc..83f63e6 100644
--- a/doc/pacman.8.txt
+++ b/doc/pacman.8.txt
@@ -266,6 +266,19 @@ Upgrade Options (apply to '-S' and '-U')[[UO]]
 *\--needed*::
Do not reinstall the targets that are already up-to-date.
 
+*\--lowspeedlimit* ::
+Sets the speed in bytes per second that a download should be below during
+`LowSpeedTime` seconds to abort the transfer for being too slow. Setting
+'speed' to 0 will disable the speed check. Defaults to 1 byte per second.
+Note that this option will not affect external programs specified by
+`XferCommand`.
+
+*\--lowspeedlimit* ::
+Sets the time in seconds that a download should be below the 
`LowSpeedLimit`
+transfer speed to abort the transfer for being too slow. Setting 'time' to
+0 will disable the speed check. Defaults to 10 seconds. Note that this
+option will not affect external programs specified by `XferCommand`.
+
 
 Query Options (apply to '-Q')[[QO]]
 ---
diff --git a/src/pacman/conf.h b/src/pacman/conf.h
index c61ed2a..fe296cc 100644
--- a/src/pacman/conf.h
+++ b/src/pacman/conf.h
@@ -205,7 +205,9 @@ enum {
OP_VERBOSE,
OP_DOWNLOADONLY,
OP_REFRESH,
-   OP_ASSUMEINSTALLED
+   OP_ASSUMEINSTALLED,
+   OP_LOWSPEEDLIMIT,
+   OP_LOWSPEEDTIME
 };
 
 /* clean method */
diff --git a/src/pacman/pacman.c b/src/pacman/pacman.c
index be52d1b..f6a5f33 100644
--- a/src/pacman/pacman.c
+++ b/src/pacman/pacman.c
@@ -194,6 +194,12 @@ static void usage(int op, const char * const myname)
addlist(_("  --ignoreignore a 
package upgrade (can be used more than once)\n"));
addlist(_("  --ignoregroup \n"
  "   ignore a 
group upgrade (can be used more than once)\n"));
+#ifdef HAVE_LIBCURL
+   addlist(_("  --lowspeedlimit \n"
+ "   bytes per 
second that a download should be below\n"));
+   addlist(_("  --lowspeedtime \n"
+ "   time in 
seconds that a download should be below lowspeedlimit\n"));
+#endif
/* pass through */
case PM_OP_REMOVE:
addlist(_("  -d, --nodeps skip 
dependency version checks (-dd to skip all checks)\n"));
@@ -713,6 +719,12 @@ static int parsearg_upgrade(int opt)
case OP_IGNOREGROUP:
parsearg_util_addlist(&(config->ignoregrp));
break;
+   case OP_LOWSPEEDLIMIT:
+   config->lowspeedlimit = parse_positive_long(optarg);
+   break;
+   case OP_LOWSPEEDTIME:
+   config->lowspeedtime = parse_positive_long(optarg);
+   break;
default: return 1;
}
return 0;
@@ -928,6 +940,8 @@ static int parseargs(int argc, char *argv[])
{"logfile",required_argument, 0, OP_LOGFILE},
{"ignoregroup", required_argument, 0, OP_IGNOREGROUP},
{"needed", no_argument,   0, OP_NEEDED},
+   {"lowspeedlimit", required_argument, 0, OP_LOWSPEEDLIMIT},
+   {"lowspeedtime", required_argument, 0, OP_LOWSPEEDTIME},
{"asexplicit", no_argument,   0, OP_ASEXPLICIT},
{"arch",   required_argument, 0, OP_ARCH},
{"print-format", required_argument, 0, OP_PRINTFORMAT},
-- 
2.9.3


Re: [pacman-dev] [PATCH v6 1/1] Add configuration options for libcurl's "low speed" timeout

2016-08-30 Thread Christian Hesse
Dave Reisner  on Tue, 2016/08/30 08:46:
> On Tue, Aug 30, 2016 at 02:12:23PM +0200, Christian Hesse wrote:
> > From: Christian Hesse 
> > 
> > Add LowSpeedLimit and LowSpeedTime configuration options to correspond
> > to libcurl's CURLOPT_LOW_SPEED_LIMIT and CURLOPT_LOW_SPEED_TIME options.
> > This allows, e.g., transfers behind corporate virus-scanning firewalls
> > to survive the delays. Increasing the timeout may not be desirable in
> > all situations; similarly, disabling the check prevents detection of
> > disappearing networks.  
> 
> FWIW, I'm strongly opposed to having a 1:1 mapping between pacman
> options and curl config options. Please look at the bigger picture --
> it's dead connection detection. We might reimplement that in the future
> in some other way (e.g. via the progress callback or by some other
> transfer library entirely).
> 
> To that end, I think it would be reasonable to add a boolean toggle for
> the dead connection detection (default on). The patch in its current
> state makes me rather itchy from an API perspective.

That is what my stupid-proxy patch does...
Now it is up to Allan to decide. ;)
-- 
main(a){char*c=/*Schoene Gruesse */"B?IJj;MEH"
"CX:;",b;for(a/*Best regards my address:*/=0;b=c[a++];)
putchar(b-1/(/*Chriscc -ox -xc - && ./x*/b/42*2-3)*42);}


pgp5_R_bbzJTd.pgp
Description: OpenPGP digital signature


Re: [pacman-dev] [PATCH v6 1/1] Add configuration options for libcurl's "low speed" timeout

2016-08-30 Thread Dave Reisner
On Tue, Aug 30, 2016 at 02:12:23PM +0200, Christian Hesse wrote:
> From: Christian Hesse 
> 
> Add LowSpeedLimit and LowSpeedTime configuration options to correspond
> to libcurl's CURLOPT_LOW_SPEED_LIMIT and CURLOPT_LOW_SPEED_TIME options.
> This allows, e.g., transfers behind corporate virus-scanning firewalls
> to survive the delays. Increasing the timeout may not be desirable in
> all situations; similarly, disabling the check prevents detection of
> disappearing networks.

FWIW, I'm strongly opposed to having a 1:1 mapping between pacman
options and curl config options. Please look at the bigger picture --
it's dead connection detection. We might reimplement that in the future
in some other way (e.g. via the progress callback or by some other
transfer library entirely).

To that end, I think it would be reasonable to add a boolean toggle for
the dead connection detection (default on). The patch in its current
state makes me rather itchy from an API perspective.

> Adds a utility function, parse_positive_long(), which yields a positive
> (but not unsigned) long integer from a null-terminated string using
> strtol(). A string containing anything but a base-10 number will cause
> the function to return -1.
> 
> Original-Patch-by: Andrew Hills 
> Signed-off-by: Christian Hesse 
> ---
>  doc/pacman.conf.5.txt | 12 
>  etc/pacman.conf.in|  4 
>  lib/libalpm/alpm.h| 10 ++
>  lib/libalpm/dload.c   |  4 ++--
>  lib/libalpm/handle.c  | 32 
>  lib/libalpm/handle.h  |  9 +
>  src/pacman/conf.c | 24 
>  src/pacman/conf.h |  4 
>  src/pacman/util.c | 19 +++
>  src/pacman/util.h |  1 +
>  10 files changed, 117 insertions(+), 2 deletions(-)
> 
> diff --git a/doc/pacman.conf.5.txt b/doc/pacman.conf.5.txt
> index c665870..2346331 100644
> --- a/doc/pacman.conf.5.txt
> +++ b/doc/pacman.conf.5.txt
> @@ -209,6 +209,18 @@ Options
>   Displays name, version and size of target packages formatted
>   as a table for upgrade, sync and remove operations.
>  
> +*LowSpeedLimit* = speed::
> +Sets the speed in bytes per second that a download should be below during
> +`LowSpeedTime` seconds to abort the transfer for being too slow. Setting
> +'speed' to 0 will disable the speed check. Defaults to 1 byte per second.
> +Note that this option will not affect external programs specified by
> +`XferCommand`.
> +
> +*LowSpeedTime* = time::
> +Sets the time in seconds that a download should be below the 
> `LowSpeedLimit`
> +transfer speed to abort the transfer for being too slow. Setting 'time' 
> to
> +0 will disable the speed check. Defaults to 10 seconds. Note that this
> +option will not affect external programs specified by `XferCommand`.
>  
>  Repository Sections
>  ---
> diff --git a/etc/pacman.conf.in b/etc/pacman.conf.in
> index 53071e5..ddc7397 100644
> --- a/etc/pacman.conf.in
> +++ b/etc/pacman.conf.in
> @@ -29,6 +29,10 @@ Architecture = auto
>  #NoUpgrade   =
>  #NoExtract   =
>  
> +# Downloader options
> +#LowSpeedLimit = 1
> +#LowSpeedTime = 10
> +
>  # Misc options
>  #UseSyslog
>  #Color
> diff --git a/lib/libalpm/alpm.h b/lib/libalpm/alpm.h
> index 168d71b..c338119 100644
> --- a/lib/libalpm/alpm.h
> +++ b/lib/libalpm/alpm.h
> @@ -811,6 +811,16 @@ const char *alpm_option_get_dbpath(alpm_handle_t 
> *handle);
>  /** Get the name of the database lock file. Read-only. */
>  const char *alpm_option_get_lockfile(alpm_handle_t *handle);
>  
> +/** Returns the libcurl low speed limit in bytes per second. */
> +long alpm_option_get_download_lowspeedlimit(alpm_handle_t *handle);
> +/** Sets the libcurl low speed limit in bytes per second. */
> +int alpm_option_set_download_lowspeedlimit(alpm_handle_t *handle, long 
> lowspeedlimit);
> +
> +/** Returns the libcurl low speed time in seconds. */
> +long alpm_option_get_download_lowspeedtime(alpm_handle_t *handle);
> +/** Sets the libcurl low speed time in seconds. */
> +int alpm_option_set_download_lowspeedtime(alpm_handle_t *handle, long 
> lowspeedtime);
> +
>  /** @name Accessors to the list of package cache directories.
>   * @{
>   */
> diff --git a/lib/libalpm/dload.c b/lib/libalpm/dload.c
> index f4e6a27..7490dd6 100644
> --- a/lib/libalpm/dload.c
> +++ b/lib/libalpm/dload.c
> @@ -305,8 +305,8 @@ static void curl_set_handle_opts(struct dload_payload 
> *payload,
>   curl_easy_setopt(curl, CURLOPT_FOLLOWLOCATION, 1L);
>   curl_easy_setopt(curl, CURLOPT_PROGRESSFUNCTION, dload_progress_cb);
>   curl_easy_setopt(curl, CURLOPT_PROGRESSDATA, (void *)payload);
> - curl_easy_setopt(curl, CURLOPT_LOW_SPEED_LIMIT, 1L);
> - curl_easy_setopt(curl, CURLOPT_LOW_SPEED_TIME, 10L);
> + curl_easy_setopt(curl, CURLOPT_LOW_SPEED_LIMIT, 
> handle->download.lowspeedlimit);
> + curl_easy_setopt(curl, CURLOPT_LOW_SPEED_TIME, 
> handle->download.lowspeedtime);
>   curl_easy_se

[pacman-dev] [PATCH v6 1/1] Add configuration options for libcurl's "low speed" timeout

2016-08-30 Thread Christian Hesse
From: Christian Hesse 

Add LowSpeedLimit and LowSpeedTime configuration options to correspond
to libcurl's CURLOPT_LOW_SPEED_LIMIT and CURLOPT_LOW_SPEED_TIME options.
This allows, e.g., transfers behind corporate virus-scanning firewalls
to survive the delays. Increasing the timeout may not be desirable in
all situations; similarly, disabling the check prevents detection of
disappearing networks.

Adds a utility function, parse_positive_long(), which yields a positive
(but not unsigned) long integer from a null-terminated string using
strtol(). A string containing anything but a base-10 number will cause
the function to return -1.

Original-Patch-by: Andrew Hills 
Signed-off-by: Christian Hesse 
---
 doc/pacman.conf.5.txt | 12 
 etc/pacman.conf.in|  4 
 lib/libalpm/alpm.h| 10 ++
 lib/libalpm/dload.c   |  4 ++--
 lib/libalpm/handle.c  | 32 
 lib/libalpm/handle.h  |  9 +
 src/pacman/conf.c | 24 
 src/pacman/conf.h |  4 
 src/pacman/util.c | 19 +++
 src/pacman/util.h |  1 +
 10 files changed, 117 insertions(+), 2 deletions(-)

diff --git a/doc/pacman.conf.5.txt b/doc/pacman.conf.5.txt
index c665870..2346331 100644
--- a/doc/pacman.conf.5.txt
+++ b/doc/pacman.conf.5.txt
@@ -209,6 +209,18 @@ Options
Displays name, version and size of target packages formatted
as a table for upgrade, sync and remove operations.
 
+*LowSpeedLimit* = speed::
+Sets the speed in bytes per second that a download should be below during
+`LowSpeedTime` seconds to abort the transfer for being too slow. Setting
+'speed' to 0 will disable the speed check. Defaults to 1 byte per second.
+Note that this option will not affect external programs specified by
+`XferCommand`.
+
+*LowSpeedTime* = time::
+Sets the time in seconds that a download should be below the 
`LowSpeedLimit`
+transfer speed to abort the transfer for being too slow. Setting 'time' to
+0 will disable the speed check. Defaults to 10 seconds. Note that this
+option will not affect external programs specified by `XferCommand`.
 
 Repository Sections
 ---
diff --git a/etc/pacman.conf.in b/etc/pacman.conf.in
index 53071e5..ddc7397 100644
--- a/etc/pacman.conf.in
+++ b/etc/pacman.conf.in
@@ -29,6 +29,10 @@ Architecture = auto
 #NoUpgrade   =
 #NoExtract   =
 
+# Downloader options
+#LowSpeedLimit = 1
+#LowSpeedTime = 10
+
 # Misc options
 #UseSyslog
 #Color
diff --git a/lib/libalpm/alpm.h b/lib/libalpm/alpm.h
index 168d71b..c338119 100644
--- a/lib/libalpm/alpm.h
+++ b/lib/libalpm/alpm.h
@@ -811,6 +811,16 @@ const char *alpm_option_get_dbpath(alpm_handle_t *handle);
 /** Get the name of the database lock file. Read-only. */
 const char *alpm_option_get_lockfile(alpm_handle_t *handle);
 
+/** Returns the libcurl low speed limit in bytes per second. */
+long alpm_option_get_download_lowspeedlimit(alpm_handle_t *handle);
+/** Sets the libcurl low speed limit in bytes per second. */
+int alpm_option_set_download_lowspeedlimit(alpm_handle_t *handle, long 
lowspeedlimit);
+
+/** Returns the libcurl low speed time in seconds. */
+long alpm_option_get_download_lowspeedtime(alpm_handle_t *handle);
+/** Sets the libcurl low speed time in seconds. */
+int alpm_option_set_download_lowspeedtime(alpm_handle_t *handle, long 
lowspeedtime);
+
 /** @name Accessors to the list of package cache directories.
  * @{
  */
diff --git a/lib/libalpm/dload.c b/lib/libalpm/dload.c
index f4e6a27..7490dd6 100644
--- a/lib/libalpm/dload.c
+++ b/lib/libalpm/dload.c
@@ -305,8 +305,8 @@ static void curl_set_handle_opts(struct dload_payload 
*payload,
curl_easy_setopt(curl, CURLOPT_FOLLOWLOCATION, 1L);
curl_easy_setopt(curl, CURLOPT_PROGRESSFUNCTION, dload_progress_cb);
curl_easy_setopt(curl, CURLOPT_PROGRESSDATA, (void *)payload);
-   curl_easy_setopt(curl, CURLOPT_LOW_SPEED_LIMIT, 1L);
-   curl_easy_setopt(curl, CURLOPT_LOW_SPEED_TIME, 10L);
+   curl_easy_setopt(curl, CURLOPT_LOW_SPEED_LIMIT, 
handle->download.lowspeedlimit);
+   curl_easy_setopt(curl, CURLOPT_LOW_SPEED_TIME, 
handle->download.lowspeedtime);
curl_easy_setopt(curl, CURLOPT_HEADERFUNCTION, dload_parseheader_cb);
curl_easy_setopt(curl, CURLOPT_WRITEHEADER, (void *)payload);
curl_easy_setopt(curl, CURLOPT_NETRC, CURL_NETRC_OPTIONAL);
diff --git a/lib/libalpm/handle.c b/lib/libalpm/handle.c
index e9439a0..8cfe0a1 100644
--- a/lib/libalpm/handle.c
+++ b/lib/libalpm/handle.c
@@ -247,6 +247,18 @@ const char SYMEXPORT 
*alpm_option_get_lockfile(alpm_handle_t *handle)
return handle->lockfile;
 }
 
+long SYMEXPORT alpm_option_get_download_lowspeedlimit(alpm_handle_t *handle)
+{
+   CHECK_HANDLE(handle, return -1);
+   return handle->download.lowspeedlimit;
+}
+
+long SYMEXPORT alpm_option_get_download_lowspeedtime(alpm_handle_t *handle)
+{
+   CHECK_HANDLE(handle, return -1);

[pacman-dev] I'm back (mostly)

2016-08-30 Thread Allan McRae
Hi all,

I had a low presence over the last couple of months due to purchasing a
house, moving and the subsequent delay of internet connection.  But I am
back!

Ping any patches that I have missed in my flurry tonight and I will
review them.

Cheers,
Allan


Re: [pacman-dev] [PATCH v2] Make DUFLAGS be overrideable during configure

2016-08-30 Thread Allan McRae
On 20/08/16 06:52, Alastair Hughes wrote:
> Not all du implementations on linux accept --apparent-size, so let the
> user configure the arguments passed to du if required.
> 
> This fixes FS#47943.
> 
> Signed-off-by: Alastair Hughes 
> ---

Why this approach over the previous style of:

DUFLAGS="${DUFLAGS- -sk}"

?

>  configure.ac | 14 +-
>  1 file changed, 9 insertions(+), 5 deletions(-)
> 
> diff --git a/configure.ac b/configure.ac
> index dd4ac04..40062f2 100644
> --- a/configure.ac
> +++ b/configure.ac
> @@ -327,12 +327,12 @@ AC_CHECK_MEMBERS([struct statfs.f_flags],,,[[#include 
> 
>  GCC_VISIBILITY_CC
>  
>  # Host-dependant definitions
> +DEFAULT_DUFLAGS=" -sk --apparent-size"
>  INODECMD="stat -c '%i %n'"
>  OWNERCMD="stat -c '%u:%g'"
>  MODECMD="stat -c '%a'"
>  SIZECMD="stat -c %s"
>  SEDINPLACE="sed --follow-symlinks -i"
> -DUFLAGS="-sk --apparent-size"
>  STRIP_BINARIES="--strip-all"
>  STRIP_SHARED="--strip-unneeded"
>  STRIP_STATIC="--strip-debug"
> @@ -343,7 +343,7 @@ case "${host_os}" in
>   MODECMD="stat -f '%Lp'"
>   SIZECMD="stat -f %z"
>   SEDINPLACE="sed -i \"\""
> - DUFLAGS="-sk"
> + DEFAULT_DUFLAGS=" -sk"
>   ;;
>   darwin*)
>   host_os_darwin=yes
> @@ -352,13 +352,12 @@ case "${host_os}" in
>   MODECMD="/usr/bin/stat -f '%Lp'"
>   SIZECMD="/usr/bin/stat -f %z"
>   SEDINPLACE="/usr/bin/sed -i ''"
> - DUFLAGS="-sk"
> + DEFAULT_DUFLAGS=" -sk"
>   STRIP_BINARIES=""
>   STRIP_SHARED="-S"
>   STRIP_STATIC="-S"
>   ;;
>  esac
> -
>  AM_CONDITIONAL([DARWIN], test "x$host_os_darwin" = "xyes")
>  AC_PATH_PROGS([DUPATH], [du], [du], [/usr/bin$PATH_SEPARATOR/bin] )
>  AC_SUBST(INODECMD)
> @@ -366,11 +365,16 @@ AC_SUBST(OWNERCMD)
>  AC_SUBST(MODECMD)
>  AC_SUBST(SIZECMD)
>  AC_SUBST(SEDINPLACE)
> -AC_SUBST(DUFLAGS)
>  AC_SUBST(STRIP_BINARIES)
>  AC_SUBST(STRIP_SHARED)
>  AC_SUBST(STRIP_STATIC)
>  
> +# Flags for du
> +if test "${DUFLAGS+set}" != "set"; then
> +DUFLAGS="${DEFAULT_DUFLAGS}"
> +fi
> +AC_ARG_VAR(DUFLAGS, [flags for du, overriding the default])
> +
>  # Variables plugged into makepkg.conf
>  CARCH="${host%%-*}"
>  CHOST="${host}"
> 


Re: [pacman-dev] [PATCH] Add option MaxDlSpeed to limit download speed

2016-08-30 Thread Allan McRae
On 20/08/16 04:12, Olivier Brunel wrote:
> Signed-off-by: Olivier Brunel 
> ---
>  doc/pacman.8.txt  |  5 +
>  doc/pacman.conf.5.txt |  6 ++
>  lib/libalpm/alpm.h|  3 +++
>  lib/libalpm/dload.c   |  1 +
>  lib/libalpm/handle.c  | 26 ++
>  lib/libalpm/handle.h  |  1 +
>  src/pacman/conf.c | 11 +++
>  src/pacman/conf.h |  4 +++-
>  src/pacman/pacman.c   | 11 +++
>  src/pacman/util.c | 45 +
>  src/pacman/util.h |  2 ++
>  11 files changed, 114 insertions(+), 1 deletion(-)
> 


I will look at this once we have the download timeout patches in so that
we have a consistent format for downloader options.

Thanks,
Allan


Re: [pacman-dev] [PATCH v2 2/3] bacman: proper option handling plus more options

2016-08-30 Thread Allan McRae
On 15/08/16 06:39, Gordian Edenhofer wrote:
> * Add option for printing fewer status updates
> * Add option for controlling the output directory
> * Add option for specyfying the the maximum number of jobs
> * Adjust output to the number of jobs
> * Rewrite usage page
> * Alter version information
> 
> Signed-off-by: Gordian Edenhofer 
> ---
> * Streamline option syntax and version function with other scripts (mainly 
> paccache)
> * Use the provided parseopts instead of getopt
> 

Multiple patches please.  Each patch should contain a single change.
E.g. (1) switch to parsepots, (2) rewrite usage, ...

A


Re: [pacman-dev] [PATCH v2 3/3] bacman: add manual page

2016-08-30 Thread Allan McRae
On 15/08/16 06:39, Gordian Edenhofer wrote:
> Signed-off-by: Gordian Edenhofer 
> ---
> * Fix some typos
> * Adjust man page according to v2 of this patch series
> 
>  doc/.gitignore   |  1 +
>  doc/Makefile.am  |  4 
>  doc/bacman.8.txt | 71 
> 
>  3 files changed, 76 insertions(+)
>  create mode 100644 doc/bacman.8.txt
> 

Contrib man pages now go to contrib/doc.

Allan


Re: [pacman-dev] [PATCH v2 1/3] bacman: allow for parallel packaging

2016-08-30 Thread Allan McRae
On 15/08/16 06:39, Gordian Edenhofer wrote:
> * move the actual assembly process into its own function
> * allow for packaging multiple packages with one command
> * handle SIGHUP SIGINT SIGTERM and remove working dirs accordingly
> * add some comments
> 
> Signed-off-by: Gordian Edenhofer 
> ---
> * Clean up working directories with force on abort
> * Sleep for 100ms in between parallelization work

Why sleep?

Two patches are needed:

1) allow bacman to take multiple package arguments
2) parallelize...


The second probably would not be accepted...

Allan


Re: [pacman-dev] [PATCH] pacman.8: fix typo in the documentation of --asexplicit

2016-08-30 Thread Allan McRae
On 20/08/16 21:46, Lukas Fleischer wrote:
> Add a space between the option and its argument. Also, do not enclose
> the argument in asterisk characters.
> 
> Fixes a typo introduced in aa4c61f (Document database checking options,
> 2014-12-28).

Thanks,
A


[pacman-dev] [GIT] The official pacman repository branch, master, updated. v5.0.1-54-g406c9b6

2016-08-30 Thread Allan McRae
This is an automated email from the git hooks/post-receive script. It was
generated because a ref change was pushed to the repository containing
the project "The official pacman repository".

The branch, master has been updated
   via  406c9b66b4f2cc54ea59e57de92db6566e59c4e7 (commit)
   via  6c15cc4d22e049486a5f773f5d81777b6087510b (commit)
   via  6d8e3d2a9171df8cbe479d0a582c468970802aed (commit)
   via  1ec7fa89ad6ad650cedeafd5c7cbc36dcf29239f (commit)
   via  69aee3e391d85a1c44de1c2e78f62a1e27453bca (commit)
   via  58140dba7440997e9d318fb56ed939a9c81fddf8 (commit)
   via  56de155296a57fb3fcd8ae64aed00fd18fe2f22e (commit)
   via  5b839c58ee1e78312edf69cd2cb5f96d8b649d01 (commit)
   via  56ae9603768bdc850f32bc789a474ae836305033 (commit)
   via  be1ffedaf6fc44aeb9da235d64889dac71e9bf24 (commit)
   via  c981f5ad76cb77363dcb4ebcc199670c3378995f (commit)
   via  af83a585745d6108d7af2d4a2d1eae9b28a8c4b6 (commit)
   via  5fcd60e2641c9293c2783aad509baf217e77aa6f (commit)
   via  681509fd445ed6012e6ecf89b49e9c00d83b70cd (commit)
   via  1291c04961e6c27e93ef376583d46ec2aa3036a1 (commit)
   via  80d97fcf7526f16d9eb097b8061956662207ed78 (commit)
   via  0f0b192d8a59134e9a58781c58aa3202df2eda2c (commit)
   via  6ac2ee21b30f3c5f331d19349f96bb8e5b020b47 (commit)
   via  7a9d8b7001f3f90471dc94ab31ec017f32ef8760 (commit)
  from  839417e8c65e73e632c77d3ecd68539494850de1 (commit)

Those revisions listed above that are new to this repository have
not appeared on any other notification email; so we list those
revisions in full, below.

- Log -
commit 406c9b66b4f2cc54ea59e57de92db6566e59c4e7
Author: Florian Pritz 
Date:   Sun Aug 7 15:37:23 2016 +0200

Document contrib/verify-pacman-repo-db.pl

Signed-off-by: Florian Pritz 

commit 6c15cc4d22e049486a5f773f5d81777b6087510b
Author: Florian Pritz 
Date:   Mon Jul 18 15:14:54 2016 +0200

contrib: Add documentation Makefile

Makefile.am is mostly copied from ./doc/Makefile.am

Signed-off-by: Florian Pritz 

commit 6d8e3d2a9171df8cbe479d0a582c468970802aed
Author: Florian Pritz 
Date:   Mon Jul 18 15:12:55 2016 +0200

contrib: Add verify-pacman-repo-db.pl

From the documentation:

verify-pacman-repo-db looks at a pacman repo database and verifies its
content with the actual package files. The database is expected to be in
the same directory as the packages (or symlinks to the packages).

The following properties are verified for each package in the database:

- existence of the package file
- file size
- MD5 and SHA256 checksum (--checksum)

Signed-off-by: Florian Pritz 

commit 1ec7fa89ad6ad650cedeafd5c7cbc36dcf29239f
Author: Allan McRae 
Date:   Tue Aug 30 19:06:45 2016 +1000

Fix file name in scripts potfile

commit 69aee3e391d85a1c44de1c2e78f62a1e27453bca
Author: Giulio Fidente 
Date:   Sun Aug 7 15:55:59 2016 +1000

Merge Giolio Fidente into "Pacman Development Team" contribution

As discussed on mailing list:
https://lists.archlinux.org/pipermail/pacman-dev/2016-July/021239.html

Signed-off-by: Allan McRae 

commit 58140dba7440997e9d318fb56ed939a9c81fddf8
Author: Ivy Foster 
Date:   Fri Jul 8 22:11:25 2016 -0500

Normalize alpm download callback's frontend cb arguments

When curl calls alpm's dlcb, alpm calls the frontend's cb with the
following (dlsize, totalsize) arguments:

0, -1: initialize
0, 0: no change since last call
x {x>0, x0}: data downloaded, total size known
x {x>0}, x: download finished

If total size is not known, do not call frontend cb (no change to
original behavior); alpm's callback shouldn't be called if there is a
download error.

See agregory's original spec here:
https://wiki.archlinux.org/index.php/User:Apg#download_callback

Signed-off-by: Allan McRae 

commit 56de155296a57fb3fcd8ae64aed00fd18fe2f22e
Author: Jack O'Connor 
Date:   Mon Aug 1 13:06:00 2016 +1000

libmakepkg: look for architecture-specific hashes in get_integlist

`makepkg -g` looks for existing checksums in the PKGBUILD file, so that
it can generate new sums of the same type. Previously it only checked
variables of the form "sha256sums", and not "sha256sums_x86_64". That
meant it would always fall back to MD5 for packages with only
architecture-specific sources. This change makes it look at
architecture-specific checksums too to determine the type.

Signed-off-by: Jack O'Connor 
Signed-off-by: Allan McRae 

commit 5b839c58ee1e78312edf69cd2cb5f96d8b649d01
Author: Allan McRae 
Date:   Sun Jul 3 23:05:41 2016 +1000

Add newline to the end of error messages for signature format issues

Signed-off-by: Allan McRae 

commit 56ae9603768bdc850f32bc789a474ae836305033
Author: Olivier Brunel 
Date: