Re: [regression] Newer gits cannot clone any remote repos
> The threaded index-pack code did not fail for > me on cygwin at all during development, including tests, but failed > immediately I installed v1.7.11. On real repositories, it failed > intermittently. On some repos it always failed, on some it never > failed and on some others it would sometimes fail, sometimes not. Then why did you commit it? If it has so high random failure rate. -- To unsubscribe from this list: send the line "unsubscribe git" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [regression] Newer gits cannot clone any remote repos
Torsten Bögershausen wrote:
> On 13.11.12 19:55, Ramsay Jones wrote:
>> Douglas Mencken wrote:
>>> *Any* git clone fails with:
>>>
>>> fatal: premature end of pack file, 106 bytes missing
>>> fatal: index-pack failed
>>>
>>> At first, I tried 1.8.0, and it failed. Then I tried to build 1.7.10.5
>>> then, and it worked. Then I tried 1.7.12.2, but it fails the same way
>>> as 1.8.0.
>>> So I decided to git bisect.
>>>
>>> b8a2486f1524947f232f657e9f2ebf44e3e7a243 is the first bad commit
>>> ``index-pack: support multithreaded delta resolving''
>>
>> This looks like the same problem I had on cygwin, which lead to
>> commit c0f86547c ("index-pack: Disable threading on cygwin", 26-06-2012).
>>
>> I didn't notice which platform you are on, but maybe you also have a
>> thread-unsafe pread()? Could you try re-building git with the
>> NO_THREAD_SAFE_PREAD build variable set?
>>
>> HTH.
>>
>> ATB,
>> Ramsay Jones
>
> This is interesting.
> I had the same problem on a PowerPC
> (Old PowerBook G4 running Linux).
>
> Using NO_THREAD_SAFE_PREAD helped, thanks for the hint.
> (After recompiling without NO_THREAD_SAFE_PREAD I could clone
> from this machine again, so the problem is not really reproducable)
Yes, the failures would be intermittent (and often not easily
reproducible). The threaded index-pack code did not fail for
me on cygwin at all during development, including tests, but failed
immediately I installed v1.7.11. On real repositories, it failed
intermittently. On some repos it always failed, on some it never
failed and on some others it would sometimes fail, sometimes not.
ATB,
Ramsay Jones
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [regression] Newer gits cannot clone any remote repos
Douglas Mencken wrote:
>> Could you try re-building git with the
>> NO_THREAD_SAFE_PREAD build variable set?
>
> Yeah! It works!!!
>
> --- evil/Makefile
> +++ good/Makefile
> @@ -957,6 +957,7 @@
> HAVE_PATHS_H = YesPlease
> LIBC_CONTAINS_LIBINTL = YesPlease
> HAVE_DEV_TTY = YesPlease
> + NO_THREAD_SAFE_PREAD = YesPlease
> endif
> ifeq ($(uname_S),GNU/kFreeBSD)
> NO_STRLCPY = YesPlease
>
> With this, I do have correctly working git clone.
OK, good.
You didn't mention which platform you are on; from the above Makefile
hunk, however, I can deduce that you are on *some* version of Linux.
Hmm, it doesn't seem too likely that your pread() is thread-unsafe
(possible, just unlikely), so it could be a more fundamental problem
with the threaded index-pack code (ie commit b8a2486f1 et. seq.).
However, as a first step could you try running the test program
(given below) on your system to determine if your pread() is thread-safe
or not. (gcc -I. -o test-pread test-pread.c; ./test-pread)
Also, what is the output of "uname -a".
ATB,
Ramsay Jones
-- >8 --
#include "git-compat-util.h"
#include "thread-utils.h"
#define DATA_FILE "junk.data"
#define MAX_DATA 256 * 1024
#define NUM_THREADS 3
#define TRIALS 50
struct thread_data {
pthread_t t;
int fd;
int cnt;
int fails;
unsigned long n;
};
static struct thread_data t[NUM_THREADS+1];
int create_data_file(void)
{
int i, fd = open(DATA_FILE, O_CREAT | O_TRUNC | O_WRONLY, 0600);
if (fd < 0)
return -1;
for (i = 0; i < MAX_DATA; i++)
if (write(fd, &i, sizeof(int)) < 0) {
close(fd);
unlink(DATA_FILE);
return -1;
}
close(fd);
return 0;
}
void *read_thread(void *data)
{
struct thread_data *d = (struct thread_data *)data;
int i, j, rd;
for (i = 0; i < TRIALS; i += MAX_DATA) {
for (j = 0; j < MAX_DATA; j++) {
ssize_t sz = read(d->fd, &rd, sizeof(int));
if (sz < 0 || rd != j)
d->fails++;
d->cnt++;
}
lseek(d->fd, 0, SEEK_SET);
}
return NULL;
}
void *pread_thread(void *data)
{
struct thread_data *d = (struct thread_data *)data;
int i, j, rd;
for (i = 0; i < TRIALS; i++) {
ssize_t sz;
d->n = d->n * 1103515245 + 12345;
j = d->n % MAX_DATA;
sz = pread(d->fd, &rd, sizeof(int), j * sizeof(int));
if (sz < 0 || rd != j)
d->fails++;
d->cnt++;
}
return NULL;
}
int main(int argc, char *argv[])
{
int fd, i;
if (create_data_file() < 0) {
printf("can't create data file\n");
return 1;
}
if ((fd = open(DATA_FILE, O_RDONLY)) < 0) {
printf("can't open data file\n");
unlink(DATA_FILE);
return 1;
}
for (i = 0; i < NUM_THREADS+1; i++) {
int ret;
t[i].fd = fd;
t[i].cnt = 0;
t[i].fails = 0;
t[i].n = i * 16381;
ret = pthread_create(&t[i].t, NULL,
(i == 0) ? read_thread : pread_thread,
&t[i]);
if (ret) {
printf("can't create thread %d (%s)\n", i,
strerror(ret));
unlink(DATA_FILE);
return 1;
}
}
for (i = 0; i < NUM_THREADS+1; i++)
pthread_join(t[i].t, NULL);
close(fd);
for (i = 0; i < NUM_THREADS+1; i++)
printf("%2d: trials %d, failed %d\n", i, t[i].cnt, t[i].fails);
unlink(DATA_FILE);
return 0;
}
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [regression] Newer gits cannot clone any remote repos
On Wed, Nov 14, 2012 at 2:55 AM, Douglas Mencken wrote: >> Could you try re-building git with the >> NO_THREAD_SAFE_PREAD build variable set? > > Yeah! It works!!! > > --- evil/Makefile > +++ good/Makefile > @@ -957,6 +957,7 @@ > HAVE_PATHS_H = YesPlease > LIBC_CONTAINS_LIBINTL = YesPlease > HAVE_DEV_TTY = YesPlease > + NO_THREAD_SAFE_PREAD = YesPlease > endif > ifeq ($(uname_S),GNU/kFreeBSD) > NO_STRLCPY = YesPlease > > With this, I do have correctly working git clone. Sorry you had to figure that out the hard way. Could you make it a proper patch? I'm surprised that Linux pread does not behave the same way across platforms though. Or maybe it only happens with certain Linux versions. What version are you using? -- Duy -- To unsubscribe from this list: send the line "unsubscribe git" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [regression] Newer gits cannot clone any remote repos
Douglas Mencken writes: I cannot reproduce the problem (on openSUSE 12.2). >>> >>> You do need multiple CPU/multi-core machine, as I got it. >> >> Which is what I have. > > Then try to build *vanilla* git 1.8.0, Which is what I did. > not OpenSuSE's one (with a lot of patches inside srcrpm). Which lot of patches? Andreas. -- Andreas Schwab, sch...@linux-m68k.org GPG Key fingerprint = 58CA 54C7 6D53 942B 1756 01D3 44D5 214B 8276 4ED5 "And now for something completely different." -- To unsubscribe from this list: send the line "unsubscribe git" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [regression] Newer gits cannot clone any remote repos
>>> I cannot reproduce the problem (on openSUSE 12.2). >> >> You do need multiple CPU/multi-core machine, as I got it. > > Which is what I have. Then try to build *vanilla* git 1.8.0, not OpenSuSE's one (with a lot of patches inside srcrpm). -- To unsubscribe from this list: send the line "unsubscribe git" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [regression] Newer gits cannot clone any remote repos
Douglas Mencken writes: >> I cannot reproduce the problem (on openSUSE 12.2). > > You do need multiple CPU/multi-core machine, as I got it. Which is what I have. Andreas. -- Andreas Schwab, sch...@linux-m68k.org GPG Key fingerprint = 58CA 54C7 6D53 942B 1756 01D3 44D5 214B 8276 4ED5 "And now for something completely different." -- To unsubscribe from this list: send the line "unsubscribe git" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [regression] Newer gits cannot clone any remote repos
> I cannot reproduce the problem (on openSUSE 12.2). You do need multiple CPU/multi-core machine, as I got it. -- To unsubscribe from this list: send the line "unsubscribe git" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [regression] Newer gits cannot clone any remote repos
Torsten Bögershausen writes: > Are there more people running PowerPC (on the server side) ? I cannot reproduce the problem (on openSUSE 12.2). Andreas. -- Andreas Schwab, sch...@linux-m68k.org GPG Key fingerprint = 58CA 54C7 6D53 942B 1756 01D3 44D5 214B 8276 4ED5 "And now for something completely different." -- To unsubscribe from this list: send the line "unsubscribe git" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [regression] Newer gits cannot clone any remote repos
On 13.11.12 19:55, Ramsay Jones wrote:
> Douglas Mencken wrote:
>> *Any* git clone fails with:
>>
>> fatal: premature end of pack file, 106 bytes missing
>> fatal: index-pack failed
>>
>> At first, I tried 1.8.0, and it failed. Then I tried to build 1.7.10.5
>> then, and it worked. Then I tried 1.7.12.2, but it fails the same way
>> as 1.8.0.
>> So I decided to git bisect.
>>
>> b8a2486f1524947f232f657e9f2ebf44e3e7a243 is the first bad commit
>> ``index-pack: support multithreaded delta resolving''
>
> This looks like the same problem I had on cygwin, which lead to
> commit c0f86547c ("index-pack: Disable threading on cygwin", 26-06-2012).
>
> I didn't notice which platform you are on, but maybe you also have a
> thread-unsafe pread()? Could you try re-building git with the
> NO_THREAD_SAFE_PREAD build variable set?
>
> HTH.
>
> ATB,
> Ramsay Jones
This is interesting.
I had the same problem on a PowerPC
(Old PowerBook G4 running Linux).
Using NO_THREAD_SAFE_PREAD helped, thanks for the hint.
(After recompiling without NO_THREAD_SAFE_PREAD I could clone
from this machine again, so the problem is not really reproducable)
Are there more people running PowerPC (on the server side) ?
/Torsten
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [regression] Newer gits cannot clone any remote repos
Douglas Mencken wrote:
> *Any* git clone fails with:
>
> fatal: premature end of pack file, 106 bytes missing
> fatal: index-pack failed
>
> At first, I tried 1.8.0, and it failed. Then I tried to build 1.7.10.5
> then, and it worked. Then I tried 1.7.12.2, but it fails the same way
> as 1.8.0.
> So I decided to git bisect.
>
> b8a2486f1524947f232f657e9f2ebf44e3e7a243 is the first bad commit
> ``index-pack: support multithreaded delta resolving''
This looks like the same problem I had on cygwin, which lead to
commit c0f86547c ("index-pack: Disable threading on cygwin", 26-06-2012).
I didn't notice which platform you are on, but maybe you also have a
thread-unsafe pread()? Could you try re-building git with the
NO_THREAD_SAFE_PREAD build variable set?
HTH.
ATB,
Ramsay Jones
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [regression] Newer gits cannot clone any remote repos
> On Mon, Nov 12, 2012 at 12:12 PM, Kevin wrote:
> > Maybe handy to say that you're on a Powerpc platform.
Oh, and yes, I'm on 2 x 2-core ("4-core") machine.
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [regression] Newer gits cannot clone any remote repos
Maybe handy to say that you're on a Powerpc platform. On Mon, Nov 12, 2012 at 5:32 PM, Douglas Mencken wrote: > *Any* git clone fails with: > > fatal: premature end of pack file, 106 bytes missing > fatal: index-pack failed > > At first, I tried 1.8.0, and it failed. Then I tried to build 1.7.10.5 > then, and it worked. Then I tried 1.7.12.2, but it fails the same way > as 1.8.0. > So I decided to git bisect. > -- To unsubscribe from this list: send the line "unsubscribe git" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
