Re: cygrunsrv + sshd + rsync = 20 times too slow -- throttled?

2021-09-16 Thread Keith Christian via Cygwin
Ken, Thank you for the reply, this is great news! Looking forward to speedier rsyncs with Cygwin. Keith On Thu, Sep 16, 2021, 16:49 Ken Brown via Cygwin wrote: > On 9/16/2021 6:00 PM, Keith Christian via Cygwin wrote: > > I've been following his thread with interest both here a

Re: cygrunsrv + sshd + rsync = 20 times too slow -- throttled?

2021-09-16 Thread Ken Brown via Cygwin
On 9/16/2021 6:00 PM, Keith Christian via Cygwin wrote: I've been following his thread with interest both here and on the Cygwin developers list. I, too rsync between Cygwin and Linux machines. Lots of good debate showing the genius of the folks that support the cygwin infrastructure. Looks li

Re: cygrunsrv + sshd + rsync = 20 times too slow -- throttled?

2021-09-16 Thread Keith Christian via Cygwin
I've been following his thread with interest both here and on the Cygwin developers list. I, too rsync between Cygwin and Linux machines. Lots of good debate showing the genius of the folks that support the cygwin infrastructure. Looks like the lively discussion on both lists has stopped. What

Re: mmap failure [was: cygrunsrv + sshd + rsync = 20 times too slow -- throttled?]

2021-09-08 Thread Corinna Vinschen via Cygwin
On Sep 6 21:34, Brian Inglis wrote: > On 2021-09-06 15:24, Ken Brown via Cygwin wrote: > > On 9/6/2021 4:54 PM, Peter Dons Tychsen wrote: > > > Hi there, > > > > > > On Mon, 2021-09-06 at 14:40 -0400, Ken Brown via Cygwin wrote: > > > > > No, wait.  I get what you say.  The optimzation settings o

Re: mmap failure [was: cygrunsrv + sshd + rsync = 20 times too slow -- throttled?]

2021-09-07 Thread Sam Edge via Cygwin
On 07/09/2021 23:44, Ken Brown via Cygwin wrote: > > MS can't add a new named field to a documented struct without breaking a lot of code.  I think it's extremely unlikely that they would do that.  On the other hand, I think it's very likely that a reader of the Cygwin code would be confused by c

Re: mmap failure [was: cygrunsrv + sshd + rsync = 20 times too slow -- throttled?]

2021-09-07 Thread Ken Brown via Cygwin
On 9/7/2021 5:52 PM, Lavrentiev, Anton (NIH/NLM/NCBI) [C] wrote: With undocumented structure member initialization an issue, maybe better to future proof using e.g. MEM_EXTENDED_PARAMETER mmap_ext = { 0 }; // or memset or bzero I don't see what this would accomplish. We're already init

RE: mmap failure [was: cygrunsrv + sshd + rsync = 20 times too slow -- throttled?]

2021-09-07 Thread Lavrentiev, Anton (NIH/NLM/NCBI) [C] via Cygwin
> > > > With undocumented structure member initialization an issue, maybe better to > > future proof using e.g. > > > > MEM_EXTENDED_PARAMETER mmap_ext = { 0 }; // or memset or bzero > > I don't see what this would accomplish. We're already initializing every > member > after Corinna's last

Re: mmap failure [was: cygrunsrv + sshd + rsync = 20 times too slow -- throttled?]

2021-09-07 Thread Peter Dons Tychsen
Hi Ken, On Mon, 2021-09-06 at 17:24 -0400, Ken Brown via Cygwin wrote: > You're looking at the wrong source code.  The bug didn't occur until > the code > was changed to do the following: You are right. I do not know why i looked at an old checkout of the code. Shame on me! Sorry for wasting you

Re: mmap failure [was: cygrunsrv + sshd + rsync = 20 times too slow -- throttled?]

2021-09-07 Thread Ken Brown via Cygwin
On 9/6/2021 11:34 PM, Brian Inglis wrote: On 2021-09-06 15:24, Ken Brown via Cygwin wrote: On 9/6/2021 4:54 PM, Peter Dons Tychsen wrote: Hi there, On Mon, 2021-09-06 at 14:40 -0400, Ken Brown via Cygwin wrote: No, wait.  I get what you say.  The optimzation settings of the test case should h

Re: mmap failure [was: cygrunsrv + sshd + rsync = 20 times too slow -- throttled?]

2021-09-06 Thread Brian Inglis
On 2021-09-06 15:24, Ken Brown via Cygwin wrote: On 9/6/2021 4:54 PM, Peter Dons Tychsen wrote: Hi there, On Mon, 2021-09-06 at 14:40 -0400, Ken Brown via Cygwin wrote: No, wait.  I get what you say.  The optimzation settings of the test case should have no influence on the code inside the DLL

Re: mmap failure [was: cygrunsrv + sshd + rsync = 20 times too slow -- throttled?]

2021-09-06 Thread Ken Brown via Cygwin
On 9/6/2021 5:24 PM, Ken Brown via Cygwin wrote: On 9/6/2021 4:54 PM, Peter Dons Tychsen wrote: Hi there, On Mon, 2021-09-06 at 14:40 -0400, Ken Brown via Cygwin wrote: No, wait.  I get what you say.  The optimzation settings of the test case should have no influence on the code inside the DLL

Re: mmap failure [was: cygrunsrv + sshd + rsync = 20 times too slow -- throttled?]

2021-09-06 Thread Ken Brown via Cygwin
On 9/6/2021 4:54 PM, Peter Dons Tychsen wrote: Hi there, On Mon, 2021-09-06 at 14:40 -0400, Ken Brown via Cygwin wrote: No, wait.  I get what you say.  The optimzation settings of the test case should have no influence on the code inside the DLL.  That doesn't make sense for sure.  However, I r

Re: mmap failure [was: cygrunsrv + sshd + rsync = 20 times too slow -- throttled?]

2021-09-06 Thread Peter Dons Tychsen
Hi there, On Mon, 2021-09-06 at 14:40 -0400, Ken Brown via Cygwin wrote: > > No, wait.  I get what you say.  The optimzation settings of the test > > case should have no influence on the code inside the DLL.  That > > doesn't > > make sense for sure.  However, I ran the testcase under GDB, I could

Re: mmap failure [was: cygrunsrv + sshd + rsync = 20 times too slow -- throttled?]

2021-09-06 Thread Peter Dons Tychsen via Cygwin
Hi there, On Mon, 2021-09-06 at 14:40 -0400, Ken Brown via Cygwin wrote: > > No, wait.  I get what you say.  The optimzation settings of the test > > case should have no influence on the code inside the DLL.  That > > doesn't > > make sense for sure.  However, I ran the testcase under GDB, I could

Re: mmap failure [was: cygrunsrv + sshd + rsync = 20 times too slow -- throttled?]

2021-09-06 Thread Ken Brown via Cygwin
On 9/6/2021 2:07 PM, Corinna Vinschen via Cygwin wrote: On Sep 6 19:59, Corinna Vinschen via Cygwin wrote: On Sep 6 13:38, Ken Brown via Cygwin wrote: On 9/6/2021 1:12 PM, Ken Brown via Cygwin wrote: On 9/6/2021 11:32 AM, Corinna Vinschen via Cygwin wrote: On Sep  5 09:24, Ken Brown via Cyg

Re: mmap failure [was: cygrunsrv + sshd + rsync = 20 times too slow -- throttled?]

2021-09-06 Thread Corinna Vinschen via Cygwin
On Sep 6 19:59, Corinna Vinschen via Cygwin wrote: > On Sep 6 13:38, Ken Brown via Cygwin wrote: > > On 9/6/2021 1:12 PM, Ken Brown via Cygwin wrote: > > > On 9/6/2021 11:32 AM, Corinna Vinschen via Cygwin wrote: > > > > On Sep  5 09:24, Ken Brown via Cygwin wrote: > > > > > On 9/4/2021 8:04 PM,

Re: mmap failure [was: cygrunsrv + sshd + rsync = 20 times too slow -- throttled?]

2021-09-06 Thread Corinna Vinschen via Cygwin
On Sep 6 13:38, Ken Brown via Cygwin wrote: > On 9/6/2021 1:12 PM, Ken Brown via Cygwin wrote: > > On 9/6/2021 11:32 AM, Corinna Vinschen via Cygwin wrote: > > > On Sep  5 09:24, Ken Brown via Cygwin wrote: > > > > On 9/4/2021 8:04 PM, Ken Brown via Cygwin wrote: > > > > > On 9/4/2021 6:58 PM, Ken

Re: mmap failure [was: cygrunsrv + sshd + rsync = 20 times too slow -- throttled?]

2021-09-06 Thread Eliot Moss
On 9/6/2021 1:38 PM, Ken Brown via Cygwin wrote: On 9/6/2021 1:12 PM, Ken Brown via Cygwin wrote: On 9/6/2021 11:32 AM, Corinna Vinschen via Cygwin wrote: On Sep  5 09:24, Ken Brown via Cygwin wrote: On 9/4/2021 8:04 PM, Ken Brown via Cygwin wrote: On 9/4/2021 6:58 PM, Ken Brown via Cygwin wr

Re: mmap failure [was: cygrunsrv + sshd + rsync = 20 times too slow -- throttled?]

2021-09-06 Thread Ken Brown via Cygwin
On 9/6/2021 1:12 PM, Ken Brown via Cygwin wrote: On 9/6/2021 11:32 AM, Corinna Vinschen via Cygwin wrote: On Sep  5 09:24, Ken Brown via Cygwin wrote: On 9/4/2021 8:04 PM, Ken Brown via Cygwin wrote: On 9/4/2021 6:58 PM, Ken Brown via Cygwin wrote: Here are the correct commits: 8169e39ab Cyg

Re: mmap failure [was: cygrunsrv + sshd + rsync = 20 times too slow -- throttled?]

2021-09-06 Thread Ken Brown via Cygwin
On 9/6/2021 11:32 AM, Corinna Vinschen via Cygwin wrote: On Sep 5 09:24, Ken Brown via Cygwin wrote: On 9/4/2021 8:04 PM, Ken Brown via Cygwin wrote: On 9/4/2021 6:58 PM, Ken Brown via Cygwin wrote: Here are the correct commits: 8169e39ab Cygwin: C++17: register keyword is deprecated 3ca80b3

Re: mmap failure [was: cygrunsrv + sshd + rsync = 20 times too slow -- throttled?]

2021-09-06 Thread Corinna Vinschen via Cygwin
On Sep 5 09:24, Ken Brown via Cygwin wrote: > On 9/4/2021 8:04 PM, Ken Brown via Cygwin wrote: > > On 9/4/2021 6:58 PM, Ken Brown via Cygwin wrote: > > > Here are the correct commits: > > > > > > 8169e39ab Cygwin: C++17: register keyword is deprecated > > > 3ca80b360 Cygwin: dumper: fix up GCC pr

Re: cygrunsrv + sshd + rsync = 20 times too slow -- throttled?

2021-09-06 Thread Achim Gratz
05.09.2021 17:11, Brian Inglis: The suggestion was intended as a tip to ensure *complete* locally rebuilt package contents are installed, Setup has its "from_cwd" installation mode for precisely that reason: installing a local package without the need to create a full install hierarchy and s

Re: cygrunsrv + sshd + rsync = 20 times too slow -- throttled?

2021-09-05 Thread Brian Inglis
On 2021-09-05 02:18, Achim Gratz wrote: 04.09.2021 18:45, Brian Inglis: [...] then to install all binary packages for dogfooding: Would you please stop telling folks to do things that potentially breaks their systems? There are quite a few more steps to take if you want to emulate what set

Re: mmap failure [was: cygrunsrv + sshd + rsync = 20 times too slow -- throttled?]

2021-09-05 Thread Ken Brown via Cygwin
On 9/4/2021 8:04 PM, Ken Brown via Cygwin wrote: On 9/4/2021 6:58 PM, Ken Brown via Cygwin wrote: On 9/4/2021 6:54 PM, Ken Brown via Cygwin wrote: On 9/4/2021 6:37 PM, Ken Brown via Cygwin wrote: I've reduced the procps failure to the following test case: $ cat mmap_test.c #include #include

Re: cygrunsrv + sshd + rsync = 20 times too slow -- throttled?

2021-09-05 Thread Achim Gratz
04.09.2021 18:45, Brian Inglis: [...] then to install all binary packages for dogfooding: Would you please stop telling folks to do things that potentially breaks their systems? There are quite a few more steps to take if you want to emulate what setup does. Then again you cannot do that o

Re: mmap failure [was: cygrunsrv + sshd + rsync = 20 times too slow -- throttled?]

2021-09-04 Thread Ken Brown via Cygwin
On 9/4/2021 6:58 PM, Ken Brown via Cygwin wrote: On 9/4/2021 6:54 PM, Ken Brown via Cygwin wrote: On 9/4/2021 6:37 PM, Ken Brown via Cygwin wrote: I've reduced the procps failure to the following test case: $ cat mmap_test.c #include #include #include int main () {    void *addr;    int pa

Re: mmap failure [was: cygrunsrv + sshd + rsync = 20 times too slow -- throttled?]

2021-09-04 Thread Brian Inglis
On 2021-09-04 16:37, Ken Brown via Cygwin wrote: I've reduced the procps failure to the following test case: $ cat mmap_test.c #include #include #include int main () {   void *addr;   int page_size = getpagesize ();   addr = mmap (0, page_size, PROT_READ | PROT_WRITE,    MAP

Re: mmap failure [was: cygrunsrv + sshd + rsync = 20 times too slow -- throttled?]

2021-09-04 Thread Ken Brown via Cygwin
On 9/4/2021 6:54 PM, Ken Brown via Cygwin wrote: On 9/4/2021 6:37 PM, Ken Brown via Cygwin wrote: I've reduced the procps failure to the following test case: $ cat mmap_test.c #include #include #include int main () {    void *addr;    int page_size = getpagesize ();    addr = mmap (0, page

Re: mmap failure [was: cygrunsrv + sshd + rsync = 20 times too slow -- throttled?]

2021-09-04 Thread Ken Brown via Cygwin
On 9/4/2021 6:37 PM, Ken Brown via Cygwin wrote: I've reduced the procps failure to the following test case: $ cat mmap_test.c #include #include #include int main () {   void *addr;   int page_size = getpagesize ();   addr = mmap (0, page_size, PROT_READ | PROT_WRITE,    MAP

mmap failure [was: cygrunsrv + sshd + rsync = 20 times too slow -- throttled?]

2021-09-04 Thread Ken Brown via Cygwin
I've reduced the procps failure to the following test case: $ cat mmap_test.c #include #include #include int main () { void *addr; int page_size = getpagesize (); addr = mmap (0, page_size, PROT_READ | PROT_WRITE, MAP_PRIVATE | MAP_ANONYMOUS, -1, 0); if (addr == MAP_FA

Re: cygrunsrv + sshd + rsync = 20 times too slow -- throttled?

2021-09-04 Thread Brian Inglis
On 2021-09-03 14:59, Chris Roehrig wrote: On Fri Sep 3 2021, at 12:55 PM, Corinna Vinschen wrote: On Sep 2 12:03, Chris Roehrig wrote: On Thu Sep 2 2021, at 8:25 AM, Ken Brown via Cygwin wrote: On 9/1/2021 5:11 PM, Chris Roehrig wrote: I rebuild procps 3.3.17.29-2480 from source and it appea

Re: cygrunsrv + sshd + rsync = 20 times too slow -- throttled?

2021-09-04 Thread Achim Gratz
Am 03.09.2021 um 22:59 schrieb Chris Roehrig: I got procps working I think (both with and without the revert). That likely wasn't what Corinna wanted to know, though. Please re-install the procps-ng, cygwin and cygwin-devel packages from setup (and revert any other alterations you may have ma

Re: cygrunsrv + sshd + rsync = 20 times too slow -- throttled?

2021-09-03 Thread Chris Roehrig
On Fri Sep 3 2021, at 12:55 PM, Corinna Vinschen wrote: > [resent, this time with the ML in To] > > On Sep 2 12:03, Chris Roehrig wrote: >> >> On Thu Sep 2 2021, at 8:25 AM, Ken Brown via Cygwin >> wrote: >>> On 9/1/2021 5:11 PM, Chris Roehrig wrote: I rebuild procps 3.3.17.29-2480 fro

Re: cygrunsrv + sshd + rsync = 20 times too slow -- throttled?

2021-09-03 Thread Corinna Vinschen via Cygwin
[resent, this time with the ML in To] On Sep 2 12:03, Chris Roehrig wrote: > > On Thu Sep 2 2021, at 8:25 AM, Ken Brown via Cygwin wrote: > > On 9/1/2021 5:11 PM, Chris Roehrig wrote: > >> I rebuild procps 3.3.17.29-2480 from source and it appears to work, so > >> maybe the stock procps packag

Re: cygrunsrv + sshd + rsync = 20 times too slow -- throttled?

2021-09-03 Thread Ken Brown via Cygwin
On 9/2/2021 3:03 PM, Chris Roehrig wrote: On Thu Sep 2 2021, at 8:25 AM, Ken Brown via Cygwin wrote: On 9/1/2021 5:11 PM, Chris Roehrig wrote: I rebuild procps 3.3.17.29-2480 from source and it appears to work, so maybe the stock procps package is incompatible with the current master branch.

Re: cygrunsrv + sshd + rsync = 20 times too slow -- throttled?

2021-09-02 Thread Chris Roehrig
On Thu Sep 2 2021, at 8:25 AM, Ken Brown via Cygwin wrote: > On 9/1/2021 5:11 PM, Chris Roehrig wrote: >> I rebuild procps 3.3.17.29-2480 from source and it appears to work, so maybe >> the stock procps package is incompatible with the current master branch. > > Maybe, but it could also be a C

Re: cygrunsrv + sshd + rsync = 20 times too slow -- throttled?

2021-09-02 Thread Ken Brown via Cygwin
On 9/1/2021 5:11 PM, Chris Roehrig wrote: I rebuild procps 3.3.17.29-2480 from source and it appears to work, so maybe the stock procps package is incompatibility with the current master branch. Maybe, but it could also be a Cygwin bug. I'll do a bisection of the Cygwin sources to see if I c

Re: cygrunsrv + sshd + rsync = 20 times too slow -- throttled?

2021-09-01 Thread Chris Roehrig
I rebuild procps 3.3.17.29-2480 from source and it appears to work, so maybe the stock procps package is incompatibility with the current master branch. (However, I built it against the stock /usr/include, not the current branch...) I first needed to 'make /proc/libprocps.la', and there was an

Re: cygrunsrv + sshd + rsync = 20 times too slow -- throttled?

2021-08-31 Thread Brian Inglis
I've found you must also copy the matching cygwin-console-helper.exe for everything to work correctly! On 2021-08-31 14:23, Chris Roehrig wrote: I did a 'git pull' of the latest topic/pipe and rebuilt and I now do indeed get 100MB/s transfers using both rsync and scp (without pipe_byte). (It t

Re: cygrunsrv + sshd + rsync = 20 times too slow -- throttled?

2021-08-31 Thread Chris Roehrig
I did a 'git pull' of the latest topic/pipe and rebuilt and I now do indeed get 100MB/s transfers using both rsync and scp (without pipe_byte). (It turns out last time I forgot 'make install' -- Doh!) I still get the procps error however. On Tue Aug 31 2021, at 12:53 PM, Chris Roehrig wrote:

Re: cygrunsrv + sshd + rsync = 20 times too slow -- throttled?

2021-08-31 Thread Chris Roehrig
Thanks, I did some more tests: scp also shows no improvement with topic/pipe.I tried running cygsshd with CYGWIN=pipe_byte as well as empty (in the registry HKLM/SYSTEM/CurrentControlSet/Services/cygsshd/Parameters/Environment/), using net stop cygsshd + net start cygsshd to restart

Re: cygrunsrv + sshd + rsync = 20 times too slow -- throttled?

2021-08-31 Thread Ken Brown via Cygwin
On 8/30/2021 7:58 PM, Chris Roehrig wrote: I got it to build and tried out the topic/pipe branch (checked out on Monday around 4:30pm PDT): 1. I didn't see any improvement in my sshd+rsync time, still 3-4 MB/sec. 2. I get the following error from procps: procps:ps/output.c:2195: please re

Re: cygrunsrv + sshd + rsync = 20 times too slow -- throttled?

2021-08-30 Thread Chris Roehrig
I got it to build and tried out the topic/pipe branch (checked out on Monday around 4:30pm PDT): 1. I didn't see any improvement in my sshd+rsync time, still 3-4 MB/sec. 2. I get the following error from procps: procps:ps/output.c:2195: please report this bug (I also get this using the main

Re: cygrunsrv + sshd + rsync = 20 times too slow -- throttled?

2021-08-30 Thread Takashi Yano via Cygwin
On Sun, 29 Aug 2021 22:15:29 -0400 Ken Brown wrote: > On 8/29/2021 8:22 PM, Takashi Yano via Cygwin wrote: > > On Mon, 30 Aug 2021 09:13:14 +0900 > > Takashi Yano wrote: > >> On Sun, 29 Aug 2021 17:04:56 -0400 > >> Ken Brown wrote: > >>> On 8/29/2021 5:07 AM, Takashi Yano via Cygwin wrote: > O

Re: cygrunsrv + sshd + rsync = 20 times too slow -- throttled?

2021-08-29 Thread Ken Brown via Cygwin
On 8/29/2021 8:22 PM, Takashi Yano via Cygwin wrote: On Mon, 30 Aug 2021 09:13:14 +0900 Takashi Yano wrote: On Sun, 29 Aug 2021 17:04:56 -0400 Ken Brown wrote: On 8/29/2021 5:07 AM, Takashi Yano via Cygwin wrote: On Sat, 28 Aug 2021 18:41:02 +0900 Takashi Yano wrote: On Sat, 28 Aug 2021 10:43

Re: cygrunsrv + sshd + rsync = 20 times too slow -- throttled?

2021-08-29 Thread Takashi Yano via Cygwin
On Mon, 30 Aug 2021 09:13:14 +0900 Takashi Yano wrote: > On Sun, 29 Aug 2021 17:04:56 -0400 > Ken Brown wrote: > > On 8/29/2021 5:07 AM, Takashi Yano via Cygwin wrote: > > > On Sat, 28 Aug 2021 18:41:02 +0900 > > > Takashi Yano wrote: > > >> On Sat, 28 Aug 2021 10:43:27 +0200 > > >> Corinna Vinsche

Re: cygrunsrv + sshd + rsync = 20 times too slow -- throttled?

2021-08-29 Thread Takashi Yano via Cygwin
On Sun, 29 Aug 2021 17:04:56 -0400 Ken Brown wrote: > On 8/29/2021 5:07 AM, Takashi Yano via Cygwin wrote: > > On Sat, 28 Aug 2021 18:41:02 +0900 > > Takashi Yano wrote: > >> On Sat, 28 Aug 2021 10:43:27 +0200 > >> Corinna Vinschen wrote: > >>> On Aug 28 02:21, Takashi Yano via Cygwin wrote: >

Re: cygrunsrv + sshd + rsync = 20 times too slow -- throttled?

2021-08-29 Thread Ken Brown via Cygwin
On 8/29/2021 4:41 AM, Takashi Yano wrote: Hi Ken, On Sat, 28 Aug 2021 16:55:52 -0400 Ken Brown wrote: On 8/28/2021 11:43 AM, Takashi Yano via Cygwin wrote: On Sat, 28 Aug 2021 13:58:08 +0200 Corinna Vinschen wrote: On Aug 28 18:41, Takashi Yano via Cygwin wrote: On Sat, 28 Aug 2021 10:43:27

Re: cygrunsrv + sshd + rsync = 20 times too slow -- throttled?

2021-08-29 Thread Ken Brown via Cygwin
On 8/29/2021 3:24 PM, Chris Roehrig wrote: I'd be happy to test it out if you can give me some commands to git it and build it to replace my existing stock Cygwin installation. (Or it is just the cygwin.dll I'd need to replace?) My daily backup scripts use a lot of pipes and named pipes. It's

Re: cygrunsrv + sshd + rsync = 20 times too slow -- throttled?

2021-08-29 Thread Ken Brown via Cygwin
On 8/29/2021 3:37 PM, Takashi Yano wrote: On Sun, 29 Aug 2021 11:57:04 -0400 Ken Brown wrote: On 8/29/2021 5:07 AM, Takashi Yano via Cygwin wrote: On Sat, 28 Aug 2021 18:41:02 +0900 Takashi Yano wrote: On Sat, 28 Aug 2021 10:43:27 +0200 Corinna Vinschen wrote: On Aug 28 02:21, Takashi Yano vi

Re: cygrunsrv + sshd + rsync = 20 times too slow -- throttled?

2021-08-29 Thread Ken Brown via Cygwin
On 8/29/2021 5:07 AM, Takashi Yano via Cygwin wrote: On Sat, 28 Aug 2021 18:41:02 +0900 Takashi Yano wrote: On Sat, 28 Aug 2021 10:43:27 +0200 Corinna Vinschen wrote: On Aug 28 02:21, Takashi Yano via Cygwin wrote: On Fri, 27 Aug 2021 12:00:50 -0400 Ken Brown wrote: Two years ago I thought I

Re: cygrunsrv + sshd + rsync = 20 times too slow -- throttled?

2021-08-29 Thread Takashi Yano via Cygwin
On Sun, 29 Aug 2021 11:57:04 -0400 Ken Brown wrote: > On 8/29/2021 5:07 AM, Takashi Yano via Cygwin wrote: > > On Sat, 28 Aug 2021 18:41:02 +0900 > > Takashi Yano wrote: > >> On Sat, 28 Aug 2021 10:43:27 +0200 > >> Corinna Vinschen wrote: > >>> On Aug 28 02:21, Takashi Yano via Cygwin wrote: >

Re: cygrunsrv + sshd + rsync = 20 times too slow -- throttled?

2021-08-29 Thread Chris Roehrig
I'd be happy to test it out if you can give me some commands to git it and build it to replace my existing stock Cygwin installation. (Or it is just the cygwin.dll I'd need to replace?) My daily backup scripts use a lot of pipes and named pipes. -- Chris On Sun Aug 29 2021, at 8:57 AM, Ken Br

Re: cygrunsrv + sshd + rsync = 20 times too slow -- throttled?

2021-08-29 Thread Ken Brown via Cygwin
On 8/29/2021 5:07 AM, Takashi Yano via Cygwin wrote: On Sat, 28 Aug 2021 18:41:02 +0900 Takashi Yano wrote: On Sat, 28 Aug 2021 10:43:27 +0200 Corinna Vinschen wrote: On Aug 28 02:21, Takashi Yano via Cygwin wrote: On Fri, 27 Aug 2021 12:00:50 -0400 Ken Brown wrote: Two years ago I thought I

Re: cygrunsrv + sshd + rsync = 20 times too slow -- throttled?

2021-08-29 Thread Corinna Vinschen via Cygwin
On Aug 29 00:43, Takashi Yano via Cygwin wrote: > On Sat, 28 Aug 2021 13:58:08 +0200 > Corinna Vinschen wrote: > > On Aug 28 18:41, Takashi Yano via Cygwin wrote: > > > On Sat, 28 Aug 2021 10:43:27 +0200 > > > Corinna Vinschen wrote: > > > > On Aug 28 02:21, Takashi Yano via Cygwin wrote: > > > > >

Re: cygrunsrv + sshd + rsync = 20 times too slow -- throttled?

2021-08-29 Thread Takashi Yano via Cygwin
On Sat, 28 Aug 2021 18:41:02 +0900 Takashi Yano wrote: > On Sat, 28 Aug 2021 10:43:27 +0200 > Corinna Vinschen wrote: > > On Aug 28 02:21, Takashi Yano via Cygwin wrote: > > > On Fri, 27 Aug 2021 12:00:50 -0400 > > > Ken Brown wrote: > > > > Two years ago I thought I needed nt_create to avoid probl

Re: cygrunsrv + sshd + rsync = 20 times too slow -- throttled?

2021-08-29 Thread Takashi Yano via Cygwin
Hi Ken, On Sat, 28 Aug 2021 16:55:52 -0400 Ken Brown wrote: > On 8/28/2021 11:43 AM, Takashi Yano via Cygwin wrote: > > On Sat, 28 Aug 2021 13:58:08 +0200 > > Corinna Vinschen wrote: > >> On Aug 28 18:41, Takashi Yano via Cygwin wrote: > >>> On Sat, 28 Aug 2021 10:43:27 +0200 > >>> Corinna Vinsche

Re: cygrunsrv + sshd + rsync = 20 times too slow -- throttled?

2021-08-28 Thread Ken Brown via Cygwin
On 8/28/2021 11:43 AM, Takashi Yano via Cygwin wrote: On Sat, 28 Aug 2021 13:58:08 +0200 Corinna Vinschen wrote: On Aug 28 18:41, Takashi Yano via Cygwin wrote: On Sat, 28 Aug 2021 10:43:27 +0200 Corinna Vinschen wrote: On Aug 28 02:21, Takashi Yano via Cygwin wrote: On Fri, 27 Aug 2021 12:00

Re: cygrunsrv + sshd + rsync = 20 times too slow -- throttled?

2021-08-28 Thread Takashi Yano via Cygwin
On Sat, 28 Aug 2021 13:58:08 +0200 Corinna Vinschen wrote: > On Aug 28 18:41, Takashi Yano via Cygwin wrote: > > On Sat, 28 Aug 2021 10:43:27 +0200 > > Corinna Vinschen wrote: > > > On Aug 28 02:21, Takashi Yano via Cygwin wrote: > > > > On Fri, 27 Aug 2021 12:00:50 -0400 > > > > Ken Brown wrote: >

Re: cygrunsrv + sshd + rsync = 20 times too slow -- throttled?

2021-08-28 Thread Ken Brown via Cygwin
On 8/28/2021 4:43 AM, Corinna Vinschen via Cygwin wrote: On Aug 28 02:21, Takashi Yano via Cygwin wrote: On Fri, 27 Aug 2021 12:00:50 -0400 Ken Brown wrote: Two years ago I thought I needed nt_create to avoid problems when calling set_pipe_non_blocking. Are you saying that's not an issue? Is

Re: cygrunsrv + sshd + rsync = 20 times too slow -- throttled?

2021-08-28 Thread Corinna Vinschen via Cygwin
On Aug 28 18:41, Takashi Yano via Cygwin wrote: > On Sat, 28 Aug 2021 10:43:27 +0200 > Corinna Vinschen wrote: > > On Aug 28 02:21, Takashi Yano via Cygwin wrote: > > > On Fri, 27 Aug 2021 12:00:50 -0400 > > > Ken Brown wrote: > > > > Two years ago I thought I needed nt_create to avoid problems whe

Re: cygrunsrv + sshd + rsync = 20 times too slow -- throttled?

2021-08-28 Thread Takashi Yano via Cygwin
On Sat, 28 Aug 2021 10:43:27 +0200 Corinna Vinschen wrote: > On Aug 28 02:21, Takashi Yano via Cygwin wrote: > > On Fri, 27 Aug 2021 12:00:50 -0400 > > Ken Brown wrote: > > > Two years ago I thought I needed nt_create to avoid problems when calling > > > set_pipe_non_blocking. Are you saying that

Re: cygrunsrv + sshd + rsync = 20 times too slow -- throttled?

2021-08-28 Thread Corinna Vinschen via Cygwin
On Aug 28 02:21, Takashi Yano via Cygwin wrote: > On Fri, 27 Aug 2021 12:00:50 -0400 > Ken Brown wrote: > > Two years ago I thought I needed nt_create to avoid problems when calling > > set_pipe_non_blocking. Are you saying that's not an issue? Is > > set_pipe_non_blocking unnecessary? Is that

Re: cygrunsrv + sshd + rsync = 20 times too slow -- throttled?

2021-08-27 Thread Takashi Yano via Cygwin
On Sat, 28 Aug 2021 11:00:24 +0900 Takashi Yano wrote: > On Sat, 28 Aug 2021 02:21:11 +0900 > Takashi Yano wrote: > > > On Fri, 27 Aug 2021 12:00:50 -0400 > > Ken Brown wrote: > > > > > On 8/27/2021 7:24 AM, Takashi Yano wrote: > > > > On Thu, 26 Aug 2021 18:18:29 -0400 > > > > Ken Brown wrote: >

Re: cygrunsrv + sshd + rsync = 20 times too slow -- throttled?

2021-08-27 Thread Takashi Yano via Cygwin
On Sat, 28 Aug 2021 02:21:11 +0900 Takashi Yano wrote: > On Fri, 27 Aug 2021 12:00:50 -0400 > Ken Brown wrote: > > > On 8/27/2021 7:24 AM, Takashi Yano wrote: > > > On Thu, 26 Aug 2021 18:18:29 -0400 > > > Ken Brown wrote: > > >> On 8/26/2021 11:56 AM, Ken Brown via Cygwin wrote: > > [...] > > >>

Re: cygrunsrv + sshd + rsync = 20 times too slow -- throttled?

2021-08-27 Thread Takashi Yano via Cygwin
On Fri, 27 Aug 2021 12:00:50 -0400 Ken Brown wrote: > On 8/27/2021 7:24 AM, Takashi Yano wrote: > > On Thu, 26 Aug 2021 18:18:29 -0400 > > Ken Brown wrote: > >> On 8/26/2021 11:56 AM, Ken Brown via Cygwin wrote: > [...] > >> In case you want to try out my proposed change, I've just rebased the >

Re: cygrunsrv + sshd + rsync = 20 times too slow -- throttled?

2021-08-27 Thread Ken Brown via Cygwin
On 8/27/2021 7:24 AM, Takashi Yano wrote: On Thu, 26 Aug 2021 18:18:29 -0400 Ken Brown wrote: On 8/26/2021 11:56 AM, Ken Brown via Cygwin wrote: [...] In case you want to try out my proposed change, I've just rebased the patches to the current master and pushed them to a new topic/pipe branch.

Re: cygrunsrv + sshd + rsync = 20 times too slow -- throttled?

2021-08-27 Thread Takashi Yano via Cygwin
On Thu, 26 Aug 2021 18:18:29 -0400 Ken Brown wrote: > On 8/26/2021 11:56 AM, Ken Brown via Cygwin wrote: > > On 8/25/2021 5:29 PM, Takashi Yano wrote: > >> On Wed, 25 Aug 2021 13:52:19 -0400 > >> Ken Brown wrote: > >>> On 8/25/2021 7:18 AM, Takashi Yano via Cygwin wrote: > On Tue, 24 Aug 2021

Re: cygrunsrv + sshd + rsync = 20 times too slow -- throttled?

2021-08-26 Thread Ken Brown via Cygwin
On 8/26/2021 11:56 AM, Ken Brown via Cygwin wrote: On 8/25/2021 5:29 PM, Takashi Yano wrote: On Wed, 25 Aug 2021 13:52:19 -0400 Ken Brown wrote: On 8/25/2021 7:18 AM, Takashi Yano via Cygwin wrote: On Tue, 24 Aug 2021 12:49:52 -0700 Chris Roehrig wrote: I have a network of Windows, Linux and M

Re: cygrunsrv + sshd + rsync = 20 times too slow -- throttled?

2021-08-26 Thread Ken Brown via Cygwin
On 8/25/2021 5:29 PM, Takashi Yano wrote: On Wed, 25 Aug 2021 13:52:19 -0400 Ken Brown wrote: On 8/25/2021 7:18 AM, Takashi Yano via Cygwin wrote: On Tue, 24 Aug 2021 12:49:52 -0700 Chris Roehrig wrote: I have a network of Windows, Linux and Mac machines and I use rsync to synchronize various

Re: cygrunsrv + sshd + rsync = 20 times too slow -- throttled?

2021-08-26 Thread Ken Brown via Cygwin
On 8/25/2021 4:33 PM, Mario Emmenlauer wrote: On 25.08.21 19:52, Ken Brown via Cygwin wrote: A couple years ago I had an idea for changing the pipe implementation to avoid overlapped I/O:    https://cygwin.com/pipermail/cygwin-patches/2019q2/009393.html    https://cygwin.com/pipermail/cygwin-p

Re: cygrunsrv + sshd + rsync = 20 times too slow -- throttled?

2021-08-26 Thread Ken Brown via Cygwin
On 8/25/2021 2:18 PM, Chris Roehrig wrote: On Wed Aug 25 2021, at 10:52 AM, Ken Brown via Cygwin wrote: A couple years ago I had an idea for changing the pipe implementation to avoid overlapped I/O: https://cygwin.com/pipermail/cygwin-patches/2019q2/009393.html https://cygwin.com/pipermai

Re: cygrunsrv + sshd + rsync = 20 times too slow -- throttled?

2021-08-25 Thread Takashi Yano via Cygwin
On Wed, 25 Aug 2021 13:52:19 -0400 Ken Brown wrote: > On 8/25/2021 7:18 AM, Takashi Yano via Cygwin wrote: > > On Tue, 24 Aug 2021 12:49:52 -0700 > > Chris Roehrig wrote: > >> I have a network of Windows, Linux and Mac machines and I use rsync to > >> synchronize various directories between them.

Re: cygrunsrv + sshd + rsync = 20 times too slow -- throttled?

2021-08-25 Thread Mario Emmenlauer
On 25.08.21 19:52, Ken Brown via Cygwin wrote: A couple years ago I had an idea for changing the pipe implementation to avoid overlapped I/O:   https://cygwin.com/pipermail/cygwin-patches/2019q2/009393.html   https://cygwin.com/pipermail/cygwin-patches/2019q2/009423.html I never followed up

Re: cygrunsrv + sshd + rsync = 20 times too slow -- throttled?

2021-08-25 Thread Chris Roehrig
It looks like there was a previous (2013) patch and attempt to add a pipe_nooverlap CYGWIN option that was rejected by the maintainers: https://stackoverflow.com/questions/10385424/good-alternatives-to-cygwin-cygwin-doesnt-support-natively-support-win32-app Is this something that can be revisite

Re: cygrunsrv + sshd + rsync = 20 times too slow -- throttled?

2021-08-25 Thread Chris Roehrig
On Wed Aug 25 2021, at 10:52 AM, Ken Brown via Cygwin wrote: > A couple years ago I had an idea for changing the pipe implementation to > avoid overlapped I/O: > > https://cygwin.com/pipermail/cygwin-patches/2019q2/009393.html > https://cygwin.com/pipermail/cygwin-patches/2019q2/009423.html >

Re: cygrunsrv + sshd + rsync = 20 times too slow -- throttled?

2021-08-25 Thread Ken Brown via Cygwin
On 8/25/2021 7:18 AM, Takashi Yano via Cygwin wrote: On Tue, 24 Aug 2021 12:49:52 -0700 Chris Roehrig wrote: I have a network of Windows, Linux and Mac machines and I use rsync to synchronize various directories between them. I'm trying to figure out why my rsync transfers are so slow (<4 MB/s

Re: cygrunsrv + sshd + rsync = 20 times too slow -- throttled?

2021-08-25 Thread Chris Roehrig
The FILE_FLAG_OVERLAPPED avenue looks promising. I get exactly the same results as you using 'scp': 4MB/s in either direction (with either remote endpoint.) I guess the next step is to install a build environment and build rsync and sshd ... On Wed Aug 25 2021, at 4:18 AM, Takashi Ya

Re: cygrunsrv + sshd + rsync = 20 times too slow -- throttled?

2021-08-25 Thread Chris Roehrig
It was set to "Programs".Changing it to "Background services" didn't make a difference. TCPOptimizer can adjust 2 registry entries that I think are related to that Windows Setting: [HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows NT\CurrentVersion\Multimedia\SystemProfile] "NetworkT

Re: cygrunsrv + sshd + rsync = 20 times too slow -- throttled?

2021-08-25 Thread Takashi Yano via Cygwin
On Tue, 24 Aug 2021 12:49:52 -0700 Chris Roehrig wrote: > I have a network of Windows, Linux and Mac machines and I use rsync to > synchronize various directories between them. > > I'm trying to figure out why my rsync transfers are so slow (<4 MB/s) only > when the remote endpoint is Cygwin rsy

Re: cygrunsrv + sshd + rsync = 20 times too slow -- throttled?

2021-08-24 Thread Mark Geisert
NightStrike via Cygwin wrote: Older versions of windows had a setting to optimize the OS for either background services or foreground applications. One of the things this did was throttle network usage. I don't know if windows 10 has the same setting though. Yes, it does. Getting to it is a pa

Re: cygrunsrv + sshd + rsync = 20 times too slow -- throttled?

2021-08-24 Thread Chris Roehrig
Here's the other direction with rsync also in --daemon mode: # ON WINDOWS (from a mintty terminal): dd if=/dev/urandom of=/tmp/bigfile.bin bs=1M count=2000 # create 2GB file printf "[TMP]\npath = /tmp\n" > /etc/rsyncd.conf # create rsyncd.conf

Re: cygrunsrv + sshd + rsync = 20 times too slow -- throttled?

2021-08-24 Thread NightStrike via Cygwin
On Tue, Aug 24, 2021, 09:50 Chris Roehrig wrote: > I have a network of Windows, Linux and Mac machines and I use rsync to > synchronize various directories between them. > > I'm trying to figure out why my rsync transfers are so slow (<4 MB/s) only > when the remote endpoint is Cygwin rsync over

Re: cygrunsrv + sshd + rsync = 20 times too slow -- throttled?

2021-08-24 Thread Chris Roehrig
Here's my test in a nutshell: # ON WINDOWS: install Cygwin and enable cygsshd ssh-host-config -y # set up authorized keys, etc to make things easier # LINUX: Create a 2GB random file on Linux: dd if=/dev/urandom of=/tmp/bigfile.bin bs=1M count=2000 # WINDOWS: rsync "pull"

Re: cygrunsrv + sshd + rsync = 20 times too slow -- throttled?

2021-08-24 Thread Mark Geisert
Chris Roehrig wrote: I have a network of Windows, Linux and Mac machines and I use rsync to synchronize various directories between them. I'm trying to figure out why my rsync transfers are so slow (<4 MB/s) only when the remote endpoint is Cygwin rsync over sshd (with both a Linux or Cygwin r

cygrunsrv + sshd + rsync = 20 times too slow -- throttled?

2021-08-24 Thread Chris Roehrig
I have a network of Windows, Linux and Mac machines and I use rsync to synchronize various directories between them. I'm trying to figure out why my rsync transfers are so slow (<4 MB/s) only when the remote endpoint is Cygwin rsync over sshd (with both a Linux or Cygwin rsync client). In all