On Sunday, January 12, 2014 5:57:16 PM UTC-8, Daniel Farina wrote: > > On Sun, Jan 12, 2014 at 5:46 PM, Joe Van Dyk <[email protected]<javascript:>> > wrote: > > On Sunday, January 12, 2014 5:45:49 PM UTC-8, Joe Van Dyk wrote: > >> > >> On Sunday, January 12, 2014 5:24:41 PM UTC-8, Daniel Farina wrote: > >>> > >>> > >>> On Jan 12, 2014 4:54 PM, "Joe Van Dyk" <[email protected]> wrote: > >>> > > >>> > When doing a backup-fetch, I get about 6.5-8 megabytes per second > >>> > download from S3. I was expecting it to be a bit faster. Measured it > through > >>> > iftop and nethogs. > >>> > > >>> > I'm noticing that wal-e is using close to 100% of cpu, top shows it > >>> > hovering at about 97%. > >>> > >>> Python 2.6 you say? This version cannot block 3DES encipherment > because > >>> of a missing feature of the ssl module. This fix helped my > surprisingly > >>> slow cases a lot. > >>> > >>> To confirm, try running Linux 'perf top' and see what symbols crop up. > >>> The 3DES symbols in openssl were perspicuously named, which is how > this bug > >>> was diagnosed. > >> > >> > >> I upgraded to Python 2.7.6, no changes in speed. I'll check out perf > >> tomorrow. > > Unfortunately. The 3DES blocking helped my cases a lot. > > > Should wal-e be using such a high cpu percentage? > > I haven't tuned WAL-E very much for CPU usage, but 6 megabytes per > second is suspicious. Can you get a faster rate if you send /dev/zero > to the disk? >
I can write at about 68 MB/s. > > My recollection is the bottleneck (as seen by 'perf top') is in > "hypervisor_page" or something named like that. That may be > correlated to something else that can be optimized, yet this > hypothesized indirection obscures the issue somewhat. > > Searching the web didn't nail a common cause of that bottleneck, and I > haven't researched beyond that. > -- You received this message because you are subscribed to the Google Groups "wal-e" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/groups/opt_out.
