Welcome,

I've been using btrfs for over a 3 months to store my personal data on my NAS server. Almost all interactions with files on the server are done using unison synchronizer. After another use of bedup (https://github.com/g2p/bedup) on my btrfs volume I experienced huge perfomance loss with synchronization. It now takes over 3 hours what have taken only 15 minutes! File browsing is not affected; but it takes forever to read contents of the files!

When I use `iotop -o -d 30` (which measures I/O activity for 30-second interval) I can see:

Total DISK READ:      98.66 K/s | Total DISK WRITE:     826.55 K/s
  TID  PRIO  USER     DISK READ  DISK WRITE  SWAPIN     IO> COMMAND
4296 be/4 root 3.99 K/s 408.59 K/s 0.00 % 98.64 % [btrfs-transacti]
 6407 be/4 adam       94.14 K/s    0.00 B/s  0.00 % 85.24 % unison -server
  311 be/4 root        0.00 B/s    0.00 B/s  0.00 % 58.20 % [md1_raid6]
  354 be/3 root        0.00 B/s    2.26 K/s  0.00 % 24.29 % [jbd2/md0-8]
  306 be/4 root        0.00 B/s    0.00 B/s  0.00 %  4.79 % [md0_raid1]
 1229 be/4 syslog      0.00 B/s  136.15 B/s  0.00 %  0.00 % rsyslogd -c5
1744 be/4 root 0.00 B/s 136.15 B/s 0.00 % 0.00 % console-kit-daemon --no-daemon

I expect no writes at all since the statistics were taken during the "Looking for changes" phse. Normally, the `unison -server` process shold have at least 5 M/s disk read speed. (The block device the btrfs is build on has a measured capability of 50 M/s sequential throughput)

When I pause the `unison -server` process (with htop), the disk activity persists of another 5-30 seconds, so I am infer, that the btrfs is doing some house-keeping work, and this is the reason I decided to post the email on this list. I suspect, that this house-keeping work has a time granularity of 5-30 seconds, and during this time access to the filesystem is delayed. The problem is not specific to the unison. This background process is triggered by just reading the file contents. Once the system is through and the file is read, than all subsequent attempts to read it are fine, even if I drop the cache (i.e. echo 3 > /proc/sys/vm/drop_caches). But after a while (after reboot) the performance hit recurs.

The questions are:
1. What sort of work is btrfs doing?
What is it writing (and why is it writing 100x bytes more than reading)?
2. Why does it take it so long?
3. What can I do to speed-up the process?
4. What can I do to prevent it from happening again?

Here are details about my system that might help you with the diagnose. If it is not enough,

I suspect it has something to do with snapshots I make for backup. I have 35 of them, and I ask bedup to find duplicates across all subvolumes. But on the other hand it is supposed to work since kernel 3.5, and the filesystem has never seen kernel older than 3.6.

My filesystem /dev/vg-adama-docs/lv-adama-docs is 372GB in size, and is a quite complex setup: It is based on logical volume (LVM2), which has a single physical volume made by dm-crypt device /dev/dm-1, which subsequently sits on top of /dev/md1 linux raid 6, which is built with 4 identical 186GB GPT partitions on each of my SATA 3TB hard drives.

There are 272k files on the system (excluding 35 snaphosts), 23k folders and 104 GB data.
$ df /mnt/adama-docs -h
Filesystem Size Used Avail Use% Mounted on /dev/mapper/vg--adama--docs-lv--adama--docs 373G 85G 288G 23% /mnt/adama-docs

I was always using the latest kernel (its 3.7.1-030701-generic at the moment) on my Ubuntu Quantal server.

--

Adam Ryczkowski
Skype:sisteczko <skype:sisteczko>

--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Reply via email to