Re: Btrfs progs release 4.8.3

2016-11-14 Thread David Sterba 
On Sun, Nov 13, 2016 at 02:28:02PM +0100, Henk Slager wrote:
> On Fri, Nov 11, 2016 at 4:38 PM, David Sterba  wrote:
> > Hi,
> >
> > btrfs-progs version 4.8.3 have been released. Handful of fixes and lots of
> > cleanups.
> >
> > Changes:
> >   * check:
> > * support for clearing space cache (v1)
> > * size reduction of inode backref structure
> >   * send:
> > * fix handling of multiple snapshots (-p and -c options)
> > * transfer buffer increased (should reduce number of context switches)
> > * reuse existing file for output (-f), eg. when root cannot create 
> > files (NFS)
> >   * dump-tree:
> > * print missing items for various structures
> > * new: dev stats, balance status item
> > * sync key names with kernel (the persistent items)
> >   * subvol show: now able to print the toplevel subvolume -- the creation 
> > time
> > might be wrong though
> >   * mkfs:
> > * store the creation time of toplevel root inode
> 
> It looks like commit 5c4d53450b2c6ff7169c99f9158c14ae96b7b0a8
>  (btrfs-progs: mkfs: store creation time of the toplevel subvolume'')
> is not enough to display the creation time of a newly created fs with
> tools v4.8.3, or am I missing something?

This is known and mentioned on the changelog entry above mkfs.

> With kernel 4.8.6-2-default as well as 4.9.0-rc4-1-default:
> 
> # /net/src/btrfs-progs/mkfs.btrfs -L ctimetest -m single /dev/loop0
> btrfs-progs v4.8.3
> See http://btrfs.wiki.kernel.org for more information.
> 
> Performing full device TRIM (100.00GiB) ...
> Label:  ctimetest
> UUID:   d65486f0-368b-4b2a-962b-176cd945feb5
> Node size:  16384
> Sector size:4096
> Filesystem size:100.00GiB
> Block group profiles:
>   Data: single8.00MiB
>   Metadata: single8.00MiB
>   System:   single4.00MiB
> SSD detected:   no
> Incompat features:  extref, skinny-metadata
> Number of devices:  1
> Devices:
>IDSIZE  PATH
> 1   100.00GiB  /dev/loop0
> 
> # mount /dev/loop0 /mnt
> # cd /mnt
> # ls > test
> # sync
> # /net/src/btrfs-progs/btrfs sub sh .
> /mnt
> Name:   
> UUID:   -
> Parent UUID:-
> Received UUID:  -
> Creation time:  -
> Subvolume ID:   5
> Generation: 9
> Gen at creation:0
> Parent ID:  0
> Top level ID:   0
> Flags:  -
> Snapshot(s):
> 
> I noticed that   btrfs_set_stack_timespec_sec(_item.otime, now);
> is called twice during mkfs.btrfs, but during btrfs sub sh
> get_ri.otime is 0 just before it is formatted/printed.

Your finding is right, I saw the same and would have fixed already, but
it turned out to be more than a simple fix. The on-disk items are
connected to in-memory ones, and the corresponding root item does not
keep them in sync. Ie. setting the otime in one place does not propagate
to the point where 'subvol show' reads it.

> My idea was to
> patch the code (kernel and/or progs) such that I can also put a time
> in some exiting filesystems.

Well, it's possible to write a oneshot tool to set the creation time, if
it's really desired. It would be an arbitrary value set by the user,
because the information about otime cannot be simply derived from the
existing timestamps.
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Btrfs progs release 4.8.3

2016-11-13 Thread Henk Slager 
On Fri, Nov 11, 2016 at 4:38 PM, David Sterba  wrote:
> Hi,
>
> btrfs-progs version 4.8.3 have been released. Handful of fixes and lots of
> cleanups.
>
> Changes:
>   * check:
> * support for clearing space cache (v1)
> * size reduction of inode backref structure
>   * send:
> * fix handling of multiple snapshots (-p and -c options)
> * transfer buffer increased (should reduce number of context switches)
> * reuse existing file for output (-f), eg. when root cannot create files 
> (NFS)
>   * dump-tree:
> * print missing items for various structures
> * new: dev stats, balance status item
> * sync key names with kernel (the persistent items)
>   * subvol show: now able to print the toplevel subvolume -- the creation time
> might be wrong though
>   * mkfs:
> * store the creation time of toplevel root inode

It looks like commit 5c4d53450b2c6ff7169c99f9158c14ae96b7b0a8
 (btrfs-progs: mkfs: store creation time of the toplevel subvolume'')
is not enough to display the creation time of a newly created fs with
tools v4.8.3, or am I missing something?

With kernel 4.8.6-2-default as well as 4.9.0-rc4-1-default:

# /net/src/btrfs-progs/mkfs.btrfs -L ctimetest -m single /dev/loop0
btrfs-progs v4.8.3
See http://btrfs.wiki.kernel.org for more information.

Performing full device TRIM (100.00GiB) ...
Label:  ctimetest
UUID:   d65486f0-368b-4b2a-962b-176cd945feb5
Node size:  16384
Sector size:4096
Filesystem size:100.00GiB
Block group profiles:
  Data: single8.00MiB
  Metadata: single8.00MiB
  System:   single4.00MiB
SSD detected:   no
Incompat features:  extref, skinny-metadata
Number of devices:  1
Devices:
   IDSIZE  PATH
1   100.00GiB  /dev/loop0

# mount /dev/loop0 /mnt
# cd /mnt
# ls > test
# sync
# /net/src/btrfs-progs/btrfs sub sh .
/mnt
Name:   
UUID:   -
Parent UUID:-
Received UUID:  -
Creation time:  -
Subvolume ID:   5
Generation: 9
Gen at creation:0
Parent ID:  0
Top level ID:   0
Flags:  -
Snapshot(s):

I noticed that   btrfs_set_stack_timespec_sec(_item.otime, now);
is called twice during mkfs.btrfs, but during btrfs sub sh
get_ri.otime is 0 just before it is formatted/printed. My idea was to
patch the code (kernel and/or progs) such that I can also put a time
in some exiting filesystems.

> * print UUID in the summary
>   * build: travis CI for devel
>   * other:
> * lots of cleanups and refactoring
> * switched to on-stack path structure
> * fixes from coverity, asan, ubsan
> * new tests
> * updates in testing infrastructure
> * fixed convert test 005
>
> Changes since rc1:
>   * fixed convert test 005
>   * updates in testing infrastructure
>   * mkfs: print UUID in the summary
>
> Tarballs: https://www.kernel.org/pub/linux/kernel/people/kdave/btrfs-progs/
> Git: git://git.kernel.org/pub/scm/linux/kernel/git/kdave/btrfs-progs.git
>
> Shortlog:
>
> David Sterba (147):
>   btrfs-progs: tests: switch to dump- commands from inspect
>   btrfs-progs: convert: switch more messages to common helpers
>   btrfs-progs: qgroups show: handle errors when resolving root id
>   btrfs-progs: remove unused function btrfs_get_path_rootid
>   btrfs-progs: rename lookup_ino_rootid
>   btrfs-progs: use existing rootid resolving helper in 
> btrfs_list_get_path_rootid
>   btrfs-progs: opencode trivial helper __free_all_subvolumn
>   btrfs-progs: remove leading underscores from several helper
>   btrfs-progs: use symbolic tree name when searching
>   btrfs-progs: list: constify prefix arugment
>   btrfs-progs: use enum for list layout type
>   btrfs-progs: list: rename some helpers
>   btrfs-progs: list: switch to common message helpers
>   btrfs-progs: subvol list: setup list filters later
>   btrfs-progs: return void from btrfs_list_setup_filter
>   btrfs-progs: subvol list: cleanup layout argument setup
>   btrfs-progs: subvol list: remove useless comments
>   btrfs-progs: subvol list: simplify value assignments
>   btrfs-progs: subvol list: consilidate naming of otime varaibles
>   btrfs-progs: subvol list: add simplified helper for adding root backrefs
>   btrfs-progs: subvol list: consolidate uuid types accross functions
>   btrfs-progs: remove trivial helper root_lookup_init
>   btrfs-progs: subvol list: remove ugly goto construct
>   btrfs-progs: subvol list: better error message if subvol insertion fails
>   btrfs-progs: subvol show: print more details about toplevel subvolume
>   btrfs-progs: dump-tree: print missing dev_item data
>   btrfs-progs: dump-tree: print missing chunk data
>

Btrfs progs release 4.8.3

2016-11-11 Thread David Sterba 
Hi,

btrfs-progs version 4.8.3 have been released. Handful of fixes and lots of
cleanups.

Changes:
  * check:
* support for clearing space cache (v1)
* size reduction of inode backref structure
  * send:
* fix handling of multiple snapshots (-p and -c options)
* transfer buffer increased (should reduce number of context switches)
* reuse existing file for output (-f), eg. when root cannot create files 
(NFS)
  * dump-tree:
* print missing items for various structures
* new: dev stats, balance status item
* sync key names with kernel (the persistent items)
  * subvol show: now able to print the toplevel subvolume -- the creation time
might be wrong though
  * mkfs:
* store the creation time of toplevel root inode
* print UUID in the summary
  * build: travis CI for devel
  * other:
* lots of cleanups and refactoring
* switched to on-stack path structure
* fixes from coverity, asan, ubsan
* new tests
* updates in testing infrastructure
* fixed convert test 005

Changes since rc1:
  * fixed convert test 005
  * updates in testing infrastructure
  * mkfs: print UUID in the summary

Tarballs: https://www.kernel.org/pub/linux/kernel/people/kdave/btrfs-progs/
Git: git://git.kernel.org/pub/scm/linux/kernel/git/kdave/btrfs-progs.git

Shortlog:

David Sterba (147):
  btrfs-progs: tests: switch to dump- commands from inspect
  btrfs-progs: convert: switch more messages to common helpers
  btrfs-progs: qgroups show: handle errors when resolving root id
  btrfs-progs: remove unused function btrfs_get_path_rootid
  btrfs-progs: rename lookup_ino_rootid
  btrfs-progs: use existing rootid resolving helper in 
btrfs_list_get_path_rootid
  btrfs-progs: opencode trivial helper __free_all_subvolumn
  btrfs-progs: remove leading underscores from several helper
  btrfs-progs: use symbolic tree name when searching
  btrfs-progs: list: constify prefix arugment
  btrfs-progs: use enum for list layout type
  btrfs-progs: list: rename some helpers
  btrfs-progs: list: switch to common message helpers
  btrfs-progs: subvol list: setup list filters later
  btrfs-progs: return void from btrfs_list_setup_filter
  btrfs-progs: subvol list: cleanup layout argument setup
  btrfs-progs: subvol list: remove useless comments
  btrfs-progs: subvol list: simplify value assignments
  btrfs-progs: subvol list: consilidate naming of otime varaibles
  btrfs-progs: subvol list: add simplified helper for adding root backrefs
  btrfs-progs: subvol list: consolidate uuid types accross functions
  btrfs-progs: remove trivial helper root_lookup_init
  btrfs-progs: subvol list: remove ugly goto construct
  btrfs-progs: subvol list: better error message if subvol insertion fails
  btrfs-progs: subvol show: print more details about toplevel subvolume
  btrfs-progs: dump-tree: print missing dev_item data
  btrfs-progs: dump-tree: print missing chunk data
  btrfs-progs: dump-tree: print missing dev_extent data
  btrfs-progs: dump-tree: factor inode_item dump to function
  btrfs-progs: dump-tree: add helper to print timespec
  btrfs-progs: dump-tree: print missing inode_item data
  btrfs-progs: dump-tree: print missing dir_item data
  btrfs-progs: dump-tree: print more root_item data
  btrfs-progs: dump-tree: print missing data for file extent item
  btrfs-progs: mkfs: store creation time of the toplevel subvolume
  btrfs-progs: introduce key type for persistent temporary items
  btrfs-progs: print-tree: extract offset from the item key
  btrfs-progs: introduce key type for persistent permanent items
  btrfs-progs: switch dev stats item to the permanent item key
  btrfs-progs: teach print_leaf about permanent item subtypes
  btrfs-progs: teach print_leaf about temporary item subtypes
  btrfs-progs: add balance status structures
  btrfs-progs: dump-tree: print balance status item
  btrfs-progs: add dev stats on-disk structure
  btrfs-progs: dump-tree: print dev stats
  btrfs-progs: dump-tree: consolidate data key names
  btrfs-progs: remove unused parameter from print_inode_item
  btrfs-progs: dump-tree: pass item size instead of item
  btrfs-progs: dump-tree: return void from print_* functions
  btrfs-progs: dump-tree: simplify and fix check for empty uuid
  btrfs-progs: dump-tree: constify char argument in print_root_ref
  btrfs-progs: dump-tree: add untyped item ptr helper and use it
  btrfs-progs: dump-tree: rename extent buffer variable in btrfs_print_leaf
  btrfs-progs: dump-tree: move variable declarations closer to their use
  btrfs-progs: dump-tree: move remaining item variables to the scope of use
  btrfs-progs: dump-tree: cleanup types for item number iteration
  btrfs-progs: dump-tree: convert key-to-string to table
  btrfs-progs: dump-tree: convert