Re: [Bug-tar] seek_hole proposal [v2]

[Sergey Poznyakoff]

Hi Pavel,

Modified patch applied (b684326e).

Regards,
Sergey

Re: [Bug-tar] seek_hole proposal [v2]

[Sergey Poznyakoff]

Paul Eggert  ha escrit:

> Sergey probably just hasn't had time to review the patch.

Yes, that's right.  But I'm in process.  Neither that patch nor other
ones submitted by Pavel remain unnoticed.

Regards,
Sergey

Re: [Bug-tar] seek_hole proposal [v2]

[Paul Eggert]

Pavel Raiskup wrote:

so far there is no response from tar
maintainers -- sounds like something we don't want to have in GNU tar?


tar should do the right thing with SEEK_HOLE etc. Sergey probably just hasn't 
had time to review the patch.

Re: [Bug-tar] seek_hole proposal [v2]

[Pavel Raiskup]

On Wednesday, March 05, 2014 12:14:17 Joerg Schilling wrote:
 Pavel Raiskup prais...@redhat.com wrote:

  Note that after discussion [1] I still think that existing ST_IS_SPARSE
  macro is better for file-sparseness detection than using SEEK_HOLE (not
  worth having additional syscalls open~seek~close).
 
 Your code is not compatible to the SEEK_HOLE interface. A file is sparse in 
 case that pathconf()/fpathconf(f, _PC_MIN_HOLE_SIZE) return a positive number 
 and lseek(f, (off_t)0, SEEK_HOLE) returns a number  stat.st_size.

Yes, apart from pathconf, the patch-v1 contained bug - I should react on
hole_offset, not data_offset.  And there was yet another bug - we need to
reposition the file offset to the beginning in this case (no FS support);
so there needs to be done this change:

   if (offset == 0 /* first loop */
-   data_offset == st-stat.st_size)
-return false;
+   data_offset == 0
+   hole_offset == st-stat.st_size)
+{
+  lseek (fd, 0, SEEK_SET);
+  return false;
+}

Joerg, thanks for pointing that out.

 see man pathconf:
 [...]
 
 In other cases, the file stil may be sparse, but the filesystem does not 
 support SEEK_HOLE. I e.g. doubt that Linux correctly implements SEEK_HOLE
 for NFS.

Correct, Linux's NFS returns just the virtual hole at the end of sparse
file, which is now OK - we would still fallback to raw hole detection.
Fixed patch v2 attached.

Pavel
From 0563cfd261f6c47f25924ae6fef542230fdf2794 Mon Sep 17 00:00:00 2001
From: Pavel Raiskup prais...@redhat.com
Date: Sun, 23 Feb 2014 13:12:54 +0100
Subject: [PATCH] tar: use SEEK_HOLE for hole detection

Reuse the SEEK_HOLE/SEEK_DATA feature of lseek when possible.
This makes the sparse file archivation to be quite faster as tar
does not need to perform additional read of whole file in order to
detect file sparse map.

This lseek feature is not yet fully POSIX but it is fairly widely
implemented these days.

Also implement --hole-detection option for proper method
selection.

* src/common.h (HOLE_DETECTION_RAW, HOLE_DETECTION_SEEK)
(HOLE_DETECTION_ALL, hole_detection): New constants and variable.
* src/sparse.c (sparse_scan_file_wholesparse): New function as a
method for detecting sparse files without any data.
(sparse_scan_file_raw): Renamed from sparse_scan_file, removed the
completely-sparse detection if-branch.
(sparse_scan_file_seek): Implements method for hole detection
using lseek.
(sparse_scan_file): Reimplemented function as a wrapper for all
methods.
* src/tar.c (HOLE_DETECTION_OPTION): New option.
(parse_opt): Handle new --hole-detection option.
* tests/sparse02.at: Use --hole-detection=raw as the seek method
creates little bit bigger archives causing test to fail.
* tests/checkseekhole.c: SEEK_HOLE detection helper.
* tests/sparsemv.at: Likewise.
* tests/sparsemvp.at: Likewise.
* tests/sparse05.at: New test-case.
* tests/testsuite.at: Cover new testcase.
* tests/Makefile.am: Likewise.
* doc/tar.1: Document.
* doc/tar.texi: Likewise.
---
 doc/tar.1 |   6 ++
 doc/tar.texi  |  70 +++-
 src/common.h  |   6 ++
 src/sparse.c  | 173 --
 src/tar.c |  16 +
 tests/Makefile.am |   5 +-
 tests/checkseekhole.c |  90 ++
 tests/sparse02.at |   2 +-
 tests/sparse05.at |  56 
 tests/sparsemv.at |   1 +
 tests/sparsemvp.at|   1 +
 tests/testsuite.at|  16 +
 12 files changed, 392 insertions(+), 50 deletions(-)
 create mode 100644 tests/checkseekhole.c
 create mode 100644 tests/sparse05.at

diff --git a/doc/tar.1 b/doc/tar.1
index b33f55b..d2898b6 100644
--- a/doc/tar.1
+++ b/doc/tar.1
@@ -259,6 +259,12 @@ When listing or extracting, the actual contents of \fIFILE\fR is not
 inspected, it is needed only due to syntactical requirements.  It is
 therefore common practice to use \fB/dev/null\fR in its place.
 .TP
+\fB\-\-hole\-detection\fR=\fIMETHOD\fR
+Use method to detect holes in sparse files.  This option implies
+\fB\-\-sparse\fR.  Currently there are \fIseek\fR and \fIraw\fR methods
+implemented.  Default is \fIseek\fR with fallback to \fIraw\fR when not
+applicable.
+.TP
 \fB\-G\fR, \fB\-\-incremental\fR
 Handle old GNU-format incremental backups.
 .TP
diff --git a/doc/tar.texi b/doc/tar.texi
index 9bb5a83..0e894f9 100644
--- a/doc/tar.texi
+++ b/doc/tar.texi
@@ -2748,6 +2748,13 @@ they refer to, instead of creating usual hard link members.
 @command{tar} will print out a short message summarizing the operations and
 options to @command{tar} and exit. @xref{help}.
 
+@opsummary{hole-detection}
+@item --hole-detection=@var{method}
+Use method to detect holes in sparse files.  This option implies
+@option{--sparse}.  Currently there are @var{seek} and @var{raw} methods
+implemented.  Default is @var{seek} with fallback to @var{raw} when not
+applicable. @xref{sparse}.
+
 @opsummary{ignore-case}