The branch, v4-2-test has been updated via e9d22f1 s4:torture:vfs_fruit: add a test for stream names via d6ed2f9 s4:torture:vfs_fruit: pass xattr name as arg to torture_setup_local_xattr() via 7ed25fc vfs_catia: run translation on stream names via b827f28 vfs_streams_xattr: stream names may contain colons via 34fc0cd s4:torture:vfs_fruit: copyfile via 9f7ca62 vfs:fruit: implement copyfile style copy_chunk via bb56c5f smb2:ioctl: support for OS X AAPL copyfile style copy_chunk via 9923741 s3:util: add internal function for transfer_file that uses pread/pwrite from baf5328 ctdb-build: Fix building of PCP PMDA module
https://git.samba.org/?p=samba.git;a=shortlog;h=v4-2-test - Log ----------------------------------------------------------------- commit e9d22f1efd008445f56aadf1f4f0b43c5ef2399b Author: Ralph Boehme <s...@samba.org> Date: Sun May 10 11:58:32 2015 +0200 s4:torture:vfs_fruit: add a test for stream names Bug: https://bugzilla.samba.org/show_bug.cgi?id=11278 Signed-off-by: Ralph Boehme <s...@samba.org> Reviewed-by: Stefan Metzmacher <me...@samba.org> (cherry picked from commit 7258061e5e9cd4b68f1c010c3667c3fd2b0663cc) Autobuild-User(v4-2-test): Stefan Metzmacher <me...@samba.org> Autobuild-Date(v4-2-test): Tue Aug 18 01:07:03 CEST 2015 on sn-devel-104 commit d6ed2f96e7a08ffbafc0c680aca7b3d94cb1747f Author: Ralph Boehme <s...@samba.org> Date: Thu Aug 6 13:48:54 2015 +0200 s4:torture:vfs_fruit: pass xattr name as arg to torture_setup_local_xattr() Bug: https://bugzilla.samba.org/show_bug.cgi?id=11278 Signed-off-by: Ralph Boehme <s...@samba.org> Reviewed-by: Stefan Metzmacher <me...@samba.org> (cherry picked from commit fe4909f1cab72f80715a996a63290462102aabc6) commit 7ed25fca4376d29b2fdd21fdb9f996fe313427a8 Author: Ralph Boehme <s...@samba.org> Date: Sat May 9 15:12:41 2015 +0200 vfs_catia: run translation on stream names With vfs_fruit option "fruit:encoding = native" we're already converting stream names that contain illegal NTFS characters from their on-the-wire Unicode Private Range encoding to their native ASCII representation. Unfortunately the reverse mapping for stream names was not perfomed. Bug: https://bugzilla.samba.org/show_bug.cgi?id=11278 Signed-off-by: Ralph Boehme <s...@samba.org> Reviewed-by: Stefan Metzmacher <me...@samba.org> (cherry picked from commit 1db11998bf1b0eef5f543377700b03ab8739338d) commit b827f2821c75c000a1aa11b74abeffa1369d44f7 Author: Ralph Boehme <s...@samba.org> Date: Sat May 9 15:02:03 2015 +0200 vfs_streams_xattr: stream names may contain colons With vfs_fruit option "fruit:encoding = native" we're already converting stream names that contain illegal NTFS characters from their on-the-wire Unicode Private Range encoding to their native ASCII representation. As as result the name of xattrs storing the streams (via vfs_streams_xattr) may contain a colon, so we have to use strrchr_m() instead of strchr_m() for matching the stream type suffix. Bug: https://bugzilla.samba.org/show_bug.cgi?id=11278 Signed-off-by: Ralph Boehme <s...@samba.org> Reviewed-by: Stefan Metzmacher <me...@samba.org> (cherry picked from commit fb9a64ea37dd4b0cd754fe6d421417a4c8ccbc57) commit 34fc0cd146aaa0ead8c9e9c0ca4d7b0a6da6924f Author: Ralph Boehme <s...@samba.org> Date: Wed Jun 10 15:30:04 2015 +0200 s4:torture:vfs_fruit: copyfile Bug: https://bugzilla.samba.org/show_bug.cgi?id=11317 Signed-off-by: Ralph Boehme <s...@samba.org> Reviewed-by: Stefan Metzmacher <me...@samba.org> (cherry picked from commit 43820da1ca2ae09a030a510f42fc1b5d848f7fcc) commit 9f7ca62453ac6c1edc1ce855352529fa3434e6d0 Author: Ralph Boehme <s...@samba.org> Date: Wed Apr 22 22:29:16 2015 +0200 vfs:fruit: implement copyfile style copy_chunk Implement Apple's special copy_chunk ioctl that requests a copy of the whole file along with all attached metadata. These copy_chunk requests have a chunk count of 0 that we translate to a copy_chunk_send VFS call overloading the parameters src_off = dest_off = num = 0. Bug: https://bugzilla.samba.org/show_bug.cgi?id=11317 Signed-off-by: Ralph Boehme <s...@samba.org> Reviewed-by: Stefan Metzmacher <me...@samba.org> (cherry picked from commit e34c879471fe6a4a5c88144394bf621e910cc82b) commit bb56c5f0aa938c34693402ce3f082b3a9c3c0465 Author: Ralph Boehme <s...@samba.org> Date: Wed Apr 22 22:29:16 2015 +0200 smb2:ioctl: support for OS X AAPL copyfile style copy_chunk Apple's special copy_chunk ioctl that requests a copy of the whole file along with all attached metadata. These copy_chunk requests have a chunk count of 0 that we translate to a copy_chunk_send VFS call overloading the parameters src_off = dest_off = num = 0. Bug: https://bugzilla.samba.org/show_bug.cgi?id=11317 Signed-off-by: Ralph Boehme <s...@samba.org> Reviewed-by: Stefan Metzmacher <me...@samba.org> (cherry picked from commit 58480da5066bd33bc73aeb72bd17bd4797c22110) commit 99237419e2565b1f6c792e88023552ddcce2875e Author: Ralph Boehme <s...@samba.org> Date: Mon Apr 27 12:16:16 2015 +0200 s3:util: add internal function for transfer_file that uses pread/pwrite read/write aren't overloaded in the streams VFS modules, using pread/pwrite instead this makes it possible to use transfer_file() with named streams. Bug: https://bugzilla.samba.org/show_bug.cgi?id=11317 Signed-off-by: Ralph Boehme <s...@samba.org> Reviewed-by: Stefan Metzmacher <me...@samba.org> (cherry picked from commit cda8c24a676232bc5834c523407caef8ea9ff038) ----------------------------------------------------------------------- Summary of changes: docs-xml/manpages/vfs_fruit.8.xml | 12 + selftest/target/Samba3.pm | 6 +- selftest/target/Samba4.pm | 6 +- source3/include/transfer_file.h | 6 + source3/include/vfs.h | 1 + source3/lib/util_transfer_file.c | 83 ++++- source3/modules/vfs_catia.c | 58 +++- source3/modules/vfs_fruit.c | 218 +++++++++++++ source3/modules/vfs_streams_xattr.c | 16 +- source3/smbd/smb2_ioctl_network_fs.c | 39 ++- source3/smbd/vfs.c | 18 +- source4/torture/vfs/fruit.c | 593 ++++++++++++++++++++++++++++++++++- 12 files changed, 1042 insertions(+), 14 deletions(-) Changeset truncated at 500 lines: diff --git a/docs-xml/manpages/vfs_fruit.8.xml b/docs-xml/manpages/vfs_fruit.8.xml index e407b54..9f77d9b 100644 --- a/docs-xml/manpages/vfs_fruit.8.xml +++ b/docs-xml/manpages/vfs_fruit.8.xml @@ -214,6 +214,18 @@ </listitem> </varlistentry> + <varlistentry> + <term>fruit:copyfile = yes | no</term> + <listitem> + <para>Whether to enable OS X specific copychunk ioctl + that requests a copy of a whole file along with all + attached metadata.</para> + <para>WARNING: the copyfile request is blocking the + client while the server does the copy.</para>. + <para>The default is <emphasis>no</emphasis>.</para> + </listitem> + </varlistentry> + </variablelist> </refsect1> diff --git a/selftest/target/Samba3.pm b/selftest/target/Samba3.pm index 09e6e2c..c95631f 100755 --- a/selftest/target/Samba3.pm +++ b/selftest/target/Samba3.pm @@ -1110,6 +1110,9 @@ sub provision($$$$$$) # sending messages works, and that the %m sub works. message command = mv %s $shrdir/message.%m + # fruit:copyfile is a global option + fruit:copyfile = yes + # Begin extra options $extra_options # End extra options @@ -1231,7 +1234,8 @@ sub provision($$$$$$) [vfs_fruit] path = $shrdir - vfs objects = catia fruit streams_xattr + vfs objects = catia fruit streams_xattr acl_xattr + ea support = yes fruit:ressource = file fruit:metadata = netatalk fruit:locking = netatalk diff --git a/selftest/target/Samba4.pm b/selftest/target/Samba4.pm index f7e2447..83b4e74 100755 --- a/selftest/target/Samba4.pm +++ b/selftest/target/Samba4.pm @@ -814,6 +814,9 @@ sub provision($$$$$$$$$) lanman auth = yes allow nt4 crypto = yes + # fruit:copyfile is a global option + fruit:copyfile = yes + $extra_smbconf_options [tmp] @@ -884,7 +887,8 @@ sub provision($$$$$$$$$) [vfs_fruit] path = $ctx->{share} - vfs objects = catia fruit streams_xattr + vfs objects = catia fruit streams_xattr acl_xattr + ea support = yes fruit:ressource = file fruit:metadata = netatalk fruit:locking = netatalk diff --git a/source3/include/transfer_file.h b/source3/include/transfer_file.h index 546104f..7e7ac03 100644 --- a/source3/include/transfer_file.h +++ b/source3/include/transfer_file.h @@ -27,6 +27,12 @@ ssize_t transfer_file_internal(void *in_file, ssize_t (*read_fn)(void *, void *, size_t), ssize_t (*write_fn)(void *, const void *, size_t)); +ssize_t transfer_file_internal_offset(void *in_file, + void *out_file, + size_t n, + ssize_t (*pread_fn)(void *, void *, size_t, off_t), + ssize_t (*pwrite_fn)(void *, const void *, size_t, off_t)); + off_t transfer_file(int infd, int outfd, off_t n); #endif /* __TRANSFER_FILE_H__ */ diff --git a/source3/include/vfs.h b/source3/include/vfs.h index 1843ef4..9d60d82 100644 --- a/source3/include/vfs.h +++ b/source3/include/vfs.h @@ -288,6 +288,7 @@ typedef struct files_struct { * possibly the simplest approach. Thanks, Jeremy for the idea. */ struct tevent_req *deferred_close; + bool aapl_copyfile_supported; } files_struct; struct vuid_cache_entry { diff --git a/source3/lib/util_transfer_file.c b/source3/lib/util_transfer_file.c index 00a2c9d..6750087 100644 --- a/source3/lib/util_transfer_file.c +++ b/source3/lib/util_transfer_file.c @@ -94,6 +94,71 @@ ssize_t transfer_file_internal(void *in_file, return (ssize_t)total; } +ssize_t transfer_file_internal_offset(void *in_file, + void *out_file, + size_t n, + ssize_t (*pread_fn)(void *, void *, size_t, off_t), + ssize_t (*pwrite_fn)(void *, const void *, size_t, off_t)) +{ + char *buf; + size_t total = 0; + ssize_t read_ret; + ssize_t write_ret; + size_t num_to_read_thistime; + size_t num_written = 0; + off_t offset = 0; + + if (n == 0) { + return 0; + } + + if ((buf = SMB_MALLOC_ARRAY(char, TRANSFER_BUF_SIZE)) == NULL) { + return -1; + } + + do { + num_to_read_thistime = MIN((n - total), TRANSFER_BUF_SIZE); + + read_ret = (*pread_fn)(in_file, buf, num_to_read_thistime, offset); + if (read_ret == -1) { + DEBUG(0,("transfer_file_internal: read failure. " + "Error = %s\n", strerror(errno) )); + SAFE_FREE(buf); + return -1; + } + if (read_ret == 0) { + break; + } + + num_written = 0; + + while (num_written < read_ret) { + write_ret = (*pwrite_fn)(out_file, buf + num_written, + read_ret - num_written, + offset + num_written); + + if (write_ret == -1) { + DEBUG(0,("transfer_file_internal: " + "write failure. Error = %s\n", + strerror(errno) )); + SAFE_FREE(buf); + return -1; + } + if (write_ret == 0) { + return (ssize_t)total; + } + + num_written += (size_t)write_ret; + } + + total += (size_t)read_ret; + offset += (off_t)read_ret; + } while (total < n); + + SAFE_FREE(buf); + return (ssize_t)total; +} + static ssize_t sys_read_fn(void *file, void *buf, size_t len) { int *fd = (int *)file; @@ -108,8 +173,22 @@ static ssize_t sys_write_fn(void *file, const void *buf, size_t len) return sys_write(*fd, buf, len); } +static ssize_t sys_pread_fn(void *file, void *buf, size_t len, off_t offset) +{ + int *fd = (int *)file; + + return sys_pread(*fd, buf, len, offset); +} + +static ssize_t sys_pwrite_fn(void *file, const void *buf, size_t len, off_t offset) +{ + int *fd = (int *)file; + + return sys_pwrite(*fd, buf, len, offset); +} + off_t transfer_file(int infd, int outfd, off_t n) { - return (off_t)transfer_file_internal(&infd, &outfd, (size_t)n, - sys_read_fn, sys_write_fn); + return (off_t)transfer_file_internal_offset(&infd, &outfd, (size_t)n, + sys_pread_fn, sys_pwrite_fn); } diff --git a/source3/modules/vfs_catia.c b/source3/modules/vfs_catia.c index 6743dfe..ff11a9a 100644 --- a/source3/modules/vfs_catia.c +++ b/source3/modules/vfs_catia.c @@ -705,11 +705,17 @@ catia_streaminfo(struct vfs_handle_struct *handle, struct files_struct *fsp, const char *path, TALLOC_CTX *mem_ctx, - unsigned int *num_streams, - struct stream_struct **streams) + unsigned int *_num_streams, + struct stream_struct **_streams) { char *mapped_name = NULL; NTSTATUS status; + int i; + unsigned int num_streams = 0; + struct stream_struct *streams = NULL; + + *_num_streams = 0; + *_streams = NULL; status = catia_string_replace_allocate(handle->conn, path, &mapped_name, vfs_translate_to_unix); @@ -719,10 +725,54 @@ catia_streaminfo(struct vfs_handle_struct *handle, } status = SMB_VFS_NEXT_STREAMINFO(handle, fsp, mapped_name, - mem_ctx, num_streams,streams); + mem_ctx, &num_streams, &streams); TALLOC_FREE(mapped_name); + if (!NT_STATUS_IS_OK(status)) { + return status; + } - return status; + /* + * Translate stream names just like the base names + */ + for (i = 0; i < num_streams; i++) { + /* + * Strip ":" prefix and ":$DATA" suffix to get a + * "pure" stream name and only translate that. + */ + void *old_ptr = streams[i].name; + char *stream_name = streams[i].name + 1; + char *stream_type = strrchr_m(stream_name, ':'); + + if (stream_type != NULL) { + *stream_type = '\0'; + stream_type += 1; + } + + status = catia_string_replace_allocate(handle->conn, stream_name, + &mapped_name, vfs_translate_to_windows); + if (!NT_STATUS_IS_OK(status)) { + TALLOC_FREE(streams); + return status; + } + + if (stream_type != NULL) { + streams[i].name = talloc_asprintf(streams, ":%s:%s", + mapped_name, stream_type); + } else { + streams[i].name = talloc_asprintf(streams, ":%s", + mapped_name); + } + TALLOC_FREE(mapped_name); + TALLOC_FREE(old_ptr); + if (streams[i].name == NULL) { + TALLOC_FREE(streams); + return NT_STATUS_NO_MEMORY; + } + } + + *_num_streams = num_streams; + *_streams = streams; + return NT_STATUS_OK; } static NTSTATUS diff --git a/source3/modules/vfs_fruit.c b/source3/modules/vfs_fruit.c index d02d0fe..fb9350b 100644 --- a/source3/modules/vfs_fruit.c +++ b/source3/modules/vfs_fruit.c @@ -29,6 +29,7 @@ #include "messages.h" #include "libcli/security/security.h" #include "../libcli/smb/smb2_create_ctx.h" +#include "lib/util/tevent_ntstatus.h" /* * Enhanced OS X and Netatalk compatibility @@ -123,8 +124,10 @@ struct fruit_config_data { enum fruit_locking locking; enum fruit_encoding encoding; bool use_aapl; + bool use_copyfile; bool readdir_attr_enabled; bool unix_info_enabled; + bool copyfile_enabled; bool veto_appledouble; /* @@ -1351,6 +1354,9 @@ static int init_fruit_config(vfs_handle_struct *handle) config->readdir_attr_rsize = true; } + config->use_copyfile = lp_parm_bool(-1, FRUIT_PARAM_TYPE_NAME, + "copyfile", false); + if (lp_parm_bool(SNUM(handle->conn), "readdir_attr", "aapl_finder_info", true)) { config->readdir_attr_finder_info = true; @@ -1835,6 +1841,11 @@ static NTSTATUS check_aapl(vfs_handle_struct *handle, config->readdir_attr_enabled = true; } + if (config->use_copyfile) { + server_caps |= SMB2_CRTCTX_AAPL_SUPPORTS_OSX_COPYFILE; + config->copyfile_enabled = true; + } + /* * The client doesn't set the flag, so we can't check * for it and just set it unconditionally @@ -3264,6 +3275,15 @@ static NTSTATUS fruit_create_file(vfs_handle_struct *handle, return status; } + if (config->copyfile_enabled) { + /* + * Set a flag in the fsp. Gets used in copychunk to + * check whether the special Apple copyfile semantics + * for copychunk should be allowed in a copychunk + * request with a count of 0. + */ + (*result)->aapl_copyfile_supported = true; + } if (is_ntfs_stream_smb_fname(smb_fname) || (*result == NULL) || ((*result)->is_directory)) { @@ -3473,6 +3493,202 @@ static NTSTATUS fruit_fset_nt_acl(vfs_handle_struct *handle, return NT_STATUS_OK; } +struct fruit_copy_chunk_state { + struct vfs_handle_struct *handle; + off_t copied; + struct files_struct *src_fsp; + struct files_struct *dst_fsp; + bool is_copyfile; +}; + +static void fruit_copy_chunk_done(struct tevent_req *subreq); +static struct tevent_req *fruit_copy_chunk_send(struct vfs_handle_struct *handle, + TALLOC_CTX *mem_ctx, + struct tevent_context *ev, + struct files_struct *src_fsp, + off_t src_off, + struct files_struct *dest_fsp, + off_t dest_off, + off_t num) +{ + struct tevent_req *req, *subreq; + struct fruit_copy_chunk_state *fruit_copy_chunk_state; + NTSTATUS status; + struct fruit_config_data *config; + off_t to_copy = num; + + DEBUG(10,("soff: %zd, doff: %zd, len: %zd\n", + src_off, dest_off, num)); + + SMB_VFS_HANDLE_GET_DATA(handle, config, + struct fruit_config_data, + return NULL); + + req = tevent_req_create(mem_ctx, &fruit_copy_chunk_state, + struct fruit_copy_chunk_state); + if (req == NULL) { + return NULL; + } + fruit_copy_chunk_state->handle = handle; + fruit_copy_chunk_state->src_fsp = src_fsp; + fruit_copy_chunk_state->dst_fsp = dest_fsp; + + /* + * Check if this a OS X copyfile style copychunk request with + * a requested chunk count of 0 that was translated to a + * copy_chunk_send VFS call overloading the parameters src_off + * = dest_off = num = 0. + */ + if ((src_off == 0) && (dest_off == 0) && (num == 0) && + src_fsp->aapl_copyfile_supported && + dest_fsp->aapl_copyfile_supported) + { + status = vfs_stat_fsp(src_fsp); + if (tevent_req_nterror(req, status)) { + return tevent_req_post(req, ev); + } + + to_copy = src_fsp->fsp_name->st.st_ex_size; + fruit_copy_chunk_state->is_copyfile = true; + } + + subreq = SMB_VFS_NEXT_COPY_CHUNK_SEND(handle, + mem_ctx, + ev, + src_fsp, + src_off, + dest_fsp, + dest_off, + to_copy); + if (tevent_req_nomem(subreq, req)) { + return tevent_req_post(req, ev); + } + + tevent_req_set_callback(subreq, fruit_copy_chunk_done, req); + return req; +} + +static void fruit_copy_chunk_done(struct tevent_req *subreq) +{ + struct tevent_req *req = tevent_req_callback_data( + subreq, struct tevent_req); + struct fruit_copy_chunk_state *state = tevent_req_data( + req, struct fruit_copy_chunk_state); + NTSTATUS status; + unsigned int num_streams = 0; + struct stream_struct *streams = NULL; + int i; + struct smb_filename *src_fname_tmp = NULL; + struct smb_filename *dst_fname_tmp = NULL; + + status = SMB_VFS_NEXT_COPY_CHUNK_RECV(state->handle, + subreq, + &state->copied); + TALLOC_FREE(subreq); + if (tevent_req_nterror(req, status)) { + return; + } + + if (!state->is_copyfile) { + tevent_req_done(req); + return; + } + + /* + * Now copy all reamining streams. We know the share supports + * streams, because we're in vfs_fruit. We don't do this async + * because streams are few and small. + */ + status = vfs_streaminfo(state->handle->conn, NULL, + state->src_fsp->fsp_name->base_name, + req, &num_streams, &streams); + if (tevent_req_nterror(req, status)) { + return; + } + + if (num_streams == 1) { + /* There is always one stream, ::$DATA. */ + tevent_req_done(req); + return; + } + + for (i = 0; i < num_streams; i++) { + DEBUG(10, ("%s: stream: '%s'/%zd\n", + __func__, streams[i].name, streams[i].size)); + + src_fname_tmp = synthetic_smb_fname( + req, + state->src_fsp->fsp_name->base_name, + streams[i].name, + NULL); + if (tevent_req_nomem(src_fname_tmp, req)) { + return; + } + + if (is_ntfs_default_stream_smb_fname(src_fname_tmp)) { + TALLOC_FREE(src_fname_tmp); + continue; + } + + dst_fname_tmp = synthetic_smb_fname( + req, + state->dst_fsp->fsp_name->base_name, + streams[i].name, + NULL); + if (tevent_req_nomem(dst_fname_tmp, req)) { + TALLOC_FREE(src_fname_tmp); + return; + } + + status = copy_file(req, + state->handle->conn, + src_fname_tmp, + dst_fname_tmp, + OPENX_FILE_CREATE_IF_NOT_EXIST, + 0, false); -- Samba Shared Repository