Re: MS-WDV (was Re: Help with buckets)

2022-12-06 Thread Emmanuel Dreyfus 
On Fri, Dec 02, 2022 at 03:17:05PM +, Joe Orton wrote:
> I think this might need to do something more complex, maybe running the 
> PROPFIND in a subrequest properly and capturing (buffering) the output 
> in a custom filter, rather than using the mod_dav internal API directly. 
> Have you tried using ap_sub_req_method_uri()? Not sure this has been 
> tried before with mod_dav so might well be something I'm missing.

I did it the way you suggested. The patch gets huge, I just send the
reelvant bits. Does it looks good to you?

In register_hooks() I have
ap_register_output_filter("DAV_MSEXT_OUT", dav_msext_output, NULL,
  AP_FTYPE_RESOURCE);
Then:

static apr_status_t dav_msext_output(ap_filter_t *f,
 apr_bucket_brigade *bb)
{
apr_bucket_brigade *bbsub = f->ctx;
apr_bucket *b;

b = APR_BRIGADE_FIRST(bb);
while (b != APR_BRIGADE_SENTINEL(bb)) {
apr_bucket *nb;
if (APR_BUCKET_IS_EOS(b))
break;

nb = APR_BUCKET_NEXT(b);
APR_BUCKET_REMOVE(b);
APR_BRIGADE_INSERT_TAIL(bbsub, b);
b = nb;
}

return ap_pass_brigade(f->next, bb);
}

static void dav_msdavext_combined_propfind(request_rec *r)
{
apr_bucket_brigade *bbsub;
apr_bucket_brigade *bb;
ap_filter_t *f;
apr_bucket *b;
request_rec *rr = NULL;
apr_off_t length;

bbsub = apr_brigade_create(r->pool, r->output_filters->c->bucket_alloc);

rr = ap_sub_req_method_uri("PROPFIND", r->uri, r, r->output_filters);
if (!rr || rr->status != HTTP_OK)
goto out;

f = ap_add_output_filter("DAV_MSEXT_OUT", bbsub, rr, rr->connection);
if (!f)
goto out;

if (ap_run_sub_req(rr) != OK)
goto out;

ap_remove_output_filter(f);

if (apr_brigade_length(bbsub, 1, ) != APR_SUCCESS)
goto out;

bb = apr_brigade_create(r->pool,r->output_filters->c->bucket_alloc);

apr_brigade_printf(bb, NULL, NULL, 
   "%016" APR_UINT64_T_HEX_FMT, length);

APR_BRIGADE_CONCAT(bb, bbsub);

ap_destroy_sub_req(rr);
rr = NULL;

rr = ap_sub_req_lookup_uri(r->uri, r, r->output_filters);
if (!rr || rr->status != HTTP_OK || rr->filename == NULL ||
rr->finfo.filetype != APR_REG)
goto out;

apr_brigade_printf(bb, NULL, NULL, 
   "%016" APR_UINT64_T_HEX_FMT, rr->finfo.size);

ap_set_content_type(r, "multipart/MSDAVEXTPrefixEncoded");

ap_pass_brigade(r->output_filters, bb);

/* plain GET rocessing happens afterward */
out:
if (rr)
ap_destroy_sub_req(rr);

return;
}


-- 
Emmanuel Dreyfus
m...@netbsd.org

Re: MS-WDV (was Re: Help with buckets)

2022-12-04 Thread Ruediger Pluem 



On 12/2/22 4:42 PM, Emmanuel Dreyfus wrote:
> On Fri, Dec 02, 2022 at 03:17:05PM +, Joe Orton wrote:
>> I think this might need to do something more complex, maybe running the 
>> PROPFIND in a subrequest properly and capturing (buffering) the output 
>> in a custom filter, rather than using the mod_dav internal API directly. 
>> Have you tried using ap_sub_req_method_uri()? Not sure this has been 
>> tried before with mod_dav so might well be something I'm missing.
> 
> I can try that, but whatever the method is, we need to produce the
> propfind data before sending its size.
> 
> I see two unsatisfying alternatives:
> 
> 1) produce propfind data in a buffer, output the size, then the buffer
> 
> 2) produce propfind data, discarding it as it comes but coutning its size, 
> then output the size, and produce the propfind data a second time.
> 
> First approach wastes memory, second approach wastes CPU. And second approach
> needs a mechanism to ensure propfind data does not change between the two
> times it is produced. I am not sure that can be guaranteed.

I think we cannot guarantee this, which leaves us with 1.

Regards

Rüdiger

Re: MS-WDV (was Re: Help with buckets)

2022-12-02 Thread Emmanuel Dreyfus 
On Fri, Dec 02, 2022 at 03:17:05PM +, Joe Orton wrote:
> I think this might need to do something more complex, maybe running the 
> PROPFIND in a subrequest properly and capturing (buffering) the output 
> in a custom filter, rather than using the mod_dav internal API directly. 
> Have you tried using ap_sub_req_method_uri()? Not sure this has been 
> tried before with mod_dav so might well be something I'm missing.

I can try that, but whatever the method is, we need to produce the
propfind data before sending its size.

I see two unsatisfying alternatives:

1) produce propfind data in a buffer, output the size, then the buffer

2) produce propfind data, discarding it as it comes but coutning its size, 
then output the size, and produce the propfind data a second time.

First approach wastes memory, second approach wastes CPU. And second approach
needs a mechanism to ensure propfind data does not change between the two
times it is produced. I am not sure that can be guaranteed.

-- 
Emmanuel Dreyfus
m...@netbsd.org

Re: MS-WDV (was Re: Help with buckets)

2022-12-02 Thread Joe Orton 
On Fri, Dec 02, 2022 at 08:53:07AM +, Emmanuel Dreyfus wrote:
> Hello
> 
> I made some progress with the combined GET+PROPFIND specified
> by MS-WDV (for a summary, see
> https://lists.apache.org/thread/57s1vvl6k9qpdv5ym7mtcl29bd933w7k )
> 
> Attached is the diff against trunk, form comments.

Hi Emmanuel,

That's a very weird protocol design, wow. Have you tested this with a 
long PROPFIND response? It needs to buffer the entire response in the 
output brigade to work out the length in the "multipart" prefix, but 
mod_dav will flush output brigades down the filter chain regularly 
during the multistatus response processing.

I think this might need to do something more complex, maybe running the 
PROPFIND in a subrequest properly and capturing (buffering) the output 
in a custom filter, rather than using the mod_dav internal API directly. 
Have you tried using ap_sub_req_method_uri()? Not sure this has been 
tried before with mod_dav so might well be something I'm missing.

Regards, Joe

MS-WDV (was Re: Help with buckets)

2022-12-02 Thread Emmanuel Dreyfus 
Hello

I made some progress with the combined GET+PROPFIND specified
by MS-WDV (for a summary, see
https://lists.apache.org/thread/57s1vvl6k9qpdv5ym7mtcl29bd933w7k )

Attached is the diff against trunk, form comments.



-- 
Emmanuel Dreyfus
m...@netbsd.org
Index: dav/main/mod_dav.c
===
--- dav/main/mod_dav.c	(revision 1905652)
+++ dav/main/mod_dav.c	(working copy)
@@ -84,7 +84,7 @@
 int locktimeout;
 int allow_depthinfinity;
 int allow_lockdiscovery;
-
+int enable_msext;
 } dav_dir_conf;
 
 /* per-server configuration */
@@ -206,6 +206,8 @@
  allow_depthinfinity);
 newconf->allow_lockdiscovery = DAV_INHERIT_VALUE(parent, child,
  allow_lockdiscovery);
+newconf->enable_msext = DAV_INHERIT_VALUE(parent, child,
+  enable_msext);
 
 return newconf;
 }
@@ -319,6 +321,20 @@
 }
 
 /*
+ * Command handler for the DAVmsExt directive, which is FLAG.
+ */
+static const char *dav_cmd_davmsext(cmd_parms *cmd, void *config, int arg)
+{
+dav_dir_conf *conf = (dav_dir_conf *)config;
+
+if (arg)
+conf->enable_msext = DAV_ENABLED_ON;
+else
+conf->enable_msext = DAV_ENABLED_OFF;
+return NULL;
+}
+
+/*
  * Command handler for DAVMinTimeout directive, which is TAKE1
  */
 static const char *dav_cmd_davmintimeout(cmd_parms *cmd, void *config,
@@ -558,10 +574,17 @@
 DAV_DECLARE(apr_status_t) dav_finish_multistatus(request_rec *r,
  apr_bucket_brigade *bb)
 {
+ap_fputs(r->output_filters, bb, "" DEBUG_CR);
+
+return OK;
+}
+
+
+/* Send the response to the first filter */
+static apr_status_t dav_pass_brigade(request_rec *r, apr_bucket_brigade *bb)
+{
 apr_bucket *b;
 
-ap_fputs(r->output_filters, bb, "" DEBUG_CR);
-
 /* indicate the end of the response body */
 b = apr_bucket_eos_create(r->connection->bucket_alloc);
 APR_BRIGADE_INSERT_TAIL(bb, b);
@@ -590,6 +613,7 @@
 apr_pool_destroy(subpool);
 
 dav_finish_multistatus(r, bb);
+dav_pass_brigade(r, bb);
 }
 
 /*
@@ -1691,6 +1715,7 @@
 /* handle the OPTIONS method */
 static int dav_method_options(request_rec *r)
 {
+dav_dir_conf *conf;
 const dav_hooks_locks *locks_hooks = DAV_GET_HOOKS_LOCKS(r);
 const dav_hooks_vsn *vsn_hooks = DAV_GET_HOOKS_VSN(r);
 const dav_hooks_binding *binding_hooks = DAV_GET_HOOKS_BINDING(r);
@@ -1801,6 +1826,11 @@
 /* this tells MSFT products to skip looking for FrontPage extensions */
 apr_table_setn(r->headers_out, "MS-Author-Via", "DAV");
 
+/* MS-WDV extensions */
+conf = ap_get_module_config(r->per_dir_config, _module);
+if (conf && conf->enable_msext == DAV_ENABLED_ON) 
+	apr_table_setn(r->headers_out, "X-MSDAVEXT", "1");
+
 /*
  * Determine which methods are allowed on the resource.
  * Three cases:  resource is null (3), is lock-null (7.4), or exists.
@@ -2146,7 +2176,7 @@
 }
 
 /* handle the PROPFIND method */
-static int dav_method_propfind(request_rec *r)
+static int dav_method_propfind(request_rec *r, apr_bucket_brigade *bb)
 {
 dav_resource *resource;
 int depth;
@@ -2237,7 +2267,10 @@
 
 ctx.doc = doc;
 ctx.r = r;
-ctx.bb = apr_brigade_create(r->pool, r->connection->bucket_alloc);
+if (bb)
+ctx.bb = bb;
+else
+ctx.bb = apr_brigade_create(r->pool, r->connection->bucket_alloc);
 apr_pool_create(, r->pool);
 apr_pool_tag(ctx.scratchpool, "mod_dav-scratch");
 
@@ -2288,6 +2321,8 @@
 }
 
 dav_finish_multistatus(r, ctx.bb);
+if (!bb)
+dav_pass_brigade(r, ctx.bb);
 
 /* the response has been sent. */
 return DONE;
@@ -4965,12 +5000,70 @@
 return dav_created(r, lookup.rnew->unparsed_uri, "Binding", 0);
 }
 
+static void dav_msdavext_combined(request_rec *r)
+{
+dav_dir_conf *conf;
+const char *msdavext_hdr;
+apr_bucket_brigade *bb;
+apr_bucket *b;
+request_rec *rr = NULL;
+apr_off_t length;
+char buf[16+1]; /* +1 for trailing \0 */
 
+if (r->main)
+goto out;
+
+if (r->method_number != M_GET && r->method_number != M_POST)
+goto out;
+
+conf = ap_get_module_config(r->per_dir_config, _module);
+if (conf->enable_msext != DAV_ENABLED_ON) 
+goto out;
+
+msdavext_hdr = apr_table_get(r->headers_in, "X-MSDAVEXT");
+if (msdavext_hdr == NULL || strcmp(msdavext_hdr, "PROPFIND") !=0)
+goto out;
+
+bb = apr_brigade_create(r->pool,r->output_filters->c->bucket_alloc);
+if (dav_method_propfind(r, bb) != DONE)
+goto out;
+
+if (apr_brigade_length(bb, 1, ) != APR_SUCCESS)
+ 	goto out;
+
+b = apr_bucket_transient_create(buf, 16, 
+r->output_filters->c->bucket_alloc);
+(void)apr_snprintf(buf, sizeof(buf),
+   "%016"