On Thu, Jun 9, 2011 at 12:13 AM, <minf...@apache.org> wrote: > Author: minfrin > Date: Wed Jun 8 22:13:21 2011 > New Revision: 1133582 > > URL: http://svn.apache.org/viewvc?rev=1133582&view=rev > Log: > mod_data: Introduce a filter to support RFC2397 data URLs.
Why is this in the core? The example in the documentation doesn't make sense, this data encoded this way is inline, not the whole response. I don't get why this is a filter in httpd core, its a feature of an app server when its generate its HTML? > Added: > httpd/httpd/trunk/docs/manual/mod/mod_data.xml > httpd/httpd/trunk/docs/manual/mod/mod_data.xml.meta > httpd/httpd/trunk/modules/filters/mod_data.c > Modified: > httpd/httpd/trunk/CHANGES > httpd/httpd/trunk/docs/manual/filter.xml > httpd/httpd/trunk/docs/manual/new_features_2_4.xml > httpd/httpd/trunk/modules/filters/config.m4 > > Modified: httpd/httpd/trunk/CHANGES > URL: > http://svn.apache.org/viewvc/httpd/httpd/trunk/CHANGES?rev=1133582&r1=1133581&r2=1133582&view=diff > ============================================================================== > --- httpd/httpd/trunk/CHANGES [utf-8] (original) > +++ httpd/httpd/trunk/CHANGES [utf-8] Wed Jun 8 22:13:21 2011 > @@ -2,6 +2,9 @@ > > Changes with Apache 2.3.13 > > + *) mod_data: Introduce a filter to support RFC2397 data URLs. [Graham > + Leggett] > + > *) mod_userdir/mod_alias/mod_vhost_alias: Correctly set DOCUMENT_ROOT, > CONTEXT_DOCUMENT_ROOT, CONTEXT_PREFIX. PR 26052. PR 46198. > [Stefan Fritsch] > > Modified: httpd/httpd/trunk/docs/manual/filter.xml > URL: > http://svn.apache.org/viewvc/httpd/httpd/trunk/docs/manual/filter.xml?rev=1133582&r1=1133581&r2=1133582&view=diff > ============================================================================== > --- httpd/httpd/trunk/docs/manual/filter.xml (original) > +++ httpd/httpd/trunk/docs/manual/filter.xml Wed Jun 8 22:13:21 2011 > @@ -38,6 +38,13 @@ > <module>mod_include</module> > <module>mod_charset_lite</module> > <module>mod_reflector</module> > + <module>mod_buffer</module> > + <module>mod_data</module> > + <module>mod_ratelimit</module> > + <module>mod_reqtimeout</module> > + <module>mod_request</module> > + <module>mod_sed</module> > + <module>mod_substitute</module> > </modulelist> > <directivelist> > <directive module="mod_filter">FilterChain</directive> > > Added: httpd/httpd/trunk/docs/manual/mod/mod_data.xml > URL: > http://svn.apache.org/viewvc/httpd/httpd/trunk/docs/manual/mod/mod_data.xml?rev=1133582&view=auto > ============================================================================== > --- httpd/httpd/trunk/docs/manual/mod/mod_data.xml (added) > +++ httpd/httpd/trunk/docs/manual/mod/mod_data.xml Wed Jun 8 22:13:21 2011 > @@ -0,0 +1,69 @@ > +<?xml version="1.0"?> > +<!DOCTYPE modulesynopsis SYSTEM "../style/modulesynopsis.dtd"> > +<?xml-stylesheet type="text/xsl" href="../style/manual.en.xsl"?> > +<!-- $LastChangedRevision: 966890 $ --> > + > +<!-- > + Licensed to the Apache Software Foundation (ASF) under one or more > + contributor license agreements. See the NOTICE file distributed with > + this work for additional information regarding copyright ownership. > + The ASF licenses this file to You under the Apache License, Version 2.0 > + (the "License"); you may not use this file except in compliance with > + the License. You may obtain a copy of the License at > + > + http://www.apache.org/licenses/LICENSE-2.0 > + > + Unless required by applicable law or agreed to in writing, software > + distributed under the License is distributed on an "AS IS" BASIS, > + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. > + See the License for the specific language governing permissions and > + limitations under the License. > +--> > + > +<modulesynopsis metafile="mod_data.xml.meta"> > + > +<name>mod_data</name> > +<description>Convert response body into an RFC2397 data URL</description> > +<status>Extension</status> > +<sourcefile>mod_data.c</sourcefile> > +<identifier>data_module</identifier> > +<compatibility>Available in Apache 2.3 and later</compatibility> > + > +<summary> > + <p>This module provides the ability to convert a response into > + an <a href="http://tools.ietf.org/html/rfc2397";>RFC2397 data URL</a>. > + </p> > + > + <p>Data URLs can be embedded inline within web pages using something > + like the <module>mod_include</module> module, to remove the need for > + clients to make separate connections to fetch what may potentially be > + many small images. Data URLs may also be included into pages generated > + by scripting languages such as PHP.</p> > + > + <example><title>An example of a data URL</title> > + data:image/gif;base64,R0lGODdhMAAwAPAAAAAAAP///ywAAAAAMAAw<br /> > + AAAC8IyPqcvt3wCcDkiLc7C0qwyGHhSWpjQu5yqmCYsapyuvUUlvONmOZtfzgFz<br /> > + ByTB10QgxOR0TqBQejhRNzOfkVJ+5YiUqrXF5Y5lKh/DeuNcP5yLWGsEbtLiOSp<br /> > + a/TPg7JpJHxyendzWTBfX0cxOnKPjgBzi4diinWGdkF8kjdfnycQZXZeYGejmJl<br /> > + ZeGl9i2icVqaNVailT6F5iJ90m6mvuTS4OK05M0vDk0Q4XUtwvKOzrcd3iq9uis<br /> > + F81M1OIcR7lEewwcLp7tuNNkM3uNna3F2JQFo97Vriy/Xl4/f1cf5VWzXyym7PH<br /> > + hhx4dbgYKAAA7<br /> > + </example> > + > + <p>The filter takes no parameters, and can be added to the filter stack > + using the <directive module="core">SetOutputFilter</directive> directive, > + or any of the directives supported by the <module>mod_filter</module> > + module.</p> > + > + <example><title>Configuring the filter</title> > + <Location /data/images><br /> > + <indent> > + SetOutputFilter DATA > + </indent> > + </Location><br /> > + </example> > + > +</summary> > +<seealso><a href="../filter.html">Filters</a></seealso> > + > +</modulesynopsis> > > Added: httpd/httpd/trunk/docs/manual/mod/mod_data.xml.meta > URL: > http://svn.apache.org/viewvc/httpd/httpd/trunk/docs/manual/mod/mod_data.xml.meta?rev=1133582&view=auto > ============================================================================== > --- httpd/httpd/trunk/docs/manual/mod/mod_data.xml.meta (added) > +++ httpd/httpd/trunk/docs/manual/mod/mod_data.xml.meta Wed Jun 8 22:13:21 > 2011 > @@ -0,0 +1,12 @@ > +<?xml version="1.0" encoding="UTF-8" ?> > +<!-- GENERATED FROM XML: DO NOT EDIT --> > + > +<metafile reference="mod_data.xml"> > + <basename>mod_data</basename> > + <path>/mod/</path> > + <relpath>..</relpath> > + > + <variants> > + <variant>en</variant> > + </variants> > +</metafile> > > Modified: httpd/httpd/trunk/docs/manual/new_features_2_4.xml > URL: > http://svn.apache.org/viewvc/httpd/httpd/trunk/docs/manual/new_features_2_4.xml?rev=1133582&r1=1133581&r2=1133582&view=diff > ============================================================================== > --- httpd/httpd/trunk/docs/manual/new_features_2_4.xml (original) > +++ httpd/httpd/trunk/docs/manual/new_features_2_4.xml Wed Jun 8 22:13:21 > 2011 > @@ -74,6 +74,9 @@ > <dt><module>mod_buffer</module></dt> > <dd>Provides for buffering the input and output filter stacks</dd> > > + <dt><module>mod_data</module></dt> > + <dd>Convert response body into an RFC2397 data URL</dd> > + > <dt><module>mod_lua</module></dt> > <dd>Embeds the <a href="http://www.lua.org/";>Lua</a> language into > httpd, > for configuration and small business logic functions.</dd> > > Modified: httpd/httpd/trunk/modules/filters/config.m4 > URL: > http://svn.apache.org/viewvc/httpd/httpd/trunk/modules/filters/config.m4?rev=1133582&r1=1133581&r2=1133582&view=diff > ============================================================================== > --- httpd/httpd/trunk/modules/filters/config.m4 (original) > +++ httpd/httpd/trunk/modules/filters/config.m4 Wed Jun 8 22:13:21 2011 > @@ -5,6 +5,7 @@ dnl APACHE_MODULE(name, helptext[, objec > APACHE_MODPATH_INIT(filters) > > APACHE_MODULE(buffer, Filter Buffering, , , yes) > +APACHE_MODULE(data, RFC2397 data encoder, , , yes) > APACHE_MODULE(ratelimit, Output Bandwidth Limiting, , , yes) > APACHE_MODULE(reqtimeout, Limit time waiting for request from client, , , > yes) > APACHE_MODULE(ext_filter, external filter module, , , most) > > Added: httpd/httpd/trunk/modules/filters/mod_data.c > URL: > http://svn.apache.org/viewvc/httpd/httpd/trunk/modules/filters/mod_data.c?rev=1133582&view=auto > ============================================================================== > --- httpd/httpd/trunk/modules/filters/mod_data.c (added) > +++ httpd/httpd/trunk/modules/filters/mod_data.c Wed Jun 8 22:13:21 2011 > @@ -0,0 +1,237 @@ > +/* Licensed to the Apache Software Foundation (ASF) under one or more > + * contributor license agreements. See the NOTICE file distributed with > + * this work for additional information regarding copyright ownership. > + * The ASF licenses this file to You under the Apache License, Version 2.0 > + * (the "License"); you may not use this file except in compliance with > + * the License. You may obtain a copy of the License at > + * > + * http://www.apache.org/licenses/LICENSE-2.0 > + * > + * Unless required by applicable law or agreed to in writing, software > + * distributed under the License is distributed on an "AS IS" BASIS, > + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. > + * See the License for the specific language governing permissions and > + * limitations under the License. > + */ > + > +/* > + * mod_data.c --- Turn the response into an rfc2397 data URL, suitable for > + * displaying as inline content on a page. > + */ > + > +#include "apr.h" > +#include "apr_strings.h" > +#include "apr_buckets.h" > +#include "apr_base64.h" > +#include "apr_lib.h" > + > +#include "ap_config.h" > +#include "util_filter.h" > +#include "httpd.h" > +#include "http_config.h" > +#include "http_log.h" > +#include "http_request.h" > +#include "http_protocol.h" > + > +#define DATA_FILTER "DATA" > + > +module AP_MODULE_DECLARE_DATA data_module; > + > +typedef struct data_ctx > +{ > + unsigned char overflow[3]; > + int count; > + apr_bucket_brigade *bb; > +} data_ctx; > + > +/** > + * Create a data URL as follows: > + * > + * data:[<mime-type>;][charset=<charset>;]base64,<payload> > + * > + * Where: > + * > + * mime-type: The mime type of the original response. > + * charset: The optional character set corresponding to the mime type. > + * payload: A base64 version of the response body. > + * > + * The content type of the response is set to text/plain. > + * > + * The Content-Length header, if present, is updated with the new content > + * length based on the increase in size expected from the base64 conversion. > + * If the Content-Length header is too large to fit into an int, we remove > + * the Content-Length header instead. > + */ > +static apr_status_t data_out_filter(ap_filter_t *f, apr_bucket_brigade *bb) > +{ > + apr_bucket *e; > + request_rec *r = f->r; > + data_ctx *ctx = f->ctx; > + apr_status_t rv = APR_SUCCESS; > + > + /* first time in? create a context */ > + if (!ctx) { > + char *type; > + char *charset; > + char *end; > + const char *content_length; > + > + /* base64-ing won't work on subrequests, it would be nice if > + * it did. Within subrequests, we have no EOS to check for, > + * so we don't know when to flush the tail to the network. > + */ > + if (!ap_is_initial_req(f->r)) { > + ap_remove_output_filter(f); > + return ap_pass_brigade(f->next, bb); > + } > + > + ctx = f->ctx = apr_pcalloc(r->pool, sizeof(*ctx)); > + ctx->bb = apr_brigade_create(r->pool, f->c->bucket_alloc); > + > + type = apr_pstrdup(r->pool, r->content_type); > + if (type) { > + charset = strchr(type, ' '); > + if (charset) { > + *charset++ = 0; > + end = strchr(charset, ' '); > + if (end) { > + *end++ = 0; > + } > + } > + } > + > + apr_brigade_printf(ctx->bb, NULL, NULL, "data:%s%s;base64,", > + type ? type : "", charset ? charset : ""); > + > + content_length = apr_table_get(r->headers_out, "Content-Length"); > + if (content_length) { > + apr_off_t len, clen; > + apr_brigade_length(ctx->bb, 1, &len); > + clen = apr_atoi64(content_length); > + if (clen >= 0 && clen < APR_INT32_MAX) { > + ap_set_content_length(r, len + apr_base64_encode_len(clen) - > 1); > + } > + else { > + apr_table_unset(r->headers_out, "Content-Length"); > + } > + } > + > + ap_set_content_type(r, "text/plain"); > + > + } > + > + /* Do nothing if asked to filter nothing. */ > + if (APR_BRIGADE_EMPTY(bb)) { > + return ap_pass_brigade(f->next, bb); > + } > + > + while (APR_SUCCESS == rv && !APR_BRIGADE_EMPTY(bb)) { > + const char *data; > + apr_size_t size; > + apr_size_t tail; > + apr_size_t len; > + /* buffer big enough for 8000 encoded bytes (6000 raw bytes) and > terminator */ > + char buffer[APR_BUCKET_BUFF_SIZE + 1]; > + char encoded[((sizeof(ctx->overflow)) / 3) * 4 + 1]; > + > + /* make sure we don't read more than 6000 bytes at a time */ > + apr_brigade_partition(bb, (APR_BUCKET_BUFF_SIZE / 4 * 3), &e); > + > + e = APR_BRIGADE_FIRST(bb); > + > + /* EOS means we are done. */ > + if (APR_BUCKET_IS_EOS(e)) { > + > + /* write away the tail */ > + if (ctx->count) { > + len = apr_base64_encode_binary(encoded, ctx->overflow, > + ctx->count); > + apr_brigade_write(ctx->bb, NULL, NULL, encoded, len - 1); > + ctx->count = 0; > + } > + > + /* pass the EOS across */ > + APR_BUCKET_REMOVE(e); > + APR_BRIGADE_INSERT_TAIL(ctx->bb, e); > + > + /* pass what we have down the chain */ > + ap_remove_output_filter(f); > + rv = ap_pass_brigade(f->next, ctx->bb); > + continue; > + } > + > + /* metadata buckets are preserved as is */ > + if (APR_BUCKET_IS_METADATA(e)) { > + /* > + * Remove meta data bucket from old brigade and insert into the > + * new. > + */ > + APR_BUCKET_REMOVE(e); > + APR_BRIGADE_INSERT_TAIL(ctx->bb, e); > + continue; > + } > + > + /* size will never be more than 6000 bytes */ > + if (APR_SUCCESS == (rv = apr_bucket_read(e, &data, &size, > + APR_BLOCK_READ))) { > + > + /* fill up and write out our overflow buffer if partially used */ > + while (size && ctx->count && ctx->count < sizeof(ctx->overflow)) > { > + ctx->overflow[ctx->count++] = *data++; > + size--; > + } > + if (ctx->count == sizeof(ctx->overflow)) { > + len = apr_base64_encode_binary(encoded, ctx->overflow, > + sizeof(ctx->overflow)); > + apr_brigade_write(ctx->bb, NULL, NULL, encoded, len - 1); > + ctx->count = 0; > + } > + > + /* write the main base64 chunk */ > + tail = size % sizeof(ctx->overflow); > + size -= tail; > + if (size) { > + len = apr_base64_encode_binary(buffer, > + (const unsigned char *) data, size); > + apr_brigade_write(ctx->bb, NULL, NULL, buffer, len - 1); > + } > + > + /* save away any tail in the overflow buffer */ > + if (tail) { > + memcpy(ctx->overflow, data + size, tail); > + ctx->count += tail; > + } > + > + apr_bucket_delete(e); > + > + /* pass what we have down the chain */ > + rv = ap_pass_brigade(f->next, ctx->bb); > + if (rv) { > + /* should break out of the loop, since our write to the > client > + * failed in some way. */ > + continue; > + } > + > + } > + > + } > + > + return rv; > + > +} > + > +static const command_rec data_cmds[] = { { NULL } }; > + > +static void register_hooks(apr_pool_t *p) > +{ > + ap_register_output_filter(DATA_FILTER, data_out_filter, NULL, > + AP_FTYPE_RESOURCE); > +} > +AP_DECLARE_MODULE(data) = { STANDARD20_MODULE_STUFF, > + NULL, /* create per-directory config structure */ > + NULL, /* merge per-directory config structures */ > + NULL, /* create per-server config structure */ > + NULL, /* merge per-server config structures */ > + data_cmds, /* command apr_table_t */ > + register_hooks /* register hooks */ > +}; > > >
