Re: generic question on input filters

André Warnier Wed, 08 Apr 2009 16:21:16 -0700

Torsten Foertsch wrote:
[...]
Thanks for the previous answer and tips.

I have written and tried the filter, and now I have a problem (at theend). Anyone knows what that means ?

Suspicion : I cannot read the content, and print nothing when I amcalled. Instead of accumulating the content and process it all at onceat the end, should I process it chunk by chunk ?Is it OK then when the Content-Length header is wrong (since I amchanging the size of the data) ?




main server error log:

[Wed Apr 08 22:42:37 2009] [notice] Apache/2.2.3 (Linux/SUSE)mod_jk/1.2.28-dev-727724 mod_perl/2.0.2 Perl/v5.8.8 configured --resuming normal operations

VirtualHost error log (logging by the filter, a few runs that seem towork fine, then suddenly the error. Sorry for the clutter, that's thehtml form sending stuff):

[Wed Apr 08 22:43:22 2009] [warn] [client 10.13.16.93] FSWB: 1st call,referer: http://host.customer.com/starweb/wideoteka/servlet.starweb[Wed Apr 08 22:43:22 2009] [warn] [client 10.13.16.93] FSWB: buffer:[__sessionNumber=67&__pageid=daneIndeksu&__a1=&__a2=&__a3=&__hiddenstyle=A&__dirtyFlag=undefined&__action=216&Near=&SelectedIndexTerms=__05.12.1971%3B%3B%3A%3A&SelectedIndexTerms=__05.1970%3B%3B%3A%3A&SelectedIndexTerms=__06+SIERPNIA+2008+R.%3B%3B%3A%3A&SelectedIndexTerms=__06+SIERPNIA+2008+R.+-+SKR%D3T%3B%3B%3A%3A&SelectedIndexTerms=__06.02.1972%3B%3B%3A%3A&SelectedIndexTerms=__06.04.1969%3B%3B%3A%3A&SelectedIndexTerms=__06.05.1968%3B%3B%3A%3A&SelectedIndexTerms=__06.06.1971%3B%3B%3A%3A&SelectedIndexTerms=__06.10.1965%3B%3B%3A%3A&SelectedIndexTerms=__06.11.1970%3B%3B%3A%3A&SelectedIndexTerms=__06.12.1968+ROK%3B%3B%3A%3A&SelectedIndexTerms=__07+SIERPIE%D1+2008%3B%3B%3A%3A&SelectedIndexTerms=__07+SIERPNIA+2008+R.%3B%3B%3A%3A&SelectedIndexTerms=__07+ZG%A3O%3F+SI%CA%3B%3B%3A%3A&SelectedIndexTerms=__07+ZG%A3O%3F+SI%CA%3B%3B%3A%3A&__ReportId=histIndeksu%5E0],referer: http://host.customer.com/starweb/wideoteka/servlet.starweb[Wed Apr 08 22:43:22 2009] [warn] [client 10.13.16.93] FSWB: splittedparams: __sessionNumber=67 || __pageid=daneIndeksu || __a1= || __a2= ||__a3= || __hiddenstyle=A || __dirtyFlag=undefined || __action=216 ||Near= || SelectedIndexTerms=__05.12.1971%3B%3B%3A%3A ||SelectedIndexTerms=__05.1970%3B%3B%3A%3A ||SelectedIndexTerms=__06+SIERPNIA+2008+R.%3B%3B%3A%3A ||SelectedIndexTerms=__06+SIERPNIA+2008+R.+-+SKR%D3T%3B%3B%3A%3A ||SelectedIndexTerms=__06.02.1972%3B%3B%3A%3A ||SelectedIndexTerms=__06.04.1969%3B%3B%3A%3A ||SelectedIndexTerms=__06.05.1968%3B%3B%3A%3A ||SelectedIndexTerms=__06.06.1971%3B%3B%3A%3A ||SelectedIndexTerms=__06.10.1965%3B%3B%3A%3A ||SelectedIndexTerms=__06.11.1970%3B%3B%3A%3A ||SelectedIndexTerms=__06.12.1968+ROK%3B%3B%3A%3A ||SelectedIndexTerms=__07+SIERPIE%D1+2008%3B%3B%3A%3A ||SelectedIndexTerms=__07+SIERPNIA+2008+R.%3B%3B%3A%3A ||SelectedIndexTerms=__07+ZG%A3O%3F+SI%CA%3B%3B%3A%3A ||SelectedIndexTerms=__07+ZG%A3O%3F+SI%CA%3B%3B%3A%3A ||__ReportId=histIndeksu%5E0, referer:http://host.customer.com/starweb/wideoteka/servlet.starweb[Wed Apr 08 22:43:22 2009] [warn] [client 10.13.16.93] FSWB: newcontent:[__sessionNumber=67&__pageid=daneIndeksu&__a1=&__a2=&__a3=&__hiddenstyle=A&__dirtyFlag=undefined&__action=216&Near=&SelectedIndexTerms%3D__05.12.1971%3B%3B%3A%3A&SelectedIndexTerms%3D__05.1970%3B%3B%3A%3A&SelectedIndexTerms%3D__06%2BSIERPNIA%2B2008%2BR.%3B%3B%3A%3A&SelectedIndexTerms%3D__06%2BSIERPNIA%2B2008%2BR.%2B-%2BSKROT%3B%3B%3A%3A&SelectedIndexTerms%3D__06.02.1972%3B%3B%3A%3A&SelectedIndexTerms%3D__06.04.1969%3B%3B%3A%3A&SelectedIndexTerms%3D__06.05.1968%3B%3B%3A%3A&SelectedIndexTerms%3D__06.06.1971%3B%3B%3A%3A&SelectedIndexTerms%3D__06.10.1965%3B%3B%3A%3A&SelectedIndexTerms%3D__06.11.1970%3B%3B%3A%3A&SelectedIndexTerms%3D__06.12.1968%2BROK%3B%3B%3A%3A&SelectedIndexTerms%3D__07%2BSIERPIEN%2B2008%3B%3B%3A%3A&SelectedIndexTerms%3D__07%2BSIERPNIA%2B2008%2BR.%3B%3B%3A%3A&SelectedIndexTerms%3D__07%2BZGLO%3F%2BSIE%3B%3B%3A%3A&SelectedIndexTerms%3D__07%2BZGLO%3F%2BSIE%3B%3B%3A%3A&__ReportId%3DhistIndeksu%5E0],referer: http://host.customer.com/starweb/wideoteka/servlet.starweb[Wed Apr 08 22:45:35 2009] [warn] [client 10.13.16.93] FSWB: 1st call,referer: http://host.customer.com/starweb/wideoteka/servlet.starweb[Wed Apr 08 22:45:35 2009] [warn] [client 10.13.16.93] FSWB: buffer:[__sessionNumber=67&__pageid=daneIndeksu&__a1=&__a2=&__a3=&__hiddenstyle=A&__dirtyFlag=undefined&__action=213&Near=ab&SelectedIndexTerms=__07.01.1968%3B%3B%3A%3A&SelectedIndexTerms=__07.01.1973%3B%3B%3A%3A&SelectedIndexTerms=__07.02.1959+NR+9%3B%3B%3A%3A&SelectedIndexTerms=__07.02.1967%3B%3B%3A%3A&SelectedIndexTerms=__07.03.1971%3B%3B%3A%3A&SelectedIndexTerms=__07.04.1968%3B%3B%3A%3A&SelectedIndexTerms=__07.05.1972%3B%3B%3A%3A&SelectedIndexTerms=__07.07.1968%3B%3B%3A%3A&SelectedIndexTerms=__07.08.1972%3B%3B%3A%3A&SelectedIndexTerms=__07.09.1969%3B%3B%3A%3A&SelectedIndexTerms=__07.09.1970%3B%3B%3A%3A&SelectedIndexTerms=__07.11.1971%3B%3B%3A%3A&SelectedIndexTerms=__08.03.1970%3B%3B%3A%3A&SelectedIndexTerms=__08.04.1962%3B%3B%3A%3A&SelectedIndexTerms=__08.04.1973%3B%3B%3A%3A&__ReportId=histIndeksu%5E0],referer: http://host.customer.com/starweb/wideoteka/servlet.starweb[Wed Apr 08 22:45:35 2009] [warn] [client 10.13.16.93] FSWB: splittedparams: __sessionNumber=67 || __pageid=daneIndeksu || __a1= || __a2= ||__a3= || __hiddenstyle=A || __dirtyFlag=undefined || __action=213 ||Near=ab || SelectedIndexTerms=__07.01.1968%3B%3B%3A%3A ||SelectedIndexTerms=__07.01.1973%3B%3B%3A%3A ||SelectedIndexTerms=__07.02.1959+NR+9%3B%3B%3A%3A ||SelectedIndexTerms=__07.02.1967%3B%3B%3A%3A ||SelectedIndexTerms=__07.03.1971%3B%3B%3A%3A ||SelectedIndexTerms=__07.04.1968%3B%3B%3A%3A ||SelectedIndexTerms=__07.05.1972%3B%3B%3A%3A ||SelectedIndexTerms=__07.07.1968%3B%3B%3A%3A ||SelectedIndexTerms=__07.08.1972%3B%3B%3A%3A ||SelectedIndexTerms=__07.09.1969%3B%3B%3A%3A ||SelectedIndexTerms=__07.09.1970%3B%3B%3A%3A ||SelectedIndexTerms=__07.11.1971%3B%3B%3A%3A ||SelectedIndexTerms=__08.03.1970%3B%3B%3A%3A ||SelectedIndexTerms=__08.04.1962%3B%3B%3A%3A ||SelectedIndexTerms=__08.04.1973%3B%3B%3A%3A ||__ReportId=histIndeksu%5E0, referer:http://host.customer.com/starweb/wideoteka/servlet.starweb[Wed Apr 08 22:45:35 2009] [warn] [client 10.13.16.93] FSWB: newcontent:[__sessionNumber=67&__pageid=daneIndeksu&__a1=&__a2=&__a3=&__hiddenstyle=A&__dirtyFlag=undefined&__action=213&Near=ab&SelectedIndexTerms%3D__07.01.1968%3B%3B%3A%3A&SelectedIndexTerms%3D__07.01.1973%3B%3B%3A%3A&SelectedIndexTerms%3D__07.02.1959%2BNR%2B9%3B%3B%3A%3A&SelectedIndexTerms%3D__07.02.1967%3B%3B%3A%3A&SelectedIndexTerms%3D__07.03.1971%3B%3B%3A%3A&SelectedIndexTerms%3D__07.04.1968%3B%3B%3A%3A&SelectedIndexTerms%3D__07.05.1972%3B%3B%3A%3A&SelectedIndexTerms%3D__07.07.1968%3B%3B%3A%3A&SelectedIndexTerms%3D__07.08.1972%3B%3B%3A%3A&SelectedIndexTerms%3D__07.09.1969%3B%3B%3A%3A&SelectedIndexTerms%3D__07.09.1970%3B%3B%3A%3A&SelectedIndexTerms%3D__07.11.1971%3B%3B%3A%3A&SelectedIndexTerms%3D__08.03.1970%3B%3B%3A%3A&SelectedIndexTerms%3D__08.04.1962%3B%3B%3A%3A&SelectedIndexTerms%3D__08.04.1973%3B%3B%3A%3A&__ReportId%3DhistIndeksu%5E0],referer: http://host.customer.com/starweb/wideoteka/servlet.starweb[Wed Apr 08 22:45:48 2009] [warn] [client 10.13.16.93] FSWB: 1st call,referer: http://host.customer.com/starweb/wideoteka/servlet.starweb[Wed Apr 08 22:45:48 2009] [warn] [client 10.13.16.93] FSWB: buffer:[__sessionNumber=67&__pageid=daneIndeksu&__a1=&__a2=&__a3=&__hiddenstyle=A&__dirtyFlag=undefined&__action=216&Near=ab&SelectedIndexTerms=__AB+OVO%3B%3B%3A%3A&SelectedIndexTerms=__AB+URBE+CONDITA%3B%3B%3A%3A&SelectedIndexTerms=__ABADDON%3B%3B%3A%3A&SelectedIndexTerms=__ABAKANOWICZ+W+POZNANIU%3B%3B%3A%3A&SelectedIndexTerms=__ABAKANOWICZ+W+STARMACH+GALLERY%3B%3B%3A%3A&SelectedIndexTerms=__ABAKANY%3B%3B%3A%3A&SelectedIndexTerms=__ABAKANY+CZY+STEREOFORMY%3B%3B%3A%3A&SelectedIndexTerms=__ABAKANY+W+KRAKOWIE+%7CA+00%3A14%3A31%3A15+%7CB+00%3A17%3A49%3A15+%7CC+3+%7CD+18%3B%3B%3A%3A&SelectedIndexTerms=__ABANDON%3B%3B%3A%3A&SelectedIndexTerms=__ABBA%3B%3B%3A%3A&SelectedIndexTerms=__ABBA%3B%3B%3A%3A&SelectedIndexTerms=__ABBA+-+MONEY%3B%3B%3A%3A&SelectedIndexTerms=__ABBA+EBAN+AT+UN%3B%3B%3A%3A&SelectedIndexTerms=__ABBA+PATER%3B%3B%3A%3A&SelectedIndexTerms=__ABBA+W+STUDIO+2%3B%3B%3A%3A&__ReportId=histIndeksu%5E0],referer: http://host.customer.com/starweb/wideoteka/servlet.starweb[Wed Apr 08 22:45:48 2009] [warn] [client 10.13.16.93] FSWB: splittedparams: __sessionNumber=67 || __pageid=daneIndeksu || __a1= || __a2= ||__a3= || __hiddenstyle=A || __dirtyFlag=undefined || __action=216 ||Near=ab || SelectedIndexTerms=__AB+OVO%3B%3B%3A%3A ||SelectedIndexTerms=__AB+URBE+CONDITA%3B%3B%3A%3A ||SelectedIndexTerms=__ABADDON%3B%3B%3A%3A ||SelectedIndexTerms=__ABAKANOWICZ+W+POZNANIU%3B%3B%3A%3A ||SelectedIndexTerms=__ABAKANOWICZ+W+STARMACH+GALLERY%3B%3B%3A%3A ||SelectedIndexTerms=__ABAKANY%3B%3B%3A%3A ||SelectedIndexTerms=__ABAKANY+CZY+STEREOFORMY%3B%3B%3A%3A ||SelectedIndexTerms=__ABAKANY+W+KRAKOWIE+%7CA+00%3A14%3A31%3A15+%7CB+00%3A17%3A49%3A15+%7CC+3+%7CD+18%3B%3B%3A%3A|| SelectedIndexTerms=__ABANDON%3B%3B%3A%3A ||SelectedIndexTerms=__ABBA%3B%3B%3A%3A ||SelectedIndexTerms=__ABBA%3B%3B%3A%3A ||SelectedIndexTerms=__ABBA+-+MONEY%3B%3B%3A%3A ||SelectedIndexTerms=__ABBA+EBAN+AT+UN%3B%3B%3A%3A ||SelectedIndexTerms=__ABBA+PATER%3B%3B%3A%3A ||SelectedIndexTerms=__ABBA+W+STUDIO+2%3B%3B%3A%3A ||__ReportId=histIndeksu%5E0, referer:http://host.customer.com/starweb/wideoteka/servlet.starweb[Wed Apr 08 22:45:48 2009] [warn] [client 10.13.16.93] FSWB: newcontent:[__sessionNumber=67&__pageid=daneIndeksu&__a1=&__a2=&__a3=&__hiddenstyle=A&__dirtyFlag=undefined&__action=216&Near=ab&SelectedIndexTerms%3D__AB%2BOVO%3B%3B%3A%3A&SelectedIndexTerms%3D__AB%2BURBE%2BCONDITA%3B%3B%3A%3A&SelectedIndexTerms%3D__ABADDON%3B%3B%3A%3A&SelectedIndexTerms%3D__ABAKANOWICZ%2BW%2BPOZNANIU%3B%3B%3A%3A&SelectedIndexTerms%3D__ABAKANOWICZ%2BW%2BSTARMACH%2BGALLERY%3B%3B%3A%3A&SelectedIndexTerms%3D__ABAKANY%3B%3B%3A%3A&SelectedIndexTerms%3D__ABAKANY%2BCZY%2BSTEREOFORMY%3B%3B%3A%3A&SelectedIndexTerms%3D__ABAKANY%2BW%2BKRAKOWIE%2B%7CA%2B00%3A14%3A31%3A15%2B%7CB%2B00%3A17%3A49%3A15%2B%7CC%2B3%2B%7CD%2B18%3B%3B%3A%3A&SelectedIndexTerms%3D__ABANDON%3B%3B%3A%3A&SelectedIndexTerms%3D__ABBA%3B%3B%3A%3A&SelectedIndexTerms%3D__ABBA%3B%3B%3A%3A&SelectedIndexTerms%3D__ABBA%2B-%2BMONEY%3B%3B%3A%3A&SelectedIndexTerms%3D__ABBA%2BEBAN%2BAT%2BUN%3B%3B%3A%3A&SelectedIndexTerms%3D__ABBA%2BPATER%3B%3B%3A%3A&SelectedIndexTerms%3D__ABBA%2BW%2BSTUDIO%2B2%3B%3B%3A%3A&__ReportId%3DhistIndeksu%5E0],referer: http://host.customer.com/starweb/wideoteka/servlet.starweb[Wed Apr 08 22:46:22 2009] [warn] [client 10.13.16.93] FSWB: 1st call,referer: http://host.customer.com/starweb/wideoteka/servlet.starweb[Wed Apr 08 22:46:22 2009] [warn] [client 10.13.16.93] FSWB: buffer:[__sessionNumber=67&__pageid=daneIndeksu&__a1=&__a2=&__a3=&__hiddenstyle=A&__dirtyFlag=undefined&__action=213&Near=ABBADON+-+ANIO%A3+ZAG%A3ADY&SelectedIndexTerms=__ABBA%3A+THE+MAMMA+MIA+STORY%3B%3B%3A%3A&SelectedIndexTerms=__ABBADO+IN+BERLIN+-+THE+FIRST+YEAR%3B%3B%3A%3A&SelectedIndexTerms=__ABBADON+-+ANIO%A3+ZAG%A3ADY%3B%3B%3A%3A&SelectedIndexTerms=__ABBAS+KIAROSTANI%3B%3B%3A%3A&SelectedIndexTerms=__ABBEYS+%26+MONASTERIES+%2F+ARMENIAN+MONASTERY+OF+SRPOTS+ISRAEL%3B%3B%3A%3A&SelectedIndexTerms=__ABBEYS+%26+MONASTERIES+%2F+CARMELITE+MONKS+%26+NUNS%3B%3B%3A%3A&SelectedIndexTerms=__ABBEYS+%26+MONASTERIES+%2F+COPTIC+MONASTERRIES+ANBA+BISHOY%3B%3B%3A%3A&SelectedIndexTerms=__ABBEYS+%26+MONASTERIES+%2F+ETHIOPIAN-ORTODOX+JERUSALEM%3B%3B%3A%3A&SelectedIndexTerms=__ABBEYS+%26+MONASTERIES+%2F+FRANCISKANES+-+CONVENTUELS+ASSISI%3B%3B%3A%3A&SelectedIndexTerms=__ABBEYS+%26+MONASTERIES+%2F+GREEK-ORTHODOX+MONASTERY+OF+ST.+JOHN%3B%3B%3A%3A&SelectedIndexTerms=__ABBEYS+%26+MONASTERIES+%2F+GREEK-ORTHODOX+OF+ZERBISTA%3B%3B%3A%3A&SelectedIndexTerms=__ABBEYS+%26+MONASTERIES+%2F+MONASTERIO+DI+CAMALDOLI+ITALY%3B%3B%3A%3A&SelectedIndexTerms=__ABBEYS+%26+MONASTERIES+%2F+ORTHODOX+MONASTERY+HOLY+TRINI],referer: http://host.customer.com/starweb/wideoteka/servlet.starweb


main server error log :

[Wed Apr 08 22:46:22 2009] [crit] [Wed Apr 08 22:46:22 2009] filehttp_filters.c, line 1324, assertion "!(((&(bb)->list))->next == (structapr_bucket *)((char *)((&(bb)->list)) - ((long) (((char *) (&(((structapr_bucket*)((void *)0))->link))) - ((char *) ((void *)0))))))" failed

[Wed Apr 08 22:46:22 2009] [notice] child pid 5518 exit signal Aborted (6)


Here is the filter code, non-essential parts omitted :

sub handler : FilterRequestHandler {
        my $f = shift;
        my $r = $f->r;
        my $logger = $r->log;
        my $ctx = $f->ctx;
        unless (defined $ctx) {
                # first invocation
                $logger->warn("FSWB: 1st call") if DEBUG;
                # remove the request Content-Length header (NOT!)
                #$r->headers_in->unset('Content-Length');
                $f->ctx({ 'post' => ''});
                $ctx = $f->ctx;
        }
    my $post = $ctx->{'post'}; # empty first time around,  but not later
        
        while ($f->read(my $buffer, BUFLEN)) {
                # read from brigade and accumulate
                $logger->warn("FSWB: buffer: [$buffer]") if DEBUG;
                $post .= $buffer;
        }
        # this brigade exhausted, but maybe there's more
        unless ($f->seen_eos) {
                # there is more then
                $ctx->{'post'} = $post; # save accumulated data in context
                return OK;
        }
        # have seen eos, so we can get to work
        my $filtered = '';
        my @qparams = split('&',$post);
        $logger->warn("FSWB: splitted params: " . join(' || ',@qparams)) if 
DEBUG;
        foreach (@qparams) {
                # if the parameter does not contain any % sign at all, take a 
shortcut
                unless ($_ =~ /%/) {
                        $filtered .= '&' . $_;
                        next;
                }
                my $unpar = uri_unescape($_);
                #my @chars = split('',$cleanpar);
                $unpar =~ s/([\x80-\xFF])/$CVTTBL->{$1} || $1/ge;
                $filtered .= '&' . uri_escape($unpar);
        }
        $filtered =~ s/^\&//;
        $logger->warn("FSWB: new content: [$filtered]") if DEBUG;
        # re-add a Content-size header ?
        my $size = length($filtered);
        $r->headers_in->set('Content-Length',"$size");
        # and forward content
        $f->print($filtered);
        
    return OK;
}

Re: generic question on input filters

Reply via email to