On 06.09.2012 11:58, Eliezer Croitoru wrote:
On 9/5/2012 9:56 AM, Eliezer Croitoru wrote:
any leads,?
Well there is a nice progress.
I reviewed the 2.7 store_url_rewrite and I divided the task into more
detailed smaller tasks.
FTR: squid-2.7 ports are exempted as suitable in most cases for
back-porting to stable despite our "no new features" policy.
I am happy for this to be done as a series of patches instead of a
singular change. It can be assembled in trunk and back-ported as a
singular later.
1. Research the url_rewrite interface code and Introduce a modified
version of url_rewrite as url_store_rewrite_program.
- this task is kind of done(passed compiling and running on 3.2.1) by
now and I want to get some ideas on naming conventions for the code
to
fit the project amazing code looks.
list of changed files and code:
create a mimic file of redirect.cc in ./store_rewrite.cc and change
No need.
What I meant earlier about src/redirect.cc being usable is that most of
the code is an exact duplicate. You should only need to:
* write a new start() function ie storeurlRewriteStart()
* add new storeurl_rewriters global
* adapt redirectRegisterWithCacheManager() to register a new
"storeurl_rewrite" report (when that part is written)
* adapt redirectInit() to setup both url_rewrite and storeurl-rewrite
helpers.
* adapt redirectShutdown() to cleanup both url_rewrite and
storeurl-rewrite helpers.
The new storeurlRewriteStart() to be used by store code for this
re-writing and sets up the redirect interface using the new store
helpers and whatever callback the changed URL is to be sent to. The
entire rest of the code for helper management is identical.
all the methods and variables to fit store_rewrite.
strip all the url_rewrite data manipulating actions.
change the debugging info.
(after the store related planning tasks get back here to redo)
./structs.h
adding the proper variables for:
helper naming, bypass(on\off), acl_access namespace, child configs
the ??_rewrites_host of url_rewrite dosnt belong for store_rewrite
process at all.
./cache_cf.cc
state the default configuration for the helper
What do you mean by this?
doConfigure() post-configuration checking?
I don't think there is anything which needs new cache_cf.cc code. The
parsing side if things is identical for url_rewrite_*. The different
defaults and locations are all coded in cf.data.pre ...
./cf.data.pre
stating all config directives for the the helper (copy and modify
from url_rewrite_program)
Okay. However, if merging the stages to trunk separately this will need
to be the final step, since it makes the directives publicly visible. We
want to make these changes and doc/release-notes/release-3.*.sgml at the
same time when it is suitable for public use.
Also, remove the storeurl_* entries at the top marking them as
"obsolete"/unavilable type.
./ClientRequestContext.h
adding int for state
adding bool for done
./client_side_request.h
stating the start method as squidexternal something
Just "extern" will do. We are killing the SQUIDCEXTERN mess.
./client_side_request.cc
adding calls and callouts
./protos.h
stating the start init and shutdown methods.
We are in the process of killing protos.h. Please create a
store_urlrewrite.h header for these definitions instead.
However, see the comments above about redirect.cc. There should not
need to be any new files created for this. Which removes the protos.h,
main.cc and Makefile.am changes.
./main.cc:
calling init and shutdown methods at start/reconfigure etc..
./Makefile.am && ./Makefile.in
adding the source ?.cc file into the commands
Other than the notes above. Okay.
If you have a patch for that please submit for audit :-)
We can pause there for the infrastructure to look fine before moving on
to the store details. I've been waiting on assistance from Henrik or
Alex on that for a while. They are the ones who know the answers to your
questions below AFAIK.
2. Research the workflow of storing objects in memory and store and
introduce psudo for a new workflow of storing objects to avoid bad
effects on cache objects usage in any form that can be.
- I do know that squid uses some hash look-up and I have seen in the
things about it.
- as far I understood from the code:
client_request builds the request of the http object.
creates a mem-object and on the way creates a checksum.
a transfer from of the mem-object to a "store" happens.
if a store rebuild happens it takes all of the data from the file in
the store.
? question how cachemgr gets the list of urls in memory?
so probable points of failure:
using the wrong url to fetch the object.
wrong arguments for checksum.
storing with wrong arguments\url leading to faulty rebuild.
I do remember that when I looked at a stored old store_url_rewrite
cahce file long time ago there were two urls in the file what leads
me
(it's a bit fogy) to think that the stored file was the memobject
cache rather then a set of arguments such as refresh time related
info,method,url,request,response.
I will look at it later but if someone have solid knowledge on how
the store routing was or implemented before i'm rushing into the code
every piece of info will help me when looking into it.
Eliezer
