Re: Best practices for xss security in CMS? - Related Question

[Pete Freitag]

On Wed, Mar 5, 2014 at 11:16 AM, Nick Gleason wrote:

>
> Hi Pete,
> I've been researching CSP and it sounds like a pretty cool option.  But, I
> just wanted to follow up on this comment that you made
> below:-- it will also block inline
> scripts and style elements--
>
> Are you saying that even if you have the "self" or "default" values in
> place, it will block a regular old script in your page?  For instance, if
> you just have something like this:CODE HEREThat will be a
> problem?  Why?
>

Hi Nick,

Yes if you have the following:

Content-Security-Policy: default-src 'self';

It will block any code here tags in your page, you can
only use 

This is a recognized problem in CSP1.0 and CSP 1.1 is currently in
development right now with two solutions for this use case, you can specify
a nonce in the header, so you would do something like this:

Content-Security-Policy: script-src 'self' 'nonce-random_string_123';

Then you can do this:

code here

You can do the same for inline style tags. See
http://w3c.github.io/webappsec/specs/content-security-policy/csp-specification.dev.html#nonce-usage-for-script-elements

The second option in CSP1.1 is hash whitelisting, where you compute a hash
of the script contents and put that in the header. See
http://w3c.github.io/webappsec/specs/content-security-policy/csp-specification.dev.html#hash-usage-for-script-elements


--
Pete Freitag - Adobe Community Professional
http://foundeo.com/ - ColdFusion Consulting & Products
http://hackmycf.com - Is your ColdFusion Server Secure?
http://www.youtube.com/watch?v=ubESB87vl5U - FuseGuard your CFML in 10
minutes


~|
Order the Adobe Coldfusion Anthology now!
http://www.amazon.com/Adobe-Coldfusion-Anthology/dp/1430272155/?tag=houseoffusion
Archive: 
http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:357861
Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm
Unsubscribe: http://www.houseoffusion.com/groups/cf-talk/unsubscribe.cfm

Re: Best practices for xss security in CMS? - Related Question

[Nick Gleason]

Hi Pete,
I've been researching CSP and it sounds like a pretty cool option.  But, I 
just wanted to follow up on this comment that you made 
below:-- it will also block inline 
scripts and style elements--

Are you saying that even if you have the "self" or "default" values in 
place, it will block a regular old script in your page?  For instance, if 
you just have something like this:CODE HEREThat will be a 
problem?  Why?
That seems like a pretty big issue since there are lots of legitimate 
reasons to have a javascript in a web page.
I just want to make sure that I am understanding the situation.
Thanks!
Nick

 


~|
Order the Adobe Coldfusion Anthology now!
http://www.amazon.com/Adobe-Coldfusion-Anthology/dp/1430272155/?tag=houseoffusion
Archive: 
http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:357850
Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm
Unsubscribe: http://www.houseoffusion.com/groups/cf-talk/unsubscribe.cfm

Re: Best practices for xss security in CMS? - Related Question

[Pete Freitag]

On Mon, Mar 3, 2014 at 5:11 PM, Nick Gleason  wrote:

>
> Pete,
>
> Much appreciated.  I guess where I'm being a bit of a dunce is that in your
> example, if a malicious url.query variable was passed in by a hacker,
> wouldn't the display only be available on that single request?  And if I
> come to the same search form 2 minutes later and do a normal search, won't
> it be clean?  I guess that, assuming we have no sql injection to the db, I
> don't see how that attack stays persistent (as it would possibly for a
> comment or forum post).  Sorry to be over-simple on this.
>

Hi Nick,

It is not a persistent attack unless it gets saved to a DB somewhere, etc.
But it is still considered harmful, for example if I send you a link that
uses javascript to write a login form on the page that submits to my
server... then that is not a good thing. Normal visitors can't parse a
query string to recognize that it is rewriting the DOM, especially when
they see HTTPS they expect that everything on the page is as you intended
it. Or the attacker might simply grab cookies and use them to hijack the
victim's session. These examples are a bit more targeted, they will
probably effect every user of the site but the attacker can use the hole to
eventually get the info they are after.


> Re: the content security policy, that looks very interesting.  Watching a
> presentation on it now.  One quick question.  If we are using that on a
> site
> and then an admin comes in and uses an iframe widget from youtube to
> display
> a video on a page in the site, does that get filtered by CSP (and require
> an
> exception for youtube)?  I gather that would need to be excepted in the
> frame-src header, right?
>

Yes you would have to allow the youtube.com domain in your CSP header, eg:

Content-Security-Policy: default-src 'self'; frame-src 'self' youtube.com;

One thing to keep in mind with Content-Security-Policy is that when you
enable it, it will also block inline scripts and style elements, you can
override that using unsalfe-inline but then you also loose a lot of the
benefits of CSP.


~|
Order the Adobe Coldfusion Anthology now!
http://www.amazon.com/Adobe-Coldfusion-Anthology/dp/1430272155/?tag=houseoffusion
Archive: 
http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:357846
Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm
Unsubscribe: http://www.houseoffusion.com/groups/cf-talk/unsubscribe.cfm

Re: Best practices for xss security in CMS? - Related Question

[Money Pit]

To clarify, I was oversimplifying above when I said 'code is being executed
on your server'.  Pete's script example would of course need to link up
with some other vulnerability for that to happen (i.e. an unpatched exploit
of some kind).

Since you can't predict such things, you minimize the number of liberties
someone can take with your server's tender innocence.


-- 
--m@Robertson--
Janitor, The Robertson Team
mysecretbase.com


~|
Order the Adobe Coldfusion Anthology now!
http://www.amazon.com/Adobe-Coldfusion-Anthology/dp/1430272155/?tag=houseoffusion
Archive: 
http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:357817
Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm
Unsubscribe: http://www.houseoffusion.com/groups/cf-talk/unsubscribe.cfm

Re: Best practices for xss security in CMS? - Related Question

[Money Pit]

Nick you are correct, strictly speaking.  That simple example is harmless,
it runs only one time and is 'visible' only to the single client.  Consider
what happens if the payload that is executed is nowhere nearly as benign.
At that point, code of some kind is being executed on your server that does
something you don't intend, and regardless of the fact it only executes
once, it could make all sorts of mischief depending on its level of
sophistication.

-- 
--m@Robertson--
Janitor, The Robertson Team
mysecretbase.com


~|
Order the Adobe Coldfusion Anthology now!
http://www.amazon.com/Adobe-Coldfusion-Anthology/dp/1430272155/?tag=houseoffusion
Archive: 
http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:357816
Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm
Unsubscribe: http://www.houseoffusion.com/groups/cf-talk/unsubscribe.cfm

RE: Best practices for xss security in CMS?

[Nick Gleason]

Hi Russ,

This is very interesting.  In this case, we limit failed logins to a fairly
small number before the login is disabled so in theory that would prevent
dictionary style attacks, even against fairly weak logins.  If you think
that is flawed, let me know.

We've discussed adding an IP filter, although I was thinking that we would
try to do it within the application code rather than at the web server in
case someone doesn't have access to the web server configuration.  I suppose
it could be done in web.config as well (on IIS), but it seems like it would
be easier for client to manage to have the IP list within the user's record.

It would be nice if we could essentially ban all foreign IPs from admin
access (when it made sense for a client), but when researching that a while
back it seemed a little tricky.

With google style 2 factor authentication, I get the idea of requesting a
numeric code in a text message - that doesn't sound terribly complicated.
But, I'm sure that people would want to elect to "stay logged in on this
computer" and I'm not clear on how best to manage that.

Thanks again.

Nick





~|
Order the Adobe Coldfusion Anthology now!
http://www.amazon.com/Adobe-Coldfusion-Anthology/dp/1430272155/?tag=houseoffusion
Archive: 
http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:357814
Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm
Unsubscribe: http://www.houseoffusion.com/groups/cf-talk/unsubscribe.cfm

RE: Best practices for xss security in CMS?

[Russ Michaels]

You could manage the web.config ip filter via cf.
You can also have the option to disable 2 factor authentication for a
specific computer for 30 days which is a common option, using either a
cookie or ip logging.

Russ Michaels
www.michaels.me.uk
cfmldeveloper.com
cflive.net
cfsearch.com
On 3 Mar 2014 22:22, "Nick Gleason"  wrote:

>
> Hi Russ,
>
> This is very interesting.  In this case, we limit failed logins to a fairly
> small number before the login is disabled so in theory that would prevent
> dictionary style attacks, even against fairly weak logins.  If you think
> that is flawed, let me know.
>
> We've discussed adding an IP filter, although I was thinking that we would
> try to do it within the application code rather than at the web server in
> case someone doesn't have access to the web server configuration.  I
> suppose
> it could be done in web.config as well (on IIS), but it seems like it would
> be easier for client to manage to have the IP list within the user's
> record.
>
> It would be nice if we could essentially ban all foreign IPs from admin
> access (when it made sense for a client), but when researching that a while
> back it seemed a little tricky.
>
> With google style 2 factor authentication, I get the idea of requesting a
> numeric code in a text message - that doesn't sound terribly complicated.
> But, I'm sure that people would want to elect to "stay logged in on this
> computer" and I'm not clear on how best to manage that.
>
> Thanks again.
>
> Nick
>
>
>
>
>
> 

~|
Order the Adobe Coldfusion Anthology now!
http://www.amazon.com/Adobe-Coldfusion-Anthology/dp/1430272155/?tag=houseoffusion
Archive: 
http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:357815
Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm
Unsubscribe: http://www.houseoffusion.com/groups/cf-talk/unsubscribe.cfm

RE: Best practices for xss security in CMS? - Related Question

[Nick Gleason]

Pete,  

Much appreciated.  I guess where I'm being a bit of a dunce is that in your
example, if a malicious url.query variable was passed in by a hacker,
wouldn't the display only be available on that single request?  And if I
come to the same search form 2 minutes later and do a normal search, won't
it be clean?  I guess that, assuming we have no sql injection to the db, I
don't see how that attack stays persistent (as it would possibly for a
comment or forum post).  Sorry to be over-simple on this.

Re: the content security policy, that looks very interesting.  Watching a
presentation on it now.  One quick question.  If we are using that on a site
and then an admin comes in and uses an iframe widget from youtube to display
a video on a page in the site, does that get filtered by CSP (and require an
exception for youtube)?  I gather that would need to be excepted in the
frame-src header, right?

Nick





~|
Order the Adobe Coldfusion Anthology now!
http://www.amazon.com/Adobe-Coldfusion-Anthology/dp/1430272155/?tag=houseoffusion
Archive: 
http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:357813
Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm
Unsubscribe: http://www.houseoffusion.com/groups/cf-talk/unsubscribe.cfm

Re: Best practices for xss security in CMS? - Related Question

[Pete Freitag]

On Sun, Mar 2, 2014 at 11:21 PM, Nick Gleason wrote:

>
> Hi guys,
>
> Following up on this thread I have a related question - what are some
> examples of XSS scenarios other than comments and forum posts.
> Any other prominent risk scenarios for XSS?
>

There are a lot of scenarios, essentially anywhere you output a variable
that originated in some part from an external source.

So for example, let's say you have a search form for your site with some
code like this:

Your search for #url.query# returned #search.recordcount#
results

There is an XSS risk there because someone could create a link to
/search.cfm?query=alert('xss') (now if you try that
example in a modern browser you will find that it might not actually work
due to the builtin XSS protection in browsers, but the hole is there and
there are ways to bypass the browsers xss protection).

So basically any time you take a variable that comes from the user or some
other untrusted source and output it, you have the potential for an XSS
hole.

Also you should checkout Content-Security-Policy headers this can help
reduce XSS risks significantly on browsers that support it. See:
http://content-security-policy.com/ for more info or come to my
cf.Objective(2014) presentation :)


--
Pete Freitag - Adobe Community Professional
http://foundeo.com/ - ColdFusion Consulting & Products
http://hackmycf.com - Is your ColdFusion Server Secure?
http://www.youtube.com/watch?v=ubESB87vl5U - FuseGuard your CFML in 10
minutes


~|
Order the Adobe Coldfusion Anthology now!
http://www.amazon.com/Adobe-Coldfusion-Anthology/dp/1430272155/?tag=houseoffusion
Archive: 
http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:357812
Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm
Unsubscribe: http://www.houseoffusion.com/groups/cf-talk/unsubscribe.cfm

Re: Best practices for xss security in CMS?

[Dave Watts]

> Dave, this is an interesting idea which we haven't pursued yet.  I don't
> have a clear sense of how the server configuration would work here.  Would
> you have two separate db servers (one for authored content and one for
> published content) that would sync up?  Or would you have an authoring
> infrastructure that would then generate more traditional static html?  I'm
> just trying to get a sense of how the separation would work.

It can vary, but it's usually pretty simple: an authoring environment
and a production environment where content is published. This by
itself really has nothing to do with preventing XSS on its face, but
it prevents unauthorized users from being able to create content - the
production environment simply has no way to allow users to create or
edit content. These environments typically either share a database, or
data is migrated automatically from one database to another.

Of course, we still need to sanitize content prior to production, but
we only have to worry about people with access to the physical network
where the authoring environment lives. This is often a fairly small
group, and hopefully a more trustworthy group.

Dave Watts, CTO, Fig Leaf Software
1-202-527-9569
http://www.figleaf.com/
http://training.figleaf.com/

Fig Leaf Software is a Veteran-Owned Small Business (VOSB) on
GSA Schedule, and provides the highest caliber vendor-authorized
instruction at our training centers, online, or onsite.

~|
Order the Adobe Coldfusion Anthology now!
http://www.amazon.com/Adobe-Coldfusion-Anthology/dp/1430272155/?tag=houseoffusion
Archive: 
http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:357811
Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm
Unsubscribe: http://www.houseoffusion.com/groups/cf-talk/unsubscribe.cfm

Re: Best practices for xss security in CMS?

[Russ Michaels]

will it all depends on several factors. how many forms do you have within
your site that result in content being inserted into the database and then
displayed on the page. If your CMS is the only place this happens and this
is password protected then you can afford to be more flexible about what
tags you allow.
However relying on passwords alone is almost pointless these days unless
you at the very least enforce some password strength, as your security is
then only as strong as the person with the weakest password.
You could consider some extra steps for any back end/cms system, such a
restricting access by IP address, which is done at web server level. How
you do this depends on how many users you have and if they have static IP's
and if they are likley connect remotely from mobile devices etc.  If you
only have a small number of static IP's to allow, then do that.
If adding specific IP's is bot viable, then use a VPN, and then just allow
the IP of the VPN server, which will allow your users to connect from
anywhere and any device as long as they have a vpn connection.
The other other is 2 factor authentication. This is actually  easier than
it seems, take a look at google authenticator for a real simple solution.



On Mon, Mar 3, 2014 at 4:12 AM, Nick Gleason  wrote:

>
> Hi Russ,
>
> Yes, we can definitely turn these tags on and off.  The challenge is that
> if
> we follow OWASP closely, then we shut off tags that clients genuinely need
> (e.g. iframe for youtube content).  So, we're trying to figure out how to
> give clients adequate features without opening up too much risk.  Of
> course,
> publishing is behind a login so there is that kind of restriction in place
> before you even get to an editor to publish.
>
> I'm curious how wordpress handles this issue.  From the little research I
> have done, it seems that none of these tags (iframe, embed, object) are
> blocked by default in wordpress installations.  I would think that would
> open them up to some risk, but perhaps having the editor behind a secure
> login mitigates the risk to a large extent.
>
> Thanks again!
>
> Nick
>
> -Original Message-
> From: Russ Michaels [mailto:r...@michaels.me.uk]
> Sent: Friday, February 28, 2014 9:39 AM
> To: cf-talk
> Subject: Re: Best practices for xss security in CMS?
>
>
> with any decent editor including CKeditor and tinyMCE, you can specify down
> to a granular level which html tags and attributes are allowed/not allowed,
> just check the docs and there should be a config file somewhere in your CMS
> that instantiates the editor where you can modify these settings.
> So it is pretty easy to do as you need.
> It is also a good idea to restrict other tags to avoid numpty editors from
> just copying and pasting content which screws up the layout.
>
>
>
> On Fri, Feb 28, 2014 at 4:29 PM, Dave Watts  wrote:
>
> >
> > > I'm very interested in your feedback on best practices when 1)
> > > trying to mitigate risk of XSS and other hacks while 2) providing
> > > CMS functionality that includes a web editor that clients use to
> publish
> web pages.
> > > For example, there are many tags like 

RE: Best practices for xss security in CMS? - Related Question

[Nick Gleason]

Hi guys,

Following up on this thread I have a related question - what are some
examples of XSS scenarios other than comments and forum posts.  As I have
researched the topic, it seems like a lot of the XSS examples given relate
to users posting to comments and forums.  That's good to understand but is
not a prominent part of our system at the moment.  So, I'm hoping to get
some other scenarios / examples where there may be risk.  Many of our forms
submit data but don't necessarily display back to other users the way that
comments would.

Any other prominent risk scenarios for XSS?

N

-Original Message-
From: Russ Michaels [mailto:r...@michaels.me.uk] 
Sent: Friday, February 28, 2014 11:58 AM
To: cf-talk
Subject: Re: Best practices for xss security in CMS?


tsk, not reading properly before replying is very naughty, I will set
Charlie Arehart on you.

I am quite confident that fuseguard would do a better job than a generic WAF
on a CF site, and anyone of shared hosting wont really have the option to do
a server wide solution.
but certainly if you use multiple technologies on your server then I agree
that a generic  WAF would be the better way to go, and there are some IIS
modules I  which you can enable just on your own site using the web.config
(helicon do this), so don't need server access, apache is probably the same.



On Fri, Feb 28, 2014 at 7:10 PM, Adam Cameron  wrote:

>
> Sorry, I only read as far as "disabling Javascript" and was commenting 
> on that. The fact remains that anything done *clientside* is not 
> reliable. It seems we're not disagreeing there,
>
> Certainly having a WAF is borderline essential on anything other than 
> a trivial site. I'm not entirely sure doing @ CF level is the correct 
> place to do it, but that's an aside.
>
> Sorry for confusion.
>
> --
> Adam
>
>
> On 1 March 2014 07:59, Russ Michaels  wrote:
>
> >
> > I disagree 100%
> > scanning All form fields globally for any dodgy content is the 
> > complete opposite of narrow sighted, it is a much more efficient way 
> > to make sure nothing gets through rather than instead trying to do 
> > these checks in multiple different places and potentially missing one.
> >
> >
> >
> > On Fri, Feb 28, 2014 at 6:56 PM, Adam Cameron  wrote:
> >
> > >
> > > That's a bit narrow-sighted.
> > >
> > > Hackers don't disable JS to bypass clientside pre-validation, they 
> > > just post the form directly. Often the server code is not coded in 
> > > such a
> way
> > to
> > > be aware how a post is made (via a legit form, or just by a POST
> > request).
> > >
> > > *Always* consider client-side pre-validation a "nice to have" and
> really
> > > more a UX ("hey, you malformed that phone number, wanna try again?"
> sort
> > of
> > > thing) consideration than actual validation. And *always *do 
> > > validation
> > on
> > > the server.
> > >
> > > --
> > > Adam
> > >
> > >
> > >
> > >
> > > On 1 March 2014 07:44, Russ Michaels  wrote:
> > >
> > > >
> > > > although these days if a user has javascript disabled they wont 
> > > > be
> able
> > > to
> > > > use the cms at all as it is a requirement for the editor and all 
> > > > the
> > > AJAXy
> > > > stuff.
> > > > but what you can do, is apply filtering to all form fields at a
> global
> > > > level, so any form submission any page will have anything dodgy
> > removed.
> > > > I believe FuseGuard will do this for you.
> >
>
>
> 



~|
Order the Adobe Coldfusion Anthology now!
http://www.amazon.com/Adobe-Coldfusion-Anthology/dp/1430272155/?tag=houseoffusion
Archive: 
http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:357809
Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm
Unsubscribe: http://www.houseoffusion.com/groups/cf-talk/unsubscribe.cfm

RE: Best practices for xss security in CMS?

[Nick Gleason]

Hi Adam,

Can you tell me a little more about what you mean by coding in order to
prevent posting directly to a form and bypassing validation?

Nick

-Original Message-
From: Adam Cameron [mailto:dacc...@gmail.com] 
Sent: Friday, February 28, 2014 10:56 AM
To: cf-talk
Subject: Re: Best practices for xss security in CMS?


That's a bit narrow-sighted.

Hackers don't disable JS to bypass clientside pre-validation, they just post
the form directly. Often the server code is not coded in such a way to be
aware how a post is made (via a legit form, or just by a POST request).

*Always* consider client-side pre-validation a "nice to have" and really
more a UX ("hey, you malformed that phone number, wanna try again?" sort of
thing) consideration than actual validation. And *always *do validation on
the server.

--
Adam




On 1 March 2014 07:44, Russ Michaels  wrote:

>
> although these days if a user has javascript disabled they wont be able to
> use the cms at all as it is a requirement for the editor and all the AJAXy
> stuff.
> but what you can do, is apply filtering to all form fields at a global
> level, so any form submission any page will have anything dodgy removed.
> I believe FuseGuard will do this for you.
>
>
> On Fri, Feb 28, 2014 at 6:34 PM, Adam Cameron  wrote:
>
> >
> > Also bear in mind that is only half the work. Whatever "pre-validation"
> or
> > UX tweaks one does on the client, one still needs to do the actual
> > validation on the server too.
> >
> >
> > On 1 March 2014 06:38, Russ Michaels  wrote:
> >
> > >
> > > with any decent editor including CKeditor and tinyMCE, you can specify
> > down
> > > to a granular level which html tags and attributes are allowed/not
> > allowed,
> > > just check the docs and there should be a config file somewhere in
your
> > CMS
> > > that instantiates the editor where you can modify these settings.
> > > So it is pretty easy to do as you need.
> > > It is also a good idea to restrict other tags to avoid numpty editors
> > from
> > > just copying and pasting content which screws up the layout.
> > >
> > >
> > >
> > > On Fri, Feb 28, 2014 at 4:29 PM, Dave Watts 
> wrote:
> > >
> > > >
> > > > > I'm very interested in your feedback on best practices when 1)
> trying
> > > to
> > > > > mitigate risk of XSS and other hacks while 2) providing CMS
> > > functionality
> > > > > that includes a web editor that clients use to publish web pages.
> > > > > For example, there are many tags like 

RE: Best practices for xss security in CMS?

[Nick Gleason]

Right now we are using a combination of portcullis plus home grown filters
within the application as well within the web server (which we control).

We would definitely consider looking at Fuseguard as well (but haven't yet).

N

-Original Message-
From: Adam Cameron [mailto:dacc...@gmail.com] 
Sent: Friday, February 28, 2014 11:10 AM
To: cf-talk
Subject: Re: Best practices for xss security in CMS?


Sorry, I only read as far as "disabling Javascript" and was commenting on
that. The fact remains that anything done *clientside* is not reliable. It
seems we're not disagreeing there,

Certainly having a WAF is borderline essential on anything other than a
trivial site. I'm not entirely sure doing @ CF level is the correct place to
do it, but that's an aside.

Sorry for confusion.

--
Adam


On 1 March 2014 07:59, Russ Michaels  wrote:

>
> I disagree 100%
> scanning All form fields globally for any dodgy content is the complete
> opposite of narrow sighted, it is a much more efficient way to make sure
> nothing gets through rather than instead trying to do these checks in
> multiple different places and potentially missing one.
>
>
>
> On Fri, Feb 28, 2014 at 6:56 PM, Adam Cameron  wrote:
>
> >
> > That's a bit narrow-sighted.
> >
> > Hackers don't disable JS to bypass clientside pre-validation, they just
> > post the form directly. Often the server code is not coded in such a way
> to
> > be aware how a post is made (via a legit form, or just by a POST
> request).
> >
> > *Always* consider client-side pre-validation a "nice to have" and really
> > more a UX ("hey, you malformed that phone number, wanna try again?" sort
> of
> > thing) consideration than actual validation. And *always *do validation
> on
> > the server.
> >
> > --
> > Adam
> >
> >
> >
> >
> > On 1 March 2014 07:44, Russ Michaels  wrote:
> >
> > >
> > > although these days if a user has javascript disabled they wont be
able
> > to
> > > use the cms at all as it is a requirement for the editor and all the
> > AJAXy
> > > stuff.
> > > but what you can do, is apply filtering to all form fields at a global
> > > level, so any form submission any page will have anything dodgy
> removed.
> > > I believe FuseGuard will do this for you.
>




~|
Order the Adobe Coldfusion Anthology now!
http://www.amazon.com/Adobe-Coldfusion-Anthology/dp/1430272155/?tag=houseoffusion
Archive: 
http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:357807
Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm
Unsubscribe: http://www.houseoffusion.com/groups/cf-talk/unsubscribe.cfm

RE: Best practices for xss security in CMS?

[Nick Gleason]

Hi Russ,

Yes, we can definitely turn these tags on and off.  The challenge is that if
we follow OWASP closely, then we shut off tags that clients genuinely need
(e.g. iframe for youtube content).  So, we're trying to figure out how to
give clients adequate features without opening up too much risk.  Of course,
publishing is behind a login so there is that kind of restriction in place
before you even get to an editor to publish.

I'm curious how wordpress handles this issue.  From the little research I
have done, it seems that none of these tags (iframe, embed, object) are
blocked by default in wordpress installations.  I would think that would
open them up to some risk, but perhaps having the editor behind a secure
login mitigates the risk to a large extent.

Thanks again!

Nick

-Original Message-
From: Russ Michaels [mailto:r...@michaels.me.uk] 
Sent: Friday, February 28, 2014 9:39 AM
To: cf-talk
Subject: Re: Best practices for xss security in CMS?


with any decent editor including CKeditor and tinyMCE, you can specify down
to a granular level which html tags and attributes are allowed/not allowed,
just check the docs and there should be a config file somewhere in your CMS
that instantiates the editor where you can modify these settings.
So it is pretty easy to do as you need.
It is also a good idea to restrict other tags to avoid numpty editors from
just copying and pasting content which screws up the layout.



On Fri, Feb 28, 2014 at 4:29 PM, Dave Watts  wrote:

>
> > I'm very interested in your feedback on best practices when 1) 
> > trying to mitigate risk of XSS and other hacks while 2) providing 
> > CMS functionality that includes a web editor that clients use to publish
web pages.
> > For example, there are many tags like 

RE: Best practices for xss security in CMS?

[Nick Gleason]

Hi Guys, thanks for all the responses - much appreciated.

Dave, this is an interesting idea which we haven't pursued yet.  I don't
have a clear sense of how the server configuration would work here.  Would
you have two separate db servers (one for authored content and one for
published content) that would sync up?  Or would you have an authoring
infrastructure that would then generate more traditional static html?  I'm
just trying to get a sense of how the separation would work.

N

-Original Message-
From: Dave Watts [mailto:dwa...@figleaf.com] 
Sent: Friday, February 28, 2014 8:29 AM
To: cf-talk
Subject: Re: Best practices for xss security in CMS?


> I'm very interested in your feedback on best practices when 1) trying 
> to mitigate risk of XSS and other hacks while 2) providing CMS 
> functionality that includes a web editor that clients use to publish web
pages.
> For example, there are many tags like 

Re: Best practices for xss security in CMS?

[Russ Michaels]

tsk, not reading properly before replying is very naughty, I will set
Charlie Arehart on you.

I am quite confident that fuseguard would do a better job than a generic
WAF on a CF site, and anyone of shared hosting wont really have the option
to do a server wide solution.
but certainly if you use multiple technologies on your server then I agree
that a generic  WAF would be the better way to go, and there are some IIS
modules I  which you can enable just on your own site using the web.config
(helicon do this), so don't need server access, apache is probably the same.



On Fri, Feb 28, 2014 at 7:10 PM, Adam Cameron  wrote:

>
> Sorry, I only read as far as "disabling Javascript" and was commenting on
> that. The fact remains that anything done *clientside* is not reliable. It
> seems we're not disagreeing there,
>
> Certainly having a WAF is borderline essential on anything other than a
> trivial site. I'm not entirely sure doing @ CF level is the correct place
> to do it, but that's an aside.
>
> Sorry for confusion.
>
> --
> Adam
>
>
> On 1 March 2014 07:59, Russ Michaels  wrote:
>
> >
> > I disagree 100%
> > scanning All form fields globally for any dodgy content is the complete
> > opposite of narrow sighted, it is a much more efficient way to make sure
> > nothing gets through rather than instead trying to do these checks in
> > multiple different places and potentially missing one.
> >
> >
> >
> > On Fri, Feb 28, 2014 at 6:56 PM, Adam Cameron  wrote:
> >
> > >
> > > That's a bit narrow-sighted.
> > >
> > > Hackers don't disable JS to bypass clientside pre-validation, they just
> > > post the form directly. Often the server code is not coded in such a
> way
> > to
> > > be aware how a post is made (via a legit form, or just by a POST
> > request).
> > >
> > > *Always* consider client-side pre-validation a "nice to have" and
> really
> > > more a UX ("hey, you malformed that phone number, wanna try again?"
> sort
> > of
> > > thing) consideration than actual validation. And *always *do validation
> > on
> > > the server.
> > >
> > > --
> > > Adam
> > >
> > >
> > >
> > >
> > > On 1 March 2014 07:44, Russ Michaels  wrote:
> > >
> > > >
> > > > although these days if a user has javascript disabled they wont be
> able
> > > to
> > > > use the cms at all as it is a requirement for the editor and all the
> > > AJAXy
> > > > stuff.
> > > > but what you can do, is apply filtering to all form fields at a
> global
> > > > level, so any form submission any page will have anything dodgy
> > removed.
> > > > I believe FuseGuard will do this for you.
> >
>
>
> 

~|
Order the Adobe Coldfusion Anthology now!
http://www.amazon.com/Adobe-Coldfusion-Anthology/dp/1430272155/?tag=houseoffusion
Archive: 
http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:357804
Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm
Unsubscribe: http://www.houseoffusion.com/groups/cf-talk/unsubscribe.cfm

Re: Best practices for xss security in CMS?

[Adam Cameron]

Sorry, I only read as far as "disabling Javascript" and was commenting on
that. The fact remains that anything done *clientside* is not reliable. It
seems we're not disagreeing there,

Certainly having a WAF is borderline essential on anything other than a
trivial site. I'm not entirely sure doing @ CF level is the correct place
to do it, but that's an aside.

Sorry for confusion.

-- 
Adam


On 1 March 2014 07:59, Russ Michaels  wrote:

>
> I disagree 100%
> scanning All form fields globally for any dodgy content is the complete
> opposite of narrow sighted, it is a much more efficient way to make sure
> nothing gets through rather than instead trying to do these checks in
> multiple different places and potentially missing one.
>
>
>
> On Fri, Feb 28, 2014 at 6:56 PM, Adam Cameron  wrote:
>
> >
> > That's a bit narrow-sighted.
> >
> > Hackers don't disable JS to bypass clientside pre-validation, they just
> > post the form directly. Often the server code is not coded in such a way
> to
> > be aware how a post is made (via a legit form, or just by a POST
> request).
> >
> > *Always* consider client-side pre-validation a "nice to have" and really
> > more a UX ("hey, you malformed that phone number, wanna try again?" sort
> of
> > thing) consideration than actual validation. And *always *do validation
> on
> > the server.
> >
> > --
> > Adam
> >
> >
> >
> >
> > On 1 March 2014 07:44, Russ Michaels  wrote:
> >
> > >
> > > although these days if a user has javascript disabled they wont be able
> > to
> > > use the cms at all as it is a requirement for the editor and all the
> > AJAXy
> > > stuff.
> > > but what you can do, is apply filtering to all form fields at a global
> > > level, so any form submission any page will have anything dodgy
> removed.
> > > I believe FuseGuard will do this for you.
>


~|
Order the Adobe Coldfusion Anthology now!
http://www.amazon.com/Adobe-Coldfusion-Anthology/dp/1430272155/?tag=houseoffusion
Archive: 
http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:357803
Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm
Unsubscribe: http://www.houseoffusion.com/groups/cf-talk/unsubscribe.cfm

Re: Best practices for xss security in CMS?

[Russ Michaels]

I disagree 100%
scanning All form fields globally for any dodgy content is the complete
opposite of narrow sighted, it is a much more efficient way to make sure
nothing gets through rather than instead trying to do these checks in
multiple different places and potentially missing one.



On Fri, Feb 28, 2014 at 6:56 PM, Adam Cameron  wrote:

>
> That's a bit narrow-sighted.
>
> Hackers don't disable JS to bypass clientside pre-validation, they just
> post the form directly. Often the server code is not coded in such a way to
> be aware how a post is made (via a legit form, or just by a POST request).
>
> *Always* consider client-side pre-validation a "nice to have" and really
> more a UX ("hey, you malformed that phone number, wanna try again?" sort of
> thing) consideration than actual validation. And *always *do validation on
> the server.
>
> --
> Adam
>
>
>
>
> On 1 March 2014 07:44, Russ Michaels  wrote:
>
> >
> > although these days if a user has javascript disabled they wont be able
> to
> > use the cms at all as it is a requirement for the editor and all the
> AJAXy
> > stuff.
> > but what you can do, is apply filtering to all form fields at a global
> > level, so any form submission any page will have anything dodgy removed.
> > I believe FuseGuard will do this for you.
> >
> >
> > On Fri, Feb 28, 2014 at 6:34 PM, Adam Cameron  wrote:
> >
> > >
> > > Also bear in mind that is only half the work. Whatever "pre-validation"
> > or
> > > UX tweaks one does on the client, one still needs to do the actual
> > > validation on the server too.
> > >
> > >
> > > On 1 March 2014 06:38, Russ Michaels  wrote:
> > >
> > > >
> > > > with any decent editor including CKeditor and tinyMCE, you can
> specify
> > > down
> > > > to a granular level which html tags and attributes are allowed/not
> > > allowed,
> > > > just check the docs and there should be a config file somewhere in
> your
> > > CMS
> > > > that instantiates the editor where you can modify these settings.
> > > > So it is pretty easy to do as you need.
> > > > It is also a good idea to restrict other tags to avoid numpty editors
> > > from
> > > > just copying and pasting content which screws up the layout.
> > > >
> > > >
> > > >
> > > > On Fri, Feb 28, 2014 at 4:29 PM, Dave Watts 
> > wrote:
> > > >
> > > > >
> > > > > > I'm very interested in your feedback on best practices when 1)
> > trying
> > > > to
> > > > > > mitigate risk of XSS and other hacks while 2) providing CMS
> > > > functionality
> > > > > > that includes a web editor that clients use to publish web pages.
> > > > > > For example, there are many tags like 

Re: Best practices for xss security in CMS?

[Adam Cameron]

That's a bit narrow-sighted.

Hackers don't disable JS to bypass clientside pre-validation, they just
post the form directly. Often the server code is not coded in such a way to
be aware how a post is made (via a legit form, or just by a POST request).

*Always* consider client-side pre-validation a "nice to have" and really
more a UX ("hey, you malformed that phone number, wanna try again?" sort of
thing) consideration than actual validation. And *always *do validation on
the server.

-- 
Adam




On 1 March 2014 07:44, Russ Michaels  wrote:

>
> although these days if a user has javascript disabled they wont be able to
> use the cms at all as it is a requirement for the editor and all the AJAXy
> stuff.
> but what you can do, is apply filtering to all form fields at a global
> level, so any form submission any page will have anything dodgy removed.
> I believe FuseGuard will do this for you.
>
>
> On Fri, Feb 28, 2014 at 6:34 PM, Adam Cameron  wrote:
>
> >
> > Also bear in mind that is only half the work. Whatever "pre-validation"
> or
> > UX tweaks one does on the client, one still needs to do the actual
> > validation on the server too.
> >
> >
> > On 1 March 2014 06:38, Russ Michaels  wrote:
> >
> > >
> > > with any decent editor including CKeditor and tinyMCE, you can specify
> > down
> > > to a granular level which html tags and attributes are allowed/not
> > allowed,
> > > just check the docs and there should be a config file somewhere in your
> > CMS
> > > that instantiates the editor where you can modify these settings.
> > > So it is pretty easy to do as you need.
> > > It is also a good idea to restrict other tags to avoid numpty editors
> > from
> > > just copying and pasting content which screws up the layout.
> > >
> > >
> > >
> > > On Fri, Feb 28, 2014 at 4:29 PM, Dave Watts 
> wrote:
> > >
> > > >
> > > > > I'm very interested in your feedback on best practices when 1)
> trying
> > > to
> > > > > mitigate risk of XSS and other hacks while 2) providing CMS
> > > functionality
> > > > > that includes a web editor that clients use to publish web pages.
> > > > > For example, there are many tags like 

Re: Best practices for xss security in CMS?

[Russ Michaels]

although these days if a user has javascript disabled they wont be able to
use the cms at all as it is a requirement for the editor and all the AJAXy
stuff.
but what you can do, is apply filtering to all form fields at a global
level, so any form submission any page will have anything dodgy removed.
I believe FuseGuard will do this for you.


On Fri, Feb 28, 2014 at 6:34 PM, Adam Cameron  wrote:

>
> Also bear in mind that is only half the work. Whatever "pre-validation" or
> UX tweaks one does on the client, one still needs to do the actual
> validation on the server too.
>
>
> On 1 March 2014 06:38, Russ Michaels  wrote:
>
> >
> > with any decent editor including CKeditor and tinyMCE, you can specify
> down
> > to a granular level which html tags and attributes are allowed/not
> allowed,
> > just check the docs and there should be a config file somewhere in your
> CMS
> > that instantiates the editor where you can modify these settings.
> > So it is pretty easy to do as you need.
> > It is also a good idea to restrict other tags to avoid numpty editors
> from
> > just copying and pasting content which screws up the layout.
> >
> >
> >
> > On Fri, Feb 28, 2014 at 4:29 PM, Dave Watts  wrote:
> >
> > >
> > > > I'm very interested in your feedback on best practices when 1) trying
> > to
> > > > mitigate risk of XSS and other hacks while 2) providing CMS
> > functionality
> > > > that includes a web editor that clients use to publish web pages.
> > > > For example, there are many tags like 

Re: Best practices for xss security in CMS?

[Adam Cameron]

Also bear in mind that is only half the work. Whatever "pre-validation" or
UX tweaks one does on the client, one still needs to do the actual
validation on the server too.


On 1 March 2014 06:38, Russ Michaels  wrote:

>
> with any decent editor including CKeditor and tinyMCE, you can specify down
> to a granular level which html tags and attributes are allowed/not allowed,
> just check the docs and there should be a config file somewhere in your CMS
> that instantiates the editor where you can modify these settings.
> So it is pretty easy to do as you need.
> It is also a good idea to restrict other tags to avoid numpty editors from
> just copying and pasting content which screws up the layout.
>
>
>
> On Fri, Feb 28, 2014 at 4:29 PM, Dave Watts  wrote:
>
> >
> > > I'm very interested in your feedback on best practices when 1) trying
> to
> > > mitigate risk of XSS and other hacks while 2) providing CMS
> functionality
> > > that includes a web editor that clients use to publish web pages.
> > > For example, there are many tags like 

Re: Best practices for xss security in CMS?

[Russ Michaels]

with any decent editor including CKeditor and tinyMCE, you can specify down
to a granular level which html tags and attributes are allowed/not allowed,
just check the docs and there should be a config file somewhere in your CMS
that instantiates the editor where you can modify these settings.
So it is pretty easy to do as you need.
It is also a good idea to restrict other tags to avoid numpty editors from
just copying and pasting content which screws up the layout.



On Fri, Feb 28, 2014 at 4:29 PM, Dave Watts  wrote:

>
> > I'm very interested in your feedback on best practices when 1) trying to
> > mitigate risk of XSS and other hacks while 2) providing CMS functionality
> > that includes a web editor that clients use to publish web pages.
> > For example, there are many tags like 

Re: Best practices for xss security in CMS?

[Dave Watts]

> I'm very interested in your feedback on best practices when 1) trying to
> mitigate risk of XSS and other hacks while 2) providing CMS functionality
> that includes a web editor that clients use to publish web pages.
> For example, there are many tags like 

Re: Best practices for xss security in CMS?

[Nick Gleason]

Thanks very much Pete.
We have implemented Portcullis among other things and that will also block 
tags like the ones mentioned.  I think that may be similar to the ones that 
you mention.  I expect that Fuseguard has something similar.  
I guess my follow up question may have to be with what kind of policy to 
create.  Blocking those tags 100% of the time feels draconian.  Blocking 
them 0% of the time feels risky.  
I expect that we need to develop rules for allowing some people (e.g. web 
master, super user, etc.) to use them while perhaps blocking others.  Does 
anyone on this list have experience with how to make those trade-offs 
effectively?
Nick

 


~|
Order the Adobe Coldfusion Anthology now!
http://www.amazon.com/Adobe-Coldfusion-Anthology/dp/1430272155/?tag=houseoffusion
Archive: 
http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:357716
Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm
Unsubscribe: http://www.houseoffusion.com/groups/cf-talk/unsubscribe.cfm

Re: Best practices for xss security in CMS?

[Pete Freitag]

Hi Nick,

It is tricky to handle HTML content while avoiding XSS, there are a two
tools I'm aware of that can help you here:

1) scrubHTML() - This is one I built in pure CFML and I think it is pretty
easy to build a whitelist of allowed html using it:
https://github.com/foundeo/cfml-security it will use your whitelist and
only allow HTML tags and attributes that you allow to come out the other
end, anything not matching the whitelist is removed.
2) AntiSamy - written in java, widely used, but its policy files can be
tricky to work with, example using it with CFML:
http://www.petefreitag.com/item/760.cfm


--
Pete Freitag - Adobe Community Professional
http://foundeo.com/ - ColdFusion Consulting & Products
http://hackmycf.com - Is your ColdFusion Server Secure?
http://www.youtube.com/watch?v=ubESB87vl5U - FuseGuard your CFML in 10
minutes



On Wed, Feb 19, 2014 at 11:08 PM, Nick Gleason wrote:

>
> Hi All,
> I'm very interested in your feedback on best practices when 1) trying to
> mitigate risk of XSS and other hacks while 2) providing CMS functionality
> that includes a web editor that clients use to publish web pages.
> For example, there are many tags like 

Re: Best practices

[Maureen]

Even worse is when they copy code directly from tutorials and have
names like myTable, myQuery, myForm, foo and bar.

On Thu, Jun 21, 2012 at 9:08 AM, Justin Scott  wrote:
>

> That reminds me of my days writing vScript for the Virtual Advanced
> BBS (way back in 1995) where all of the variables were predefined
> based on letters and numbers, so: a0, a1 ... z8, z9.  Talk about
> torture... and you'd better not need more than 260 of them in any one
> script either.  Shudd

~|
Order the Adobe Coldfusion Anthology now!
http://www.amazon.com/Adobe-Coldfusion-Anthology/dp/1430272155/?tag=houseoffusion
Archive: 
http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:351647
Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm
Unsubscribe: http://www.houseoffusion.com/groups/cf-talk/unsubscribe.cfm

Re: Best practices

[Justin Scott]

> I recently had to help with some code with really
> outlandish variable and field names.
>  (not really, but a good paraphrase)

That reminds me of my days writing vScript for the Virtual Advanced
BBS (way back in 1995) where all of the variables were predefined
based on letters and numbers, so: a0, a1 ... z8, z9.  Talk about
torture... and you'd better not need more than 260 of them in any one
script either.  Shudder.


-Justin

~|
Order the Adobe Coldfusion Anthology now!
http://www.amazon.com/Adobe-Coldfusion-Anthology/dp/1430272155/?tag=houseoffusion
Archive: 
http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:351639
Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm
Unsubscribe: http://www.houseoffusion.com/groups/cf-talk/unsubscribe.cfm

Re: Best practices

[Dave Watts]

> I recently had to help with some code with really outlandish variable and 
> field names.
>         (not really, but a good paraphrase)

I've been telling this story to students for I guess around ten years now.

I've done quite a bit of work reviewing other people's apps, and
suggesting improvements or fixing problems, etc. Once, I was hired by
a company, that in the peak craziness of the dotcom era had bought
some sort of forum for developers - kind of like StackOverflow in
concept. Anyway, it was having performance and scaling problems once
people actually started using it, and that's why they hired me.

The app had been developed by a single person, who I was told lived in
a shack in the woods and was very hard to reach, so I was never able
to talk to him.

Anyway, all the variables throughout the program were named after
characters from "The Lion King". Of course, the program had nothing to
do with the movie, I guess when you live in a shack in the woods
writing code this is the kind of thing you do. I ended up having to
write a "cast of characters" which mapped to what the variables
actually did:

Application.Simba - number of logged-in users
...

I reported all this to the client, of course, but I don't know if the
developer ever suffered any negative repercussions - other than being
the butt of my jokes in the hundreds of CF classes I've taught since.

Dave Watts, CTO, Fig Leaf Software
http://www.figleaf.com/
http://training.figleaf.com/

Fig Leaf Software is a Veteran-Owned Small Business (VOSB) on
GSA Schedule, and provides the highest caliber vendor-authorized
instruction at our training centers, online, or ons

~|
Order the Adobe Coldfusion Anthology now!
http://www.amazon.com/Adobe-Coldfusion-Anthology/dp/1430272155/?tag=houseoffusion
Archive: 
http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:351632
Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm
Unsubscribe: http://www.houseoffusion.com/groups/cf-talk/unsubscribe.cfm

RE: Best practices

[Stephens, Larry V]

And, IMO, there's a point too often overlooked: commenting and style (as in 
indenting code and naming stuff). 

I've sometimes been weak about comments in my code - but I'm trying to do 
better. I have trouble remembering what I was trying to do when I revisit code 
after a few months, let alone looking at someone else's code. And while I'll 
buy that some code is self-documenting I don't buy that on a much grander scale 
than a half-dozen lines or so.

I am insistent that and code written for my department be properly indented and 
that includes javascript (and I don't mean online libraries). I'm willing to 
give up the fractional difference in load or execution time for code I can 
decipher without developing ulcers.

I recently had to help with some code with really outlandish variable and field 
names.
 (not really, but a good paraphrase)

Name stuff so the next person that looks at the code has a clue what you're 
doing.

 

-Original Message-
From: Maureen [mailto:mamamaur...@gmail.com] 
Sent: Wednesday, June 20, 2012 6:18 PM
To: cf-talk
Subject: Re: Best practices


If I wrap a large amount of code in cfoutput tags, I always comment the 
starting and ending tags to describe what they wrap.  It makes it easier to 
match them when debugging.

On Wed, Jun 20, 2012 at 3:05 PM, Rob Voyle  wrote:
>
> Hi folks
>
> Thanks for the input and help.
> I had not been thinking in terms of speed but of accuracy, which 
> doesn't seem to be an issue. The page isn't that big that speed is 
> going to be a problem. It is actually much easier to code without the 
>  as I have several paragraphs with variable scattered throughout.



~|
Order the Adobe Coldfusion Anthology now!
http://www.amazon.com/Adobe-Coldfusion-Anthology/dp/1430272155/?tag=houseoffusion
Archive: 
http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:351630
Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm
Unsubscribe: http://www.houseoffusion.com/groups/cf-talk/unsubscribe.cfm

Re: Best practices

[Maureen]

If I wrap a large amount of code in cfoutput tags, I always comment
the starting and ending tags to describe what they wrap.  It makes it
easier to match them when debugging.

On Wed, Jun 20, 2012 at 3:05 PM, Rob Voyle  wrote:
>
> Hi folks
>
> Thanks for the input and help.
> I had not been thinking in terms of speed but of accuracy, which doesn't seem 
> to
> be an issue. The page isn't that big that speed is going to be a problem. It 
> is
> actually much easier to code without the  as I have several paragraphs
> with variable scattered throughout.

~|
Order the Adobe Coldfusion Anthology now!
http://www.amazon.com/Adobe-Coldfusion-Anthology/dp/1430272155/?tag=houseoffusion
Archive: 
http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:351628
Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm
Unsubscribe: http://www.houseoffusion.com/groups/cf-talk/unsubscribe.cfm

Re: Best practices

[Rob Voyle]

Hi folks

Thanks for the input and help.
I had not been thinking in terms of speed but of accuracy, which doesn't seem 
to 
be an issue. The page isn't that big that speed is going to be a problem. It is 
actually much easier to code without the  as I have several paragraphs 
with variable scattered throughout.

Thanks
Rob
Robert J. Voyle, Psy.D.
Director, Clergy Leadership Institute
For Coaching and Training in Appreciative Inquiry
Author: Restoring Hope: Appreciative Strategies
 to Resolve Grief and Resentment
http://www.appreciativeway.com/
503-647-2378 or 503-647-2382
   


~|
Order the Adobe Coldfusion Anthology now!
http://www.amazon.com/Adobe-Coldfusion-Anthology/dp/1430272155/?tag=houseoffusion
Archive: 
http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:351627
Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm
Unsubscribe: http://www.houseoffusion.com/groups/cf-talk/unsubscribe.cfm

Re: Best practices

[Justin Scott]

> While the general statement you made about bytecode is true, the
> conclusion you draw from it is one that I'd be reluctant to make
> without load testing.

Indeed, I had this debate with someone a few years ago and we beat a
server into the ground for a few hours with both scenarios and the
results were essentially the same either way.  I don't have the exact
numbers anymore, but it was along the lines of a difference of less
than 10ms when the results of millions of iterations were averaged out
(that was on ColdFusion 8 Enterprise on a Dell PowerEdge 2850 server
if memory serves).  I remember this being a big deal back in 1999
under CF4, but in current versions it doesn't appear to matter from a
performance standpoint which cfoutput approach is used.


-Justin Scott

~|
Order the Adobe Coldfusion Anthology now!
http://www.amazon.com/Adobe-Coldfusion-Anthology/dp/1430272155/?tag=houseoffusion
Archive: 
http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:351626
Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm
Unsubscribe: http://www.houseoffusion.com/groups/cf-talk/unsubscribe.cfm

Re: Best practices

[Dave Watts]

> > Everything between cfoutput tags needs to be parsed. So a big
> > page would slow performance, by how much is prob negligible
> > but worth testing to find out.
>
> Remember that this would only be a hit once each time the file was
> changed, as once it's compiled down to bytecode it doesn't have to be
> parsed again.  Back in the CF5 days the code was parsed with each page
> view, but that hasn't been the case since CFMX 6.

While the general statement you made about bytecode is true, the
conclusion you draw from it is one that I'd be reluctant to make
without load testing. CF still has to substitute values for
expressions in your CFOUTPUT blocks, and we don't know whether the
cost of multiple CFOUTPUTs vs single, larger CFOUTPUTs is a
compilation cost or an execution cost. At least, I don't know, and
I've never seen any real evidence one way or the other.

Dave Watts, CTO, Fig Leaf Software
http://www.figleaf.com/
http://training.figleaf.com/

Fig Leaf Software is a Veteran-Owned Small Business (VOSB) on
GSA Schedule, and provides the highest caliber vendor-authorized
instruction at our training centers, online, or onsite

~|
Order the Adobe Coldfusion Anthology now!
http://www.amazon.com/Adobe-Coldfusion-Anthology/dp/1430272155/?tag=houseoffusion
Archive: 
http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:351625
Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm
Unsubscribe: http://www.houseoffusion.com/groups/cf-talk/unsubscribe.cfm

Re: Best practices

[Justin Scott]

> Everything between cfoutput tags needs to be parsed. So a big
> page would slow performance, by how much is prob negligible
> but worth testing to find out.

Remember that this would only be a hit once each time the file was
changed, as once it's compiled down to bytecode it doesn't have to be
parsed again.  Back in the CF5 days the code was parsed with each page
view, but that hasn't been the case since CFMX 6.


-Justin

~|
Order the Adobe Coldfusion Anthology now!
http://www.amazon.com/Adobe-Coldfusion-Anthology/dp/1430272155/?tag=houseoffusion
Archive: 
http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:351623
Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm
Unsubscribe: http://www.houseoffusion.com/groups/cf-talk/unsubscribe.cfm

Re: Best practices

[Russ Michaels]

Everything between cfoutput tags needs to be parsed. So a big page would
slow performance, by how much is prob negligible but worth testing to find
out.
If there are only a few vars in the whole page then only putting the
cfoutput where needed will speed things up. Depends how important those
milliseconds are to you.

Regards
Russ Michaels
On Jun 20, 2012 2:52 AM, "Nathan Strutz"  wrote:

>
> Matt's dead on. It really doesn't matter very much, anymore. It's a style
> preference.
>
> That said, I would add that style is very important! The ability to scan a
> file and know what it's doing without guessing is an important thing.
> Having templates that match and create a cohesive feeling application is
> also important.
>
> With that, I'd say wrapping entire templates is ok so long as you don't
> force a lot of double-## escaping characters, because that looks ugly. Try
> to keep CFML out of your javascript except where it's necessary, because
> that looks ugly. Same for stylesheets, because ID selectors get double-hash
> marks, which removes you from your context.
>
> nathan strutz
> [www.dopefly.com] [hi.im/nathanstrutz] [about.me/nathanstrutz]
>
>
> On Tue, Jun 19, 2012 at 4:25 PM, Matt Quackenbush  >wrote:
>
> >
> > Years ago it was much more performant to use the single tag wrapped
> around
> > everything. Nowadays it is kinda more about personal preference than
> > performance.
> >
> > HTH
> >
> > Sent from my Samsung Galaxy SII
> > On Jun 19, 2012 6:20 PM, "Rob Voyle"  wrote:
> >
> > >
> > > Hi Folks
> > >
> > > I am wondering what is considered best practices for the  tag
> > >
> > > I have a large page many tables, paragraphs etc. that has text and a
> > > series of
> > > variables scattered thru it.
> > > The simplest coding is to put a  at the begining and a
> > >  at
> > > the end and us #variable# thru the page
> > >
> > > or should I use a separate #variable# for each new
> > > variable.
> > >
> > > Thanks
> > > Rob
> > > Robert J. Voyle, Psy.D.
> > > Director, Clergy Leadership Institute
> > > For Coaching and Training in Appreciative Inquiry
> > > Author: Restoring Hope: Appreciative Strategies
> > > to Resolve Grief and Resentment
> > > http://www.appreciativeway.com/
> > > 503-647-2378 or 503-647-2382
> > >
> > >
> > >
> > >
> >
> >
>
> 

~|
Order the Adobe Coldfusion Anthology now!
http://www.amazon.com/Adobe-Coldfusion-Anthology/dp/1430272155/?tag=houseoffusion
Archive: 
http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:351621
Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm
Unsubscribe: http://www.houseoffusion.com/groups/cf-talk/unsubscribe.cfm

Re: Best practices

[Nathan Strutz]

Matt's dead on. It really doesn't matter very much, anymore. It's a style
preference.

That said, I would add that style is very important! The ability to scan a
file and know what it's doing without guessing is an important thing.
Having templates that match and create a cohesive feeling application is
also important.

With that, I'd say wrapping entire templates is ok so long as you don't
force a lot of double-## escaping characters, because that looks ugly. Try
to keep CFML out of your javascript except where it's necessary, because
that looks ugly. Same for stylesheets, because ID selectors get double-hash
marks, which removes you from your context.

nathan strutz
[www.dopefly.com] [hi.im/nathanstrutz] [about.me/nathanstrutz]


On Tue, Jun 19, 2012 at 4:25 PM, Matt Quackenbush wrote:

>
> Years ago it was much more performant to use the single tag wrapped around
> everything. Nowadays it is kinda more about personal preference than
> performance.
>
> HTH
>
> Sent from my Samsung Galaxy SII
> On Jun 19, 2012 6:20 PM, "Rob Voyle"  wrote:
>
> >
> > Hi Folks
> >
> > I am wondering what is considered best practices for the  tag
> >
> > I have a large page many tables, paragraphs etc. that has text and a
> > series of
> > variables scattered thru it.
> > The simplest coding is to put a  at the begining and a
> >  at
> > the end and us #variable# thru the page
> >
> > or should I use a separate #variable# for each new
> > variable.
> >
> > Thanks
> > Rob
> > Robert J. Voyle, Psy.D.
> > Director, Clergy Leadership Institute
> > For Coaching and Training in Appreciative Inquiry
> > Author: Restoring Hope: Appreciative Strategies
> > to Resolve Grief and Resentment
> > http://www.appreciativeway.com/
> > 503-647-2378 or 503-647-2382
> >
> >
> >
> >
>
> 

~|
Order the Adobe Coldfusion Anthology now!
http://www.amazon.com/Adobe-Coldfusion-Anthology/dp/1430272155/?tag=houseoffusion
Archive: 
http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:351620
Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm
Unsubscribe: http://www.houseoffusion.com/groups/cf-talk/unsubscribe.cfm

Re: Best practices

[AJ Mercer]

If you use cfsetting show output only (cant recall attribute)
You have to wrap html in cfoutput tags
On Jun 20, 2012 7:20 AM, "Rob Voyle"  wrote:

>
> Hi Folks
>
> I am wondering what is considered best practices for the  tag
>
> I have a large page many tables, paragraphs etc. that has text and a
> series of
> variables scattered thru it.
> The simplest coding is to put a  at the begining and a
>  at
> the end and us #variable# thru the page
>
> or should I use a separate #variable# for each new
> variable.
>
> Thanks
> Rob
> Robert J. Voyle, Psy.D.
> Director, Clergy Leadership Institute
> For Coaching and Training in Appreciative Inquiry
> Author: Restoring Hope: Appreciative Strategies
> to Resolve Grief and Resentment
> http://www.appreciativeway.com/
> 503-647-2378 or 503-647-2382
>
>
>
> 

~|
Order the Adobe Coldfusion Anthology now!
http://www.amazon.com/Adobe-Coldfusion-Anthology/dp/1430272155/?tag=houseoffusion
Archive: 
http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:351619
Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm
Unsubscribe: http://www.houseoffusion.com/groups/cf-talk/unsubscribe.cfm

Re: Best practices

[Matt Quackenbush]

Years ago it was much more performant to use the single tag wrapped around
everything. Nowadays it is kinda more about personal preference than
performance.

HTH

Sent from my Samsung Galaxy SII
On Jun 19, 2012 6:20 PM, "Rob Voyle"  wrote:

>
> Hi Folks
>
> I am wondering what is considered best practices for the  tag
>
> I have a large page many tables, paragraphs etc. that has text and a
> series of
> variables scattered thru it.
> The simplest coding is to put a  at the begining and a
>  at
> the end and us #variable# thru the page
>
> or should I use a separate #variable# for each new
> variable.
>
> Thanks
> Rob
> Robert J. Voyle, Psy.D.
> Director, Clergy Leadership Institute
> For Coaching and Training in Appreciative Inquiry
> Author: Restoring Hope: Appreciative Strategies
> to Resolve Grief and Resentment
> http://www.appreciativeway.com/
> 503-647-2378 or 503-647-2382
>
>
>
> 

~|
Order the Adobe Coldfusion Anthology now!
http://www.amazon.com/Adobe-Coldfusion-Anthology/dp/1430272155/?tag=houseoffusion
Archive: 
http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:351618
Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm
Unsubscribe: http://www.houseoffusion.com/groups/cf-talk/unsubscribe.cfm

Re: Best practices: Google mapping

[Larry Lyons]

>Is that Jason Dean format?
>
>
>On Thu, Dec 15, 2011 at 2:36 PM, Larry Lyons  wrote:
>>

My iPad has a completely different idea of my typing. (Its becoming typecast 
actually)

I meant to say you may want to try the jquery google maps plugin. It accepts 
json.


http://code.google.com/p/jquery-ui-map/ 

~|
Order the Adobe Coldfusion Anthology now!
http://www.amazon.com/Adobe-Coldfusion-Anthology/dp/1430272155/?tag=houseoffusion
Archive: 
http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:349235
Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm
Unsubscribe: http://www.houseoffusion.com/groups/cf-talk/unsubscribe.cfm

Re: Best practices: Google mapping

[Raymond Camden]

Is that Jason Dean format?


On Thu, Dec 15, 2011 at 2:36 PM, Larry Lyons  wrote:
>
>> Is there a current state of the art for dealing with Google maps and
>> coldfusion, especially asynchronously.  I've used cf_googlemap
>> extensively in the past, but has anything eclipsed it?
> You could try the google maps jQuert plugin. One of the options is accepting 
> Jason input.
>
> http://code.google.com/p/jquery-ui-map/
>
> 

~|
Order the Adobe Coldfusion Anthology now!
http://www.amazon.com/Adobe-Coldfusion-Anthology/dp/1430272155/?tag=houseoffusion
Archive: 
http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:349173
Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm
Unsubscribe: http://www.houseoffusion.com/groups/cf-talk/unsubscribe.cfm

Re: Best practices: Google mapping

[Larry Lyons]

> Is there a current state of the art for dealing with Google maps and 
> coldfusion, especially asynchronously.  I've used cf_googlemap 
> extensively in the past, but has anything eclipsed it? 
You could try the google maps jQuert plugin. One of the options is accepting 
Jason input.

http://code.google.com/p/jquery-ui-map/ 

~|
Order the Adobe Coldfusion Anthology now!
http://www.amazon.com/Adobe-Coldfusion-Anthology/dp/1430272155/?tag=houseoffusion
Archive: 
http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:349172
Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm
Unsubscribe: http://www.houseoffusion.com/groups/cf-talk/unsubscribe.cfm

Re: Best practices: Google mapping

[Raymond Camden]

I've got a few blog entries on using Google Maps w/ CF outside of
CFMAP as well.


On Wed, Dec 14, 2011 at 4:19 PM, Jeff Gladnick  wrote:
>
> Is there a current state of the art for dealing with Google maps and 
> coldfusion, especially asynchronously.  I've used cf_googlemap extensively in 
> the past, but has anything eclipsed it?
>



-- 
===
Raymond Camden, Adobe Developer Evangelist

Email : raymondcam...@gmail.com
Blog : www.raymondcamden.com
Twitter: cfjedimaste

~|
Order the Adobe Coldfusion Anthology now!
http://www.amazon.com/Adobe-Coldfusion-Anthology/dp/1430272155/?tag=houseoffusion
Archive: 
http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:349164
Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm
Unsubscribe: http://www.houseoffusion.com/groups/cf-talk/unsubscribe.cfm

Re: Best practices: Google mapping

[Steve 'Cutter' Blades]

You can try my CFGMap project on RIAForge. Should be well documented, 
but you're welcome to ask questions off list.

http://cfgmap.riaforge.org/

Steve 'Cutter' Blades
Adobe Community Professional
Adobe Certified Expert
Advanced Macromedia ColdFusion MX 7 Developer

http://cutterscrossing.com


Co-Author "Learning Ext JS 3.2" Packt Publishing 2010
https://www.packtpub.com/learning-ext-js-3-2-for-building-dynamic-desktop-style-user-interfaces/book

"The best way to predict the future is to help create it"


On 12/14/2011 5:19 PM, Jeff Gladnick wrote:
> Is there a current state of the art for dealing with Google maps and 
> coldfusion, especially asynchronously.  I've used cf_googlemap extensively in 
> the past, but has anything eclipsed it?
>
> 

~|
Order the Adobe Coldfusion Anthology now!
http://www.amazon.com/Adobe-Coldfusion-Anthology/dp/1430272155/?tag=houseoffusion
Archive: 
http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:349163
Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm
Unsubscribe: http://www.houseoffusion.com/groups/cf-talk/unsubscribe.cfm

RE: Best Practices for Web Site Traffic Tracking

[Rick Faircloth]

Yes.. I do see a Win32-Intel binary distribution.
I was going by information found in the FAQ's.

Thanks!

-Original Message-
From: Rey Bango [mailto:[EMAIL PROTECTED] 
Sent: Wednesday, February 21, 2007 2:23 PM
To: CF-Talk
Subject: Re: Best Practices for Web Site Traffic Tracking

Actually, check the downloads. They have Windows binaries if I recall.

Rey





~|
Create robust enterprise, web RIAs.
Upgrade & integrate Adobe Coldfusion MX7 with Flex 2
http://www.adobe.com/products/coldfusion/flex2/

Archive: 
http://www.houseoffusion.com/groups/CF-Talk/message.cfm/messageid:270384
Subscription: http://www.houseoffusion.com/groups/CF-Talk/subscribe.cfm
Unsubscribe: http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=89.70.4

Re: Best Practices for Web Site Traffic Tracking

[Rey Bango]

Actually, check the downloads. They have Windows binaries if I recall.

Rey

Rick Faircloth wrote:
> Thanks for point that out to me, Rey... unfortunately
> it only runs on Linux and I'm not capable of porting it to Windows.
> 
> Rick
> 
> -Original Message-
> From: Rey Bango [mailto:[EMAIL PROTECTED] 
> Sent: Wednesday, February 21, 2007 11:52 AM
> To: CF-Talk
> Subject: Re: Best Practices for Web Site Traffic Tracking
> 
> Rick,
> 
> I couldn't remember the name of the free stats server that I had seen 
> awhile back. I found it:
> 
> http://www.mrunix.net/webalizer/
> 
> Haven't used it but it might work for you.
> 
> Rey...
> 
> 
> 
> 
> 
> 

~|
Upgrade to Adobe ColdFusion MX7
Experience Flex 2 & MX7 integration & create powerful cross-platform RIAs
http://www.adobe.com/products/coldfusion/flex2/

Archive: 
http://www.houseoffusion.com/groups/CF-Talk/message.cfm/messageid:270373
Subscription: http://www.houseoffusion.com/groups/CF-Talk/subscribe.cfm
Unsubscribe: http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=89.70.4

RE: Best Practices for Web Site Traffic Tracking

[Rick Faircloth]

Thanks for point that out to me, Rey... unfortunately
it only runs on Linux and I'm not capable of porting it to Windows.

Rick

-Original Message-
From: Rey Bango [mailto:[EMAIL PROTECTED] 
Sent: Wednesday, February 21, 2007 11:52 AM
To: CF-Talk
Subject: Re: Best Practices for Web Site Traffic Tracking

Rick,

I couldn't remember the name of the free stats server that I had seen 
awhile back. I found it:

http://www.mrunix.net/webalizer/

Haven't used it but it might work for you.

Rey...





~|
ColdFusion MX7 and Flex 2 
Build sales & marketing dashboard RIA’s for your business. Upgrade now
http://www.adobe.com/products/coldfusion/flex2

Archive: 
http://www.houseoffusion.com/groups/CF-Talk/message.cfm/messageid:270358
Subscription: http://www.houseoffusion.com/groups/CF-Talk/subscribe.cfm
Unsubscribe: 
http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=11502.10531.4

Re: Best Practices for Web Site Traffic Tracking

[Cutter (CFRelated)]

But, we've found that bots will also show a screen res of 0 x 0, so if 
you check this sort of thing you can then exclude anything without a 
screen res.

Cutter
__
http://blog.cutterscrossing.com

Claude Schneegans wrote:
>  >>Anyone know of a discussion/tutorial on the best way to go about
> creating a website traffic management/reporting system?
> 
> Only my own experience on the subject.
> You'll have to parse the headers to distinguish robots from human 
> visitors, otherwise your statistics will be biased.
> This does not guaranty that statistics are perfect, there will still be 
> some email tracking robots that mimic Explorer,
> but it will be better than nothing. And having statistics about robots 
> visiting your web site is a plus.
> 

~|
Macromedia ColdFusion MX7
Upgrade to MX7 & experience time-saving features, more productivity.
http://www.adobe.com/products/coldfusion

Archive: 
http://www.houseoffusion.com/groups/CF-Talk/message.cfm/messageid:270339
Subscription: http://www.houseoffusion.com/groups/CF-Talk/subscribe.cfm
Unsubscribe: http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=89.70.4

Re: Best Practices for Web Site Traffic Tracking

[Rey Bango]

Rick,

I couldn't remember the name of the free stats server that I had seen 
awhile back. I found it:

http://www.mrunix.net/webalizer/

Haven't used it but it might work for you.

Rey...

Rick Faircloth wrote:
> Good point, Claude...
> 
> Rick
> 
> -Original Message-
> From: Claude Schneegans [mailto:[EMAIL PROTECTED] 
> Sent: Tuesday, February 20, 2007 8:10 PM
> To: CF-Talk
> Subject: Re: Best Practices for Web Site Traffic Tracking
> 
>  >>Anyone know of a discussion/tutorial on the best way to go about
> creating a website traffic management/reporting system?
> 
> Only my own experience on the subject.
> You'll have to parse the headers to distinguish robots from human 
> visitors, otherwise your statistics will be biased.
> This does not guaranty that statistics are perfect, there will still be 
> some email tracking robots that mimic Explorer,
> but it will be better than nothing. And having statistics about robots 
> visiting your web site is a plus.
> 

~|
Create Web Applications With ColdFusion MX7 & Flex 2. 
Build powerful, scalable RIAs. Free Trial
http://www.adobe.com/products/coldfusion/flex2/

Archive: 
http://www.houseoffusion.com/groups/CF-Talk/message.cfm/messageid:270335
Subscription: http://www.houseoffusion.com/groups/CF-Talk/subscribe.cfm
Unsubscribe: http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=89.70.4

Re: Best Practices for Web Site Traffic Tracking

[Mik Muller]

I'm doing something similar, but I'm not leaving a user cookie. Hmm... good 
idea. Also, I am logging the user_agent (but not if the user_agent appears to 
be a bot, I don't log their activity... for some sites crawlers was 2/3 of the 
site log activity). This is what I keep:

DATETIME, GROUPID, ITEMID, IP_NUMBER, QUERY_STRING, HTTP_REFERER*, SCRIPT_NAME, 
USER_AGENT, USERID 

This way I can really drill down into what's going on. To keep the tables 
smaller, I should probably put ipnumber and useragent into different tables and 
link them in. And add the cookie/session ID as well (then I'll know who's who 
even if they're not logged in!). Then I think I'll be covered.

The coolest thing about this system is getting the email when someone joins the 
site. I include a link to the live log report and filter for the user's IP 
number when they joined. This shows how the person got to the website, what 
they looked at as they clicked around, joining and verifying (I can see how 
long this process took, important), and what they did after that... until I got 
the email and clicked in to watch. Kinda voyeristic, but informative.

I have over 100 active sites on the server, and each site has its own ms sql 
db, and each db stores its own live log.

* Referer is only logged cfif listLen(cgi.http_referer,"/") gt 1 and 
listGetAt(cgi.http_referer,2,"/") neq cgi.server_name

Mik


At 03:22 PM 2/20/2007, you wrote:
>I've seen dedicated systems solely for parsing the logs. Can chew up a 
>lot of CPU resource. But, they (WebTrends) do have a hosted service that 
>only requires you adding a small scriptlet to pages. They work out 
>pricing according to your projected page hits, then incrementally raise 
>you if you go over your projections. Takes a lot of load off of your 
>systems, but does require some budget planning.
>
>When a user comes to a site we look for a cookie with an id. If it 
>exists we set a session var with the value (never checking for the 
>cookie again). If it doesn't, then we set the cookie and the (user) 
>session var. We also test for an unexpired 'session' id. If one doesn't 
>exist, then we set it. We then record the 'session' id and user id (once 
>per visit), and get the new record's id (for foreign key). Every page 
>click gets recorded to a table, including the timestamp, page name, 
>query string, site (since we do more than one), and the foreign key 
>tying it back to a session and user. We can then query on the foreign 
>key, ordering by timestamp, and see each page hit within the user's 
>session, from start to finish. We can, within the session, record other 
>information about a user (from a form submission or whatever) that we 
>also tie back through the foreign key. This now helps us tie specific 
>user information (when available) to a specific session, knowing what 
>time they arrived, time on site, time on page, what products they looked 
>at, what forms they submitted, etc.
>
>Chews up a lot of CPU.
>
>Cutter
>___
>http://blog.cutterscrossing.com
>
>Rick Faircloth wrote:
>> You've hit on one of the reasons I've asked about this.
>> 
>> I don't have any really high-volume sites, but for some of the
>> sites that I do track, the database entries really pile up over
>> the years.  And I haven't built in a system for archiving.
>> 
>> So when I go to get stats, it takes a lng time for the
>> database (MySQL, not dedicated) to serve up the results.
>> And it spikes the CPU usage, as well, which impacts serving sites.
>> 
>> One of the problems with my system is that *all* the stats are
>> re-created with *every* request.  For instance, cumulative stats
>> are always calculated, even if I'm just asking for stats for a certain date.
>> I know seems like a stupid way to go about the setup, but it
>> wasn't an issue when the database wasn't wasn't so full.  But now I've
>> got one site that has 970,000 records for traffic that the system has
>> to parse each time a request is made for *every* type of information:
>> cumulative visits per page, total unique visitors today, visits today by 
>> page,
>> etc., etc...
>> 
>> And beyond the performance issues, I know there is other data that
>> I could use like tracking movement through a site, but I don't know
>> how to go about doing that.
>> 
>> With so much on my plate, I just *hate* the thought of re-building the
>> system, but it looks like I'll have to.  I thought I'd gather some
>> *best practices* before I begin.
>> 
>> So, WebTrends doesn't use a database?  They just parse s

RE: Best Practices for Web Site Traffic Tracking

[Rick Faircloth]

Good point, Claude...

Rick

-Original Message-
From: Claude Schneegans [mailto:[EMAIL PROTECTED] 
Sent: Tuesday, February 20, 2007 8:10 PM
To: CF-Talk
Subject: Re: Best Practices for Web Site Traffic Tracking

 >>Anyone know of a discussion/tutorial on the best way to go about
creating a website traffic management/reporting system?

Only my own experience on the subject.
You'll have to parse the headers to distinguish robots from human 
visitors, otherwise your statistics will be biased.
This does not guaranty that statistics are perfect, there will still be 
some email tracking robots that mimic Explorer,
but it will be better than nothing. And having statistics about robots 
visiting your web site is a plus.

-- 
___
REUSE CODE! Use custom tags;
See http://www.contentbox.com/claude/customtags/tagstore.cfm
(Please send any spam to this address: [EMAIL PROTECTED])
Thanks.




~|
ColdFusion MX7 by Adobe®
Dyncamically transform webcontent into Adobe PDF with new ColdFusion MX7. 
Free Trial. http://www.adobe.com/products/coldfusion

Archive: 
http://www.houseoffusion.com/groups/CF-Talk/message.cfm/messageid:270271
Subscription: http://www.houseoffusion.com/groups/CF-Talk/subscribe.cfm
Unsubscribe: 
http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=11502.10531.4

Re: Best Practices for Web Site Traffic Tracking

[Claude Schneegans]

 >>Anyone know of a discussion/tutorial on the best way to go about
creating a website traffic management/reporting system?

Only my own experience on the subject.
You'll have to parse the headers to distinguish robots from human 
visitors, otherwise your statistics will be biased.
This does not guaranty that statistics are perfect, there will still be 
some email tracking robots that mimic Explorer,
but it will be better than nothing. And having statistics about robots 
visiting your web site is a plus.

-- 
___
REUSE CODE! Use custom tags;
See http://www.contentbox.com/claude/customtags/tagstore.cfm
(Please send any spam to this address: [EMAIL PROTECTED])
Thanks.


~|
ColdFusion MX7 and Flex 2 
Build sales & marketing dashboard RIA’s for your business. Upgrade now
http://www.adobe.com/products/coldfusion/flex2

Archive: 
http://www.houseoffusion.com/groups/CF-Talk/message.cfm/messageid:270265
Subscription: http://www.houseoffusion.com/groups/CF-Talk/subscribe.cfm
Unsubscribe: http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=89.70.4

RE: Best Practices for Web Site Traffic Tracking

[Rick Faircloth]

I had looked at SmarterStats once before and decided to
create my own, but now I think it would be a wise choice.

Now, I'm doing SEO/SEM for clients, too, and the amount
of info SmarterStats provides is significant.

The Pro version at $199 would be sufficient for now.

Thanks for bringing that to my attention again, Casey!

Rick

-Original Message-
From: Casey Dougall [mailto:[EMAIL PROTECTED] 
Sent: Tuesday, February 20, 2007 5:00 PM
To: CF-Talk
Subject: Re: Best Practices for Web Site Traffic Tracking

SmarterStats is cheep and provides a wealth of information about your
visitor.




~|
ColdFusion MX7 and Flex 2 
Build sales & marketing dashboard RIA’s for your business. Upgrade now
http://www.adobe.com/products/coldfusion/flex2

Archive: 
http://www.houseoffusion.com/groups/CF-Talk/message.cfm/messageid:270253
Subscription: http://www.houseoffusion.com/groups/CF-Talk/subscribe.cfm
Unsubscribe: http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=89.70.4

RE: Best Practices for Web Site Traffic Tracking

[Rick Faircloth]

Thanks for the tip, Adam...

I've been using GA on the sites that I do SEO/SEM for,
but I probably should expand that to the other sites, as well.

Thanks for the feedback!

Rick

-Original Message-
From: Adam Howitt [mailto:[EMAIL PROTECTED] 
Sent: Tuesday, February 20, 2007 4:49 PM
To: CF-Talk
Subject: Re: Best Practices for Web Site Traffic Tracking

Rick,
I would seriously look at Google Analytics:
Benefits
1. it's free
2. it is extensively documented
3. you can manage multiple websites thru one interface
4. there are books written by real authors on the reports
5. the reports are the most user friendly of any I have seen
6. cross section data cuts like "of the people using firefox, what was
their exit page"
7. e-commerce data
Drawbacks
1. It's javascript based - but unless you are serving content to a
niche population with javascript disabled then lets skip that one.
2. It doesn't monitor your 404s unless you add the JavaScript to your
404 page since it isn't log based.

My take is that since it's free and takes about 10 minutes to add to a
page header on your site to be included everywhere - why wouldn't you
install it and try it?  If you install it and don't like it I'm sure
they'll give you your money back :-)

-- 
~~~
Adam Howitt
http://www.walkjogrun.net/?f
I'm running a marathon for the Leukemia and Lymphoma charity:
http://www.active.com/donate/tntil/adamhowitt

On 2/20/07, Rick Faircloth <[EMAIL PROTECTED]> wrote:
> Whew... I think my server would choke... :o)
> Thanks for the info on the procedure, however.
>
> Rick
>
> -Original Message-
> From: Cutter (CFRelated) [mailto:[EMAIL PROTECTED]
> Sent: Tuesday, February 20, 2007 3:22 PM
> To: CF-Talk
> Subject: Re: Best Practices for Web Site Traffic Tracking
>
> I've seen dedicated systems solely for parsing the logs. Can chew up a
> lot of CPU resource. But, they (WebTrends) do have a hosted service that
> only requires you adding a small scriptlet to pages. They work out
> pricing according to your projected page hits, then incrementally raise
> you if you go over your projections. Takes a lot of load off of your
> systems, but does require some budget planning.
>
> When a user comes to a site we look for a cookie with an id. If it
> exists we set a session var with the value (never checking for the
> cookie again). If it doesn't, then we set the cookie and the (user)
> session var. We also test for an unexpired 'session' id. If one doesn't
> exist, then we set it. We then record the 'session' id and user id (once
> per visit), and get the new record's id (for foreign key). Every page
> click gets recorded to a table, including the timestamp, page name,
> query string, site (since we do more than one), and the foreign key
> tying it back to a session and user. We can then query on the foreign
> key, ordering by timestamp, and see each page hit within the user's
> session, from start to finish. We can, within the session, record other
> information about a user (from a form submission or whatever) that we
> also tie back through the foreign key. This now helps us tie specific
> user information (when available) to a specific session, knowing what
> time they arrived, time on site, time on page, what products they looked
> at, what forms they submitted, etc.
>
> Chews up a lot of CPU.
>
> Cutter
> ___
> http://blog.cutterscrossing.com
>



~|
ColdFusion MX7 and Flex 2 
Build sales & marketing dashboard RIA’s for your business. Upgrade now
http://www.adobe.com/products/coldfusion/flex2

Archive: 
http://www.houseoffusion.com/groups/CF-Talk/message.cfm/messageid:270250
Subscription: http://www.houseoffusion.com/groups/CF-Talk/subscribe.cfm
Unsubscribe: http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=89.70.4

Re: Best Practices for Web Site Traffic Tracking

[Casey Dougall]

SmarterStats is cheep and provides a wealth of information about your
visitor.


~|
Create robust enterprise, web RIAs.
Upgrade & integrate Adobe Coldfusion MX7 with Flex 2
http://www.adobe.com/products/coldfusion/flex2/

Archive: 
http://www.houseoffusion.com/groups/CF-Talk/message.cfm/messageid:270247
Subscription: http://www.houseoffusion.com/groups/CF-Talk/subscribe.cfm
Unsubscribe: 
http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=11502.10531.4

Re: Best Practices for Web Site Traffic Tracking

[Adam Howitt]

Rick,
I would seriously look at Google Analytics:
Benefits
1. it's free
2. it is extensively documented
3. you can manage multiple websites thru one interface
4. there are books written by real authors on the reports
5. the reports are the most user friendly of any I have seen
6. cross section data cuts like "of the people using firefox, what was
their exit page"
7. e-commerce data
Drawbacks
1. It's javascript based - but unless you are serving content to a
niche population with javascript disabled then lets skip that one.
2. It doesn't monitor your 404s unless you add the JavaScript to your
404 page since it isn't log based.

My take is that since it's free and takes about 10 minutes to add to a
page header on your site to be included everywhere - why wouldn't you
install it and try it?  If you install it and don't like it I'm sure
they'll give you your money back :-)

-- 
~~~
Adam Howitt
http://www.walkjogrun.net/?f
I'm running a marathon for the Leukemia and Lymphoma charity:
http://www.active.com/donate/tntil/adamhowitt

On 2/20/07, Rick Faircloth <[EMAIL PROTECTED]> wrote:
> Whew... I think my server would choke... :o)
> Thanks for the info on the procedure, however.
>
> Rick
>
> -Original Message-
> From: Cutter (CFRelated) [mailto:[EMAIL PROTECTED]
> Sent: Tuesday, February 20, 2007 3:22 PM
> To: CF-Talk
> Subject: Re: Best Practices for Web Site Traffic Tracking
>
> I've seen dedicated systems solely for parsing the logs. Can chew up a
> lot of CPU resource. But, they (WebTrends) do have a hosted service that
> only requires you adding a small scriptlet to pages. They work out
> pricing according to your projected page hits, then incrementally raise
> you if you go over your projections. Takes a lot of load off of your
> systems, but does require some budget planning.
>
> When a user comes to a site we look for a cookie with an id. If it
> exists we set a session var with the value (never checking for the
> cookie again). If it doesn't, then we set the cookie and the (user)
> session var. We also test for an unexpired 'session' id. If one doesn't
> exist, then we set it. We then record the 'session' id and user id (once
> per visit), and get the new record's id (for foreign key). Every page
> click gets recorded to a table, including the timestamp, page name,
> query string, site (since we do more than one), and the foreign key
> tying it back to a session and user. We can then query on the foreign
> key, ordering by timestamp, and see each page hit within the user's
> session, from start to finish. We can, within the session, record other
> information about a user (from a form submission or whatever) that we
> also tie back through the foreign key. This now helps us tie specific
> user information (when available) to a specific session, knowing what
> time they arrived, time on site, time on page, what products they looked
> at, what forms they submitted, etc.
>
> Chews up a lot of CPU.
>
> Cutter
> ___
> http://blog.cutterscrossing.com
>

~|
ColdFusion MX7 and Flex 2 
Build sales & marketing dashboard RIA’s for your business. Upgrade now
http://www.adobe.com/products/coldfusion/flex2

Archive: 
http://www.houseoffusion.com/groups/CF-Talk/message.cfm/messageid:270245
Subscription: http://www.houseoffusion.com/groups/CF-Talk/subscribe.cfm
Unsubscribe: http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=89.70.4

RE: Best Practices for Web Site Traffic Tracking

[Rick Faircloth]

Whew... I think my server would choke... :o)
Thanks for the info on the procedure, however.

Rick

-Original Message-
From: Cutter (CFRelated) [mailto:[EMAIL PROTECTED] 
Sent: Tuesday, February 20, 2007 3:22 PM
To: CF-Talk
Subject: Re: Best Practices for Web Site Traffic Tracking

I've seen dedicated systems solely for parsing the logs. Can chew up a 
lot of CPU resource. But, they (WebTrends) do have a hosted service that 
only requires you adding a small scriptlet to pages. They work out 
pricing according to your projected page hits, then incrementally raise 
you if you go over your projections. Takes a lot of load off of your 
systems, but does require some budget planning.

When a user comes to a site we look for a cookie with an id. If it 
exists we set a session var with the value (never checking for the 
cookie again). If it doesn't, then we set the cookie and the (user) 
session var. We also test for an unexpired 'session' id. If one doesn't 
exist, then we set it. We then record the 'session' id and user id (once 
per visit), and get the new record's id (for foreign key). Every page 
click gets recorded to a table, including the timestamp, page name, 
query string, site (since we do more than one), and the foreign key 
tying it back to a session and user. We can then query on the foreign 
key, ordering by timestamp, and see each page hit within the user's 
session, from start to finish. We can, within the session, record other 
information about a user (from a form submission or whatever) that we 
also tie back through the foreign key. This now helps us tie specific 
user information (when available) to a specific session, knowing what 
time they arrived, time on site, time on page, what products they looked 
at, what forms they submitted, etc.

Chews up a lot of CPU.

Cutter
___
http://blog.cutterscrossing.com

Rick Faircloth wrote:
> You've hit on one of the reasons I've asked about this.
> 
> I don't have any really high-volume sites, but for some of the
> sites that I do track, the database entries really pile up over
> the years.  And I haven't built in a system for archiving.
> 
> So when I go to get stats, it takes a lng time for the
> database (MySQL, not dedicated) to serve up the results.
> And it spikes the CPU usage, as well, which impacts serving sites.
> 
> One of the problems with my system is that *all* the stats are
> re-created with *every* request.  For instance, cumulative stats
> are always calculated, even if I'm just asking for stats for a certain
date.
> I know seems like a stupid way to go about the setup, but it
> wasn't an issue when the database wasn't wasn't so full.  But now I've
> got one site that has 970,000 records for traffic that the system has
> to parse each time a request is made for *every* type of information:
> cumulative visits per page, total unique visitors today, visits today by
page,
> etc., etc...
> 
> And beyond the performance issues, I know there is other data that
> I could use like tracking movement through a site, but I don't know
> how to go about doing that.
> 
> With so much on my plate, I just *hate* the thought of re-building the
> system, but it looks like I'll have to.  I thought I'd gather some
> *best practices* before I begin.
> 
> So, WebTrends doesn't use a database?  They just parse server
> logs to get the info?  Sure would take a lot of work off the CPU
> and database.
> 
> Rick
> 
> 
> -Original Message-
> From: Cutter (CFRelated) [mailto:[EMAIL PROTECTED] 
> Sent: Tuesday, February 20, 2007 2:37 PM
> To: CF-Talk
> Subject: Re: Best Practices for Web Site Traffic Tracking
> 
> Rick,
> 
> Last week we deployed new code, here at work, redesigning our 
> application and session startup and management, specifically for 
> improving our own click-through user/session tracking on our client's 
> sites. Now, we're talking about a shared application templated system 
> that services 1600+ sites (separate app for each site, same code base), 
> but I can tell you that we have a single, dedicated MS SQL 
> server/machine that is solely used for stats input, processing, and 
> reporting. We have greatly improved (or, our DBA has) how we process 
> this data, but it does chew up a ton of time and resources, with extreme 
> care and attention being paid to indexes and table access locks. If you 
> are maintaining a large, high traffic site, then I would definitely 
> weigh your options carefully. Rolling your own stats can be very 
> beneficial, especially if you need the ability to create specific 
> information tracking (like clicking through steps in a Flash or Flex 
> application), but if you don't re

RE: Best Practices for Web Site Traffic Tracking

[Rick Faircloth]

Of the features they list...

- Key Metrics Performance Dashboard
- 5-Point Scenario Analysis
- Visual Path and Content Analysis
- Scenario "Conversion" Analysis
- Geographic Report
- On-site Advertising
- Basic Search Engine Reporting

all I really need is the last one, "Basic Search Engine Reporting",
which I assume includes entry, path, and exit from the website.
(And the path info I really can do without)

So WebTrends does more that I could build for $800 for sure, but
it's way overkill for me and for my clients...

Right now, my "home-grown" solution provides...

- Total Unique Visitors for Today
- Total Page Views for Today
- Average Page Views Per Unique Visitor
- Total Page Views for Today For Individual Pages

- Total Unique Visitors by Date Range
- Total Page Views by Date Range
- Total Visits by Date Range for Individual Pages

Cumulative stats include:

- Total Page Views
- Average Daily Page Views
- Total Unique Visitors
- Single-Day Max Page Views
- Single-Day Min Page Views
- Total Page Views By Individual Pages
- Top 10 Referrers

I probably have all the stats I need, I just need to break
the system down into components, so that every stat
is not calculated every time!  Like I wrote earlier, that seems
obvious, but it wasn't a problem when the number of records
in the database weren't close to 1 million...

Rick

-Original Message-
From: Robertson-Ravo, Neil (RX)
[mailto:[EMAIL PROTECTED] 
Sent: Tuesday, February 20, 2007 2:54 PM
To: CF-Talk
Subject: Re: Best Practices for Web Site Traffic Tracking

Can you build it for less or = to $800 with the same features? :-)



"This e-mail is from Reed Exhibitions (Gateway House, 28 The Quadrant,
Richmond, Surrey, TW9 1DN, United Kingdom), a division of Reed Business,
Registered in England, Number 678540.  It contains information which is
confidential and may also be privileged.  It is for the exclusive use of the
intended recipient(s).  If you are not the intended recipient(s) please note
that any form of distribution, copying or use of this communication or the
information in it is strictly prohibited and may be unlawful.  If you have
received this communication in error please return it to the sender or call
our switchboard on +44 (0) 20 89107910.  The opinions expressed within this
communication are not necessarily those expressed by Reed Exhibitions." 
Visit our website at http://www.reedexpo.com

-Original Message-
From: Rick Faircloth
To: CF-Talk
Sent: Tue Feb 20 19:32:20 2007
Subject: RE: Best Practices for Web Site Traffic Tracking

Yes, but it doesn't have to be quite that extensive...something like
their Standard Package...(whew, even the Standard version of WebTrends
is around $800!!... yes, I'll build my own!)

Rick

-Original Message-
From: Robertson-Ravo, Neil (RX)
[mailto:[EMAIL PROTECTED] 
Sent: Tuesday, February 20, 2007 2:02 PM
To: CF-Talk
Subject: Re: Best Practices for Web Site Traffic Tracking

You mean like a WebTrends kind of thing?




"This e-mail is from Reed Exhibitions (Gateway House, 28 The Quadrant,
Richmond, Surrey, TW9 1DN, United Kingdom), a division of Reed Business,
Registered in England, Number 678540.  It contains information which is
confidential and may also be privileged.  It is for the exclusive use of the
intended recipient(s).  If you are not the intended recipient(s) please note
that any form of distribution, copying or use of this communication or the
information in it is strictly prohibited and may be unlawful.  If you have
received this communication in error please return it to the sender or call
our switchboard on +44 (0) 20 89107910.  The opinions expressed within this
communication are not necessarily those expressed by Reed Exhibitions." 
Visit our website at http://www.reedexpo.com

-Original Message-
From: Rick Faircloth
To: CF-Talk
Sent: Tue Feb 20 18:58:13 2007
Subject: Best Practices for Web Site Traffic Tracking

Hi, all.

Anyone know of a discussion/tutorial on the best way to go about
creating a website traffic management/reporting system?

I've been using my own methods for several years, and, while they
work well enough, I know there are bound to be better methods.

I'm looking not just for coding, but for overall discussion of
traffic management, such as archiving older data, tracking visitors
movement through a website. just about anything.

Of course, I'm look for CF-based info.

Rick











~|
Create robust enterprise, web RIAs.
Upgrade & integrate Adobe Coldfusion MX7 with Flex 2
http://www.adobe.com/products/coldfusion/flex2/

Archive: 
http://www.houseoffusion.com/groups/CF-Talk/message.cfm/messageid:270229
Subscription: http://www.houseoffusion.com/groups/CF-Talk/subscribe.cfm
Unsubscribe: 
http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=11502.10531.4

Re: Best Practices for Web Site Traffic Tracking

[Cutter (CFRelated)]

I've seen dedicated systems solely for parsing the logs. Can chew up a 
lot of CPU resource. But, they (WebTrends) do have a hosted service that 
only requires you adding a small scriptlet to pages. They work out 
pricing according to your projected page hits, then incrementally raise 
you if you go over your projections. Takes a lot of load off of your 
systems, but does require some budget planning.

When a user comes to a site we look for a cookie with an id. If it 
exists we set a session var with the value (never checking for the 
cookie again). If it doesn't, then we set the cookie and the (user) 
session var. We also test for an unexpired 'session' id. If one doesn't 
exist, then we set it. We then record the 'session' id and user id (once 
per visit), and get the new record's id (for foreign key). Every page 
click gets recorded to a table, including the timestamp, page name, 
query string, site (since we do more than one), and the foreign key 
tying it back to a session and user. We can then query on the foreign 
key, ordering by timestamp, and see each page hit within the user's 
session, from start to finish. We can, within the session, record other 
information about a user (from a form submission or whatever) that we 
also tie back through the foreign key. This now helps us tie specific 
user information (when available) to a specific session, knowing what 
time they arrived, time on site, time on page, what products they looked 
at, what forms they submitted, etc.

Chews up a lot of CPU.

Cutter
___
http://blog.cutterscrossing.com

Rick Faircloth wrote:
> You've hit on one of the reasons I've asked about this.
> 
> I don't have any really high-volume sites, but for some of the
> sites that I do track, the database entries really pile up over
> the years.  And I haven't built in a system for archiving.
> 
> So when I go to get stats, it takes a lng time for the
> database (MySQL, not dedicated) to serve up the results.
> And it spikes the CPU usage, as well, which impacts serving sites.
> 
> One of the problems with my system is that *all* the stats are
> re-created with *every* request.  For instance, cumulative stats
> are always calculated, even if I'm just asking for stats for a certain date.
> I know seems like a stupid way to go about the setup, but it
> wasn't an issue when the database wasn't wasn't so full.  But now I've
> got one site that has 970,000 records for traffic that the system has
> to parse each time a request is made for *every* type of information:
> cumulative visits per page, total unique visitors today, visits today by page,
> etc., etc...
> 
> And beyond the performance issues, I know there is other data that
> I could use like tracking movement through a site, but I don't know
> how to go about doing that.
> 
> With so much on my plate, I just *hate* the thought of re-building the
> system, but it looks like I'll have to.  I thought I'd gather some
> *best practices* before I begin.
> 
> So, WebTrends doesn't use a database?  They just parse server
> logs to get the info?  Sure would take a lot of work off the CPU
> and database.
> 
> Rick
> 
> 
> -Original Message-
> From: Cutter (CFRelated) [mailto:[EMAIL PROTECTED] 
> Sent: Tuesday, February 20, 2007 2:37 PM
> To: CF-Talk
> Subject: Re: Best Practices for Web Site Traffic Tracking
> 
> Rick,
> 
> Last week we deployed new code, here at work, redesigning our 
> application and session startup and management, specifically for 
> improving our own click-through user/session tracking on our client's 
> sites. Now, we're talking about a shared application templated system 
> that services 1600+ sites (separate app for each site, same code base), 
> but I can tell you that we have a single, dedicated MS SQL 
> server/machine that is solely used for stats input, processing, and 
> reporting. We have greatly improved (or, our DBA has) how we process 
> this data, but it does chew up a ton of time and resources, with extreme 
> care and attention being paid to indexes and table access locks. If you 
> are maintaining a large, high traffic site, then I would definitely 
> weigh your options carefully. Rolling your own stats can be very 
> beneficial, especially if you need the ability to create specific 
> information tracking (like clicking through steps in a Flash or Flex 
> application), but if you don't require that sort of degree of 
> specialized tracking then you may be much better off with balancing your 
> application structure (page names, sub directory pathing, etc.) and 
> using something that does log parsing (like WebTrends).
> 
> Cutter
> 
> http://

RE: Best Practices for Web Site Traffic Tracking

[Rick Faircloth]

I use them for my sites that I do SEO/SEM for.
And since it is free, I could use for all my sites.
And the "Site Overlay" is nice...but the data processing
is manual...
(Man, they can certainly churn out some quick stats!)

But I'm considering doing some things like generating
traffic reports at night with scheduled tasks and emailing them to website
owners.  And to do that I'll have to be able to have
access to the data.

The data analysis that my clients want is very basic.
"How many people visited my site today?"  Nothing about
conversion rates or anything like that.  They don't even
pay attention to how many visits for each page.

My biggest need is to work up an stats archive, so all the
records don't have to be calculated every time.

Just thinking out loud...

Rick



-Original Message-
From: Rey Bango [mailto:[EMAIL PROTECTED] 
Sent: Tuesday, February 20, 2007 2:42 PM
To: CF-Talk
Subject: Re: Best Practices for Web Site Traffic Tracking

Rick,

Also checkout Google Analytics (http://www.google.com/analytics/) which 
is Google's implementation of Urchin.

Rey

Rick Faircloth wrote:
> Yes, but it doesn't have to be quite that extensive...something like
> their Standard Package...(whew, even the Standard version of WebTrends
> is around $800!!... yes, I'll build my own!)
> 
> Rick
> 
> -Original Message-
> From: Robertson-Ravo, Neil (RX)
> [mailto:[EMAIL PROTECTED] 
> Sent: Tuesday, February 20, 2007 2:02 PM
> To: CF-Talk
> Subject: Re: Best Practices for Web Site Traffic Tracking
> 
> You mean like a WebTrends kind of thing?
> 
> 
> 
> 
> "This e-mail is from Reed Exhibitions (Gateway House, 28 The Quadrant,
> Richmond, Surrey, TW9 1DN, United Kingdom), a division of Reed Business,
> Registered in England, Number 678540.  It contains information which is
> confidential and may also be privileged.  It is for the exclusive use of
the
> intended recipient(s).  If you are not the intended recipient(s) please
note
> that any form of distribution, copying or use of this communication or the
> information in it is strictly prohibited and may be unlawful.  If you have
> received this communication in error please return it to the sender or
call
> our switchboard on +44 (0) 20 89107910.  The opinions expressed within
this
> communication are not necessarily those expressed by Reed Exhibitions." 
> Visit our website at http://www.reedexpo.com
> 
> -Original Message-
> From: Rick Faircloth
> To: CF-Talk
> Sent: Tue Feb 20 18:58:13 2007
> Subject: Best Practices for Web Site Traffic Tracking
> 
> Hi, all.
> 
> Anyone know of a discussion/tutorial on the best way to go about
> creating a website traffic management/reporting system?
> 
> I've been using my own methods for several years, and, while they
> work well enough, I know there are bound to be better methods.
> 
> I'm looking not just for coding, but for overall discussion of
> traffic management, such as archiving older data, tracking visitors
> movement through a website. just about anything.
> 
> Of course, I'm look for CF-based info.
> 
> Rick
> 
> 
> 
> 
> 
> 
> 
> 



~|
Create robust enterprise, web RIAs.
Upgrade & integrate Adobe Coldfusion MX7 with Flex 2
http://www.adobe.com/products/coldfusion/flex2/

Archive: 
http://www.houseoffusion.com/groups/CF-Talk/message.cfm/messageid:270221
Subscription: http://www.houseoffusion.com/groups/CF-Talk/subscribe.cfm
Unsubscribe: 
http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=11502.10531.4

RE: Best Practices for Web Site Traffic Tracking

[Rick Faircloth]

You've hit on one of the reasons I've asked about this.

I don't have any really high-volume sites, but for some of the
sites that I do track, the database entries really pile up over
the years.  And I haven't built in a system for archiving.

So when I go to get stats, it takes a lng time for the
database (MySQL, not dedicated) to serve up the results.
And it spikes the CPU usage, as well, which impacts serving sites.

One of the problems with my system is that *all* the stats are
re-created with *every* request.  For instance, cumulative stats
are always calculated, even if I'm just asking for stats for a certain date.
I know seems like a stupid way to go about the setup, but it
wasn't an issue when the database wasn't wasn't so full.  But now I've
got one site that has 970,000 records for traffic that the system has
to parse each time a request is made for *every* type of information:
cumulative visits per page, total unique visitors today, visits today by page,
etc., etc...

And beyond the performance issues, I know there is other data that
I could use like tracking movement through a site, but I don't know
how to go about doing that.

With so much on my plate, I just *hate* the thought of re-building the
system, but it looks like I'll have to.  I thought I'd gather some
*best practices* before I begin.

So, WebTrends doesn't use a database?  They just parse server
logs to get the info?  Sure would take a lot of work off the CPU
and database.

Rick


-Original Message-
From: Cutter (CFRelated) [mailto:[EMAIL PROTECTED] 
Sent: Tuesday, February 20, 2007 2:37 PM
To: CF-Talk
Subject: Re: Best Practices for Web Site Traffic Tracking

Rick,

Last week we deployed new code, here at work, redesigning our 
application and session startup and management, specifically for 
improving our own click-through user/session tracking on our client's 
sites. Now, we're talking about a shared application templated system 
that services 1600+ sites (separate app for each site, same code base), 
but I can tell you that we have a single, dedicated MS SQL 
server/machine that is solely used for stats input, processing, and 
reporting. We have greatly improved (or, our DBA has) how we process 
this data, but it does chew up a ton of time and resources, with extreme 
care and attention being paid to indexes and table access locks. If you 
are maintaining a large, high traffic site, then I would definitely 
weigh your options carefully. Rolling your own stats can be very 
beneficial, especially if you need the ability to create specific 
information tracking (like clicking through steps in a Flash or Flex 
application), but if you don't require that sort of degree of 
specialized tracking then you may be much better off with balancing your 
application structure (page names, sub directory pathing, etc.) and 
using something that does log parsing (like WebTrends).

Cutter

http://blog.cutterscrossing.com

Rick Faircloth wrote:
> Hi, all.
> 
> Anyone know of a discussion/tutorial on the best way to go about
> creating a website traffic management/reporting system?
> 
> I've been using my own methods for several years, and, while they
> work well enough, I know there are bound to be better methods.
> 
> I'm looking not just for coding, but for overall discussion of
> traffic management, such as archiving older data, tracking visitors
> movement through a website. just about anything.
> 
> Of course, I'm look for CF-based info.
> 
> Rick
> 
> 
> 
> 



~|
Create Web Applications With ColdFusion MX7 & Flex 2. 
Build powerful, scalable RIAs. Free Trial
http://www.adobe.com/products/coldfusion/flex2/

Archive: 
http://www.houseoffusion.com/groups/CF-Talk/message.cfm/messageid:270218
Subscription: http://www.houseoffusion.com/groups/CF-Talk/subscribe.cfm
Unsubscribe: http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=89.70.4

Re: Best Practices for Web Site Traffic Tracking

[Robertson-Ravo, Neil (RX)]

Can you build it for less or = to $800 with the same features? :-)



"This e-mail is from Reed Exhibitions (Gateway House, 28 The Quadrant,
Richmond, Surrey, TW9 1DN, United Kingdom), a division of Reed Business,
Registered in England, Number 678540.  It contains information which is
confidential and may also be privileged.  It is for the exclusive use of the
intended recipient(s).  If you are not the intended recipient(s) please note
that any form of distribution, copying or use of this communication or the
information in it is strictly prohibited and may be unlawful.  If you have
received this communication in error please return it to the sender or call
our switchboard on +44 (0) 20 89107910.  The opinions expressed within this
communication are not necessarily those expressed by Reed Exhibitions." 
Visit our website at http://www.reedexpo.com

-Original Message-
From: Rick Faircloth
To: CF-Talk
Sent: Tue Feb 20 19:32:20 2007
Subject: RE: Best Practices for Web Site Traffic Tracking

Yes, but it doesn't have to be quite that extensive...something like
their Standard Package...(whew, even the Standard version of WebTrends
is around $800!!... yes, I'll build my own!)

Rick

-Original Message-
From: Robertson-Ravo, Neil (RX)
[mailto:[EMAIL PROTECTED] 
Sent: Tuesday, February 20, 2007 2:02 PM
To: CF-Talk
Subject: Re: Best Practices for Web Site Traffic Tracking

You mean like a WebTrends kind of thing?




"This e-mail is from Reed Exhibitions (Gateway House, 28 The Quadrant,
Richmond, Surrey, TW9 1DN, United Kingdom), a division of Reed Business,
Registered in England, Number 678540.  It contains information which is
confidential and may also be privileged.  It is for the exclusive use of the
intended recipient(s).  If you are not the intended recipient(s) please note
that any form of distribution, copying or use of this communication or the
information in it is strictly prohibited and may be unlawful.  If you have
received this communication in error please return it to the sender or call
our switchboard on +44 (0) 20 89107910.  The opinions expressed within this
communication are not necessarily those expressed by Reed Exhibitions." 
Visit our website at http://www.reedexpo.com

-Original Message-
From: Rick Faircloth
To: CF-Talk
Sent: Tue Feb 20 18:58:13 2007
Subject: Best Practices for Web Site Traffic Tracking

Hi, all.

Anyone know of a discussion/tutorial on the best way to go about
creating a website traffic management/reporting system?

I've been using my own methods for several years, and, while they
work well enough, I know there are bound to be better methods.

I'm looking not just for coding, but for overall discussion of
traffic management, such as archiving older data, tracking visitors
movement through a website. just about anything.

Of course, I'm look for CF-based info.

Rick









~|
Create Web Applications With ColdFusion MX7 & Flex 2. 
Build powerful, scalable RIAs. Free Trial
http://www.adobe.com/products/coldfusion/flex2/

Archive: 
http://www.houseoffusion.com/groups/CF-Talk/message.cfm/messageid:270215
Subscription: http://www.houseoffusion.com/groups/CF-Talk/subscribe.cfm
Unsubscribe: http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=89.70.4

RE: Best Practices for Web Site Traffic Tracking

[Rick Faircloth]

Thanks for the info, Rey... I'll check out the references.

Rick

-Original Message-
From: Rey Bango [mailto:[EMAIL PROTECTED] 
Sent: Tuesday, February 20, 2007 2:38 PM
To: CF-Talk
Subject: Re: Best Practices for Web Site Traffic Tracking

Rick,

Webmaster World has one of the best forums for that topic:

http://www.webmasterworld.com/analytics/

Check them out.

I recommend Sawmill for website tracking data (http://sawmill.net). I've 
also read some VERY good things about Mint (http://www.haveamint.com/) 
although its only a *nix-based solution. If you're running Windows, 
you're out of luck on that one.

Rey...

Rick Faircloth wrote:
> Hi, all.
> 
> Anyone know of a discussion/tutorial on the best way to go about
> creating a website traffic management/reporting system?
> 
> I've been using my own methods for several years, and, while they
> work well enough, I know there are bound to be better methods.
> 
> I'm looking not just for coding, but for overall discussion of
> traffic management, such as archiving older data, tracking visitors
> movement through a website. just about anything.
> 
> Of course, I'm look for CF-based info.
> 
> Rick
> 
> 
> 
> 



~|
Upgrade to Adobe ColdFusion MX7
Experience Flex 2 & MX7 integration & create powerful cross-platform RIAs
http://www.adobe.com/products/coldfusion/flex2/

Archive: 
http://www.houseoffusion.com/groups/CF-Talk/message.cfm/messageid:270212
Subscription: http://www.houseoffusion.com/groups/CF-Talk/subscribe.cfm
Unsubscribe: http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=89.70.4

Re: Best Practices for Web Site Traffic Tracking

[Rey Bango]

Rick,

Also checkout Google Analytics (http://www.google.com/analytics/) which 
is Google's implementation of Urchin.

Rey

Rick Faircloth wrote:
> Yes, but it doesn't have to be quite that extensive...something like
> their Standard Package...(whew, even the Standard version of WebTrends
> is around $800!!... yes, I'll build my own!)
> 
> Rick
> 
> -Original Message-
> From: Robertson-Ravo, Neil (RX)
> [mailto:[EMAIL PROTECTED] 
> Sent: Tuesday, February 20, 2007 2:02 PM
> To: CF-Talk
> Subject: Re: Best Practices for Web Site Traffic Tracking
> 
> You mean like a WebTrends kind of thing?
> 
> 
> 
> 
> "This e-mail is from Reed Exhibitions (Gateway House, 28 The Quadrant,
> Richmond, Surrey, TW9 1DN, United Kingdom), a division of Reed Business,
> Registered in England, Number 678540.  It contains information which is
> confidential and may also be privileged.  It is for the exclusive use of the
> intended recipient(s).  If you are not the intended recipient(s) please note
> that any form of distribution, copying or use of this communication or the
> information in it is strictly prohibited and may be unlawful.  If you have
> received this communication in error please return it to the sender or call
> our switchboard on +44 (0) 20 89107910.  The opinions expressed within this
> communication are not necessarily those expressed by Reed Exhibitions." 
> Visit our website at http://www.reedexpo.com
> 
> -Original Message-
> From: Rick Faircloth
> To: CF-Talk
> Sent: Tue Feb 20 18:58:13 2007
> Subject: Best Practices for Web Site Traffic Tracking
> 
> Hi, all.
> 
> Anyone know of a discussion/tutorial on the best way to go about
> creating a website traffic management/reporting system?
> 
> I've been using my own methods for several years, and, while they
> work well enough, I know there are bound to be better methods.
> 
> I'm looking not just for coding, but for overall discussion of
> traffic management, such as archiving older data, tracking visitors
> movement through a website. just about anything.
> 
> Of course, I'm look for CF-based info.
> 
> Rick
> 
> 
> 
> 
> 
> 
> 
> 

~|
Macromedia ColdFusion MX7
Upgrade to MX7 & experience time-saving features, more productivity.
http://www.adobe.com/products/coldfusion

Archive: 
http://www.houseoffusion.com/groups/CF-Talk/message.cfm/messageid:270211
Subscription: http://www.houseoffusion.com/groups/CF-Talk/subscribe.cfm
Unsubscribe: http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=89.70.4

Re: Best Practices for Web Site Traffic Tracking

[Cutter (CFRelated)]

Rick,

Last week we deployed new code, here at work, redesigning our 
application and session startup and management, specifically for 
improving our own click-through user/session tracking on our client's 
sites. Now, we're talking about a shared application templated system 
that services 1600+ sites (separate app for each site, same code base), 
but I can tell you that we have a single, dedicated MS SQL 
server/machine that is solely used for stats input, processing, and 
reporting. We have greatly improved (or, our DBA has) how we process 
this data, but it does chew up a ton of time and resources, with extreme 
care and attention being paid to indexes and table access locks. If you 
are maintaining a large, high traffic site, then I would definitely 
weigh your options carefully. Rolling your own stats can be very 
beneficial, especially if you need the ability to create specific 
information tracking (like clicking through steps in a Flash or Flex 
application), but if you don't require that sort of degree of 
specialized tracking then you may be much better off with balancing your 
application structure (page names, sub directory pathing, etc.) and 
using something that does log parsing (like WebTrends).

Cutter

http://blog.cutterscrossing.com

Rick Faircloth wrote:
> Hi, all.
> 
> Anyone know of a discussion/tutorial on the best way to go about
> creating a website traffic management/reporting system?
> 
> I've been using my own methods for several years, and, while they
> work well enough, I know there are bound to be better methods.
> 
> I'm looking not just for coding, but for overall discussion of
> traffic management, such as archiving older data, tracking visitors
> movement through a website. just about anything.
> 
> Of course, I'm look for CF-based info.
> 
> Rick
> 
> 
> 
> 

~|
ColdFusion MX7 and Flex 2 
Build sales & marketing dashboard RIA’s for your business. Upgrade now
http://www.adobe.com/products/coldfusion/flex2

Archive: 
http://www.houseoffusion.com/groups/CF-Talk/message.cfm/messageid:270209
Subscription: http://www.houseoffusion.com/groups/CF-Talk/subscribe.cfm
Unsubscribe: 
http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=11502.10531.4

Re: Best Practices for Web Site Traffic Tracking

[Rey Bango]

Rick,

Webmaster World has one of the best forums for that topic:

http://www.webmasterworld.com/analytics/

Check them out.

I recommend Sawmill for website tracking data (http://sawmill.net). I've 
also read some VERY good things about Mint (http://www.haveamint.com/) 
although its only a *nix-based solution. If you're running Windows, 
you're out of luck on that one.

Rey...

Rick Faircloth wrote:
> Hi, all.
> 
> Anyone know of a discussion/tutorial on the best way to go about
> creating a website traffic management/reporting system?
> 
> I've been using my own methods for several years, and, while they
> work well enough, I know there are bound to be better methods.
> 
> I'm looking not just for coding, but for overall discussion of
> traffic management, such as archiving older data, tracking visitors
> movement through a website. just about anything.
> 
> Of course, I'm look for CF-based info.
> 
> Rick
> 
> 
> 
> 

~|
ColdFusion MX7 and Flex 2 
Build sales & marketing dashboard RIA’s for your business. Upgrade now
http://www.adobe.com/products/coldfusion/flex2

Archive: 
http://www.houseoffusion.com/groups/CF-Talk/message.cfm/messageid:270208
Subscription: http://www.houseoffusion.com/groups/CF-Talk/subscribe.cfm
Unsubscribe: http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=89.70.4

RE: Best Practices for Web Site Traffic Tracking

[Rick Faircloth]

Yes, but it doesn't have to be quite that extensive...something like
their Standard Package...(whew, even the Standard version of WebTrends
is around $800!!... yes, I'll build my own!)

Rick

-Original Message-
From: Robertson-Ravo, Neil (RX)
[mailto:[EMAIL PROTECTED] 
Sent: Tuesday, February 20, 2007 2:02 PM
To: CF-Talk
Subject: Re: Best Practices for Web Site Traffic Tracking

You mean like a WebTrends kind of thing?




"This e-mail is from Reed Exhibitions (Gateway House, 28 The Quadrant,
Richmond, Surrey, TW9 1DN, United Kingdom), a division of Reed Business,
Registered in England, Number 678540.  It contains information which is
confidential and may also be privileged.  It is for the exclusive use of the
intended recipient(s).  If you are not the intended recipient(s) please note
that any form of distribution, copying or use of this communication or the
information in it is strictly prohibited and may be unlawful.  If you have
received this communication in error please return it to the sender or call
our switchboard on +44 (0) 20 89107910.  The opinions expressed within this
communication are not necessarily those expressed by Reed Exhibitions." 
Visit our website at http://www.reedexpo.com

-Original Message-
From: Rick Faircloth
To: CF-Talk
Sent: Tue Feb 20 18:58:13 2007
Subject: Best Practices for Web Site Traffic Tracking

Hi, all.

Anyone know of a discussion/tutorial on the best way to go about
creating a website traffic management/reporting system?

I've been using my own methods for several years, and, while they
work well enough, I know there are bound to be better methods.

I'm looking not just for coding, but for overall discussion of
traffic management, such as archiving older data, tracking visitors
movement through a website. just about anything.

Of course, I'm look for CF-based info.

Rick







~|
ColdFusion MX7 and Flex 2 
Build sales & marketing dashboard RIA’s for your business. Upgrade now
http://www.adobe.com/products/coldfusion/flex2

Archive: 
http://www.houseoffusion.com/groups/CF-Talk/message.cfm/messageid:270207
Subscription: http://www.houseoffusion.com/groups/CF-Talk/subscribe.cfm
Unsubscribe: http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=89.70.4

Re: Best Practices for Web Site Traffic Tracking

[Robertson-Ravo, Neil (RX)]

You mean like a WebTrends kind of thing?




"This e-mail is from Reed Exhibitions (Gateway House, 28 The Quadrant,
Richmond, Surrey, TW9 1DN, United Kingdom), a division of Reed Business,
Registered in England, Number 678540.  It contains information which is
confidential and may also be privileged.  It is for the exclusive use of the
intended recipient(s).  If you are not the intended recipient(s) please note
that any form of distribution, copying or use of this communication or the
information in it is strictly prohibited and may be unlawful.  If you have
received this communication in error please return it to the sender or call
our switchboard on +44 (0) 20 89107910.  The opinions expressed within this
communication are not necessarily those expressed by Reed Exhibitions." 
Visit our website at http://www.reedexpo.com

-Original Message-
From: Rick Faircloth
To: CF-Talk
Sent: Tue Feb 20 18:58:13 2007
Subject: Best Practices for Web Site Traffic Tracking

Hi, all.

Anyone know of a discussion/tutorial on the best way to go about
creating a website traffic management/reporting system?

I've been using my own methods for several years, and, while they
work well enough, I know there are bound to be better methods.

I'm looking not just for coding, but for overall discussion of
traffic management, such as archiving older data, tracking visitors
movement through a website. just about anything.

Of course, I'm look for CF-based info.

Rick





~|
Create robust enterprise, web RIAs.
Upgrade & integrate Adobe Coldfusion MX7 with Flex 2
http://www.adobe.com/products/coldfusion/flex2/

Archive: 
http://www.houseoffusion.com/groups/CF-Talk/message.cfm/messageid:270205
Subscription: http://www.houseoffusion.com/groups/CF-Talk/subscribe.cfm
Unsubscribe: http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=89.70.4

Re: Best practices - error handling

[Mike Kear]

Actually so far i have had very little actual information - mostly people
saying they want it too.

Michael DInowitz said he'd see if he could dig out some information he had
- did you ever manage to do that Michael?

I'm heading off to the beach on Sunday, so if i havent got anything to work
with by then i'll have to do some other projects instead.  But based on the
level of interest and the number of "me too" messages I'd have to suggest
there is an opportunity for a breezo or a tutorial somewhere for someone who
has the knowledge to do an up to date presentation on teh current best
practices for handing errors in the day of the OOP ColdFusion.


Cheers
Mike Kear
Windsor, NSW, Australia
Adobe Certified Advanced ColdFusion Developer
AFP Webworks
http://afpwebworks.com
ColdFusion, PHP, ASP, ASP.NET hosting from AUD$15/month


On 1/12/07, Andrew Grosset <[EMAIL PROTECTED]> wrote:
>
> >I would also be interested in that information.
>
> me to!
>
> Andrew
>


~|
Create robust enterprise, web RIAs.
Upgrade & integrate Adobe Coldfusion MX7 with Flex 2
http://ad.doubleclick.net/clk;56760587;14748456;a?http://www.adobe.com/products/coldfusion/flex2/?sdid=LVNU

Archive: 
http://www.houseoffusion.com/groups/CF-Talk/message.cfm/messageid:266359
Subscription: http://www.houseoffusion.com/groups/CF-Talk/subscribe.cfm
Unsubscribe: http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=89.70.4

Re: Best practices - error handling

[Andrew Grosset]

>I would also be interested in that information.

me to!

Andrew

~|
Create robust enterprise, web RIAs.
Upgrade & integrate Adobe Coldfusion MX7 with Flex 2
http://ad.doubleclick.net/clk;56760587;14748456;a?http://www.adobe.com/products/coldfusion/flex2/?sdid=LVNU

Archive: 
http://www.houseoffusion.com/groups/CF-Talk/message.cfm/messageid:266358
Subscription: http://www.houseoffusion.com/groups/CF-Talk/subscribe.cfm
Unsubscribe: http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=89.70.4

Re: Best practices - error handling

[Michael Dinowitz]

Let me dig up the ppt and post it up and see if I can do a voice over or 
something. 

On a related note, if anyone knows someone in the NY area who can take 
dictation and wants to hear me talking on a LOT of technical subjects, please 
contact me off list. I can get 3-4 articles out a week if I could talk them out 
to a person (no software). We're even willing to pay a little for this.

~|
Create robust enterprise, web RIAs.
Upgrade & integrate Adobe Coldfusion MX7 with Flex 2
http://ad.doubleclick.net/clk;56760587;14748456;a?http://www.adobe.com/products/coldfusion/flex2/?sdid=LVNU

Archive: 
http://www.houseoffusion.com/groups/CF-Talk/message.cfm/messageid:265727
Subscription: http://www.houseoffusion.com/groups/CF-Talk/subscribe.cfm
Unsubscribe: 
http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=11502.10531.4

Re: Best practices - error handling

[Matt Robertson]

On 1/4/07, Rey Bango wrote:
> I'd be interested in this info as well.

Me three

-- 
[EMAIL PROTECTED]
Janitor, The Robertson Team
mysecretbase.com

~|
Create robust enterprise, web RIAs.
Upgrade & integrate Adobe Coldfusion MX7 with Flex 2
http://ad.doubleclick.net/clk;56760587;14748456;a?http://www.adobe.com/products/coldfusion/flex2/?sdid=LVNU

Archive: 
http://www.houseoffusion.com/groups/CF-Talk/message.cfm/messageid:265650
Subscription: http://www.houseoffusion.com/groups/CF-Talk/subscribe.cfm
Unsubscribe: 
http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=11502.10531.4

RE: Best practices - error handling

[Ciliotta, Mario]

I would also be interested in that information.

Mike is anything posted on the House of Fusion?

Mario 

-Original Message-
From: Rey Bango [mailto:[EMAIL PROTECTED] 
Sent: Thursday, January 04, 2007 10:43 AM
To: CF-Talk
Subject: Re: Best practices - error handling

I'd be interested in this info as well.

Rey

Mike Kear wrote:
> Every January, when I go to the beach for a holiday, I take the 
> opportunity to review one or more of my common methods and techniques
> to update to the current best practice.Last year I decided to
> commit to learning what i needed to know about OOP and how it applies
> to CF.   It revolutionised how i go about building sides. A couple of
> years ago I committed ot learning about CSS and Standards, and it 
> revolutionised how i code HTML.
> 
> This year, I'm going to review how i handle errors, and 404s, to see 
> if I'm doing it the best way, given that I'm now using Application.cfc 
> rather than Application.cfm and using more OOP techniques for almost 
> everything.
> 
> Is there a resource I can look to that will show me the current best 
> practice for handling errors and 404s, so I can compare with what I'm 
> doing?  (Aside frorm the CF documentation I mean - I've already put 
> that on the list to study again)
> 
> 



~|
Create robust enterprise, web RIAs.
Upgrade & integrate Adobe Coldfusion MX7 with Flex 2
http://ad.doubleclick.net/clk;56760587;14748456;a?http://www.adobe.com/products/coldfusion/flex2/?sdid=LVNU

Archive: 
http://www.houseoffusion.com/groups/CF-Talk/message.cfm/messageid:265636
Subscription: http://www.houseoffusion.com/groups/CF-Talk/subscribe.cfm
Unsubscribe: 
http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=11502.10531.4

Re: Best practices - error handling

[Rey Bango]

I'd be interested in this info as well.

Rey

Mike Kear wrote:
> Every January, when I go to the beach for a holiday, I take the
> opportunity to review one or more of my common methods and techniques
> to update to the current best practice.Last year I decided to
> commit to learning what i needed to know about OOP and how it applies
> to CF.   It revolutionised how i go about building sides. A couple of
> years ago I committed ot learning about CSS and Standards, and it
> revolutionised how i code HTML.
> 
> This year, I'm going to review how i handle errors, and 404s, to see
> if I'm doing it the best way, given that I'm now using Application.cfc
> rather than Application.cfm and using more OOP techniques for almost
> everything.
> 
> Is there a resource I can look to that will show me the current best
> practice for handling errors and 404s, so I can compare with what I'm
> doing?  (Aside frorm the CF documentation I mean - I've already put
> that on the list to study again)
> 
> 

~|
Create robust enterprise, web RIAs.
Upgrade & integrate Adobe Coldfusion MX7 with Flex 2
http://ad.doubleclick.net/clk;56760587;14748456;a?http://www.adobe.com/products/coldfusion/flex2/?sdid=LVNU

Archive: 
http://www.houseoffusion.com/groups/CF-Talk/message.cfm/messageid:265622
Subscription: http://www.houseoffusion.com/groups/CF-Talk/subscribe.cfm
Unsubscribe: 
http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=11502.10531.4

Re: Best practices - error handling

[Will Tomlinson]

Mike,

Mike Dinowitz might have something for you. He gave a great error handling 
preso at CFUNITED. It also included some good OO error handling techniques. 

Will

~|
Create robust enterprise, web RIAs.
Upgrade & integrate Adobe Coldfusion MX7 with Flex 2
http://ad.doubleclick.net/clk;56760587;14748456;a?http://www.adobe.com/products/coldfusion/flex2/?sdid=LVNU

Archive: 
http://www.houseoffusion.com/groups/CF-Talk/message.cfm/messageid:265608
Subscription: http://www.houseoffusion.com/groups/CF-Talk/subscribe.cfm
Unsubscribe: http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=89.70.4

Re: best practices for encryption

[daniel kessler]

ok, great.  Thanks guys.

>You can also use cfinclude and cfsavecontent to read the file, which is
>faster. 
>
>-Original Message-
>From: Rob Wilkerson [mailto:[EMAIL PROTECTED] 
>Sent: 30 October 2006 20:00
>To: CF-Talk
>Subject: Re: best practices for encryption
>
>On 10/30/06, daniel kessler <[EMAIL PROTECTED]> wrote:
>> How do you read something that is above web root?
>> And to destroy it, do you just stick it in a var then overwrite the var
>with  when done?
>
>Using CFFILE and supplying an absolute path.  The content of the file is
>simply the value of the key.  Read the file into memory, use the value that
>way.  Once the request is complete the value is destroyed.
>Or you could destroy it manually using your example.

~|
Introducing the Fusion Authority Quarterly Update. 80 pages of hard-hitting,
up-to-date ColdFusion information by your peers, delivered to your door four 
times a year.
http://www.fusionauthority.com/quarterly

Archive: 
http://www.houseoffusion.com/groups/CF-Talk/message.cfm/messageid:258526
Subscription: http://www.houseoffusion.com/groups/CF-Talk/subscribe.cfm
Unsubscribe: 
http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=11502.10531.4

RE: best practices for encryption

[Snake]

You can also use cfinclude and cfsavecontent to read the file, which is
faster. 

-Original Message-
From: Rob Wilkerson [mailto:[EMAIL PROTECTED] 
Sent: 30 October 2006 20:00
To: CF-Talk
Subject: Re: best practices for encryption

On 10/30/06, daniel kessler <[EMAIL PROTECTED]> wrote:
> How do you read something that is above web root?
> And to destroy it, do you just stick it in a var then overwrite the var
with  when done?

Using CFFILE and supplying an absolute path.  The content of the file is
simply the value of the key.  Read the file into memory, use the value that
way.  Once the request is complete the value is destroyed.
Or you could destroy it manually using your example.



~|
Introducing the Fusion Authority Quarterly Update. 80 pages of hard-hitting,
up-to-date ColdFusion information by your peers, delivered to your door four 
times a year.
http://www.fusionauthority.com/quarterly

Archive: 
http://www.houseoffusion.com/groups/CF-Talk/message.cfm/messageid:258471
Subscription: http://www.houseoffusion.com/groups/CF-Talk/subscribe.cfm
Unsubscribe: http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=89.70.4

Re: best practices for encryption

[Rob Wilkerson]

On 10/30/06, daniel kessler <[EMAIL PROTECTED]> wrote:
> How do you read something that is above web root?
> And to destroy it, do you just stick it in a var then overwrite the var with 
>  when done?

Using CFFILE and supplying an absolute path.  The content of the file
is simply the value of the key.  Read the file into memory, use the
value that way.  Once the request is complete the value is destroyed.
Or you could destroy it manually using your example.

~|
Introducing the Fusion Authority Quarterly Update. 80 pages of hard-hitting,
up-to-date ColdFusion information by your peers, delivered to your door four 
times a year.
http://www.fusionauthority.com/quarterly

Archive: 
http://www.houseoffusion.com/groups/CF-Talk/message.cfm/messageid:258467
Subscription: http://www.houseoffusion.com/groups/CF-Talk/subscribe.cfm
Unsubscribe: http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=89.70.4

Re: best practices for encryption

[daniel kessler]

How do you read something that is above web root?
And to destroy it, do you just stick it in a var then overwrite the var with 
 when done?

>I recently had the same situation come up and ended up choosing the  
>security-by-obscurity approach.  I generated a key as you did and  
>stored it in a file outside of the web root.  I read the key as  
>needed and destroy it to keep it out of memory.  I'd be interested in  
>how others handled this, as well.
>
>On Sep 25, 2006, at 3:35 PM, Ray Champagne wrote:
>
>>

~|
Introducing the Fusion Authority Quarterly Update. 80 pages of hard-hitting,
up-to-date ColdFusion information by your peers, delivered to your door four 
times a year.
http://www.fusionauthority.com/quarterly

Archive: 
http://www.houseoffusion.com/groups/CF-Talk/message.cfm/messageid:258462
Subscription: http://www.houseoffusion.com/groups/CF-Talk/subscribe.cfm
Unsubscribe: 
http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=11502.10531.4

Re: best practices for encryption

[Matt Robertson]

On 9/25/06, Rob Wilkerson <[EMAIL PROTECTED]> wrote:
> I generated a key as you did and
> stored it in a file outside of the web root.  I read the key as
> needed and destroy it to keep it out of memory.

I should mention that I only store stuff in Application.cfm when I am
not working with anything important, which isn't what Ray is doing
from the sound of it.  What you're doing sounds like its making the
best of a bad situation.  A simple cfinclude can include a file
anywhere on the drive regardless of whether its below the web root.

-- 
[EMAIL PROTECTED]
Janitor, MSB Web Systems
mysecretbase.com

~|
Introducing the Fusion Authority Quarterly Update. 80 pages of hard-hitting,
up-to-date ColdFusion information by your peers, delivered to your door four 
times a year.
http://www.fusionauthority.com/quarterly

Archive: 
http://www.houseoffusion.com/groups/CF-Talk/message.cfm/messageid:254164
Subscription: http://www.houseoffusion.com/groups/CF-Talk/subscribe.cfm
Unsubscribe: http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=89.70.4

Re: best practices for encryption

[Rob Wilkerson]

I recently had the same situation come up and ended up choosing the  
security-by-obscurity approach.  I generated a key as you did and  
stored it in a file outside of the web root.  I read the key as  
needed and destroy it to keep it out of memory.  I'd be interested in  
how others handled this, as well.

On Sep 25, 2006, at 3:35 PM, Ray Champagne wrote:

> So, first time I've ever ran into the need to encrypt data in my  
> DB, and I
> already have a question.  When using the Encrypt function in CF,  
> one must
> supply a key, I'm using the GenerateSecretKey function to get said  
> key.  My
> question is, once I've stored the encrypted field in the database,  
> where do
> I store the key so that I can use the decrypt function on the other  
> side to
> retrieve the data?  Should generate a new key for every string, or  
> use one
> that will work on the entire DB?  Never been down this road before,  
> so any
> pointers would be helpful.
>
>
>
>
>
>
>
> 

~|
Introducing the Fusion Authority Quarterly Update. 80 pages of hard-hitting,
up-to-date ColdFusion information by your peers, delivered to your door four 
times a year.
http://www.fusionauthority.com/quarterly

Archive: 
http://www.houseoffusion.com/groups/CF-Talk/message.cfm/messageid:254162
Subscription: http://www.houseoffusion.com/groups/CF-Talk/subscribe.cfm
Unsubscribe: 
http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=11502.10531.4

RE: best practices for encryption

[Ray Champagne]

Heh - OK.  That's really what I was wondering.  It sounded to me like
putting all the latest alarms and anti-theft devices on your car, then
leaving the window down while you shop.  I guess I wasn't all that far
off

> -Original Message-
> From: Matt Robertson [mailto:[EMAIL PROTECTED]
> Sent: Monday, September 25, 2006 5:12 PM
> To: CF-Talk
> Subject: Re: best practices for encryption
> Importance: High
> 
> On 9/25/06, Ray Champagne <[EMAIL PROTECTED]> wrote:
> > I was
> > more worried about where and how to store the generated key to decrypt
the
> > data "on the other side".
> 
> That right there is the weak link in the chain no matter what you do.
> Someone can hack the box and get that key, and at that point they own
> you.  Because of that Iike to use public/private key encryption along
> the lines of the RSA method, where you only store the encryption
> algorithm on the box and the user pastes in the decryption key
> themselves via a form, and stores it off-box.  If your customer is
> serious about encryption and limitation of liability thats the way to
> go.
> 
> http://developer.perthweb.com.au/textcrypt.html
> 
> Beyond that, when using low-level stuff I stick the key in an
> application var that is fed by Application.cfm
> 
> --
> [EMAIL PROTECTED]
> Janitor, MSB Web Systems
> mysecretbase.com
> 
> 

~|
Introducing the Fusion Authority Quarterly Update. 80 pages of hard-hitting,
up-to-date ColdFusion information by your peers, delivered to your door four 
times a year.
http://www.fusionauthority.com/quarterly

Archive: 
http://www.houseoffusion.com/groups/CF-Talk/message.cfm/messageid:254160
Subscription: http://www.houseoffusion.com/groups/CF-Talk/subscribe.cfm
Unsubscribe: 
http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=11502.10531.4

Re: best practices for encryption

[Matt Robertson]

On 9/25/06, Ray Champagne <[EMAIL PROTECTED]> wrote:
> I was
> more worried about where and how to store the generated key to decrypt the
> data "on the other side".

That right there is the weak link in the chain no matter what you do.
Someone can hack the box and get that key, and at that point they own
you.  Because of that Iike to use public/private key encryption along
the lines of the RSA method, where you only store the encryption
algorithm on the box and the user pastes in the decryption key
themselves via a form, and stores it off-box.  If your customer is
serious about encryption and limitation of liability thats the way to
go.

http://developer.perthweb.com.au/textcrypt.html

Beyond that, when using low-level stuff I stick the key in an
application var that is fed by Application.cfm

-- 
[EMAIL PROTECTED]
Janitor, MSB Web Systems
mysecretbase.com

~|
Introducing the Fusion Authority Quarterly Update. 80 pages of hard-hitting,
up-to-date ColdFusion information by your peers, delivered to your door four 
times a year.
http://www.fusionauthority.com/quarterly

Archive: 
http://www.houseoffusion.com/groups/CF-Talk/message.cfm/messageid:254159
Subscription: http://www.houseoffusion.com/groups/CF-Talk/subscribe.cfm
Unsubscribe: 
http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=11502.10531.4

RE: best practices for encryption

[Ray Champagne]

But to back up a little bit, I wasn't asking how to encrypt, although I
appreciate the advice (and I'm considering what way to go right now).  I was
more worried about where and how to store the generated key to decrypt the
data "on the other side".

> -Original Message-
> From: Matt Robertson [mailto:[EMAIL PROTECTED]
> Sent: Monday, September 25, 2006 4:49 PM
> To: CF-Talk
> Subject: Re: best practices for encryption
> 
> The correct answer to your question really depends on the specifics of
> the job at hand.  If, for example, you are storing account login
> passwords I would say that a salted hash is a mighty good option, if
> not the best.  But that won't work for a lot of things.
> 
> What are you up to?
> 
> --
> [EMAIL PROTECTED]
> Janitor, MSB Web Systems
> mysecretbase.com
> 
> 

~|
Introducing the Fusion Authority Quarterly Update. 80 pages of hard-hitting,
up-to-date ColdFusion information by your peers, delivered to your door four 
times a year.
http://www.fusionauthority.com/quarterly

Archive: 
http://www.houseoffusion.com/groups/CF-Talk/message.cfm/messageid:254153
Subscription: http://www.houseoffusion.com/groups/CF-Talk/subscribe.cfm
Unsubscribe: http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=89.70.4

RE: best practices for encryption

[Ray Champagne]

Storing customer account numbers.  Not a huge deal, but the client wants to
cover all his bases, since, technically, the account number could tie into
their bank account if the person hacking tried hard enough.

> -Original Message-
> From: Matt Robertson [mailto:[EMAIL PROTECTED]
> Sent: Monday, September 25, 2006 4:49 PM
> To: CF-Talk
> Subject: Re: best practices for encryption
> 
> The correct answer to your question really depends on the specifics of
> the job at hand.  If, for example, you are storing account login
> passwords I would say that a salted hash is a mighty good option, if
> not the best.  But that won't work for a lot of things.
> 
> What are you up to?
> 
> --
> [EMAIL PROTECTED]
> Janitor, MSB Web Systems
> mysecretbase.com
> 
> 

~|
Introducing the Fusion Authority Quarterly Update. 80 pages of hard-hitting,
up-to-date ColdFusion information by your peers, delivered to your door four 
times a year.
http://www.fusionauthority.com/quarterly

Archive: 
http://www.houseoffusion.com/groups/CF-Talk/message.cfm/messageid:254152
Subscription: http://www.houseoffusion.com/groups/CF-Talk/subscribe.cfm
Unsubscribe: http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=89.70.4

Re: best practices for encryption

[Matt Robertson]

The correct answer to your question really depends on the specifics of
the job at hand.  If, for example, you are storing account login
passwords I would say that a salted hash is a mighty good option, if
not the best.  But that won't work for a lot of things.

What are you up to?

-- 
[EMAIL PROTECTED]
Janitor, MSB Web Systems
mysecretbase.com

~|
Introducing the Fusion Authority Quarterly Update. 80 pages of hard-hitting,
up-to-date ColdFusion information by your peers, delivered to your door four 
times a year.
http://www.fusionauthority.com/quarterly

Archive: 
http://www.houseoffusion.com/groups/CF-Talk/message.cfm/messageid:254148
Subscription: http://www.houseoffusion.com/groups/CF-Talk/subscribe.cfm
Unsubscribe: 
http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=11502.10531.4

Re: best practices for encryption

[Teddy Payne]

If you use Encrypt, you will need the same key to Decrypt the data back.

There are multiple types of encryption that CF supports, so you may want to
research into which one fits you best.

You may want to look up the ideas of adding an additional string to your
encryption.  A good idea is to have a UUID and a the string you want to
encrypt concatenated together to provide that much more level of complexity
to decrypt.  I have heard this called "adding salt."

Also, when you pass the value to some action page, it is in clear text, so
you might want to put the process to encrypt a string for DB insertion in a
SSL encrypter folder.

I have seen the key stored in the application.cfm and .cfc files for ease of
decryption or stored in a file that is not in the webroot.  Either way, you
need to make sure it is not stored in a publically accessible area.

Teddy

On 9/25/06, Ray Champagne <[EMAIL PROTECTED]> wrote:
>
> So, first time I've ever ran into the need to encrypt data in my DB, and I
> already have a question.  When using the Encrypt function in CF, one must
> supply a key, I'm using the GenerateSecretKey function to get said
> key.  My
> question is, once I've stored the encrypted field in the database, where
> do
> I store the key so that I can use the decrypt function on the other side
> to
> retrieve the data?  Should generate a new key for every string, or use one
> that will work on the entire DB?  Never been down this road before, so any
> pointers would be helpful.
>
>
>
>
>
>
>
> 

~|
Introducing the Fusion Authority Quarterly Update. 80 pages of hard-hitting,
up-to-date ColdFusion information by your peers, delivered to your door four 
times a year.
http://www.fusionauthority.com/quarterly

Archive: 
http://www.houseoffusion.com/groups/CF-Talk/message.cfm/messageid:254119
Subscription: http://www.houseoffusion.com/groups/CF-Talk/subscribe.cfm
Unsubscribe: 
http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=11502.10531.4

Re: Best practices for CFMX 6.1 to CFMX 7 Standard Upgrade

[Robert Munn]

Well, you can't use archive and deploy, as far as I know. I've been thinking 
for some time of writing a version of archive and deploy for myself to use on 
Standard installs, but honestly since my company has Ent. I've never been 
sufficiently motivated to do it.  Do you have the motivation to write even a 
basic migration system? When I look at what it takes, I start to think that the 
Ent. license looks awfully cheap, no joke. 

AFAIK, you can upgrade to 7 from 6.1 without uninstalling, but in Enterprise 
you sacrifice certain (very nice) additions if you do that. I don't know if you 
can upgrade from 6.1 to 7 Standard. If that's an option, that's the first thing 
to check out. If you don't lose anything by upgrading, why not just upgrade 
from the existing install? If you stand to lose something and prefer to install 
a clean copy, pen and paper or screenshots of the CF Admin are the way I've 
done upgrades in the past. Just  make sure you check everything. 

I haven't seen anything specific from Macromedia that says if you are upgrading 
Standard (Pro) you should uninstall 6.x and install 7 fresh. That is the advice 
for Ent. if you want the extra goodies. I assume that means that you can 
upgrade without doing an uninstall, but I haven't done it myself. If you have a 
test machine to try it on, give it a go. Even if that means you have to install 
6.1 on a clean system and then try the upgrade, it is worth the time to try it 
in a test system before you do it live.

You can also make copies of the 6.1 config files as a backup. Most of the 
system settings are stored in XML files in one spot or another in the cf root. 

Maybe you can be the guy to write the unofficial 6.1 to 7 Pro upgrade guide. 
I'm sure plenty of others would benefit from it.

Rob 



>Rob,
>
>I'm installing Standard. Any advice on that?
>
>Rey...
>
>Robert Munn wrote:
>>

~|
Logware (www.logware.us): a new and convenient web-based time tracking 
application. Start tracking and documenting hours spent on a project or with a 
client with Logware today. Try it for free with a 15 day trial account.
http://www.houseoffusion.com/banners/view.cfm?bannerid=67

Message: http://www.houseoffusion.com/lists.cfm/link=i:4:210999
Archives: http://www.houseoffusion.com/cf_lists/threads.cfm/4
Subscription: http://www.houseoffusion.com/lists.cfm/link=s:4
Unsubscribe: http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=89.70.4
Donations & Support: http://www.houseoffusion.com/tiny.cfm/54

Re: Best practices for CFMX 6.1 to CFMX 7 Standard Upgrade

[Rey Bango]

Rob,

I'm installing Standard. Any advice on that?

Rey...

Robert Munn wrote:
> Are you upgrading Pro or Enterprise? If you are upgrading Enterprise, you can 
> use the Archive and Deploy features as you suggested. I did it with a couple 
> of systems, one of which had six separate server instances on it. Just go 
> into the CF Admin for each instance, export all of the settings (except for 
> the CFIDE mapping) into a car file, then uninstall CFMX 6.1. Make sure you 
> stop all services if you have multiple server instances running as services.
> 
> Install CFMX 7 Enterprise. Run the configuration wizard from the browser 
> after the initial install finishes. Create your server instances using the 
> server manager features in CFMX 7 Admin. Go into the CF Admin for each 
> instance and deploy the car file for that instance. That should be it.
> 
> Rob 
> 
> 
>>I need to upgrade a client from CFMX 6.1 to CFMX 7. Can someone point me 
>>to a good upgrade article or perhaps lend some insight into the best 
>>approach for doing the upgrade?
>>
>>I'm very interested in how to preserve the 6.1 settings and carry them 
>>over to CFMX 7 (car file perhaps). Should I have both CFMX 6.1 and CFMX 
>>7 installed and then just repoint IIS to CFMX 7?
>>
>>Any direction would be appreciated.
>>
>>Rey...
>>
>>
>>-- 
>>http://www.ReyBango.com
> 
> 
> 

~|
Logware (www.logware.us): a new and convenient web-based time tracking 
application. Start tracking and documenting hours spent on a project or with a 
client with Logware today. Try it for free with a 15 day trial account.
http://www.houseoffusion.com/banners/view.cfm?bannerid=67

Message: http://www.houseoffusion.com/lists.cfm/link=i:4:210995
Archives: http://www.houseoffusion.com/cf_lists/threads.cfm/4
Subscription: http://www.houseoffusion.com/lists.cfm/link=s:4
Unsubscribe: http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=89.70.4
Donations & Support: http://www.houseoffusion.com/tiny.cfm/54

RE: Best Practices - Large Data Load

[Dawson, Michael]

If you have any money look at DT/Studio from Embarcadero. 

-Original Message-
From: C. Hatton Humphrey [mailto:[EMAIL PROTECTED] 
Sent: Wednesday, April 13, 2005 11:06 AM
To: CF-Talk
Subject: Re: Best Practices - Large Data Load

> Good god don't do it in CF - you will kill it.  This should all be 
> done inside SQL Server - DTS / SP type thing.

This is something that has to be portable and web-launchable;  I haven't
done much direct DB - DB interaction so I'll fiddle around with the
syntax and see how it works out.

Hatton



~|
Logware (www.logware.us): a new and convenient web-based time tracking 
application. Start tracking and documenting hours spent on a project or with a 
client with Logware today. Try it for free with a 15 day trial account.
http://www.houseoffusion.com/banners/view.cfm?bannerid=67

Message: http://www.houseoffusion.com/lists.cfm/link=i:4:202692
Archives: http://www.houseoffusion.com/cf_lists/threads.cfm/4
Subscription: http://www.houseoffusion.com/lists.cfm/link=s:4
Unsubscribe: http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=89.70.4
Donations & Support: http://www.houseoffusion.com/tiny.cfm/54

Re: Best Practices - Large Data Load

[C. Hatton Humphrey]

> I second to motion to use DTS if possible (going from MS-to-MS tools it's a
> no brainer, but it also supports some others).  You can create a package,
> accept variable inputs, define all aspects of the transfer and the bastard
> is very FAST.

The situation is always going to be where both of the databases are
the same server... the DB to DB copy is the best option... I've never
written a DTS package so I'm not 100% certian what I'd need to do.

The nice thing is that this is a one-shot query that will only need to
be run one or two times total.

Hatton

~|
Find out how CFTicket can increase your company's customer support 
efficiency by 100%
http://www.houseoffusion.com/banners/view.cfm?bannerid=49

Message: http://www.houseoffusion.com/lists.cfm/link=i:4:202612
Archives: http://www.houseoffusion.com/cf_lists/threads.cfm/4
Subscription: http://www.houseoffusion.com/lists.cfm/link=s:4
Unsubscribe: http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=89.70.4
Donations & Support: http://www.houseoffusion.com/tiny.cfm/54

RE: Best Practices - Large Data Load

[Jim Davis]

> -Original Message-
> From: C. Hatton Humphrey [mailto:[EMAIL PROTECTED]
> Sent: Wednesday, April 13, 2005 12:06 PM
> To: CF-Talk
> Subject: Re: Best Practices - Large Data Load
> 
> > Good god don't do it in CF - you will kill it.  This should all be done
> > inside SQL Server - DTS / SP type thing.
> 
> This is something that has to be portable and web-launchable;  I
> haven't done much direct DB - DB interaction so I'll fiddle around
> with the syntax and see how it works out.

If possible I agree with others: try to eliminate as many moving parts as
you can.

But if you can't you could do this in CF - but it will always be slower and
less capable than direct DB-to-DB transfer.

I second to motion to use DTS if possible (going from MS-to-MS tools it's a
no brainer, but it also supports some others).  You can create a package,
accept variable inputs, define all aspects of the transfer and the bastard
is very FAST.

I use it all the time to schedule (within SQL Server)
production-to-development data backups (copying production data from my web
host to my local development box).

It works perfectly and I don't need to involve any other tools at all.

Jim Davis





~|
Logware (www.logware.us): a new and convenient web-based time tracking 
application. Start tracking and documenting hours spent on a project or with a 
client with Logware today. Try it for free with a 15 day trial account.
http://www.houseoffusion.com/banners/view.cfm?bannerid=67

Message: http://www.houseoffusion.com/lists.cfm/link=i:4:202594
Archives: http://www.houseoffusion.com/cf_lists/threads.cfm/4
Subscription: http://www.houseoffusion.com/lists.cfm/link=s:4
Unsubscribe: 
http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=11502.10531.4
Donations & Support: http://www.houseoffusion.com/tiny.cfm/54

RE: Best Practices - Large Data Load

[Robertson-Ravo, Neil (RX)]

Use DTS - you can pass in variables - and it will no doubt be faster than
using crappy dynamic SQL.







-Original Message-
From: C. Hatton Humphrey [mailto:[EMAIL PROTECTED] 
Sent: 13 April 2005 17:10
To: CF-Talk
Subject: Re: Best Practices - Large Data Load

Okay, I got it rewritten into workable SQL; now I just have to change
it to a dynamic query so I can pass in database names to a stored
procedure;

The inital CF based method was taking something in the order of 30
minutes to chug through; the cross-db method took a total of 44
seconds on my dev box... should take much less on the production
server.

I knew I was hammering with a wrench, I just needed a redirect to find
the right tool!

Thanks!

On 4/13/05, C. Hatton Humphrey <[EMAIL PROTECTED]> wrote:
> > Good god don't do it in CF - you will kill it.  This should all be done
> > inside SQL Server - DTS / SP type thing.
> 
> This is something that has to be portable and web-launchable;  I
> haven't done much direct DB - DB interaction so I'll fiddle around
> with the syntax and see how it works out.
> 
> Hatton
>



~|
Find out how CFTicket can increase your company's customer support 
efficiency by 100%
http://www.houseoffusion.com/banners/view.cfm?bannerid=49

Message: http://www.houseoffusion.com/lists.cfm/link=i:4:202588
Archives: http://www.houseoffusion.com/cf_lists/threads.cfm/4
Subscription: http://www.houseoffusion.com/lists.cfm/link=s:4
Unsubscribe: http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=89.70.4
Donations & Support: http://www.houseoffusion.com/tiny.cfm/54

RE: Best Practices - Large Data Load

[Robertson-Ravo, Neil (RX)]

DTS and SP are Web launchable..



-Original Message-
From: C. Hatton Humphrey [mailto:[EMAIL PROTECTED] 
Sent: 13 April 2005 17:06
To: CF-Talk
Subject: Re: Best Practices - Large Data Load

> Good god don't do it in CF - you will kill it.  This should all be done
> inside SQL Server - DTS / SP type thing.

This is something that has to be portable and web-launchable;  I
haven't done much direct DB - DB interaction so I'll fiddle around
with the syntax and see how it works out.

Hatton



~|
Find out how CFTicket can increase your company's customer support 
efficiency by 100%
http://www.houseoffusion.com/banners/view.cfm?bannerid=49

Message: http://www.houseoffusion.com/lists.cfm/link=i:4:202585
Archives: http://www.houseoffusion.com/cf_lists/threads.cfm/4
Subscription: http://www.houseoffusion.com/lists.cfm/link=s:4
Unsubscribe: http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=89.70.4
Donations & Support: http://www.houseoffusion.com/tiny.cfm/54

Re: Best Practices - Large Data Load

[C. Hatton Humphrey]

Okay, I got it rewritten into workable SQL; now I just have to change
it to a dynamic query so I can pass in database names to a stored
procedure;

The inital CF based method was taking something in the order of 30
minutes to chug through; the cross-db method took a total of 44
seconds on my dev box... should take much less on the production
server.

I knew I was hammering with a wrench, I just needed a redirect to find
the right tool!

Thanks!

On 4/13/05, C. Hatton Humphrey <[EMAIL PROTECTED]> wrote:
> > Good god don't do it in CF - you will kill it.  This should all be done
> > inside SQL Server - DTS / SP type thing.
> 
> This is something that has to be portable and web-launchable;  I
> haven't done much direct DB - DB interaction so I'll fiddle around
> with the syntax and see how it works out.
> 
> Hatton
>

~|
Find out how CFTicket can increase your company's customer support 
efficiency by 100%
http://www.houseoffusion.com/banners/view.cfm?bannerid=49

Message: http://www.houseoffusion.com/lists.cfm/link=i:4:202583
Archives: http://www.houseoffusion.com/cf_lists/threads.cfm/4
Subscription: http://www.houseoffusion.com/lists.cfm/link=s:4
Unsubscribe: 
http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=11502.10531.4
Donations & Support: http://www.houseoffusion.com/tiny.cfm/54

Re: Best Practices - Large Data Load

[C. Hatton Humphrey]

> Good god don't do it in CF - you will kill it.  This should all be done
> inside SQL Server - DTS / SP type thing.

This is something that has to be portable and web-launchable;  I
haven't done much direct DB - DB interaction so I'll fiddle around
with the syntax and see how it works out.

Hatton

~|
Find out how CFTicket can increase your company's customer support 
efficiency by 100%
http://www.houseoffusion.com/banners/view.cfm?bannerid=49

Message: http://www.houseoffusion.com/lists.cfm/link=i:4:202582
Archives: http://www.houseoffusion.com/cf_lists/threads.cfm/4
Subscription: http://www.houseoffusion.com/lists.cfm/link=s:4
Unsubscribe: 
http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=11502.10531.4
Donations & Support: http://www.houseoffusion.com/tiny.cfm/54

Re: Best Practices - Large Data Load

[Aaron Rouse]

Sometimes, in our situations at least, db to db connections are not allowed. 
For instance one of ours that does this type of "copying" of data the other 
side of the fense will only grant us ODBC access. 

 On 4/13/05, Adrocknaphobia <[EMAIL PROTECTED]> wrote: 
> 
> Yes, remove CF from the equation. Whats it doing in the middle that a
> db to db connection wouldn't handle?
> 
> -Adam
> 
>


~|
Logware (www.logware.us): a new and convenient web-based time tracking 
application. Start tracking and documenting hours spent on a project or with a 
client with Logware today. Try it for free with a 15 day trial account.
http://www.houseoffusion.com/banners/view.cfm?bannerid=67

Message: http://www.houseoffusion.com/lists.cfm/link=i:4:202577
Archives: http://www.houseoffusion.com/cf_lists/threads.cfm/4
Subscription: http://www.houseoffusion.com/lists.cfm/link=s:4
Unsubscribe: http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=89.70.4
Donations & Support: http://www.houseoffusion.com/tiny.cfm/54

Re: Best Practices - Large Data Load

[Aaron Rouse]

We have some interfaces here that take the same amount of records then just 
flat out loop over it and insert them. Takes for ever to run. I have been 
able to replace one so far with a dblink in Oracle so now I just run a 
stored proc and it inserts the data when selected.

On 4/13/05, C. Hatton Humphrey <[EMAIL PROTECTED]> wrote: 
> 
> I have a query that is returning roughly 60,000 records from one
> database that I need to insert into another database. Right now I'm
> taking the old query, storing it in a session and then having the
> system process the inserts 150 or so at a time and then submitting a
> form to itself.
> 
> Is there a better/faster way to do this? It's taking a *LONG* time
> for the process to run.
> 
> Hatton
> 
> 

~|
Find out how CFTicket can increase your company's customer support 
efficiency by 100%
http://www.houseoffusion.com/banners/view.cfm?bannerid=49

Message: http://www.houseoffusion.com/lists.cfm/link=i:4:202571
Archives: http://www.houseoffusion.com/cf_lists/threads.cfm/4
Subscription: http://www.houseoffusion.com/lists.cfm/link=s:4
Unsubscribe: http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=89.70.4
Donations & Support: http://www.houseoffusion.com/tiny.cfm/54

RE: Best Practices - Large Data Load

[Robertson-Ravo, Neil (RX)]

Good god don't do it in CF - you will kill it.  This should all be done
inside SQL Server - DTS / SP type thing.



-Original Message-
From: C. Hatton Humphrey [mailto:[EMAIL PROTECTED] 
Sent: 13 April 2005 16:42
To: CF-Talk
Subject: Best Practices - Large Data Load

I have a query that is returning roughly 60,000 records from one
database that I need to insert into another database.  Right now I'm
taking the old query, storing it in a session and then having the
system process the inserts 150 or so at a time and then submitting a
form to itself.

Is there a better/faster way to do this?  It's taking a *LONG* time
for the process to run.

Hatton



~|
Discover CFTicket - The leading ColdFusion Help Desk and Trouble 
Ticket application

http://www.houseoffusion.com/banners/view.cfm?bannerid=48

Message: http://www.houseoffusion.com/lists.cfm/link=i:4:202569
Archives: http://www.houseoffusion.com/cf_lists/threads.cfm/4
Subscription: http://www.houseoffusion.com/lists.cfm/link=s:4
Unsubscribe: http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=89.70.4
Donations & Support: http://www.houseoffusion.com/tiny.cfm/54

Re: Best Practices - Large Data Load

[Adrocknaphobia]

Yes, remove CF from the equation. Whats it doing in the middle that a
db to db connection wouldn't handle?

-Adam

On 4/13/05, C. Hatton Humphrey <[EMAIL PROTECTED]> wrote:
> I have a query that is returning roughly 60,000 records from one
> database that I need to insert into another database.  Right now I'm
> taking the old query, storing it in a session and then having the
> system process the inserts 150 or so at a time and then submitting a
> form to itself.
> 
> Is there a better/faster way to do this?  It's taking a *LONG* time
> for the process to run.
> 
> Hatton
> 
> 

~|
Discover CFTicket - The leading ColdFusion Help Desk and Trouble 
Ticket application

http://www.houseoffusion.com/banners/view.cfm?bannerid=48

Message: http://www.houseoffusion.com/lists.cfm/link=i:4:202568
Archives: http://www.houseoffusion.com/cf_lists/threads.cfm/4
Subscription: http://www.houseoffusion.com/lists.cfm/link=s:4
Unsubscribe: http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=89.70.4
Donations & Support: http://www.houseoffusion.com/tiny.cfm/54