make (and save) a hash of ip -> domain-name
and only do the dns lookup once per ip
On Thu, 15 Mar 2001 [EMAIL PROTECTED] wrote:
> Below I am sharing a script that will parse CLF Apache web.log files and
> place the data into Excel-friendly .csv files.
>
> I do have a question, however: the script takes a LONG time to run because
> of all of the DNS lookups. Is there any way to speed this up?
>
> Thanks.
>
> Ryan C. Christiansen
> Web Developer
> Intellisol International
>
>
> REBOL []
>
> log-file: read/lines ftp://username:[EMAIL PROTECTED]/logs/web.log
>
> first-line: parse log-file/1 none
> first-line/4: remove first-line/4 {[}
> first-line/5: remove first-line/5 {]}
> checksum-string: rejoin [first-line/4 " " first-line/5]
> checksum-date: make date! checksum-string
>
> csv-file-name: make file! (rejoin [checksum-date ".csv"])
> write csv-file-name {User IP Address, User Domain Address, Date Hit, Time
> Hit, File Hit, Bytes Transferred, Referring Page, Browser Type}
> write/append csv-file-name (newline newline)
>
> foreach log-line log-file [
>
> current-line: parse log-line none
>
> current-line/4: remove current-line/4 {[}
> current-line/5: remove current-line/5 {]}
> date-string: rejoin [current-line/4 " " current-line/5]
> hit-date: make date! date-string
>
> either not-equal? hit-date checksum-date [
> csv-file-name: make file! (rejoin [hit-date ".csv"])
> write csv-file-name {User IP Address, User Domain Address, Date
> Hit, Time Hit, File Hit, Bytes Transferred, Referring Page, Browser Type}
> write/append csv-file-name (newline newline)
>
> current-line: parse log-line none
> IP-address: make tuple! current-line/1
> domain-address: read join dns:// IP-address
> current-line/4: remove current-line/4 {[}
> current-line/5: remove current-line/5 {]}
> date-string: rejoin [current-line/4 " " current-line/5]
> hit-date: make date! date-string
> parse date-string [thru ":" copy text to end (hit-time: make time!
> text)]
> hit-file: current-line/6
> hit-bytes: current-line/8
> referring-page: make url! current-line/9
> browser-type: current-line/10
>
> write/append csv-file-name (rejoin [IP-address "," domain-address
> "," hit-date "," hit-time "," hit-file "," hit-bytes "," referring-page ","
> browser-type newline])
>
> checksum-date: hit-date
> ][
> current-line: parse log-line none
> IP-address: make tuple! current-line/1
> domain-address: read join dns:// IP-address
> current-line/4: remove current-line/4 {[}
> current-line/5: remove current-line/5 {]}
> date-string: rejoin [current-line/4 " " current-line/5]
> hit-date: make date! date-string
> parse date-string [thru ":" copy text to end (hit-time: make time!
> text)]
> hit-file: current-line/6
> hit-bytes: current-line/8
> referring-page: make url! current-line/9
> browser-type: current-line/10
>
> write/append csv-file-name (rejoin [IP-address "," domain-address
> "," hit-date "," hit-time "," hit-file "," hit-bytes "," referring-page ","
> browser-type newline])
>
> print (rejoin [IP-address "," domain-address "," hit-date ","
> hit-time "," hit-file "," hit-bytes "," referring-page "," browser-type
> newline])
>
> next log-file
> ]
> ]
>
> --
> To unsubscribe from this list, please send an email to
> [EMAIL PROTECTED] with "unsubscribe" in the
> subject, without the quotes.
>
--
To unsubscribe from this list, please send an email to
[EMAIL PROTECTED] with "unsubscribe" in the
subject, without the quotes.
