greg wm wrote:
i used wget to copy the entire http://nonviolentpeaceforce.org site to
http://nvpf.org/np. the former is in m$ asp, the latter captured as html.
for example, http://nonviolentpeaceforce.org/spanish/welcome.asp was
captured to http://nvpf.org/np/spanish/welcome.asp.html
as you can see, the capture is mostly fine, including spanish characters
in the text (eg año), however the spanish characters in the menus didn't
do quite so well (eg Misi?n)
fixed! see below.
in the file año appears as año which is apparently "good", but
Misi?n appears as Misión, which is apparently "bad".
first question: why is that bad?
if i tell galeon, instead of automatic encoding, use western iso-8859-1,
then, presto, the page appears nicely. but i don't have to do that to
see the original, nor do i have to do that for anybody else's pages, and
of course i can't expect our audience to go and fiddle with that in
their browsers.
second question: why doesn't the meta http-equiv header do anything?
right after the title the file says <meta http-equiv="Content-Type"
content="text/html; charset=iso-8859-1">. why isn't that good enough?
why does it make no difference at all what i change it to? i tried
utf-8, Utf-8, UTF-8, Windows-1252, none have any effect tho i can see
them if i tell my browser to view source.
overridden by apache's http headers, apparently. see below.
fourth question: can wget be tweaked to do better?
>
i think those menus were rendered out of some .asp database or
whatever, differently than the rest of the text of the page. but so
what? why didn't wget capture something identical to what my
browser shows?
the command i ran was
wget -ENKkrl19 -nH -w2 -owget.log http://nonviolentpeaceforce.org
my locale is en_IE.UTF-8, so why did wget save in latin-1 format?
the wget manual page mentions nothing at all about character sets.
well whatever, thunk i, no problem, i'll just find and replace. well
ha. i haven't yet managed to craft sed to capture the buggers! it's
all making me feel dang defeated..
Brian Foster wrote:
> there are other alternatives. e.g., convert the page
> (file) to UTF-8 (e.g., using iconv(1)), being sure to
> change the meta charset setting to utf-8.
>
> finally, vim(1): vim confuses things here (I am _not_
> trying to start an editor war!). vim guesses what the
> file's charset is, and adjusts accordingly so that you
> can view/edit it in a locale using a different charset.
> hence, a lot of things that cat displays as rubbish
> display Ok in vim. if, in vim, you do a “:set” command,
> you'll probably see an entry like “fileencoding=utf-8”.
> that means vim thinks the file is UTF-8. (probably,
> for the ISO-8859-1 files, it says “fileencoding=latin1”;
> Latin1 is a common informal(?) name for ISO-8859-1.)
> “:help fileencoding” for more information.
thank you brian! perhaps iconv might have done the trick, anyway i used
vim. vim :se fileencoding revealed that wget saved the files in
latin-1, and :se fileencoding=utf-8 for each file cleaned up the mess.
wasn't even a big job after using :map such that each file was fixed
with a single keystroke.
William A. Rowe, Jr. wrote:
> What happens if you remove the defaultcharset entirely; have Apache
> provide no hinting at the encoding; does the browser respect the meta
> tag?
>
> The http headers are authoritative, and override any metadata. If you
rather control your encoding with meta tags, turn off charsets entirely.
that is probably the winning answer. i already applied the above
solution so i dunno for sure, but look..
wget --save-headers from the original m$ .asp server:
HTTP/1.1 200 OK
Server: Microsoft-IIS/5.0
Date: Sat, 20 Aug 2005 21:18:55 GMT
Connection: keep-alive
Connection: Keep-Alive
Content-Length: 11003
Content-Type: text/html
Set-Cookie: ASPSESSIONIDAQQBRDRA=KNIPOCMDJPKMMANLNHFMKMGH; path=/
Cache-control: private
wget --save-headers from my apache server:
HTTP/1.1 200 OK
Date: Sun, 21 Aug 2005 04:10:34 GMT
Server: Apache/2.0.52 (CentOS)
Last-Modified: Sun, 21 Aug 2005 01:34:43 GMT
ETag: "260261-2b33-9134b2c0"
Accept-Ranges: bytes
Content-Length: 11059
Connection: close
Content-Type: text/html; charset=UTF-8
now i wouldn't have thought that the following httpd.conf directive
would result in overriding the meta http-equiv headers, but, there does
seem to be a strong odor..
# Specify a default charset for all pages sent out. This is
# always a good idea and opens the door for future internationalisation
# of your web site, should you ever want it. Specifying it as
# a default does little harm; as the standard dictates that a page
# is in iso-8859-1 (latin1) unless specified otherwise i.e. you
# are merely stating the obvious. There are also some security
# reasons in browsers, related to javascript and URL parsing
# which encourage you to always set a default char set.
#
AddDefaultCharset UTF-8
greg
Greg Whitley Mott
IT Coordinator
NonviolentPeaceforce.org
---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: [EMAIL PROTECTED]
" from the digest: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
