Re: Special characters
On Tue, 2007-08-21 at 14:21 -0400, Dan King wrote: > I am having issues running a web application, called OTRS, that uses > DBI and DBD::Oracle. When I insert special characters, such as é or â > they show up as question marks in the database when looking at them > from sqlplus or through the web application. I am running the web app > on Apache v1 with an Oracle 10g backend both on Solaris 9. The reason > I am asking the question here is I am wondering if mod perl does any > conversion of text that is typed into a text box. > It's not that mod_perl is converting the text. It's that your program is NOT converting it. You are using conflicting character sets. For instance, maybe data in your database is stored in UTF8, but on your website you're using ISO-8859-15. You need to decode all text coming into your Perl program, and encode it again on the way out. Have a look at this tutorial I wrote about using UTF8 with MySQL - may help. http://www.perlmonks.org/index.pl?node_id=620803 Clint
RE: Special characters
You've said that you've verified that the data is in the DB correctly from the
shell, but it's possible that it's trying to convert to another charset it when
you pull it out for some reason. There seems to be some stuff in the
DBD::Oracle documentation talking about charsets and unicode, i'd start by
looking there.
http://search.cpan.org/~pythian/DBD-Oracle-1.19/Oracle.pm#DBD::Oracle_and_Unicode
HTH,
Adam
From: Dan King [mailto:[EMAIL PROTECTED]
Sent: Tuesday, August 21, 2007 2:22 PM
To: modperl@perl.apache.org
Subject: Special characters
I am having issues running a web application, called OTRS, that uses DBI and
DBD::Oracle. When I insert special characters, such as é or â they show up as
question marks in the database when looking at them from sqlplus or through the
web application. I am running the web app on Apache v1 with an Oracle 10g
backend both on Solaris 9. The reason I am asking the question here is I am
wondering if mod perl does any conversion of text that is typed into a text
box.
I have been able to successfully insert and read the special characters using
sqlplus from the command line of the server running the web app, apache and
Oracle as well as creating a small perl test script that looks as follows:
#!/usr/local/bin/perl
use warnings;
use strict;
use Data::Dumper;
use DBI;
use DBD::Oracle;
my $dbh = DBI->connect("dbi:Oracle:host=localhost;sid=QAT", "rm_ticket",
"rm_ticket");
my $sql = qq{ UPDATE signature SET text='çè' WHERE id=1 };
my $sth = $dbh->prepare($sql);
$sth->execute();
$sth->finish();
$dbh->disconnect;
The reading script looks as follows:
#!/usr/local/bin/perl
use warnings;
use strict;
use Data::Dumper;
use DBI;
use DBD::Oracle;
my $dbh = DBI->connect("dbi:Oracle:host=localhost;sid=QAT", "rm_ticket",
"rm_ticket");
my $sql = qq{ SELECT text from signature };
my $sth = $dbh->prepare($sql);
$sth->execute();
my($text);
$sth->bind_columns(undef, \$text);
print "Text of signature:\n\n";
while( $sth->fetch() ) {
print "Object: $text\n";
}
$sth->finish();
$dbh->disconnect;
Does anyone know why I can successfully write and read from the database when I
am not using the web application but when I use the web application the special
characters fail to work properly?
Dan King
Software Developer
Canadian Resident Matching Service
613.237.0075 ext. 241
(Toll free) 877.CARMS.42
171 Nepean Street, Suite 300
Ottawa, ON, CANK2P 0B4
www.carms.ca
This e-mail message, including any attachments, is for the sole use of the
intended recipients and may contain confidential and or privileged information.
If you are not the intended recipient or this information has been forwarded
in error, please contact the sender by reply e-mail and destroy copies of the
original message. Ce message (incluant toute pièce jointe) s'adresse
uniquement au(x) destinataire(s) prévu(s) ou à une personne autorisée à le
recevoir en son (leur) nom. Il pourrait contenir des renseignements
confidentiels ou protégés. Si vous l'avez reçu par erreur, nous vous prions
d'en informer l'auteur dans les meilleurs délais, de ne pas divulguer son
contenu et de le supprimer de votre système. Merci.
RE: Special characters
I am using ISO-8859-1 in the database and the correct encoding is being used on the clients web browser. That is why I thought that it may be an apache or mod_perl issue. Thank you for the recommendations though and I will look into and experiment with all of them. Dan King Software Developer Canadian Resident Matching Service 613.237.0075 ext. 241 (Toll free) 877.CARMS.42 171 Nepean Street, Suite 300 Ottawa, ON, CANK2P 0B4 www.carms.ca This e-mail message, including any attachments, is for the sole use of the intended recipients and may contain confidential and or privileged information. If you are not the intended recipient or this information has been forwarded in error, please contact the sender by reply e-mail and destroy copies of the original message. Ce message (incluant toute pièce jointe) s'adresse uniquement au(x) destinataire(s) prévu(s) ou à une personne autorisée à le recevoir en son (leur) nom. Il pourrait contenir des renseignements confidentiels ou protégés. Si vous l'avez reçu par erreur, nous vous prions d'en informer l'auteur dans les meilleurs délais, de ne pas divulguer son contenu et de le supprimer de votre système. Merci. -Original Message- From: Clinton Gormley [mailto:[EMAIL PROTECTED] Sent: August 21, 2007 3:01 PM To: Dan King Cc: modperl@perl.apache.org Subject: Re: Special characters On Tue, 2007-08-21 at 14:21 -0400, Dan King wrote: > I am having issues running a web application, called OTRS, that uses > DBI and DBD::Oracle. When I insert special characters, such as é or â > they show up as question marks in the database when looking at them > from sqlplus or through the web application. I am running the web app > on Apache v1 with an Oracle 10g backend both on Solaris 9. The reason > I am asking the question here is I am wondering if mod perl does any > conversion of text that is typed into a text box. > It's not that mod_perl is converting the text. It's that your program is NOT converting it. You are using conflicting character sets. For instance, maybe data in your database is stored in UTF8, but on your website you're using ISO-8859-15. You need to decode all text coming into your Perl program, and encode it again on the way out. Have a look at this tutorial I wrote about using UTF8 with MySQL - may help. http://www.perlmonks.org/index.pl?node_id=620803 Clint
Re: Special characters
You need to know the encoding of the characters you are using in perl
and handle them accordingly.
The following link should have sufficient information to help you
trouble shoot this.
http://search.cpan.org/~jhi/perl-5.8.0/pod/perluniintro.pod
On 8/21/07, Dan King <[EMAIL PROTECTED]> wrote:
>
>
>
>
> I am having issues running a web application, called OTRS, that uses DBI and
> DBD::Oracle. When I insert special characters, such as é or â they show up
> as question marks in the database when looking at them from sqlplus or
> through the web application. I am running the web app on Apache v1 with an
> Oracle 10g backend both on Solaris 9. The reason I am asking the question
> here is I am wondering if mod perl does any conversion of text that is typed
> into a text box.
>
>
>
> I have been able to successfully insert and read the special characters
> using sqlplus from the command line of the server running the web app,
> apache and Oracle as well as creating a small perl test script that looks as
> follows:
>
>
>
>
>
>
>
> #!/usr/local/bin/perl
>
>
>
> use warnings;
>
>
>
> use strict;
>
>
>
> use Data::Dumper;
>
>
>
> use DBI;
>
>
>
> use DBD::Oracle;
>
>
>
>
>
>
>
> my $dbh = DBI->connect("dbi:Oracle:host=localhost;sid=QAT",
> "rm_ticket", "rm_ticket");
>
>
>
> my $sql = qq{ UPDATE signature SET text='çè' WHERE id=1 };
>
>
>
> my $sth = $dbh->prepare($sql);
>
>
>
> $sth->execute();
>
>
>
> $sth->finish();
>
>
>
> $dbh->disconnect;
>
>
>
>
>
>
>
> The reading script looks as follows:
>
>
>
>
>
>
>
> #!/usr/local/bin/perl
>
>
>
> use warnings;
>
>
>
> use strict;
>
>
>
> use Data::Dumper;
>
>
>
> use DBI;
>
>
>
> use DBD::Oracle;
>
>
>
>
>
>
>
> my $dbh = DBI->connect("dbi:Oracle:host=localhost;sid=QAT",
> "rm_ticket", "rm_ticket");
>
>
>
> my $sql = qq{ SELECT text from signature };
>
>
>
> my $sth = $dbh->prepare($sql);
>
>
>
> $sth->execute();
>
>
>
> my($text);
>
>
>
> $sth->bind_columns(undef, \$text);
>
>
>
> print "Text of signature:\n\n";
>
>
>
> while( $sth->fetch() ) {
>
>
>
> print "Object: $text\n";
>
>
>
> }
>
>
>
> $sth->finish();
>
>
>
> $dbh->disconnect;
>
>
>
>
>
>
>
> Does anyone know why I can successfully write and read from the database
> when I am not using the web application but when I use the web application
> the special characters fail to work properly?
>
>
>
>
>
> Dan King
> Software Developer
> Canadian Resident Matching Service
> 613.237.0075 ext. 241
> (Toll free) 877.CARMS.42
> 171 Nepean Street, Suite 300
> Ottawa, ON, CANK2P 0B4
> www.carms.ca
>
>
>
> This e-mail message, including any attachments, is for the sole use of the
> intended recipients and may contain confidential and or privileged
> information. If you are not the intended recipient or this information has
> been forwarded in error, please contact the sender by reply e-mail and
> destroy copies of the original message. Ce message (incluant toute pièce
> jointe) s'adresse uniquement au(x) destinataire(s) prévu(s) ou à une
> personne autorisée à le recevoir en son (leur) nom. Il pourrait contenir des
> renseignements confidentiels ou protégés. Si vous l'avez reçu par erreur,
> nous vous prions d'en informer l'auteur dans les meilleurs délais, de ne pas
> divulguer son contenu et de le supprimer de votre système. Merci.
>
>
RE: Special characters
Hi everyone,
Thanks for everyones help and suggestions. I found the problem with the special
characters. I guess mod_perl doesn't take the environment variables from the
server so you have to set them in the httpd.conf file. The code I used to fix
it is below:
PerlSetEnv NLS_LANG AMERICAN_AMERICA.WE8ISO8859P1 PerlPassEnv NLS_LANG
Once again thanks for your help.
Dan King
Software Developer
Canadian Resident Matching Service
613.237.0075 ext. 241
(Toll free) 877.CARMS.42
171 Nepean Street, Suite 300
Ottawa, ON, CANK2P 0B4
www.carms.ca
This e-mail message, including any attachments, is for the sole use of the
intended recipients and may contain confidential and or privileged information.
If you are not the intended recipient or this information has been forwarded
in error, please contact the sender by reply e-mail and destroy copies of the
original message. Ce message (incluant toute pièce jointe) s'adresse
uniquement au(x) destinataire(s) prévu(s) ou à une personne autorisée à le
recevoir en son (leur) nom. Il pourrait contenir des renseignements
confidentiels ou protégés. Si vous l'avez reçu par erreur, nous vous prions
d'en informer l'auteur dans les meilleurs délais, de ne pas divulguer son
contenu et de le supprimer de votre système. Merci.
-Original Message-
From: Manoj Bist [mailto:[EMAIL PROTECTED]
Sent: August 21, 2007 3:16 PM
To: Dan King
Cc: modperl@perl.apache.org
Subject: Re: Special characters
You need to know the encoding of the characters you are using in perl
and handle them accordingly.
The following link should have sufficient information to help you
trouble shoot this.
http://search.cpan.org/~jhi/perl-5.8.0/pod/perluniintro.pod
On 8/21/07, Dan King <[EMAIL PROTECTED]> wrote:
>
>
>
>
> I am having issues running a web application, called OTRS, that uses DBI and
> DBD::Oracle. When I insert special characters, such as é or â they show up
> as question marks in the database when looking at them from sqlplus or
> through the web application. I am running the web app on Apache v1 with an
> Oracle 10g backend both on Solaris 9. The reason I am asking the question
> here is I am wondering if mod perl does any conversion of text that is typed
> into a text box.
>
>
>
> I have been able to successfully insert and read the special characters
> using sqlplus from the command line of the server running the web app,
> apache and Oracle as well as creating a small perl test script that looks as
> follows:
>
>
>
>
>
>
>
> #!/usr/local/bin/perl
>
>
>
> use warnings;
>
>
>
> use strict;
>
>
>
> use Data::Dumper;
>
>
>
> use DBI;
>
>
>
> use DBD::Oracle;
>
>
>
>
>
>
>
> my $dbh = DBI->connect("dbi:Oracle:host=localhost;sid=QAT",
> "rm_ticket", "rm_ticket");
>
>
>
> my $sql = qq{ UPDATE signature SET text='çè' WHERE id=1 };
>
>
>
> my $sth = $dbh->prepare($sql);
>
>
>
> $sth->execute();
>
>
>
> $sth->finish();
>
>
>
> $dbh->disconnect;
>
>
>
>
>
>
>
> The reading script looks as follows:
>
>
>
>
>
>
>
> #!/usr/local/bin/perl
>
>
>
> use warnings;
>
>
>
> use strict;
>
>
>
> use Data::Dumper;
>
>
>
> use DBI;
>
>
>
> use DBD::Oracle;
>
>
>
>
>
>
>
> my $dbh = DBI->connect("dbi:Oracle:host=localhost;sid=QAT",
> "rm_ticket", "rm_ticket");
>
>
>
> my $sql = qq{ SELECT text from signature };
>
>
>
> my $sth = $dbh->prepare($sql);
>
>
>
> $sth->execute();
>
>
>
> my($text);
>
>
>
> $sth->bind_columns(undef, \$text);
>
>
>
> print "Text of signature:\n\n";
>
>
>
> while( $sth->fetch() ) {
>
>
>
> print "Object: $text\n";
>
>
>
> }
>
>
>
> $sth->finish();
>
>
>
> $dbh->disconnect;
>
>
>
>
>
>
>
> Does anyone know why I can successfully write and read from the database
> when I am not using the web application but when I use the web application
> the special characters fail to work properly?
>
>
>
>
>
> Dan King
> Software Developer
> Canadian Resident Matching Service
> 613.237.0075 ext. 241
> (Toll free) 877.CARMS.42
> 171 Nepean Street, Suite 300
> Ottawa, ON, CANK2P 0B4
> www.carms.ca
>
>
>
> This e-mail message, including any attachments, is for the sole use of the
> intended recipients and may contain confidential and or privileged
> information. If you are not the intended recipient or this information has
> been forwarded in error, please contact the sender by reply e-mail and
> destroy copies of the original message. Ce message (incluant toute pièce
> jointe) s'adresse uniquement au(x) destinataire(s) prévu(s) ou à une
> personne autorisée à le recevoir en son (leur) nom. Il pourrait contenir des
> renseignements confidentiels ou protégés. Si vous l'avez reçu par erreur,
> nous vous prions d'en informer l'auteur dans les meilleurs délais, de ne pas
> divulguer son contenu et de le supprimer de votre système. Merci.
>
>
Re: Special characters
Dan King wrote: > I guess mod_perl doesn't take the environment variables from the server so > you have to set them in the httpd.conf file. The code I used to fix it is > below: That's not exactly true. From the mod_perl docs: However, Apache (or mod_perl) don't pass on environment variables from the shell by default; you'll have to specify these using either the standard PassEnv or mod_perl's PerlPassEnv directives. It's not that they can't, it's that by default they don't pass anything you don't tell it to. And it's not mod_perl, it's Apache. > PerlSetEnv NLS_LANG AMERICAN_AMERICA.WE8ISO8859P1 PerlPassEnv NLS_LANG If you're setting NLS_LANG correctly in your environment, you can simply make that PerlPassEnv NLS_LANG Or if you want to make it available to anything else that might need it PassEnv NLS_LANG -- Michael Peters Developer Plus Three, LP
