php-general Digest 21 May 2010 17:16:06 -0000 Issue 6756
Topics (messages 305374 through 305400):
Re: PDO don't know which is mysql server charset?
305374 by: cleverpig
305376 by: Lester Caine
305377 by: cleverpig
305378 by: Lester Caine
305379 by: Peter Lind
305380 by: Lester Caine
305381 by: Peter Lind
Re: Brandon Rampersad wants to chat
305375 by: Jochen Schultz
how to update array keys and keep element order ?
305382 by: cr.vegelin.gmail.com
305383 by: Richard Quadling
305385 by: cr.vegelin.gmail.com
305391 by: Al
305392 by: Richard Quadling
Remove blank lines from a file
305384 by: Anton Heuschen
305387 by: Ashley Sheridan
305389 by: Al
305396 by: Anton Heuschen
305399 by: Bob McConnell
305400 by: Gary
Re: How to get input from socket client
305386 by: Bob McConnell
Re: Question about a security function
305388 by: David Otton
305390 by: Ashley Sheridan
305393 by: Al
305394 by: David Otton
305395 by: Al
305397 by: Jim Lucas
305398 by: Al
Administrivia:
To subscribe to the digest, e-mail:
[email protected]
To unsubscribe from the digest, e-mail:
[email protected]
To post to the list, e-mail:
[email protected]
----------------------------------------------------------------------
--- Begin Message ---
But if I use PDO's query function,there is all right without set charset:
$result=$myPDO->query("SELECT * FROM guests ORDER BY lname,fname");
while($row=$result->fetch(PDO::FETCH_ASSOC)){
echo $row["fname"]." ". $row["lname"];
}
2010/5/21 cleverpig <[email protected]>:
> hi, everybody!
>
> I encountered a problem using the PDO:
> My mysql charset is utf-8, when using PDO's prepare () and statement's
> execute() methods,
> there is not the correct encoding resultset:
> $ Statement = $ pdo-> prepare ("SELECT * FROM guests ORDER BY?");
> $ Statement-> execute (array ('lname'));
> while ($ row = $ statement-> fetch (PDO:: FETCH_ASSOC)) (
> echo $ row ['fname']. "". $ row ['lname'];
> )
>
> But when I was make charset_set_results to be gbk, everything was normal:
> $ Pdo-> exec ('SET character_set_results = gbk');
>
> If PDO don't know which charset is fit to me,it will use default
> charset(utf-8)?This is really mysql server charset!
>
> Anybody help me?Thanks!
>
> PS: I am using wamp (php5.3.1)
>
> --
> cleverpig(Dan)
> Location: Beijing
> Address: Room 4018,No.A2 South Avenue Fuxingmen Beijing,P.R.China
> Zipcode: 100031
> MSN: [email protected]
> QQ: 149291732
> Skype: cleverpigatmatrix
> Facebook ID:cleverpig
> Blog: cleverpig.name/dan/
> Tags: del.icio.us/cleverpig
> Twitter: twitter.com/cleverpig
> 新浪微博: t.sina.com.cn/cleverpig
> Organization: www.beijing-open-party.org
> or...@facebook: http://www.facebook.com/group.php?gid=8159558294
>
--
cleverpig(Dan)
Location: Beijing
Address: Room 4018,No.A2 South Avenue Fuxingmen Beijing,P.R.China
Zipcode: 100031
MSN: [email protected]
QQ: 149291732
Skype: cleverpigatmatrix
Facebook ID:cleverpig
Blog: cleverpig.name/dan/
Tags: del.icio.us/cleverpig
Twitter: twitter.com/cleverpig
新浪微博: t.sina.com.cn/cleverpig
Organization: www.beijing-open-party.org
or...@facebook: http://www.facebook.com/group.php?gid=8159558294
--- End Message ---
--- Begin Message ---
cleverpig wrote:
> But if I use PDO's query function,there is all right without set charset:
> $result=$myPDO->query("SELECT * FROM guests ORDER BY lname,fname");
> while($row=$result->fetch(PDO::FETCH_ASSOC)){
> echo $row["fname"]." ". $row["lname"];
> }
>
>> I encountered a problem using the PDO:
>> My mysql charset is utf-8, when using PDO's prepare () and statement's
>> execute() methods,
>> there is not the correct encoding resultset:
>> $ Statement = $ pdo-> prepare ("SELECT * FROM guests ORDER BY?");
>> $ Statement-> execute (array ('lname'));
>> while ($ row = $ statement-> fetch (PDO:: FETCH_ASSOC)) (
>> echo $ row ['fname']. "". $ row ['lname'];
>> )
>>
>> But when I was make charset_set_results to be gbk, everything was normal:
>> $ Pdo-> exec ('SET character_set_results = gbk');
>>
>> If PDO don't know which charset is fit to me,it will use default
>> charset(utf-8)?This is really mysql server charset!
The tikiwiki project has just picked up on this problem, and it may be that you
are missing a key piece of information! What THEY are finding is that while the
data is being displayed correctly, what is stored in the database is ACTUALLY
'double encoded'. So as long as you 'double decode' it things are OK, but the
data stored in MySQL is actually corrupt and unreadable by other applications!
What they are currently trying to work out is how to process damaged databases
in addition to sorting out how the connection should be set up.
http://dev.tikiwiki.org/UTF-8+Testcase has more information ....
--
Lester Caine - G8HFL
-----------------------------
Contact - http://lsces.co.uk/wiki/?page=contact
L.S.Caine Electronic Services - http://lsces.co.uk
EnquirySolve - http://enquirysolve.com/
Model Engineers Digital Workshop - http://medw.co.uk//
Firebird - http://www.firebirdsql.org/index.php
--- End Message ---
--- Begin Message ---
thanks Lester!
What's double decode? Your meaning is like What's i said:
using $ Pdo-> exec ('SET character_set_results = gbk');
to make charset_set_results to be gbk?
2010/5/21 Lester Caine <[email protected]>:
> cleverpig wrote:
>> But if I use PDO's query function,there is all right without set charset:
>> $result=$myPDO->query("SELECT * FROM guests ORDER BY lname,fname");
>> while($row=$result->fetch(PDO::FETCH_ASSOC)){
>> echo $row["fname"]." ". $row["lname"];
>> }
>>
>>> I encountered a problem using the PDO:
>>> My mysql charset is utf-8, when using PDO's prepare () and statement's
>>> execute() methods,
>>> there is not the correct encoding resultset:
>>> $ Statement = $ pdo-> prepare ("SELECT * FROM guests ORDER BY?");
>>> $ Statement-> execute (array ('lname'));
>>> while ($ row = $ statement-> fetch (PDO:: FETCH_ASSOC)) (
>>> echo $ row ['fname']. "". $ row ['lname'];
>>> )
>>>
>>> But when I was make charset_set_results to be gbk, everything was normal:
>>> $ Pdo-> exec ('SET character_set_results = gbk');
>>>
>>> If PDO don't know which charset is fit to me,it will use default
>>> charset(utf-8)?This is really mysql server charset!
>
> The tikiwiki project has just picked up on this problem, and it may be that
> you
> are missing a key piece of information! What THEY are finding is that while
> the
> data is being displayed correctly, what is stored in the database is ACTUALLY
> 'double encoded'. So as long as you 'double decode' it things are OK, but the
> data stored in MySQL is actually corrupt and unreadable by other applications!
>
> What they are currently trying to work out is how to process damaged databases
> in addition to sorting out how the connection should be set up.
> http://dev.tikiwiki.org/UTF-8+Testcase has more information ....
>
> --
> Lester Caine - G8HFL
> -----------------------------
> Contact - http://lsces.co.uk/wiki/?page=contact
> L.S.Caine Electronic Services - http://lsces.co.uk
> EnquirySolve - http://enquirysolve.com/
> Model Engineers Digital Workshop - http://medw.co.uk//
> Firebird - http://www.firebirdsql.org/index.php
>
> --
> PHP General Mailing List (http://www.php.net/)
> To unsubscribe, visit: http://www.php.net/unsub.php
>
>
--
cleverpig(Dan)
Location: Beijing
Address: Room 4018,No.A2 South Avenue Fuxingmen Beijing,P.R.China
Zipcode: 100031
MSN: [email protected]
QQ: 149291732
Skype: cleverpigatmatrix
Facebook ID:cleverpig
Blog: cleverpig.name/dan/
Tags: del.icio.us/cleverpig
Twitter: twitter.com/cleverpig
新浪微博: t.sina.com.cn/cleverpig
Organization: www.beijing-open-party.org
or...@facebook: http://www.facebook.com/group.php?gid=8159558294
--- End Message ---
--- Begin Message ---
cleverpig wrote:
thanks Lester!
What's double decode? Your meaning is like What's i said:
using $ Pdo-> exec ('SET character_set_results = gbk');
to make charset_set_results to be gbk?
http://tikiwiki.org/UTF-8 seems to have documented everything ;)
Basically if the data going into a field is wrong, as long as it is read out the
right way then it will LOOK OK, but if you don't know how the data is stored
then 'decoding' it may be difficult. I'm not sure quite how PDO will handle your
gbk data, although in that case it would be single byte and any multi byte
characters would be dropped ... I think.
The problem here is that PHP still does not know how to handle UTF8 properly, so
if a multibyte character string is provided and is stored in what is essentially
a single byte per character field in the database, as long as all the extra
bytes are saved, reading it back will be OK ... unless the bytes are then
processed to another character set before saving! THEN as long as the
'unprocessing' when you read back is transparent, the data will read OK, but
trying to read the data directly in the database would not give the same result.
Your request to read back as gbk will only work if what was stored can actually
be correctly converted to gbk!
--
Lester Caine - G8HFL
-----------------------------
Contact - http://lsces.co.uk/wiki/?page=contact
L.S.Caine Electronic Services - http://lsces.co.uk
EnquirySolve - http://enquirysolve.com/
Model Engineers Digital Workshop - http://medw.co.uk//
Firebird - http://www.firebirdsql.org/index.php
--- End Message ---
--- Begin Message ---
On 21 May 2010 10:16, Lester Caine <[email protected]> wrote:
> cleverpig wrote:
>>
>> thanks Lester!
>>
>> What's double decode? Your meaning is like What's i said:
>> using $ Pdo-> exec ('SET character_set_results = gbk');
>> to make charset_set_results to be gbk?
>
> http://tikiwiki.org/UTF-8 seems to have documented everything ;)
>
> Basically if the data going into a field is wrong, as long as it is read out
> the right way then it will LOOK OK, but if you don't know how the data is
> stored then 'decoding' it may be difficult. I'm not sure quite how PDO will
> handle your gbk data, although in that case it would be single byte and any
> multi byte characters would be dropped ... I think.
>
> The problem here is that PHP still does not know how to handle UTF8
> properly
It's not *just* that PHP isn't handling utf8 perfectly. Encoding
between database and server is a rather complex issue, you're dealing
with:
* database encoding
* database connection encoding
* php internal encoding
* output encoding
Messing up just *one* of these will give bad output - so you need to
make sure that all of them are aligned.
Regards
Peter
--
<hype>
WWW: http://plphp.dk / http://plind.dk
LinkedIn: http://www.linkedin.com/in/plind
Flickr: http://www.flickr.com/photos/fake51
BeWelcome: Fake51
Couchsurfing: Fake51
</hype>
--- End Message ---
--- Begin Message ---
Peter Lind wrote:
The problem here is that PHP still does not know how to handle UTF8
> properly
It's not*just* that PHP isn't handling utf8 perfectly. Encoding
between database and server is a rather complex issue, you're dealing
with:
* database encoding
* database connection encoding
* php internal encoding
* output encoding
Messing up just *one* of these will give bad output - so you need to
make sure that all of them are aligned.
And not simply assume that because everything is SET to UTF8 - that is actually
what is being handled :(
The holes in the process such as with MySQL/PDO need to be well documented,
which is what tikiwiki are working through now, but should be in the PHP
documentation?
--
Lester Caine - G8HFL
-----------------------------
Contact - http://lsces.co.uk/wiki/?page=contact
L.S.Caine Electronic Services - http://lsces.co.uk
EnquirySolve - http://enquirysolve.com/
Model Engineers Digital Workshop - http://medw.co.uk//
Firebird - http://www.firebirdsql.org/index.php
--- End Message ---
--- Begin Message ---
On 21 May 2010 10:47, Lester Caine <[email protected]> wrote:
> Peter Lind wrote:
>>>
>>> The problem here is that PHP still does not know how to handle UTF8
>>> > properly
>
>> It's not*just* that PHP isn't handling utf8 perfectly. Encoding
>> between database and server is a rather complex issue, you're dealing
>> with:
>> * database encoding
>> * database connection encoding
>> * php internal encoding
>> * output encoding
>>
>> Messing up just *one* of these will give bad output - so you need to
>> make sure that all of them are aligned.
>
> And not simply assume that because everything is SET to UTF8 - that is
> actually what is being handled :(
> The holes in the process such as with MySQL/PDO need to be well documented,
> which is what tikiwiki are working through now, but should be in the PHP
> documentation?
>
A good reference in the PHP docs would be valuable - otherwise you're
searching wikis and blogs for the answer.
Regards
Peter
--
<hype>
WWW: http://plphp.dk / http://plind.dk
LinkedIn: http://www.linkedin.com/in/plind
Flickr: http://www.flickr.com/photos/fake51
BeWelcome: Fake51
Couchsurfing: Fake51
</hype>
--- End Message ---
--- Begin Message ---
You are inviting the whole general list at once?
"Powerful spam protection" doesn't work for outgoing mails?
Brandon Rampersad schrieb:
-----------------------------------------------------------------------
Brandon Rampersad wants to stay in better touch using some of Google's
coolest new
products.
If you already have Gmail or Google Talk, visit:
http://mail.google.com/mail/b-b89a7a894d-04fb7fdbad-NeHRsydO6kBsIkVOT8hXbjPA1ZU
You'll need to click this link to be able to chat with Brandon Rampersad.
To get Gmail - a free email account from Google with over 2,800 megabytes of
storage - and chat with Brandon Rampersad, visit:
http://mail.google.com/mail/a-b89a7a894d-04fb7fdbad-NeHRsydO6kBsIkVOT8hXbjPA1ZU
Gmail offers:
- Instant messaging right inside Gmail
- Powerful spam protection
- Built-in search for finding your messages and a helpful way of organizing
emails into "conversations"
- No pop-up ads or untargeted banners - just text ads and related information
that are relevant to the content of your messages
All this, and its yours for free. But wait, there's more! By opening a Gmail
account, you also get access to Google Talk, Google's instant messaging
service:
http://www.google.com/talk/
Google Talk offers:
- Web-based chat that you can use anywhere, without a download
- A contact list that's synchronized with your Gmail account
- Free, high quality PC-to-PC voice calls when you download the Google Talk
client
We're working hard to add new features and make improvements, so we might also
ask for your comments and suggestions periodically. We appreciate your help in
making our products even better!
Thanks,
The Google Team
To learn more about Gmail and Google Talk, visit:
http://mail.google.com/mail/help/about.html
http://www.google.com/talk/about.html
(If clicking the URLs in this message does not work, copy and paste them into
the address bar of your browser).
--
Sport Import GmbH - Amtsgericht Oldenburg - Tel: +49-4405-9280-63
Industriestrasse 39 - HRB 1202900 -
26188 Edewecht - GF: Michael Müllmann
--- End Message ---
--- Begin Message ---
How do I update an array key without disturbing the element order ?
Suppose an existing array("FR", values ...)
where I want to change 0 => "FR" to "country" => "FR"
and keep the original element order.
TIA, Cor
--- End Message ---
--- Begin Message ---
On 21 May 2010 10:56, <[email protected]> wrote:
> How do I update an array key without disturbing the element order ?
> Suppose an existing array("FR", values ...)
> where I want to change 0 => "FR" to "country" => "FR"
> and keep the original element order.
>
> TIA, Cor
>
Something like ...
function arrayKeyChange(array &$array, $oldKey, $newKey) {
if (!array_key_exists($oldKey, $array)) {
trigger_error('Requested key does not exist.', E_USER_WARNING);
return False;
}
if (array_key_exists($oldKey, $array) && array_key_exists($newKey,
$array)) {
trigger_error('Will result in duplicate keys.', E_USER_WARNING);
return False;
}
// Split keys/values
$keys = array_keys($array);
$values = array_values($array);
// Find position of oldKey
$oldKeyPos = array_search($oldKey, $keys, True);
// Replace key
$keys[$oldKeyPos] = $newKey;
// Combine keys/values
$array = array_combine($keys, $values);
return True;
}
--
-----
Richard Quadling
"Standing on the shoulders of some very clever giants!"
EE : http://www.experts-exchange.com/M_248814.html
EE4Free : http://www.experts-exchange.com/becomeAnExpert.jsp
Zend Certified Engineer : http://zend.com/zce.php?c=ZEND002498&r=213474731
ZOPA : http://uk.zopa.com/member/RQuadling
--- End Message ---
--- Begin Message ---
----- Original Message -----
From: "Richard Quadling" <[email protected]>
To: <[email protected]>
Cc: <[email protected]>
Sent: Friday, May 21, 2010 1:30 PM
Subject: Re: [PHP] how to update array keys and keep element order ?
On 21 May 2010 10:56, <[email protected]> wrote:
How do I update an array key without disturbing the element order ?
Suppose an existing array("FR", values ...)
where I want to change 0 => "FR" to "country" => "FR"
and keep the original element order.
TIA, Cor
Something like ...
function arrayKeyChange(array &$array, $oldKey, $newKey) {
if (!array_key_exists($oldKey, $array)) {
trigger_error('Requested key does not exist.', E_USER_WARNING);
return False;
}
if (array_key_exists($oldKey, $array) && array_key_exists($newKey,
$array)) {
trigger_error('Will result in duplicate keys.', E_USER_WARNING);
return False;
}
// Split keys/values
$keys = array_keys($array);
$values = array_values($array);
// Find position of oldKey
$oldKeyPos = array_search($oldKey, $keys, True);
// Replace key
$keys[$oldKeyPos] = $newKey;
// Combine keys/values
$array = array_combine($keys, $values);
return True;
}
Thanks Richard,
I will check your solution. I tried the following:
function ar_replacekeys ($array, $replacements)
{
foreach ($array as $key => $value) // 1)
{
foreach ($replacements as $oldkey => $newkey)
{
if ($key == $oldkey)
{
$array[$newkey] = $value; // 2)
unset($array[$key]); // 2)
}
}
}
}
Works fine, except that replaced elements are placed at the end. (2)
Also tested: foreach ($array as &$key => $value) (1)
but this gave "Key element cannot be a reference"
--- End Message ---
--- Begin Message ---
On 5/21/2010 5:56 AM, [email protected] wrote:
How do I update an array key without disturbing the element order ?
Suppose an existing array("FR", values ...)
where I want to change 0 => "FR" to "country" => "FR"
and keep the original element order.
TIA, Cor
I short on time; but, it seems array_combine() or array_fill_keys() would do it
for you.
--- End Message ---
--- Begin Message ---
On 21 May 2010 13:34, <[email protected]> wrote:
>
> ----- Original Message ----- From: "Richard Quadling"
> <[email protected]>
> To: <[email protected]>
> Cc: <[email protected]>
> Sent: Friday, May 21, 2010 1:30 PM
> Subject: Re: [PHP] how to update array keys and keep element order ?
>
>
>> On 21 May 2010 10:56, <[email protected]> wrote:
>>>
>>> How do I update an array key without disturbing the element order ?
>>> Suppose an existing array("FR", values ...)
>>> where I want to change 0 => "FR" to "country" => "FR"
>>> and keep the original element order.
>>>
>>> TIA, Cor
>>>
>>
>> Something like ...
>>
>> function arrayKeyChange(array &$array, $oldKey, $newKey) {
>> if (!array_key_exists($oldKey, $array)) {
>> trigger_error('Requested key does not exist.', E_USER_WARNING);
>> return False;
>> }
>> if (array_key_exists($oldKey, $array) && array_key_exists($newKey,
>> $array)) {
>> trigger_error('Will result in duplicate keys.', E_USER_WARNING);
>> return False;
>> }
>>
>> // Split keys/values
>> $keys = array_keys($array);
>> $values = array_values($array);
>>
>> // Find position of oldKey
>> $oldKeyPos = array_search($oldKey, $keys, True);
>>
>> // Replace key
>> $keys[$oldKeyPos] = $newKey;
>>
>> // Combine keys/values
>> $array = array_combine($keys, $values);
>>
>> return True;
>> }
>
> Thanks Richard,
>
> I will check your solution. I tried the following:
>
> function ar_replacekeys ($array, $replacements)
> {
> foreach ($array as $key => $value) // 1)
> {
> foreach ($replacements as $oldkey => $newkey)
> {
> if ($key == $oldkey)
> {
> $array[$newkey] = $value; // 2)
> unset($array[$key]); // 2)
> }
> }
> }
> }
>
> Works fine, except that replaced elements are placed at the end. (2)
> Also tested: foreach ($array as &$key => $value) (1)
> but this gave "Key element cannot be a reference"
>
>
>
>
<?php
function arrayKeyChange(...){...} // As previously supplied.
$arr = array('Continent'=>'Europe', 'FR', 'Currency'=>'Euro');
arrayKeyChange($arr, 0, 'country');
arrayKeyChange($arr, 'country', 'Country');
print_r($arr);
?>
outputs ...
Array
(
[Continent] => Europe
[Country] => FR
[Currency] => Euro
)
--
-----
Richard Quadling
"Standing on the shoulders of some very clever giants!"
EE : http://www.experts-exchange.com/M_248814.html
EE4Free : http://www.experts-exchange.com/becomeAnExpert.jsp
Zend Certified Engineer : http://zend.com/zce.php?c=ZEND002498&r=213474731
ZOPA : http://uk.zopa.com/member/RQuadling
--- End Message ---
--- Begin Message ---
Hi Im trying do something like this, have a function which uploads my
file and returns file pointer ... but at same time ... I want to
remove all Blank lines in a file and update it before it goes to the
final location ...
What I tried was to do a write of file and use some regexp replace to
remove a blank ... either I am not doing the replace correct or my
understanding of the file buffer and what I can do with it between the
browser and saving is not correct,
Anyway my code looks something like this :
$uploadfile = $this->uploaddir;
$mtran = mt_rand(999,999999);
$NewName = date("Ymd_Gis").$mtran.".csv";
$uploadfile = $uploadfile.$NewName;
try{
if
(move_uploaded_file($_FILES['attachfile']['tmp_name'], $uploadfile))
{
$handle = fopen($uploadfile, "r+");
$lines = file($uploadfile,
FILE_SKIP_EMPTY_LINES); //FILE_IGNORE_NEW_LINES |
foreach ($lines as $line_num => $line) {
$line =
preg_replace("/(^[\r\n]*|[\r\n]+)[\s\t]*[\r\n]+/", "", $line);
if(strlen($line) > 0)
$line=trim($line);
$line=$line."\n";
fwrite($handle, $line);
}
fclose($handle);
--- End Message ---
--- Begin Message ---
On Fri, 2010-05-21 at 14:03 +0200, Anton Heuschen wrote:
> Hi Im trying do something like this, have a function which uploads my
> file and returns file pointer ... but at same time ... I want to
> remove all Blank lines in a file and update it before it goes to the
> final location ...
>
> What I tried was to do a write of file and use some regexp replace to
> remove a blank ... either I am not doing the replace correct or my
> understanding of the file buffer and what I can do with it between the
> browser and saving is not correct,
>
> Anyway my code looks something like this :
>
>
> $uploadfile = $this->uploaddir;
> $mtran = mt_rand(999,999999);
> $NewName = date("Ymd_Gis").$mtran.".csv";
> $uploadfile = $uploadfile.$NewName;
>
> try{
> if
> (move_uploaded_file($_FILES['attachfile']['tmp_name'], $uploadfile))
> {
> $handle = fopen($uploadfile, "r+");
> $lines = file($uploadfile,
> FILE_SKIP_EMPTY_LINES); //FILE_IGNORE_NEW_LINES |
> foreach ($lines as $line_num => $line) {
> $line =
> preg_replace("/(^[\r\n]*|[\r\n]+)[\s\t]*[\r\n]+/", "", $line);
> if(strlen($line) > 0)
> $line=trim($line);
> $line=$line."\n";
> fwrite($handle, $line);
> }
> fclose($handle);
>
If the files aren't too large in size, what about using something like
file('somefile.txt', FILE_IGNORE_NEW_LINES | FILE_SKIP_EMPTY_LINES);
which should pull into an array only those lines with content, and then
just write that back out to the same file line by line?
Thanks,
Ash
http://www.ashleysheridan.co.uk
--- End Message ---
--- Begin Message ---
On 5/21/2010 8:03 AM, Anton Heuschen wrote:
Hi Im trying do something like this, have a function which uploads my
file and returns file pointer ... but at same time ... I want to
remove all Blank lines in a file and update it before it goes to the
final location ...
What I tried was to do a write of file and use some regexp replace to
remove a blank ... either I am not doing the replace correct or my
understanding of the file buffer and what I can do with it between the
browser and saving is not correct,
Anyway my code looks something like this :
$uploadfile = $this->uploaddir;
$mtran = mt_rand(999,999999);
$NewName = date("Ymd_Gis").$mtran.".csv";
$uploadfile = $uploadfile.$NewName;
try{
if
(move_uploaded_file($_FILES['attachfile']['tmp_name'], $uploadfile))
{
$handle = fopen($uploadfile, "r+");
$lines = file($uploadfile,
FILE_SKIP_EMPTY_LINES); //FILE_IGNORE_NEW_LINES |
foreach ($lines as $line_num => $line) {
$line =
preg_replace("/(^[\r\n]*|[\r\n]+)[\s\t]*[\r\n]+/", "", $line);
if(strlen($line)> 0)
$line=trim($line);
$line=$line."\n";
fwrite($handle, $line);
}
fclose($handle);
Suggest using file() which does must of the work for you. Then use foreach() to
scan for empty lines. Recreate the array
Here is a quickie, not tested. Don't know what is the end of line on empties.
So you need to adjust as needed.
$orgArray= file(path);
foreach($org as $line){
$tl=trim($line);
if(empty($tl)) continue;
$newArray[]=$line;
}
file_put_contents($filename, $newArray);
--- End Message ---
--- Begin Message ---
On 21 May 2010 15:16, Ashley Sheridan <[email protected]> wrote:
> On Fri, 2010-05-21 at 14:03 +0200, Anton Heuschen wrote:
>
> Hi Im trying do something like this, have a function which uploads my
> file and returns file pointer ... but at same time ... I want to
> remove all Blank lines in a file and update it before it goes to the
> final location ...
>
> What I tried was to do a write of file and use some regexp replace to
> remove a blank ... either I am not doing the replace correct or my
> understanding of the file buffer and what I can do with it between the
> browser and saving is not correct,
>
> Anyway my code looks something like this :
>
>
> $uploadfile = $this->uploaddir;
> $mtran = mt_rand(999,999999);
> $NewName = date("Ymd_Gis").$mtran.".csv";
> $uploadfile = $uploadfile.$NewName;
>
> try{
> if
> (move_uploaded_file($_FILES['attachfile']['tmp_name'], $uploadfile))
> {
> $handle = fopen($uploadfile, "r+");
> $lines = file($uploadfile,
> FILE_SKIP_EMPTY_LINES); //FILE_IGNORE_NEW_LINES |
> foreach ($lines as $line_num => $line) {
> $line =
> preg_replace("/(^[\r\n]*|[\r\n]+)[\s\t]*[\r\n]+/", "", $line);
> if(strlen($line) > 0)
> $line=trim($line);
> $line=$line."\n";
> fwrite($handle, $line);
> }
> fclose($handle);
>
>
>
> If the files aren't too large in size, what about using something like
> file('somefile.txt', FILE_IGNORE_NEW_LINES | FILE_SKIP_EMPTY_LINES); which
> should pull into an array only those lines with content, and then just write
> that back out to the same file line by line?
>
> Thanks,
> Ash
> http://www.ashleysheridan.co.uk
>
>
>
Hi,
I actually had that ... removed it in last example as I was trying other
stuff and it did not seem to work either ?
$lines = file($uploadfile, FILE_SKIP_EMPTY_LINES); //FILE_IGNORE_NEW_LINES
|
see I removed the FILE_IGNORE line - it was in earlier and only tried
FILE_SKIP_EMPTY .... but still the final file had all the spaces again ...
So in the file it would look like (from the original file the user uploads
that is)
1
2
3
4
5
6
but when the file is saved to the server it must look like
1
2
3
4
5
6
but it never does and still looks like the first block.
--- End Message ---
--- Begin Message ---
From: Anton Heuschen
> On 21 May 2010 15:16, Ashley Sheridan <[email protected]>
wrote:
>> On Fri, 2010-05-21 at 14:03 +0200, Anton Heuschen wrote:
>>
>> Hi Im trying do something like this, have a function which uploads my
>> file and returns file pointer ... but at same time ... I want to
>> remove all Blank lines in a file and update it before it goes to the
>> final location ...
>>
>> What I tried was to do a write of file and use some regexp replace to
>> remove a blank ... either I am not doing the replace correct or my
>> understanding of the file buffer and what I can do with it between
the
>> browser and saving is not correct,
>>
>> Anyway my code looks something like this :
>>
>>
>> $uploadfile = $this->uploaddir;
>> $mtran = mt_rand(999,999999);
>> $NewName = date("Ymd_Gis").$mtran.".csv";
>> $uploadfile = $uploadfile.$NewName;
>>
>> try{
>> if
>> (move_uploaded_file($_FILES['attachfile']['tmp_name'], $uploadfile))
>> {
>> $handle = fopen($uploadfile, "r+");
>> $lines = file($uploadfile,
>> FILE_SKIP_EMPTY_LINES); //FILE_IGNORE_NEW_LINES |
>> foreach ($lines as $line_num => $line) {
>> $line =
>> preg_replace("/(^[\r\n]*|[\r\n]+)[\s\t]*[\r\n]+/", "", $line);
>> if(strlen($line) > 0)
>> $line=trim($line);
>> $line=$line."\n";
>> fwrite($handle, $line);
>> }
>> fclose($handle);
>>
>>
>>
>> If the files aren't too large in size, what about using something
like
>> file('somefile.txt', FILE_IGNORE_NEW_LINES | FILE_SKIP_EMPTY_LINES);
which
>> should pull into an array only those lines with content, and then
just write
>> that back out to the same file line by line?
>
> I actually had that ... removed it in last example as I was trying
other
> stuff and it did not seem to work either ?
>
> $lines = file($uploadfile, FILE_SKIP_EMPTY_LINES);
//FILE_IGNORE_NEW_LINES
> |
>
>
> see I removed the FILE_IGNORE line - it was in earlier and only tried
> FILE_SKIP_EMPTY .... but still the final file had all the spaces again
...
>
> So in the file it would look like (from the original file the user
uploads
> that is)
>
> 1
> 2
>
> 3
> 4
>
>
> 5
>
> 6
>
>
> but when the file is saved to the server it must look like
>
>
> 1
> 2
> 3
> 4
> 5
> 6
>
> but it never does and still looks like the first block.
Are those lines actually empty, or do they have other non-printing
characters in them? Isn't there a generic whitespace value that could be
used in place of '\s\t'?
Can you look at the output file with a binary or hex editor to see what
is actually in those 'empty' lines?
Does that regular expression work correctly on UTF-8 input?
Bob McConnell
--- End Message ---
--- Begin Message ---
Anton Heuschen writes:
> Hi Im trying do something like this, have a function which uploads my
> file and returns file pointer ... but at same time ... I want to
> remove all Blank lines in a file and update it
sed -i '/^$/d' yourFile
--
Gary
--- End Message ---
--- Begin Message ---
From: Ryan Sun
> Thanks for your reply
You are welcome.
>> The other problem has to do with thinking an fread() will always give
>> you everything you sent in an fwrite()
> Interestingly, I use 'telnet 127.0.0.1 1037' for testing later(on
> windows) and everything works, the php server got the input from
> telnet client, so I assume there is something wrong in the php client,
> the fwrite statement...
What are the actual symptoms? Are you sure both ends are using the same
character encoding?
TCP transports a series of octets, or bytes. Again it is up to the
applications on both ends of the stream to agree on interpretation of
those octets, either separately or in sequence. For example, if integers
or floats are sent in a native format, but one end is little Endean
while the other is big Endean, there will be some differences in what
the two ends read from that stream. Likewise, if one end is expecting
ASCII, while the other is sending UTF-8, there may be the occasional
multi-byte value that gets scrambled.
On the other hand, I may be way out in left field.
Bob McConnell
--- End Message ---
--- Begin Message ---
On 20 May 2010 16:51, Al <[email protected]> wrote:
> I'm not being clear. First pass is thru the blacklist, which effectually
> tells hacker to not bother and totally deletes the entry.
>
> If the raw entry gets past the blacklist, it must then only contain my
> whitelist tags. e.g., the two examples you cited were caught by the
> whitelist parser.
Ah, gotcha. That seems like a much better approach to me. But if the
whitelist's going to stop the submission, then why bother with a
blacklist at all?
--- End Message ---
--- Begin Message ---
On Fri, 2010-05-21 at 14:24 +0100, David Otton wrote:
> On 20 May 2010 16:51, Al <[email protected]> wrote:
>
> > I'm not being clear. First pass is thru the blacklist, which effectually
> > tells hacker to not bother and totally deletes the entry.
> >
> > If the raw entry gets past the blacklist, it must then only contain my
> > whitelist tags. e.g., the two examples you cited were caught by the
> > whitelist parser.
>
> Ah, gotcha. That seems like a much better approach to me. But if the
> whitelist's going to stop the submission, then why bother with a
> blacklist at all?
>
I still think you might be better off using BBCode, which is used on
websites just for this very purpose. When any input comes back, you can
remove all the HTML completely and replace the BBCode tags that you
allow. This should guarantee that the only HTML in the text is what you
put there. That way, the only chance someone has to enter malicious code
is to manipulate your replacement algorithm.
Thanks,
Ash
http://www.ashleysheridan.co.uk
--- End Message ---
--- Begin Message ---
On 5/21/2010 9:24 AM, David Otton wrote:
On 20 May 2010 16:51, Al<[email protected]> wrote:
I'm not being clear. First pass is thru the blacklist, which effectually
tells hacker to not bother and totally deletes the entry.
If the raw entry gets past the blacklist, it must then only contain my
whitelist tags. e.g., the two examples you cited were caught by the
whitelist parser.
Ah, gotcha. That seems like a much better approach to me. But if the
whitelist's going to stop the submission, then why bother with a
blacklist at all?
Like I said above, First pass is thru the blacklist, which effectually
tells hackers to not bother and totally deletes the entry.
Also, it's possible that one of my non-techie users can unwittingly enter hack
code. I want to make a big deal of it. My error messages says in red "Illegal
code entered. It was not saved. Reenter your text without it." Remember, I show
them the error segment so they know exactly what the problem is. There is also
another msg which says to contact tech support with a link.
--- End Message ---
--- Begin Message ---
On 21 May 2010 14:21, Ashley Sheridan <[email protected]> wrote:
> I still think you might be better off using BBCode, which is used on
> websites just for this very purpose. When any input comes back, you can
> remove all the HTML completely and replace the BBCode tags that you
> allow. This should guarantee that the only HTML in the text is what you
> put there. That way, the only chance someone has to enter malicious code
> is to manipulate your replacement algorithm.
We don't know what the use case is. It's likely that HTML is a fixed
requirement here.
In any case, stripping the HTML from a post and leaving just the
BBCode is almost as difficult as stripping out all tags except <p>.
There are so many text encodings and weird quirks out there that I
wouldn't trust any code I'd written myself to do it. HTMLPurifier is
widely adopted and tested, and actively maintained.
--- End Message ---
--- Begin Message ---
On 5/21/2010 9:21 AM, Ashley Sheridan wrote:
On Fri, 2010-05-21 at 14:24 +0100, David Otton wrote:
On 20 May 2010 16:51, Al<[email protected]> wrote:
I'm not being clear. First pass is thru the blacklist, which effectually
tells hacker to not bother and totally deletes the entry.
If the raw entry gets past the blacklist, it must then only contain my
whitelist tags. e.g., the two examples you cited were caught by the
whitelist parser.
Ah, gotcha. That seems like a much better approach to me. But if the
whitelist's going to stop the submission, then why bother with a
blacklist at all?
I still think you might be better off using BBCode, which is used on
websites just for this very purpose. When any input comes back, you can
remove all the HTML completely and replace the BBCode tags that you
allow. This should guarantee that the only HTML in the text is what you
put there. That way, the only chance someone has to enter malicious code
is to manipulate your replacement algorithm.
Thanks,
Ash
http://www.ashleysheridan.co.uk
I've used BBcode several times in the pass for this reason. But, found I was
forever having to add new ones for special situations that could easily be
handled with plain old HTML elements. Some of my users have a rudimentary
knowledge of html so they can use it. Most just use my proxy tags e.g., a
partial list:;
Text Emphasis => <blue>foo</blue>, <bold>foo</bold>, <green>foo</green>,...
Titles and Headers => <blue-title>foo</blue-title>, <blue-subtitle>..
Containers => <container location; width; border>any content</container>
Lists => <list>*foo...*foo</list>
Horiz and Blank Lines, etc. => <black-line>, <blue-line>, <blank-line>,
URL and email Links => <url "www.foo.com">Label</url>; [w/wo http:],
Images => <image position width% relPath>caption</image>;
--- End Message ---
--- Begin Message ---
Al wrote:
>
>
> On 5/21/2010 9:24 AM, David Otton wrote:
>> On 20 May 2010 16:51, Al<[email protected]> wrote:
>>
>>> I'm not being clear. First pass is thru the blacklist, which effectually
>>> tells hacker to not bother and totally deletes the entry.
>>>
>>> If the raw entry gets past the blacklist, it must then only contain my
>>> whitelist tags. e.g., the two examples you cited were caught by the
>>> whitelist parser.
>>
>> Ah, gotcha. That seems like a much better approach to me. But if the
>> whitelist's going to stop the submission, then why bother with a
>> blacklist at all?
>
> Like I said above, First pass is thru the blacklist, which effectually
> tells hackers to not bother and totally deletes the entry.
>
> Also, it's possible that one of my non-techie users can unwittingly
> enter hack code. I want to make a big deal of it. My error messages says
> in red "Illegal code entered. It was not saved. Reenter your text
> without it." Remember, I show them the error segment so they know
> exactly what the problem is. There is also another msg which says to
> contact tech support with a link.
>
>
>
Do you actually "show them" the error. That would give away your mystical
powers of detection... :)
--
Jim Lucas
"Some men are born to greatness, some achieve greatness,
and some have greatness thrust upon them."
Twelfth Night, Act II, Scene V
by William Shakespeare
--- End Message ---
--- Begin Message ---
On 5/21/2010 10:36 AM, Jim Lucas wrote:
Al wrote:
On 5/21/2010 9:24 AM, David Otton wrote:
On 20 May 2010 16:51, Al<[email protected]> wrote:
I'm not being clear. First pass is thru the blacklist, which effectually
tells hacker to not bother and totally deletes the entry.
If the raw entry gets past the blacklist, it must then only contain my
whitelist tags. e.g., the two examples you cited were caught by the
whitelist parser.
Ah, gotcha. That seems like a much better approach to me. But if the
whitelist's going to stop the submission, then why bother with a
blacklist at all?
Like I said above, First pass is thru the blacklist, which effectually
tells hackers to not bother and totally deletes the entry.
Also, it's possible that one of my non-techie users can unwittingly
enter hack code. I want to make a big deal of it. My error messages says
in red "Illegal code entered. It was not saved. Reenter your text
without it." Remember, I show them the error segment so they know
exactly what the problem is. There is also another msg which says to
contact tech support with a link.
Do you actually "show them" the error. That would give away your mystical
powers of detection... :)
Keep in mind that my users are authenticated before being allowed access. So,
I'm covering the situations where my user's PW has been stolen or the hacker got
past the auth.
Fact is, I mainly want to prevent malicious scripts from being placed on my
pages rendered as HTML. I just spent some time helping a website I designed
some years ago, but have not been involved for two years, investigate a hacking.
The folks maintaining the site ignored all of my recommendations for good
security practices. Bottom line: 920 html and php files that generate html have
a script that sends every visitor's IP and the page's URL to a website in RU. I
don't know why they want this info, the site belongs to a running club.
--- End Message ---
