Wow. I had no idea that you guys would be so into this. Thanks.
Oh, and just so you know, I lifted that data string in my example from the php online manual page for the base64_encode() function. I was just using it for testing purposes.
And, given how you said processing that 30 MB data set "gave my system a decent work out", I'm thinking that our decision to use a mediumblob (that limits to about 16MB, or about 12.3 MB before it's encoded) was a pretty good one. The next larger option was a largeblob (maximum around 4GB ... Ooo.)
Thanks again you guys, Lloyd
James Lance wrote:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
Here's the situation. I'm encoding files (base 64 encoding) to put them into a database. I want to also include some metadata with the encoded string, namely filename and mime type. Here's an example of the format I decided to use.
"filename: somefilename.txt;mime-type: text/plain; data: VGhpcyBpcyBhbiBlbmNvZGVkIHN0cmluZw=="
So, I think the regexp that would allow me to extract the filename, mime-type, and the data itself, is this:
/^filename:\s+([-_.a-zA-Z0-9]+);mime-type:\s+([-_/a-zA-Z0-9]+);data:\s(.*)$
I want you to know I wasted a good half-hour ~ 45 minutes because of you :-)
I thought that I would show you a completely different way of splitting out that data:
$var = "filename: somefilename.txt;mime-type: text/plain; data: VGhpcyBpcyBhbiBlbmNvZGVkIHN0cmluZw==";
my %info = (map(split(/\s*:\s*/,$_),split(/\s*;\s*/,$var)));
If you can get rid of the spaces between the : and ; like: "filename:somefilename.txt;mime-type:text/plain;data:VGhpcyBpcyBhbiBlbmNvZGVkIHN0cmluZw=="; then you can clear out the \s* filters. You can get at your data through $info{filename}, $info{data} etc
Then I thought that I would check the time differences between using map and split and the regex that Jacob suggested:
s/^filename: ([^;]+);mime-type: ([^;]+); data: //;
The regex is about twice as fast with that data string that you have there.
I tried using a data input that was 30 megs in size and the regex was about 8 times faster. I have included the test results if you are interested:
very small data set:
10001 trials of map (155.096ms total), 15us/trial
10001 trials of regex (94.459ms total), 9us/trial
30 meg data set: 1001 trials of map (274.845s total), 274.571ms/trial 1001 trials of regex (42.634s total), 42.591ms/trial
the 30 meg test trial gave my system a decent work out. it had that cpu usage up at %60.
That was a fun waste of time... :-)
- -- James Lance
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.4 (GNU/Linux)
iD8DBQFArD6exwWH9N2FUHoRAi6GAJ9769N+FODYBz6N5dWLtD6EviXJTACg0yiR KS/qg4FKpfE921mHwt78UUI= =a0hS -----END PGP SIGNATURE-----
____________________
BYU Unix Users Group http://uug.byu.edu/ ___________________________________________________________________
List Info: http://uug.byu.edu/cgi-bin/mailman/listinfo/uug-list
--
--------------------
"Open Source is not a crime."
-Seen on a bumper sticker
--------------------____________________
BYU Unix Users Group http://uug.byu.edu/ ___________________________________________________________________
List Info: http://uug.byu.edu/cgi-bin/mailman/listinfo/uug-list
