RE: zipping and unzipping a string
It sounds like your 'database guy' has turned his own laziness into your challenge. (Translated: cut his work load and quadrupled yours)I can't think of any reason not to use the text data type when it's best suited for the data in question. Hopefully he manages the server all together so you can win the 'challenge' and change his mind at the same time... Store the data in a text file, store the name of the file in the database and see if he'd rather use scattered hard drive storage rather than the database that was meant to hold it in the first place. A few hundred text files should help to change his mind. It's almost worth the extra work just to teach him a lesson in data storage. Ps. when he comes around, make him port all the files' data into the shiny new Text field. I can think of one or two other useless ways to do this but there's seriously no point what-so-ever. ..:.:.:.:.:.:.:.:.:.:.:.:.:.:. Bobby Hartsfield http://acoderslife.com -Original Message- From: John Blayter [mailto:[EMAIL PROTECTED] Sent: Saturday, October 15, 2005 11:54 PM To: CF-Talk Subject: zipping and unzipping a string I am looking for a way to zip and unzip a string without going to the hard drive to perform I/O operations. I have started hacking away at some code but I don't know enough Java to get it working. I have a WDDX packet that can be between 9 - 14K that we want to store in a SQL server database. The database guy has hatred towards using a text data type and has challenged us to find a way to fit a 14K wddx packet into a varchar(8000) field. Any help would be greatly appreciated. Thanks, -- John Blayter email: [EMAIL PROTECTED] ~| Find out how CFTicket can increase your company's customer support efficiency by 100% http://www.houseoffusion.com/banners/view.cfm?bannerid=49 Message: http://www.houseoffusion.com/lists.cfm/link=i:4:221149 Archives: http://www.houseoffusion.com/cf_lists/threads.cfm/4 Subscription: http://www.houseoffusion.com/lists.cfm/link=s:4 Unsubscribe: http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=11502.10531.4 Donations Support: http://www.houseoffusion.com/tiny.cfm/54
Re: zipping and unzipping a string
Some of his concerns are actually valid. It does take SQL server longer to return a query with a TEXT field than it does that with just a varchar and in this application a couple extra MS does count. The main issue with why we are going this route is because we are going to load 65,000 of these into the database. Right now we are pre-loading them into a structure in the server scope. This sounds fine and all but it takes at least 30 minutes to populate this structure with 5,500 entries from the database using an asynchronous gateway. Using the server scope for all 65,000 entries will not work because we will hit the memory ceiling of the JVM. Using the database as a caching storage will allow us to manage the cache for all servers in the cluster in a single place. Another advantage is that the cache will not disappear if the server or service gets restarted. So it may sound minuscule or even petty but I can not store these files on the server as it takes too long to do read operations on each one when I need it. I can store them as a text field in the database but I am looking for a faster way to use a combination of a varchar data type with compression in the application level to shave off a couple more MS. On 10/16/05, Bobby Hartsfield [EMAIL PROTECTED] wrote: It sounds like your 'database guy' has turned his own laziness into your challenge. (Translated: cut his work load and quadrupled yours)I can't think of any reason not to use the text data type when it's best suited for the data in question. Hopefully he manages the server all together so you can win the 'challenge' and change his mind at the same time... Store the data in a text file, store the name of the file in the database and see if he'd rather use scattered hard drive storage rather than the database that was meant to hold it in the first place. A few hundred text files should help to change his mind. It's almost worth the extra work just to teach him a lesson in data storage. Ps. when he comes around, make him port all the files' data into the shiny new Text field. I can think of one or two other useless ways to do this but there's seriously no point what-so-ever. ..:.:.:.:.:.:.:.:.:.:.:.:.:.:. Bobby Hartsfield http://acoderslife.com -Original Message- From: John Blayter [mailto:[EMAIL PROTECTED] Sent: Saturday, October 15, 2005 11:54 PM To: CF-Talk Subject: zipping and unzipping a string I am looking for a way to zip and unzip a string without going to the hard drive to perform I/O operations. I have started hacking away at some code but I don't know enough Java to get it working. I have a WDDX packet that can be between 9 - 14K that we want to store in a SQL server database. The database guy has hatred towards using a text data type and has challenged us to find a way to fit a 14K wddx packet into a varchar(8000) field. Any help would be greatly appreciated. Thanks, -- John Blayter email: [EMAIL PROTECTED] ~| Logware (www.logware.us): a new and convenient web-based time tracking application. Start tracking and documenting hours spent on a project or with a client with Logware today. Try it for free with a 15 day trial account. http://www.houseoffusion.com/banners/view.cfm?bannerid=67 Message: http://www.houseoffusion.com/lists.cfm/link=i:4:221152 Archives: http://www.houseoffusion.com/cf_lists/threads.cfm/4 Subscription: http://www.houseoffusion.com/lists.cfm/link=s:4 Unsubscribe: http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=89.70.4 Donations Support: http://www.houseoffusion.com/tiny.cfm/54
RE: zipping and unzipping a string
in this application a couple extra MS does count. for a faster way to use a combination of a varchar data type with compression in the application level to shave off a couple more MS. Don't you think that running a compress/decompress routine on the data is going to negate those few ms you save on the database side anyway? I would think it would actually take longer, actually. Just my $0.02. -Justin Scott ~| Logware (www.logware.us): a new and convenient web-based time tracking application. Start tracking and documenting hours spent on a project or with a client with Logware today. Try it for free with a 15 day trial account. http://www.houseoffusion.com/banners/view.cfm?bannerid=67 Message: http://www.houseoffusion.com/lists.cfm/link=i:4:221155 Archives: http://www.houseoffusion.com/cf_lists/threads.cfm/4 Subscription: http://www.houseoffusion.com/lists.cfm/link=s:4 Unsubscribe: http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=89.70.4 Donations Support: http://www.houseoffusion.com/tiny.cfm/54
RE: zipping and unzipping a string
I still see no valid concerns. It may take sql server a ms or two longer to return the type if its full, yes. But you're going to do nothing but add more time by compressing/decompressing or using the CF server to process something that is better suited for the database to process. If time is your concern, I still say make a text data type. I know its not a good idea to store the data in text files... the point was to hope your dba realized that too. ..:.:.:.:.:.:.:.:.:.:.:.:.:.:. Bobby Hartsfield http://acoderslife.com -Original Message- From: John Blayter [mailto:[EMAIL PROTECTED] Sent: Sunday, October 16, 2005 8:40 AM To: CF-Talk Subject: Re: zipping and unzipping a string Some of his concerns are actually valid. It does take SQL server longer to return a query with a TEXT field than it does that with just a varchar and in this application a couple extra MS does count. The main issue with why we are going this route is because we are going to load 65,000 of these into the database. Right now we are pre-loading them into a structure in the server scope. This sounds fine and all but it takes at least 30 minutes to populate this structure with 5,500 entries from the database using an asynchronous gateway. Using the server scope for all 65,000 entries will not work because we will hit the memory ceiling of the JVM. Using the database as a caching storage will allow us to manage the cache for all servers in the cluster in a single place. Another advantage is that the cache will not disappear if the server or service gets restarted. So it may sound minuscule or even petty but I can not store these files on the server as it takes too long to do read operations on each one when I need it. I can store them as a text field in the database but I am looking for a faster way to use a combination of a varchar data type with compression in the application level to shave off a couple more MS. On 10/16/05, Bobby Hartsfield [EMAIL PROTECTED] wrote: It sounds like your 'database guy' has turned his own laziness into your challenge. (Translated: cut his work load and quadrupled yours)I can't think of any reason not to use the text data type when it's best suited for the data in question. Hopefully he manages the server all together so you can win the 'challenge' and change his mind at the same time... Store the data in a text file, store the name of the file in the database and see if he'd rather use scattered hard drive storage rather than the database that was meant to hold it in the first place. A few hundred text files should help to change his mind. It's almost worth the extra work just to teach him a lesson in data storage. Ps. when he comes around, make him port all the files' data into the shiny new Text field. I can think of one or two other useless ways to do this but there's seriously no point what-so-ever. ..:.:.:.:.:.:.:.:.:.:.:.:.:.:. Bobby Hartsfield http://acoderslife.com -Original Message- From: John Blayter [mailto:[EMAIL PROTECTED] Sent: Saturday, October 15, 2005 11:54 PM To: CF-Talk Subject: zipping and unzipping a string I am looking for a way to zip and unzip a string without going to the hard drive to perform I/O operations. I have started hacking away at some code but I don't know enough Java to get it working. I have a WDDX packet that can be between 9 - 14K that we want to store in a SQL server database. The database guy has hatred towards using a text data type and has challenged us to find a way to fit a 14K wddx packet into a varchar(8000) field. Any help would be greatly appreciated. Thanks, -- John Blayter email: [EMAIL PROTECTED] ~| Logware (www.logware.us): a new and convenient web-based time tracking application. Start tracking and documenting hours spent on a project or with a client with Logware today. Try it for free with a 15 day trial account. http://www.houseoffusion.com/banners/view.cfm?bannerid=67 Message: http://www.houseoffusion.com/lists.cfm/link=i:4:221156 Archives: http://www.houseoffusion.com/cf_lists/threads.cfm/4 Subscription: http://www.houseoffusion.com/lists.cfm/link=s:4 Unsubscribe: http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=89.70.4 Donations Support: http://www.houseoffusion.com/tiny.cfm/54
zipping and unzipping a string
I am looking for a way to zip and unzip a string without going to the hard drive to perform I/O operations. I have started hacking away at some code but I don't know enough Java to get it working. I have a WDDX packet that can be between 9 - 14K that we want to store in a SQL server database. The database guy has hatred towards using a text data type and has challenged us to find a way to fit a 14K wddx packet into a varchar(8000) field. Any help would be greatly appreciated. Thanks, -- John Blayter email: [EMAIL PROTECTED] ~| Logware (www.logware.us): a new and convenient web-based time tracking application. Start tracking and documenting hours spent on a project or with a client with Logware today. Try it for free with a 15 day trial account. http://www.houseoffusion.com/banners/view.cfm?bannerid=67 Message: http://www.houseoffusion.com/lists.cfm/link=i:4:221142 Archives: http://www.houseoffusion.com/cf_lists/threads.cfm/4 Subscription: http://www.houseoffusion.com/lists.cfm/link=s:4 Unsubscribe: http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=11502.10531.4 Donations Support: http://www.houseoffusion.com/tiny.cfm/54
RE: zipping and unzipping a string
From: John Blayter [mailto:[EMAIL PROTECTED] I am looking for a way to zip and unzip a string without going to the hard drive to perform I/O operations. I have started hacking away at some code but I don't know enough Java to get it working. I have a WDDX packet that can be between 9 - 14K that we want to store in a SQL server database. The database guy has hatred towards using a text data type and has challenged us to find a way to fit a 14K wddx packet into a varchar(8000) field. Any help would be greatly appreciated. Thanks, I would tell your DBA that large data types exist for a reason... large data! What is his reasoning for not wanting the text data type used? What is the purpose of the field, will it be searched? Indexed? Is the DBA familiar with Text In Row (alleviates the double I/O or text retrieval)... etc... Mike ~| Find out how CFTicket can increase your company's customer support efficiency by 100% http://www.houseoffusion.com/banners/view.cfm?bannerid=49 Message: http://www.houseoffusion.com/lists.cfm/link=i:4:221144 Archives: http://www.houseoffusion.com/cf_lists/threads.cfm/4 Subscription: http://www.houseoffusion.com/lists.cfm/link=s:4 Unsubscribe: http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=11502.10531.4 Donations Support: http://www.houseoffusion.com/tiny.cfm/54
RE: zipping and unzipping a string
-Original Message- From: Michael T. Tangorre [mailto:[EMAIL PROTECTED] Sent: Sunday, October 16, 2005 12:06 AM To: CF-Talk Subject: RE: zipping and unzipping a string From: John Blayter [mailto:[EMAIL PROTECTED] I am looking for a way to zip and unzip a string without going to the hard drive to perform I/O operations. I have started hacking away at some code but I don't know enough Java to get it working. I have a WDDX packet that can be between 9 - 14K that we want to store in a SQL server database. The database guy has hatred towards using a text data type and has challenged us to find a way to fit a 14K wddx packet into a varchar(8000) field. Any help would be greatly appreciated. Thanks, I would tell your DBA that large data types exist for a reason... large data! What is his reasoning for not wanting the text data type used? What is the purpose of the field, will it be searched? Indexed? Is the DBA familiar with Text In Row (alleviates the double I/O or text retrieval)... etc... Yeah... I gotta agree. You shouldn't change your data because of a personal hatred towards a perfectly usable feature. Does he have actual reasons? Anything that will offset the development time needed (and they completely lack of flexibility assumed) from hacking large data to fit in a smaller field? Look at the data. Use the data type most appropriate to the data. That's really DB 101 there. If you have problems after that optimize, consider kludges, etc. But not to do the logical thing because of a unsubstantiated personal opinion is just plain silly. Jim Davis ~| Logware (www.logware.us): a new and convenient web-based time tracking application. Start tracking and documenting hours spent on a project or with a client with Logware today. Try it for free with a 15 day trial account. http://www.houseoffusion.com/banners/view.cfm?bannerid=67 Message: http://www.houseoffusion.com/lists.cfm/link=i:4:221147 Archives: http://www.houseoffusion.com/cf_lists/threads.cfm/4 Subscription: http://www.houseoffusion.com/lists.cfm/link=s:4 Unsubscribe: http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=89.70.4 Donations Support: http://www.houseoffusion.com/tiny.cfm/54
Re: zipping and unzipping a string
I'm not 100% sure, but SQL server 6 or 7 (I think, from memory) has a max record size of 8000 bytes or something weird. We had a couple of varchar 4000 fields and all the normal fields on top of that (PK, FKs, etc etc). It was fine till it started truncating info. Much research later and it turned out that the max record length was being exceeded. Hence the text field type and its data pointer goodness... One confirmation here: http://west-wind.com/weblog/posts/207.aspx If your wddx access does not need to be lightning fast, another option is to store it on the HDD and just store a reference to the file in your database table. HTH Aaron Jim Davis wrote: -Original Message- From: Michael T. Tangorre [mailto:[EMAIL PROTECTED] Sent: Sunday, October 16, 2005 12:06 AM To: CF-Talk Subject: RE: zipping and unzipping a string From: John Blayter [mailto:[EMAIL PROTECTED] I am looking for a way to zip and unzip a string without going to the hard drive to perform I/O operations. I have started hacking away at some code but I don't know enough Java to get it working. I have a WDDX packet that can be between 9 - 14K that we want to store in a SQL server database. The database guy has hatred towards using a text data type and has challenged us to find a way to fit a 14K wddx packet into a varchar(8000) field. Any help would be greatly appreciated. Thanks, I would tell your DBA that large data types exist for a reason... large data! What is his reasoning for not wanting the text data type used? What is the purpose of the field, will it be searched? Indexed? Is the DBA familiar with Text In Row (alleviates the double I/O or text retrieval)... etc... Yeah... I gotta agree. You shouldn't change your data because of a personal hatred towards a perfectly usable feature. Does he have actual reasons? Anything that will offset the development time needed (and they completely lack of flexibility assumed) from hacking large data to fit in a smaller field? Look at the data. Use the data type most appropriate to the data. That's really DB 101 there. If you have problems after that optimize, consider kludges, etc. But not to do the logical thing because of a unsubstantiated personal opinion is just plain silly. Jim Davis ~| Logware (www.logware.us): a new and convenient web-based time tracking application. Start tracking and documenting hours spent on a project or with a client with Logware today. Try it for free with a 15 day trial account. http://www.houseoffusion.com/banners/view.cfm?bannerid=67 Message: http://www.houseoffusion.com/lists.cfm/link=i:4:221148 Archives: http://www.houseoffusion.com/cf_lists/threads.cfm/4 Subscription: http://www.houseoffusion.com/lists.cfm/link=s:4 Unsubscribe: http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=89.70.4 Donations Support: http://www.houseoffusion.com/tiny.cfm/54
