Re: Performance issue with large data(such as images)
newton.dave wrote: > > >> "" doesn't work at >> all. > > Stream it from an action/servlet/etc? > > Dave > > - > To unsubscribe, e-mail: user-unsubscr...@struts.apache.org > For additional commands, e-mail: user-h...@struts.apache.org > > > Yes, I streamed it from an action. However, I could show the above table using an action to stream images. -- View this message in context: http://www.nabble.com/Performance-issue-with-large-data%28such-as-images%29-tp23945029p23981525.html Sent from the Struts - User mailing list archive at Nabble.com. - To unsubscribe, e-mail: user-unsubscr...@struts.apache.org For additional commands, e-mail: user-h...@struts.apache.org
Re: Performance issue with large data(such as images)
fireapple wrote: Any way to resolve the security issue? If the user save the portrait in the hard drive(outside of app), I couldn't find a way to display it using URL. "" doesn't work at all. Stream it from an action/servlet/etc? Dave - To unsubscribe, e-mail: user-unsubscr...@struts.apache.org For additional commands, e-mail: user-h...@struts.apache.org
RE: Performance issue with large data(such as images)
Security Management wrote: > > I have had problems when trying to show a bunch of thumbnails, because you > have a URL that points to an action in your jsp that causes a database hit > to pull the image. If I'm trying to show 150 thumbnails, the page takes > forever to load due to the db connections. Any solution to that? I'm > using > DBCP > > - > To unsubscribe, e-mail: user-unsubscr...@struts.apache.org > For additional commands, e-mail: user-h...@struts.apache.org > > > - > To unsubscribe, e-mail: user-unsubscr...@struts.apache.org > For additional commands, e-mail: user-h...@struts.apache.org > > > I also use the URL that points to an action in my jsp, such as: http://www.nabble.com/Performance-issue-with-large-data%28such-as-images%29-tp23945029p23969896.html Sent from the Struts - User mailing list archive at Nabble.com. - To unsubscribe, e-mail: user-unsubscr...@struts.apache.org For additional commands, e-mail: user-h...@struts.apache.org
Re: Performance issue with large data(such as images)
Richard Sayre wrote: > > I should have also mentioned that there are security issues with > persisting to the hard drive. > > I would recommend saving to the database and loading the images as > needed. Do not save them to session. I do not think you will have a > performance problem. > > Rich > > - > To unsubscribe, e-mail: user-unsubscr...@struts.apache.org > For additional commands, e-mail: user-h...@struts.apache.org > > > Any way to resolve the security issue? If the user save the portrait in the hard drive(outside of app), I couldn't find a way to display it using URL. "" doesn't work at all. -- View this message in context: http://www.nabble.com/Performance-issue-with-large-data%28such-as-images%29-tp23945029p23969795.html Sent from the Struts - User mailing list archive at Nabble.com. - To unsubscribe, e-mail: user-unsubscr...@struts.apache.org For additional commands, e-mail: user-h...@struts.apache.org
RE: Performance issue with large data(such as images)
I should have also mentioned that there are security issues with persisting to the hard drive. I would recommend saving to the database and loading the images as needed. Do not save them to session. I do not think you will have a performance problem. Rich I have had problems when trying to show a bunch of thumbnails, because you have a URL that points to an action in your jsp that causes a database hit to pull the image. If I'm trying to show 150 thumbnails, the page takes forever to load due to the db connections. Any solution to that? I'm using DBCP - To unsubscribe, e-mail: user-unsubscr...@struts.apache.org For additional commands, e-mail: user-h...@struts.apache.org - To unsubscribe, e-mail: user-unsubscr...@struts.apache.org For additional commands, e-mail: user-h...@struts.apache.org
RE: Performance issue with large data(such as images)
This is a little off topic, but can anybody recommend a photo resizing utility to run after the upload? I too have a need to allow the user to upload a user photo, which could be stored in a BLOB, but I know it is not that simple. Many sites do post processing to adjust the image resolution, size or even type, and some allow the user to crop the image as well. I'm curious if anybody has experience with an open source photo manipulation framework that plays well with Struts 2. Steve Mitchell http://www.Byteworksinc.com -Original Message- From: Richard Sayre [mailto:richardsa...@gmail.com] Sent: Wednesday, June 10, 2009 12:41 PM To: Struts Users Mailing List Subject: Re: Performance issue with large data(such as images) >Thanks, Rich, I''m happy to help. > If I use BLOB, I have to make a lot of change to my code. Therefore, I have > to persist it as String. I used base64 to encode and decode the byte[] into > String. Because I resize the images to small ones. I use a VARCHAR length > about 10k~20k to persist the String. Thats fine, if you don't want to do a hugh code change then keep it the same > "How often do you need to load all 300 images? If it is forum style I > would imangine a limited number of posting per page say max 20. So > the would be a maximum of 20 images per page load." > If this is true, which method is the best among the above three? Which method is 'best' depends on what you want. For fast access and low memory - persist to hard drive > "If you must use the least amount of memory as possible, persist the > images to the hard drive and store the file reference to the url in > the database. " > If I want to persist images to the hard drive of server. Shall I persist the > data inside WAR or outside? Because user can change their portrait, does it > mean it must be persisted outside WAR? No just make a base dir in your app called 'userAvatar' or something similar. Then persist the image as user_.gif ( = the user id). When they chage their image override the file. Also as stated by Pawel, make sure you have a performance problem before you try to solve a performance problem. > -- > View this message in context: > http://www.nabble.com/Performance-issue-with-large-data%28such-as-images%29-tp23945029p23948048.html > Sent from the Struts - User mailing list archive at Nabble.com. > > > - > To unsubscribe, e-mail: user-unsubscr...@struts.apache.org > For additional commands, e-mail: user-h...@struts.apache.org > > - To unsubscribe, e-mail: user-unsubscr...@struts.apache.org For additional commands, e-mail: user-h...@struts.apache.org -- NOTICE: This electronic mail message and any attached files are confidential. The information is exclusively for the use of the individual or entity intended as the recipient. If you are not the intended recipient, any use, copying, printing, reviewing, retention, disclosure, distribution or forwarding of the message or any attached file is not authorized and is strictly prohibited. If you have received this electronic mail message in error, please advise the sender by reply electronic mail immediately and permanently delete the original transmission, any attachments and any copies of this message from your computer system. Thank you. == - To unsubscribe, e-mail: user-unsubscr...@struts.apache.org For additional commands, e-mail: user-h...@struts.apache.org
Re: Performance issue with large data(such as images)
I should have also mentioned that there are security issues with persisting to the hard drive. I would recommend saving to the database and loading the images as needed. Do not save them to session. I do not think you will have a performance problem. Rich - To unsubscribe, e-mail: user-unsubscr...@struts.apache.org For additional commands, e-mail: user-h...@struts.apache.org
Re: Performance issue with large data(such as images)
>Thanks, Rich, I''m happy to help. > If I use BLOB, I have to make a lot of change to my code. Therefore, I have > to persist it as String. I used base64 to encode and decode the byte[] into > String. Because I resize the images to small ones. I use a VARCHAR length > about 10k~20k to persist the String. Thats fine, if you don't want to do a hugh code change then keep it the same > "How often do you need to load all 300 images? If it is forum style I > would imangine a limited number of posting per page say max 20. So > the would be a maximum of 20 images per page load." > If this is true, which method is the best among the above three? Which method is 'best' depends on what you want. For fast access and low memory - persist to hard drive > "If you must use the least amount of memory as possible, persist the > images to the hard drive and store the file reference to the url in > the database. " > If I want to persist images to the hard drive of server. Shall I persist the > data inside WAR or outside? Because user can change their portrait, does it > mean it must be persisted outside WAR? No just make a base dir in your app called 'userAvatar' or something similar. Then persist the image as user_.gif ( = the user id). When they chage their image override the file. Also as stated by Pawel, make sure you have a performance problem before you try to solve a performance problem. > -- > View this message in context: > http://www.nabble.com/Performance-issue-with-large-data%28such-as-images%29-tp23945029p23948048.html > Sent from the Struts - User mailing list archive at Nabble.com. > > > - > To unsubscribe, e-mail: user-unsubscr...@struts.apache.org > For additional commands, e-mail: user-h...@struts.apache.org > > - To unsubscribe, e-mail: user-unsubscr...@struts.apache.org For additional commands, e-mail: user-h...@struts.apache.org
Re: Performance issue with large data(such as images)
Hi All, just as a side note, 300 users is a very small quantity of users. So i would recomend to check if there is a performance problem at all. Best greetings, Paweł Wielgus. 2009/6/9 Jim Collings : > Can't put em in your database as BLOB's with all the other user info? > > On Tue, Jun 9, 2009 at 4:26 PM, Dave Newton wrote: >> fireapple wrote: >>> >>> If I want to persist images to the hard drive of server. Shall I persist >>> the >>> data inside WAR or outside? Because user can change their portrait, does >>> it >>> mean it must be persisted outside WAR? >> >> You can't write stuff into your WAR. >> >> Dave >> >> - >> To unsubscribe, e-mail: user-unsubscr...@struts.apache.org >> For additional commands, e-mail: user-h...@struts.apache.org >> >> > > - > To unsubscribe, e-mail: user-unsubscr...@struts.apache.org > For additional commands, e-mail: user-h...@struts.apache.org > > - To unsubscribe, e-mail: user-unsubscr...@struts.apache.org For additional commands, e-mail: user-h...@struts.apache.org
Re: Performance issue with large data(such as images)
Can't put em in your database as BLOB's with all the other user info? On Tue, Jun 9, 2009 at 4:26 PM, Dave Newton wrote: > fireapple wrote: >> >> If I want to persist images to the hard drive of server. Shall I persist >> the >> data inside WAR or outside? Because user can change their portrait, does >> it >> mean it must be persisted outside WAR? > > You can't write stuff into your WAR. > > Dave > > - > To unsubscribe, e-mail: user-unsubscr...@struts.apache.org > For additional commands, e-mail: user-h...@struts.apache.org > > - To unsubscribe, e-mail: user-unsubscr...@struts.apache.org For additional commands, e-mail: user-h...@struts.apache.org
Re: Performance issue with large data(such as images)
fireapple wrote: If I want to persist images to the hard drive of server. Shall I persist the data inside WAR or outside? Because user can change their portrait, does it mean it must be persisted outside WAR? You can't write stuff into your WAR. Dave - To unsubscribe, e-mail: user-unsubscr...@struts.apache.org For additional commands, e-mail: user-h...@struts.apache.org
Re: Performance issue with large data(such as images)
Richard Sayre wrote: > > You should persist the images as bytes, in a binary field or something > similar. > >> 1. Because the image is large, it's not efficient to store it in >> objects(beans). > > Well this depends on the Bean. Is the bean stored in session? If yes > then it will take up some memory. If you dont have alot of memory I > would not recommend this. > >> 2. Because I have many users(say 300), if I store all the maps(userId and >> image) in the session, the session will be huge. > > Again if your requirement is low memory usage then dont store large > objects in momory for a long time > >> 3. Because I have many users(say 300), if I access the db for each user's >> image, the worse case is I will have to access db for 300 times. > > Well this is the trade off Memory vs Resources to Connect to a DB. > Even if the images are not stored in session they will still be held > in memory when getting them from the DB. > > How often do you need to load all 300 images? If it is forum style I > would imangine a limited number of posting per page say max 20. So > the would be a maximum of 20 images per page load. > > You could also Store the images in the hard drive of the server, and > store a URL to the image in the database. This would reduce the > memory overhead. Did you try to do any test to see what the impact of > retrieveing the data from the database is? > > If you must use the least amount of memory as possible, persist the > images to the hard drive and store the file reference to the url in > the database. Or even store the images as userId_avatar.gif. That > way you can link the file to a user id and have no need to store the > url. Use a base directory for storing the images. > > Before you guess if something is a performance issue, you should test > to see what the actual cost of each method of image storing and > retrival is. > > Rich > > Thanks, Rich, "You should persist the images as bytes, in a binary field or something similar." If I use BLOB, I have to make a lot of change to my code. Therefore, I have to persist it as String. I used base64 to encode and decode the byte[] into String. Because I resize the images to small ones. I use a VARCHAR length about 10k~20k to persist the String. "How often do you need to load all 300 images? If it is forum style I would imangine a limited number of posting per page say max 20. So the would be a maximum of 20 images per page load." If this is true, which method is the best among the above three? "If you must use the least amount of memory as possible, persist the images to the hard drive and store the file reference to the url in the database. " If I want to persist images to the hard drive of server. Shall I persist the data inside WAR or outside? Because user can change their portrait, does it mean it must be persisted outside WAR? -- View this message in context: http://www.nabble.com/Performance-issue-with-large-data%28such-as-images%29-tp23945029p23948048.html Sent from the Struts - User mailing list archive at Nabble.com. - To unsubscribe, e-mail: user-unsubscr...@struts.apache.org For additional commands, e-mail: user-h...@struts.apache.org
Re: Performance issue with large data(such as images)
You should persist the images as bytes, in a binary field or something similar. > 1. Because the image is large, it's not efficient to store it in > objects(beans). Well this depends on the Bean. Is the bean stored in session? If yes then it will take up some memory. If you dont have alot of memory I would not recommend this. > 2. Because I have many users(say 300), if I store all the maps(userId and > image) in the session, the session will be huge. Again if your requirement is low memory usage then dont store large objects in momory for a long time > 3. Because I have many users(say 300), if I access the db for each user's > image, the worse case is I will have to access db for 300 times. Well this is the trade off Memory vs Resources to Connect to a DB. Even if the images are not stored in session they will still be held in memory when getting them from the DB. How often do you need to load all 300 images? If it is forum style I would imangine a limited number of posting per page say max 20. So the would be a maximum of 20 images per page load. You could also Store the images in the hard drive of the server, and store a URL to the image in the database. This would reduce the memory overhead. Did you try to do any test to see what the impact of retrieveing the data from the database is? If you must use the least amount of memory as possible, persist the images to the hard drive and store the file reference to the url in the database. Or even store the images as userId_avatar.gif. That way you can link the file to a user id and have no need to store the url. Use a base directory for storing the images. Before you guess if something is a performance issue, you should test to see what the actual cost of each method of image storing and retrival is. Rich On Tue, Jun 9, 2009 at 12:49 PM, fireapple wrote: > > Firstly, I persist the portraits (images) of all users in the db as Strings > (large one, say length 5000). Then, I want to get the String and display the > portraits of users on the web page(like a topic in a forum). Here I know > three solutions but none of them is efficient: > 1. Because the image is large, it's not efficient to store it in > objects(beans). > 2. Because I have many users(say 300), if I store all the maps(userId and > image) in the session, the session will be huge. > 3. Because I have many users(say 300), if I access the db for each user's > image, the worse case is I will have to access db for 300 times. > > Is there any solution (such as a design pattern or interceptor) for this > performance issue? Many thanks! > -- > View this message in context: > http://www.nabble.com/Performance-issue-with-large-data%28such-as-images%29-tp23945029p23945029.html > Sent from the Struts - User mailing list archive at Nabble.com. > > > - > To unsubscribe, e-mail: user-unsubscr...@struts.apache.org > For additional commands, e-mail: user-h...@struts.apache.org > > - To unsubscribe, e-mail: user-unsubscr...@struts.apache.org For additional commands, e-mail: user-h...@struts.apache.org
Performance issue with large data(such as images)
Firstly, I persist the portraits (images) of all users in the db as Strings (large one, say length 5000). Then, I want to get the String and display the portraits of users on the web page(like a topic in a forum). Here I know three solutions but none of them is efficient: 1. Because the image is large, it's not efficient to store it in objects(beans). 2. Because I have many users(say 300), if I store all the maps(userId and image) in the session, the session will be huge. 3. Because I have many users(say 300), if I access the db for each user's image, the worse case is I will have to access db for 300 times. Is there any solution (such as a design pattern or interceptor) for this performance issue? Many thanks! -- View this message in context: http://www.nabble.com/Performance-issue-with-large-data%28such-as-images%29-tp23945029p23945029.html Sent from the Struts - User mailing list archive at Nabble.com. - To unsubscribe, e-mail: user-unsubscr...@struts.apache.org For additional commands, e-mail: user-h...@struts.apache.org
