Re: [galaxy-dev] Quota will not decrease with permanent delete
On Jan 16, 2012, at 8:31 AM, Peter Cock wrote: On Mon, Jan 16, 2012 at 1:20 PM, graham etherington (TSL) graham.ethering...@sainsbury-laboratory.ac.uk wrote: Hi Nate, We are getting quite large discrepancies between 'Size on disk' and 'Usage' numbers, so I've run the scripts/set_user_disk_usage.py script. When I run it, some of the 'old usage' numbers are negative numbers, e.g. User1 old usage: -226417886261, new usage: 139894688577 User2 old usage: 263698720473, new usage: 68974834617 User3 old usage: 397108490587, new usage: 406950046043 User4 old usage: -57441684074, new usage: 7536071906 This seems a little strange (how can a user have a 'minus' usage?). Is there an explanation for this? Negative numbers here are probably a sign of integer overflow (something too large for a signed 32 bit integer would be my guess). Possible, although I would guess that set_user_disk_usage.py was never run after the disk accounting was added (harmless, since it can be run at any time). Users who already had a bunch of data then started at '0', deleted a bunch of data, and end up with negative usage. --nate Peter ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Quota will not decrease with permanent delete
Hi Nate, We are getting quite large discrepancies between 'Size on disk' and 'Usage' numbers, so I've run the scripts/set_user_disk_usage.py script. When I run it, some of the 'old usage' numbers are negative numbers, e.g. User1 old usage: -226417886261, new usage: 139894688577 User2 old usage: 263698720473, new usage: 68974834617 User3 old usage: 397108490587, new usage: 406950046043 User4 old usage: -57441684074, new usage: 7536071906 This seems a little strange (how can a user have a 'minus' usage?). Is there an explanation for this? Cheers, Graham Dr. Graham Etherington Bioinformatics Support Officer, The Sainsbury Laboratory, Norwich Research Park, Norwich NR4 7UH. UK Tel: +44 (0)1603 450601 On 06/12/2011 18:17, Nate Coraor n...@bx.psu.edu wrote: On Dec 1, 2011, at 8:37 AM, Glen Beane wrote: You aren't the only one having problem in the disk-usage reported in the upper right. I've also had problems with this total disk usage in our local instance of Galaxy I've deleted all but a handful of files, and we have a cron job that purges the files from disk after a certain number of days. I've deleted everything but one history, and for the last few weeks the size of that history has been reported as about 200GB. I don't have any other histories, or any deleted but not purged datasets. However the total disk usage shown by Galaxy is about 1.5TB Hi Glen, There's a script to recalculate if the total becomes off, see scripts/set_user_disk_usage.py. I've found that on the public site, the totals are occasionally off by small amounts, meaning there must be somewhere where data is associated with or disassociated from a user and is not accounted for. I haven't yet found a reproducible way to make the total incorrect, so I haven't found the bug yet. If you happen to notice when this occurs, please let me know. Otherwise, it should be safe to run the usage script via cron to update totals that become incorrect. Thanks, --nate On Nov 30, 2011, at 2:24 PM, Mary Anne Alliegro wrote: Hi Galaxy Users, I have permanently deleted numerous files. My usage % has decreased, but this is NOT reflected in my Gb report (upper right)- it remains the same. Am I missing some phantom trash bin? If not, will Galaxy recalculate my Gb usage so that I may proceed with my project? Thank you, Mary Anne pastedGraphic.pdf ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ -- Glen L. Beane Senior Software Engineer The Jackson Laboratory (207) 288-6153 ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Quota will not decrease with permanent delete
On Mon, Jan 16, 2012 at 1:20 PM, graham etherington (TSL) graham.ethering...@sainsbury-laboratory.ac.uk wrote: Hi Nate, We are getting quite large discrepancies between 'Size on disk' and 'Usage' numbers, so I've run the scripts/set_user_disk_usage.py script. When I run it, some of the 'old usage' numbers are negative numbers, e.g. User1 old usage: -226417886261, new usage: 139894688577 User2 old usage: 263698720473, new usage: 68974834617 User3 old usage: 397108490587, new usage: 406950046043 User4 old usage: -57441684074, new usage: 7536071906 This seems a little strange (how can a user have a 'minus' usage?). Is there an explanation for this? Negative numbers here are probably a sign of integer overflow (something too large for a signed 32 bit integer would be my guess). Peter ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Quota will not decrease with permanent delete
On Dec 6, 2011, at 2:16 PM, Mary Anne Alliegro wrote: No, it is still reads over 120GB yet the usage is only 26% Hi Mary Anne, Sorry I didn't get back to you sooner. Your usage is currently ~75GB. The difference before may have been due to some other data in your account which was not permanently deleted. If you still find that your disk usage isn't reflecting your actual history sizes, please have a look at this page for details on how to view deleted data: http://wiki.g2.bx.psu.edu/Learn/Managing%20Datasets And let us know if this doesn't answer any remaining questions. Thanks for using Galaxy, --nate Mary Anne Alliegro Marine Biological Laboratory JOSEPHINE BAY PAUL CENTER FOR COMPARATIVE MOLECULAR BIOLOGY AND EVOLUTION 7 MBL Street Woods Hole, MA 02543 maallie...@mbl.edu 508-289-7419 From: Nate Coraor n...@bx.psu.edu To: Mary Anne Alliegro maallie...@mbl.edu Cc: galaxy-...@bx.psu.edu Sent: Tuesday, December 6, 2011 1:14:32 PM Subject: Re: [galaxy-dev] Quota will not decrease with permanent delete On Nov 30, 2011, at 2:24 PM, Mary Anne Alliegro wrote: Hi Galaxy Users, I have permanently deleted numerous files. My usage % has decreased, but this is NOT reflected in my Gb report (upper right)- it remains the same. Am I missing some phantom trash bin? If not, will Galaxy recalculate my Gb usage so that I may proceed with my project? Thank you, Mary Anne Hi Mary Anne, I believe your usage total is now correct, but if this is not the case, please let us know. Thanks, --nate pastedGraphic.pdf ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Quota will not decrease with permanent delete
On Nov 30, 2011, at 2:24 PM, Mary Anne Alliegro wrote: Hi Galaxy Users, I have permanently deleted numerous files. My usage % has decreased, but this is NOT reflected in my Gb report (upper right)- it remains the same. Am I missing some phantom trash bin? If not, will Galaxy recalculate my Gb usage so that I may proceed with my project? Thank you, Mary Anne Hi Mary Anne, I believe your usage total is now correct, but if this is not the case, please let us know. Thanks, --nate pastedGraphic.pdf ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Quota will not decrease with permanent delete
On Dec 1, 2011, at 8:37 AM, Glen Beane wrote: You aren't the only one having problem in the disk-usage reported in the upper right. I've also had problems with this total disk usage in our local instance of Galaxy I've deleted all but a handful of files, and we have a cron job that purges the files from disk after a certain number of days. I've deleted everything but one history, and for the last few weeks the size of that history has been reported as about 200GB. I don't have any other histories, or any deleted but not purged datasets. However the total disk usage shown by Galaxy is about 1.5TB Hi Glen, There's a script to recalculate if the total becomes off, see scripts/set_user_disk_usage.py. I've found that on the public site, the totals are occasionally off by small amounts, meaning there must be somewhere where data is associated with or disassociated from a user and is not accounted for. I haven't yet found a reproducible way to make the total incorrect, so I haven't found the bug yet. If you happen to notice when this occurs, please let me know. Otherwise, it should be safe to run the usage script via cron to update totals that become incorrect. Thanks, --nate On Nov 30, 2011, at 2:24 PM, Mary Anne Alliegro wrote: Hi Galaxy Users, I have permanently deleted numerous files. My usage % has decreased, but this is NOT reflected in my Gb report (upper right)- it remains the same. Am I missing some phantom trash bin? If not, will Galaxy recalculate my Gb usage so that I may proceed with my project? Thank you, Mary Anne pastedGraphic.pdf ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ -- Glen L. Beane Senior Software Engineer The Jackson Laboratory (207) 288-6153 ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Quota will not decrease with permanent delete
You aren't the only one having problem in the disk-usage reported in the upper right. I've also had problems with this total disk usage in our local instance of Galaxy I've deleted all but a handful of files, and we have a cron job that purges the files from disk after a certain number of days. I've deleted everything but one history, and for the last few weeks the size of that history has been reported as about 200GB. I don't have any other histories, or any deleted but not purged datasets. However the total disk usage shown by Galaxy is about 1.5TB On Nov 30, 2011, at 2:24 PM, Mary Anne Alliegro wrote: Hi Galaxy Users, I have permanently deleted numerous files. My usage % has decreased, but this is NOT reflected in my Gb report (upper right)- it remains the same. Am I missing some phantom trash bin? If not, will Galaxy recalculate my Gb usage so that I may proceed with my project? Thank you, Mary Anne pastedGraphic.pdf ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ -- Glen L. Beane Senior Software Engineer The Jackson Laboratory (207) 288-6153 ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
