Re: Looking for big data sets - improving data handling
we are at about 1GB now but we expect to be at 10 or more in a month. Husain On Wed, Mar 17, 2010 at 4:36 PM, Oren Teich wrote: > Hi all, > we're in the process of improving our data handling, extending taps to make > sure it supports bigger data sets of 10GB or larger. To make sure we're > covering the right situations, I'm looking for people who have big databases > that they can share with us to test with. We'll destroy the data once we're > done, we just want to make sure we cover the situation you are using! > > Let me know if you've got a dataset we can test against. > > Oren > > -- > You received this message because you are subscribed to the Google Groups > "Heroku" group. > To post to this group, send email to her...@googlegroups.com. > To unsubscribe from this group, send email to > heroku+unsubscr...@googlegroups.com > . > For more options, visit this group at > http://groups.google.com/group/heroku?hl=en. > -- You received this message because you are subscribed to the Google Groups "Heroku" group. To post to this group, send email to her...@googlegroups.com. To unsubscribe from this group, send email to heroku+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/heroku?hl=en.
Looking for big data sets - improving data handling
Hi all, we're in the process of improving our data handling, extending taps to make sure it supports bigger data sets of 10GB or larger. To make sure we're covering the right situations, I'm looking for people who have big databases that they can share with us to test with. We'll destroy the data once we're done, we just want to make sure we cover the situation you are using! Let me know if you've got a dataset we can test against. Oren -- You received this message because you are subscribed to the Google Groups "Heroku" group. To post to this group, send email to her...@googlegroups.com. To unsubscribe from this group, send email to heroku+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/heroku?hl=en.
Re: daily cron scheduled time?
You can open a support ticket to request a specific time. On Wed, 2010-03-17 at 09:48 -0700, Jesse wrote: > I am adding the daily cron addon to my app > it seems to schedule the daily cron to run each day, but at the time > of day the addon was enabled > > is there a way to set a specific time of day? > > enable the addon > at 2010-03-17 09:32 PDT > heroku addons:add cron:daily > > check the scheduling > heroku info > > ... > Stack: bamboo-ree-1.8.7 > Data size: ... tables > Next cron: 2010-03-18 09:32 PDT (scheduled) > Last cron: 2010-03-16 17:54 PDT (finished) > Addons: Daily Cron, New Rel... > > ideally I would like to run my cron job at say 10:00pm > my plan right now is to be on my computer at 10:00pm, remove the addon > and then add it back, is there an easier way? > > http://docs.heroku.com/cron > > Thanks, > - Jesse > -- You received this message because you are subscribed to the Google Groups "Heroku" group. To post to this group, send email to her...@googlegroups.com. To unsubscribe from this group, send email to heroku+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/heroku?hl=en.
Re: How to restore a bundle?
But let's say you have a modest sized database (modest relative to the total amounts allowable by Heroku) of 10GB. That way exceeds the 1GB soft limit on the temp directory, and there doesn't seem to be a straightforward way to get 10GB back into somewhere where you could call pg_restore on it is there? On Mar 17, 12:32 pm, Michael wrote: > Both pg_dump and pg_restore can be called from ruby. Try dumping the > database to temp, compressing, and uploading to s3. Restoring is the > same process in reverse. > > On Mar 17, 11:00 am, Mike wrote: > > > I'm going to take the lack of response to indicate that it is not > > possible to restore a bundle. > > > In light of that, is it actually even feasible to run an application > > on Heroku with a data backup policy? > > > I've seen numerous accounts of people out there using cron scripts to > > download bundles as a backup strategy, but while the bundles can help > > you get a copy of your data, what good is that if there's no way to > > use it? > > > There's always db:push, but someone in my other thread indicated that > > the time it requires nears infinity at only 500 MB of data. > > > What is everyone out there doing for backups? > > > On Mar 14, 8:25 pm, Mike wrote: > > > > I've been struggling with how to get a large dataset onto my Heroku > > > application (see my thread > > > here:http://groups.google.com/group/heroku/browse_thread/thread/2ef6fd99a5...). > > > An approach I had thought would be good would be generating a local > > > bundle of some type and uploading it to Heroku where it could be > > > reanimated, However, I'm not sure if this is possible. > > > > This got me thinking about how if that is not possible, whether these > > > bundles could be used for backup at all. They are the advertised > > > preferred way of backing up a Heroku > > > application:http://docs.heroku.com/backups > > > > However, in the documentation, even while it is stated that for best > > > backup procedures data should be backed up in the form of downloading > > > the bundles, there is no mention of how to restore a bundle that has > > > been downloaded. Under the documentation for Heroku's command-line > > > tool, the only bundle commands listed are: bundles, bundles:capture, > > > bundles:download, bundles:animate, and bundles:destroy. > > > > How can I re-upload a bundle to an application and then set the > > > application to overwrite its database with the bundle's information? > > > Is it possible? > > > > And if it is possible, is it possible for me to locally create a > > > bundle in order to use this to upload a large multi-gig database to my > > > application? > > -- You received this message because you are subscribed to the Google Groups "Heroku" group. To post to this group, send email to her...@googlegroups.com. To unsubscribe from this group, send email to heroku+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/heroku?hl=en.
daily cron scheduled time?
I am adding the daily cron addon to my app it seems to schedule the daily cron to run each day, but at the time of day the addon was enabled is there a way to set a specific time of day? enable the addon at 2010-03-17 09:32 PDT heroku addons:add cron:daily check the scheduling heroku info ... Stack: bamboo-ree-1.8.7 Data size: ... tables Next cron: 2010-03-18 09:32 PDT (scheduled) Last cron: 2010-03-16 17:54 PDT (finished) Addons: Daily Cron, New Rel... ideally I would like to run my cron job at say 10:00pm my plan right now is to be on my computer at 10:00pm, remove the addon and then add it back, is there an easier way? http://docs.heroku.com/cron Thanks, - Jesse -- You received this message because you are subscribed to the Google Groups "Heroku" group. To post to this group, send email to her...@googlegroups.com. To unsubscribe from this group, send email to heroku+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/heroku?hl=en.
Re: How to restore a bundle?
Both pg_dump and pg_restore can be called from ruby. Try dumping the database to temp, compressing, and uploading to s3. Restoring is the same process in reverse. On Mar 17, 11:00 am, Mike wrote: > I'm going to take the lack of response to indicate that it is not > possible to restore a bundle. > > In light of that, is it actually even feasible to run an application > on Heroku with a data backup policy? > > I've seen numerous accounts of people out there using cron scripts to > download bundles as a backup strategy, but while the bundles can help > you get a copy of your data, what good is that if there's no way to > use it? > > There's always db:push, but someone in my other thread indicated that > the time it requires nears infinity at only 500 MB of data. > > What is everyone out there doing for backups? > > On Mar 14, 8:25 pm, Mike wrote: > > > > > I've been struggling with how to get a large dataset onto my Heroku > > application (see my thread > > here:http://groups.google.com/group/heroku/browse_thread/thread/2ef6fd99a5...). > > An approach I had thought would be good would be generating a local > > bundle of some type and uploading it to Heroku where it could be > > reanimated, However, I'm not sure if this is possible. > > > This got me thinking about how if that is not possible, whether these > > bundles could be used for backup at all. They are the advertised > > preferred way of backing up a Heroku > > application:http://docs.heroku.com/backups > > > However, in the documentation, even while it is stated that for best > > backup procedures data should be backed up in the form of downloading > > the bundles, there is no mention of how to restore a bundle that has > > been downloaded. Under the documentation for Heroku's command-line > > tool, the only bundle commands listed are: bundles, bundles:capture, > > bundles:download, bundles:animate, and bundles:destroy. > > > How can I re-upload a bundle to an application and then set the > > application to overwrite its database with the bundle's information? > > Is it possible? > > > And if it is possible, is it possible for me to locally create a > > bundle in order to use this to upload a large multi-gig database to my > > application? -- You received this message because you are subscribed to the Google Groups "Heroku" group. To post to this group, send email to her...@googlegroups.com. To unsubscribe from this group, send email to heroku+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/heroku?hl=en.
Re: How to restore a bundle?
I'm going to take the lack of response to indicate that it is not possible to restore a bundle. In light of that, is it actually even feasible to run an application on Heroku with a data backup policy? I've seen numerous accounts of people out there using cron scripts to download bundles as a backup strategy, but while the bundles can help you get a copy of your data, what good is that if there's no way to use it? There's always db:push, but someone in my other thread indicated that the time it requires nears infinity at only 500 MB of data. What is everyone out there doing for backups? On Mar 14, 8:25 pm, Mike wrote: > I've been struggling with how to get a large dataset onto my Heroku > application (see my thread > here:http://groups.google.com/group/heroku/browse_thread/thread/2ef6fd99a5...). > An approach I had thought would be good would be generating a local > bundle of some type and uploading it to Heroku where it could be > reanimated, However, I'm not sure if this is possible. > > This got me thinking about how if that is not possible, whether these > bundles could be used for backup at all. They are the advertised > preferred way of backing up a Heroku > application:http://docs.heroku.com/backups > > However, in the documentation, even while it is stated that for best > backup procedures data should be backed up in the form of downloading > the bundles, there is no mention of how to restore a bundle that has > been downloaded. Under the documentation for Heroku's command-line > tool, the only bundle commands listed are: bundles, bundles:capture, > bundles:download, bundles:animate, and bundles:destroy. > > How can I re-upload a bundle to an application and then set the > application to overwrite its database with the bundle's information? > Is it possible? > > And if it is possible, is it possible for me to locally create a > bundle in order to use this to upload a large multi-gig database to my > application? -- You received this message because you are subscribed to the Google Groups "Heroku" group. To post to this group, send email to her...@googlegroups.com. To unsubscribe from this group, send email to heroku+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/heroku?hl=en.
Re: Cannot install taps
Thank you for your reply, Ben. The command # yum whatprovides */sqlite3.h says that the file is provded by the RPM package "sqlite3-devel" -- is this installed on your system so the Gem can be built against it? The file that exists in your file-system may be a placeholder or something provided by another package. What sqlite* packages have you installed? ( yum list sqlite* ) $ yum list sqlite* gave me: Loaded plugins: presto, refresh-packagekit Installed Packages sqlite.i686 3.6.20-1.fc12 @updates sqlite-devel.i6863.6.20-1.fc12 @updates sqlite-doc.i686 3.6.20-1.fc12 @updates Available Packages sqlite-tcl.i686 3.6.20-1.fc12 updates sqlite2.i686 2.8.17-5.fc12 fedora sqlite2-devel.i686 2.8.17-5.fc12 fedora sqlite2-tcl.i686 2.8.17-5.fc12 fedora sqliteman.i686 1.2.1-2.fc12fedora I'm sorry I have to say that I don't understand "Is this installed on your system so the Gem can be built against it?". ^^^ Do you mean I have to install sqlite3-devel before gem installation? Speaking of the installation order, I installed sqlite3-devel after gem was installed. Thank you, Nao -- You received this message because you are subscribed to the Google Groups "Heroku" group. To post to this group, send email to her...@googlegroups.com. To unsubscribe from this group, send email to heroku+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/heroku?hl=en.
Which Sendgrid has Whitelabeling?
Hi I want to start using Sendgrid and I need their Whitelabel option which is available in their Silver (and above) package. However, given that there does not seem to be a direct correlation between the Heroku Sendgrid AddOns (Free, Pro, Premium) and Sendgrid's own packages (free, Basic, Silver, Gold, Platinum) I'm not sure which addon to purchase in order to get access to Sendgrid Whitelabeling. Can anyone help me out? Cheers Ray -- You received this message because you are subscribed to the Google Groups "Heroku" group. To post to this group, send email to her...@googlegroups.com. To unsubscribe from this group, send email to heroku+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/heroku?hl=en.
