Re: Heroku technical questions
Oren, That's a fair enough answer. :-) I'll give it a shot. Thanks again for your help. -Chris On Tue, Mar 16, 2010 at 10:59 AM, Oren Teich wrote: > I don't know the technical details of how DNS works well enough. Either try > it with a throw away domain, maybe someone here knows, or contact support. > Oren > > On Mon, Mar 15, 2010 at 5:50 PM, Chris wrote: >> >> Oren, >> Thanks for the response. I'm still not totally sure of one thing >> though. Your response to the DNS question leads me to believe that I >> can have specified subdomains OR wildcard domains, but NOT both >> specified AND wildcard (for a single domain). >> If you could clarify this I would appreciate it. >> >> Thanks, >> -Chris >> >> >> >> >> 9) DNS related: Is it possible to have say 2 set subdomains for a >> >> given domain (ex: forums.mydomain.com, chat.mydomain.com) that point >> >> to separate apps, and then have a catchall domain point to a 3rd app? >> >> Ex: subdomainXYZ.mydomain.com points to a given app for any value of >> >> XYZ. The docs seem to indicate that I need to use the heroku console >> >> to add any subdomains, so what I'm looking for is how I would add a >> >> catchall domain in Heroku. >> >> >> > >> > Wildcard domain, or add each domain independently, your choice. >> > >> > Oren >> > >> >> -- >> You received this message because you are subscribed to the Google Groups >> "Heroku" group. >> To post to this group, send email to her...@googlegroups.com. >> To unsubscribe from this group, send email to >> heroku+unsubscr...@googlegroups.com. >> For more options, visit this group at >> http://groups.google.com/group/heroku?hl=en. >> > > -- > You received this message because you are subscribed to the Google Groups > "Heroku" group. > To post to this group, send email to her...@googlegroups.com. > To unsubscribe from this group, send email to > heroku+unsubscr...@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/heroku?hl=en. > -- You received this message because you are subscribed to the Google Groups "Heroku" group. To post to this group, send email to her...@googlegroups.com. To unsubscribe from this group, send email to heroku+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/heroku?hl=en.
Re: Heroku technical questions
I don't know the technical details of how DNS works well enough. Either try it with a throw away domain, maybe someone here knows, or contact support. Oren On Mon, Mar 15, 2010 at 5:50 PM, Chris wrote: > Oren, > Thanks for the response. I'm still not totally sure of one thing > though. Your response to the DNS question leads me to believe that I > can have specified subdomains OR wildcard domains, but NOT both > specified AND wildcard (for a single domain). > If you could clarify this I would appreciate it. > > Thanks, > -Chris > > >> > >> 9) DNS related: Is it possible to have say 2 set subdomains for a > >> given domain (ex: forums.mydomain.com, chat.mydomain.com) that point > >> to separate apps, and then have a catchall domain point to a 3rd app? > >> Ex: subdomainXYZ.mydomain.com points to a given app for any value of > >> XYZ. The docs seem to indicate that I need to use the heroku console > >> to add any subdomains, so what I'm looking for is how I would add a > >> catchall domain in Heroku. > >> > > > > Wildcard domain, or add each domain independently, your choice. > > > > Oren > > > > -- > You received this message because you are subscribed to the Google Groups > "Heroku" group. > To post to this group, send email to her...@googlegroups.com. > To unsubscribe from this group, send email to > heroku+unsubscr...@googlegroups.com > . > For more options, visit this group at > http://groups.google.com/group/heroku?hl=en. > > -- You received this message because you are subscribed to the Google Groups "Heroku" group. To post to this group, send email to her...@googlegroups.com. To unsubscribe from this group, send email to heroku+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/heroku?hl=en.
Re: pull/push in one step?
+1 An additional option that I would like to see that would make import/export of databases easier is to be able to do a database dump (ex: pg_dump -o app_db_name > "some s3 bucket/key") where the database gets dumped from the Heroku Postgresql server to an S3 bucket (and the reverse for pulling a database into Heroku). This would allow us to use S3Fox or similar app to upload a big database to S3 and then have Heroku pull it in bypassing Taps. Of course this would necessitate the end user using Postgresql for their development environment, but my personal opinion is that anyone who is dealing with huge datasets should be doing that already anyway if they are planning on using Heroku. This would also allow for importing of single tables which is currently a problem on Heroku. If doing a 'heroku db:copy app1 app2' is not feasible directly on Heroku servers without first pulling the database down locally (for various technical reasons) I personally would still prefer to dump to S3 rather than pull local since that should still be significantly faster. My $0.02. -Chris On Tue, Mar 16, 2010 at 8:03 AM, Daniele wrote: > +1 > > Really nice feature! > > On 15 Mar, 18:57, Mike Doel wrote: >> I imagine we're not alone in having both a staging and production >> environment up on heroku. We frequently use heroku db:pull to grab the >> production database and will occasionally put it up to our staging >> environment with heroku db:push. >> >> Is it possible to do this in one step? Something like: >> >> heroku db:copy app1 app2 >> >> Ideally, this would happen without having to download locally and uploading >> as separate steps (i.e. direct server-to-server transfer). > > -- > You received this message because you are subscribed to the Google Groups > "Heroku" group. > To post to this group, send email to her...@googlegroups.com. > To unsubscribe from this group, send email to > heroku+unsubscr...@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/heroku?hl=en. > > -- You received this message because you are subscribed to the Google Groups "Heroku" group. To post to this group, send email to her...@googlegroups.com. To unsubscribe from this group, send email to heroku+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/heroku?hl=en.
Re: Heroku technical questions
Oren, Thanks for the response. I'm still not totally sure of one thing though. Your response to the DNS question leads me to believe that I can have specified subdomains OR wildcard domains, but NOT both specified AND wildcard (for a single domain). If you could clarify this I would appreciate it. Thanks, -Chris >> >> 9) DNS related: Is it possible to have say 2 set subdomains for a >> given domain (ex: forums.mydomain.com, chat.mydomain.com) that point >> to separate apps, and then have a catchall domain point to a 3rd app? >> Ex: subdomainXYZ.mydomain.com points to a given app for any value of >> XYZ. The docs seem to indicate that I need to use the heroku console >> to add any subdomains, so what I'm looking for is how I would add a >> catchall domain in Heroku. >> > > Wildcard domain, or add each domain independently, your choice. > > Oren > -- You received this message because you are subscribed to the Google Groups "Heroku" group. To post to this group, send email to her...@googlegroups.com. To unsubscribe from this group, send email to heroku+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/heroku?hl=en.
Re: pull/push in one step?
+1 Really nice feature! On 15 Mar, 18:57, Mike Doel wrote: > I imagine we're not alone in having both a staging and production environment > up on heroku. We frequently use heroku db:pull to grab the production > database and will occasionally put it up to our staging environment with > heroku db:push. > > Is it possible to do this in one step? Something like: > > heroku db:copy app1 app2 > > Ideally, this would happen without having to download locally and uploading > as separate steps (i.e. direct server-to-server transfer). -- You received this message because you are subscribed to the Google Groups "Heroku" group. To post to this group, send email to her...@googlegroups.com. To unsubscribe from this group, send email to heroku+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/heroku?hl=en.
Re: Best way to add a huge dataset?
What I've been told for tmp folder size is about 1gb but it's a soft limit. -Terence On Mon, 2010-03-15 at 08:52 -0700, Mike wrote: > Also in that link you linked to: > Slug Size: 500MB - Hard > > Man, that probably includes the temp directory. > > Getting data onto Heroku is pretty friggin hard > > On Mar 15, 9:27 am, Daniele wrote: > > On 15 Mar, 01:13, Mike wrote: > > > > > That's a really good idea on having a controller take the upload into > > > temp. Is there a size limit on the temp directory? > > > > I don't know, sorry. > > > > > Or a time limit on > > > how long a dyno can be locked to a single upload before being > > > restarted? > > > > "Request Length: 30 seconds - Hard" (http://legal.heroku.com/aup) > > > > A bit low timeout for a middle-size upload... > -- You received this message because you are subscribed to the Google Groups "Heroku" group. To post to this group, send email to her...@googlegroups.com. To unsubscribe from this group, send email to heroku+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/heroku?hl=en.
Re: Using Zerigo to simplify S3 urls
Thanks for that Matt. It sees to say that I should use something like docs.energiehelpine.co.uk as the bucket name and then map docs.enegiehelpine.co.uk.s3.amazonaws.com to docs.enegiehelpine.co.uk The problem is, I can't figure out how I do that last step using zerigo! Any help much appreciated, DAZ On Mar 13, 6:11 pm, Matt Buck wrote: > http://docs.amazonwebservices.com/AmazonS3/latest/VirtualHosting.html... > > HTH, > Matt Buck > > > > On Sat, Mar 13, 2010 at 10:38 AM, DAZ wrote: > > Hi, > > > I'm storing some pdf documents on Amazon S3. To access them I use urls > > like: > >http://energie.documents.s3.amazonaws.com/news.pdf > > > Is it possible I can configure Zerigo (or anything else) to map > >http://www.energiehelpline.co.uk/documents/ > > > to > > >http://energie.documents.s3.amazonaws.com/ > > > This would help to simplify the urls used for these documents. > > > Thanks, > > > DAZ > > > -- > > You received this message because you are subscribed to the Google Groups > > "Heroku" group. > > To post to this group, send email to her...@googlegroups.com. > > To unsubscribe from this group, send email to > > heroku+unsubscr...@googlegroups.com > > . > > For more options, visit this group at > >http://groups.google.com/group/heroku?hl=en. -- You received this message because you are subscribed to the Google Groups "Heroku" group. To post to this group, send email to her...@googlegroups.com. To unsubscribe from this group, send email to heroku+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/heroku?hl=en.
Cannot install taps
I tried to do "heroku db:pull" in vain with the following message.
= quote start =
$ heroku db:pull
Taps Load Error: Could not find RubyGem taps (>= 0.2.23, < 0.3.0)
You may need to install or update the taps gem to use db commands.
On most systems this will be:
sudo gem install taps
= quote end =
I looked and confirmed that it was not installed.
= quote start =
$ gem list
*** LOCAL GEMS ***
actionmailer (2.3.5, 2.3.4)
actionpack (2.3.5, 2.3.4)
activerecord (2.3.5, 2.3.4, 2.2.2)
activeresource (2.3.5, 2.3.4)
activesupport (2.3.5, 2.3.4, 2.2.2)
configuration (1.1.0)
heroku (1.8.2)
json_pure (1.2.2)
launchy (0.3.5)
mime-types (1.16)
rack (1.1.0, 1.0.1, 1.0.0)
rails (2.3.5, 2.3.4)
rake (0.8.7)
rest-client (1.3.1)
sequel (3.0.0)
sinatra (0.9.2)
thor (0.9.9)
= quote end =
So I did what I had been told with the following error.
= quote start =
$ sudo gem install taps
Building native extensions. This could take a while...
ERROR: Error installing taps:
ERROR: Failed to build gem native extension.
/usr/bin/ruby extconf.rb
checking for fdatasync() in -lrt... no
checking for sqlite3.h... no
*** extconf.rb failed ***
Could not create Makefile due to some reason, probably lack of
necessary libraries and/or headers. Check the mkmf.log file for more
details. You may need configuration options.
Provided configuration options:
--with-opt-dir
--without-opt-dir
--with-opt-include
--without-opt-include=${opt-dir}/include
--with-opt-lib
--without-opt-lib=${opt-dir}/lib
--with-make-prog
--without-make-prog
--srcdir=.
--curdir
--ruby=/usr/bin/ruby
--with-sqlite3-dir
--without-sqlite3-dir
--with-sqlite3-include
--without-sqlite3-include=${sqlite3-dir}/include
--with-sqlite3-lib
--without-sqlite3-lib=${sqlite3-dir}/lib
--with-rtlib
--without-rtlib
Gem files will remain installed in /usr/lib/ruby/gems/1.8/gems/sqlite3-
ruby-1.2.5 for inspection.
Results logged to /usr/lib/ruby/gems/1.8/gems/sqlite3-ruby-1.2.5/ext/
sqlite3_api/gem_make.out
= quote end =
I looked into the "gem_make.out" log, but I found nothing but an error
message exactly same with the above.
I could certainly locate 'sqlite.h'.
= quote start =
$ locate sqlite3.h
/usr/include/sqlite3.h
/usr/share/doc/sqlite-doc-3.6.20/c3ref/sqlite3.html
= quote end =
So I tried the following option in vain, not feeling it was a
appropriate choice.
= quote start =
$ sudo gem install taps -- --without-sqlite3-include=/usr/include/
sqlite3.h
Building native extensions. This could take a while...
ERROR: Error installing taps:
ERROR: Failed to build gem native extension.
/usr/bin/ruby extconf.rb --without-sqlite3-include=/usr/include/
sqlite3.h
checking for fdatasync() in -lrt... no
checking for sqlite3.h... no
*** extconf.rb failed ***
Could not create Makefile due to some reason, probably lack of
necessary libraries and/or headers. Check the mkmf.log file for more
details. You may need configuration options.
(... The rest is same as the above)
= quote end =
The platform is as follows:
Fedora release 12 (Constantine)
Linux 2.6.31.12-174.2.3.fc12.i686 #1 SMP Mon Jan 18 20:22:46
UTC 2010 i686 i686 i386 GNU/Linux
I'll appreciate if somebody can kindly help me to solve the problem.
Thank you in advance.
--
You received this message because you are subscribed to the Google Groups
"Heroku" group.
To post to this group, send email to her...@googlegroups.com.
To unsubscribe from this group, send email to
heroku+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/heroku?hl=en.
pull/push in one step?
I imagine we're not alone in having both a staging and production environment up on heroku. We frequently use heroku db:pull to grab the production database and will occasionally put it up to our staging environment with heroku db:push. Is it possible to do this in one step? Something like: heroku db:copy app1 app2 Ideally, this would happen without having to download locally and uploading as separate steps (i.e. direct server-to-server transfer). Mike -- You received this message because you are subscribed to the Google Groups "Heroku" group. To post to this group, send email to her...@googlegroups.com. To unsubscribe from this group, send email to heroku+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/heroku?hl=en.
Re: Heroku technical questions
Hrm, I seem to have lost the first few repies. 1) you can see an archive of all issues here: http://status.heroku.com/past 2) Sigh, I had a long long response here. Short version - we've got advanced monitoring. The pagers went off before any user knew about the problem. We were working on it within 2 m On Mon, Mar 15, 2010 at 9:44 AM, Oren Teich wrote: > at this time. >> [/QUOTE] >> >> However, my app would still not start at Mar 14, 2010 - 4:25 UTC. >> > > That sounds odd for sure. In the future, make sure to submit a ticket if > you're seeing errors. We had no other reports that I'm aware of. It's > possible you got unlucky somehow. > > >> So, what exactly is the definition of "New and idled apps". My >> understanding is that as long as I'm paying for dynos that my app will >> not go idle, but if I increase the dynos from say 4 to 10 are the 6 >> new dynos considered 'new' and won't start in such a situation? What >> about workers? > > >> What about accidentally pushing a 'new' version of code during such an >> outage? I assume this would effectively bring my running app offline. >> > > When issues like this occur, the system automatically identifies it, and > prevents users from hurting things. You can't push, you can't change your > dynos or workers. We make sure your app continues to run while we find and > resolve the issue. > > >> >> 4) If I have a running app that is being concurrently hit by say 100 >> users. What is the exact process the server takes when I push a new >> version of code? In other words, do those 100 users immediately get >> disconnected, or does Heroku spin up the new app, redirecting all new >> requests to the new version once it is online, and then terminate the >> 'old' version dynos after all existing requests to those dynos have >> been satisfied? Basically, what is the user experience for someone >> who is in the middle of submitting their credit card info at the >> precise time that I push a new version of code? >> > > The latter. Open request are served on the old dynos, which are then > replaced with new ones. Users don't see anything. > > >> >> 5) I'm a little concerned about recent posts dealing with getting >> large datasets in and out of the database on Heroku. I've read plenty >> of creative ideas on how to get the data in and out, but nothing that >> would really be considered acceptable for a high traffic site that >> needs virtually zero downtime (especially with the posts saying that >> Taps gets really slow after about 500 MB). Does Heroku have any plans >> of making it easier to get large datasets in and out of the database? >> What about scaling the database? It's easy to spin up app dynos, but >> what about moving from a Fugu database to a Zilla? What kind of >> downtime are we looking at for such a transition? (How about from >> Ronin -> Fugu?) The faqs state: "Switching to/from a dedicated >> database usually takes one business day as our support staff processes >> the requests, and verifies that data migrations completed >> successfully." I assume this is for switching between say Koi and >> Ronin, but is it also true for Ronin->Fugu->Zilla? >> > > There's two issues: one the ingress/outgress of big data. We're working on > improvements and docs for this. > > Two - dedicated DB migrations. We handle this for you, and will coordinate > with you. It takes 1-2 days to do, with a few minutes of down time each > time you switch. That's between any dedicated option. We'll work with you > directly to coordinate. > > >> 6) I assume I would be looking to move away from Heroku long before I >> reached 2TB of data limit on the database (since at that point I would >> be looking for redundant systems that can have read/write traffic >> separated, etc which Heroku doesn't appear to support), but how would >> I get this data off Heroku? >> >> > We haven't had anyone leave due to traffic or size yet. If you start > hitting performance bottlenecks, we'd love to work with you to optimize the > system or make any changes needed. > That said, you can always request a data dump for us if the tools aren't > working for you. > > 7) What systems are used to store the database data? In other words, >> how safe is my data in the event of system failure (hard disk, etc)? >> Single disk, Raid 1, Raid 5, Raid 10, etc? >> > > Very safe. There's a combination of protection at every layer. SW raid > across EBS (which are themselves RAIDed), daily backups of all data, etc. > > >> 8) In the event of system failure, what kind of support can I expect >> from Heroku to get the database back online? (Meaning that a hard >> disk failure with a response of "restore from your backup bundle" >> would not be considered acceptable. Transaction logs would need to be >> replayed, etc.) >> >> > We provide full support for this. We'll take care of it for you. > > >> 9) DNS related: Is it possible to have say 2 set subdomains for a >> given domain (ex: forums
Re: Heroku technical questions
Hrm, I seem to have lost the first few repies. 1) you can see an archive of all issues here: http://status.heroku.com/past 2) Sigh, I had a long long response here. Short version - we've got advanced monitoring. The pagers went off before any user knew about the problem. We were working on it within 5 minutes. Oren On Mon, Mar 15, 2010 at 9:44 AM, Oren Teich wrote: > at this time. >> [/QUOTE] >> >> However, my app would still not start at Mar 14, 2010 - 4:25 UTC. >> > > That sounds odd for sure. In the future, make sure to submit a ticket if > you're seeing errors. We had no other reports that I'm aware of. It's > possible you got unlucky somehow. > > >> So, what exactly is the definition of "New and idled apps". My >> understanding is that as long as I'm paying for dynos that my app will >> not go idle, but if I increase the dynos from say 4 to 10 are the 6 >> new dynos considered 'new' and won't start in such a situation? What >> about workers? > > >> What about accidentally pushing a 'new' version of code during such an >> outage? I assume this would effectively bring my running app offline. >> > > When issues like this occur, the system automatically identifies it, and > prevents users from hurting things. You can't push, you can't change your > dynos or workers. We make sure your app continues to run while we find and > resolve the issue. > > >> >> 4) If I have a running app that is being concurrently hit by say 100 >> users. What is the exact process the server takes when I push a new >> version of code? In other words, do those 100 users immediately get >> disconnected, or does Heroku spin up the new app, redirecting all new >> requests to the new version once it is online, and then terminate the >> 'old' version dynos after all existing requests to those dynos have >> been satisfied? Basically, what is the user experience for someone >> who is in the middle of submitting their credit card info at the >> precise time that I push a new version of code? >> > > The latter. Open request are served on the old dynos, which are then > replaced with new ones. Users don't see anything. > > >> >> 5) I'm a little concerned about recent posts dealing with getting >> large datasets in and out of the database on Heroku. I've read plenty >> of creative ideas on how to get the data in and out, but nothing that >> would really be considered acceptable for a high traffic site that >> needs virtually zero downtime (especially with the posts saying that >> Taps gets really slow after about 500 MB). Does Heroku have any plans >> of making it easier to get large datasets in and out of the database? >> What about scaling the database? It's easy to spin up app dynos, but >> what about moving from a Fugu database to a Zilla? What kind of >> downtime are we looking at for such a transition? (How about from >> Ronin -> Fugu?) The faqs state: "Switching to/from a dedicated >> database usually takes one business day as our support staff processes >> the requests, and verifies that data migrations completed >> successfully." I assume this is for switching between say Koi and >> Ronin, but is it also true for Ronin->Fugu->Zilla? >> > > There's two issues: one the ingress/outgress of big data. We're working on > improvements and docs for this. > > Two - dedicated DB migrations. We handle this for you, and will coordinate > with you. It takes 1-2 days to do, with a few minutes of down time each > time you switch. That's between any dedicated option. We'll work with you > directly to coordinate. > > >> 6) I assume I would be looking to move away from Heroku long before I >> reached 2TB of data limit on the database (since at that point I would >> be looking for redundant systems that can have read/write traffic >> separated, etc which Heroku doesn't appear to support), but how would >> I get this data off Heroku? >> >> > We haven't had anyone leave due to traffic or size yet. If you start > hitting performance bottlenecks, we'd love to work with you to optimize the > system or make any changes needed. > That said, you can always request a data dump for us if the tools aren't > working for you. > > 7) What systems are used to store the database data? In other words, >> how safe is my data in the event of system failure (hard disk, etc)? >> Single disk, Raid 1, Raid 5, Raid 10, etc? >> > > Very safe. There's a combination of protection at every layer. SW raid > across EBS (which are themselves RAIDed), daily backups of all data, etc. > > >> 8) In the event of system failure, what kind of support can I expect >> from Heroku to get the database back online? (Meaning that a hard >> disk failure with a response of "restore from your backup bundle" >> would not be considered acceptable. Transaction logs would need to be >> replayed, etc.) >> >> > We provide full support for this. We'll take care of it for you. > > >> 9) DNS related: Is it possible to have say 2 set subdomains for a >> given domai
Re: Heroku technical questions
Hrm, I seem to have lost the first few repies. 1) you can see an archive of all issues here: http://status.heroku.com/past 2) Sigh, I had a long long response here. Short version - we've got advanced monitoring. The pagers went off before any user knew about the problem. We were working on it within 5 minutes. Oren On Mon, Mar 15, 2010 at 9:44 AM, Oren Teich wrote: > at this time. >> [/QUOTE] >> >> However, my app would still not start at Mar 14, 2010 - 4:25 UTC. >> > > That sounds odd for sure. In the future, make sure to submit a ticket if > you're seeing errors. We had no other reports that I'm aware of. It's > possible you got unlucky somehow. > > >> So, what exactly is the definition of "New and idled apps". My >> understanding is that as long as I'm paying for dynos that my app will >> not go idle, but if I increase the dynos from say 4 to 10 are the 6 >> new dynos considered 'new' and won't start in such a situation? What >> about workers? > > >> What about accidentally pushing a 'new' version of code during such an >> outage? I assume this would effectively bring my running app offline. >> > > When issues like this occur, the system automatically identifies it, and > prevents users from hurting things. You can't push, you can't change your > dynos or workers. We make sure your app continues to run while we find and > resolve the issue. > > >> >> 4) If I have a running app that is being concurrently hit by say 100 >> users. What is the exact process the server takes when I push a new >> version of code? In other words, do those 100 users immediately get >> disconnected, or does Heroku spin up the new app, redirecting all new >> requests to the new version once it is online, and then terminate the >> 'old' version dynos after all existing requests to those dynos have >> been satisfied? Basically, what is the user experience for someone >> who is in the middle of submitting their credit card info at the >> precise time that I push a new version of code? >> > > The latter. Open request are served on the old dynos, which are then > replaced with new ones. Users don't see anything. > > >> >> 5) I'm a little concerned about recent posts dealing with getting >> large datasets in and out of the database on Heroku. I've read plenty >> of creative ideas on how to get the data in and out, but nothing that >> would really be considered acceptable for a high traffic site that >> needs virtually zero downtime (especially with the posts saying that >> Taps gets really slow after about 500 MB). Does Heroku have any plans >> of making it easier to get large datasets in and out of the database? >> What about scaling the database? It's easy to spin up app dynos, but >> what about moving from a Fugu database to a Zilla? What kind of >> downtime are we looking at for such a transition? (How about from >> Ronin -> Fugu?) The faqs state: "Switching to/from a dedicated >> database usually takes one business day as our support staff processes >> the requests, and verifies that data migrations completed >> successfully." I assume this is for switching between say Koi and >> Ronin, but is it also true for Ronin->Fugu->Zilla? >> > > There's two issues: one the ingress/outgress of big data. We're working on > improvements and docs for this. > > Two - dedicated DB migrations. We handle this for you, and will coordinate > with you. It takes 1-2 days to do, with a few minutes of down time each > time you switch. That's between any dedicated option. We'll work with you > directly to coordinate. > > >> 6) I assume I would be looking to move away from Heroku long before I >> reached 2TB of data limit on the database (since at that point I would >> be looking for redundant systems that can have read/write traffic >> separated, etc which Heroku doesn't appear to support), but how would >> I get this data off Heroku? >> >> > We haven't had anyone leave due to traffic or size yet. If you start > hitting performance bottlenecks, we'd love to work with you to optimize the > system or make any changes needed. > That said, you can always request a data dump for us if the tools aren't > working for you. > > 7) What systems are used to store the database data? In other words, >> how safe is my data in the event of system failure (hard disk, etc)? >> Single disk, Raid 1, Raid 5, Raid 10, etc? >> > > Very safe. There's a combination of protection at every layer. SW raid > across EBS (which are themselves RAIDed), daily backups of all data, etc. > > >> 8) In the event of system failure, what kind of support can I expect >> from Heroku to get the database back online? (Meaning that a hard >> disk failure with a response of "restore from your backup bundle" >> would not be considered acceptable. Transaction logs would need to be >> replayed, etc.) >> >> > We provide full support for this. We'll take care of it for you. > > >> 9) DNS related: Is it possible to have say 2 set subdomains for a >> given domai
Re: Heroku technical questions
> > at this time. > [/QUOTE] > > However, my app would still not start at Mar 14, 2010 - 4:25 UTC. > That sounds odd for sure. In the future, make sure to submit a ticket if you're seeing errors. We had no other reports that I'm aware of. It's possible you got unlucky somehow. > So, what exactly is the definition of "New and idled apps". My > understanding is that as long as I'm paying for dynos that my app will > not go idle, but if I increase the dynos from say 4 to 10 are the 6 > new dynos considered 'new' and won't start in such a situation? What > about workers? > What about accidentally pushing a 'new' version of code during such an > outage? I assume this would effectively bring my running app offline. > When issues like this occur, the system automatically identifies it, and prevents users from hurting things. You can't push, you can't change your dynos or workers. We make sure your app continues to run while we find and resolve the issue. > > 4) If I have a running app that is being concurrently hit by say 100 > users. What is the exact process the server takes when I push a new > version of code? In other words, do those 100 users immediately get > disconnected, or does Heroku spin up the new app, redirecting all new > requests to the new version once it is online, and then terminate the > 'old' version dynos after all existing requests to those dynos have > been satisfied? Basically, what is the user experience for someone > who is in the middle of submitting their credit card info at the > precise time that I push a new version of code? > The latter. Open request are served on the old dynos, which are then replaced with new ones. Users don't see anything. > > 5) I'm a little concerned about recent posts dealing with getting > large datasets in and out of the database on Heroku. I've read plenty > of creative ideas on how to get the data in and out, but nothing that > would really be considered acceptable for a high traffic site that > needs virtually zero downtime (especially with the posts saying that > Taps gets really slow after about 500 MB). Does Heroku have any plans > of making it easier to get large datasets in and out of the database? > What about scaling the database? It's easy to spin up app dynos, but > what about moving from a Fugu database to a Zilla? What kind of > downtime are we looking at for such a transition? (How about from > Ronin -> Fugu?) The faqs state: "Switching to/from a dedicated > database usually takes one business day as our support staff processes > the requests, and verifies that data migrations completed > successfully." I assume this is for switching between say Koi and > Ronin, but is it also true for Ronin->Fugu->Zilla? > There's two issues: one the ingress/outgress of big data. We're working on improvements and docs for this. Two - dedicated DB migrations. We handle this for you, and will coordinate with you. It takes 1-2 days to do, with a few minutes of down time each time you switch. That's between any dedicated option. We'll work with you directly to coordinate. > 6) I assume I would be looking to move away from Heroku long before I > reached 2TB of data limit on the database (since at that point I would > be looking for redundant systems that can have read/write traffic > separated, etc which Heroku doesn't appear to support), but how would > I get this data off Heroku? > > We haven't had anyone leave due to traffic or size yet. If you start hitting performance bottlenecks, we'd love to work with you to optimize the system or make any changes needed. That said, you can always request a data dump for us if the tools aren't working for you. 7) What systems are used to store the database data? In other words, > how safe is my data in the event of system failure (hard disk, etc)? > Single disk, Raid 1, Raid 5, Raid 10, etc? > Very safe. There's a combination of protection at every layer. SW raid across EBS (which are themselves RAIDed), daily backups of all data, etc. > 8) In the event of system failure, what kind of support can I expect > from Heroku to get the database back online? (Meaning that a hard > disk failure with a response of "restore from your backup bundle" > would not be considered acceptable. Transaction logs would need to be > replayed, etc.) > > We provide full support for this. We'll take care of it for you. > 9) DNS related: Is it possible to have say 2 set subdomains for a > given domain (ex: forums.mydomain.com, chat.mydomain.com) that point > to separate apps, and then have a catchall domain point to a 3rd app? > Ex: subdomainXYZ.mydomain.com points to a given app for any value of > XYZ. The docs seem to indicate that I need to use the heroku console > to add any subdomains, so what I'm looking for is how I would add a > catchall domain in Heroku. > > Wildcard domain, or add each domain independently, your choice. Oren -- You received this message because you are subscrib
Re: Best way to add a huge dataset?
Also in that link you linked to: Slug Size: 500MB - Hard Man, that probably includes the temp directory. Getting data onto Heroku is pretty friggin hard On Mar 15, 9:27 am, Daniele wrote: > On 15 Mar, 01:13, Mike wrote: > > > That's a really good idea on having a controller take the upload into > > temp. Is there a size limit on the temp directory? > > I don't know, sorry. > > > Or a time limit on > > how long a dyno can be locked to a single upload before being > > restarted? > > "Request Length: 30 seconds - Hard" (http://legal.heroku.com/aup) > > A bit low timeout for a middle-size upload... -- You received this message because you are subscribed to the Google Groups "Heroku" group. To post to this group, send email to her...@googlegroups.com. To unsubscribe from this group, send email to heroku+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/heroku?hl=en.
Re: Best way to add a huge dataset?
On 15 Mar, 01:13, Mike wrote: > That's a really good idea on having a controller take the upload into > temp. Is there a size limit on the temp directory? I don't know, sorry. > Or a time limit on > how long a dyno can be locked to a single upload before being > restarted? "Request Length: 30 seconds - Hard" (http://legal.heroku.com/aup) A bit low timeout for a middle-size upload... -- You received this message because you are subscribed to the Google Groups "Heroku" group. To post to this group, send email to her...@googlegroups.com. To unsubscribe from this group, send email to heroku+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/heroku?hl=en.
