On 10/05/2016 01:01, Gregory Haynes wrote: > > On Mon, May 9, 2016, at 03:54 PM, John Dickinson wrote: >> On 9 May 2016, at 13:16, Gregory Haynes wrote: >>> >>> This is a bit of an aside but I am sure others are wondering the same >>> thing - Is there some info (specs/etherpad/ML thread/etc) that has more >>> details on the bottleneck you're running in to? Given that the only >>> clients of your service are the public facing DNS servers I am now even >>> more surprised that you're hitting a python-inherent bottleneck. >> >> In Swift's case, the summary is that it's hard[0] to write a network >> service in Python that shuffles data between the network and a block >> device (hard drive) and effectively utilizes all of the hardware >> available. So far, we've done very well by fork()'ing child processes, >> using cooperative concurrency via eventlet, and basic "write more >> efficient code" optimizations. However, when it comes down to it, >> managing all of the async operations across many cores and many drives >> is really hard, and there just isn't a good, efficient interface for >> that in Python. > > This is a pretty big difference from hitting an unsolvable performance > issue in the language and instead is a case of language preference - > which is fine. I don't really want to fall in to the language-comparison > trap, but I think more detailed reasoning for why it is preferable over > python in specific use cases we have hit is good info to include / > discuss in the document you're drafting :). Essentially its a matter of > weighing the costs (which lots of people have hit on so I won't) with > the potential benefits and so unless the benefits are made very clear > (especially if those benefits are technical) its pretty hard to evaluate > IMO. > > There seemed to be an assumption in some of the designate rewrite posts > that there is some language-inherent performance issue causing a > bottleneck. If this does actually exist then that is a good reason for > rewriting in another language and is something that would be very useful > to clearly document as a case where we support this type of thing. I am > highly suspicious that this is the case though, but I am trying hard to > keep an open mind...
The way this component works makes it quite difficult to make any major improvement. MiniDNS (the component) takes data and sends a zone transfer every time a recordset gets updated. That is a full (AXFR) zone transfer, so every record in the zone gets sent to each of the DNS servers that end users can hit. This can be quite a large number - ns[1-6].example.com. may well be tens or hundreds of servers behind anycast IPs and load balancers. In many cases, internal zones (or even external zones) can be quite large - I have seen zones that are 200-300Mb. If a zone is high traffic (like say cloud.example.com. where a record is added / removed for each boot / destroy, or the reverse DNS zones for a cloud), there can be a lot of data sent out from this component. We are a small development team, and after looking at our options, and judging the amount of developer hours we had available, a different language was the route we decided on. I was going to go implement a few POCs and see what was most suitable. Golang was then being proposed as a new "blessed" language, and as it was a language that we had a pre-existing POC in we decided to keep this within the potential new list of languages. As I said before, we did not just randomly decide this. We have been talking about it for a while, and at this summit we dedicated an entire session to it, and decided to do it. >> >> Initial results from a golang reimplementation of the object server in >> Python are very positive[1]. We're not proposing to rewrite Swift >> entirely in Golang. Specifically, we're looking at improving object >> replication time in Swift. This service must discover what data is on >> a drive, talk to other servers in the cluster about what they have, >> and coordinate any data sync process that's needed. >> >> [0] Hard, not impossible. Of course, given enough time, we can do >> anything in a Turing-complete language, right? But we're not talking >> about possible, we're talking about efficient tools for the job at >> hand. > > Sorry to be a pedant but that is just plain false - there are plenty of > intractable problems. > >> >> [1] http://d.not.mn/python_vs_golang_gets.png and >> http://d.not.mn/python_vs_golang_puts.png >> >> >> --John > > __________________________________________________________________________ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > __________________________________________________________________________ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev