I have attached another patch to AVRO-1499 that solves a big unicode encoding problem. Because we are using string.size instead of string.bytesize, we end up writing the number of characters instead of the number of bytes in Ruby 1.9+. I also took the liberty of removing the collect_hash mixin. If someone could review it, that would be great.
I had to set up a VM to test this against Ruby 1.8, because I can’t install Ruby 1.8 on my OSX work station anymore.** There is still a failing test on Ruby 2.0+, due to encoding of union[double, long] types. I have fixed that in my fork as well, but that one is a bit more involved. A CI server that runs against different the different language versions would be a great addition to this project, to make verifying changes less of a pain. Willem ** This s the main reason why I suggest dropping support for Ruby 1.8. Yes, it is possible to build the gem in a way to be compatible with Ruby 1.8 and up, but it is really deterring people from becoming contributors. On Jun 26, 2014, at 8:01 AM, Martin Kleppmann <mar...@kleppmann.com> wrote: > IMHO it's not a big problem to support many versions of Ruby in a single gem. > I would much rather have one gem that works in all widespread versions of > Ruby (1.8.7, 1.9.3, 2.0, 2.1) than the complexity of multiple branches or > forks. > > On the whole, Avro should work in modern versions of Ruby; if anything is > broken, we should be able to fix it without breaking compatibility with 1.8. > For example, this patch I posted a while ago fixes a major encoding issue > under Ruby 2.0:https://issues.apache.org/jira/browse/AVRO-1499 -- I'd love > someone to give it a review please, so that we can get it committed. > > Continuing to support Ruby 1.8 means we can't use Ruby 1.9 language features > (e.g. the compact syntax for hashes with symbol keys) which is a shame, but > not a major problem IMHO. All the unicode handling can be dynamically > enabled/disabled at runtime depending on whether the Ruby VM supports it. > > Willem, it would be great to get your changes merged upstream. I'm happy to > help you get them committed. Is there something in particular that is forcing > you to drop 1.8 support? > > As long as Avro works in modern versions of Ruby, and supporting Ruby 1.8.7 > isn't overly burdensome, I think it would be ok to drop Ruby 1.8.7 support in > Avro 1.8.0, but keep it until then. > > Martin > > On 26 Jun 2014, at 03:46, Sean Busbey <bus...@cloudera.com> wrote: >> Personally, I'd rather see #2. >> >> I think it's very hard to know what the current use of Ruby 1.8 is. Support >> from the MRI community only ended ~1 year ago[1]. JRuby still supports >> running in 1.8 mode. They'll be dropping it in their next major release, >> but there isn't a schedule for that yet and they expect the current major >> release line to continue for some time after that[2]. Additionally, Heroku >> won't be ending support until the end of this month[3]. Even after that, >> it's not clear to me that they won't allow users to keep using it. >> >> As mentioned previously I'm a JRuby-in-1.9-mode user and I usually just >> work with the Java libraries directly. So this won't directly impact me, >> but I agree that it sucks when upgrades break things. So I don't feel like >> #1 is an option. >> >> We could also investigate maintaining a single gem that just had two >> implementations with in it, with the active one determined by the Ruby >> version. >> >> [1]: https://www.ruby-lang.org/en/news/2013/06/30/we-retire-1-8-7/ >> [2]: https://groups.google.com/d/msg/jruby-users/qmLpZ7qDwZo/J_iYViplcq4J >> [3]: https://devcenter.heroku.com/articles/ruby-support#ruby-versions >> >> -Sean >> >> On Wed, Jun 25, 2014 at 6:41 PM, Doug Cutting <cutt...@apache.org> wrote: >> >>> On Wed, Jun 25, 2014 at 2:16 AM, Willem van Bergen <wil...@vanbergen.org> >>> wrote: >>>> It's possible to have 2 different gems, but this is not very common in >>> the Ruby world. >>>> Because Ruby 1.8 is not maintained anymore, not even for security >>> issues, most >>>> people have moved on to newer versions. >>> >>> I can see a couple of options: >>> 1. Assume that no one actually uses Ruby 1.8 anymore, and upgrade to >>> 1.9 in Avro 1.7.7. A change that doesn't break anyone isn't >>> incompatible. >>> 2. Assume some folks still use Ruby 1.8 and add a ruby1.9 fork in >>> Avro 1.7.7. Ruby users who upgrade to Avro 1.7.7 would need to opt-in >>> to the Ruby 1.9 version. >>> 3. Wait until we release 1.8.0 to upgrade Avro to support Ruby 1.9. >>> >>> (3) seems like a bad option unless we're confident we're going to >>> release a 1.8.0 soon, which I am not. Folks hate getting broken by >>> upgrades. Avro is a dependency of a lot of Java applications. Having >>> an incompatible release makes it hard for one component to upgrade >>> without forcing all to upgrade. Either you end up with a broken stack >>> or with one that can never upgrade. Which of (1) or (2) seems more >>> palatable to Ruby folks? Are there other options? >>> >>> Doug >>> >> >> >> >> -- >> Sean >
