Sorry for coming back to this thread late. I have some feedback on the compatibility aspects of 2.0.
We are working with a variety of BI vendors to certify Drill and provide native connectors for Drill. Having native access from BI tools helps with seamless experience for the users with performance and functionality. This work is in progress and they are (and will be) working with 1.x versions of Drill as part of the development because thats what we have now. Some of these connectors will be available before 2.0 and some of them can come in post 2.0 as certification is a long process. We don't want to be in a situation where the native connectors are just released by certain BI vendor and the connector is immediately obsolete or doesn't work because we have 2.0 release out now. So the general requirement should be that we maintain backward compatibility with certain number of prior releases. This is very important for the success of the project and adoption by eco system. I am happy to discuss further. -Neeraja On Tue, Apr 5, 2016 at 8:44 AM, Jacques Nadeau <[email protected]> wrote: > I'm going to take this as lazy consensus. I'll create the branch. > > Once created, all merges to the master (1.x branch) should also go to the > v2 branch unless we have a discussion here that they aren't applicable. > When committing, please make sure to commit to both locations. > > thanks, > Jacques > > > -- > Jacques Nadeau > CTO and Co-Founder, Dremio > > On Sat, Mar 26, 2016 at 7:26 PM, Jacques Nadeau <[email protected]> > wrote: > > > Re Compatibility: > > > > I actually don't even think 1.0 clients work with 1.6 server, do they? > > > > I would probably decrease the cross-compatibility requirement burden. A > > nice goal would be cross compatibility across an extended series of > > releases. However, given all the things we've learned in the last year, > we > > shouldn't try to maintain more legacy than is necessary. As such, I > propose > > that we consider the requirement of 2.0 to be: > > > > 1.lastX works with 2.firstX. (For example, if 1.8 is the last minor > > release of the 1.x series, 1.8 would work with 2.0.) > > > > This simplifies testing (we don't have to worry about things like does > 1.1 > > work with 2.3, etc) and gives people an upgrade path as they desire. This > > also allows us to decide what pieces of the compatibility shim go in the > > 2.0 server versus the 1.lastX client. (I actually lean towards allowing a > > full break between v1 and v2 server/client but understand that that level > > or coordination is hard in many organizations since analysts are separate > > from IT). Hopefully, what I'm proposing can be a good compromise between > > progress and deployment ease. > > > > Thoughts? > > > > Re: Branches/Dangers > > > > Good points on this Julian. > > > > How about this: > > > > - small fixes and enhancements PRs should be made against v1 > > - new feature PRs should be made against v2 > > - v2 should continue to always pass all precommit tests during its life > > - v2 becomes master in two months > > > > I definitely don't want to create instability in the v2 branch. > > > > The other option I see is we can only do bug fix releases and branch the > > current master into a maintenance branch and treat master as v2. > > > > Other ideas? > > > > > > -- > > Jacques Nadeau > > CTO and Co-Founder, Dremio > > > > On Sat, Mar 26, 2016 at 6:07 PM, Julian Hyde <[email protected]> wrote: > > > >> Do you plan to be doing significant development on both the v1 and v2 > >> branches, and if so, for how long? I have been bitten badly by that > pattern > >> in the past. Developers put lots of unrelated, destabilizing changes > into > >> v2, it look longer than expected to stabilize v2, product management > lost > >> confidence in v2 and shifted resources back to v1, and v2 never caught > up > >> with v1. > >> > >> One important question: Which branch will you ask people to target for > >> pull requests? v1, v2 or both? If they submit to v2, and v2 is broken, > how > >> will you know whether the patches are good? > >> > >> My recommendation is to choose one of the following: (1) put a strict > >> time limit of say 2 months after which v2 would become the master branch > >> (and v1 master would become a maintenance branch), or (2) make v2 > focused > >> on a particular architectural feature; create multiple independent > feature > >> branches with breaking API changes if you need to. > >> > >> Julian > >> > >> > >> > On Mar 26, 2016, at 1:41 PM, Paul Rogers <[email protected]> > wrote: > >> > > >> > Hi All, > >> > > >> > 2.0 is a good opportunity to enhance our ZK information. See > >> DRILL-4543: Advertise Drill-bit ports, status, capabilities in > ZooKeeper. > >> This change will simplify YARN integration. > >> > > >> > This enhancement will change the “public API” in ZK. To Parth’s point, > >> we can do so in a way that old clients work - as long as a Drill-bit > uses > >> default ports. > >> > > >> > I’ve marked this JIRA as a candidate for 2.0. > >> > > >> > Thanks, > >> > > >> > - Paul > >> > > >> >> On Mar 24, 2016, at 4:11 PM, Parth Chandra <[email protected]> > wrote: > >> >> > >> >> What's our proposal for backward compatibility between 1.x and 2.x? > >> >> My thoughts: > >> >> Optional - Allow a mixture of 1.x and 2.x drillbits in a cluster. > >> >> Required - 1.x clients should be able to talk to 2.x drillbits. > >> >> > >> >> > >> >> > >> >> On Thu, Mar 24, 2016 at 8:55 AM, Jacques Nadeau <[email protected]> > >> wrote: > >> >> > >> >>> There are some changes that either have reviews pending or are in > >> progress > >> >>> that would require breaking changes to Drill core. > >> >>> > >> >>> Examples Include: > >> >>> DRILL-4455 (arrow integration) > >> >>> DRILL-4417 (jdbc/odbc/rpc changes) > >> >>> DRILL-4534 (improve null performance) > >> >>> > >> >>> I've created a new 2.0.0 release version in JIRA and moved these > >> tasks to > >> >>> that umbrella. > >> >>> > >> >>> I'd like to propose a new v2 release branch where we can start > >> >>> incorporating these changes without disrupting v1 stability and > >> >>> compatibility. > >> >>> > >> >>> > >> >>> -- > >> >>> Jacques Nadeau > >> >>> CTO and Co-Founder, Dremio > >> >>> > >> > > >> > >> > > >
