Hello, I put together a JDiff compatibility tool that was committed to trunk. Would someone be kind enough to backport it into 0.94.8?
Here is the commit for the tool: https://github.com/apache/hbase/commit/379f88f0b55b817cfa395326d5da0e3d6552ef28 Here is the review: https://reviews.apache.org/r/10361/ Thanks in advance, Aleks S. On Tue, Apr 9, 2013 at 10:47 AM, Aleksandr Shulman <al...@cloudera.com>wrote: > I put together a tool that leverages Tom White's work. > Here is a review for it: https://reviews.apache.org/r/10361/ > > It will diff the public java api definitions of hbase in two git repos and > generate an HTML report. The tool will reside in the /dev-support folder. > The documentation is inline in the file. > > I'd appreciate your input in how I can make it more useful and usable for > us. > > Once we agree on the definitions of what classes are indeed public, we can > fine-tune this tool to ignore everything else. > > -Aleks S. > > > On Mon, Apr 8, 2013 at 3:55 PM, Todd Lipcon <t...@cloudera.com> wrote: > >> The advantage of the annotations is that Tom White already did the work >> for >> jdiff to ignore non-public classes over in Hadoop land. We could leverage >> that work, whether we re-use the o.a.h.classification annotations or add >> our own copies in org.apache.hbase.*. >> >> -Todd >> >> On Mon, Apr 8, 2013 at 3:08 PM, lars hofhansl <la...@apache.org> wrote: >> >> > It seems we could just generally document that: >> > - no RPC incompatibilities >> > - no API breaking changes to any user facing classes (now we'll pay >> better >> > attention to this) >> > - best effort to keep coprocessor API changes backward compatible >> > >> > If - on the other hand - we wanted to automate API checks then we'd need >> > tagging (either in form of an annotation or Javadoc) >> > >> > +1 on the javadoc tagging if you're willing to take than on. As other >> have >> > said -1 on pulling Interface Audience in. >> > Your set of classes looks good. >> > >> > -- Lars >> > >> > >> > >> > ________________________________ >> > From: Elliott Clark <ecl...@apache.org> >> > To: "dev@hbase.apache.org" <dev@hbase.apache.org> >> > Sent: Monday, April 8, 2013 1:49 PM >> > Subject: Re: Declaring HBase Public API in 0.94 >> > >> > Please don't pull in @InterfaceAudience. Keeping 0.2x compatibility was >> > something that was hard won in 0.94, it would be a real shame to loose >> that >> > now. >> > >> > >> > On Mon, Apr 8, 2013 at 11:07 AM, Aleksandr Shulman <al...@cloudera.com >> > >wrote: >> > >> > > Hi everyone, >> > > >> > > In light of all the conversation on compatibility, I wanted to float >> the >> > > idea of documenting which Java packages, classes, and methods we want >> to >> > > declare as being API compatible in 0.94.x. I'd like your input on: >> > > 1. JavaDoc vs. using AudienceInterface >> > > 2. What the javadoc notation should look like >> > > 3. Which pieces of code should be tagged >> > > >> > > What do I mean by documenting API compatibility? That means that we >> > suggest >> > > the anyone building applications use specific methods because they >> would >> > > continue to be both binary and RPC-compatible going forward. Any >> > > application written, either running on a node of a cluster or on a >> remote >> > > machine, would continue to work properly without recompile for all >> > versions >> > > of 0.94.x running on the cluster. >> > > >> > > *Benefits:* >> > > It would prevent developers from using calls that are subject to >> change. >> > > This would give developers more confidence in using the platform, >> which >> > > will encourage more development on our platform. >> > > 0.94 will still be with us for some time and I think the >> > > better-late-than-never approach will save us pain down the road. >> Finally, >> > > it would allow us to more easily verify that we are in fact API >> > compatible. >> > > >> > > *Can we use AudienceInterface?* >> > > HBase 0.94 can be compiled against both hadoop 0.2x, 1.x, and 2.0.x. >> In >> > the >> > > case of 0.2x, the AudienceInterface classes were not bundled. >> Therefore, >> > we >> > > cannot expect HBase 0.94 to support it. For that reason, I think >> JavaDoc >> > > might be better. >> > > On the other hand, perhaps we might just want to bundle >> AudienceInterface >> > > with 0.94 going forward? Then we can have consistent annotations in >> 0.94, >> > > 0.95, and 0.96 without worrying about the hadoop version. >> > > >> > > Please correct me if I'm wrong about any of the above. >> > > >> > > *Clarification of RPC compatibility:* >> > > We care about RPC compatibility when we create clients that bundle >> their >> > > dependency jars with them. These jars are used to form a request that >> is >> > > executed on a remote machine (i.e. the cluster). If the cluster is >> > upgraded >> > > and no longer recognizes the command, then this will break RPC >> > > compatibility. >> > > >> > > *Clarification of Binary compatibility:* >> > > We care about binary compatibility when a client is created and >> compiled, >> > > and the jars on which is depends change. It should still be able to >> form >> > > requests using those jars. If the cluster is upgraded and the compiled >> > > client code cannot find a method it was depending on to be there, we >> > break >> > > binary compatibility. A recent example is in 0.94.2, where the return >> > type >> > > of HColumnDescriptor.setMaximumVersions was changed and those who >> > upgraded >> > > received this error: >> > > >> > > java.lang.NoSuchMethodError: org.apache.hadoop.hbase.** >> > > HColumnDescriptor.**setMaxVersions(I)V >> > > >> > > *What we currently have:* >> > > We have an @audience annotation set up in 0.95/0.96. In 0.94, I >> suggest >> > > either adding JavaDoc or pulling in the AudienceInterface annotation. >> > > >> > > *Suggested Javadoc language:* >> > > @custom.94_api >> > > >> > > *Granularity:* >> > > Just to the class level. The native java access level (e.g. public, >> > > protected, etc.) should indicate what should be kept compatible. >> > > >> > > *Suggested classes:* >> > > Here is a first-cut of things that should be declared and documented >> as >> > > public APIs. This list was obtained from looking at some MapReduce >> over >> > > HBase example code. >> > > >> > > *JAVA API:* >> > > >> > > *org.apache.hadoop.hbase (some selected classes, see below) >> > > org.apache.hadoop.hbase.client.* >> > > org.apache.hadoop.hbase.filter.* >> > > org.apache.hadoop.hbase.io.hfile.Compression.Algorithm >> > > org.apache.hadoop.hbase.util.* >> > > org.apache.hadoop.hbase.mapreduce.** >> > > >> > > *REST API: >> > > org.apache.hadoop.hbase.rest.client.** >> > > >> > > *Thrift API: >> > > All methods defined in: >> > > /hbase/src/main/resources/org/apache/hadoop/hbase/thrift/Hbase.thrift* >> > > >> > > *Selected classes in org.apache.hadoop.hbase:* >> > > >> > > *import org.apache.hadoop.hbase.ClusterStatus; >> > > import org.apache.hadoop.hbase.HBaseConfiguration; >> > > import org.apache.hadoop.hbase.HColumnDescriptor; >> > > import org.apache.hadoop.hbase.HRegionInfo; >> > > import org.apache.hadoop.hbase.HRegionLocation; >> > > import org.apache.hadoop.hbase.HServerAddress; >> > > import org.apache.hadoop.hbase.HTableDescriptor; >> > > import org.apache.hadoop.hbase.KeyValue;* >> > > >> > > -- >> > > Best Regards, >> > > >> > > Aleks Shulman >> > > 847.814.5804 >> > > Cloudera >> > > >> > >> >> >> >> -- >> Todd Lipcon >> Software Engineer, Cloudera >> > > > > -- > Best Regards, > > Aleks Shulman > 847.814.5804 > Cloudera > -- Best Regards, Aleks Shulman 847.814.5804 Cloudera
