Re: 0.21 found interface but class was expected
Hi Steve, we had a similar problem. We've compiled our code with version 0.21 but included the wrong jars into the classpath. (version 0.20.2; NInputFormat.java). It seems that Hadoop changed this class to an interface, maybe you've a simliar problem. Hope this helps. Sebastian Zitat von Steve Lewis : Cassandra sees this error with 0.21 of hadoop Exception in thread "main" java.lang.IncompatibleClassChangeError: Found interface org.apache.hadoop.mapreduce.JobContext, but class was expected I see something similar Error: Found interface org.apache.hadoop.mapreduce.TaskInputOutputContext, but class was expected I find this especially puzzling since org.apache.hadoop.mapreduce.TaskInputOutputContext IS a class not an interface Does anyone have bright ideas??? -- Steven M. Lewis PhD 4221 105th Ave Ne Kirkland, WA 98033 206-384-1340 (cell) Institute for Systems Biology Seattle WA
Re: 0.21 found interface but class was expected
I have a long rant at http://lordjoesoftware.blogspot.com/ on this but the moral is that there seems to have been a deliberate decision that 0,20 code will may not be comparable with - I have NEVER seen a major library so directly abandon backward compatability On Fri, Nov 12, 2010 at 8:04 AM, Sebastian Schoenherr < sebastian.schoenh...@student.uibk.ac.at> wrote: > Hi Steve, > we had a similar problem. We've compiled our code with version 0.21 but > included the wrong jars into the classpath. (version 0.20.2; > NInputFormat.java). It seems that Hadoop changed this class to an interface, > maybe you've a simliar problem. > Hope this helps. > Sebastian > > > Zitat von Steve Lewis : > > > Cassandra sees this error with 0.21 of hadoop >> >> Exception in thread "main" java.lang.IncompatibleClassChangeError: Found >> interface org.apache.hadoop.mapreduce.JobContext, but class was expected >> >> I see something similar >> Error: Found interface org.apache.hadoop.mapreduce.TaskInputOutputContext, >> but class was expected >> >> I find this especially puzzling >> since org.apache.hadoop.mapreduce.TaskInputOutputContext IS a class not an >> interface >> >> Does anyone have bright ideas??? >> >> -- >> Steven M. Lewis PhD >> 4221 105th Ave Ne >> Kirkland, WA 98033 >> 206-384-1340 (cell) >> Institute for Systems Biology >> Seattle WA >> >> > > > -- Steven M. Lewis PhD 4221 105th Ave Ne Kirkland, WA 98033 206-384-1340 (cell) Institute for Systems Biology Seattle WA
Re: 0.21 found interface but class was expected
As much as I love ranting I can't help but wonder if there were any promises to make 0.21+ be backward compatible with <0.20 ? Just curious? On Sat, Nov 13, 2010 at 02:50PM, Steve Lewis wrote: > I have a long rant at http://lordjoesoftware.blogspot.com/ on this but > the moral is that there seems to have been a deliberate decision that 0,20 > code will may not be comparable with - > I have NEVER seen a major library so directly abandon backward compatability > > > On Fri, Nov 12, 2010 at 8:04 AM, Sebastian Schoenherr < > sebastian.schoenh...@student.uibk.ac.at> wrote: > > > Hi Steve, > > we had a similar problem. We've compiled our code with version 0.21 but > > included the wrong jars into the classpath. (version 0.20.2; > > NInputFormat.java). It seems that Hadoop changed this class to an interface, > > maybe you've a simliar problem. > > Hope this helps. > > Sebastian > > > > > > Zitat von Steve Lewis : > > > > > > Cassandra sees this error with 0.21 of hadoop > >> > >> Exception in thread "main" java.lang.IncompatibleClassChangeError: Found > >> interface org.apache.hadoop.mapreduce.JobContext, but class was expected > >> > >> I see something similar > >> Error: Found interface org.apache.hadoop.mapreduce.TaskInputOutputContext, > >> but class was expected > >> > >> I find this especially puzzling > >> since org.apache.hadoop.mapreduce.TaskInputOutputContext IS a class not an > >> interface > >> > >> Does anyone have bright ideas??? > >> > >> -- > >> Steven M. Lewis PhD > >> 4221 105th Ave Ne > >> Kirkland, WA 98033 > >> 206-384-1340 (cell) > >> Institute for Systems Biology > >> Seattle WA > >> > >> > > > > > > > > > -- > Steven M. Lewis PhD > 4221 105th Ave Ne > Kirkland, WA 98033 > 206-384-1340 (cell) > Institute for Systems Biology > Seattle WA signature.asc Description: Digital signature
Re: 0.21 found interface but class was expected
Java libraries are VERY reluctant to change major classes in a way that breaks backward compatability - NOTE that while the 0.18 packages are deprecated, they are separate from the 0.20 packages allowing 0.18 code to run on 0.20 systems - this is true of virtually all Java libraries On Sat, Nov 13, 2010 at 3:08 PM, Konstantin Boudnik wrote: > As much as I love ranting I can't help but wonder if there were any > promises > to make 0.21+ be backward compatible with <0.20 ? > > Just curious? > > On Sat, Nov 13, 2010 at 02:50PM, Steve Lewis wrote: > > I have a long rant at http://lordjoesoftware.blogspot.com/ on this but > > the moral is that there seems to have been a deliberate decision that > 0,20 > > code will may not be comparable with - > > I have NEVER seen a major library so directly abandon backward > compatability > > > > > > On Fri, Nov 12, 2010 at 8:04 AM, Sebastian Schoenherr < > > sebastian.schoenh...@student.uibk.ac.at> wrote: > > > > > Hi Steve, > > > we had a similar problem. We've compiled our code with version 0.21 but > > > included the wrong jars into the classpath. (version 0.20.2; > > > NInputFormat.java). It seems that Hadoop changed this class to an > interface, > > > maybe you've a simliar problem. > > > Hope this helps. > > > Sebastian > > > > > > > > > Zitat von Steve Lewis : > > > > > > > > > Cassandra sees this error with 0.21 of hadoop > > >> > > >> Exception in thread "main" java.lang.IncompatibleClassChangeError: > Found > > >> interface org.apache.hadoop.mapreduce.JobContext, but class was > expected > > >> > > >> I see something similar > > >> Error: Found interface > org.apache.hadoop.mapreduce.TaskInputOutputContext, > > >> but class was expected > > >> > > >> I find this especially puzzling > > >> since org.apache.hadoop.mapreduce.TaskInputOutputContext IS a class > not an > > >> interface > > >> > > >> Does anyone have bright ideas??? > > >> > > >> -- > > >> Steven M. Lewis PhD > > >> 4221 105th Ave Ne > > >> Kirkland, WA 98033 > > >> 206-384-1340 (cell) > > >> Institute for Systems Biology > > >> Seattle WA > > >> > > >> > > > > > > > > > > > > > > > -- > > Steven M. Lewis PhD > > 4221 105th Ave Ne > > Kirkland, WA 98033 > > 206-384-1340 (cell) > > Institute for Systems Biology > > Seattle WA > > -BEGIN PGP SIGNATURE- > Version: GnuPG v1.4.10 (GNU/Linux) > > iF4EAREIAAYFAkzfGnwACgkQenyFlstYjhK6RwD+IdUVZuqXACV9+9By7fMiy/MO > Uxyt4o4Z4naBzvjMu0cBAMkHLuHFHxuM5Yzb7doeC8eAzq+brsBzVHDKGeUD5FG4 > =dr5x > -END PGP SIGNATURE- > > -- Steven M. Lewis PhD 4221 105th Ave Ne Kirkland, WA 98033 206-384-1340 (cell) Institute for Systems Biology Seattle WA
Re: 0.21 found interface but class was expected
It doesn't answer my question. I guess I will have to look for the answer somewhere else On Sat, Nov 13, 2010 at 03:22PM, Steve Lewis wrote: > Java libraries are VERY reluctant to change major classes in a way that > breaks backward compatability - > NOTE that while the 0.18 packages are deprecated, they are separate from > the 0.20 packages allowing > 0.18 code to run on 0.20 systems - this is true of virtually all Java > libraries > > On Sat, Nov 13, 2010 at 3:08 PM, Konstantin Boudnik wrote: > > > As much as I love ranting I can't help but wonder if there were any > > promises > > to make 0.21+ be backward compatible with <0.20 ? > > > > Just curious? > > > > On Sat, Nov 13, 2010 at 02:50PM, Steve Lewis wrote: > > > I have a long rant at http://lordjoesoftware.blogspot.com/ on this but > > > the moral is that there seems to have been a deliberate decision that > > 0,20 > > > code will may not be comparable with - > > > I have NEVER seen a major library so directly abandon backward > > compatability > > > > > > > > > On Fri, Nov 12, 2010 at 8:04 AM, Sebastian Schoenherr < > > > sebastian.schoenh...@student.uibk.ac.at> wrote: > > > > > > > Hi Steve, > > > > we had a similar problem. We've compiled our code with version 0.21 but > > > > included the wrong jars into the classpath. (version 0.20.2; > > > > NInputFormat.java). It seems that Hadoop changed this class to an > > interface, > > > > maybe you've a simliar problem. > > > > Hope this helps. > > > > Sebastian > > > > > > > > > > > > Zitat von Steve Lewis : > > > > > > > > > > > > Cassandra sees this error with 0.21 of hadoop > > > >> > > > >> Exception in thread "main" java.lang.IncompatibleClassChangeError: > > Found > > > >> interface org.apache.hadoop.mapreduce.JobContext, but class was > > expected > > > >> > > > >> I see something similar > > > >> Error: Found interface > > org.apache.hadoop.mapreduce.TaskInputOutputContext, > > > >> but class was expected > > > >> > > > >> I find this especially puzzling > > > >> since org.apache.hadoop.mapreduce.TaskInputOutputContext IS a class > > not an > > > >> interface > > > >> > > > >> Does anyone have bright ideas??? > > > >> > > > >> -- > > > >> Steven M. Lewis PhD > > > >> 4221 105th Ave Ne > > > >> Kirkland, WA 98033 > > > >> 206-384-1340 (cell) > > > >> Institute for Systems Biology > > > >> Seattle WA > > > >> > > > >> > > > > > > > > > > > > > > > > > > > > > -- > > > Steven M. Lewis PhD > > > 4221 105th Ave Ne > > > Kirkland, WA 98033 > > > 206-384-1340 (cell) > > > Institute for Systems Biology > > > Seattle WA > > > > -BEGIN PGP SIGNATURE- > > Version: GnuPG v1.4.10 (GNU/Linux) > > > > iF4EAREIAAYFAkzfGnwACgkQenyFlstYjhK6RwD+IdUVZuqXACV9+9By7fMiy/MO > > Uxyt4o4Z4naBzvjMu0cBAMkHLuHFHxuM5Yzb7doeC8eAzq+brsBzVHDKGeUD5FG4 > > =dr5x > > -END PGP SIGNATURE- > > > > > > > -- > Steven M. Lewis PhD > 4221 105th Ave Ne > Kirkland, WA 98033 > 206-384-1340 (cell) > Institute for Systems Biology > Seattle WA signature.asc Description: Digital signature
Re: 0.21 found interface but class was expected
It is considered good manners :) Seriously, if you want to attract a community you have an obligation to tell them when you're going to jerk the rug out from under their feet. On Sat, Nov 13, 2010 at 3:27 PM, Konstantin Boudnik wrote: > It doesn't answer my question. I guess I will have to look for the answer > somewhere else > > On Sat, Nov 13, 2010 at 03:22PM, Steve Lewis wrote: >> Java libraries are VERY reluctant to change major classes in a way that >> breaks backward compatability - >> NOTE that while the 0.18 packages are deprecated, they are separate from >> the 0.20 packages allowing >> 0.18 code to run on 0.20 systems - this is true of virtually all Java >> libraries >> >> On Sat, Nov 13, 2010 at 3:08 PM, Konstantin Boudnik wrote: >> >> > As much as I love ranting I can't help but wonder if there were any >> > promises >> > to make 0.21+ be backward compatible with <0.20 ? >> > >> > Just curious? >> > >> > On Sat, Nov 13, 2010 at 02:50PM, Steve Lewis wrote: >> > > I have a long rant at http://lordjoesoftware.blogspot.com/ on this but >> > > the moral is that there seems to have been a deliberate decision that >> > 0,20 >> > > code will may not be comparable with - >> > > I have NEVER seen a major library so directly abandon backward >> > compatability >> > > >> > > >> > > On Fri, Nov 12, 2010 at 8:04 AM, Sebastian Schoenherr < >> > > sebastian.schoenh...@student.uibk.ac.at> wrote: >> > > >> > > > Hi Steve, >> > > > we had a similar problem. We've compiled our code with version 0.21 but >> > > > included the wrong jars into the classpath. (version 0.20.2; >> > > > NInputFormat.java). It seems that Hadoop changed this class to an >> > interface, >> > > > maybe you've a simliar problem. >> > > > Hope this helps. >> > > > Sebastian >> > > > >> > > > >> > > > Zitat von Steve Lewis : >> > > > >> > > > >> > > > Cassandra sees this error with 0.21 of hadoop >> > > >> >> > > >> Exception in thread "main" java.lang.IncompatibleClassChangeError: >> > Found >> > > >> interface org.apache.hadoop.mapreduce.JobContext, but class was >> > expected >> > > >> >> > > >> I see something similar >> > > >> Error: Found interface >> > org.apache.hadoop.mapreduce.TaskInputOutputContext, >> > > >> but class was expected >> > > >> >> > > >> I find this especially puzzling >> > > >> since org.apache.hadoop.mapreduce.TaskInputOutputContext IS a class >> > not an >> > > >> interface >> > > >> >> > > >> Does anyone have bright ideas??? >> > > >> >> > > >> -- >> > > >> Steven M. Lewis PhD >> > > >> 4221 105th Ave Ne >> > > >> Kirkland, WA 98033 >> > > >> 206-384-1340 (cell) >> > > >> Institute for Systems Biology >> > > >> Seattle WA >> > > >> >> > > >> >> > > > >> > > > >> > > > >> > > >> > > >> > > -- >> > > Steven M. Lewis PhD >> > > 4221 105th Ave Ne >> > > Kirkland, WA 98033 >> > > 206-384-1340 (cell) >> > > Institute for Systems Biology >> > > Seattle WA >> > >> > -BEGIN PGP SIGNATURE- >> > Version: GnuPG v1.4.10 (GNU/Linux) >> > >> > iF4EAREIAAYFAkzfGnwACgkQenyFlstYjhK6RwD+IdUVZuqXACV9+9By7fMiy/MO >> > Uxyt4o4Z4naBzvjMu0cBAMkHLuHFHxuM5Yzb7doeC8eAzq+brsBzVHDKGeUD5FG4 >> > =dr5x >> > -END PGP SIGNATURE- >> > >> > >> >> >> -- >> Steven M. Lewis PhD >> 4221 105th Ave Ne >> Kirkland, WA 98033 >> 206-384-1340 (cell) >> Institute for Systems Biology >> Seattle WA > > -BEGIN PGP SIGNATURE- > Version: GnuPG v1.4.10 (GNU/Linux) > > iF4EAREIAAYFAkzfHswACgkQenyFlstYjhKtUAD+Nu/gL5DQ+v9iC89dIaHltvCK > Oa6HOwVWNXWksUxhZhgBAMueLiItX6y4jhCKA5xCOqAmfxA0KZpTkyZr4+ozrazg > =wScC > -END PGP SIGNATURE- > > -- Lance Norskog goks...@gmail.com
Re: 0.21 found interface but class was expected
We do have policies against breaking APIs between consecutive major versions except for very rare exceptions (eg UnixUserGroupInformation went away when security was added). We do *not* have any current policies that existing code can work against different major versions without a recompile in between. Switching an implementation class to an interface is a case where a simple recompile of the dependent app should be sufficient to avoid issues. For whatever reason, the JVM bytecode for invoking an interface method (invokeinterface) is different than invoking a virtual method in a class (invokevirtual). -Todd On Sat, Nov 13, 2010 at 5:28 PM, Lance Norskog wrote: > It is considered good manners :) > > Seriously, if you want to attract a community you have an obligation > to tell them when you're going to jerk the rug out from under their > feet. > > On Sat, Nov 13, 2010 at 3:27 PM, Konstantin Boudnik > wrote: > > It doesn't answer my question. I guess I will have to look for the answer > somewhere else > > > > On Sat, Nov 13, 2010 at 03:22PM, Steve Lewis wrote: > >> Java libraries are VERY reluctant to change major classes in a way that > >> breaks backward compatability - > >> NOTE that while the 0.18 packages are deprecated, they are separate > from > >> the 0.20 packages allowing > >> 0.18 code to run on 0.20 systems - this is true of virtually all Java > >> libraries > >> > >> On Sat, Nov 13, 2010 at 3:08 PM, Konstantin Boudnik > wrote: > >> > >> > As much as I love ranting I can't help but wonder if there were any > >> > promises > >> > to make 0.21+ be backward compatible with <0.20 ? > >> > > >> > Just curious? > >> > > >> > On Sat, Nov 13, 2010 at 02:50PM, Steve Lewis wrote: > >> > > I have a long rant at http://lordjoesoftware.blogspot.com/ on this > but > >> > > the moral is that there seems to have been a deliberate decision > that > >> > 0,20 > >> > > code will may not be comparable with - > >> > > I have NEVER seen a major library so directly abandon backward > >> > compatability > >> > > > >> > > > >> > > On Fri, Nov 12, 2010 at 8:04 AM, Sebastian Schoenherr < > >> > > sebastian.schoenh...@student.uibk.ac.at> wrote: > >> > > > >> > > > Hi Steve, > >> > > > we had a similar problem. We've compiled our code with version > 0.21 but > >> > > > included the wrong jars into the classpath. (version 0.20.2; > >> > > > NInputFormat.java). It seems that Hadoop changed this class to an > >> > interface, > >> > > > maybe you've a simliar problem. > >> > > > Hope this helps. > >> > > > Sebastian > >> > > > > >> > > > > >> > > > Zitat von Steve Lewis : > >> > > > > >> > > > > >> > > > Cassandra sees this error with 0.21 of hadoop > >> > > >> > >> > > >> Exception in thread "main" > java.lang.IncompatibleClassChangeError: > >> > Found > >> > > >> interface org.apache.hadoop.mapreduce.JobContext, but class was > >> > expected > >> > > >> > >> > > >> I see something similar > >> > > >> Error: Found interface > >> > org.apache.hadoop.mapreduce.TaskInputOutputContext, > >> > > >> but class was expected > >> > > >> > >> > > >> I find this especially puzzling > >> > > >> since org.apache.hadoop.mapreduce.TaskInputOutputContext IS a > class > >> > not an > >> > > >> interface > >> > > >> > >> > > >> Does anyone have bright ideas??? > >> > > >> > >> > > >> -- > >> > > >> Steven M. Lewis PhD > >> > > >> 4221 105th Ave Ne > >> > > >> Kirkland, WA 98033 > >> > > >> 206-384-1340 (cell) > >> > > >> Institute for Systems Biology > >> > > >> Seattle WA > >> > > >> > >> > > >> > >> > > > > >> > > > > >> > > > > >> > > > >> > > > >> > > -- > >> > > Steven M. Lewis PhD > >> > > 4221 105th Ave Ne > >> > > Kirkland, WA 98033 > >> > > 206-384-1340 (cell) > >> > > Institute for Systems Biology > >> > > Seattle WA > >> > > >> > -BEGIN PGP SIGNATURE- > >> > Version: GnuPG v1.4.10 (GNU/Linux) > >> > > >> > iF4EAREIAAYFAkzfGnwACgkQenyFlstYjhK6RwD+IdUVZuqXACV9+9By7fMiy/MO > >> > Uxyt4o4Z4naBzvjMu0cBAMkHLuHFHxuM5Yzb7doeC8eAzq+brsBzVHDKGeUD5FG4 > >> > =dr5x > >> > -END PGP SIGNATURE- > >> > > >> > > >> > >> > >> -- > >> Steven M. Lewis PhD > >> 4221 105th Ave Ne > >> Kirkland, WA 98033 > >> 206-384-1340 (cell) > >> Institute for Systems Biology > >> Seattle WA > > > > -BEGIN PGP SIGNATURE- > > Version: GnuPG v1.4.10 (GNU/Linux) > > > > iF4EAREIAAYFAkzfHswACgkQenyFlstYjhKtUAD+Nu/gL5DQ+v9iC89dIaHltvCK > > Oa6HOwVWNXWksUxhZhgBAMueLiItX6y4jhCKA5xCOqAmfxA0KZpTkyZr4+ozrazg > > =wScC > > -END PGP SIGNATURE- > > > > > > > > -- > Lance Norskog > goks...@gmail.com > -- Todd Lipcon Software Engineer, Cloudera
Re: 0.21 found interface but class was expected
On Sat, Nov 13, 2010 at 9:50 PM, Todd Lipcon wrote: > We do have policies against breaking APIs between consecutive major versions > except for very rare exceptions (eg UnixUserGroupInformation went away when > security was added). > > We do *not* have any current policies that existing code can work against > different major versions without a recompile in between. Switching an > implementation class to an interface is a case where a simple recompile of > the dependent app should be sufficient to avoid issues. For whatever reason, > the JVM bytecode for invoking an interface method (invokeinterface) is > different than invoking a virtual method in a class (invokevirtual). > > -Todd > > On Sat, Nov 13, 2010 at 5:28 PM, Lance Norskog wrote: > >> It is considered good manners :) >> >> Seriously, if you want to attract a community you have an obligation >> to tell them when you're going to jerk the rug out from under their >> feet. >> >> On Sat, Nov 13, 2010 at 3:27 PM, Konstantin Boudnik >> wrote: >> > It doesn't answer my question. I guess I will have to look for the answer >> somewhere else >> > >> > On Sat, Nov 13, 2010 at 03:22PM, Steve Lewis wrote: >> >> Java libraries are VERY reluctant to change major classes in a way that >> >> breaks backward compatability - >> >> NOTE that while the 0.18 packages are deprecated, they are separate >> from >> >> the 0.20 packages allowing >> >> 0.18 code to run on 0.20 systems - this is true of virtually all Java >> >> libraries >> >> >> >> On Sat, Nov 13, 2010 at 3:08 PM, Konstantin Boudnik >> wrote: >> >> >> >> > As much as I love ranting I can't help but wonder if there were any >> >> > promises >> >> > to make 0.21+ be backward compatible with <0.20 ? >> >> > >> >> > Just curious? >> >> > >> >> > On Sat, Nov 13, 2010 at 02:50PM, Steve Lewis wrote: >> >> > > I have a long rant at http://lordjoesoftware.blogspot.com/ on this >> but >> >> > > the moral is that there seems to have been a deliberate decision >> that >> >> > 0,20 >> >> > > code will may not be comparable with - >> >> > > I have NEVER seen a major library so directly abandon backward >> >> > compatability >> >> > > >> >> > > >> >> > > On Fri, Nov 12, 2010 at 8:04 AM, Sebastian Schoenherr < >> >> > > sebastian.schoenh...@student.uibk.ac.at> wrote: >> >> > > >> >> > > > Hi Steve, >> >> > > > we had a similar problem. We've compiled our code with version >> 0.21 but >> >> > > > included the wrong jars into the classpath. (version 0.20.2; >> >> > > > NInputFormat.java). It seems that Hadoop changed this class to an >> >> > interface, >> >> > > > maybe you've a simliar problem. >> >> > > > Hope this helps. >> >> > > > Sebastian >> >> > > > >> >> > > > >> >> > > > Zitat von Steve Lewis : >> >> > > > >> >> > > > >> >> > > > Cassandra sees this error with 0.21 of hadoop >> >> > > >> >> >> > > >> Exception in thread "main" >> java.lang.IncompatibleClassChangeError: >> >> > Found >> >> > > >> interface org.apache.hadoop.mapreduce.JobContext, but class was >> >> > expected >> >> > > >> >> >> > > >> I see something similar >> >> > > >> Error: Found interface >> >> > org.apache.hadoop.mapreduce.TaskInputOutputContext, >> >> > > >> but class was expected >> >> > > >> >> >> > > >> I find this especially puzzling >> >> > > >> since org.apache.hadoop.mapreduce.TaskInputOutputContext IS a >> class >> >> > not an >> >> > > >> interface >> >> > > >> >> >> > > >> Does anyone have bright ideas??? >> >> > > >> >> >> > > >> -- >> >> > > >> Steven M. Lewis PhD >> >> > > >> 4221 105th Ave Ne >> >> > > >> Kirkland, WA 98033 >> >> > > >> 206-384-1340 (cell) >> >> > > >> Institute for Systems Biology >> >> > > >> Seattle WA >> >> > > >> >> >> > > >> >> >> > > > >> >> > > > >> >> > > > >> >> > > >> >> > > >> >> > > -- >> >> > > Steven M. Lewis PhD >> >> > > 4221 105th Ave Ne >> >> > > Kirkland, WA 98033 >> >> > > 206-384-1340 (cell) >> >> > > Institute for Systems Biology >> >> > > Seattle WA >> >> > >> >> > -BEGIN PGP SIGNATURE- >> >> > Version: GnuPG v1.4.10 (GNU/Linux) >> >> > >> >> > iF4EAREIAAYFAkzfGnwACgkQenyFlstYjhK6RwD+IdUVZuqXACV9+9By7fMiy/MO >> >> > Uxyt4o4Z4naBzvjMu0cBAMkHLuHFHxuM5Yzb7doeC8eAzq+brsBzVHDKGeUD5FG4 >> >> > =dr5x >> >> > -END PGP SIGNATURE- >> >> > >> >> > >> >> >> >> >> >> -- >> >> Steven M. Lewis PhD >> >> 4221 105th Ave Ne >> >> Kirkland, WA 98033 >> >> 206-384-1340 (cell) >> >> Institute for Systems Biology >> >> Seattle WA >> > >> > -BEGIN PGP SIGNATURE- >> > Version: GnuPG v1.4.10 (GNU/Linux) >> > >> > iF4EAREIAAYFAkzfHswACgkQenyFlstYjhKtUAD+Nu/gL5DQ+v9iC89dIaHltvCK >> > Oa6HOwVWNXWksUxhZhgBAMueLiItX6y4jhCKA5xCOqAmfxA0KZpTkyZr4+ozrazg >> > =wScC >> > -END PGP SIGNATURE- >> > >> > >> >> >> >> -- >> Lance Norskog >> goks...@gmail.com >> > > > > -- > Todd Lipcon > Software Engineer, Cloudera > Cloudera and Yahoo have back ported the interesting security components and some enhancements into their 0.20 based distributions. Out of the box, I know hive does not (yet
Re: 0.21 found interface but class was expected
Oh, thank you Todd! For a second there I thought that Hadoop developers have promised a full binary compatibility (in true Solaris sense of the word). Now I understand that such thing never been promised. Even though Hadoop haven't come over 'major' version change yet. Seriously. Steve, you are talking about leaving and breathing system here. To best of my understanding first stable Hadoop version was suppose to be 1.0 - a major version according to your own terms. Which apparently hasn't came around yet. Now, what exactly you are frustrated about? Cos On Sat, Nov 13, 2010 at 06:50PM, Todd Lipcon wrote: > We do have policies against breaking APIs between consecutive major versions > except for very rare exceptions (eg UnixUserGroupInformation went away when > security was added). > > We do *not* have any current policies that existing code can work against > different major versions without a recompile in between. Switching an > implementation class to an interface is a case where a simple recompile of > the dependent app should be sufficient to avoid issues. For whatever reason, > the JVM bytecode for invoking an interface method (invokeinterface) is > different than invoking a virtual method in a class (invokevirtual). > > -Todd > > On Sat, Nov 13, 2010 at 5:28 PM, Lance Norskog wrote: > > > It is considered good manners :) > > > > Seriously, if you want to attract a community you have an obligation > > to tell them when you're going to jerk the rug out from under their > > feet. > > > > On Sat, Nov 13, 2010 at 3:27 PM, Konstantin Boudnik > > wrote: > > > It doesn't answer my question. I guess I will have to look for the answer > > somewhere else > > > > > > On Sat, Nov 13, 2010 at 03:22PM, Steve Lewis wrote: > > >> Java libraries are VERY reluctant to change major classes in a way that > > >> breaks backward compatability - > > >> NOTE that while the 0.18 packages are deprecated, they are separate > > from > > >> the 0.20 packages allowing > > >> 0.18 code to run on 0.20 systems - this is true of virtually all Java > > >> libraries > > >> > > >> On Sat, Nov 13, 2010 at 3:08 PM, Konstantin Boudnik > > wrote: > > >> > > >> > As much as I love ranting I can't help but wonder if there were any > > >> > promises > > >> > to make 0.21+ be backward compatible with <0.20 ? > > >> > > > >> > Just curious? > > >> > > > >> > On Sat, Nov 13, 2010 at 02:50PM, Steve Lewis wrote: > > >> > > I have a long rant at http://lordjoesoftware.blogspot.com/ on this > > but > > >> > > the moral is that there seems to have been a deliberate decision > > that > > >> > 0,20 > > >> > > code will may not be comparable with - > > >> > > I have NEVER seen a major library so directly abandon backward > > >> > compatability > > >> > > > > >> > > > > >> > > On Fri, Nov 12, 2010 at 8:04 AM, Sebastian Schoenherr < > > >> > > sebastian.schoenh...@student.uibk.ac.at> wrote: > > >> > > > > >> > > > Hi Steve, > > >> > > > we had a similar problem. We've compiled our code with version > > 0.21 but > > >> > > > included the wrong jars into the classpath. (version 0.20.2; > > >> > > > NInputFormat.java). It seems that Hadoop changed this class to an > > >> > interface, > > >> > > > maybe you've a simliar problem. > > >> > > > Hope this helps. > > >> > > > Sebastian > > >> > > > > > >> > > > > > >> > > > Zitat von Steve Lewis : > > >> > > > > > >> > > > > > >> > > > Cassandra sees this error with 0.21 of hadoop > > >> > > >> > > >> > > >> Exception in thread "main" > > java.lang.IncompatibleClassChangeError: > > >> > Found > > >> > > >> interface org.apache.hadoop.mapreduce.JobContext, but class was > > >> > expected > > >> > > >> > > >> > > >> I see something similar > > >> > > >> Error: Found interface > > >> > org.apache.hadoop.mapreduce.TaskInputOutputContext, > > >> > > >> but class was expected > > >> > > >> > > >> > > >> I find this especially puzzling > > >> > > >> since org.apache.hadoop.mapreduce.TaskInputOutputContext IS a > > class > > >> > not an > > >> > > >> interface > > >> > > >> > > >> > > >> Does anyone have bright ideas??? > > >> > > >> > > >> > > >> -- > > >> > > >> Steven M. Lewis PhD > > >> > > >> 4221 105th Ave Ne > > >> > > >> Kirkland, WA 98033 > > >> > > >> 206-384-1340 (cell) > > >> > > >> Institute for Systems Biology > > >> > > >> Seattle WA > > >> > > >> > > >> > > >> > > >> > > > > > >> > > > > > >> > > > > > >> > > > > >> > > > > >> > > -- > > >> > > Steven M. Lewis PhD > > >> > > 4221 105th Ave Ne > > >> > > Kirkland, WA 98033 > > >> > > 206-384-1340 (cell) > > >> > > Institute for Systems Biology > > >> > > Seattle WA > > >> > > > >> > -BEGIN PGP SIGNATURE- > > >> > Version: GnuPG v1.4.10 (GNU/Linux) > > >> > > > >> > iF4EAREIAAYFAkzfGnwACgkQenyFlstYjhK6RwD+IdUVZuqXACV9+9By7fMiy/MO > > >> > Uxyt4o4Z4naBzvjMu0cBAMkHLuHFHxuM5Yzb7doeC8eAzq+brsBzVHDKGeUD5FG4 > > >> > =dr5x > > >> > -END PGP SIGNATURE- > > >> > > > >> > > > >> > > >> > > >> -- > > >> Steven M. Lewis PhD > > >> 4221
Re: 0.21 found interface but class was expected
[Yes, gmail people, this likely went to your junk folder. ] On Nov 13, 2010, at 5:28 PM, Lance Norskog wrote: > It is considered good manners :) > > Seriously, if you want to attract a community you have an obligation > to tell them when you're going to jerk the rug out from under their > feet. The rug has been jerked in various ways for every micro version since as long as I've been with Hadoop. Such jerkings have always (eventually) been for the positive with a happy ending almost every time. No pain, no gain. Oh, one other thing. Here we are, several fairly significant conferences later (both as the main focus and as one of the leading topics) and I still don't understand why people have concerns about "attracting a community". When you have what seems like 100s of companies creating products either built around or integrating Hadoop (the full gamut of several stealth startups to Major Players like IBM), it doesn't really seem like that is much of an issue anymore. At this point, I'm actually in the opposite camp: the community has grown TOO fast to the point that major problems in the source won't be able to be fixed because folks will expect less breakage. This is especially easy for sites with a few hundred nodes (or with enough frosting on top) because everything seems to be working for them. Many of them will not really understand that at super large scales, some things just don't work. In order to fix some of the issues, breakage will occur. The end result can either be a community divided into multiple camps due to forking or a community that has learned to tolerate these minor inconveniences when they pop up. I for one would rather be in the latter, but it sure seems like some parts of the community (and in many ways, the ASF itself) would rather it be the former. Time will tell.
Re: 0.21 found interface but class was expected
On Nov 14, 2010, at 12:05 AM, Allen Wittenauer wrote: > The rug has been jerked in various ways for every micro version since > as long as I've been with Hadoop. s,micro,minor, But i'm sure you knew that.
Re: 0.21 found interface but class was expected
Hi Steve, Sorry to hear about the problems you had. The issue you hit was a result of MAPREDUCE-954, and there was some discussion on that JIRA about compatibility. I believe the thinking was that the context classes are framework classes, so users don't extend/implement them in the normal course of use, and it's also understood that users would recompile their apps (i.e. source compatibility). However, tools like MRUnit which extend/implement these classes do need to be updated when a change like this happens. We tried hard to make 0.21 as backwards compatible with 0.20 as possible, a big part of which was going through all the APIs and annotating their audience and stability (see http://developer.yahoo.com/blogs/hadoop/posts/2010/05/towards_enterpriseclass_compat/ for background). The new MapReduce API (in org.apache.hadoop.mapreduce), which is what we are talking about here, is not yet declared stable (unlike the old API) and these classes are marked with @InterfaceStability.Evolving to show that they can change even between minor releases. I think we could improve visibility to users by publishing a list of incompatible changes in the API for each release - so I've opened HADOOP-7035 for this. Cheers, Tom On Sun, Nov 14, 2010 at 7:41 AM, Konstantin Boudnik wrote: > Oh, thank you Todd! For a second there I thought that Hadoop developers have > promised a full binary compatibility (in true Solaris sense of the word). > > Now I understand that such thing never been promised. Even though Hadoop > haven't come over 'major' version change yet. > > Seriously. Steve, you are talking about leaving and breathing system here. To > best of my understanding first stable Hadoop version was suppose to be 1.0 - a > major version according to your own terms. Which apparently hasn't came around > yet. > > Now, what exactly you are frustrated about? > Cos > > On Sat, Nov 13, 2010 at 06:50PM, Todd Lipcon wrote: >> We do have policies against breaking APIs between consecutive major versions >> except for very rare exceptions (eg UnixUserGroupInformation went away when >> security was added). >> >> We do *not* have any current policies that existing code can work against >> different major versions without a recompile in between. Switching an >> implementation class to an interface is a case where a simple recompile of >> the dependent app should be sufficient to avoid issues. For whatever reason, >> the JVM bytecode for invoking an interface method (invokeinterface) is >> different than invoking a virtual method in a class (invokevirtual). >> >> -Todd >> >> On Sat, Nov 13, 2010 at 5:28 PM, Lance Norskog wrote: >> >> > It is considered good manners :) >> > >> > Seriously, if you want to attract a community you have an obligation >> > to tell them when you're going to jerk the rug out from under their >> > feet. >> > >> > On Sat, Nov 13, 2010 at 3:27 PM, Konstantin Boudnik >> > wrote: >> > > It doesn't answer my question. I guess I will have to look for the answer >> > somewhere else >> > > >> > > On Sat, Nov 13, 2010 at 03:22PM, Steve Lewis wrote: >> > >> Java libraries are VERY reluctant to change major classes in a way that >> > >> breaks backward compatability - >> > >> NOTE that while the 0.18 packages are deprecated, they are separate >> > from >> > >> the 0.20 packages allowing >> > >> 0.18 code to run on 0.20 systems - this is true of virtually all Java >> > >> libraries >> > >> >> > >> On Sat, Nov 13, 2010 at 3:08 PM, Konstantin Boudnik >> > wrote: >> > >> >> > >> > As much as I love ranting I can't help but wonder if there were any >> > >> > promises >> > >> > to make 0.21+ be backward compatible with <0.20 ? >> > >> > >> > >> > Just curious? >> > >> > >> > >> > On Sat, Nov 13, 2010 at 02:50PM, Steve Lewis wrote: >> > >> > > I have a long rant at http://lordjoesoftware.blogspot.com/ on this >> > but >> > >> > > the moral is that there seems to have been a deliberate decision >> > that >> > >> > 0,20 >> > >> > > code will may not be comparable with - >> > >> > > I have NEVER seen a major library so directly abandon backward >> > >> > compatability >> > >> > > >> > >> > > >> > >> > > On Fri, Nov 12, 2010 at 8:04 AM, Sebastian Schoenherr < >> > >> > > sebastian.schoenh...@student.uibk.ac.at> wrote: >> > >> > > >> > >> > > > Hi Steve, >> > >> > > > we had a similar problem. We've compiled our code with version >> > 0.21 but >> > >> > > > included the wrong jars into the classpath. (version 0.20.2; >> > >> > > > NInputFormat.java). It seems that Hadoop changed this class to an >> > >> > interface, >> > >> > > > maybe you've a simliar problem. >> > >> > > > Hope this helps. >> > >> > > > Sebastian >> > >> > > > >> > >> > > > >> > >> > > > Zitat von Steve Lewis : >> > >> > > > >> > >> > > > >> > >> > > > Cassandra sees this error with 0.21 of hadoop >> > >> > > >> >> > >> > > >> Exception in thread "main" >> > java.lang.IncompatibleClassChangeError: >> > >> > Found >> > >> > > >> interface org.apache.hadoo
Re: 0.21 found interface but class was expected
On 14/11/10 08:05, Allen Wittenauer wrote: [Yes, gmail people, this likely went to your junk folder. ] On Nov 13, 2010, at 5:28 PM, Lance Norskog wrote: It is considered good manners :) Seriously, if you want to attract a community you have an obligation to tell them when you're going to jerk the rug out from under their feet. The rug has been jerked in various ways for every micro version since as long as I've been with Hadoop. Such jerkings have always (eventually) been for the positive with a happy ending almost every time. No pain, no gain. I think Steve Lewis is probably miffed at the way something came out and broke things. With 0.20.x being so stable for a while, it's set up expectations about stability that don't match what the developers have been working off, which is "we are a 0.x project and anything not marked as stable isn't". One thing Hadoop is very sensitive about is preserving filesystem data, I haven't heard anything bad there. It's just here we've done some transitions that are forcing recompiles and that has follow on effects. Steve Lewis: one way to stay ahead of these problems is to hook your hudson server up to checking out and building hadoop, then testing your code against it. This is how you can find and report problems before releases. It'd be great to get you involved in this, though I will warn you that merging and retesting against trunk can be quite time consuming at times. Oh, one other thing. Here we are, several fairly significant conferences later (both as the main focus and as one of the leading topics) and I still don't understand why people have concerns about "attracting a community". When you have what seems like 100s of companies creating products either built around or integrating Hadoop (the full gamut of several stealth startups to Major Players like IBM), it doesn't really seem like that is much of an issue anymore. At this point, I'm actually in the opposite camp: the community has grown TOO fast to the point that major problems in the source won't be able to be fixed because folks will expect less breakage. This is especially easy for sites with a few hundred nodes (or with enough frosting on top) because everything seems to be working for them. Many of them will not really understand that at super large scales, some things just don't work. In order to fix some of the issues, breakage will occur. 1. I view a few hundred nodes as large. That's because the nodes are getting bigger, that many machines can still be multi-PB. Even 20-50 nodes is a place for fun, and its where most people are playing. these are the majority of end users, even if the few large clusters run by facebook, Yahoo! and LinkedIn are in a different league. 2. I am with Allan here in that there are big changes we need to get in, and there is a lot of inertia about doing this. But it's the cost of success: there is too much value in files in the filesystem, too much of a tangible cost of any performance problems, that people whose business depends on Hadoop are worried about the costs of changes. In particular, the NN and JT are scale limits in the big clusters, and anything that uses more memory or keeps things around for longer threatens the big clusters, no matter the benefit to the smaller ones. The end result can either be a community divided into multiple camps due to forking or a community that has learned to tolerate these minor inconveniences when they pop up. I for one would rather be in the latter, but it sure seems like some parts of the community (and in many ways, the ASF itself) would rather it be the former. I don't think forking helps, but there might be benefits for -more agility in deployments, better Hadoop on VM work, where even the JT asks for machines as part of its execution plan. -a design of the NN that scales better by having bits of the filesystem handled by other namenodes, so reducing the conflict between the very large clusters (anything with -Xmx32g for the NN heap) and everyone else. -Steve