Thank you for the support. My idea was (naively) that I could build from Apache source code the same Hadoop versions as what HDP defined as a release. Specifically the goal is to **not** use HDP artifacts, so not a true clone but a look-a-like. This would give a good transition point to a team that was on HDP to move to BigTop. Then we could help migrate them to a BigTop release in the future.
My goal is to show that BigTop is the successor to HDP. Removing patches from bigtop-packages/src/common/<component>/ only works on your first try. (Clean git pull) After that the patches get sticky.(You can delete them but they still get applied.) If a Bigtop feature is to build any arbitrary version of a hadoop package (Use git to build) should we support version specific patches? I'm happy to help do the work to make this happen, but I want to make sure it's in alignment with the project goals. On Thu, Jun 3, 2021 at 4:44 AM Jun HE <ju...@apache.org> wrote: > I'm a little confused. So the idea is to build source code from HDP using > Bigtop? If that is true, then as Masatake mentioned removing a patch is not > enough. You may need to change do-component-build, install_<component>, and > even those svc and configuration files, to make it work with existing HDP > components. > > Masatake Iwasaki <iwasak...@oss.nttdata.co.jp> 于2021年6月3日周四 下午2:36写道： > > > > I want to build an old version of HDP. > > > > I think you need some considerations. > > > > * Products of HDP x.y.z is expected to be built against products of HDP > > x.y.z. > > The versions of products of HDP and Bigtop do not match. > > You will need to fix scripts of Bigtop for resolving mismatch. > > (do-component-build and install_*.sh tend to be relevant.) > > > > * Even if you succeeded to build Bigtop package from HDP source code, > > the contents should be quite different from HDP one. > > (You should just use .srpm of HDP for building .rpm of HDP if > > available.) > > > > * Artifacts of HDP usually depend on HDP specific artifacts > > published in Hortonworks Maven repositories. > > (This could be resolved by just tweaking ~/.m2/settings.xml or pom.xml > > of products.) > > > > > > > Practical example: *In master:* I want to build spark 2.3.0. If I use > > > bigtop.bom or git to build both fail when they try to apply > > > patch2-snappy-java-1.1.8.diff. > > > > The master branch of Bigtop is expected to build Spark 3 against Hadoop > 3. > > > > > https://github.com/apache/bigtop/commit/f47d4bcb16e67f50b2403f4fc5071b60ec88c081 > > > > Just removing patches should not be enough. > > > > > > On 2021/06/03 10:26, Jun HE wrote: > > > Hi Matt, > > > > > > Are you just trying to disable a certain patch temporarily? If yes, my > > > understanding is that just delete that patch > > > in bigtop-packages/src/common/<component>/ would work. Pls have a try. > > > > > > Regards, > > > > > > Jun > > > > > > Matt Andruff <m...@andruffsolutions.com> 于2021年6月3日周四 上午4:16写道： > > > > > >> Good Day Team, > > >> > > >> I want to build an old version of HDP. Theoretically I should be able > > to > > >> do that with BigTop, but I'm having issues because patches are running > > and > > >> not finding what they're looking for and failing my build. > > >> > > >> How do I turn off a patch intended for a different version (say spark > > >> 2.3.0)? > > >> > > >> Practical example: *In master:* I want to build spark 2.3.0. If I > use > > >> bigtop.bom or git to build both fail when they try to apply > > >> patch2-snappy-java-1.1.8.diff. > > >> Which is totally needed for a fix in some later version of spark, but > > not > > >> the version I'm using. As far as I can tell, there isn't a way to turn > > >> on/off specific version patches, or perhaps this patch isn't following > > the > > >> paradigm. > > >> > > >> Should I open a ticket to create a new functionality that only applies > > >> patches to specific versions? Or is this already baked in and I have > > hit a > > >> subtle bug? > > >> Thoughts? > > >> > > > > > >
