The branch, master has been updated via 8eac5a6bd5f .gitignore: add ignore rules for a few dev tools via 63ea86804fa autobuild: Drop 'py2' flag via 689a1ee747b autobuild: Replace samba-buildpy2-only with samba-nopython-py2 via e11969def18 autobuild: Remove ${EXTRA_PYTHON} variable via 9124e44d710 autobuild: Tidy up unnecessary line-breaks in 'TESTS=' via 8c5a5a58c4e autobuild: Remove the PY3_ONLY variable via 6ba99c36727 autobuild: Update variable name to make more sense via 5f3beb6e410 autobuild: Drop py2 autobuild jobs from 08b91f5e037 gitlab-ci: Rename DOCKER to CONTAINER
https://git.samba.org/?p=samba.git;a=shortlog;h=master - Log ----------------------------------------------------------------- commit 8eac5a6bd5f945f74a29da3904003980dc9532fa Author: Joe Guo <j...@catalyst.net.nz> Date: Fri Feb 8 11:25:44 2019 +1300 .gitignore: add ignore rules for a few dev tools Signed-off-by: Joe Guo <j...@catalyst.net.nz> Reviewed-by: Douglas Bagnall <douglas.bagn...@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abart...@samba.org> Autobuild-User(master): Andrew Bartlett <abart...@samba.org> Autobuild-Date(master): Fri Feb 15 06:07:07 CET 2019 on sn-devel-144 commit 63ea86804fa119f2bad6f376f249aefc1791d120 Author: Tim Beale <timbe...@catalyst.net.nz> Date: Fri Feb 15 12:20:10 2019 +1300 autobuild: Drop 'py2' flag This isn't used any more. It was only being set, never referenced. Signed-off-by: Tim Beale <timbe...@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abart...@samba.org> commit 689a1ee747b3ae1e03bcdae2706ccbb552005a85 Author: Tim Beale <timbe...@catalyst.net.nz> Date: Fri Feb 15 12:17:49 2019 +1300 autobuild: Replace samba-buildpy2-only with samba-nopython-py2 For Samba 4.11, the minimum python2 functionality we will support (for now, at least - we may change our minds) is for the --disable-python target, i.e. if you're excluding all the python functionality from samba, then WAF should still support being built with python2. The use case here is old unix platforms that want to use smbd, but don't have python3 support. Signed-off-by: Tim Beale <timbe...@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abart...@samba.org> commit e11969def18cb0ea803bc4a0ea2eb0fdfed8edee Author: Tim Beale <timbe...@catalyst.net.nz> Date: Fri Feb 15 11:58:51 2019 +1300 autobuild: Remove ${EXTRA_PYTHON} variable We no longer build the python2 bindings, only python3. So we can get rid of this variable now. Signed-off-by: Tim Beale <timbe...@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abart...@samba.org> commit 9124e44d71009861875bd94ead53c8e680757882 Author: Tim Beale <timbe...@catalyst.net.nz> Date: Fri Feb 15 11:45:53 2019 +1300 autobuild: Tidy up unnecessary line-breaks in 'TESTS=' Now that we've dropped the {PY3_ONLY} variable, there's no need for line-breaks in some of the 'TESTS=' values. We can tidy this up a bit. Signed-off-by: Tim Beale <timbe...@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abart...@samba.org> commit 8c5a5a58c4e9904310c89ca9cd9a0cafb040f848 Author: Tim Beale <timbe...@catalyst.net.nz> Date: Fri Feb 15 11:44:21 2019 +1300 autobuild: Remove the PY3_ONLY variable This variable is no longer needed as all the tests run using python3 now. Signed-off-by: Tim Beale <timbe...@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abart...@samba.org> commit 6ba99c367275764f336dc3201c38ba907818d838 Author: Tim Beale <timbe...@catalyst.net.nz> Date: Wed Feb 13 12:41:34 2019 +1300 autobuild: Update variable name to make more sense When we switched from python2 being the default to python3, we didn't update this variable name. It's now handling the python2 case, but it's a boolean flag named 'py3', which is rather confusing. Signed-off-by: Tim Beale <timbe...@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abart...@samba.org> commit 5f3beb6e4109ce0d1a648e80dbf3024040d3c54e Author: Tim Beale <timbe...@catalyst.net.nz> Date: Wed Feb 13 12:12:18 2019 +1300 autobuild: Drop py2 autobuild jobs Samba v4.11 will no longer support python2, so let's drop the autobuild jobs. This will save some gitlab/sn-devel time and money, as it's less work for CI to do. Note that this highlights some previous inconsistencies: - samba-none-env-py2 was being built for gitlab but not sn-devel. - samba-nt4-py2 was being built for sn-devel but not gitlab I've left samba-buildpy2-only for now, which will be addressed in a subsequent patch. Signed-off-by: Tim Beale <timbe...@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abart...@samba.org> ----------------------------------------------------------------------- Summary of changes: .gitignore | 12 +++++ .gitlab-ci-private.yml | 8 ---- .gitlab-ci.yml | 17 +------- script/autobuild.py | 116 ++++++++++++++++++++++--------------------------- 4 files changed, 66 insertions(+), 87 deletions(-) Changeset truncated at 500 lines: diff --git a/.gitignore b/.gitignore index ac05c21ddc8..1b307ca80d6 100644 --- a/.gitignore +++ b/.gitignore @@ -69,3 +69,15 @@ semantic.cache .agignore # generated by configure python/samba/provision/kerberos_implementation.py + +# vagrant +.vagrant/ + +# Visual Studio Code +.vscode/ + +# read by pyenv to auto switch python envs +.python-version + +# read by direnv to auto load environment variables +.envrc diff --git a/.gitlab-ci-private.yml b/.gitlab-ci-private.yml index 4a6e5036f80..4e83c197b89 100644 --- a/.gitlab-ci-private.yml +++ b/.gitlab-ci-private.yml @@ -18,10 +18,6 @@ samba: extends: .private_template # this one takes about 4 hours to finish -samba-py2: - extends: .private_template - # this one takes about 4 hours to finish - samba-fileserver: extends: .private_template # this one takes about 1 hours to finish @@ -30,9 +26,5 @@ samba-ad-dc: extends: .private_template # this one takes about 1 hours to finish -samba-ad-dc-py2: - extends: .private_template - # this one takes about 1 hours to finish - samba-nt4: extends: .private_template diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml index 8dce4af6e0b..d999cf4693f 100644 --- a/.gitlab-ci.yml +++ b/.gitlab-ci.yml @@ -61,11 +61,10 @@ samba-none-env: extends: .shared_template # this one takes about 1 hours to finish -samba-none-env-py2: +samba-nopython: extends: .shared_template - # this one takes about 1 hours to finish -samba-nopython: +samba-nopython-py2: extends: .shared_template samba-systemkrb5: @@ -84,16 +83,9 @@ samba-ad-dc-2: samba-ad-dc-backup: extends: .shared_template -samba-ad-dc-2-py2: - extends: .shared_template - # this one takes about 1 hours to finish - samba-libs: extends: .shared_template -samba-libs-py2: - extends: .shared_template - samba-static: extends: .shared_template @@ -103,12 +95,7 @@ ctdb: samba-ctdb: extends: .shared_template -samba-buildpy2-only: - extends: .shared_template - samba-ad-dc-ntvfs: extends: .shared_template # this one takes about 100 mins to finish -samba-ad-dc-ntvfs-py2: - extends: .shared_template diff --git a/script/autobuild.py b/script/autobuild.py index b43adcfc33a..f2a29497cf2 100755 --- a/script/autobuild.py +++ b/script/autobuild.py @@ -36,27 +36,21 @@ cleanup_list = [] builddirs = { "ctdb": "ctdb", "samba": ".", - "samba-py2": ".", "samba-nt4": ".", - "samba-nt4-py2": ".", "samba-fileserver": ".", "samba-xc": ".", "samba-o3": ".", "samba-ctdb": ".", "samba-libs": ".", - "samba-libs-py2": ".", "samba-static": ".", "samba-none-env": ".", "samba-ad-dc": ".", - "samba-ad-dc-py2": ".", "samba-ad-dc-ntvfs": ".", - "samba-ad-dc-ntvfs-py2": ".", "samba-ad-dc-2": ".", - "samba-ad-dc-2-py2": ".", "samba-ad-dc-backup": ".", "samba-systemkrb5": ".", "samba-nopython": ".", - "samba-buildpy2-only": ".", + "samba-nopython-py2": ".", "ldb": "lib/ldb", "tdb": "lib/tdb", "talloc": "lib/talloc", @@ -71,20 +65,15 @@ if os.environ.get("AUTOBUILD_SKIP_SAMBA_O3", "0") == "1": defaulttasks.remove("samba-o3") ctdb_configure_params = " --enable-developer --picky-developer ${PREFIX}" -samba_configure_params = " --picky-developer ${PREFIX} ${EXTRA_PYTHON} --with-profiling-data" +samba_configure_params = " --picky-developer ${PREFIX} --with-profiling-data" samba_libs_envvars = "PYTHONPATH=${PYTHON_PREFIX}:$PYTHONPATH" samba_libs_envvars += " PKG_CONFIG_PATH=$PKG_CONFIG_PATH:${PREFIX_DIR}/lib/pkgconfig" samba_libs_envvars += " ADDITIONAL_CFLAGS='-Wmissing-prototypes'" samba_libs_configure_base = samba_libs_envvars + " ./configure --abi-check --enable-debug --picky-developer -C ${PREFIX}" -samba_libs_configure_libs = samba_libs_configure_base + " --bundled-libraries=cmocka,popt,NONE ${EXTRA_PYTHON}" +samba_libs_configure_libs = samba_libs_configure_base + " --bundled-libraries=cmocka,popt,NONE" samba_libs_configure_bundled_libs = " --bundled-libraries=!talloc,!pytalloc-util,!tdb,!pytdb,!ldb,!pyldb,!pyldb-util,!tevent,!pytevent,!popt" -samba_libs_configure_samba = samba_libs_configure_base + samba_libs_configure_bundled_libs + " ${EXTRA_PYTHON}" - -if os.environ.get("AUTOBUILD_NO_EXTRA_PYTHON", "0") == "1": - extra_python = "" -else: - extra_python = "--extra-python=/usr/bin/python2" +samba_libs_configure_samba = samba_libs_configure_base + samba_libs_configure_bundled_libs tasks = { "ctdb": [("random-sleep", "../script/random-sleep.sh 60 600", "text/plain"), @@ -101,8 +90,7 @@ tasks = { ("configure", "./configure.developer --with-selftest-prefix=./bin/ab" + samba_configure_params, "text/plain"), ("make", "make -j", "text/plain"), ("test", "make test FAIL_IMMEDIATELY=1 " - "TESTS='${PY3_ONLY}" - "--exclude-env=none " + "TESTS='--exclude-env=none " "--exclude-env=nt4_dc " "--exclude-env=nt4_member " "--exclude-env=ad_dc " @@ -132,8 +120,7 @@ tasks = { ("configure", "./configure.developer --without-ads --with-selftest-prefix=./bin/ab" + samba_configure_params, "text/plain"), ("make", "make -j", "text/plain"), ("test", "make test FAIL_IMMEDIATELY=1 " - "TESTS='${PY3_ONLY}" - "--include-env=nt4_dc --include-env=nt4_member'", "text/plain"), + "TESTS='--include-env=nt4_dc --include-env=nt4_member'", "text/plain"), ("install", "make install", "text/plain"), ("check-clean-tree", "script/clean-source-tree.sh", "text/plain"), ("clean", "make clean", "text/plain")], @@ -142,16 +129,14 @@ tasks = { ("configure", "./configure.developer --without-ad-dc --without-ldap --without-ads --without-json --with-selftest-prefix=./bin/ab" + samba_configure_params, "text/plain"), ("make", "make -j", "text/plain"), ("test", "make test FAIL_IMMEDIATELY=1 " - "TESTS='${PY3_ONLY}" - "--include-env=fileserver'", "text/plain"), + "TESTS='--include-env=fileserver'", "text/plain"), ("check-clean-tree", "script/clean-source-tree.sh", "text/plain")], "samba-ad-dc": [("random-sleep", "script/random-sleep.sh 60 600", "text/plain"), ("configure", "./configure.developer --with-selftest-prefix=./bin/ab" + samba_configure_params, "text/plain"), ("make", "make -j", "text/plain"), ("test", "make test FAIL_IMMEDIATELY=1 " - "TESTS='${PY3_ONLY}" - "--include-env=ad_dc " + "TESTS='--include-env=ad_dc " "--include-env=fl2003dc " "--include-env=fl2008r2dc " "--include-env=ad_member " @@ -163,8 +148,7 @@ tasks = { ("configure", "./configure.developer --with-selftest-prefix=./bin/ab" + samba_configure_params, "text/plain"), ("make", "make -j", "text/plain"), ("test", "make test FAIL_IMMEDIATELY=1 " - "TESTS='${PY3_ONLY}" - "--include-env=chgdcpass " + "TESTS='--include-env=chgdcpass " "--include-env=vampire_2000_dc " "--include-env=fl2000dc " "--include-env=ad_dc_no_nss " @@ -179,9 +163,7 @@ tasks = { ("configure", "./configure.developer --with-selftest-prefix=./bin/ab" + samba_configure_params, "text/plain"), ("make", "make -j", "text/plain"), ("test", "make test FAIL_IMMEDIATELY=1 " - "TESTS='${PY3_ONLY}" - "--include-env=ad_dc_ntvfs " - "'", + "TESTS='--include-env=ad_dc_ntvfs'", "text/plain"), ("check-clean-tree", "script/clean-source-tree.sh", "text/plain")], @@ -191,8 +173,7 @@ tasks = { ("configure", "./configure.developer --with-selftest-prefix=./bin/ab" + samba_configure_params, "text/plain"), ("make", "make -j", "text/plain"), ("test", "make test FAIL_IMMEDIATELY=1 " - "TESTS='${PY3_ONLY}" - "--include-env=backupfromdc " + "TESTS='--include-env=backupfromdc " "--include-env=restoredc " "--include-env=renamedc " "--include-env=offlinebackupdc " @@ -222,8 +203,7 @@ tasks = { ("configure", "ADDITIONAL_CFLAGS='-O3 -Wp,-D_FORTIFY_SOURCE=2' ./configure.developer --with-selftest-prefix=./bin/ab --abi-check-disable" + samba_configure_params, "text/plain"), ("make", "make -j", "text/plain"), ("test", "make quicktest FAIL_IMMEDIATELY=1 " - "TESTS='${PY3_ONLY}" - "--include-env=ad_dc'", "text/plain"), + "TESTS='--include-env=ad_dc'", "text/plain"), ("install", "make install", "text/plain"), ("check-clean-tree", "script/clean-source-tree.sh", "text/plain"), ("clean", "make clean", "text/plain")], @@ -283,8 +263,7 @@ tasks = { ("make", "make -j", "text/plain"), ("test", "make test " "FAIL_IMMEDIATELY=1 " - "TESTS='${PY3_ONLY}" - "--include-env=none'", + "TESTS='--include-env=none'", "text/plain")], "samba-static": [ @@ -314,8 +293,7 @@ tasks = { # we currently cannot run a full make test, a limited list of tests could be run # via "make test TESTS=sometests" ("test", "make test FAIL_IMMEDIATELY=1 " - "TESTS='${PY3_ONLY}" - "--include-env=ktest'", "text/plain"), + "TESTS='--include-env=ktest'", "text/plain"), ("install", "make install", "text/plain"), ("check-clean-tree", "script/clean-source-tree.sh", "text/plain"), ("clean", "make clean", "text/plain") @@ -359,15 +337,46 @@ tasks = { ("libs-clean", "make clean", "text/plain") ], + # check we can do the same thing using python2 + "samba-nopython-py2": [ + ("random-sleep", "script/random-sleep.sh 60 600", "text/plain"), + ("configure", "PYTHON=python2 ./configure.developer --picky-developer ${PREFIX} --with-profiling-data --disable-python --without-ad-dc", "text/plain"), + ("make", "PYTHON=python2 make -j", "text/plain"), + ("install", "PYTHON=python2 make install", "text/plain"), + ("check-clean-tree", "script/clean-source-tree.sh", "text/plain"), + ("clean", "PYTHON=python2 make clean", "text/plain"), + + ("talloc-configure", "cd lib/talloc && PYTHON=python2 " + samba_libs_configure_base + " --bundled-libraries=cmocka,NONE --disable-python", "text/plain"), + ("talloc-make", "cd lib/talloc && PYTHON=python2 make", "text/plain"), + ("talloc-install", "cd lib/talloc && PYTHON=python2 make install", "text/plain"), + + ("tdb-configure", "cd lib/tdb && PYTHON=python2 " + samba_libs_configure_base + " --bundled-libraries=cmocka,NONE --disable-python", "text/plain"), + ("tdb-make", "cd lib/tdb && PYTHON=python2 make", "text/plain"), + ("tdb-install", "cd lib/tdb && PYTHON=python2 make install", "text/plain"), + ("tevent-configure", "cd lib/tevent && PYTHON=python2 " + samba_libs_configure_base + " --bundled-libraries=cmocka,NONE --disable-python", "text/plain"), + ("tevent-make", "cd lib/tevent && PYTHON=python2 make", "text/plain"), + ("tevent-install", "cd lib/tevent && PYTHON=python2 make install", "text/plain"), + + ("ldb-configure", "cd lib/ldb && PYTHON=python2 " + samba_libs_configure_base + " --bundled-libraries=cmocka,NONE --disable-python", "text/plain"), + ("ldb-make", "cd lib/ldb && PYTHON=python2 make", "text/plain"), + ("ldb-install", "cd lib/ldb && PYTHON=python2 make install", "text/plain"), + + # retry against installed library packages + ("libs-configure", "PYTHON=python2 " + samba_libs_configure_base + samba_libs_configure_bundled_libs + " --disable-python --without-ad-dc", "text/plain"), + ("libs-make", "PYTHON=python2 make -j", "text/plain"), + ("libs-install", "PYTHON=python2 make install", "text/plain"), + ("libs-check-clean-tree", "script/clean-source-tree.sh", "text/plain"), + ("libs-clean", "PYTHON=python2 make clean", "text/plain") + ], "ldb": [ ("random-sleep", "../../script/random-sleep.sh 60 600", "text/plain"), - ("configure", "./configure --enable-developer -C ${PREFIX} ${EXTRA_PYTHON}", "text/plain"), + ("configure", "./configure --enable-developer -C ${PREFIX}", "text/plain"), ("make", "make", "text/plain"), ("install", "make install", "text/plain"), ("test", "make test", "text/plain"), - ("configure-no-lmdb", "./configure --enable-developer --without-ldb-lmdb -C ${PREFIX} ${EXTRA_PYTHON}", "text/plain"), + ("configure-no-lmdb", "./configure --enable-developer --without-ldb-lmdb -C ${PREFIX}", "text/plain"), ("make-no-lmdb", "make", "text/plain"), ("install-no-lmdb", "make install", "text/plain"), ("check-clean-tree", "../../script/clean-source-tree.sh", "text/plain"), @@ -376,7 +385,7 @@ tasks = { "tdb": [ ("random-sleep", "../../script/random-sleep.sh 60 600", "text/plain"), - ("configure", "./configure --enable-developer -C ${PREFIX} ${EXTRA_PYTHON}", "text/plain"), + ("configure", "./configure --enable-developer -C ${PREFIX}", "text/plain"), ("make", "make", "text/plain"), ("install", "make install", "text/plain"), ("test", "make test", "text/plain"), @@ -386,7 +395,7 @@ tasks = { "talloc": [ ("random-sleep", "../../script/random-sleep.sh 60 600", "text/plain"), - ("configure", "./configure --enable-developer -C ${PREFIX} ${EXTRA_PYTHON}", "text/plain"), + ("configure", "./configure --enable-developer -C ${PREFIX}", "text/plain"), ("make", "make", "text/plain"), ("install", "make install", "text/plain"), ("test", "make test", "text/plain"), @@ -406,7 +415,7 @@ tasks = { "tevent": [ ("random-sleep", "../../script/random-sleep.sh 60 600", "text/plain"), - ("configure", "./configure --enable-developer -C ${PREFIX} ${EXTRA_PYTHON}", "text/plain"), + ("configure", "./configure --enable-developer -C ${PREFIX}", "text/plain"), ("make", "make", "text/plain"), ("install", "make install", "text/plain"), ("test", "make test", "text/plain"), @@ -425,13 +434,6 @@ tasks = { ("check-clean-tree", "../script/clean-source-tree.sh", "text/plain"), ("clean", "make clean", "text/plain")], - "samba-buildpy2-only": [("random-sleep", "script/random-sleep.sh 60 600", "text/plain"), - ("configure", "PYTHON='python' ./configure.developer --with-selftest-prefix=./bin/ab " + samba_configure_params, "text/plain"), - ("make", "PYTHON='python' make -j", "text/plain"), - ("install", "PYTHON='python' make install", "text/plain"), - ("check-clean-tree", "script/clean-source-tree.sh", "text/plain"), - ("clean", "PYTHON='python' make clean", "text/plain")], - # these are useful for debugging autobuild 'pass': [("pass", 'echo passing && /bin/true', "text/plain")], @@ -463,9 +465,8 @@ def run_cmd(cmd, dir=".", show=None, output=False, checkfail=True): class builder(object): '''handle build of one directory''' - def __init__(self, name, sequence, cp=True, py3=False): + def __init__(self, name, sequence, cp=True): self.name = name - self.py3 = py3 if name in builddirs: self.dir = builddirs[name] else: @@ -504,13 +505,6 @@ class builder(object): (self.stage, self.cmd, self.output_mime_type) = self.sequence[self.next] self.cmd = self.cmd.replace("${PYTHON_PREFIX}", get_python_lib(plat_specific=1, standard_lib=0, prefix=self.prefix)) self.cmd = self.cmd.replace("${PREFIX}", "--prefix=%s" % self.prefix) - if self.py3: - self.cmd = self.cmd.replace("${EXTRA_PYTHON}", "%s" % extra_python) - # The trailing space is important - self.cmd = self.cmd.replace("${PY3_ONLY}", "python2 ") - else: - self.cmd = self.cmd.replace("${EXTRA_PYTHON}", "") - self.cmd = self.cmd.replace("${PY3_ONLY}", "") self.cmd = self.cmd.replace("${PREFIX_DIR}", "%s" % self.prefix) self.cmd = self.cmd.replace("${TESTS}", options.restrict_tests) # if self.output_mime_type == "text/x-subunit": @@ -544,13 +538,7 @@ class buildlist(object): os.environ['AUTOBUILD_RANDOM_SLEEP_OVERRIDE'] = '1' for n in tasknames: - if n not in tasks and n.endswith("-py2"): - b = builder(n, - tasks[n[:-4]], - cp=n is not "pidl", - py3=True) - else: - b = builder(n, tasks[n], cp=n is not "pidl") + b = builder(n, tasks[n], cp=n is not "pidl") self.tlist.append(b) if options.retry: rebase_remote = "rebaseon" -- Samba Shared Repository