Hi Scott, I would say that all your errors are from not being able to build *_librsync.so* properly. There is one thing that may help in lines 65-76 of *setup.py*. You can supply *LIBRSYNC_DIR* as an env variable, or on the command line as *--librsync-dir=*. I have *export LIBRSYNC_DIR=/usr/local* in my .bash_profile. That came from *homebrew* installing into /usr/local instead of /usr. Seems none of these installers care much for standards.
Try exporting *LIBRSYNC_DIR* and let me know how it goes after that. ...Ken On Sun, Oct 9, 2022 at 4:46 PM Scott Hannahs <s...@p-hall.net> wrote: > So there are two major failures in testing. One is four failures in the > tahoe backend, (which I don’t use). > > testing/unit/test_backend_instance.py ..............................FF.. [ > 10%] > FF........................ssssssssss [ > 18%] > > > This shows a “Key Error” so is something wrong with my GPG version? > > self = <duplicity.backends.tahoebackend.TAHOEBackend object at 0x10f58deb0> > commandline = 'tahoe cp > /var/folders/6b/85yg9bp94rv55dzbdp78thp40000kr/T/testfiles/local > var:folders/6b/85yg9bp94rv55dzbdp78thp40000kr/T/testfiles/output/file-a' > > def subprocess_popen(self, commandline): > u""" > Execute the given command line with error check. > Returns int Exitcode, string StdOut, string StdErr > > Raise a BackendException on failure. > """ > import shlex > > if isinstance(commandline, (list, tuple)): > logstr = u' '.join(commandline) > args = commandline > else: > logstr = commandline > args = shlex.split(commandline) > > logstr = self.munge_password(logstr) > log.Info(_(u"Reading results of '%s'") % logstr) > > result, stdout, stderr = self.__subprocess_popen(args) > if result != 0: > try: > > ignores = self.popen_breaks[args[0]] > E KeyError: > '/opt/sw/src/fink.build/duplicity-1.0.1-1/duplicity-1.0.1/testing/overrides/bin/tahoe' > > /opt/sw/src/fink.build/duplicity-1.0.1-1/duplicity-1.0.1/duplicity/backend.py:522: > KeyError > > During handling of the above exception, another exception occurred: > > The second testing error is more general, so many of the tests pass and > then starting at the “bad upload” test it all goes south. > > testing/functional/test_badupload.py F [ > 70%] > testing/functional/test_cleanup.py FFF [ > 71%] > testing/functional/test_final.py FFFFFFFFFFFFFFFFFFFFFFFFFFF [ > 77%] > testing/functional/test_log.py F [ > 77%] > testing/functional/test_rdiffdir.py F [ > 77%] > testing/functional/test_replicate.py FFF [ > 78%] > testing/functional/test_restart.py FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF [ > 86%] > testing/functional/test_selection.py .FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF [ > 94%] > FFFFssFFFFFFFssss [ > 98%] > > > This test has a long back trace, But I think that the basic problem is > that duplicity cannot import “librsynch”??? The end of the traceback has > the text "b"ImportError: cannot import name '_librsync' from 'duplicity' > (/opt/sw/src/fink.build/duplicity-1.0.1-1/duplicity-1.0.1/duplicity/__init__.py)" > ...return_val: 1" > > /opt/sw/src/fink.build/duplicity-1.0.1-1/duplicity-1.0.1/duplicity/backend.py:527: > BackendException > _______________________ BadUploadTest.test_missing_file > ________________________ > > self = <testing.functional.test_badupload.BadUploadTest > testMethod=test_missing_file> > > @pytest.mark.slow > def test_missing_file(self): > u""" > Test basic lost file > """ > try: > > self.backup(u"full", > u"{0}/testfiles/dir1".format(_runtest_dir), options=[u"--skip-volume=1"]) > > > /opt/sw/src/fink.build/duplicity-1.0.1-1/duplicity-1.0.1/testing/functional/test_badupload.py:44: > _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ > _ _ _ > > self = <testing.functional.test_badupload.BadUploadTest > testMethod=test_missing_file> > type = 'full' > input_dir = > '/var/folders/6b/85yg9bp94rv55dzbdp78thp40000kr/T/testfiles/dir1' > options = ['full', > '/var/folders/6b/85yg9bp94rv55dzbdp78thp40000kr/T/testfiles/dir1', ' > file:///var/folders/6b/85yg9bp94rv55dzbdp78thp40000kr/T/testfiles/output', > '--volsize', '1', '--skip-volume=1'] > kwargs = {}, before_files = set(), now = 1664937609.016365 > > def backup(self, type, input_dir, options=[], **kwargs): # pylint: > disable=redefined-builtin > u"""Run duplicity backup to default directory""" > options = [type, input_dir, self.backend_url, u"--volsize", u"1"] > + options > before_files = self.get_backend_files() > > # If a chain ends with time X and the next full chain begins at > time X, > # we may trigger an assert in dup_collections.py. If needed, > sleep to > # avoid such problems > now = time.time() > if self.last_backup == int(now): > time.sleep(1) > > > self.run_duplicity(options=options, **kwargs) > > > /opt/sw/src/fink.build/duplicity-1.0.1-1/duplicity-1.0.1/testing/functional/__init__.py:189: > _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ > _ _ _ > > self = <testing.functional.test_badupload.BadUploadTest > testMethod=test_missing_file> > options = ['full', > '/var/folders/6b/85yg9bp94rv55dzbdp78thp40000kr/T/testfiles/dir1', ' > file:///var/folders/6b/85yg9bp94rv55dzbdp78thp40000kr/T/testfiles/output', > '--volsize', '1', '--skip-volume=1'] > current_time = None, fail = None, passphrase_input = [] > > def run_duplicity(self, options=[], current_time=None, fail=None, > passphrase_input=[]): > u""" > Run duplicity binary with given arguments and options > """ > # We run under setsid and take input from /dev/null (below) because > # this way we force a failure if duplicity tries to read from the > # console unexpectedly (like for gpg password or such). > > # Check all string inputs are unicode -- we will convert to system > encoding before running the command > for item in options: > if sys.version_info.major == 2: > assert not isinstance(item, str), u"item " + unicode(item) > + u" in options is not unicode" > > for item in passphrase_input: > assert isinstance(item, u"".__class__), u"item " + > unicode(item) + u" in passphrase_input is not unicode" > > if platform.platform().startswith(u'Linux'): > cmd_list = [u'setsid'] > if self._setsid_w: > cmd_list.extend([u"-w"]) > else: > cmd_list = [] > basepython = os.environ.get(u'TOXPYTHON', None) > if basepython is not None: > cmd_list.extend([basepython]) > run_coverage = os.environ.get(u'RUN_COVERAGE', None) > if run_coverage is not None: > cmd_list.extend([u"-m", u"coverage", u"run", > u"--source=duplicity", u"-p"]) > cmd_list.extend([u"{0}/bin/duplicity".format(_top_dir)]) > cmd_list.extend(options) > cmd_list.extend([u"-v0"]) > cmd_list.extend([u"--no-print-statistics"]) > cmd_list.extend([u"--allow-source-mismatch"]) > > cmd_list.extend([u"--archive-dir={0}/testfiles/cache".format(_runtest_dir)]) > if current_time: > cmd_list.extend([u"--current-time", current_time]) > cmd_list.extend(self.class_args) > if fail: > cmd_list.extend([u"--fail", u"".__class__(fail)]) > cmdline = u" ".join([u'"%s"' % x for x in cmd_list]) > > cmdline = u" ".join([u'"%s"' % x for x in cmd_list]) > > if not passphrase_input: > cmdline += u" < /dev/null" > > # The immediately following block is the nicer way to execute > pexpect with > # unicode strings, but we need to have the pre-4.0 version for > some time yet, > # so for now this is commented out so tests execute the same way > on all systems. > > # if parse_version(pexpect.__version__) >= parse_version("4.0"): > # # pexpect.spawn only supports unicode from version 4.0 > # # there was a separate pexpect.spawnu in 3.x, but it has an > error on readline > # child = pexpect.spawn(u'/bin/sh', [u'-c', cmdline], > timeout=None, encoding=sys.getfilesystemencoding()) > # > # for passphrase in passphrase_input: > # child.expect(u'passphrase.*:') > # child.sendline(passphrase) > # else: > > # Manually encode to filesystem encoding and send to spawn as bytes > # ToDo: Remove this once we no longer have to support systems with > pexpect < 4.0 > if sys.version_info.major > 2: > child = pexpect.spawn(u'/bin/sh', [u'-c', cmdline], > timeout=None) > else: > child = pexpect.spawn(b'/bin/sh', [b'-c', > cmdline.encode(sys.getfilesystemencoding(), > > u'replace')], timeout=None) > > for passphrase in passphrase_input: > child.expect(b'passphrase.*:') > child.sendline(passphrase) > > # if the command fails, we need to clear its output > # so it will terminate cleanly. > child.expect_exact(pexpect.EOF) > lines = child.before.splitlines() > child.wait() > child.ptyproc.delayafterclose = 0.0 > return_val = child.exitstatus > > if fail: > self.assertEqual(30, return_val) > elif return_val: > print(u"\n...command:", cmdline, file=sys.stderr) > print(u"...cwd:", os.getcwd(), file=sys.stderr) > print(u"...output:", file=sys.stderr) > for line in lines: > line = line.rstrip() > if line: > print(line, file=sys.stderr) > print(u"...return_val:", return_val, file=sys.stderr) > > > raise CmdError(return_val) > E testing.functional.CmdError: 1 > > /opt/sw/src/fink.build/duplicity-1.0.1-1/duplicity-1.0.1/testing/functional/__init__.py:175: > CmdError > > During handling of the above exception, another exception occurred: > > self = <testing.functional.test_badupload.BadUploadTest > testMethod=test_missing_file> > > @pytest.mark.slow > def test_missing_file(self): > u""" > Test basic lost file > """ > try: > self.backup(u"full", > u"{0}/testfiles/dir1".format(_runtest_dir), options=[u"--skip-volume=1"]) > self.fail() > except CmdError as e: > > self.assertEqual(e.exit_status, 44, str(e)) > E AssertionError: 1 != 44 : 1 > > /opt/sw/src/fink.build/duplicity-1.0.1-1/duplicity-1.0.1/testing/functional/test_badupload.py:47: > AssertionError > ----------------------------- Captured stderr call > ----------------------------- > > ...command: > "/opt/sw/src/fink.build/duplicity-1.0.1-1/duplicity-1.0.1/bin/duplicity" > "full" "/var/folders/6b/85yg9bp94rv55dzbdp78thp40000kr/T/testfiles/dir1" " > file:///var/folders/6b/85yg9bp94rv55dzbdp78thp40000kr/T/testfiles/output" > "--volsize" "1" "--skip-volume=1" "-v0" "--no-print-statistics" > "--allow-source-mismatch" > "--archive-dir=/var/folders/6b/85yg9bp94rv55dzbdp78thp40000kr/T/testfiles/cache" > < /dev/null > ...cwd: /private/var/folders/6b/85yg9bp94rv55dzbdp78thp40000kr/T > ...output: > b'Traceback (most recent call last):' > b' File > "/opt/sw/src/fink.build/duplicity-1.0.1-1/duplicity-1.0.1/bin/duplicity", > line 37, in <module>' > b' from duplicity.dup_main import main' > b' File > "/opt/sw/src/fink.build/duplicity-1.0.1-1/duplicity-1.0.1/duplicity/dup_main.py", > line 50, in <module>' > b' from duplicity import commandline' > b' File > "/opt/sw/src/fink.build/duplicity-1.0.1-1/duplicity-1.0.1/duplicity/commandline.py", > line 43, in <module>' > b' from duplicity import backend' > b' File > "/opt/sw/src/fink.build/duplicity-1.0.1-1/duplicity-1.0.1/duplicity/backend.py", > line 44, in <module>' > b' from duplicity import dup_temp' > b' File > "/opt/sw/src/fink.build/duplicity-1.0.1-1/duplicity-1.0.1/duplicity/dup_temp.py", > line 34, in <module>' > b' from duplicity import path' > b' File > "/opt/sw/src/fink.build/duplicity-1.0.1-1/duplicity-1.0.1/duplicity/path.py", > line 49, in <module>' > b' from duplicity import librsync' > b' File > "/opt/sw/src/fink.build/duplicity-1.0.1-1/duplicity-1.0.1/duplicity/librsync.py", > line 36, in <module>' > b' from . import _librsync' > b"ImportError: cannot import name '_librsync' from 'duplicity' > (/opt/sw/src/fink.build/duplicity-1.0.1-1/duplicity-1.0.1/duplicity/__init__.py)" > ...return_val: 1 > > > On Oct 9, 2022, at 11:26 AM, Kenneth Loafman <kenn...@loafman.com> wrote: > > Ok, sounds good. Any progress? > > ...Ken > > > On Thu, Oct 6, 2022 at 3:15 PM Scott Hannahs <s...@p-hall.net> wrote: > >> I will try to reinstall Xcode and the command line tools. I had been >> using py38 which is installed in both the standard and an updated version >> in the non-standard location. >> >> On Oct 6, 2022, at 14:57, Kenneth Loafman <kenn...@loafman.com> wrote: >> >> OK, >> >> I could not remember what version of Python 3 you were running, so py3x >> meant for you to fill it in. I'm guessing either py38 or py39 for Python >> 3.8 and Python 3.9 respectively. Others follow the same model, >> py27,py35,...py310. >> >> I am not sure why it's trying to build for anything but x86_64 if you >> have Intel silicone and not Apple silicone. >> >> Here's what mine looks like for py38: >> >> ken@dione:~/workspace/duplicity-origin$ python3.8 ./setup.py build_ext >> Unable to get SCM version: defaulting to 1.0.1 >> running build_ext >> Building extension for librsync... >> building 'duplicity._librsync' extension >> creating build/temp.macosx-12-x86_64-cpython-38 >> creating build/temp.macosx-12-x86_64-cpython-38/duplicity >> clang -Wno-unused-result -Wsign-compare -Wunreachable-code -fno-common >> -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -isysroot >> /Library/Developer/CommandLineTools/SDKs/MacOSX12.sdk -I/usr/local/include >> -I/usr/local/opt/python@3.8/Frameworks/Python.framework/Versions/3.8/include/python3.8 >> -c duplicity/_librsyncmodule.c -o >> build/temp.macosx-12-x86_64-cpython-38/duplicity/_librsyncmodule.o >> creating build/lib.macosx-12-x86_64-cpython-38 >> creating build/lib.macosx-12-x86_64-cpython-38/duplicity >> clang -bundle -undefined dynamic_lookup -isysroot >> /Library/Developer/CommandLineTools/SDKs/MacOSX12.sdk >> build/temp.macosx-12-x86_64-cpython-38/duplicity/_librsyncmodule.o >> -L/usr/local/lib -lrsync -o >> build/lib.macosx-12-x86_64-cpython-38/duplicity/_ >> librsync.cpython-38-darwin.so >> ld: warning: -undefined dynamic_lookup may not work with chained fixups >> copying build/lib.macosx-12-x86_64-cpython-38/duplicity/_ >> librsync.cpython-38-darwin.so -> duplicity >> >> >> It looks like your xcode libs are out of date, and it also looks like >> you're not using the standard locations for software. That's gonna >> complicate matters. >> >> I'm on Monterey 12.6 with the current Xcode from Apple, all in standard >> places. >> >> ...Ken >> >> >> >> >> On Thu, Oct 6, 2022 at 1:13 PM Scott Hannahs <s...@p-hall.net> wrote: >> >>> Ken, >>> >>> I will leave fink out, but it should be doing exactly the same thing! >>> :-) >>> >>> the following doesn’t even build duplicity correctly, so I don’t get to >>> the tox command. >>> >>> % tar xf duplicity-1.0.1.tar >>> % cd duplicity-1.0.1 >>> % % C_INCLUDE_PATH=/opt/sw/include LIBRARY_PATH=/opt/sw/lib python3 >>> ./setup.py build_ext > build.out.txt 2>&1 >>> >>> >>> This gives the following output. >>> >>> Unable to get SCM version: defaulting to 1.0.1 >>> running build_ext >>> Building extension for librsync... >>> building 'duplicity._librsync' extension >>> clang -Wno-unused-result -Wsign-compare -Wunreachable-code -fno-common >>> -dynamic -DNDEBUG -g -fwrapv -O3 -Wall >>> -iwithsysroot/System/Library/Frameworks/System.framework/PrivateHeaders >>> -iwithsysroot/Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.9/Headers >>> -arch arm64 -arch x86_64 -Werror=implicit-function-declaration >>> -I/Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.9/include/python3.9 >>> -c duplicity/_librsyncmodule.c -o >>> build/temp.macosx-10.9-universal2-3.9/duplicity/_librsyncmodule.o >>> clang -bundle -undefined dynamic_lookup -arch arm64 -arch x86_64 >>> -Wl,-headerpad,0x1000 >>> build/temp.macosx-10.9-universal2-3.9/duplicity/_librsyncmodule.o -lrsync >>> -o build/lib.macosx-10.9-universal2-3.9/duplicity/_ >>> librsync.cpython-39-darwin.so >>> ld: warning: ignoring file /opt/sw/lib/librsync.dylib, building for >>> macOS-arm64 but attempting to link with file built for macOS-x86_64 >>> ld: warning: dylib (/opt/sw/lib/librsync.dylib) was built for newer >>> macOS version (12.4) than being linked (10.9) >>> copying build/lib.macosx-10.9-universal2-3.9/duplicity/_ >>> librsync.cpython-39-darwin.so -> duplicity >>> >>> >>> I am building on an X86 machine and my libraries are x86 and I am not >>> trying to build a universal arm 64 system. But with warnings it tries to >>> build. >>> >>> tox does not like the py3x value >>> >>> % tox -epy3 >>> py3 create: /Users/sth/Downloads/duplicity-1.0.1/.tox/py3 >>> py3 installdeps: -rrequirements.txt >>> >>> and I ran out of time this AM waiting for more output. >>> >>> -Scott >>> >>> >>> On Oct 5, 2022, at 11:54 AM, Kenneth Loafman <kenn...@loafman.com> >>> wrote: >>> >>> Could you try just building and running the tests in a 'normal' user >>> environment? Leave Fink out of the picture. >>> >>> do this: >>> >>> 1. $ tar xf duplicity-1.01.tar.gz >>> 2. $ ./setup.py build_ext >>> 3. $ tox -e py3x >>> >>> ...Thanks, >>> ...Ken >>> >>> >>> >> >
_______________________________________________ Mailing list: https://launchpad.net/~duplicity-team Post to : duplicity-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~duplicity-team More help : https://help.launchpad.net/ListHelp