#26193: Tarballs are not compressed in a run following an aborted run -----------------------------------+---------------------- Reporter: karsten | Owner: karsten Type: defect | Status: assigned Priority: Medium | Milestone: Component: Metrics/CollecTor | Version: Severity: Normal | Keywords: Actual Points: | Parent ID: Points: | Reviewer: Sponsor: | -----------------------------------+---------------------- Today I wondered why some of our 2018-05 tarballs are over 3 days old even though we're creating new tarballs every 2 days. And I wondered even more after finding a seemingly completed run from yesterday. Here's what happened:
- The May 19 run succeeded without issues. - The May 21 run was interrupted, probably due to a host reboot. Apparently, it finished compressing tarballs except for bridge extra-info descriptors, but it did not move any of them in place. - The May 23 run went through, but it did not compress any tarballs except for bridge extra-info descriptors. I think this is related to [https://gitweb.torproject.org/collector.git/tree/src/main/resources /create-tarballs.sh#n102 line 102 in the script]: {{{ for (( i = 0 ; i < ${#TARBALLS[@]} ; i++ )); do echo `date` "Creating" ${TARBALLS[$i]}'.tar' tar chf ${TARBALLS[$i]}.tar ${TARBALLS[$i]} if [ ! -f ${TARBALLS[$i]}.tar.xz ]; then # <- this one echo `date` "Compressing" ${TARBALLS[$i]}'.tar' xz -9e ${TARBALLS[$i]}.tar fi done }}} Explanation: there were still compressed files in the working directory from May 21, and as a result we did not attempt to compress new tarballs from May 23. Suggested fix: remove that `if` statement and compress the tarball regardless of whether a previous compressed tarball exists in the working directory using `xz -9e -f`: {{{ diff --git a/src/main/resources/create-tarballs.sh b/src/main/resources /create-tarballs.sh index cd16b2dc..d247c520 100755 --- a/src/main/resources/create-tarballs.sh +++ b/src/main/resources/create-tarballs.sh @@ -99,10 +99,8 @@ done for (( i = 0 ; i < ${#TARBALLS[@]} ; i++ )); do echo `date` "Creating" ${TARBALLS[$i]}'.tar' tar chf ${TARBALLS[$i]}.tar ${TARBALLS[$i]} - if [ ! -f ${TARBALLS[$i]}.tar.xz ]; then - echo `date` "Compressing" ${TARBALLS[$i]}'.tar' - xz -9e ${TARBALLS[$i]}.tar - fi + echo `date` "Compressing" ${TARBALLS[$i]}'.tar' + xz -9e -f ${TARBALLS[$i]}.tar done cd $OUTDIR/webstats/ }}} -- Ticket URL: <https://trac.torproject.org/projects/tor/ticket/26193> Tor Bug Tracker & Wiki <https://trac.torproject.org/> The Tor Project: anonymity online
_______________________________________________ tor-bugs mailing list tor-bugs@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs