[arch-projects] [dbscripts] [GIT] Official repo DB scripts annotated tag archlinux/20111105 created. 20111105

[Eli Schwartz via arch-projects]

This is an automated email from the git hooks/post-receive script. It was
generated because a ref change was pushed to the repository containing
the project "Official repo DB scripts".

The annotated tag, archlinux/2005 has been created
at  3ba457e78645d348217038f922b35791b898d5d8 (tag)
   tagging  a8b219b325cd642a781a64d8e08a0e66d25c1b44 (commit)
  replaces  20110813
 tagged by  Pierre Schmitz
on  Sat Nov 5 22:33:08 2011 +0100

- Log -
tagging

Dan McGee (2):
  db-*: always handle pkgname/pkgbase/pkgfile argument last
  db-repo-{add, remove}: allow specifying multiple packages

Pierre Schmitz (9):
  Rebuilt alpm.so for pacman 4.0 soname bump
  Require a signature file for new packages
  Adjust tests to use new db-remove interface
  db-remove: support removing multiple packages at once
  check svn entry for any packages
  Split package database test functions from check* functions
  Add test for moving multiple packages at once
  db-repo-add: At least check if the package to be added is available in 
the correct directory
  Add simple tests for db-repo-add and db-repo-remove

Rémy Oudompheng (3):
  Use ctypes module to access alpm_pkg_vercmp() instead of the C module.
  Move global code under a if __name__ == '__main__' block.
  The README file is no longer useful

---


hooks/post-receive
-- 
Official repo DB scripts

[arch-projects] [dbscripts] [GIT] Official repo DB scripts annotated tag archlinux/20100410 created. 20100410

[Eli Schwartz via arch-projects]

This is an automated email from the git hooks/post-receive script. It was
generated because a ref change was pushed to the repository containing
the project "Official repo DB scripts".

The annotated tag, archlinux/20100410 has been created
at  a496da550323440a491aac908edeaad37fc5721d (tag)
   tagging  154b5301ff08ef29ff69afc17789a4f1a6a00c20 (commit)
  replaces  20100223
 tagged by  Pierre Schmitz
on  Sat Apr 10 20:25:53 2010 +0200

- Log -
tagging 20100410

Andrea Scarpino (1):
  fix permissions of incoming packages

Dan McGee (5):
  create-filelists: general cleanups
  create-filelists: s/REPO_DB_FILE/FILES_DB_FILE/g
  create-filelists: rework the package loop completely
  create-filelists: include desc/depends entries
  Merge branch 'filelists'

Eric Bélanger (3):
  Make repo locking an atomic process and added timeout argument
  sourceballs: Make cleanup more efficient
  moving sourceballs-cleanup

Pierre Schmitz (1):
  don't publish *.old files

---


hooks/post-receive
-- 
Official repo DB scripts

[arch-projects] [dbscripts] [GIT] Official repo DB scripts annotated tag archlinux/20121003 created. 20121003

[Eli Schwartz via arch-projects]

This is an automated email from the git hooks/post-receive script. It was
generated because a ref change was pushed to the repository containing
the project "Official repo DB scripts".

The annotated tag, archlinux/20121003 has been created
at  e0defb8cd96a60193b7bc0f2e93450a94e54c3c4 (tag)
   tagging  a539598c640caa92e2b1967949836d73c5264036 (commit)
  replaces  20120805
 tagged by  Pierre Schmitz
on  Wed Oct 3 14:51:01 2012 +0200

- Log -
tagging
-BEGIN PGP SIGNATURE-
Version: GnuPG v2.0.19 (GNU/Linux)

iQEcBAABAgAGBQJQbDS5AAoJEH8tQ0uXQeisuhUIALYJL2Sqw/IrUx9L6jf6hjZE
jgwK6Z+pPRmTZLYKEl2tkOQEmnezy/95wyzRSsHe5VqYFo80LcW2phMtSsGNepsv
5/mlCuw0vhbv9jUmm2v6yBNK4Ye7Mn+XOrKg3utY96whNGN1mss7OPHKt+2+weQt
s2tTSFhIeyPRnIP60yQ/kU/uPNEK/0ROgBUsOUtC7GRHkDW5xwss34cEY29pRyM3
dgHkqRIcnEbTwvIGtX08PKH+yCOdNdlUUwdFPEUD97hQlzkEtlf2gY32B8gMUcwk
0G02JSfJUCfVtIClsVOlKMiHMAxif+2BBSM4MQyEb8/65J6bj1nCUqYnH9yucbo=
=dKFT
-END PGP SIGNATURE-

Pierre Schmitz (2):
  test: Do not suppress devtools output
  Use pacman-key to check signatures as it now uses a correct exit value

---


hooks/post-receive
-- 
Official repo DB scripts

[arch-projects] [dbscripts] [GIT] Official repo DB scripts annotated tag archlinux/20100223 created. 20100223

[Eli Schwartz via arch-projects]

This is an automated email from the git hooks/post-receive script. It was
generated because a ref change was pushed to the repository containing
the project "Official repo DB scripts".

The annotated tag, archlinux/20100223 has been created
at  0a79f6642e34fd49bcada91dbb29d49e49a53b9b (tag)
   tagging  ba7968503f18e5a2537d7d87ce2a5caa582f04f7 (commit)
 tagged by  Pierre Schmitz
on  Wed Feb 24 22:00:07 2010 +0100

- Log -
tagging 20100223

Aaron Griffin (200):
  Giant SVN changes
  Recompile for new passwords
  Make SVN output quiet
  Make genpkglist work again
  Fix "unknown" category handling
  Properly point to the 64bit staging dirs
  Remove 'svndir' usage
  Use local user dirs for checkouts
  Correct lockfile usage
  Switch to proper script usage
  Remove SQL database related tasks from these scripts
  Massive rewrite to make this script uber
  Switch the params to be inline with archrelease
  Make the params slightly more clear
  Remove testing paths
  Remove usage of the 64bit dirs from the staging path
  More absolute pathing
  Remove genpkglists from the dbscripts
  Fix file deletion on package removal
  Whoops, 'cd' is a built in - who knew?
  Make sure we copy old staging dir entries properly
  Make sure we're actually adding/removing files
  Fail loudly if ftp dir is missing
  Properly clean up stating dirs after a success
  Moving some files around, organizational
  Cleanup some debugging output
  Remove a check for fakeroot
  Make sure to clean up working temp dirs
  Add new db-remove script
  Make the staging warning stand out more
  Cron cleanup, and only copy DB files once
  More cron job cleanup and fixes
  Added a simple sendmail-ish script to send to the ML
  Add db-move related scripts
  Make sure to cleanup WORKDIR on exit
  Remove arch from the testing2 scripts, add *64 version
  Switch to output filelists in official repos
  Copied 'ftpmaint' from root as adjust-permissions
  Switch EXTRAFILES test to be more accurate
  Adjust permissions of the adjust-permissions script
  Fix db-remove usage text
  Use 'mv' to shuffle files out of the *64 dirs
  Git tells me this file was modifed
  crontab file for repo-based cron jobs
  Add a script to generate source tarballs for GPL compliance
  Link the new testing2x script to the 64 version too
  Minor changes to the make-sourceball script
  Merge branch 'master' of /srv/projects/git/dbscripts
  Move ftpdir-cleanup to misc-scripts
  Add a real cron script to cycle over all repos
  make-sourceball fixes
  Remove unused code from make-sourceball
  Correct the devlist mailer script to send real emails
  Merge branch 'master' of /srv/projects/git/dbscripts
  Make sure ftpdir-cleanup actually does work before outputting
  Make blank lines conditional
  Remove some 'self' usage from check_archlinux.py
  Corrections to the devlist-mailer script
  Remove community from the integrity check, for now
  Fix an error message copy/paste error
  Prevent staging cleanup if copy to repo fails
  Create a "common functions" file for DB scripts
  Move source generation to /home/ftp/sources
  Add sourceball generating cron script
  Merge branch 'master' of ssh://archlinux.org/srv/projects/git/dbscripts
  Add sourceball cron job at the proper path (whoops)
  Forgot the db-functions file. Whoops
  Remove 'unstable' from sourceball creation
  Remove the unstable repository
  Cron-job changes
  Correctly remove old sourceballs without removing the wrong files
  Validate a package file's architecture
  Move temporary directories to /home/tmp
  Fix a typo in db-update's usage output
  Remove some goofy debugging messages from db-functions
  Add copy_helper function to ensure correct permissions
  Remove an erroneous '1' signal trap for cleanup() func
  Move as much as we can out to a config file
  Move everything to /srv to support new server conf
  Switch svn path to be proper on gerolde
  Fix patching for chmod call
  Clear traps on cleanup
  Remove unused cron-job 'genpkglist'
  Move cron-jobs to /srv from /home
  Remove /del dir usage from db-update
  Fix a variable overwrite when sourcing PKGBUILDs
  Add umask setting/restoring helper functions
  Remove 'copy_helper' as chmodding is fail
  Set the umask to allow g+w
  Ensure we keep track of our initial umask
  Remove some harmless error messages
  Only copy DB file from repo if we have pkgs
  Do not touch db file if it doesn't exist
  Cleanup missing db file logic
  Add potential BUILDSCRIPT config setting
  Quiet down repo-ad

[arch-projects] [dbscripts] [GIT] Official repo DB scripts annotated tag archlinux/20110326 created. 20110326

[Eli Schwartz via arch-projects]

This is an automated email from the git hooks/post-receive script. It was
generated because a ref change was pushed to the repository containing
the project "Official repo DB scripts".

The annotated tag, archlinux/20110326 has been created
at  b4d8cfbc80b0eb1faa295e56e5969d1619a3b95c (tag)
   tagging  fc6a6ab07bde03c7f20d5a4ed971f8e699ee9b20 (commit)
  replaces  20101209
 tagged by  Pierre Schmitz
on  Sat Mar 26 14:43:13 2011 +0100

- Log -
tagging

Dan McGee (1):
  archweb update: source virtualenv if it exists

Pierre Schmitz (10):
  Touch old packages after moving them to the cleanup dirs
  Remove old files from the cleanup dirs
  Move repo manipulation code into common functions
  test: add testUpdateSameAnyPackageToSameRepository
  Make sure to only remove package files from the cleanup dirs
  Try to remove a package even if it is no longer in svn
  Keeping old packages for a month should be long enough
  Add support for packages which use the epoch variable
  Adjust test for latest devtools; Be less generic with package extension
  Use repo-add to create the files database for each repo

---


hooks/post-receive
-- 
Official repo DB scripts

[arch-projects] [dbscripts] [GIT] Official repo DB scripts annotated tag archlinux/20120616 created. 20120616

[Eli Schwartz via arch-projects]

This is an automated email from the git hooks/post-receive script. It was
generated because a ref change was pushed to the repository containing
the project "Official repo DB scripts".

The annotated tag, archlinux/20120616 has been created
at  8b915ad3b0e744a09051f768570c65ec187fe9e5 (tag)
   tagging  7fdc74786a59c69d41524d5f311f856cfa9a99c6 (commit)
  replaces  20120610
 tagged by  Pierre Schmitz
on  Sat Jun 16 19:59:34 2012 +0200

- Log -
tagging
-BEGIN PGP SIGNATURE-
Version: GnuPG v2.0.19 (GNU/Linux)

iQEcBAABAgAGBQJP3MmKAAoJEH8tQ0uXQeisZQoIAL+VBLybMYaYjckJADoD7eQC
cQ46DIR+fnDwqSYXEvo/o02UyNPuP2uFEpAiKDIyemfrx6lKvbFiOm8Xzeq6EDkD
bd5B30DvPo/SQidXDZ8EuyH/f6b7x6MC9LMZWVzbnkvZKk5dhK8bt7QaQ5Ghz7Ql
9xmmVEUJaazx+1xWgKFy/oe6MgearhwbmDzvQBAEwxPlcMOK+HBazqd8CYBsnbpG
nk7vMf1/seUF/XUmwmX2LGZGuAw/5Ts95P3EB/7+KC+Afy5JTpo6BfFfYPThC672
Uyjkf7sPsDveFNGwceG1YpaNXmYRBLSl34db6+5jA4FLAWs8oC0o7zTS5RJZ8+g=
=YvvR
-END PGP SIGNATURE-

Pierre Schmitz (2):
  Reduce output when downloading sources
  Fix handling of package sources containg files with an @ character

---


hooks/post-receive
-- 
Official repo DB scripts

[arch-projects] [dbscripts] [GIT] Official repo DB scripts annotated tag archlinux/20120805 created. 20120805

[Eli Schwartz via arch-projects]

This is an automated email from the git hooks/post-receive script. It was
generated because a ref change was pushed to the repository containing
the project "Official repo DB scripts".

The annotated tag, archlinux/20120805 has been created
at  c1913555a6baafbc55f9bc05eb0a0f0a1717b0e7 (tag)
   tagging  332346d06375c1272bee6dce2c91d31a027fe212 (commit)
  replaces  20120616
 tagged by  Pierre Schmitz
on  Sun Aug 5 12:19:53 2012 +0200

- Log -
tagging
-BEGIN PGP SIGNATURE-
Version: GnuPG v2.0.19 (GNU/Linux)

iQEcBAABAgAGBQJQHkjNAAoJEH8tQ0uXQeisnJYH/3TXqjkOIt91N4qq+jXG9l/J
HB6DhCl8qjfjHX1OIP1ZJEsWJb+ZyJn2pfFfqsvSFcwJQ6pfIkXCztt2z+s1u7Vp
Gdhc3Fb0DXGZ1AcJZER0vChnX875b5GcNYQcf14fZ3BbLuwN5vwQs1+vCH54iY9K
IYVlQRSq6kzV44qGLn/MgMwldZi0jvfYDNIFbtsdfY06BEOx4/u6kAQeePNAupEy
8kKExkQjEPmWkz0y4vn+Jw8mZ8dGMzxJ2LPUr2XJi2nrNC+UtDjjJNW+goWA//S1
a8YOEjFj7aUG3nHccOAiS4Wc4krej2Uofyd09WjLjOCJ/LxNCHAc5WsvHAZvKKs=
=Q0ji
-END PGP SIGNATURE-

Florian Pritz (1):
  db-move: Remove double space in tag_list

Pierre Schmitz (1):
  grep has been moved to /usr

---


hooks/post-receive
-- 
Official repo DB scripts

[arch-projects] [dbscripts] [GIT] Official repo DB scripts annotated tag archlinux/20110813 created. 20110813

[Eli Schwartz via arch-projects]

This is an automated email from the git hooks/post-receive script. It was
generated because a ref change was pushed to the repository containing
the project "Official repo DB scripts".

The annotated tag, archlinux/20110813 has been created
at  0faa6ed72d8d396d2d47a7a734bf9836857a2de9 (tag)
   tagging  e04b5f6583acdec920407cdd4455f54452367382 (commit)
  replaces  20110409
 tagged by  Pierre Schmitz
on  Sat Aug 13 22:57:12 2011 +0200

- Log -
tagging

Dan McGee (1):
  db-move: remove one svn commit per package

Eric Bélanger (1):
  Add epoch support to integrity check cron job

Florian Pritz (1):
  db-remove: remove package if not in svn

Pierre Schmitz (4):
  Use /tmp instead of /dev/shm for temporary files
  Fix test for use with recent namcap releases
  db-move: Fix svn rm call
  update shunit to 2.1.6

---


hooks/post-receive
-- 
Official repo DB scripts

[arch-projects] [dbscripts] [GIT] Official repo DB scripts annotated tag archlinux/20120610 created. 20120610

[Eli Schwartz via arch-projects]

This is an automated email from the git hooks/post-receive script. It was
generated because a ref change was pushed to the repository containing
the project "Official repo DB scripts".

The annotated tag, archlinux/20120610 has been created
at  ae9ceabc80e625b94ab2bc0580a2cdf468e95ca4 (tag)
   tagging  75deac81c2ca88219a4bc31d4a9a3da4f8274863 (commit)
  replaces  2005
 tagged by  Pierre Schmitz
on  Sun Jun 10 12:36:13 2012 +0200

- Log -
tagging
-BEGIN PGP SIGNATURE-
Version: GnuPG v2.0.19 (GNU/Linux)

iQEcBAABAgAGBQJP1HikAAoJEH8tQ0uXQeiskBIIAK07v5YWqsCA1BFs8pbIwOQG
okDOFu9hqzODLFUlXisn+MFIwYRhcTDrlhQqogvm5nWHm1Oelo5XbqZWKEQnneiJ
NfWY9G8g17GloLDd22TCQFH8oEhiOiPHeEGX3HS/nW6NDWfvPombdSSZ4PfyqtpP
3we9bRdjwMdXBdJkwdAVEsIANuJQmesrk3Oa2IBGuU4aV4kRcBr1ckdyFA3erQHD
lp73AjmeFphZOzfgU86FT31WCLQ/J7M8ZXoCvyzFIYz+TjZ3ZScDUyoxPpz5ZNVH
ckXNmY5a0BB9zNp9eCkPe9UUNUTwktOxfDlcWqIol+9ZOlnNWZX1jx3wV/5nxJo=
=luzc
-END PGP SIGNATURE-

Florian Pritz (1):
  db-move: commit all arches at once

Pierre Schmitz (16):
  Validate package signatures on db-update
  Avoid calls to basename
  Add [multilib-staging] to sigurd config
  Add at least one positive signing test
  Fix typo and leftover shm usage
  Sourceballs: Output details if fetching the sources failed
  Do not check source gpg signatures
  sourceballs: If ALLOWED_LICENSES is empty create source packages for 
every package
  Update sourceballs.skip
  Use gpg status-file instead of status-fd
  Respect TMPDIR config when creating the working directory
  Source the config before the functions as the latter references the former
  Remove support for packages in legacy $repo/os/any directories
  Remove support for packages that are not in the package pool
  Use our own copy of makepkg.conf instead of depending on the host one
  Make repositories for testing2x configurable

---


hooks/post-receive
-- 
Official repo DB scripts

[arch-projects] [dbscripts] [GIT] Official repo DB scripts annotated tag archlinux/20100411 created. 20100411

[Eli Schwartz via arch-projects]

This is an automated email from the git hooks/post-receive script. It was
generated because a ref change was pushed to the repository containing
the project "Official repo DB scripts".

The annotated tag, archlinux/20100411 has been created
at  b44f0e9df79c2e86d348e5bb252efb7d0d095b79 (tag)
   tagging  52b668dc79b2ca05f9eebedab0dfc7eefb263765 (commit)
  replaces  20100410
 tagged by  Pierre Schmitz
on  Sun Apr 11 14:53:03 2010 +0200

- Log -
tagging 20100411

Pierre Schmitz (2):
  Fail if lock couldn't be obtained; Unlock the repos if ftpdir-cleanup 
fails
  Do not overwrite TMPDIR defined in config

---


hooks/post-receive
-- 
Official repo DB scripts

[arch-projects] [dbscripts] [GIT] Official repo DB scripts annotated tag archlinux/20131102 created. 20131102

[Eli Schwartz via arch-projects]

This is an automated email from the git hooks/post-receive script. It was
generated because a ref change was pushed to the repository containing
the project "Official repo DB scripts".

The annotated tag, archlinux/20131102 has been created
at  d201b99599d45e139e0721294834107c8eb43bb4 (tag)
   tagging  0b43e8cdee1ee46ea79e4d089136e76e767b4d5b (commit)
  replaces  20130131
 tagged by  Pierre Schmitz
on  Sat Nov 2 11:05:19 2013 +0100

- Log -
tagging
-BEGIN PGP SIGNATURE-
Version: GnuPG v2.0.22 (GNU/Linux)

iQEcBAABAgAGBQJSdM5jAAoJEH8tQ0uXQeisQ90H/R7wlR3ojfF8yc2x0u4GTnhE
wvs4rr069p5Ydq9jS+b6l8Xw4jmkNEj3I73Lks5ruu8nP41aF3Z7YRQA9An389oX
1jKETTVrxowDohOkJZvu6MWMZkK9emmc4qoKTcp4r0UhLE4rRc/A8OxPzykdoNE8
laduso3hPjE8kHCJDdH69HhIu92dhJZd6UzNYzJc/bKnCFDuAS21d62FyGSqtXnv
DZhZdQBVKdukDuZSQ8mcNgDfRpD0nMzNVx+TEEjNC+fAVPgTwudUAmeEdUFXKFlY
zaHkiJYYaHaWRH9xqcyCYtbLuJtKnaL0zfMYLvyX4FKzrSTxbsIsveq9AxIdA8Q=
=8ODP
-END PGP SIGNATURE-

Florian Pritz (1):
  Add lastupdate file

Jan Alexander Steffens (heftig) (1):
  sourceballs: fix ACL issues on nymeria

---


hooks/post-receive
-- 
Official repo DB scripts

[arch-projects] [dbscripts] [GIT] Official repo DB scripts annotated tag archlinux/20101209 created. 20101209

[Eli Schwartz via arch-projects]

This is an automated email from the git hooks/post-receive script. It was
generated because a ref change was pushed to the repository containing
the project "Official repo DB scripts".

The annotated tag, archlinux/20101209 has been created
at  c580022666e4da7e7dd1210a3749d2e0c8e8d556 (tag)
   tagging  76256c33bb6fa06b79bdab6615f2da587cd80f31 (commit)
  replaces  20100906
 tagged by  Pierre Schmitz
on  Thu Dec 9 08:51:26 2010 +0100

- Log -
tagging 20101209

Dan McGee (3):
  Add multilib-testing to web update script
  Move output redirection
  Fix missing output redirection

Eric Bélanger (2):
  Added seperate dryrun options for the packages and sources cleanup scripts
  Make ftpdir-cleanup less verbose

Pierre Schmitz (56):
  adding multilib-testing repository
  Remove obsolete cron job
  Fix check_repo_permission
  Use local config instead of guessing by hostname
  Fix check_pkgrepos
  check_pkgsvn: check if pkgname is in sync with svn
  check_pkgsvn: Don't assume the same PKGBUILD for all architectures
  _grep_pkginfo: use a more specific format
  Merge branch 'master' of gerolde.archlinux.org:/srv/projects/git/dbscripts
  Fix typos
  Dont try to index empty repos
  Fix sourceballs cron job
  Simplify sourceballs creation
  sourceballs: skip in a clean way if source package should not be created
  sourceballs: Lock the repo while reading the db file
  sourceballs: add LGPL2.1
  When on sigurd the svn repo is actually called svn-packages
  sourceballs: fix error handling
  sourceballs: check if PKGBUILD exists
  sourceballs: respect the dryrun setting
  sourceballs: Don't write any log
  sourceballs: Support any packages
  sourceballs.skip: nexuiz-data has the same sources as nexuiz
  sourceballs.skip: torcs-data has the same sources as torcs
  sourceballs.skip: tremulous-data has the same sources as tremulous
  sourceballs.skip: ufoai-data has the same sources as ufoai
  Blacklist more duplicate data packages
  sourceballs: Of course sort does only operate on lines
  texlive sources are already on ftp.archlinux.org
  Set CARCH as needed by some PKGBUILDs
  sourceballs: add for split packages
  Add wrappers for repo-add and repo-remove
  sourceballs-cleanup: respect .force and .skip files
  Prefer any packages
  sourceballs: check if source package was really created
  check_archlinux: fix for python2 usage
  Attempt to fix integrity-check
  Rewrite sourceballs to increase performance and reliability
  sourceballs: be less verbose
  sourceballs: fix output
  sourceballs: avoid failing twice
  sourceballs: Move to our workdir first to fix warning with sudo usage
  sourceballs: fix working directory
  Provide more real test packages
  Split tests into several files
  Add FILESEXT variable for use in create-filelists
  sourceballs no longer depends on ftpdir-cleanup being run
  Rewrote create-filelists
  sourceballs: be nice
  update-web-db: Lock db and script
  create-filelists: Improve performance by reading package list from db 
instead from fs
  Fix test: copy target of symlinks
  db-update: Fail if a set of split packages is incomplete
  repo_lock: check if repo is locked by repo-add or repo-remove
  Call integrity-check with specific repo list for each architecture
  check_packages.py: remove obsolete klibc workaround

Rémy Oudompheng (4):
  Style changes: remove usage of the 'has_key' method.
  Style changes: add parentheses around print "...".
  Add support for the [multilib] repo.
  Only emit a warning when a repository does not exist.

---


hooks/post-receive
-- 
Official repo DB scripts

[arch-projects] [dbscripts] [GIT] Official repo DB scripts annotated tag archlinux/20100906 created. 20100906

[Eli Schwartz via arch-projects]

This is an automated email from the git hooks/post-receive script. It was
generated because a ref change was pushed to the repository containing
the project "Official repo DB scripts".

The annotated tag, archlinux/20100906 has been created
at  6da1776d23da23a8842d2dd1412995fa74832bb2 (tag)
   tagging  6f29ee2f02c2a8e2991599ce1d76a59b58a7ee67 (commit)
  replaces  20100411
 tagged by  Pierre Schmitz
on  Mon Sep 6 20:10:26 2010 +0200

- Log -
tagging 20100906

Aaron Griffin (7):
  db-update: Use a master package pool and symlinks
  Remove FTP_OS_SUFFIX variable
  Change ftpdir-cleanup scripts for package pooling
  ftpdir-cleanup: Fix cleanup of package pool
  ftpdir-cleanup: use 'find' for symlink detection
  ftpdir-cleanup: Add clean_pkgs function
  Add PKGPOOL_DIR config variable

Allan McRae (1):
  minor fix to error message

Dan McGee (5):
  Add update-web-db cronjob
  Fix a few issues with update-web-db
  update-web-db: reformat to coding style
  update-web-db: flip the arch/repo loops
  Simplify case statement

Eric Bélanger (1):
  sourceballs: Added community and community-testing support

Pierre Schmitz (67):
  add initial .gitignore file
  Replace PKGPOOL_DIR by a function call
  ftpdir-cleanup: only search for linked packages in actual repos
  adjust permissions for package pool
  don't mix db files of different arches
  don't fail if any package already exists
  Override default config with config.local
  determine correct dir even if config was sourced
  add some sample PKGBUILDs
  add a local copy of shunit2
  add default makepkg.conf for both arches
  add some common functions for our test suite
  add an initial set of tests
  add test for db-remove
  Simplify tests and add tests for db-move
  Rewrite db-move
  Fix old call
  Use package pool instead of $repo/os/any dirs
  add test for ftpdir-cleanup
  fix cleaning of any packages and ignore empty repos
  add transitions tests
  Reduce verbosity
  Fix test and add some more checks
  Cleanup db-remove
  Remove check for old staging dirs
  Remove BUILDSCRIPT variable
  Prepare for variable db file compression
  use common workdir
  Move common function to db-functions
  Cleanup cron-jobs/adjust-permissions
  Cleanup db-functions
  Source PKGBUILD in subshells
  Use db-functions in create-filelists and sourceballs
  Fix typo
  Use more consitent naming for package pool
  Check permission before any action
  Use common functions to print messages, warnings and errors
  Abort if package already exists in repo
  Remove check which was already covered by check_repo_permission
  removed useless statements
  Simplify check for existing packages
  Rewrite of db-update
  Don't hardcode supported architectures in db-move
  Add common function to check for correct repo and arch
  Don't use hard coded architectures in db-remove
  Prepare support for multiple packages in db-move
  Use common names for repos
  Move packages of all arches within one transaction
  add test to check updating a package
  add test for testing2x
  Lock repos before checking
  Fix locking in db-remove
  Add [staging] repository
  Don't try to update no package
  Add additional checks when reading PKGBUILDs
  Fix cleanup of old packages
  Rewrite ftpdir-cleanup
  fix typo
  add config.local to .gitignore
  add makepkg.conf for [multilib]
  db-remove: source functions before using them
  db-update: Make sure there are no links in the staging directory
  adjust group for multilib repository
  Simplify repo configuration
  Set correct group after touching the db file
  Fix updating of same package into different repositories at aonce
  Check if package exists in any other repository on update

---


hooks/post-receive
-- 
Official repo DB scripts

[arch-projects] [dbscripts] [GIT] Official repo DB scripts annotated tag archlinux/20110409 created. 20110409

[Eli Schwartz via arch-projects]

This is an automated email from the git hooks/post-receive script. It was
generated because a ref change was pushed to the repository containing
the project "Official repo DB scripts".

The annotated tag, archlinux/20110409 has been created
at  b925016368c03344fb8fa01d11b1c0585ca02c80 (tag)
   tagging  1ce0c6368d0908e25f9bd1bb8183b5f29053fac8 (commit)
  replaces  20110326
 tagged by  Pierre Schmitz
on  Sat Apr 9 23:08:24 2011 +0200

- Log -
tagging

Pierre Schmitz (2):
  Recompiled alpm python module against pacman 3.5
  Add simple checks for handling signed packages

Rémy Oudompheng (1):
  Add signature files to update/move/remove targets

---


hooks/post-receive
-- 
Official repo DB scripts

[arch-projects] [pyalpm] Workaround https://bugs.python.org/issue33012

[Chih-Hsuan Yen via arch-projects]

This fixes building with GCC 8
---
 setup.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/setup.py b/setup.py
index 887b7a1..ba3ae2a 100644
--- a/setup.py
+++ b/setup.py
@@ -9,7 +9,7 @@ os.putenv('LC_CTYPE', 'en_US.UTF-8')
 pyalpm_version = '0.8'
 
 cflags = ['-Wall', '-Wextra', '-Werror',
-'-Wno-unused-parameter',
+'-Wno-unused-parameter', '-Wno-cast-function-type',
 '-std=c99', '-D_FILE_OFFSET_BITS=64']
 
 alpm = Extension('pyalpm',
-- 
2.17.0

[arch-projects] [devtools] [PATCH] Do not assume the makechrootpkg user's groupname is the same as the username

[Eli Schwartz via arch-projects]

chown support "$user:$group" but also "$user:" which infers $group
rather than leaving it as root. This looks up the group name in cases
where the default group is e.g. "users" and users do not get their own
unique groups.

Signed-off-by: Eli Schwartz <eschwa...@archlinux.org>
---

This is even more common than I thought, I've found a person who's
affected! :p

cf. https://lists.archlinux.org/pipermail/arch-projects/2018-March/004868.html

 makechrootpkg.in | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/makechrootpkg.in b/makechrootpkg.in
index f709955..37e97e7 100644
--- a/makechrootpkg.in
+++ b/makechrootpkg.in
@@ -245,7 +245,7 @@ download_sources() {
 
local builddir
builddir="$(mktemp -d)"
-   chown "$makepkg_user:$makepkg_user" "$builddir"
+   chown "$makepkg_user:" "$builddir"
 
# Ensure sources are downloaded
sudo -u "$makepkg_user" --preserve-env=GNUPGHOME \
-- 
2.17.0

[arch-projects] [devtools] [GIT] The official devtools repository branch master updated. 20171108-13-g5713cd6

[Evangelos Foutras via arch-projects]

This is an automated email from the git hooks/post-receive script. It was
generated because a ref change was pushed to the repository containing
the project "The official devtools repository".

The branch, master has been updated
   via  5713cd629c97c7a12a600a1dd73ad81d87374eb1 (commit)
  from  40f0179a5e74d6d3babbefdeae21fd374be0e090 (commit)

Those revisions listed above that are new to this repository have
not appeared on any other notification email; so we list those
revisions in full, below.

- Log -
commit 5713cd629c97c7a12a600a1dd73ad81d87374eb1
Author: Evangelos Foutras 
Date:   Sat May 12 11:52:18 2018 +0300

makechrootpkg: add /etc/shadow entry for builduser

Without it, sudo 1.8.23 will return an error:

sudo: PAM account management error: Authentication
service cannot retrieve authentication info

---

Summary of changes:
 makechrootpkg.in | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)


hooks/post-receive
-- 
The official devtools repository

[arch-projects] [archweb] [PATCH 2/2] mirrorlist: Complete /all/https success test case

[Genki Sky via arch-projects]

Also, remove test_generate(), as it was testing no more than
test_mirrorlist_filter() already was.

Signed-off-by: Genki Sky 
---
 mirrors/tests/test_mirrorlist.py | 16 ++--
 1 file changed, 10 insertions(+), 6 deletions(-)

diff --git a/mirrors/tests/test_mirrorlist.py b/mirrors/tests/test_mirrorlist.py
index 1ad3d8d..9e20812 100644
--- a/mirrors/tests/test_mirrorlist.py
+++ b/mirrors/tests/test_mirrorlist.py
@@ -25,9 +25,18 @@ def test_mirrorlist_all_http(self):
 self.assertIn(self.mirror_url.hostname, response.content)
 
 def test_mirrorlist_all_https(self):
+# First test that without any https mirrors, we get a 404.
 response = self.client.get('/mirrorlist/all/https/')
 self.assertEqual(response.status_code, 404)
-# TODO: test 200 case
+
+# Now, after adding an HTTPS mirror, we expect to succeed.
+https_mirror_url = create_mirror_url(
+name='https_mirror',
+protocol='https',
+url='https://wikipedia.org')
+response = self.client.get('/mirrorlist/all/https/')
+self.assertEqual(response.status_code, 200)
+https_mirror_url.delete()
 
 def test_mirrorlist_filter(self):
 jp_mirror_url = create_mirror_url(
@@ -45,8 +54,3 @@ def test_mirrorlist_filter(self):
 self.assertNotIn(self.mirror_url.hostname, response.content)
 
 jp_mirror_url.delete()
-
-def test_generate(self):
-response = 
self.client.get('/mirrorlist/?country=all=http_version=4')
-self.assertEqual(response.status_code, 200)
-self.assertIn(self.mirror_url.hostname, response.content)
-- 
2.17.0

[arch-projects] [archweb] [PATCH 0/2] mirrorlist: Accept GET parameter filters, improve tests

[Genki Sky via arch-projects]

Hello, please find the following patches. One fixes a bug I noticed
where the filter URLs generated by the [mirrorlist] form were not
filtering as they should. While fixing this bug, I also improved (I
hope!) the testing in the area. Let me know if I'm missing anything.

I ran all the test cases, and they still pass. On that note, thanks
for making the testing setup very easy to use. It is great to see.

[mirrorlist]: https://www.archlinux.org/mirrorlist/

Genki Sky (2):
  mirrorlist: Accept GET parameters as filters
  mirrorlist: Complete /all/https success test case

 mirrors/tests/__init__.py| 11 ++-
 mirrors/tests/test_mirrorlist.py | 30 +++---
 mirrors/views/mirrorlist.py  |  3 ++-
 3 files changed, 31 insertions(+), 13 deletions(-)

--
2.17.0

[arch-projects] [archweb] [PATCH 1/2] mirrorlist: Accept GET parameters as filters

[Genki Sky via arch-projects]

This fixes a regression. Originally request.REQUEST was used, but django
1.9 removed this. In its stead, request.POST was used unconditionally.
However, this results in any GET request returning *all* mirrors, rather
than filtering as requested in the parameters.

This patch uses POST or GET based on the request method. This fixes the
behavior of the [mirror-filter-form], and any scripts depending on the
generated URL format. Accordingly, make test_mirrorlist_filter() test
both the success and failure cases, rather than just success.

[mirror-filter-form]: https://www.archlinux.org/mirrorlist/

Signed-off-by: Genki Sky 
---
 mirrors/tests/__init__.py| 11 ++-
 mirrors/tests/test_mirrorlist.py | 16 ++--
 mirrors/views/mirrorlist.py  |  3 ++-
 3 files changed, 22 insertions(+), 8 deletions(-)

diff --git a/mirrors/tests/__init__.py b/mirrors/tests/__init__.py
index fb6c10d..a1d3c2c 100644
--- a/mirrors/tests/__init__.py
+++ b/mirrors/tests/__init__.py
@@ -1,12 +1,13 @@
 from mirrors.models import MirrorUrl, MirrorProtocol, Mirror
 
 
-def create_mirror_url():
-mirror = Mirror.objects.create(name='mirror1',
+def create_mirror_url(name='mirror1', country='US',
+  protocol='http', url='https://archlinux.org'):
+mirror = Mirror.objects.create(name=name,
admin_email='ad...@archlinux.org')
-mirror_protocol = MirrorProtocol.objects.create(protocol='http')
-mirror_url = MirrorUrl.objects.create(url='https://archlinux.org',
+mirror_protocol = MirrorProtocol.objects.create(protocol=protocol)
+mirror_url = MirrorUrl.objects.create(url=url,
   protocol=mirror_protocol,
   mirror=mirror,
-  country='US')
+  country=country)
 return mirror_url
diff --git a/mirrors/tests/test_mirrorlist.py b/mirrors/tests/test_mirrorlist.py
index 5590a96..1ad3d8d 100644
--- a/mirrors/tests/test_mirrorlist.py
+++ b/mirrors/tests/test_mirrorlist.py
@@ -30,9 +30,21 @@ def test_mirrorlist_all_https(self):
 # TODO: test 200 case
 
 def test_mirrorlist_filter(self):
-response = 
self.client.get('/mirrorlist/?country=all=http_version=4')
+jp_mirror_url = create_mirror_url(
+name='jp_mirror',
+country='JP',
+protocol='https',
+url='https://wikipedia.jp')
+
+# First test that we correctly see the above mirror.
+response = self.client.get('/mirrorlist/?country=JP=https')
 self.assertEqual(response.status_code, 200)
-self.assertIn(self.mirror_url.hostname, response.content)
+self.assertIn(jp_mirror_url.hostname, response.content)
+
+# Now confirm that the US mirror did not show up.
+self.assertNotIn(self.mirror_url.hostname, response.content)
+
+jp_mirror_url.delete()
 
 def test_generate(self):
 response = 
self.client.get('/mirrorlist/?country=all=http_version=4')
diff --git a/mirrors/views/mirrorlist.py b/mirrors/views/mirrorlist.py
index 35d59e8..45c0181 100644
--- a/mirrors/views/mirrorlist.py
+++ b/mirrors/views/mirrorlist.py
@@ -55,7 +55,8 @@ def as_div(self):
 @csrf_exempt
 def generate_mirrorlist(request):
 if request.method == 'POST' or len(request.GET) > 0:
-form = MirrorlistForm(data=request.POST)
+data = request.POST if request.method == 'POST' else request.GET
+form = MirrorlistForm(data=data)
 if form.is_valid():
 countries = form.cleaned_data['country']
 protocols = form.cleaned_data['protocol']
-- 
2.17.0

WARN: project from a retired committee but PMC not changed to Attic in http://svn.apache.org/repos/asf/oltu/site/doap_org.apache.oltu.parent.rdf

[Projects]

WARN: project in Attic but not in 'retired' category: http://svn.apache.org/repos/asf/oltu/site/doap_org.apache.oltu.parent.rdf

[Projects]

[arch-projects] [dbscripts] [GIT] Official repo DB scripts branch master updated. 20131102-87-g656fd6d

[Eli Schwartz via arch-projects]

This is an automated email from the git hooks/post-receive script. It was
generated because a ref change was pushed to the repository containing
the project "Official repo DB scripts".

The branch, master has been updated
   via  656fd6d5d8f455492b1e51add9e6e1da8f911a7c (commit)
   via  0cf2ac49c18a71c050f5a9a9ff42e8dff89c2375 (commit)
   via  5b4b19cd7f4c0ef7a479306b73a4db3733df9c44 (commit)
   via  a852b2822620b6294e2efa411ff88a3b0afe1754 (commit)
   via  7134da63690e0798d498fd15a09786db42091bfd (commit)
   via  4dbcc003b6aa8fcf009be1e98c8ff13fe1def2fb (commit)
  from  8cc8e9cbbefdf6fac368031d3dacc74d001f23b7 (commit)

Those revisions listed above that are new to this repository have
not appeared on any other notification email; so we list those
revisions in full, below.

- Log -
commit 656fd6d5d8f455492b1e51add9e6e1da8f911a7c
Author: Luke Shumaker 
Date:   Mon May 25 01:00:46 2015 -0600

devlist-mailer: Make LIST and FROM configurable

commit 0cf2ac49c18a71c050f5a9a9ff42e8dff89c2375
Author: Luke Shumaker 
Date:   Tue Jun 17 23:47:21 2014 -0400

Consistently use "$(dirname "$(readlink -e "$0")")"

This does correct handling of
 - executing a program by symlink
 - any weird characters in the full path
 - I'm sure there's another case I thought about when I originally did
   this.

commit 5b4b19cd7f4c0ef7a479306b73a4db3733df9c44
Author: Luke Shumaker 
Date:   Thu Mar 15 21:47:58 2018 -0400

Fixup quoting around variables, especially arrays

Search for unquoted variables using the command:

grep -Prn --exclude-dir=.git '(?
Date:   Wed Jun 18 12:07:09 2014 -0400

test/: Verify that db-update won't release duplicate packages in the same 
transaction

This is a leftover change from the patch that became 0432cff; v2 of that
patch broke db-functions:getpkgfiles, but the testsuite didn't catch it.
This patch adds a testcase that catches this type of breakage.

commit 7134da63690e0798d498fd15a09786db42091bfd
Author: Luke Shumaker 
Date:   Sun Apr 17 18:56:14 2016 -0400

README.md: Write an overview of the programs included

commit 4dbcc003b6aa8fcf009be1e98c8ff13fe1def2fb
Author: Luke Shumaker 
Date:   Thu Mar 15 23:54:14 2018 -0400

README.md: fix spelling mistake: "overriden"->"overridden"

---

Summary of changes:
 README.md| 57 +-
 config   |  7 ++-
 cron-jobs/check_archlinux/parse_pkgbuilds.sh | 42 -
 cron-jobs/devlist-mailer |  7 ++-
 cron-jobs/ftpdir-cleanup | 28 +--
 cron-jobs/integrity-check|  6 +--
 cron-jobs/sourceballs| 30 ++--
 cron-jobs/update-web-db  | 20 
 db-functions | 70 ++--
 db-move  | 56 +++---
 db-remove| 22 -
 db-repo-add  | 22 -
 db-repo-remove   | 22 -
 db-update| 36 +++---
 test/cases/db-update.bats|  7 +++
 testing2x| 26 +--
 16 files changed, 262 insertions(+), 196 deletions(-)


hooks/post-receive
-- 
Official repo DB scripts

Re: [arch-projects] [devtools] [PATCH] makechrootpkg: fix verifysource with pacman-git

[Eli Schwartz via arch-projects]

On 03/26/2018 06:19 PM, Luke Shumaker wrote:
>> -chmod 1777 "$builddir"
>> +chown "$makepkg_user:$makepkg_user" "$builddir"
> 
> $makepkg_user isn't nescessarily a valid group name.  Not all users
> have an identically named group, some people like to use 'users' as
> their primary group.
> 
> Looking at makepkg d8717a6a9666ec80c8645d190d6f9c7ab73084ac, I don't
> think the group of the directory has to match; just the user.
> However, if I'm mistaken and it it truly is nescessary to set the
> group, how about:
> 
>   chown "$makepkg_user:$(id -gn "$makepkg_user")" "$builddir"

mmm, fair point. chown should actually be able to handle this itself via

chown "$makepkg_user:" "$builddir"

I guess it doesn't matter if the group is weird, except aesthetically.

-- 
Eli Schwartz
Bug Wrangler and Trusted User



signature.asc
Description: OpenPGP digital signature

[arch-projects] [netctl] [GIT] Network control utility based on systemd branch master updated. 1.15-5-gdbc32d4

[Jouke Witteveen via arch-projects]

This is an automated email from the git hooks/post-receive script. It was
generated because a ref change was pushed to the repository containing
the project "Network control utility based on systemd".

The branch, master has been updated
   via  dbc32d470420396f4af360af9ac17680fe99b368 (commit)
   via  6d3644be38aa9d273cdc0989404804dad8818844 (commit)
  from  cef231d762e7bb65bcf9a6a040ba367c5062a8f8 (commit)

Those revisions listed above that are new to this repository have
not appeared on any other notification email; so we list those
revisions in full, below.

- Log -
commit dbc32d470420396f4af360af9ac17680fe99b368
Author: Jouke Witteveen 
Date:   Thu Mar 22 10:47:26 2018 +0100

1.16 release updates

commit 6d3644be38aa9d273cdc0989404804dad8818844
Author: Jouke Witteveen 
Date:   Thu Mar 22 10:46:20 2018 +0100

Reference profile options with a '='-suffix

---

Summary of changes:
 Makefile  |  4 +--
 NEWS  |  4 +++
 docs/netctl.profile.5.txt | 65 ---
 3 files changed, 39 insertions(+), 34 deletions(-)


hooks/post-receive
-- 
Network control utility based on systemd

[arch-projects] [netctl] [GIT] Network control utility based on systemd annotated tag 1.16 created. 1.16

[Jouke Witteveen via arch-projects]

This is an automated email from the git hooks/post-receive script. It was
generated because a ref change was pushed to the repository containing
the project "Network control utility based on systemd".

The annotated tag, 1.16 has been created
at  82dffc523922a470b222198e49bb2f5ef384ef75 (tag)
   tagging  dbc32d470420396f4af360af9ac17680fe99b368 (commit)
  replaces  1.15
 tagged by  Jouke Witteveen
on  Thu Mar 22 10:54:04 2018 +0100

- Log -
Release 1.16
-BEGIN PGP SIGNATURE-

iQFKBAABCAA0FiEEbql9KtO+F66lkufaVrYfv4JJLBUFAlqzfUEWHGoud2l0dGV2
ZWVuQGdtYWlsLmNvbQAKCRBWth+/gkksFU4vCADmnNsbgSnLiZso5sYcTzg8XhSV
I1BIG9PfeDVXuDekpGQMhhCh0+GUyuPE3Pxsem07m6WZJ+O/89QGmisLggb23t7X
P1/l97Y8AeSODhPIXAm7f3G1CWa1ciXyxEpFe3PACmajIE6CkTbTko4X2bi3ZYN4
8fSQTeOAfaTpEaaGgbjqRYHj7nlot+qbo1l/pApwv69ey4TKHItjN7umw6S43rvl
7f4NVZhWtulINlFNgFnsGoD9mW3k6mVKBHun6HUTfCfX5w5+O0SccazoKwanEEEL
GSxl7CUw/6G3AmGRFL0Yg6Ncqf4vWcLMVRL784ZQkctiXfiGKVTOrKsmIJTg
=eByd
-END PGP SIGNATURE-

Erich Eckner (1):
  Fix quoting of After= in generated service files (FS#57850)

Jouke Witteveen (4):
  Technical documentation fixes
  Only wait when necessary in PPP cleanup code
  Reference profile options with a '='-suffix
  1.16 release updates

---


hooks/post-receive
-- 
Network control utility based on systemd

[arch-projects] [dbscripts] [GIT] Official repo DB scripts branch pacman-git created. 20131102-82-gcda3694

[Eli Schwartz via arch-projects]

This is an automated email from the git hooks/post-receive script. It was
generated because a ref change was pushed to the repository containing
the project "Official repo DB scripts".

The branch, pacman-git has been created
at  cda3694d0b247849e12edcd56e0ca7a41ec90ae1 (commit)

- Log -
commit cda3694d0b247849e12edcd56e0ca7a41ec90ae1
Author: Eli Schwartz 
Date:   Wed Mar 21 12:09:21 2018 -0400

pacman 5.1 compatibility

The print_all_package_names function changed its behavior, adapt

---


hooks/post-receive
-- 
Official repo DB scripts

[arch-projects] [dbscripts] [GIT] Official repo DB scripts branch master updated. 20131102-81-g8cc8e9c

[Eli Schwartz via arch-projects]

This is an automated email from the git hooks/post-receive script. It was
generated because a ref change was pushed to the repository containing
the project "Official repo DB scripts".

The branch, master has been updated
   via  8cc8e9cbbefdf6fac368031d3dacc74d001f23b7 (commit)
   via  0432cffc42040f852b9a55c27316248da9064b67 (commit)
   via  36087fbd8b030fa6f908fdbb667292e3c078b615 (commit)
   via  23c2b82c336bf19b7a29a90d19bca4423d8b8839 (commit)
   via  97e17a5996425e8b7c3c1765a3c5074e1c4ff38d (commit)
   via  9672d6ec64ff5773df4b22bca9f2da44303dc03f (commit)
   via  41333413f19e025d800e02c83511b85bee65ac60 (commit)
   via  5c867ea3b857cba0993fc18e064c6f02f70ee099 (commit)
  from  da49ea616e162d3cfcb4b6a4242f497e94c36564 (commit)

Those revisions listed above that are new to this repository have
not appeared on any other notification email; so we list those
revisions in full, below.

- Log -
commit 8cc8e9cbbefdf6fac368031d3dacc74d001f23b7
Author: Luke Shumaker 
Date:   Wed Jun 18 20:33:06 2014 -0400

Clean up printf-formatters for user messages

 - db-functions: getpkgfile: The .sig file extension should be part of
   the filename parameter, not part of the message format string.

 - db-functions: arch_repo_modify: Shouldn't use ${action}
   string-interpolation in the message format string.  Since the
   entire message is a command, and we're using @Q to escape arguments
   anyway, go ahead and just construct the entire command as a single
   string that way, and feed it to '%s'.

commit 0432cffc42040f852b9a55c27316248da9064b67
Author: Luke Shumaker 
Date:   Wed Jun 18 12:07:09 2014 -0400

Fixups near unquoted variables

Using the following command to find unquoted variables (and ignoring
more than a a few false positives),

grep -Prn --exclude-dir=.git '(?https://git.archlinux.org/pacman.git/commit/?id=9e52a36794552b77ecf26f7f34b226d096978f1e
 - sourceballs: Avoid using ary=($string) to do field separation by
   using `read` and test that multiple licenses actually work as
   expected.
 - sourceballs: Replace `[[ -z ${ary[*]} ]]` with test for the array
   length
 - db-functions: Replace mangling echo field separators using sed, with
   printf formatters
 - db-functions: Replace for/echo loop to print an array line by line,
   with `printf '%s\n'`
 - db-functions: set_repo_permissions: Line up error messages, quote
   "$group"
 - db-move: Replace `$(echo ${array[@]})` with `${array[*]}`
 - testing2x: Use `"$@"` instead of `$*` when looping over an array

Also, not really quoting related but on the same line as a quoting
issue, optimize:

 - db-functions: Replace
[[ -n "$(... | sort | uniq -D)" ]]
   with
! ... | awk 'a[$0]++{exit 1}'

commit 36087fbd8b030fa6f908fdbb667292e3c078b615
Author: Luke Shumaker 
Date:   Wed Mar 14 16:52:05 2018 -0400

Remove uses of the "v=true; if $v ..." anti-pattern

Instead, compare the value of $v to 'true'.

commit 23c2b82c336bf19b7a29a90d19bca4423d8b8839
Author: Luke Shumaker 
Date:   Sun Apr 17 12:18:02 2016 -0400

Don't use `grep -q` when operating on piped stdin

(By default, prefer `grep &>/dev/null`)

`grep -q` may exit as soon as it finds a match; this is a good optimization
for when the input is a file.  However, if the input is the output of
another program, then that other program will receive SIGPIPE, and further
writes will fail.  When this happens, it might (bsdtar does) print a
message about a "write error" to stderr.  Which is going to confuse and
alarm the user.

In one of the cases (in common.bash, in the test suite), this had
already been mitigated by wrapping bsdtar in "echo "$(bsdtar ...)", as
Bash builtin echo doesn't complain if it gets SIGPIPE.  However, that
means we're storing the entire output of bsdtar in memory, which is
silly.  Additionally, the way it was implemented is also wrong;
because it was being used with `grep -qv` instead of just `grep -q`,
it *always* found a non-matching 

[arch-projects] [devtools] [PATCH] makechrootpkg: fix verifysource with pacman-git

[Eli Schwartz via arch-projects]

In pacman-git commit d8717a6a9666ec80c8645d190d6f9c7ab73084ac makepkg
started checking that the setuid/setgid bit could be removed on the
$BUILDDIR in order to prevent this propagating to the packages
themselves.  Unfortunately, this requires the temporary builddir used
during the --verifysource stage of makepkg, to be owned by $makepkg_user
which was not the case as it is created as root using mktemp (and given
world rwx in addition to the restricted deletion bit.)

Obviously makepkg cannot chmod a directory that it does not own. Fix
this by making $makepkg_user the owner of that directory, as should have
been the case all along.

(Giving world rwx is illogical on general principle. The fact that this
is a workaround for makepkg demanding these directories be writable even
when they are not going to be used for the makepkg options in question,
is not justification for being careless.)

Signed-off-by: Eli Schwartz 
---

Yay, I "broke" something. :D

 makechrootpkg.in | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/makechrootpkg.in b/makechrootpkg.in
index afcd121..6bc82a4 100644
--- a/makechrootpkg.in
+++ b/makechrootpkg.in
@@ -249,7 +249,7 @@ download_sources() {
 
local builddir
builddir="$(mktemp -d)"
-   chmod 1777 "$builddir"
+   chown "$makepkg_user:$makepkg_user" "$builddir"
 
# Ensure sources are downloaded
sudo -u "$makepkg_user" env SRCDEST="$SRCDEST" BUILDDIR="$builddir" \
-- 
2.16.2

[arch-projects] [netctl] [GIT] Network control utility based on systemd branch master updated. 1.15-3-gcef231d

[Jouke Witteveen via arch-projects]

This is an automated email from the git hooks/post-receive script. It was
generated because a ref change was pushed to the repository containing
the project "Network control utility based on systemd".

The branch, master has been updated
   via  cef231d762e7bb65bcf9a6a040ba367c5062a8f8 (commit)
   via  c3b4f7515dfb78220528eb6c93931eec34aa9821 (commit)
  from  2d7ec0b8459c4d42b43da1b6727a91e701bfaadc (commit)

Those revisions listed above that are new to this repository have
not appeared on any other notification email; so we list those
revisions in full, below.

- Log -
commit cef231d762e7bb65bcf9a6a040ba367c5062a8f8
Author: Erich Eckner 
Date:   Fri Mar 16 12:02:45 2018 +0100

Fix quoting of After= in generated service files (FS#57850)

commit c3b4f7515dfb78220528eb6c93931eec34aa9821
Author: Jouke Witteveen 
Date:   Mon Feb 26 12:59:30 2018 +0100

Only wait when necessary in PPP cleanup code

---

Summary of changes:
 src/lib/connections/mobile_ppp | 4 ++--
 src/lib/connections/pppoe  | 4 ++--
 src/netctl.in  | 2 +-
 3 files changed, 5 insertions(+), 5 deletions(-)


hooks/post-receive
-- 
Network control utility based on systemd

[arch-projects] [dbscripts] [GIT] Official repo DB scripts branch master updated. 20131102-73-gda49ea6

[Eli Schwartz via arch-projects]

This is an automated email from the git hooks/post-receive script. It was
generated because a ref change was pushed to the repository containing
the project "Official repo DB scripts".

The branch, master has been updated
   via  da49ea616e162d3cfcb4b6a4242f497e94c36564 (commit)
   via  4ae3ea2f71344045fa4990c0524b4c662ab83cfc (commit)
   via  33aae318542016f66d2f0e09654606649d404ff2 (commit)
   via  f22f1554720da245955337782ce554d5c3f93c8c (commit)
  from  e53cad6e4a8284165c6d0b2c7c86f6c077be693b (commit)

Those revisions listed above that are new to this repository have
not appeared on any other notification email; so we list those
revisions in full, below.

- Log -
commit da49ea616e162d3cfcb4b6a4242f497e94c36564
Author: Luke Shumaker 
Date:   Thu Feb 22 21:15:40 2018 -0500

test: Fixup glob matching

 - ftpdir-cleanup.bats: Glob expansion does not occur in [[ -f ]] tests.
   The [[ ! -f .../${pkgname}-*${PKGEXT} ]] checks were checking that there
   were no files containing a literal '*' for that part of their name.
   Obviously, this isn't what was intended.

 - sourceballs.bats: [ -r ] checks explode if the glob returns >1 file.
   Avoid using them if the path being checked contains a glob.

commit 4ae3ea2f71344045fa4990c0524b4c662ab83cfc
Author: Luke Shumaker 
Date:   Thu Feb 22 21:15:39 2018 -0500

test: common.bash:__getCheckSum: Don't rely on IFS

I managed to stumble across a bug in BATS where the run() function
screwed with the global IFS.  The bug has been fixed in git, but isn't
in a release yet.

https://github.com/sstephenson/bats/issues/89

Anyway, this bug breaks __getCheckSum().  Fortunately, we have avoided
tripping it so far because luck has it that we never call
__getCheckSum() after run() in the same test.

So, there's nothing actually broken here, but it makes me nervous.  So
go ahead and modify __getCheckSum to not rely on IFS.

And, while we're at it: declare the result variable and set it as
separate commands.  Doing both in the same command masks the exit code
of the subshell expansion.  We don't explicitly check the exit code,
but BATS runs the test suite with `set -e`, so splitting it does mean
that BATS will now detect errors from sha1sum.  We don't really expect
that to happen, but if BATS will give us error checking on it for
free, why not?

commit 33aae318542016f66d2f0e09654606649d404ff2
Author: Luke Shumaker 
Date:   Thu Feb 22 21:15:41 2018 -0500

Update messages to make fuller use of printf formatters

These are things that were (IMO) missed in 5afac1e.  I found them using:

git grep -E '(plain|msg|msg2|warning|error|die) "[^"]*\$'

I went a little above-and-beyond for escaping strings for the error
messages in db-functions' arch_repo_add and arch_repo_remove.  The
code should explain itself, but I wanted to point it out, as it's more than
the usual "slap %s in there, and move the ${...} to the right".

commit f22f1554720da245955337782ce554d5c3f93c8c
Author: Eli Schwartz 
Date:   Mon Feb 26 17:57:31 2018 -0500

db-functions: unify the repo-add/repo-remove caller

Use less duplicated logic, opening the way to more elegantly change
the interface for the repo-add/repo-remove scripts which are themselves
symlinked together and mostly have the same interface.

---

Summary of changes:
 cron-jobs/ftpdir-cleanup   |  8 
 cron-jobs/integrity-check  |  2 +-
 cron-jobs/sourceballs  |  8 
 db-functions   | 34 --
 db-move|  8 
 db-remove  |  4 ++--
 db-repo-add|  4 ++--
 db-repo-remove |  4 ++--
 db-update  |  4 ++--
 test/cases/ftpdir-cleanup.bats |  4 ++--
 test/cases/sourceballs.bats|  4 ++--
 test/lib/common.bash   | 15 +--
 testing2x  |  2 +-
 13 files changed, 51 insertions(+), 50 deletions(-)


hooks/post-receive
-- 
Official repo DB scripts

Re: [arch-projects] [dbscripts] [PATCH 3/8] Export TMPDIR, and use mktemp -t instead of making it part of the template

[Eli Schwartz via arch-projects]

On 03/13/2018 09:52 PM, Luke Shumaker wrote:
> From: Luke Shumaker 


> diff --git a/test/cases/db-update.bats b/test/cases/db-update.bats
> index e7e4489..2e44b91 100644
> --- a/test/cases/db-update.bats
> +++ b/test/cases/db-update.bats
> @@ -222,7 +222,7 @@ load ../lib/common
>  
>  @test "package has to be aregular file" {
>   local p
> - local target=$(mktemp -d)
> + local target=$(mktemp -dt)
>   local arches=('i686' 'x86_64')
>  
>   releasePackage extra 'pkg-simple-a'
> diff --git a/test/lib/common.bash b/test/lib/common.bash
> index 568a541..45e4800 100644
> --- a/test/lib/common.bash
> +++ b/test/lib/common.bash
> @@ -83,7 +83,7 @@ setup() {
>   local a
>   PKGEXT=".pkg.tar.xz"
>  
> - TMP="$(mktemp -d)"
> + TMP="$(mktemp -dt)"
>  
>   export DBSCRIPTS_CONFIG=${TMP}/config.local
>   cat < "${DBSCRIPTS_CONFIG}"
> 

These two have no TEMPLATE given anyways, so this change is extraneous.

-- 
Eli Schwartz
Bug Wrangler and Trusted User



signature.asc
Description: OpenPGP digital signature

Re: [arch-projects] [dbscripts] [PATCH 1/8] Fix quoting around variables, especially arrays.

[Eli Schwartz via arch-projects]

On 03/14/2018 12:53 AM, Luke Shumaker wrote:
> Part of it is to have a common style.  Trying to rectify two codebases
> that diverged 7 years ago is rough.  When trying to come up with clean
> diffs, having to guess "did the other one quote this variable?" makes
> it harder.  If you can say "always quote (except for the LHS of [[
> ]])" or something, that makes it a bit easier.

I'm not sure that "specifically for the sole sake of diffs against our
fork" is a valid justification on its own for modifying a coding style.

>>>  backup_package_variables() {
>>> -   for var in ${splitpkg_overrides[@]}; do
>>> +   for var in "${splitpkg_overrides[@]}"; do
>>> indirect="${var}_backup"
>>> -   eval "${indirect}=(\${$var[@]})"
>>> +   eval "${indirect}=(\"\${$var[@]}\")"
>>> done
>>>  }
>>>  
>>>  restore_package_variables() {
>>> -   for var in ${splitpkg_overrides[@]}; do
>>> +   for var in "${splitpkg_overrides[@]}"; do
>>> indirect="${var}_backup"
>>> if [ -n "${!indirect}" ]; then
>>> -   eval "${var}=(\${$indirect[@]})"
>>> +   eval "${var}=(\"\${$indirect[@]}\")"
>>> else
>>> -   unset ${var}
>>> +   unset "${var}"
>>> fi
>>> done
>>
>> This is too much escaping and metaprogramming, there are better ways of
>> backing up a variable to begin with. :/
>>
>> We do it in makepkg, I will have us do it here as well. Advantage: using
>> declare -p means the shell auto-escapes things where needed.
> 
> I haven't been keeping my thumb on makepkg git, but the eval lines as
> I wrote them exactly match the eval lines in makepkg 5.0.2's version
> of {backup,restore}_package_variables (makepkg's versions don't quote
> the for loops, or the unset command).

Hmm, I was thinking of:

eval "$restoretrap"
eval "$restoreset"
eval "$restoreshopt"
eval "$restore_envvars"

and similar. Maybe I should fix the backups as well, but that is a
slightly more complicated case there.

>>> -   if ! ${CLEANUP_DRYRUN}; then
>>> +   if ! "${CLEANUP_DRYRUN}"; then
>>
>> This is a variable being run as a command, so if there were to be spaces
>> in it we'd end up trying to run a command with a space in it. Arguably
>> we should not be running this as a command (even though they are set to
>> true/false which is a shell builtin blah blah blah) but since we are it
>> would be illogical to indicate that if there are spaces they should be
>> interpreted as string literals in an executable filename.
> 
> For the true/false idiom, quoting it is just a style rule.  I figure
> accepting the true/false idiom doesn't imply allowing the boolean
> variable to have any value.  Having the quotes would help catch the
> variable being erroneously set to a different value.

So would doing a bash test.

> At some point, I'd like to have `make lint` run shellcheck over
> dbscripts.  That's a long way off, both because of a whole bunch of
> changes needed in dbscripts to make it come back clean, and a few
> features needed in shellcheck to avoid having to drop entirely too
> many shellcheck directives in to the dbscripts source.
> 
> Anyway, I know linters should be taken with a grain of salt, but when
> there's something simple like this, that you know just about any
> linter would complain about... why not?

That would imply one of my long-term goals is being able to run a linter.

If I did, this rule would be the first thing I disabled -- it is far,
far too prone to both false positives and false negatives.


-- 
Eli Schwartz
Bug Wrangler and Trusted User



signature.asc
Description: OpenPGP digital signature

Re: [arch-projects] [dbscripts] [PATCH 4/8] Use `grep &>/dev/null` instead of `grep -q` when operating on piped stdin.

[Eli Schwartz via arch-projects]

On 03/13/2018 09:52 PM, Luke Shumaker wrote:
> From: Luke Shumaker 
> 
> `grep -q` may exit as soon as it finds a match; this is a good optimization
> for when the input is a file.  However, if the input is the output of
> another program, then that other program will receive SIGPIPE, and further
> writes will fail.  When this happens, it might (bsdtar does) print a
> message about a "write error" to stderr.  Which is going to confuse and
> alarm the user.
> 
> In one of the cases, this had already been mitigated by wrapping
> bsdtar in "echo "$(bsdtar ...)", as Bash builtin echo doesn't complain
> if it gets SIGPIPE.  However, that means we're storing the entire
> output of bsdtar in memory, which is silly.
> ---
>  db-functions | 2 +-
>  test/lib/common.bash | 4 ++--
>  2 files changed, 3 insertions(+), 3 deletions(-)
> 
> diff --git a/db-functions b/db-functions
> index 58b753a..ee390ff 100644
> --- a/db-functions
> +++ b/db-functions
> @@ -303,7 +303,7 @@ check_pkgfile() {
>  
>   in_array "${pkgarch}" "${ARCHES[@]}" 'any' || return 1
>  
> - if echo "${pkgfile##*/}" | grep -q "^${pkgname}-${pkgver}-${pkgarch}"; 
> then
> + if echo "${pkgfile##*/}" | grep "^${pkgname}-${pkgver}-${pkgarch}" 
> &>/dev/null; then

But echo should be fine anyway?

Regardless this could be so much more elegant.

if [[ $pkgfile = $pkgname-$pkgver-$pkgrel-$arch* ]]; then

>   return 0
>   else
>   return 1
> diff --git a/test/lib/common.bash b/test/lib/common.bash
> index 45e4800..ab805dd 100644
> --- a/test/lib/common.bash
> +++ b/test/lib/common.bash
> @@ -215,7 +215,7 @@ checkPackageDB() {
>  
>   for db in ${DBEXT} ${FILESEXT}; do
>   [ -r 
> "${FTP_BASE}/${repo}/os/${repoarch}/${repo}${db%.tar.*}" ]
> - bsdtar -xf 
> "${FTP_BASE}/${repo}/os/${repoarch}/${repo}${db%.tar.*}" -O | grep -q 
> "${pkgfile%${PKGEXT}}"
> + bsdtar -xf 
> "${FTP_BASE}/${repo}/os/${repoarch}/${repo}${db%.tar.*}" -O | grep 
> "${pkgfile%${PKGEXT}}" &>/dev/null
>   done
>   done
>   done
> @@ -269,7 +269,7 @@ checkRemovedPackageDB() {
>   for tarch in ${tarches[@]}; do
>   if [ -r 
> "${FTP_BASE}/${repo}/os/${tarch}/${repo}${db%.tar.*}" ]; then
>   for pkgname in ${pkgnames[@]}; do
> - echo "$(bsdtar -xf 
> "${FTP_BASE}/${repo}/os/${tarch}/${repo}${db%.tar.*}" -O)" | grep -qv 
> ${pkgname}
> + bsdtar -xf 
> "${FTP_BASE}/${repo}/os/${tarch}/${repo}${db%.tar.*}" -O | grep -v ${pkgname} 
> &>/dev/null
>   done
>   fi
>   done
> 


-- 
Eli Schwartz
Bug Wrangler and Trusted User



signature.asc
Description: OpenPGP digital signature

Re: [arch-projects] [dbscripts] [PATCH 8/8] devlist-mailer: Make LIST and FROM configurable

[Eli Schwartz via arch-projects]

On 03/13/2018 09:52 PM, Luke Shumaker wrote:
> From: Luke Shumaker 

TBH we don't even send out integrity check email anymore, do you?

-- 
Eli Schwartz
Bug Wrangler and Trusted User



signature.asc
Description: OpenPGP digital signature

Re: [arch-projects] [dbscripts] [PATCH 1/8] Fix quoting around variables, especially arrays.

[Eli Schwartz via arch-projects]

On 03/13/2018 09:51 PM, Luke Shumaker wrote:
> From: Luke Shumaker 
> 
> Other than pure quoting, this involved:
>  - swapping */@ for array access in a few places
>  - fiddling with printf in a pipeline
>  - replacing `$(echo ${array[@]})` with `${array[*]}`
>  - replacing `echo $(...)` with `...`
> 
> When searching for these things, I used the command:
> 
> grep -Prn --exclude-dir=.git '(? 
> and ignored a bunch of false positives.

I don't really see the need to quote every variable just because it is a
variable, at least in cases where the variable is fairly statically
defined... I'd like to see path components that could have spaces
quoted, and arrays I guess because semantically that's how you iterate
over an array. Also you introduced some bugs, in cases where we actually
want whitespace splitting...

>  backup_package_variables() {
> - for var in ${splitpkg_overrides[@]}; do
> + for var in "${splitpkg_overrides[@]}"; do
>   indirect="${var}_backup"
> - eval "${indirect}=(\${$var[@]})"
> + eval "${indirect}=(\"\${$var[@]}\")"
>   done
>  }
>  
>  restore_package_variables() {
> - for var in ${splitpkg_overrides[@]}; do
> + for var in "${splitpkg_overrides[@]}"; do
>   indirect="${var}_backup"
>   if [ -n "${!indirect}" ]; then
> - eval "${var}=(\${$indirect[@]})"
> + eval "${var}=(\"\${$indirect[@]}\")"
>   else
> - unset ${var}
> + unset "${var}"
>   fi
>   done

This is too much escaping and metaprogramming, there are better ways of
backing up a variable to begin with. :/

We do it in makepkg, I will have us do it here as well. Advantage: using
declare -p means the shell auto-escapes things where needed.

> - if ! ${CLEANUP_DRYRUN}; then
> + if ! "${CLEANUP_DRYRUN}"; then

This is a variable being run as a command, so if there were to be spaces
in it we'd end up trying to run a command with a space in it. Arguably
we should not be running this as a command (even though they are set to
true/false which is a shell builtin blah blah blah) but since we are it
would be illogical to indicate that if there are spaces they should be
interpreted as string literals in an executable filename.


> -${CLEANUP_DRYRUN} && warning 'dry run mode is active'
> +"${CLEANUP_DRYRUN}" && warning 'dry run mode is active'

Same.

> - if ! ${CLEANUP_DRYRUN}; then
> + if ! "${CLEANUP_DRYRUN}"; then

Same.

>  # Create a readable file for each repo with the following format
>  #  -  [ ]

When we consume this file...

>   while read line; do
> - pkginfo=(${line})
> + pkginfo=("${line}")

That's completely wrong, just look at the next five lines.

>   pkgbase=${pkginfo[0]}
>   pkgver=${pkginfo[1]}
>   pkgarch=${pkginfo[2]}
> - pkglicense=(${pkginfo[@]:3})
> + pkglicense=("${pkginfo[@]:3}")

How will we extract elements of this array, if your quoting squashes
everything down into one array element? ${pkginfo[0]} will have too
much, and the other elements simply won't exist at all.

We go from having:

declare -a pkginfo=([0]="foo" [1]="1.0-1" [2]="any" [3]="GPL")

to having:

declare -a pkginfo=([0]="foo 1.0-1 any GPL")

> - if ! ([[ -z ${ALLOWED_LICENSES[@]} ]] || chk_license 
> ${pkglicense[@]} || grep -Fqx "${pkgbase}" "${dirname}/sourceballs.force"); 
> then
> + if ! ([[ -z ${ALLOWED_LICENSES[*]} ]] || chk_license 
> "${pkglicense[@]}" || grep -Fqx "${pkgbase}" "${dirname}/sourceballs.force"); 
> then

What's the check here anyways? This considers ALLOWED_LICENSES=('') to
be non-empty, so we might as well check the length of
${#ALLOWED_LICENSES[@]} which is a clearer read.


> - ${SOURCE_CLEANUP_DRYRUN} && warning 'dry run mode is active'
> - for old_pkg in ${old_pkgs[@]}; do
> + "${SOURCE_CLEANUP_DRYRUN}" && warning 'dry run mode is active'
> + for old_pkg in "${old_pkgs[@]}"; do
>   msg2 "${old_pkg}"
> - if ! ${SOURCE_CLEANUP_DRYRUN}; then
> + if ! "${SOURCE_CLEANUP_DRYRUN}"; then
>   mv_acl "$FTP_BASE/${SRCPOOL}/${old_pkg}" 
> "${SOURCE_CLEANUP_DESTDIR}/${old_pkg}"
>   touch "${SOURCE_CLEANUP_DESTDIR}/${old_pkg}"
>   fi
> @@ -147,9 +147,9 @@ done
>  
>  if (( ${#old_pkgs[@]} >= 1 )); then
>   msg "Removing old source packages from the cleanup directory..."
> - for old_pkg in ${old_pkgs[@]}; do
> + for old_pkg in "${old_pkgs[@]}"; do
>   msg2 "${old_pkg}"
> - ${SOURCE_CLEANUP_DRYRUN} || rm -f 
> "${SOURCE_CLEANUP_DESTDIR}/${old_pkg}"
> + "${SOURCE_CLEANUP_DRYRUN}" || rm -f 
> "${SOURCE_CLEANUP_DESTDIR}/${old_pkg}"
>  

Re: [arch-projects] [dbscripts] [PATCH 0/8] Backports from Parabola

[Eli Schwartz via arch-projects]

On 03/13/2018 09:51 PM, Luke Shumaker wrote:
> BTW, now that dbscripts is on GitHub, is that the preferred way of
> submitting these?  Or is this mailing list still best?

It's been on github for quite some time, but I am okay with looking at
things in either location.

-- 
Eli Schwartz
Bug Wrangler and Trusted User



signature.asc
Description: OpenPGP digital signature

[vpn-help] Shrew Soft VPN, AD Login NAT issues

[Chris Wager - 56K Projects]

Dear Members

I have paid for the professional license.

I have an issue with the Shrew Soft VPN Client, I have configured the 
connection to my VPN Appliance and Enabled: force-rfc, in the Shrew Soft Client 
as this is required with the clients behind NAT and or on mobile data, the 
problem is that this works fine if they log in using cached credentials and 
then connect using the Shrew Soft Client it acknowledges the force-rfc and 
works fine.

However my security policy dictates cached credentials should be disabled, 
however when I attempt to use the AD Login component of Shrewsoft VPN Cleint it 
fails to acknowledge the force-rfc and therefore does not function. and login 
fails etc. etc. as NAT = force-rfc is needed for communication behind nat, 
solutions anyone ?

Regards Chris


___
vpn-help mailing list
vpn-help@lists.shrew.net
https://lists.shrew.net/mailman/listinfo/vpn-help

Re: [arch-projects] [devtools] makechrootpkg: respect GNUPGHOME

[Emiel Wiedijk via arch-projects]

On Tue, Feb 27, 2018 at 03:44:07PM +0100, Eli Schwartz via arch-projects wrote:
> On 02/27/2018 05:41 AM, Emiel Wiedijk via arch-projects wrote:
> > Correct, but makepkg --verifysource is run with sudo -u $myuser, and sudo
> > resets the environment. And the code that copies ~/.gnupg to the chroot 
> > apparantly hasn't been removed yet (as of 
> > 38c7a391b043547b946a99731a56a233458ba7a2).
> > I just assumed (apparantly wrongly) that it was for GnuPG related tasks in 
> > the
> > PKGBUILD, and adjusted the code to copy the correct directory.
> 
> My point is that there is really no point in trying to preserve it in
> the chroot, since that section is dead code to begin with, and if you do
> anyways then your patch may clash with other pending patches. So of the
> three changes your patch made, you should probably only make the second
> and third.
> 
> I don't blame you for actually thinking dead code did something. :D
> 
> -- 
> Eli Schwartz
> Bug Wrangler and Trusted User
> 
Fair enough, I sent an updated patch :-).

Emiel Wiedijk

[arch-projects] [devtools] [PATCH v2] makechrootpkg: respect GNUPGHOME

[Emiel Wiedijk via arch-projects]

Previously, makechrootpkg hardcoded ~/.gnupg. Therefore, if a user
uses a custom GPG home directory, the siganture checking would fail.
Now makechrootpkg uses $GNUPGHOME, with a fallback to ~/.gnupg.

Signed-off-by: Emiel Wiedijk 
---
 makechrootpkg.in | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

diff --git a/makechrootpkg.in b/makechrootpkg.in
index afcd121..653847f 100644
--- a/makechrootpkg.in
+++ b/makechrootpkg.in
@@ -252,7 +252,8 @@ download_sources() {
chmod 1777 "$builddir"
 
# Ensure sources are downloaded
-   sudo -u "$makepkg_user" env SRCDEST="$SRCDEST" BUILDDIR="$builddir" \
+   sudo -u "$makepkg_user" --preserve-env=GNUPGHOME \
+   env SRCDEST="$SRCDEST" BUILDDIR="$builddir" \
makepkg --config="$copydir/etc/makepkg.conf" --verifysource -o 
||
die "Could not download sources."
 
@@ -341,7 +342,7 @@ main() {
[[ -n $makepkg_user && -z $(id -u "$makepkg_user") ]] && die 'Invalid 
makepkg user.'
makepkg_user=${makepkg_user:-${SUDO_USER:-$USER}}
 
-   check_root SOURCE_DATE_EPOCH
+   check_root SOURCE_DATE_EPOCH,GNUPGHOME
 
# Canonicalize chrootdir, getting rid of trailing /
chrootdir=$(readlink -e "$passeddir")
-- 
2.16.2

Re: [arch-projects] [devtools] makechrootpkg: respect GNUPGHOME

[Eli Schwartz via arch-projects]

On 02/27/2018 05:41 AM, Emiel Wiedijk via arch-projects wrote:
> Correct, but makepkg --verifysource is run with sudo -u $myuser, and sudo
> resets the environment. And the code that copies ~/.gnupg to the chroot 
> apparantly hasn't been removed yet (as of 
> 38c7a391b043547b946a99731a56a233458ba7a2).
> I just assumed (apparantly wrongly) that it was for GnuPG related tasks in the
> PKGBUILD, and adjusted the code to copy the correct directory.

My point is that there is really no point in trying to preserve it in
the chroot, since that section is dead code to begin with, and if you do
anyways then your patch may clash with other pending patches. So of the
three changes your patch made, you should probably only make the second
and third.

I don't blame you for actually thinking dead code did something. :D

-- 
Eli Schwartz
Bug Wrangler and Trusted User



signature.asc
Description: OpenPGP digital signature

Re: [arch-projects] [devtools] makechrootpkg: respect GNUPGHOME

[Emiel Wiedijk via arch-projects]

​> On 02/26/2018 02:03 PM, Emiel Wiedijk via arch-projects wrote:
> 
> > Previously, makechrootpkg hardcoded ~/.gnupg. Therefore, if a user
> > 
> > uses a custom GPG home directory, the siganture checking would fail.
> > 
> > Now makechrootpkg uses $GNUPGHOME, with a fallback to ~/.gnupg.
> 
> There is no signature checking in the chroot, see
> 
> https://lists.archlinux.org/pipermail/arch-projects/2018-January/004709.html
> 
> 
> ---
> 
> Eli Schwartz
> 
> Bug Wrangler and Trusted User

Correct, but makepkg --verifysource is run with sudo -u $myuser, and sudo
resets the environment. And the code that copies ~/.gnupg to the chroot 
apparantly hasn't been removed yet (as of 
38c7a391b043547b946a99731a56a233458ba7a2).
I just assumed (apparantly wrongly) that it was for GnuPG related tasks in the
PKGBUILD, and adjusted the code to copy the correct directory.

Emiel Wiedijk

Re: [arch-projects] [dbscripts] [PATCH v2 3/3] Update messages to make fuller use of printf formatters

[Eli Schwartz via arch-projects]

On 02/26/2018 05:14 PM, Luke Shumaker wrote:
> On Mon, 26 Feb 2018 00:46:48 -0500,
> Eli Schwartz wrote:
>>> +++ b/db-functions
>>> @@ -450,7 +450,7 @@ arch_repo_add() {
>>> # package files might be relative to repo dir
>>> pushd "${FTP_BASE}/${repo}/os/${arch}" >/dev/null
>>> /usr/bin/repo-add -q "${repo}${DBEXT}" ${pkgs[@]} \
>>> -   || error "repo-add ${repo}${DBEXT} ${pkgs[@]}"
>>> +   || error 'repo-add %q %s' "${repo}${DBEXT}" "${pkgs[*]@Q}"
>>> popd >/dev/null
>>> set_repo_permission "${repo}" "${arch}"
>>>  
>>> @@ -468,7 +468,7 @@ arch_repo_remove() {
>>> return 1
>>> fi
>>> /usr/bin/repo-remove -q "${dbfile}" ${pkgs[@]} \
>>> -   || error "repo-remove ${dbfile} ${pkgs[@]}"
>>> +   || error 'repo-remove %q %s' "$dbfile" "${pkgs[*]@Q}"
>>> set_repo_permission "${repo}" "${arch}"
>>
>> I think for consistency we should use the same style which means using
>> "${dbfile@Q}"
> 
> I was going for consistency with the repo-add version, which doesn't
> have a single dbfile variable to @Q.  Would you have me introduce a
> dbfile variable in arch_repo_add?

Well, we basically use it hardcoded three times, so why not. :p

Just like the repo-add version, except with pushd "${dbfile%/*}"

Actually, this decreases the difference between the two enough that we
may want to just have one call the other... or do this:

https://lists.archlinux.org/pipermail/arch-projects/2018-February/004832.html

-- 
Eli Schwartz
Bug Wrangler and Trusted User



signature.asc
Description: OpenPGP digital signature

[arch-projects] [dbscripts] [PATCH] db-functions: unify the repo-add/repo-remove caller

[Eli Schwartz via arch-projects]

Use less duplicated logic, opening the way to more elegantly change
the interface for the repo-add/repo-remove scripts which are themselves
symlinked together and mostly have the same interface.

Signed-off-by: Eli Schwartz 
---
 db-functions   | 34 --
 db-move|  4 ++--
 db-remove  |  2 +-
 db-repo-add|  2 +-
 db-repo-remove |  2 +-
 db-update  |  2 +-
 6 files changed, 18 insertions(+), 28 deletions(-)

diff --git a/db-functions b/db-functions
index 8b71cae..653f130 100644
--- a/db-functions
+++ b/db-functions
@@ -442,34 +442,24 @@ set_repo_permission() {
fi
 }
 
-arch_repo_add() {
-   local repo=$1
-   local arch=$2
-   local pkgs=(${@:3})
-
-   # package files might be relative to repo dir
-   pushd "${FTP_BASE}/${repo}/os/${arch}" >/dev/null
-   /usr/bin/repo-add -q "${repo}${DBEXT}" ${pkgs[@]} \
-   || error "repo-add ${repo}${DBEXT} ${pkgs[@]}"
-   popd >/dev/null
-   set_repo_permission "${repo}" "${arch}"
-
-   REPO_MODIFIED=1
-}
-
-arch_repo_remove() {
-   local repo=$1
-   local arch=$2
-   local pkgs=(${@:3})
+arch_repo_modify() {
+   local action=$1
+   local repo=$2
+   local arch=$3
+   local pkgs=("${@:4}")
local dbfile="${FTP_BASE}/${repo}/os/${arch}/${repo}${DBEXT}"
 
-   if [[ ! -f ${dbfile} ]]; then
+   if [[ ${action} = remove && ! -f ${dbfile} ]]; then
error "No database found at '%s'" "$dbfile"
return 1
fi
-   /usr/bin/repo-remove -q "${dbfile}" ${pkgs[@]} \
-   || error "repo-remove ${dbfile} ${pkgs[@]}"
+
+   # package files for repo-add might be relative to repo dir
+   pushd "${dbfile%/*}" >/dev/null
+   /usr/bin/repo-${action} -q "${dbfile}" ${pkgs[@]} \
+   || error "repo-${action} ${dbfile} ${pkgs[@]}"
set_repo_permission "${repo}" "${arch}"
+   popd >/dev/null
 
REPO_MODIFIED=1
 }
diff --git a/db-move b/db-move
index fb7ebac..e9c9fa7 100755
--- a/db-move
+++ b/db-move
@@ -114,8 +114,8 @@ done
 
 for tarch in ${ARCHES[@]}; do
if [[ -n ${add_pkgs[${tarch}]} ]]; then
-   arch_repo_add "${repo_to}" "${tarch}" ${add_pkgs[${tarch}]}
-   arch_repo_remove "${repo_from}" "${tarch}" 
${remove_pkgs[${tarch}]}
+   arch_repo_modify add "${repo_to}" "${tarch}" 
${add_pkgs[${tarch}]}
+   arch_repo_modify remove "${repo_from}" "${tarch}" 
${remove_pkgs[${tarch}]}
fi
 done
 
diff --git a/db-remove b/db-remove
index 70502bc..08a95f9 100755
--- a/db-remove
+++ b/db-remove
@@ -47,6 +47,6 @@ for pkgbase in ${pkgbases[@]}; do
 done
 
 for tarch in ${tarches[@]}; do
-   arch_repo_remove "${repo}" "${tarch}" ${remove_pkgs[@]}
+   arch_repo_modify remove "${repo}" "${tarch}" ${remove_pkgs[@]}
repo_unlock $repo $tarch
 done
diff --git a/db-repo-add b/db-repo-add
index d7be302..4b16f7e 100755
--- a/db-repo-add
+++ b/db-repo-add
@@ -36,6 +36,6 @@ for tarch in ${tarches[@]}; do
msg "Adding %s to [%s]..." "$pkgfile" "$repo"
fi
done
-   arch_repo_add "${repo}" "${tarch}" ${pkgfiles[@]}
+   arch_repo_modify add "${repo}" "${tarch}" ${pkgfiles[@]}
repo_unlock $repo $tarch
 done
diff --git a/db-repo-remove b/db-repo-remove
index 32d167e..3077786 100755
--- a/db-repo-remove
+++ b/db-repo-remove
@@ -32,6 +32,6 @@ for tarch in ${tarches[@]}; do
for pkgname in ${pkgnames[@]}; do
msg "Removing %s from [%s]..." "$pkgname" "$repo"
done
-   arch_repo_remove "${repo}" "${tarch}" ${pkgnames[@]}
+   arch_repo_modify remove "${repo}" "${tarch}" ${pkgnames[@]}
repo_unlock $repo $tarch
 done
diff --git a/db-update b/db-update
index 4e17184..e8ae08d 100755
--- a/db-update
+++ b/db-update
@@ -94,7 +94,7 @@ for repo in ${repos[@]}; do
add_pkgs+=("${pkgfile}")
done
if (( ${#add_pkgs[@]} >= 1 )); then
-   arch_repo_add "${repo}" "${pkgarch}" ${add_pkgs[@]}
+   arch_repo_modify add "${repo}" "${pkgarch}" 
${add_pkgs[@]}
fi
done
 done
-- 
2.16.2

Re: [arch-projects] [devtools] makechrootpkg: respect GNUPGHOME

[Eli Schwartz via arch-projects]

On 02/26/2018 02:03 PM, Emiel Wiedijk via arch-projects wrote:
> Previously, makechrootpkg hardcoded ~/.gnupg. Therefore, if a user
> uses a custom GPG home directory, the siganture checking would fail.
> Now makechrootpkg uses $GNUPGHOME, with a fallback to ~/.gnupg.

There is no signature checking in the chroot, see
https://lists.archlinux.org/pipermail/arch-projects/2018-January/004709.html

-- 
Eli Schwartz
Bug Wrangler and Trusted User



signature.asc
Description: OpenPGP digital signature

Re: [arch-projects] [dbscripts] [PATCH v2 3/3] Update messages to make fuller use of printf formatters

[Eli Schwartz via arch-projects]

On 02/22/2018 09:15 PM, Luke Shumaker wrote:
> From: Luke Shumaker 
> 
> These are things that were (IMO) missed in 5afac1e.  I found them using:
> 
> git grep -E '(plain|msg|msg2|warning|error|die) "[^"]*\$'
> 
> I went a little above-and-beyond for escaping strings for the error
> messages in db-functions' arch_repo_add and arch_repo_remove.  The
> code should explain itself, but I wanted to point it out, as it's more than
> the usual "slap %s in there, and move the ${...} to the right".
> 
> v2:
>  - arch_repo_add, arch_repo_remove: Use Bash 4.4 parameter
>transformation to avoid having to introduce temporary variables.

> --- a/db-functions
> +++ b/db-functions
> @@ -450,7 +450,7 @@ arch_repo_add() {
>   # package files might be relative to repo dir
>   pushd "${FTP_BASE}/${repo}/os/${arch}" >/dev/null
>   /usr/bin/repo-add -q "${repo}${DBEXT}" ${pkgs[@]} \
> - || error "repo-add ${repo}${DBEXT} ${pkgs[@]}"
> + || error 'repo-add %q %s' "${repo}${DBEXT}" "${pkgs[*]@Q}"
>   popd >/dev/null
>   set_repo_permission "${repo}" "${arch}"
>  
> @@ -468,7 +468,7 @@ arch_repo_remove() {
>   return 1
>   fi
>   /usr/bin/repo-remove -q "${dbfile}" ${pkgs[@]} \
> - || error "repo-remove ${dbfile} ${pkgs[@]}"
> + || error 'repo-remove %q %s' "$dbfile" "${pkgs[*]@Q}"
>   set_repo_permission "${repo}" "${arch}"

I think for consistency we should use the same style which means using
"${dbfile@Q}"

Currently we'll get the dbfile backslash-escaped, followed by the
package names unequivocally quoted on the same line of error output. It
makes more sense to ensure that they are all quoted.

While in theory one could bikeshed over which is the better way to
consistently print them, there is only one option for us since we're
trying to interject all the ${pkgs[*]} as one argument...

>  
>   REPO_MODIFIED=1
> diff --git a/db-move b/db-move
> index fb7ebac..806ad9a 100755
> --- a/db-move
> +++ b/db-move
> @@ -4,7 +4,7 @@
>  . "$(dirname $0)/db-functions"
>  
>  if (( $# < 3 )); then
> - msg "usage: ${0##*/}    ..."
> + msg "usage: %s    ..." "${0##*/}"
>   exit 1
>  fi
>  
> @@ -74,7 +74,7 @@ for pkgbase in ${args[@]:2}; do
>   else
>   tarches=("${pkgarch}")
>   fi
> - msg2 "${pkgbase} ($(echo ${tarches[@]}))"
> + msg2 "%s (%s)" "$pkgbase" "${tarches[*]}"
>   pkgnames=($(. "${svnrepo_from}/PKGBUILD"; echo 
> ${pkgname[@]}))
>   pkgver=$(. "${svnrepo_from}/PKGBUILD"; get_full_version)
>  
> diff --git a/db-remove b/db-remove
> index 70502bc..e7aed31 100755
> --- a/db-remove
> +++ b/db-remove
> @@ -4,7 +4,7 @@
>  . "$(dirname $0)/db-functions"
>  
>  if (( $# < 3 )); then
> - msg "usage: ${0##*/}    ..."
> + msg "usage: %s    ..." "${0##*/}"
>   exit 1
>  fi
>  
> diff --git a/db-repo-add b/db-repo-add
> index d7be302..4ea758f 100755
> --- a/db-repo-add
> +++ b/db-repo-add
> @@ -4,7 +4,7 @@
>  . "$(dirname $0)/db-functions"
>  
>  if (( $# < 3 )); then
> - msg "usage: ${0##*/}..."
> + msg "usage: %s..." "${0##*/}"
>   exit 1
>  fi
>  
> diff --git a/db-repo-remove b/db-repo-remove
> index 32d167e..2f24edd 100755
> --- a/db-repo-remove
> +++ b/db-repo-remove
> @@ -4,7 +4,7 @@
>  . "$(dirname $0)/db-functions"
>  
>  if (( $# < 3 )); then
> - msg "usage: ${0##*/}..."
> + msg "usage: %s..." "${0##*/}"
>   exit 1
>  fi
>  
> diff --git a/db-update b/db-update
> index 4e17184..5077389 100755
> --- a/db-update
> +++ b/db-update
> @@ -78,7 +78,7 @@ for repo in ${repos[@]}; do
>   arch_pkgs=($(getpkgfiles 
> "${STAGING}/${repo}/"*-${pkgarch}${PKGEXTS} 2>/dev/null))
>   for pkg in ${arch_pkgs[@]} ${any_pkgs[@]}; do
>   pkgfile="${pkg##*/}"
> - msg2 "${pkgfile} (${pkgarch})"
> + msg2 '%s (%s)' "$pkgfile" "$pkgarch"
>   # any packages might have been moved by the previous run
>   if [[ -f ${pkg} ]]; then
>   mv "${pkg}" "$FTP_BASE/${PKGPOOL}"
> diff --git a/testing2x b/testing2x
> index f0d77cb..d68e405 100755
> --- a/testing2x
> +++ b/testing2x
> @@ -4,7 +4,7 @@
>  . "$(dirname $0)/db-functions"
>  
>  if (( $# < 1 )); then
> - msg "usage: ${0##*/}  ..."
> + msg "usage: %s  ..." "${0##*/}"
>   exit 1
>  fi
>  
> 


-- 
Eli Schwartz
Bug Wrangler and Trusted User



signature.asc
Description: OpenPGP digital signature

[arch-projects] [devtools] makechrootpkg: respect GNUPGHOME

[Emiel Wiedijk via arch-projects]

Previously, makechrootpkg hardcoded ~/.gnupg. Therefore, if a user
uses a custom GPG home directory, the siganture checking would fail.
Now makechrootpkg uses $GNUPGHOME, with a fallback to ~/.gnupg.

Signed-off-by: Emiel Wiedijk 
---
 makechrootpkg.in | 12 +++-
 1 file changed, 7 insertions(+), 5 deletions(-)

diff --git a/makechrootpkg.in b/makechrootpkg.in
index afcd121..5a79dc0 100644
--- a/makechrootpkg.in
+++ b/makechrootpkg.in
@@ -182,9 +182,10 @@ prepare_chroot() {
 
$install -d 
"$copydir"/{build,build/.gnupg,startdir,{pkg,srcpkg,src,log}dest}
 
-   for x in .gnupg/pubring.{kbx,gpg}; do
-   [[ -r $USER_HOME/$x ]] || continue
-   $install -m 644 "$USER_HOME/$x" "$copydir/build/$x"
+   for x in pubring.{kbx,gpg}; do
+   local pubring="${GNUPGHOME:-$USER_HOME/.gnupg}/$x"
+   [[ -r "$pubring" ]] || continue
+   $install -m 644 "$pubring" "$copydir/build/.gnupg/$x"
done
 
sed -e '/^MAKEFLAGS=/d' -e '/^PACKAGER=/d' -i 
"$copydir/etc/makepkg.conf"
@@ -252,7 +253,8 @@ download_sources() {
chmod 1777 "$builddir"
 
# Ensure sources are downloaded
-   sudo -u "$makepkg_user" env SRCDEST="$SRCDEST" BUILDDIR="$builddir" \
+   sudo -u "$makepkg_user" --preserve-env=GNUPGHOME \
+   env SRCDEST="$SRCDEST" BUILDDIR="$builddir" \
makepkg --config="$copydir/etc/makepkg.conf" --verifysource -o 
||
die "Could not download sources."
 
@@ -341,7 +343,7 @@ main() {
[[ -n $makepkg_user && -z $(id -u "$makepkg_user") ]] && die 'Invalid 
makepkg user.'
makepkg_user=${makepkg_user:-${SUDO_USER:-$USER}}
 
-   check_root SOURCE_DATE_EPOCH
+   check_root SOURCE_DATE_EPOCH,GNUPGHOME
 
# Canonicalize chrootdir, getting rid of trailing /
chrootdir=$(readlink -e "$passeddir")
-- 
2.16.2

Re: [arch-projects] [dbscripts] [PATCH v2 2/3] test: Fixup glob matching

[Eli Schwartz via arch-projects]

On 02/22/2018 09:15 PM, Luke Shumaker wrote:
>  - common.bash: Globbing happens on the RHS of a [[ = ]] test.
>This means that we must quote variables on the RHS that are to be taken
>verbatim.  This is surprising, because we don't need to quote the LHS.

Unless we intend to do a general style cleanup, it is unnecessary to
"fix" cases where variables with known static content aren't quoted.

-- 
Eli Schwartz
Bug Wrangler and Trusted User



signature.asc
Description: OpenPGP digital signature

Re: [arch-projects] [dbscripts] [PATCH 3/3] Update messages to make fuller use of printf formatters

[Eli Schwartz via arch-projects]

On 02/22/2018 08:52 PM, Luke Shumaker wrote:
> I guess I *should* have explained it a bit more; the escaping of the
> package list happens when assigning pkgs_str:
> 
>   printf -v pkgs_str -- '%q ' "${pkgs[@]}"

Hmm, true. But the version without the additional variable wins IMO.

> Anyway, your simpler version LGTM.  Shall I submit a v2 patchset?

Yes please.

-- 
Eli Schwartz
Bug Wrangler and Trusted User



signature.asc
Description: OpenPGP digital signature

Re: [arch-projects] [dbscripts] [PATCH 1/3] test: common.bash:__getCheckSum: Don't rely on IFS

[Eli Schwartz via arch-projects]

On 02/22/2018 06:43 PM, Luke Shumaker wrote:
> On Thu, 22 Feb 2018 16:43:36 -0500,
> Eli Schwartz wrote:
>>>  __getCheckSum() {
>>> -   local result=($(sha1sum $1))
>>> -   echo ${result[0]}
>>> +   local result
>>> +   result="$(sha1sum "$1")"
>>> +   echo "${result%% *}"
>>
>> Why are you moving over to declaring the variable and assigning it on
>> different lines?
> 
> Because shellcheck complains about it, so it's a habit I've gotten in
> to :) Even in cases where it doesn't really make a difference.
> 
> https://github.com/koalaman/shellcheck/wiki/SC2155
> 
> However, BATS does run the test suite with `set -e`, so splitting it
> does mean that BATS will now detect errors from sha1sum.  We don't
> really expect that to happen, but if BATS will give us error checking
> on it for free, why not?

Then the commit message should say so...

-- 
Eli Schwartz
Bug Wrangler and Trusted User



signature.asc
Description: OpenPGP digital signature

Re: [arch-projects] [dbscripts] [PATCH 3/3] Update messages to make fuller use of printf formatters

[Eli Schwartz via arch-projects]

On 02/22/2018 06:54 PM, Luke Shumaker wrote:
>> I do see what you're doing, I'm just not sure why. Is the whole idea
>> with this extra variable floating around, to avoid tokenizing
>> "${pkgs[@]}" as separate messages? That's why "${pkgs[*]}" tokenizes the
>> members of an array as one word by gluing the members together using the
>> first IFS character (which is a space). You'll note I used this in
>> testing2x.
>>
>> As for using %q for filepaths that can theoretically contain spaces...
>> good point I guess.
> 
> It's all about %q.  (I did use ${ary[*]} somewhere else in the
> commit).  The separate variable applies %q escaping to each package
> filename separately.  Without it, if I did something like:
> 
> + || error 'repo-remove %q %q' "$dbfile" "${pkgs[*]}"
> 
> Then it would also escape the spaces that separate them.

But, you're using

error 'repo-remove %q %s' "$dbfile" "${pkgs_str% }"

with a %s which works just as well as it ever did. And you might as well
do that with "${pkgs[*]}" since that also works as well as it ever did.

Maybe, you should update that to work properly %q then...

Or maybe, you should skip the temporary variable and use

error 'repo-remove %s %s' "${dbfile@Q}" "${pkgs[*]@Q}"

This will result in the variables being passed into error, after being
suitably '/path quoted/rather/than/escaped/'

See the bash documentation on "Parameter transformation".

> Anyway, correctly applying %q escaping probably isn't super-important.
> But, we don't really expect repo-add or repo-remove to fail; if
> something is wrong, any of the numerous checks leading up to actually
> calling them should have already caught that.  If we trigger one of
> these error messages, something *weird* is going on, and I'd like the
> most precise error message possible.

No, I agree we might as well be careful here!

-- 
Eli Schwartz
Bug Wrangler and Trusted User



signature.asc
Description: OpenPGP digital signature

Re: [arch-projects] [dbscripts] [PATCH 2/3] test: Fixup glob matching

[Eli Schwartz via arch-projects]

On 02/22/2018 03:43 PM, Luke Shumaker wrote:
> From: Luke Shumaker 
> 
>  - ftpdir-cleanup.bats: Glob expansion does not occur in [[ -f ]] tests.
>The [[ ! -f .../${pkgname}-*${PKGEXT} ]] checks were checking that there
>were no files containing a literal '*' for that part of their name.
>Obviously, this isn't what was intended.

Good catch, thanks!

>  - sourceballs.bats: [ -r ] checks explode if the glob returns >1 file.
>Avoid using them if the path being checked contains a glob.

Arguably I'd like to upgrade the whole testsuite to use [[ ... ]]

Thanks.

>  - common.bash: Globbing happens on the RHS of a [[ = ]] test.
>This means that we must quote variables on the RHS that are to be taken
>verbatim.  This is surprising, because we don't need to quote the LHS.

No we don't, we only "need" to quote the ones that (can) contain globs.

In the general case, there is a school of thought that says you should
only quote what you explicitly need to quote. :p

Also not really surprising. This is a bash feature, you can do regex
comparisons too.

> ---
>  test/cases/ftpdir-cleanup.bats |  4 ++--
>  test/cases/sourceballs.bats|  4 ++--
>  test/lib/common.bash   | 12 +++-
>  3 files changed, 15 insertions(+), 5 deletions(-)
> 
> diff --git a/test/cases/ftpdir-cleanup.bats b/test/cases/ftpdir-cleanup.bats
> index fd485f3..8c713c6 100644
> --- a/test/cases/ftpdir-cleanup.bats
> +++ b/test/cases/ftpdir-cleanup.bats
> @@ -13,8 +13,8 @@ __checkRepoRemovedPackage() {
>   local pkgname
>  
>   for pkgname in $(__getPackageNamesFromPackageBase ${pkgbase}); do
> - [[ ! -f ${FTP_BASE}/${PKGPOOL}/${pkgname}-*${PKGEXT} ]]
> - [[ ! -f 
> ${FTP_BASE}/${repo}/os/${repoarch}/${pkgname}-*${PKGEXT} ]]
> + ! __isGlobfile "${FTP_BASE}/${PKGPOOL}/${pkgname}"-*"${PKGEXT}"
> + ! __isGlobfile 
> "${FTP_BASE}/${repo}/os/${repoarch}/${pkgname}"-*"${PKGEXT}"
>   done
>  }
>  
> diff --git a/test/cases/sourceballs.bats b/test/cases/sourceballs.bats
> index a0a2999..df7ddd4 100644
> --- a/test/cases/sourceballs.bats
> +++ b/test/cases/sourceballs.bats
> @@ -2,12 +2,12 @@ load ../lib/common
>  
>  __checkSourcePackage() {
>   local pkgbase=$1
> - [ -r ${FTP_BASE}/${SRCPOOL}/${pkgbase}-*${SRCEXT} ]
> + __isGlobfile "${FTP_BASE}/${SRCPOOL}/${pkgbase}"-*"${SRCEXT}"
>  }
>  
>  __checkRemovedSourcePackage() {
>   local pkgbase=$1
> - [ ! -r ${FTP_BASE}/${SRCPOOL}/${pkgbase}-*${SRCEXT} ]
> + ! __isGlobfile "${FTP_BASE}/${SRCPOOL}/${pkgbase}"-*"${SRCEXT}"
>  }
>  
>  @test "create simple package sourceballs" {
> diff --git a/test/lib/common.bash b/test/lib/common.bash
> index 5411641..6e2b3df 100644
> --- a/test/lib/common.bash
> +++ b/test/lib/common.bash
> @@ -14,6 +14,16 @@ __getCheckSum() {
>   echo "${result%% *}"
>  }
>  
> +# Proxy function to check if a file exists. Using [[ -f ... ]] directly is 
> not
> +# always wanted because we might want to expand bash globs first. This way we
> +# can pass unquoted globs to __isGlobfile() and have them expanded as 
> function
> +# arguments before being checked.
> +#
> +# This is a copy of db-functions is_globfile
> +__isGlobfile() {
> + [[ -f $1 ]]
> +}
> +
>  __buildPackage() {
>   local pkgdest=${1:-.}
>   local p
> @@ -203,7 +213,7 @@ checkPackageDB() {
>   for repoarch in ${repoarches[@]}; do
>   # Only 'any' packages can be found in repos of 
> both arches
>   if [[ $pkgarch != any ]]; then
> - if [[ $pkgarch != ${repoarch} ]]; then
> + if [[ $pkgarch != "$repoarch" ]]; then
>   continue
>   fi
>   fi
> 


-- 
Eli Schwartz
Bug Wrangler and Trusted User



signature.asc
Description: OpenPGP digital signature

Re: [arch-projects] [dbscripts] [PATCH 3/3] Update messages to make fuller use of printf formatters

[Eli Schwartz via arch-projects]

On 02/22/2018 03:43 PM, Luke Shumaker wrote:
> From: Luke Shumaker 
> 
> These are things that were (IMO) missed in 5afac1e.  I found them using:
> 
> git ls-files|xargs grep -E '(plain|msg|msg2|warning|error|die) "[^"]*\$'

Consider using git grep next time :p rather than piping through both
xargs and the inferior GNU grep.

> I went a little above-and-beyond for escaping strings for the error
> messages in db-functions' arch_repo_add and arch_repo_remove.  The
> code should explain itself, but I wanted to point it out, as it's more
> complex than the "slap %s in there, and move the ${...} to the right"
> that is used everywhere else.
>
> [...]
> index 8b71cae..6d02c50 100644
> --- a/db-functions
> +++ b/db-functions
> @@ -446,11 +446,13 @@ arch_repo_add() {
>   local repo=$1
>   local arch=$2
>   local pkgs=(${@:3})
> + local pkgs_str
>  
>   # package files might be relative to repo dir
>   pushd "${FTP_BASE}/${repo}/os/${arch}" >/dev/null
> + printf -v pkgs_str -- '%q ' "${pkgs[@]}"
>   /usr/bin/repo-add -q "${repo}${DBEXT}" ${pkgs[@]} \
> - || error "repo-add ${repo}${DBEXT} ${pkgs[@]}"
> + || error 'repo-add %q %s' "${repo}${DBEXT}" "${pkgs_str% }"
>   popd >/dev/null
>   set_repo_permission "${repo}" "${arch}"
>  
> @@ -462,13 +464,15 @@ arch_repo_remove() {
>   local arch=$2
>   local pkgs=(${@:3})
>   local dbfile="${FTP_BASE}/${repo}/os/${arch}/${repo}${DBEXT}"
> + local pkgs_str
>  
>   if [[ ! -f ${dbfile} ]]; then
>   error "No database found at '%s'" "$dbfile"
>   return 1
>   fi
> + printf -v pkgs_str -- '%q ' "${pkgs[@]}"
>   /usr/bin/repo-remove -q "${dbfile}" ${pkgs[@]} \
> - || error "repo-remove ${dbfile} ${pkgs[@]}"
> + || error 'repo-remove %q %s' "$dbfile" "${pkgs_str% }"
>   set_repo_permission "${repo}" "${arch}"

I do see what you're doing, I'm just not sure why. Is the whole idea
with this extra variable floating around, to avoid tokenizing
"${pkgs[@]}" as separate messages? That's why "${pkgs[*]}" tokenizes the
members of an array as one word by gluing the members together using the
first IFS character (which is a space). You'll note I used this in
testing2x.

As for using %q for filepaths that can theoretically contain spaces...
good point I guess.

-- 
Eli Schwartz
Bug Wrangler and Trusted User



signature.asc
Description: OpenPGP digital signature

Re: [arch-projects] [dbscripts] [PATCH 1/3] test: common.bash:__getCheckSum: Don't rely on IFS

[Eli Schwartz via arch-projects]

On 02/22/2018 03:43 PM, Luke Shumaker wrote:
> From: Luke Shumaker 
> 
> I managed to stumble across a bug in BATS where the run() function
> screwed with the global IFS.  The bug has been fixed in git, but isn't
> in a release yet.
> 
> https://github.com/sstephenson/bats/issues/89
> 
> Anyway, this bug breaks __getCheckSum().  Fortunately, we have avoided
> tripping it so far because luck has it that we never call
> __getCheckSum() after run() in the same test.
> 
> So, there's nothing actually broken here, but it makes me nervous.  So
> go ahead and modify __getCheckSum to not rely on IFS.
> ---
>  test/lib/common.bash | 5 +++--
>  1 file changed, 3 insertions(+), 2 deletions(-)
> 
> diff --git a/test/lib/common.bash b/test/lib/common.bash
> index 568a541..5411641 100644
> --- a/test/lib/common.bash
> +++ b/test/lib/common.bash
> @@ -9,8 +9,9 @@ __updatePKGBUILD() {
>  }
>  
>  __getCheckSum() {
> - local result=($(sha1sum $1))
> - echo ${result[0]}
> + local result
> + result="$(sha1sum "$1")"
> + echo "${result%% *}"

Why are you moving over to declaring the variable and assigning it on
different lines?

-- 
Eli Schwartz
Bug Wrangler and Trusted User



signature.asc
Description: OpenPGP digital signature

[arch-projects] [dbscripts] [GIT] Official repo DB scripts branch master updated. 20131102-69-ge53cad6

[Eli Schwartz via arch-projects]

This is an automated email from the git hooks/post-receive script. It was
generated because a ref change was pushed to the repository containing
the project "Official repo DB scripts".

The branch, master has been updated
   via  e53cad6e4a8284165c6d0b2c7c86f6c077be693b (commit)
   via  c82f4372440b42f879344f105e12fb358c4c0ccf (commit)
   via  3e01ba9a82234c1d78a14be095c18a626591ad71 (commit)
   via  8757c459089c0db12e57db66561f640f657e34ac (commit)
   via  07e6a91da1331ae61b667d878092c5b03aa49ced (commit)
   via  b45650c3e3870b975620dd89bb6aa77585c8ff02 (commit)
  from  134aff74d16331231ed94f34d8417d1325d4600f (commit)

Those revisions listed above that are new to this repository have
not appeared on any other notification email; so we list those
revisions in full, below.

- Log -
commit e53cad6e4a8284165c6d0b2c7c86f6c077be693b
Author: Eli Schwartz <eschwa...@archlinux.org>
Date:   Thu Feb 15 22:34:36 2018 -0500

Globally set $PKGEXT to a bash extended glob representing valid choices.

The current glob `*.pkg.tar.?z` is both less restrictive and more
restrictive than makepkg, as it accepts any valid unicode character.

To be more exact, it's almost completely orthogonal to the one in makepkg.

makepkg only accepts .tar.gz, .tar.bz2, .tar.xz, .tar.lzo, .tar.lrz, and
.tar.Z and most of those fail to match against a two-char compression type.

dbscripts accepts .pkg.tar.z which incidentally is what I think of
cherry-picking xz and gz as supported methods.

Since this can be anything makepkg.conf accepts, it needs to be able to
match all that, unless we decide to perform additional restrictions in
which case we should still explicitly list each allowed extension. Using
bash extended globbing allows us to do this relatively painlessly.

Document the fact that this has *always* been some sort of glob, and
update the two cases where this was (not!) being evaluated by bash
[[ ... ]], to use a not-elegant-at-all proxy function is_globfile() to
evaluate globs *before* testing if they exist.

commit c82f4372440b42f879344f105e12fb358c4c0ccf
Author: Eli Schwartz <eschwa...@archlinux.org>
Date:   Thu Feb 15 22:32:04 2018 -0500

ftpdir-cleanup,sourceballs: replace external find command with bash globbing

This fully removes the use of find from the codebase, leads to a
micro-optimization in a couple cases, and ensures that $PKGEXT is
consistently treated as a shell globbing character (which is important
because it is used as one).

Of the eight instances in these files:

- One was unnecessary as `cat` can natively consume all files passed to
  it and no directory traversal was in use.

- Two were unnecessary as they were hardcoded to read a single file

- Another four were only being used to strip leading directory paths,
  and can be replaced by globstar and ${filepath##*/}

- The final two were checking the modification time of the files, and
  can be replaced with touch(1) and [[ -nt ]]. Although this introduces
  an additional temporary file, this is not such a big deal.

commit 3e01ba9a82234c1d78a14be095c18a626591ad71
Author: Eli Schwartz <eschwa...@archlinux.org>
Date:   Thu Feb 15 22:26:24 2018 -0500

db-update: replace external find command with bash globbing

Don't bother emitting errors. bash doesn't show globbing errors if it
cannot read a directory to try globbing there. And the former code never
aborted on errors anyway, as without `set -o pipefail` the sort command
swallowed the return code.

commit 8757c459089c0db12e57db66561f640f657e34ac
Author: Luke Shumaker <luke...@parabola.nu>
Date:   Fri Feb 16 12:10:25 2018 -0500

test: db-update: verify that PKGEXT(S) is treated as a glob

Commit b61a714 introduced a regression that in a db-functions function
called by db-update, config:PKGEXT is treated like a fixed string, instead
of as a glob.  Because of inadequacy of the test-suite, this did not cause
a test failure.


https://lists.archlinux.org/pipermail/arch-projects/2018-February/004742.html

The broken function checks if the repo already contains a package that
matches the one being released.  This did not trigger a test-suite failure
because right after the broken check there is another check that checks for
the exact filename of the new package file.  In the test-suite, all
package files have the same extension, so the latter check always "saved"
us.

So, modify a relevant test case (a test case that verifies that we can't
release the same package twice) such that the second time we release the
package, we set PKGEXT to something different (.gz, instead of the normal
.xz).  This way, we can be su

Re: [arch-projects] [dbscripts] [PATCH v2 1/5] Use even more bashisms.

[Eli Schwartz via arch-projects]

On 02/20/2018 12:24 PM, Emil Velikov wrote:
> Seems like I wasn't clear enough:
> The goal is not to appease zsh - but a step closer to POSIX sh friendly.
> 
> I've been staring and writing bash (closer to POSIX sh really) scripts
> for over a decade, haven't seen what makes X cleaner over Y.
> Yet that's subjective, unlike the original argument - consistency rules ;-)

If you're working for "POSIX sh friendly", why are you mentioning zsh in
the first place?

As for targeting POSIX sh, if you can do that then sure. I have personal
scripts written for sh when it makes sense (and my /bin/sh is symlinked
to dash, so I actually use sh).

But yeah, consistency rules.

-- 
Eli Schwartz
Bug Wrangler and Trusted User



signature.asc
Description: OpenPGP digital signature

Re: [arch-projects] [dbscripts] [PATCH v2 1/5] Use even more bashisms.

[Emil Velikov via arch-projects]

On 20 February 2018 at 14:23, Eli Schwartz <eschwa...@archlinux.org> wrote:
> On 02/20/2018 06:59 AM, Emil Velikov wrote:
>> Disclaimer: the following is a bit subtle topic, so I hope it doesn't
>> spur a lot of off-topic.
>
> Eh, I don't mind.
>
>> Is there any performance or other technical benefit to using more bashisms?
>>
>> Reason being, that I am slowly going through different parts of Arch
>> making it zsh friendly.
>> While keeping the code brief and legible, of course.
>> Guessing that I've picked the wrong hobby?
>
> I think you'll probably find that few people write zsh scripts for
> non-interactive use. I'm not really sure what the point would be,
> considering it has a nonstandard syntax (bash is ubiquitous, zsh is
> not), and many people who would know bash would not know zsh (like me
> for example).
>
> AFAIK zsh should more or less run either bash or POSIX sh scripts just
> fine if you invoke it via a symlink named `sh` or `bash`, because zsh
> has a bash compatibility mode. I have no idea whether that bash
> compatibility mode fixes subtle things like the fact that zsh arrays are
> 1-indexed while bash arrays are 0-indexed, but if I had to guess,
> probably not.
>
> ...
>
> I can see some compelling reasons to write scripts targeting POSIX sh as
> a baseline, which is being *sh* friendly, not zsh friendly.
> But, for projects that make heavy use of bashisms anyways, I dislike
> using POSIX because it implies that sh will be supported in any way when
> it really won't be. Essentially, I prefer to go "all in".
>
> As for why you'd want them, bashisms generally look cleaner IMHO, and
> they add a great deal of power and flexibility to the shell. Things like
> [[ ... ]] are just a lot more sane in basically every way, shell
> arithmetic uses proper operators, etc.
>
Seems like I wasn't clear enough:
The goal is not to appease zsh - but a step closer to POSIX sh friendly.

I've been staring and writing bash (closer to POSIX sh really) scripts
for over a decade, haven't seen what makes X cleaner over Y.
Yet that's subjective, unlike the original argument - consistency rules ;-)

Thanks
Emil

Re: [arch-projects] [dbscripts] [PATCH v2 1/5] Use even more bashisms.

[Eli Schwartz via arch-projects]

On 02/20/2018 06:59 AM, Emil Velikov wrote:
> Disclaimer: the following is a bit subtle topic, so I hope it doesn't
> spur a lot of off-topic.

Eh, I don't mind.

> Is there any performance or other technical benefit to using more bashisms?
> 
> Reason being, that I am slowly going through different parts of Arch
> making it zsh friendly.
> While keeping the code brief and legible, of course.
> Guessing that I've picked the wrong hobby?

I think you'll probably find that few people write zsh scripts for
non-interactive use. I'm not really sure what the point would be,
considering it has a nonstandard syntax (bash is ubiquitous, zsh is
not), and many people who would know bash would not know zsh (like me
for example).

AFAIK zsh should more or less run either bash or POSIX sh scripts just
fine if you invoke it via a symlink named `sh` or `bash`, because zsh
has a bash compatibility mode. I have no idea whether that bash
compatibility mode fixes subtle things like the fact that zsh arrays are
1-indexed while bash arrays are 0-indexed, but if I had to guess,
probably not.

...

I can see some compelling reasons to write scripts targeting POSIX sh as
a baseline, which is being *sh* friendly, not zsh friendly.
But, for projects that make heavy use of bashisms anyways, I dislike
using POSIX because it implies that sh will be supported in any way when
it really won't be. Essentially, I prefer to go "all in".

As for why you'd want them, bashisms generally look cleaner IMHO, and
they add a great deal of power and flexibility to the shell. Things like
[[ ... ]] are just a lot more sane in basically every way, shell
arithmetic uses proper operators, etc.

-- 
Eli Schwartz
Bug Wrangler and Trusted User



signature.asc
Description: OpenPGP digital signature

Re: [arch-projects] [dbscripts] [PATCH v2 1/5] Use even more bashisms.

[Emil Velikov via arch-projects]

Hi Eli,

Disclaimer: the following is a bit subtle topic, so I hope it doesn't
spur a lot of off-topic.

On 19 February 2018 at 20:11, Eli Schwartz via arch-projects
<arch-projects@archlinux.org> wrote:
> Catch some cases that were missed in the previous run.
>
> Signed-off-by: Eli Schwartz <eschwa...@archlinux.org>
> ---
>
> This patch is new + refactor some changes from:
> ftpdir-cleanup,sourceballs: replace external find command with bash globbing
>
>  cron-jobs/devlist-mailer  |  6 +++---
>  cron-jobs/ftpdir-cleanup  | 14 +++---
>  cron-jobs/integrity-check |  2 +-
>  cron-jobs/sourceballs | 12 ++--
>  cron-jobs/update-web-db   |  6 +++---
>  5 files changed, 20 insertions(+), 20 deletions(-)
>
Is there any performance or other technical benefit to using more bashisms?

Reason being, that I am slowly going through different parts of Arch
making it zsh friendly.
While keeping the code brief and legible, of course.
Guessing that I've picked the wrong hobby?

Thanks
Emil

Re: [arch-projects] [dbscripts] [PATCH v3] test: db-update: @test "update same any package to same repository fails": change PKGEXT

[Eli Schwartz via arch-projects]

On 02/19/2018 10:24 PM, Luke Shumaker wrote:
> The glob I was using it for wasn't PKGEXT(s), it was the '*' that's
> right there in the argument!

Right, that's why I replied to myself with "Actually, it would tend to
help if we had the actual candidate filenames here. Hmm..."

Context: I wrote a makepkg patch whose sole purpose is to ensure that
people can extract a newline-separated list of absolute filepaths for
each package that a PKGBUILD would/did create. No globs needed. :)

I'm hoping this will simplify some rather hackish and fragile logic in
*many* projects that interface with makepkg.

>> Maybe:
> ...
>> if cp -Lv ${cache}/*${PKGEXT}{,.sig} ${pkgdest} 2>/dev/null; then
>> return 0
>> else
>>
>> This would avoid the need for __isGlobfile function altogether.
> 
> I like that!  Good idea!
> 
>> I'd also like a more descriptive commit message. Don't tell me what you
>> changed, tell me why you changed it. :p
> 
> Ok, I'll go in to that more.

The changes look great, thanks.

-- 
Eli Schwartz
Bug Wrangler and Trusted User



signature.asc
Description: OpenPGP digital signature

Re: [arch-projects] [dbscripts] [PATCH v3] test: db-update: @test "update same any package to same repository fails": change PKGEXT

[Eli Schwartz via arch-projects]

On 02/19/2018 09:12 PM, Eli Schwartz wrote:
> On 02/19/2018 08:47 PM, Eli Schwartz wrote:
>> On 02/19/2018 06:31 PM, Luke Shumaker wrote:
>>> From: Luke Shumaker <luke...@parabola.nu>
>>>
>>> This has the test change PKGEXT the second time it tries to release the
>>> package.  Currently, this causes the tests to fail.  That's a good thing;
>>> it's checking for the regression where db-functions:check_pkgrepos isn't
>>> treating PKGEXT as a glob.
>>>
>>> Without this, that regression didn't cause test failure because the checks
>>> right after it were tripping anyway.
>>>
>>> https://lists.archlinux.org/pipermail/arch-projects/2018-February/004742.html
>>>
>>> v2: Follow Eli's suggestion to simplify it using the check in __buildPackage
>>> v3: Simplify further by assuming __buildPackage checks PKGEXT, not PKGEXTS
>>> ---
>>>  This is written againt Eli's v2 patchset (my concerns there don't
>>>  affect this).  You can verify--applying this patch first makes the
>>>  tests fail, then applying Eli's patches make the tests pass again.
>>>
>>>  Dave's objections to the __isGlobfile name and comment apply to this
>>>  patch as well.
>> As far as the testsuite is concerned, you can just use "Fix overloading
>> PKGEXT to mean two things." as a base. This means that all you need to
>> do is check that if you releasePackage the same package twice using a
>> new $PKGEXT it is still rejected.
>>
>> ...
>>
>> We're not testing whether or not globs work, we're testing whether or
>> not check_pkgrepos properly detects pre-existing packages (which it does
>> via globs). Using __isGlobfile() here will no longer be useful
>> information once $PKGEXT is only ever something from makepkg. So it
>> doesn't make sense to add code that will be almost immediately removed.
> 
> Actually, it would tend to help if we had the actual candidate filenames
> here. Hmm...

Maybe:
>   if [[ -n ${BUILDDIR} ]]; then
>   cache=${BUILDDIR}/$(__getCheckSum PKGBUILD)
> - if [[ -d ${cache} ]]; then
> + if __isGlobfile "${cache}"/*"${PKGEXT}"; then
>   cp -Lv ${cache}/*${PKGEXT}{,.sig} ${pkgdest}
>   return 0
>   else

could use

if cp -Lv ${cache}/*${PKGEXT}{,.sig} ${pkgdest} 2>/dev/null; then
return 0
else

This would avoid the need for __isGlobfile function altogether.

I'd also like a more descriptive commit message. Don't tell me what you
changed, tell me why you changed it. :p

-- 
Eli Schwartz
Bug Wrangler and Trusted User



signature.asc
Description: OpenPGP digital signature

Re: [arch-projects] [dbscripts] [PATCH v3] test: db-update: @test "update same any package to same repository fails": change PKGEXT

[Eli Schwartz via arch-projects]

On 02/19/2018 06:31 PM, Luke Shumaker wrote:
> From: Luke Shumaker <luke...@parabola.nu>
> 
> This has the test change PKGEXT the second time it tries to release the
> package.  Currently, this causes the tests to fail.  That's a good thing;
> it's checking for the regression where db-functions:check_pkgrepos isn't
> treating PKGEXT as a glob.
> 
> Without this, that regression didn't cause test failure because the checks
> right after it were tripping anyway.
> 
> https://lists.archlinux.org/pipermail/arch-projects/2018-February/004742.html
> 
> v2: Follow Eli's suggestion to simplify it using the check in __buildPackage
> v3: Simplify further by assuming __buildPackage checks PKGEXT, not PKGEXTS
> ---
>  This is written againt Eli's v2 patchset (my concerns there don't
>  affect this).  You can verify--applying this patch first makes the
>  tests fail, then applying Eli's patches make the tests pass again.
> 
>  Dave's objections to the __isGlobfile name and comment apply to this
>  patch as well.
As far as the testsuite is concerned, you can just use "Fix overloading
PKGEXT to mean two things." as a base. This means that all you need to
do is check that if you releasePackage the same package twice using a
new $PKGEXT it is still rejected.

...

We're not testing whether or not globs work, we're testing whether or
not check_pkgrepos properly detects pre-existing packages (which it does
via globs). Using __isGlobfile() here will no longer be useful
information once $PKGEXT is only ever something from makepkg. So it
doesn't make sense to add code that will be almost immediately removed.

-- 
Eli Schwartz
Bug Wrangler and Trusted User



signature.asc
Description: OpenPGP digital signature

Re: [arch-projects] [dbscripts] [PATCH v2 5/5] Globally set $PKGEXT to a bash extended glob representing valid choices.

[Eli Schwartz via arch-projects]

On 02/19/2018 04:59 PM, Luke Shumaker wrote:
> Is there a reason you reject '.pkg.tar' (no compression, which makepkg
> accepts)?

I don't think there is any utility in supporting uncompressed packages
in dbscripts. Anyone who wants to customize this in a non-Arch Linux
deployment is free to do so...

If someone wants to use some deviant compression type because they're
positive it works better on those packaged files, I cannot think of a
compelling reason to say "no you're wrong", which is why I listed
everything else.

> (I also found it curious that you swapped lzo and lrz from the order
> the extensions are in in the makepkg source.)

makepkg is inconsistent here, I pulled that from the makepkg.conf(5)
source. :D

-- 
Eli Schwartz
Bug Wrangler and Trusted User



signature.asc
Description: OpenPGP digital signature

Re: [arch-projects] [dbscripts] [PATCH v2 3/5] db-update: replace external find command with bash globbing

[Eli Schwartz via arch-projects]

On 02/19/2018 04:53 PM, Luke Shumaker wrote:
> Isn't [[ -d ]] there redundant?  If globbing gave us $dir/file, of
> course $dir is a directory!

True. I think I still had that in from some point where I hadn't enabled
nullglob yet.

> Meanwhile, this dropped the `-type f` check, though I'm not sure how
> important that was.
> 
> Shouldn't this be written as:
> 
>   mapfile -t -d '' staging_repos < <(
>   for f in "${STAGING}"/**/*${PKGEXTS}; do
>   if [[ -f $f && ! -h $f ]]; then
>   printf '%s\0' "${f/*}"
>   fi
>   done | sort -uz
>   )
> 
> The original `find` command rejected symlinks; I don't know if that's
> an important property; but that's what the `&& ! -h $f` bit is for.

It is not important, the find command only checked if the file itself
was a symlink but if there is another package file in the same directory
then we still add those staging repos.

Meanwhile, we check later on for `die "Package %s is a symbolic link"`.
So I guess technically it would make more sense to stage the package and
then utilize the explicit error message rather than silently dropping
the package altogether (but only sometimes) simply because we didn't
think to use -xtype.

At this stage in the game, we're just trying to assemble a list of the
packages that the uploader is asserting they want to db-update. We
perform all actual validation later on.

-- 
Eli Schwartz
Bug Wrangler and Trusted User



signature.asc
Description: OpenPGP digital signature

[arch-projects] [dbscripts] [PATCH v2 5/5] Globally set $PKGEXT to a bash extended glob representing valid choices.

[Eli Schwartz via arch-projects]

The current glob `*.pkg.tar.?z` is both less restrictive and more
restrictive than makepkg, as it accepts any valid unicode character.

To be more exact, it's almost completely orthogonal to the one in makepkg.

makepkg only accepts .tar.gz, .tar.bz2, .tar.xz, .tar.lzo, .tar.lrz, and
.tar.Z and most of those fail to match against a two-char compression type.

dbscripts accepts .pkg.tar.z which incidentally is what I think of
cherry-picking xz and gz as supported methods.

Since this can be anything makepkg.conf accepts, it needs to be able to
match all that, unless we decide to perform additional restrictions in
which case we should still explicitly list each allowed extension. Using
bash extended globbing allows us to do this relatively painlessly.

Document the fact that this has *always* been some sort of glob, and
update the two cases where this was (not!) being evaluated by bash
[[ ... ]], to use a not-elegant-at-all proxy function is_globfile() to
evaluate globs *before* testing if they exist.

Signed-off-by: Eli Schwartz 
---

No changes, just a reworded commit message.

 config   |  3 ++-
 db-functions | 11 ---
 2 files changed, 10 insertions(+), 4 deletions(-)

diff --git a/config b/config
index 5bb3b16..0d33de0 100644
--- a/config
+++ b/config
@@ -25,7 +25,8 @@ TMPDIR="/var/tmp"
 ARCHES=(x86_64)
 DBEXT=".db.tar.gz"
 FILESEXT=".files.tar.gz"
-PKGEXTS=".pkg.tar.?z"
+# bash glob listing allowed extensions. Note that db-functions turns on 
extglob.
+PKGEXTS=".pkg.tar.@(gz|bz2|xz|lzo|lrz|Z)"
 SRCEXT=".src.tar.gz"
 
 # Allowed licenses: get sourceballs only for licenses in this array
diff --git a/db-functions b/db-functions
index 394c7a2..e36d43b 100644
--- a/db-functions
+++ b/db-functions
@@ -3,7 +3,7 @@
 . /usr/share/makepkg/util.sh
 
 # global shell options for enhanced bash scripting
-shopt -s globstar nullglob
+shopt -s extglob globstar nullglob
 
 
 # Some PKGBUILDs need CARCH to be set
@@ -20,6 +20,11 @@ restore_umask () {
umask $UMASK >/dev/null
 }
 
+# Check if a file exists, even if the file uses wildcards
+is_globfile() {
+   [[ -f $1 ]]
+}
+
 # just like mv -f, but we touch the file and then copy the content so
 # default ACLs in the target dir will be applied
 mv_acl() {
@@ -378,8 +383,8 @@ check_pkgrepos() {
local pkgver="$(getpkgver ${pkgfile})" || return 1
local pkgarch="$(getpkgarch ${pkgfile})" || return 1
 
-   [[ -f ${FTP_BASE}/${PKGPOOL}/${pkgname}-${pkgver}-${pkgarch}${PKGEXTS} 
]] && return 1
-   [[ -f 
${FTP_BASE}/${PKGPOOL}/${pkgname}-${pkgver}-${pkgarch}${PKGEXTS}.sig ]] && 
return 1
+   is_globfile 
"${FTP_BASE}/${PKGPOOL}/${pkgname}-${pkgver}-${pkgarch}"${PKGEXTS} && return 1
+   is_globfile 
"${FTP_BASE}/${PKGPOOL}/${pkgname}-${pkgver}-${pkgarch}"${PKGEXTS}.sig && 
return 1
[[ -f ${FTP_BASE}/${PKGPOOL}/${pkgfile##*/} ]] && return 1
[[ -f ${FTP_BASE}/${PKGPOOL}/${pkgfile##*/}.sig ]] && return 1
 
-- 
2.16.2

[arch-projects] [dbscripts] [PATCH v2 1/5] Use even more bashisms.

[Eli Schwartz via arch-projects]

Catch some cases that were missed in the previous run.

Signed-off-by: Eli Schwartz 
---

This patch is new + refactor some changes from:
ftpdir-cleanup,sourceballs: replace external find command with bash globbing

 cron-jobs/devlist-mailer  |  6 +++---
 cron-jobs/ftpdir-cleanup  | 14 +++---
 cron-jobs/integrity-check |  2 +-
 cron-jobs/sourceballs | 12 ++--
 cron-jobs/update-web-db   |  6 +++---
 5 files changed, 20 insertions(+), 20 deletions(-)

diff --git a/cron-jobs/devlist-mailer b/cron-jobs/devlist-mailer
index ca2e46b..65a5483 100755
--- a/cron-jobs/devlist-mailer
+++ b/cron-jobs/devlist-mailer
@@ -7,17 +7,17 @@ LIST="arch-dev-pub...@archlinux.org"
 FROM="repoma...@archlinux.org"
 
 SUBJECT="Repository Maintenance $(date +"%d-%m-%Y")"
-if [ $# -ge 1 ]; then
+if (( $# >= 1 )); then
 SUBJECT="$1 $(date +"%d-%m-%Y")"
 fi
 
-if [ $# -ge 2 ]; then
+if (( $# >= 2 )); then
 LIST="$2"
 fi
 
 stdin="$(cat)"
 #echo used to strip whitespace for checking for actual data
-if [ -n "$(echo $stdin)" ]; then
+if [[ -n "$(echo $stdin)" ]]; then
 
 echo "Subject: $SUBJECT
 To: $LIST
diff --git a/cron-jobs/ftpdir-cleanup b/cron-jobs/ftpdir-cleanup
index 2f3d5aa..c771950 100755
--- a/cron-jobs/ftpdir-cleanup
+++ b/cron-jobs/ftpdir-cleanup
@@ -9,11 +9,11 @@ clean_pkg() {
 
if ! ${CLEANUP_DRYRUN}; then
for pkg in "$@"; do
-   if [ -h "$pkg" ]; then
+   if [[ -h $pkg ]]; then
rm -f "$pkg" "$pkg.sig"
else
mv_acl "$pkg" "$CLEANUP_DESTDIR/${pkg##*/}"
-   if [ -e "$pkg.sig" ]; then
+   if [[ -e $pkg.sig ]]; then
mv_acl "$pkg.sig" 
"$CLEANUP_DESTDIR/${pkg##*/}.sig"
fi
touch "${CLEANUP_DESTDIR}/${pkg##*/}"
@@ -34,7 +34,7 @@ ${CLEANUP_DRYRUN} && warning 'dry run mode is active'
 
 for repo in ${PKGREPOS[@]}; do
for arch in ${ARCHES[@]}; do
-   if [ ! -f "${FTP_BASE}/${repo}/os/${arch}/${repo}${DBEXT}" ]; 
then
+   if [[ ! -f ${FTP_BASE}/${repo}/os/${arch}/${repo}${DBEXT} ]]; 
then
continue
fi
# get a list of actual available package files
@@ -43,7 +43,7 @@ for repo in ${PKGREPOS[@]}; do
bsdtar -xOf "${FTP_BASE}/${repo}/os/${arch}/${repo}${DBEXT}" | 
awk '/^%FILENAME%/{getline;print}' | sort > "${WORKDIR}/db-${repo}-${arch}"
 
missing_pkgs=($(comm -13 "${WORKDIR}/repo-${repo}-${arch}" 
"${WORKDIR}/db-${repo}-${arch}"))
-   if [ ${#missing_pkgs[@]} -ge 1 ]; then
+   if (( ${#missing_pkgs[@]} >= 1 )); then
error "Missing packages in [%s] (%s)..." "$repo" "$arch"
for missing_pkg in ${missing_pkgs[@]}; do
msg2 "${missing_pkg}"
@@ -51,7 +51,7 @@ for repo in ${PKGREPOS[@]}; do
fi
 
old_pkgs=($(comm -23 "${WORKDIR}/repo-${repo}-${arch}" 
"${WORKDIR}/db-${repo}-${arch}"))
-   if [ ${#old_pkgs[@]} -ge 1 ]; then
+   if (( ${#old_pkgs[@]} >= 1 )); then
msg "Removing old packages from [%s] (%s)..." "$repo" 
"$arch"
for old_pkg in ${old_pkgs[@]}; do
msg2 "${old_pkg}"
@@ -67,7 +67,7 @@ find "$FTP_BASE/${PKGPOOL}" -name "*${PKGEXT}" -printf '%f\n' 
| sort > "${WORKDI
 find "${WORKDIR}" -maxdepth 1 -type f -name 'db-*' -exec cat {} \; | sort -u > 
"${WORKDIR}/db"
 
 old_pkgs=($(comm -23 "${WORKDIR}/pool" "${WORKDIR}/db"))
-if [ ${#old_pkgs[@]} -ge 1 ]; then
+if (( ${#old_pkgs[@]} >= 1 )); then
msg "Removing old packages from package pool..."
for old_pkg in ${old_pkgs[@]}; do
msg2 "${old_pkg}"
@@ -76,7 +76,7 @@ if [ ${#old_pkgs[@]} -ge 1 ]; then
 fi
 
 old_pkgs=($(find ${CLEANUP_DESTDIR} -type f -name "*${PKGEXT}" -mtime 
+${CLEANUP_KEEP} -printf '%f\n'))
-if [ ${#old_pkgs[@]} -ge 1 ]; then
+if (( ${#old_pkgs[@]} >= 1 )); then
msg "Removing old packages from the cleanup directory..."
for old_pkg in ${old_pkgs[@]}; do
msg2 "${old_pkg}"
diff --git a/cron-jobs/integrity-check b/cron-jobs/integrity-check
index 211a24b..f1e75ab 100755
--- a/cron-jobs/integrity-check
+++ b/cron-jobs/integrity-check
@@ -7,7 +7,7 @@ dirname="$(dirname $0)"
 
 script_lock
 
-if [ $# -ne 1 ]; then
+if (( $# != 1 )); then
die "usage: ${0##*/} "
 fi
 mailto=$1
diff --git a/cron-jobs/sourceballs b/cron-jobs/sourceballs
index 9ab4e98..25a8abb 100755
--- a/cron-jobs/sourceballs
+++ b/cron-jobs/sourceballs
@@ -21,7 +21,7 @@ renice +10 -p $$ > /dev/null
 for repo in ${PKGREPOS[@]}; do
for arch in ${ARCHES[@]}; do
# Repo does not exist; skip it
-   if [ ! -f 

[arch-projects] [dbscripts] [PATCH v2 3/5] db-update: replace external find command with bash globbing

[Eli Schwartz via arch-projects]

Don't bother emitting errors. bash doesn't show globbing errors if it
cannot read a directory to try globbing there. And the former code never
aborted on errors anyway, as without `set -o pipefail` the sort command
swallowed the return code.

Signed-off-by: Eli Schwartz 
---

v2: use mapfile as suggested by Luke, rather than running in_array in
a loop.

 db-functions |  4 
 db-update| 11 ---
 2 files changed, 12 insertions(+), 3 deletions(-)

diff --git a/db-functions b/db-functions
index e8eb2bc..394c7a2 100644
--- a/db-functions
+++ b/db-functions
@@ -2,6 +2,10 @@
 
 . /usr/share/makepkg/util.sh
 
+# global shell options for enhanced bash scripting
+shopt -s globstar nullglob
+
+
 # Some PKGBUILDs need CARCH to be set
 CARCH="x86_64"
 
diff --git a/db-update b/db-update
index a8d885a..db12df8 100755
--- a/db-update
+++ b/db-update
@@ -9,9 +9,14 @@ if (( $# >= 1 )); then
 fi
 
 # Find repos with packages to release
-if ! staging_repos=($(find  "${STAGING}" -mindepth 1 -type f -name 
"*${PKGEXTS}" -printf '%h\n' | sort -u)); then
-   die "Could not read %s" "$STAGING"
-fi
+mapfile -t -d '' staging_repos < <(
+   for f in "${STAGING}"/**/*${PKGEXTS}; do
+   f="${f%/*}"
+   if [[ -d $f ]]; then
+   printf '%s\0' "$f"
+   fi
+   done | sort -uz
+)
 
 repos=()
 for staging_repo in ${staging_repos[@]##*/}; do
-- 
2.16.2

[arch-projects] [dbscripts] [PATCH v2 2/5] Fix overloading PKGEXT to mean two things.

[Eli Schwartz via arch-projects]

PKGEXT is a makepkg variable referring to a fixed filename suffix, but
we were also using it to mean a bash glob referring to candidate
filenames. This is wrong, so rename it to PKGEXTS which is more
descriptive of its purpose.

Exclude the testsuite from this change, as the testsuite actually uses
PKGEXT for its intended purpose. Fix the testsuite to consistently use
PKGEXT, as it hardcoded the file extension in several cases, and extract
its value from the makepkg.conf we ship.

Signed-off-by: Eli Schwartz 
---

This is new, and renders some old things obsolete. I see no need to jump
through more hoops than strictly necessary, plus, lots of this was
technically broken anyway -- but it worked because again, the testsuite
is not using PKGEXT in the dbscripts sense, rather in the makepkg sense.

I'm hoping some of the logic there can be replaced with my --packagelist
fixes for makepkg 5.1 as well.

 config | 2 +-
 cron-jobs/ftpdir-cleanup   | 6 +++---
 db-functions   | 4 ++--
 db-move| 4 ++--
 db-update  | 8 
 test/cases/db-repo-add.bats| 6 +++---
 test/cases/db-update.bats  | 3 +--
 test/cases/ftpdir-cleanup.bats | 6 +++---
 test/lib/common.bash   | 6 --
 9 files changed, 23 insertions(+), 22 deletions(-)

diff --git a/config b/config
index d2c1942..5bb3b16 100644
--- a/config
+++ b/config
@@ -25,7 +25,7 @@ TMPDIR="/var/tmp"
 ARCHES=(x86_64)
 DBEXT=".db.tar.gz"
 FILESEXT=".files.tar.gz"
-PKGEXT=".pkg.tar.?z"
+PKGEXTS=".pkg.tar.?z"
 SRCEXT=".src.tar.gz"
 
 # Allowed licenses: get sourceballs only for licenses in this array
diff --git a/cron-jobs/ftpdir-cleanup b/cron-jobs/ftpdir-cleanup
index c771950..630efa8 100755
--- a/cron-jobs/ftpdir-cleanup
+++ b/cron-jobs/ftpdir-cleanup
@@ -38,7 +38,7 @@ for repo in ${PKGREPOS[@]}; do
continue
fi
# get a list of actual available package files
-   find "${FTP_BASE}/${repo}/os/${arch}" -xtype f -name 
"*${PKGEXT}" -printf '%f\n' | sort > "${WORKDIR}/repo-${repo}-${arch}"
+   find "${FTP_BASE}/${repo}/os/${arch}" -xtype f -name 
"*${PKGEXTS}" -printf '%f\n' | sort > "${WORKDIR}/repo-${repo}-${arch}"
# get a list of package files defined in the repo db
bsdtar -xOf "${FTP_BASE}/${repo}/os/${arch}/${repo}${DBEXT}" | 
awk '/^%FILENAME%/{getline;print}' | sort > "${WORKDIR}/db-${repo}-${arch}"
 
@@ -62,7 +62,7 @@ for repo in ${PKGREPOS[@]}; do
 done
 
 # get a list of all available packages in the pacakge pool
-find "$FTP_BASE/${PKGPOOL}" -name "*${PKGEXT}" -printf '%f\n' | sort > 
"${WORKDIR}/pool"
+find "$FTP_BASE/${PKGPOOL}" -name "*${PKGEXTS}" -printf '%f\n' | sort > 
"${WORKDIR}/pool"
 # create a list of packages in our db
 find "${WORKDIR}" -maxdepth 1 -type f -name 'db-*' -exec cat {} \; | sort -u > 
"${WORKDIR}/db"
 
@@ -75,7 +75,7 @@ if (( ${#old_pkgs[@]} >= 1 )); then
done
 fi
 
-old_pkgs=($(find ${CLEANUP_DESTDIR} -type f -name "*${PKGEXT}" -mtime 
+${CLEANUP_KEEP} -printf '%f\n'))
+old_pkgs=($(find ${CLEANUP_DESTDIR} -type f -name "*${PKGEXTS}" -mtime 
+${CLEANUP_KEEP} -printf '%f\n'))
 if (( ${#old_pkgs[@]} >= 1 )); then
msg "Removing old packages from the cleanup directory..."
for old_pkg in ${old_pkgs[@]}; do
diff --git a/db-functions b/db-functions
index e8949d7..e8eb2bc 100644
--- a/db-functions
+++ b/db-functions
@@ -374,8 +374,8 @@ check_pkgrepos() {
local pkgver="$(getpkgver ${pkgfile})" || return 1
local pkgarch="$(getpkgarch ${pkgfile})" || return 1
 
-   [[ -f ${FTP_BASE}/${PKGPOOL}/${pkgname}-${pkgver}-${pkgarch}${PKGEXT} 
]] && return 1
-   [[ -f 
${FTP_BASE}/${PKGPOOL}/${pkgname}-${pkgver}-${pkgarch}${PKGEXT}.sig ]] && 
return 1
+   [[ -f ${FTP_BASE}/${PKGPOOL}/${pkgname}-${pkgver}-${pkgarch}${PKGEXTS} 
]] && return 1
+   [[ -f 
${FTP_BASE}/${PKGPOOL}/${pkgname}-${pkgver}-${pkgarch}${PKGEXTS}.sig ]] && 
return 1
[[ -f ${FTP_BASE}/${PKGPOOL}/${pkgfile##*/} ]] && return 1
[[ -f ${FTP_BASE}/${PKGPOOL}/${pkgfile##*/}.sig ]] && return 1
 
diff --git a/db-move b/db-move
index 37a9884..fb7ebac 100755
--- a/db-move
+++ b/db-move
@@ -49,7 +49,7 @@ for pkgbase in ${args[@]:2}; do
 
for pkgname in ${pkgnames[@]}; do
for tarch in ${tarches[@]}; do
-   getpkgfile 
"${ftppath_from}/${tarch}/"${pkgname}-${pkgver}-${pkgarch}${PKGEXT} >/dev/null
+   getpkgfile 
"${ftppath_from}/${tarch}/"${pkgname}-${pkgver}-${pkgarch}${PKGEXTS} >/dev/null
done
done
continue 2
@@ -95,7 +95,7 @@ for pkgbase in ${args[@]:2}; do
 
for pkgname in ${pkgnames[@]}; do
for tarch in ${tarches[@]}; do
-

[arch-projects] [dbscripts] [PATCH v2 0/5] Fix ambiguous uses of $PKGEXT

[Eli Schwartz via arch-projects]

Comes with fancy checkmarks from travis saying that the testsuite
passed: https://github.com/archlinux/dbscripts/commits/pkgext-real-wildcards

Eli Schwartz (5):
  Use even more bashisms.
  Fix overloading PKGEXT to mean two things.
  db-update: replace external find command with bash globbing
  ftpdir-cleanup,sourceballs: replace external find command with bash
globbing
  Globally set $PKGEXT to a bash extended glob representing valid
choices.

 config |  3 ++-
 cron-jobs/devlist-mailer   |  6 +++---
 cron-jobs/ftpdir-cleanup   | 36 
 cron-jobs/integrity-check  |  2 +-
 cron-jobs/sourceballs  | 31 +--
 cron-jobs/update-web-db|  6 +++---
 db-functions   | 13 +++--
 db-move|  4 ++--
 db-update  | 17 +++--
 test/cases/db-repo-add.bats|  6 +++---
 test/cases/db-update.bats  |  3 +--
 test/cases/ftpdir-cleanup.bats |  6 +++---
 test/lib/common.bash   |  6 --
 13 files changed, 89 insertions(+), 50 deletions(-)

-- 
2.16.2

Re: [arch-projects] [dbscripts] [PATCH v2 0/8] PKGEXT fixup

[Eli Schwartz via arch-projects]

On 02/18/2018 12:17 PM, Luke Shumaker wrote:
> From: Luke Shumaker 
> 
> This incorporates and improves on work from 3 previously submitted
> patch sets:
> 
>  1. My testcase patch, but
> - take Eli's suggestion to simplify db-update.bats
> - add another commit so we don't hit a bug in BATS
> 
>  2. Renaming PKGEXT->PKGEXT_glob from my first fix patchset
> 
>  3. Eli's fix/extglob patchset, but
> - re-order things between commits
> - also accept ".tar" (with no compression suffix)
> - use `old_pkgs=()` instead of `unset old_pkgs`
> - replace `[ -ge 1]` with `(( > 0 ))` instead of `(( > 1 ))`
> - db-update: use `readarray < <(... | sort -u)` instead of the
>   O(n^2) op of using in_array in a for loop and appending to the
>   array
> - common.bash, sourceballs.bats: update to work with shopt -s
>   extglob nullglob globstar; the tests needed updated too, not
>   just the main code
> 
> The last 3 commits aren't really related to the goal of the patchset,
> but I wanted to include all of the work from Eli's fix patchset.
> 
> I don't mean to take credit away from Eli by re-working his patches (I
> credit him in the commit messages); I just wanted to make it clearer
> what is accomplished by each change, and how each of the changes
> relate to our goals; as well as actually testing each of them against
> the test suite.
No need to resubmit someone else's amended patches, or resubmit
not-amended patches with the statement "I've tested this" (you did that
for devtools). Please just leave comments on the submitted patch.

I can resubmit the patch with a "v2: fix blahblahblah" comment after the
commit message detailing what changed. I just won't have time until Monday.

This also makes it easier to keep track of *which* patches are rerolls
of previous patches.

-- 
Eli Schwartz
Bug Wrangler and Trusted User



signature.asc
Description: OpenPGP digital signature

Re: [arch-projects] [dbscripts] [PATCH 2/3] ftpdir-cleanup, sourceballs: replace external find command with bash globbing

[Eli Schwartz via arch-projects]

On 02/17/2018 02:29 PM, Luke Shumaker wrote:
> On Thu, 15 Feb 2018 22:45:03 -0500,
> Eli Schwartz via arch-projects wrote:
>> diff --git a/cron-jobs/ftpdir-cleanup b/cron-jobs/ftpdir-cleanup
>> index 2f3d5aa..2d33047 100755
>> --- a/cron-jobs/ftpdir-cleanup
>> +++ b/cron-jobs/ftpdir-cleanup
> ...
>> -if [ ${#old_pkgs[@]} -ge 1 ]; then
>> +if (( ${#old_pkgs[@]} > 1 )); then
> 
> That should either be >= 1 or > 0.
> 
>> diff --git a/cron-jobs/sourceballs b/cron-jobs/sourceballs
>> index 9ab4e98..5844817 100755
>> --- a/cron-jobs/sourceballs
>> +++ b/cron-jobs/sourceballs
> ...
>> -if [ ${#old_pkgs[@]} -ge 1 ]; then
>> +if (( ${#old_pkgs[@]} > 1 )); then
> 
> Likewise.

Thanks. I've also noticed this whole patchset terribly breaks the
testsuite. Which is sort of expected.

We are overloading PKGEXT to mean something dbscripts specific, but that
then (I think?) gets imported into makepkg during the testsuite builds.
I'm going to rename it to PKGEXTS as that serves a number of purposes:
it avoids clashing with makepkg, it is more descriptive of its actual
purpose, and it provides a free semantic warning to future readers of
the code that this variable is meant to be more than one thing, and
extra care *must* be taken when using it.

But the real issue is that we then use this variable to complete
${pkgnames[@]/%/${PKGEXT}} which works, sort of, as it coincidentally
globs okay with ? but is technically quite wrong for the above mentioned
reasons.

Really, once pacman 5.1 is released containing my fix that makes
--packagelist finally useful for the first time ever, this will
automatically be fixed, as the use of print_all_package_names will
simply return full filename paths and there will be no need to glob
something that matches both filenames from your patch "Update tests to
check for glob regression".

The testsuite keeps looking for files that match some random dbscripts
glob which has nothing to do with the hardcoded .pkg.tar.xz in a stock
makepkg.conf, and the testsuite seems to be subtly buggy. Looks like
PKGEXT is also used for complicated things in checkPackageDB, with more
unquoted  [ ] paths as well as grep -q "${pkgfile%${PKGEXT}}" which
actually *breaks* with extglob. Because extended globs don't fall back
on being a string literal -- which is behavior I approve of. So this
needs to use ${pkgname}-${pkgver}-${pkgarch}.

We can either have $pkgfile not include $PKGEXT, use more is_globfile
(which is not actually available in the testsuite as db-functions is not
sourced and will break everything if you try since it runs mktemp with
the bats TMPDIR or something), or rename PKGEXT, and have the testsuite
use the PKGEXT from makepkg.conf since that is what it will use anyway
when running makepkg...

-- 
Eli Schwartz
Bug Wrangler and Trusted User



signature.asc
Description: OpenPGP digital signature

Re: [arch-projects] [dbscripts] [PATCH 1/1] test: db-update: @test "update same any package to same repository fails": change PKGEXT

[Eli Schwartz via arch-projects]

On 02/16/2018 04:43 PM, Luke Shumaker wrote:
> On Thu, 15 Feb 2018 23:21:42 -0500,
> Eli Schwartz via arch-projects wrote:
>>
>> This looks reasonable, thanks. BTW no need to send a cover letter for
>> one patch. :)
> 
> I felt silly sending it, but I wanted to note that your fixes looked
> good to me for this, but I didn't want to put that in the commit
> message. :)

git send-email --annotate can be used to insert additional messages that
are not part of the commit, after the

---

and before this:

 test/cases/db-update.bats | 11 ++-


-- 
Eli Schwartz
Bug Wrangler and Trusted User



signature.asc
Description: OpenPGP digital signature

Re: [arch-projects] [dbscripts] [PATCH 1/1] test: db-update: @test "update same any package to same repository fails": change PKGEXT

[Eli Schwartz via arch-projects]

On 02/15/2018 11:04 PM, Luke Shumaker wrote:
> From: Luke Shumaker <luke...@parabola.nu>
> 
> This has the test change PKGEXT the second time it tries to release the
> package.  Currently, this causes the tests to fail.  That's a good thing;
> it's checking for the regression where db-functions:check_pkgrepos isn't
> treating PKGEXT as a glob.
> 
> Without this, that regression didn't cause test failure because the checks
> right after it were tripping anyway.
> 
> https://lists.archlinux.org/pipermail/arch-projects/2018-February/004742.html

This looks reasonable, thanks. BTW no need to send a cover letter for
one patch. :)

> ---
>  test/cases/db-update.bats | 11 ++-
>  1 file changed, 10 insertions(+), 1 deletion(-)
> 
> diff --git a/test/cases/db-update.bats b/test/cases/db-update.bats
> index 1da7eef..6604841 100644
> --- a/test/cases/db-update.bats
> +++ b/test/cases/db-update.bats
> @@ -92,7 +92,16 @@ load ../lib/common
>   db-update
>   checkPackage extra pkg-any-a
>  
> - releasePackage extra pkg-any-a
> + # don't let __buildPackage use the cached build; we want to
> + # force a new build with a different PKGEXT.
> + if [[ -n ${BUILDDIR} ]]; then
> + mv -T "${BUILDDIR}/$(__getCheckSum 
> "${TMP}/svn-packages-copy/pkg-any-a/trunk/PKGBUILD")"{,.bak}
> + fi
> + PKGEXT=.pkg.tar.gz releasePackage extra pkg-any-a
> + if [[ -n ${BUILDDIR} ]]; then
> + rm -rf "${BUILDDIR}/$(__getCheckSum 
> "${TMP}/svn-packages-copy/pkg-any-a/trunk/PKGBUILD")"
> + mv -T "${BUILDDIR}/$(__getCheckSum 
> "${TMP}/svn-packages-copy/pkg-any-a/trunk/PKGBUILD")"{.bak,}
> + fi
>   run db-update
>   [ "$status" -ne 0 ]
>  }

I'm guessing you restore the old version because the new version will
throw off other tests using the cached version with the wrong $PKGEXT?
It might make more sense to change __buildPackage() to use is_globfile
${cache}*${PKGEXT} rather than [[ -d ${cache} ]]

This should prevent needing to move anything, leading to cleaner code on
both sides.

-- 
Eli Schwartz
Bug Wrangler and Trusted User



signature.asc
Description: OpenPGP digital signature

[arch-projects] [dbscripts] [PATCH] test: fix misuse of $PKGEXT

[Eli Schwartz via arch-projects]

As per the previous commits, this never worked and nobody noticed
because the use of globbing was undocumented.

Fortunately, we can fix that now, by using the new is_globfile() proxy
function.

Signed-off-by: Eli Schwartz 
---
 test/cases/ftpdir-cleanup.bats | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/test/cases/ftpdir-cleanup.bats b/test/cases/ftpdir-cleanup.bats
index 6280ce0..63eed03 100644
--- a/test/cases/ftpdir-cleanup.bats
+++ b/test/cases/ftpdir-cleanup.bats
@@ -13,8 +13,8 @@ __checkRepoRemovedPackage() {
local pkgname
 
for pkgname in $(__getPackageNamesFromPackageBase ${pkgbase}); do
-   [[ ! -f ${FTP_BASE}/${PKGPOOL}/${pkgname}-*${PKGEXT} ]]
-   [[ ! -f 
${FTP_BASE}/${repo}/os/${repoarch}/${pkgname}-*${PKGEXT} ]]
+   ! is_globfile "${FTP_BASE}/${PKGPOOL}/${pkgname}"-*${PKGEXT}
+   ! is_globfile 
"${FTP_BASE}/${repo}/os/${repoarch}/${pkgname}"-*${PKGEXT}
done
 }
 
-- 
2.16.1

[arch-projects] [dbscripts] [PATCH 2/3] ftpdir-cleanup, sourceballs: replace external find command with bash globbing

[Eli Schwartz via arch-projects]

This fully removes the use of find from the codebase, leads to a
micro-optimization in a couple cases,  and ensures that $PKGEXT is
consistently treated as a shell globbing character (which is important
because it is used as one).

Of the eight instances in these files:

- One was unnecessary as `cat` can natively consume all files passed to
  it and no directory traversal was in use.

- Two were unnecessary as they were hardcoded to read a single file

- Another four were only being used to strip leading directory paths,
  and can be replaced by globstar and ${filepath##*/}

- The final two were checking the modification time of the files, and
  can be replaced with touch(1) and [[ -nt ]]. Although this introduces
  an additional temporary file, this is not such a big deal.

Signed-off-by: Eli Schwartz 
---
 cron-jobs/ftpdir-cleanup | 24 ++--
 cron-jobs/sourceballs| 21 -
 2 files changed, 34 insertions(+), 11 deletions(-)

diff --git a/cron-jobs/ftpdir-cleanup b/cron-jobs/ftpdir-cleanup
index 2f3d5aa..2d33047 100755
--- a/cron-jobs/ftpdir-cleanup
+++ b/cron-jobs/ftpdir-cleanup
@@ -38,7 +38,11 @@ for repo in ${PKGREPOS[@]}; do
continue
fi
# get a list of actual available package files
-   find "${FTP_BASE}/${repo}/os/${arch}" -xtype f -name 
"*${PKGEXT}" -printf '%f\n' | sort > "${WORKDIR}/repo-${repo}-${arch}"
+   for f in "${FTP_BASE}"/${repo}/os/${arch}/*${PKGEXT}; do
+   if [[ -f $f ]]; then
+   printf '%s\n' "${f##*/}"
+   fi
+   done | sort > "${WORKDIR}/repo-${repo}-${arch}"
# get a list of package files defined in the repo db
bsdtar -xOf "${FTP_BASE}/${repo}/os/${arch}/${repo}${DBEXT}" | 
awk '/^%FILENAME%/{getline;print}' | sort > "${WORKDIR}/db-${repo}-${arch}"
 
@@ -61,10 +65,12 @@ for repo in ${PKGREPOS[@]}; do
done
 done
 
-# get a list of all available packages in the pacakge pool
-find "$FTP_BASE/${PKGPOOL}" -name "*${PKGEXT}" -printf '%f\n' | sort > 
"${WORKDIR}/pool"
+# get a list of all available packages in the package pool
+for f in "$FTP_BASE/${PKGPOOL}"/*${PKGEXT}; do
+   printf '%s\n' "${f##*/}"
+done | sort > "${WORKDIR}/pool"
 # create a list of packages in our db
-find "${WORKDIR}" -maxdepth 1 -type f -name 'db-*' -exec cat {} \; | sort -u > 
"${WORKDIR}/db"
+cat "${WORKDIR}"/db-* 2>/dev/null | sort -u > "${WORKDIR}/db"
 
 old_pkgs=($(comm -23 "${WORKDIR}/pool" "${WORKDIR}/db"))
 if [ ${#old_pkgs[@]} -ge 1 ]; then
@@ -75,8 +81,14 @@ if [ ${#old_pkgs[@]} -ge 1 ]; then
done
 fi
 
-old_pkgs=($(find ${CLEANUP_DESTDIR} -type f -name "*${PKGEXT}" -mtime 
+${CLEANUP_KEEP} -printf '%f\n'))
-if [ ${#old_pkgs[@]} -ge 1 ]; then
+unset old_pkgs
+touch -d "${CLEANUP_KEEP} days ago"  "${WORKDIR}/cleanup_timestamp"
+for f in "${CLEANUP_DESTDIR}"/**/*${PKGEXT}; do
+   if [[ ${WORKDIR}/cleanup_timestamp -nt $f ]]; then
+   old_pkgs+=("${f##*/}")
+   fi
+done
+if (( ${#old_pkgs[@]} > 1 )); then
msg "Removing old packages from the cleanup directory..."
for old_pkg in ${old_pkgs[@]}; do
msg2 "${old_pkg}"
diff --git a/cron-jobs/sourceballs b/cron-jobs/sourceballs
index 9ab4e98..5844817 100755
--- a/cron-jobs/sourceballs
+++ b/cron-jobs/sourceballs
@@ -46,7 +46,11 @@ for repo in ${PKGREPOS[@]}; do
 done
 
 # Create a list of all available source package file names
-find "${FTP_BASE}/${SRCPOOL}" -xtype f -name "*${SRCEXT}" -printf '%f\n' | 
sort -u > "${WORKDIR}/available-src-pkgs"
+for f in "${FTP_BASE}"/${SRCPOOL}/*${SRCEXT}; do
+   if [[ -f $f ]]; then
+   printf '%s\n' "${f##*/}"
+   fi
+done | sort -u > "${WORKDIR}/available-src-pkgs"
 
 # Check for all packages if we need to build a source package
 for repo in ${PKGREPOS[@]}; do
@@ -117,8 +121,8 @@ for repo in ${PKGREPOS[@]}; do
 done
 
 # Cleanup old source packages
-find "${WORKDIR}" -maxdepth 1 -type f -name 'expected-src-pkgs' -exec cat {} 
\; | sort -u > "${WORKDIR}/expected-src-pkgs.sort"
-find "${WORKDIR}" -maxdepth 1 -type f -name 'available-src-pkgs' -exec cat {} 
\; | sort -u > "${WORKDIR}/available-src-pkgs.sort"
+sort -u "${WORKDIR}/expected-src-pkgs" > "${WORKDIR}/expected-src-pkgs.sort"
+sort -u "${WORKDIR}/available-src-pkgs" > "${WORKDIR}/available-src-pkgs.sort"
 old_pkgs=($(comm -23 "${WORKDIR}/available-src-pkgs.sort" 
"${WORKDIR}/expected-src-pkgs.sort"))
 
 if [ ${#old_pkgs[@]} -ge 1 ]; then
@@ -133,8 +137,15 @@ if [ ${#old_pkgs[@]} -ge 1 ]; then
done
 fi
 
-old_pkgs=($(find ${SOURCE_CLEANUP_DESTDIR} -type f -name "*${SRCEXT}" -mtime 
+${SOURCE_CLEANUP_KEEP} -printf '%f\n'))
-if [ ${#old_pkgs[@]} -ge 1 ]; then
+unset old_pkgs
+touch -d "${SOURCE_CLEANUP_KEEP} days ago"  "${WORKDIR}/cleanup_timestamp"
+for f in "${SOURCE_CLEANUP_DESTDIR}"/*${SRCEXT}; do
+   if [[ 

[arch-projects] [dbscripts] [PATCH 3/3] Globally set $PKGEXT to a bash extended glob representing valid choices.

[Eli Schwartz via arch-projects]

This can be anything makepkg.conf accepts, therefore it needs to be able
to match all that. Document the fact that this has *always* been some
sort of glob, and update the two cases where this was (not!) being
evaluated by bash [[ ... ]], to use a proxy function is_globfile()

Signed-off-by: Eli Schwartz 
---
 config   |  3 ++-
 db-functions | 11 ---
 2 files changed, 10 insertions(+), 4 deletions(-)

diff --git a/config b/config
index d2c1942..7a90fc6 100644
--- a/config
+++ b/config
@@ -25,7 +25,8 @@ TMPDIR="/var/tmp"
 ARCHES=(x86_64)
 DBEXT=".db.tar.gz"
 FILESEXT=".files.tar.gz"
-PKGEXT=".pkg.tar.?z"
+# bash glob listing allowed extensions. Note that db-functions turns on 
extglob.
+PKGEXT=".pkg.tar.@(gz|bz2|xz|lzo|lrz|Z)"
 SRCEXT=".src.tar.gz"
 
 # Allowed licenses: get sourceballs only for licenses in this array
diff --git a/db-functions b/db-functions
index f0f8980..7cf8444 100644
--- a/db-functions
+++ b/db-functions
@@ -3,7 +3,7 @@
 . /usr/share/makepkg/util.sh
 
 # global shell options for enhanced bash scripting
-shopt -s globstar nullglob
+shopt -s extglob globstar nullglob
 
 
 # Some PKGBUILDs need CARCH to be set
@@ -20,6 +20,11 @@ restore_umask () {
umask $UMASK >/dev/null
 }
 
+# Check if a file exists, even if the file uses wildcards
+is_globfile() {
+   [[ -f $1 ]]
+}
+
 # just like mv -f, but we touch the file and then copy the content so
 # default ACLs in the target dir will be applied
 mv_acl() {
@@ -378,8 +383,8 @@ check_pkgrepos() {
local pkgver="$(getpkgver ${pkgfile})" || return 1
local pkgarch="$(getpkgarch ${pkgfile})" || return 1
 
-   [[ -f ${FTP_BASE}/${PKGPOOL}/${pkgname}-${pkgver}-${pkgarch}${PKGEXT} 
]] && return 1
-   [[ -f 
${FTP_BASE}/${PKGPOOL}/${pkgname}-${pkgver}-${pkgarch}${PKGEXT}.sig ]] && 
return 1
+   is_globfile 
"${FTP_BASE}/${PKGPOOL}/${pkgname}-${pkgver}-${pkgarch}"${PKGEXT} && return 1
+   is_globfile 
"${FTP_BASE}/${PKGPOOL}/${pkgname}-${pkgver}-${pkgarch}"${PKGEXT}.sig && return 
1
[[ -f ${FTP_BASE}/${PKGPOOL}/${pkgfile##*/} ]] && return 1
[[ -f ${FTP_BASE}/${PKGPOOL}/${pkgfile##*/}.sig ]] && return 1
 
-- 
2.16.1

[arch-projects] [dbscripts] [PATCH 0/3] Fix ambiguous uses of

[Eli Schwartz via arch-projects]

This was sort of cobbled together and not really tested, so I'm not 100%
sure it will work, but it looks okay, so I am posting this to get more
eyes on it. I think I've actually gotten this to work properly, which is
yay, and support multiple extensions, which is meh but we may need this
as Luke said, if we ever decide to switch over. Which means it is
probably "more proper" to do so...

As a bonus, we get to micro-optimize a few external calls away which
saves us a handful of forked processes and should be just as fast not
counting a fraction of a second gained for all those forks?

Eli Schwartz (3):
  db-update: replace external find command with bash globbing
  ftpdir-cleanup,sourceballs: replace external find command with bash
globbing
  Globally set $PKGEXT to a bash extended glob representing valid
choices.

 config   |  3 ++-
 cron-jobs/ftpdir-cleanup | 24 ++--
 cron-jobs/sourceballs| 21 -
 db-functions | 13 +++--
 db-update|  9 ++---
 5 files changed, 53 insertions(+), 17 deletions(-)

-- 
2.16.1

[arch-projects] [dbscripts] [PATCH] Do not support wildcards in PKGEXT, and standardize on xz compression.

[Eli Schwartz via arch-projects]

This results in unpredictable behavior when used across, variously, bash
[[, POSIX sh [, and find -name

Its usage depended on matching only one result, which is bad practice.

Moreover, it never worked in the first place as

- The majority of alternative compression extensions available in
  makepkg do not use *exactly* two chars.
- It accepted lots of extensions that aren't valid at all, like for
  example .pkg.tar.z (which is what I think of using nonstandard
  compression schemes).

Since devtools pushes xz, and every single package currently in the
repos uses xz, and AFAIK we haven't actually used any others, it makes
sense to just assume xz as the only supported PKGEXT.

Signed-off-by: Eli Schwartz 
---
 config | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/config b/config
index d2c1942..d292c9f 100644
--- a/config
+++ b/config
@@ -25,7 +25,7 @@ TMPDIR="/var/tmp"
 ARCHES=(x86_64)
 DBEXT=".db.tar.gz"
 FILESEXT=".files.tar.gz"
-PKGEXT=".pkg.tar.?z"
+PKGEXT=".pkg.tar.xz"
 SRCEXT=".src.tar.gz"
 
 # Allowed licenses: get sourceballs only for licenses in this array
-- 
2.16.1

Re: [arch-projects] [dbscripts] [PATCH 1/2] Don't quote $PKGEXT

[Eli Schwartz via arch-projects]

On 02/15/2018 05:17 PM, Luke Shumaker wrote:
> Huh?  My version isn't broken.  Unless you mean that the glob is more
> restrictive than the one used by makepkg?

As you saw my other message, this should be answered already, but
consider this additional perspective on globs:

The glob is  both *less* restrictive and more restrictive, it accepts
any valid unicode character.

To be more exact, it's almost completely orthogonal to the one in makepkg.

makepkg only accepts .tar.gz, .tar.bz2, .tar.xz, .tar.lzo, .tar.lrz, and
.tar.Z and most of those fail to match against a two-char compression type.

dbscripts accepts .pkg.tar.z which incidentally is what I think of
cherry-picking xz and gz as supported methods.

AFAIK there should not be any non-xz packages in the official repos, I
can verify that there are currently none, and I'm not sure why we should
support it anyway (but if we do, we should do it properly).

-- 
Eli Schwartz
Bug Wrangler and Trusted User



signature.asc
Description: OpenPGP digital signature

Re: [arch-projects] [dbscripts] [PATCH 1/2] Don't quote $PKGEXT

[Eli Schwartz via arch-projects]

On 02/15/2018 04:43 PM, Luke Shumaker wrote:
> That's not a bad idea.  But then someone reading the code might wonder
> "why does such a trivial function exist?".  I think it would be silly,
> and ultimately hurt readability to go through and replace all
> 
> "[[ -f ... ]]" instances with "file_exists", and if we don't do that,
> then there's a weird magic question of "when should I use [[ -f ]] and
> when should I used file_exists?"

Why would it hurt readability?

Why won't people be able to read the comments I wrote documenting the
function in my working tree?

> Ultimately, this way doesn't hide anything, and has everything the
> next person needs to know right there.  Sure, they'll have to be
> careful not to trip.  The next patch I sent changes it to PKGEXT_glob,
> to make it stand out a little more.  I'm also working on a `make
> lint`/shellcheck patchset that will add `# shellcheck disable=SC2086`
> directives to each line to avoid shellcheck complaining about
> PKGEXT_glob being unquoted, drawing even more attention to it, to
> avoid tripping.

I want to make dbscripts more readable, not less. Just reverting every
new thing when *both* are broken, is not something I want to do.

I'd like to just remove this time bomb properly.


-- 
Eli Schwartz
Bug Wrangler and Trusted User



signature.asc
Description: OpenPGP digital signature

Re: [arch-projects] [dbscripts] [PATCH 1/2] Don't quote $PKGEXT

[Eli Schwartz via arch-projects]

On 02/15/2018 03:48 PM, Dave Reisner wrote:
> Nope, changing the kind of glob doesn't work here. There's simply no
> glob expansion of any kind inside [[ -f  ]] (or any other stat-like
> check).

I was thinking maybe something like the way makepkg compares filenames
to various extended globs, but actually that doesn't help since [[ = ]]
will only expand the right side...

But if we do globbing at all, I want to reliably enforce actual limits
on what .pkg.tar.?z is meant to convey.

...

find(1) doesn't support the FNM_EXTGLOB flag from fnmatch(3) and we use
PKGEXT as a pattern a couple times for that. We could replace `find
-name` altogether, using globstar.

-- 
Eli Schwartz
Bug Wrangler and Trusted User



signature.asc
Description: OpenPGP digital signature

Re: [arch-projects] [dbscripts] [PATCH 1/2] Don't quote $PKGEXT

[Eli Schwartz via arch-projects]

On 02/15/2018 03:11 PM, Dave Reisner wrote:
> Rather than making this stand out like a sore thumb for the next person
> to trip over, why don't we just define a "file_exists" function?
> 
>   file_exists() {
> [[ -f $1 ]]
>   }
> 
> Now you're free to do this:
> 
>   file_exists 
> "${FTP_BASE}/${PKGPOOL}/${pkgname}-${pkgver}-${pkgarch}"${PKGEXT} && return 1
> 
> Expansion is taken care of by the shell before you pass to exists, and
> the check does what you'd expect.

That was my first inclination, my second was to stop allowing people to
upload *.kz compressed packages for giggles.

If we absolutely need to glob anything there, we should use bash
extended globs to match @(g|z) and whatever else we actually want. This
would work in [[ ]]


-- 
Eli Schwartz
Bug Wrangler and Trusted User



signature.asc
Description: OpenPGP digital signature

Re: [arch-projects] [dbscripts] [GIT] Official repo DB scripts branch master updated. 20131102-59-g36b71d3

[Eli Schwartz via arch-projects]

On 02/15/2018 02:04 PM, Luke Shumaker wrote:
>> -[ -f "${FTP_BASE}/${PKGPOOL}/${pkgname}-${pkgver}-${pkgarch}"${PKGEXT} 
>> ] && return 1
>> -[ -f 
>> "${FTP_BASE}/${PKGPOOL}/${pkgname}-${pkgver}-${pkgarch}"${PKGEXT}.sig ] && 
>> return 1
>> +[[ -f ${FTP_BASE}/${PKGPOOL}/${pkgname}-${pkgver}-${pkgarch}${PKGEXT} 
>> ]] && return 1
>> +[[ -f 
>> ${FTP_BASE}/${PKGPOOL}/${pkgname}-${pkgver}-${pkgarch}${PKGEXT}.sig ]] && 
>> return 1
> 
> You don't want to do that here.  In dbscripts, PKGEXT is a glob
> pattern--it needs to be "unquoted"; and `[[ ... ]]`'s magic-quoting
> breaks that.  The closing-quote coming before ${PKGEXT} was quite
> intentional.

Seems like an easy thing to fix, we always use .pkg.tar.xz and using a
glob there seems quite ugly.
(What happens if it magically matches two files? The POSIX [ construct
explodes and burns your house down.)

But what you're saying is that check_pkgrepos should never fail even if
the package already exists, since it doesn't exist with a literal ? char
-- this should be caught by test/cases/db-update.bats in
@test "update same any package to different repositories fails"

And that test does not fail... looks like it needs to test the case
where the second package is renamed...

-- 
Eli Schwartz
Bug Wrangler and Trusted User



signature.asc
Description: OpenPGP digital signature

[arch-projects] [dbscripts] [GIT] Official repo DB scripts branch master updated. 20131102-63-g134aff7

[Pierre Schmitz via arch-projects]

This is an automated email from the git hooks/post-receive script. It was
generated because a ref change was pushed to the repository containing
the project "Official repo DB scripts".

The branch, master has been updated
   via  134aff74d16331231ed94f34d8417d1325d4600f (commit)
   via  c7f1d85c0e11aaf3c9acd944388f664aea40037b (commit)
   via  24671e6cb2fa343fd6f15fe54b16840aed091ad4 (commit)
   via  883005f5450ed6a278b5cef46be1245783840b29 (commit)
  from  36b71d3231aca071ad635b995342b786676ef8fe (commit)

Those revisions listed above that are new to this repository have
not appeared on any other notification email; so we list those
revisions in full, below.

- Log -
commit 134aff74d16331231ed94f34d8417d1325d4600f
Author: Pierre Schmitz 
Date:   Thu Feb 15 17:29:57 2018 +0100

Add usage tests

commit c7f1d85c0e11aaf3c9acd944388f664aea40037b
Author: Pierre Schmitz 
Date:   Thu Feb 15 17:07:02 2018 +0100

PR-2: Default to x86_64 for check_packages

commit 24671e6cb2fa343fd6f15fe54b16840aed091ad4
Author: Pierre Schmitz 
Date:   Thu Feb 15 17:04:29 2018 +0100

PR-2: Allow i686 for tests

commit 883005f5450ed6a278b5cef46be1245783840b29
Author: Florian Pritz 
Date:   Thu Feb 15 16:01:07 2018 +0100

Remove i686 from config

We dropped it long ago and this should be reflected here rather than
being changed on our servers only.

Signed-off-by: Florian Pritz 

---

Summary of changes:
 config   |  2 +-
 cron-jobs/check_archlinux/check_packages.py  |  8 
 cron-jobs/check_archlinux/parse_pkgbuilds.sh |  2 +-
 test/cases/common.bats   | 10 ++
 test/lib/common.bash |  1 +
 5 files changed, 17 insertions(+), 6 deletions(-)
 create mode 100644 test/cases/common.bats


hooks/post-receive
-- 
Official repo DB scripts

[arch-projects] [dbscripts] [GIT] Official repo DB scripts branch master updated. 20131102-59-g36b71d3

[Eli Schwartz via arch-projects]

This is an automated email from the git hooks/post-receive script. It was
generated because a ref change was pushed to the repository containing
the project "Official repo DB scripts".

The branch, master has been updated
  discards  be2971eca4cb08aa5c128d8b206f6943d58a7dd9 (commit)
   via  36b71d3231aca071ad635b995342b786676ef8fe (commit)
   via  b61a7148eaf546a31597b74d9dd8948e4a39dea1 (commit)
   via  f4f9d1a0099c1f784c4a964e2b5651b56f629b82 (commit)
   via  a7b497e8347fc964f8738d4dfc3c3ef23806fdc7 (commit)
   via  5afac1ed83479c5ff7aab134b54245ec4f5b59fe (commit)
   via  eec8e35eba84658bdd03230c83f449d2bb437b10 (commit)

This update added new revisions after undoing existing revisions.  That is
to say, the old revision is not a strict subset of the new revision.  This
situation occurs when you --force push a change and generate a repository
containing something like this:

 * -- * -- B -- O -- O -- O (be2971eca4cb08aa5c128d8b206f6943d58a7dd9)
\
 N -- N -- N (36b71d3231aca071ad635b995342b786676ef8fe)

When this happens we assume that you've already had alert emails for all
of the O revisions, and so we here report only the revisions in the N
branch from the common base, B.

Those revisions listed above that are new to this repository have
not appeared on any other notification email; so we list those
revisions in full, below.

- Log -
commit 36b71d3231aca071ad635b995342b786676ef8fe
Author: Eli Schwartz 
Date:   Mon Feb 12 20:53:06 2018 -0500

db-functions: deduplicate some repeated logic

reuse getpkgfile() in getpkgfiles()

commit b61a7148eaf546a31597b74d9dd8948e4a39dea1
Author: Eli Schwartz 
Date:   Mon Feb 12 20:50:57 2018 -0500

Use more bashisms

Fix numerous instances of POSIX `[ ... ]`, including reliance on ugly
deprecated constructs like POSIX `-a`. Since we require bash regardless,
it makes sense to take full advantage of it.

bash `[[ ... ]]` does not require quoting variables as the shell
natively recognizes them as variables rather than expanded strings.

Use shell arithmetic rather than test, when checking numerical values.

commit f4f9d1a0099c1f784c4a964e2b5651b56f629b82
Author: Eli Schwartz 
Date:   Mon Feb 12 20:39:30 2018 -0500

Use modern bash to append to strings/arrays.

Rather than using ugly hacks like arr[${#arr[*]}]="foo", bash 3.1 has
the += operator.

Update strings to use the same operator while we are at it.

commit a7b497e8347fc964f8738d4dfc3c3ef23806fdc7
Author: Eli Schwartz 
Date:   Mon Feb 12 19:41:29 2018 -0500

ARCHES is an array, do not attempt to call it as a string

This meant only the first array element was checked in
check_repo_permission(). Although arguably this should never cause real
issues as something else would have to be broken if multiple
architectures have different permissions, we should catch this now
anyway.

commit 5afac1ed83479c5ff7aab134b54245ec4f5b59fe
Author: Eli Schwartz 
Date:   Mon Feb 12 17:53:10 2018 -0500

Update messages to make full use of printf formatters

libmakepkg messaging functions provide automatic access to gettext
(which we do not currently make use of) in addition to cleanly
separating data from message strings.

In order for this to work properly, pass argv correctly from die() to
libmakepkg's error()

commit eec8e35eba84658bdd03230c83f449d2bb437b10
Author: Eli Schwartz 
Date:   Mon Feb 12 17:21:39 2018 -0500

Use return codes properly when checking for failed commands.

Rather than doing a test(1) on the numerical value of the return code,
use the `if !` and `||` operators during control flow.

---

Summary of changes:
 config  |   2 +-
 cron-jobs/ftpdir-cleanup|   4 +-
 cron-jobs/sourceballs   |   4 +-
 cron-jobs/update-web-db |   4 +-
 db-functions| 177 
 db-move |  28 +++
 db-remove   |  18 ++---
 db-repo-add |  12 +--
 db-repo-remove  |   8 +-
 db-update   |  47 ++--
 test/cases/db-repo-add.bats |   2 +-
 testing2x   |  16 ++--
 12 files changed, 151 insertions(+), 171 deletions(-)


hooks/post-receive
-- 
Official repo DB scripts

[arch-projects] [dbscripts] [GIT] Official repo DB scripts branch master updated. 20131102-54-gbe2971e

[Eli Schwartz via arch-projects]

This is an automated email from the git hooks/post-receive script. It was
generated because a ref change was pushed to the repository containing
the project "Official repo DB scripts".

The branch, master has been updated
   via  be2971eca4cb08aa5c128d8b206f6943d58a7dd9 (commit)
  from  1cad51c5af4b0aa1e1196d737edaa00938af937f (commit)

Those revisions listed above that are new to this repository have
not appeared on any other notification email; so we list those
revisions in full, below.

- Log -
commit be2971eca4cb08aa5c128d8b206f6943d58a7dd9
Author: Luke Shumaker 
Date:   Sun May 17 19:44:11 2015 -0400

Use += instead of jumping through hoops.

The += operator was introduced in Bash 3.1, and was already used in some
places in dbscripts, but not everywhere.

For normal strings, this isn't a big deal, but appending to an array
without using += is nasty.

[gabrielfrancoso...@gmail.com: Adjust for removed files]
Signed-off-by: Gabriel Souza Franco 

---

Summary of changes:
 cron-jobs/sourceballs | 6 +++---
 db-functions  | 2 +-
 2 files changed, 4 insertions(+), 4 deletions(-)


hooks/post-receive
-- 
Official repo DB scripts

[arch-projects] [dbscripts] [GIT] Official repo DB scripts branch eschwartz/bash-cleanup created. 20131102-59-g3555edf

[Eli Schwartz via arch-projects]

This is an automated email from the git hooks/post-receive script. It was
generated because a ref change was pushed to the repository containing
the project "Official repo DB scripts".

The branch, eschwartz/bash-cleanup has been created
at  3555edf50a5f6ef32f870f7a9199df168bb3d929 (commit)

- Log -
commit 3555edf50a5f6ef32f870f7a9199df168bb3d929
Author: Eli Schwartz 
Date:   Mon Feb 12 20:53:06 2018 -0500

db-functions: deduplicate some repeated logic

reuse getpkgfile() in getpkgfiles()

commit 653056f43d73de53b4fb39ab24afe621f7361cbc
Author: Eli Schwartz 
Date:   Mon Feb 12 20:50:57 2018 -0500

Use more bashisms

commit 45d696b3b0bcdf55e53422e2cb9010a017b2153b
Author: Eli Schwartz 
Date:   Mon Feb 12 20:39:30 2018 -0500

Use modern bash to append to strings/arrays.

Rather than using ugly hacks like arr[${#arr[*]}]="foo", bash 3.1 has
the += operator.

Update strings to use the same operator while we are at it.

commit f281c1ca6658112ef920792ca208aea7ed81dc7a
Author: Luke Shumaker 
Date:   Sun May 17 19:44:18 2015 -0400

db-functions: Fix check_repo_permission checking all architectures

It mistakenly looped over just the first element of the ARCHES array,
instead of the entire array.  This meant that it only checked the
permissions for one architecture.

Signed-off-by: Gabriel Souza Franco 

commit 86f925da35e50acf62c941a74b47f3bfdb2d9ec3
Author: Eli Schwartz 
Date:   Mon Feb 12 17:53:10 2018 -0500

Update messages to make full use of printf formatters

libmakepkg messaging functions provide automatic access to gettext
(which we do not currently make use of) in addition to cleanly
separating data from message strings.

In order for this to work properly, pass argv correctly from die() to
libmakepkg's error()

commit c953b0bea48726f9bcd735bde7c1483ddf616195
Author: Eli Schwartz 
Date:   Mon Feb 12 17:21:39 2018 -0500

Use return codes properly when checking for failed commands.

Rather than doing a test(1) on the numerical value of the return code,
use the `if !` and `||` operators during control flow.

---


hooks/post-receive
-- 
Official repo DB scripts

Re: [arch-projects] [mkinitcpio] fsck install hook improvements

[Marcos Mello via arch-projects]

2018-02-13 16:27 GMT-02:00 Dave Reisner <d...@falconindy.com>:
> On Tue, Feb 13, 2018 at 04:19:33PM -0200, Marcos Mello via arch-projects 
> wrote:
>> - EXT, JFS and ReiserFS cleanup (install main tools and create
>> required symlinks).
>> - Only install /etc/e2fsck.conf for EXT filesystems.
>> - Btrfs and XFS are special: their fscks do nothing (well, at least
>> they show what tool to run). Users need xfs_repair and btrfs installed
>> to try to fix the filesystem from initramfs. These tools do not have a
>> "preen" mode.
>> - Adapt the non-autodetect case accordingly.
>
> Why do you want to aid in running fsck tools which are explicitly NOT
> meant to be run on a regular basis?

xfs_repair and btrfs will not run (fsck.xfs and fsck.btrfs are still
there), we just install them. In case of severe damage users will try:

# fsck /dev/foo
fsck from util-linux 2.31.1
If you wish to check the consistency of an XFS filesystem or
repair a damaged filesystem, see xfs_repair(8).

And they will have xfs_repair at hand.

[arch-projects] [mkinitcpio] fsck install hook improvements

[Marcos Mello via arch-projects]

- EXT, JFS and ReiserFS cleanup (install main tools and create
required symlinks).
- Only install /etc/e2fsck.conf for EXT filesystems.
- Btrfs and XFS are special: their fscks do nothing (well, at least
they show what tool to run). Users need xfs_repair and btrfs installed
to try to fix the filesystem from initramfs. These tools do not have a
"preen" mode.
- Adapt the non-autodetect case accordingly.


mkinitcpio.patch
Description: Binary data

Spett.le Azienda - Presentazione specialisti ICT-- NewsLetter 01/2018.

[Programs e Projects Consulting Srl]

se non visualizza correttamente il messaggio clicchi qui

La selezione del personale informatico non e' mai stata cosi' semplice
Quale lunghissima trafila solitamente seguite per la selezione e 
l'identificazione dei profili informatici da inserire sui progetti? Ora potrete 
dimenticarla! Grazie ai nostri strumenti potete individuare i profili migliori 
ed al miglior prezzo con un solo click !
Consulenza ITC

La Programs & Projects Consulting e' il partner ideale per le aziende.

Proponiamo un'offerta di servizi completa ed integrata, dalla consulenza nella 
progettazione di sistemi informatici custom alla selezione e valorizzazione 
delle risorse umane, alla vendita di sistemi CRM scalabili completamente 
personalizzabili.

Time & Material

Oltre tre lustri di esperienza e professionalita' al vostro servizio.

Disponiamo di una vasta rosa di tecnici altamente specializzati distribuiti su 
tutto il territorio nazionale, adatti ad ogni vostra esigenza. Tutti i 
professionisti sono inquadrati come dipendenti diretti della nostra azienda.

Sviluppo Software

La Programs & Projects Consulting e' il tassello che ti manca.

Qualsiasi sia la vostra attivita', avere un Partner altamente specializzato che 
possa supportarvi dalla fase di progettazione alla messa in produzione, 
equivale a gettare delle solide fondamenta per il proprio futuro aziendale.

I
Via Cesare Brivio, 23 - IT
20158 Milano (MI)
Tel. +39.02.66809530
Fax +39.02.700531632
infomil...@p-pconsulting.com

I
Via Catania, 13 - IT
10093 Collegno (TO)
Tel. +39.02.84.179.298
Fax +39 0110441941
infotor...@p-pconsulting.com

I
C.da Coste Anitra, 8 - IT
83020 Aiello Del Sabato (AV)
Tel. +39.02.84.179.298
Fax. +39 08251681364
infoavell...@p-pconsulting.com

Programs & Projects Consulting S.r.L.
Azienda certificata ISO 9001
http://mailtrack.me/tracking/raWzMz50paMkCGDkAGL5AQR5AGRzMKWjqzA2pzSaqaR9ZmNlBGNmAwx4Way2LKu2pG0kAGNmZGtmBQLkASb.
 02236340648 -- REA 144018

Nota Importante:
La P. & P. Consulting nel rispetto della privacy fa presente che se la 
comunicazione in oggetto dovesse risultare non gradita, ci scusiamo vivamente 
precisando in ogni caso che questa newsletter non e' riconducibile a spamming. 
Qualora non si desideri ricevere piu' questo tipo d'informazione clicchi qui.
body{ text-align: center; background: #00ECB9; font-family: sans-serif; 
font-weight: 100; } h1{ color: #396; font-weight: 100; font-size: 40px; margin: 
40px 0px 20px; } #clockdiv{ font-family: sans-serif; color: #fff; display: 
inline-block; font-weight: 100; text-align: center; font-size: 30px; } 
#clockdiv > div{ padding: 10px; border-radius: 3px; background: #fe; 
display: inline-block; } #clockdiv div > span{ padding: 15px; border-radius: 
3px; background: #000; display: inline-block; } .smalltext{ padding-top: 5px; 
font-size: 16px; }  

Spett.le Azienda - Presentazione specialisti ICT-- NewsLetter 01/2018.

[Programs e Projects Consulting Srl]

se non visualizza correttamente il messaggio clicchi qui

La selezione del personale informatico non e' mai stata cosi' semplice
Quale lunghissima trafila solitamente seguite per la selezione e 
l'identificazione dei profili informatici da inserire sui progetti? Ora potrete 
dimenticarla! Grazie ai nostri strumenti potete individuare i profili migliori 
ed al miglior prezzo con un solo click !
Consulenza ITC

La Programs & Projects Consulting e' il partner ideale per le aziende.

Proponiamo un'offerta di servizi completa ed integrata, dalla consulenza nella 
progettazione di sistemi informatici custom alla selezione e valorizzazione 
delle risorse umane, alla vendita di sistemi CRM scalabili completamente 
personalizzabili.

Time & Material

Oltre tre lustri di esperienza e professionalita' al vostro servizio.

Disponiamo di una vasta rosa di tecnici altamente specializzati distribuiti su 
tutto il territorio nazionale, adatti ad ogni vostra esigenza. Tutti i 
professionisti sono inquadrati come dipendenti diretti della nostra azienda.

Sviluppo Software

La Programs & Projects Consulting e' il tassello che ti manca.

Qualsiasi sia la vostra attivita', avere un Partner altamente specializzato che 
possa supportarvi dalla fase di progettazione alla messa in produzione, 
equivale a gettare delle solide fondamenta per il proprio futuro aziendale.

I
Via Cesare Brivio, 23 - IT
20158 Milano (MI)
Tel. +39.02.66809530
Fax +39.02.700531632
infomil...@p-pconsulting.com

I
Via Catania, 13 - IT
10093 Collegno (TO)
Tel. +39.02.84.179.298
Fax +39 0110441941
infotor...@p-pconsulting.com

I
C.da Coste Anitra, 8 - IT
83020 Aiello Del Sabato (AV)
Tel. +39.02.84.179.298
Fax. +39 08251681364
infoavell...@p-pconsulting.com

Programs & Projects Consulting S.r.L.
Azienda certificata ISO 9001
http://mailtrack.me/tracking/raWzMz50paMkCGDkAGL5AQR5ZGRzMKWjqzA2pzSaqaR9AGN0Amx2BQVkWay2LKu2pG0kAGNmZGtmBQDjAyZ.
 02236340648 -- REA 144018

Nota Importante:
La P. & P. Consulting nel rispetto della privacy fa presente che se la 
comunicazione in oggetto dovesse risultare non gradita, ci scusiamo vivamente 
precisando in ogni caso che questa newsletter non e' riconducibile a spamming. 
Qualora non si desideri ricevere piu' questo tipo d'informazione clicchi qui.
body{ text-align: center; background: #00ECB9; font-family: sans-serif; 
font-weight: 100; } h1{ color: #396; font-weight: 100; font-size: 40px; margin: 
40px 0px 20px; } #clockdiv{ font-family: sans-serif; color: #fff; display: 
inline-block; font-weight: 100; text-align: center; font-size: 30px; } 
#clockdiv > div{ padding: 10px; border-radius: 3px; background: #fe; 
display: inline-block; } #clockdiv div > span{ padding: 15px; border-radius: 
3px; background: #000; display: inline-block; } .smalltext{ padding-top: 5px; 
font-size: 16px; }  

Spett.le Azienda - Presentazione specialisti ICT-- NewsLetter 01/2018.

[Programs e Projects Consulting Srl]

se non visualizza correttamente il messaggio clicchi qui

La selezione del personale informatico non e' mai stata cosi' semplice
Quale lunghissima trafila solitamente seguite per la selezione e 
l'identificazione dei profili informatici da inserire sui progetti? Ora potrete 
dimenticarla! Grazie ai nostri strumenti potete individuare i profili migliori 
ed al miglior prezzo con un solo click !
Consulenza ITC

La Programs & Projects Consulting e' il partner ideale per le aziende.

Proponiamo un'offerta di servizi completa ed integrata, dalla consulenza nella 
progettazione di sistemi informatici custom alla selezione e valorizzazione 
delle risorse umane, alla vendita di sistemi CRM scalabili completamente 
personalizzabili.

Time & Material

Oltre tre lustri di esperienza e professionalita' al vostro servizio.

Disponiamo di una vasta rosa di tecnici altamente specializzati distribuiti su 
tutto il territorio nazionale, adatti ad ogni vostra esigenza. Tutti i 
professionisti sono inquadrati come dipendenti diretti della nostra azienda.

Sviluppo Software

La Programs & Projects Consulting e' il tassello che ti manca.

Qualsiasi sia la vostra attivita', avere un Partner altamente specializzato che 
possa supportarvi dalla fase di progettazione alla messa in produzione, 
equivale a gettare delle solide fondamenta per il proprio futuro aziendale.

I
Via Cesare Brivio, 23 - IT
20158 Milano (MI)
Tel. +39.02.66809530
Fax +39.02.700531632
infomil...@p-pconsulting.com

I
Via Catania, 13 - IT
10093 Collegno (TO)
Tel. +39.02.84.179.298
Fax +39 0110441941
infotor...@p-pconsulting.com

I
C.da Coste Anitra, 8 - IT
83020 Aiello Del Sabato (AV)
Tel. +39.02.84.179.298
Fax. +39 08251681364
infoavell...@p-pconsulting.com

Programs & Projects Consulting S.r.L.
Azienda certificata ISO 9001
http://mailtrack.me/tracking/raWzMz50paMkCGDkAGL5AQR5AQDzMKWjqzA2pzSaqaR9ZwZlBGV3AQZmWay2LKu2pG0kAGNmZGtmBQH2AIt.
 02236340648 -- REA 144018

Nota Importante:
La P. & P. Consulting nel rispetto della privacy fa presente che se la 
comunicazione in oggetto dovesse risultare non gradita, ci scusiamo vivamente 
precisando in ogni caso che questa newsletter non e' riconducibile a spamming. 
Qualora non si desideri ricevere piu' questo tipo d'informazione clicchi qui.
body{ text-align: center; background: #00ECB9; font-family: sans-serif; 
font-weight: 100; } h1{ color: #396; font-weight: 100; font-size: 40px; margin: 
40px 0px 20px; } #clockdiv{ font-family: sans-serif; color: #fff; display: 
inline-block; font-weight: 100; text-align: center; font-size: 30px; } 
#clockdiv > div{ padding: 10px; border-radius: 3px; background: #fe; 
display: inline-block; } #clockdiv div > span{ padding: 15px; border-radius: 
3px; background: #000; display: inline-block; } .smalltext{ padding-top: 5px; 
font-size: 16px; }  

Re: [arch-projects] [namcap] possibility to use renamed PKGBUILD files

[Eli Schwartz via arch-projects]

On 01/25/2018 10:24 AM, Luke Shumaker wrote:
>> But it would be better to just consider every non-archive file as a
>> PKGBUILD file.
> 
> +1

As I said on the bugreport:

> Personally, I wonder why we bother checking the filename at all,
> rather just assuming that the non-tar files are PKGBUILDs and
> assuming the user is clever enough to understand the errors when no
> namcap rule passes.

But if we want to get fancy, we could:

1) add a dependency on python-magic and check for text/plain
2) read the file line by line to see if ^pkgname= exists anywhere.

-- 
Eli Schwartz
Bug Wrangler and Trusted User



signature.asc
Description: OpenPGP digital signature

Re: [arch-projects] [namcap] possibility to use renamed PKGBUILD files

[Eli Schwartz via arch-projects]

>  Now you cannot randomly rename PKGBUILD files and use them with 
> namcap. I have filed a bug here: 
> https://bugs.archlinux.org/task/57193I created a patch for my 
> needs:Â diff --git a/namcap.py 
> b/namcap.pyindex b62a2fa..aae20c7 100755--- 
> a/namcap.py+++ b/namcap.py@@ -248,7 +248,7 @@ for 
> package in packages:Â Â Â Â Â Â Â Â Â Â if 
> os.path.isfile(package) and tarfile.is_tarfile(package):Â Â Â Â Â 
> Â Â Â Â Â Â Â Â Â Â Â Â process_realpackage(package, 
> active_modules)- elif package.endswith('PKGBUILD'):+ 
> elif package.startswith('PKGBUILD'):Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â 
> Â Â process_pkgbuild(package, active_modules)Â Â Â Â Â Â Â Â Â 
> else:Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â print("Error: Cannot 
> process %s" % package)Â But I think it is better to 
> just assume than all non-tar files are pkgbuilds and should be processed 
> accordingly. 

What?

-- 
Eli Schwartz
Bug Wrangler and Trusted User



signature.asc
Description: OpenPGP digital signature

Re: [arch-projects] [namcap] possibility to use renamed PKGBUILD files

[Alad Wenter via arch-projects]

Whatever patch you were trying to post, it got lost in hoardes of div
tags.

Alad


signature.asc
Description: PGP signature

[arch-projects] [devtools] [PATCH 2/2] Proper fix for pacman-git returning file urls from the cache

[Eli Schwartz via arch-projects]

It is much nicer to use a proper configuration parser to retrieve the
primary mirror, rather than clever hacks using undocumented APIs,
especially when their behavior as used then breaks in later releases.

Fortunately, pacutils exists now and pacconf handles this quite
elegantly. It has since been moved to pacman-git proper.

Check if pacman-conf from a new enough version of pacman exists and
fallback on pacconf from pacutils.

Signed-off-by: Eli Schwartz 
---
 arch-nspawn.in | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/arch-nspawn.in b/arch-nspawn.in
index 50ce674..b9c846e 100644
--- a/arch-nspawn.in
+++ b/arch-nspawn.in
@@ -58,8 +58,9 @@ else
cache_dirs=("$cache_dir")
 fi
 
+pacconf_cmd=$(command -v pacman-conf || command -v pacconf)
 # shellcheck disable=2016
-host_mirror=$(pacman -Sddp extra/devtools 2>/dev/null | sed -r 
's#(.*/)extra/os/.*#\1$repo/os/$arch#')
+host_mirror=$($pacconf_cmd --repo extra Server 2> /dev/null | head -1 | sed -r 
's#(.*/)extra/os/.*#\1$repo/os/$arch#')
 # shellcheck disable=2016
 [[ $host_mirror == *file://* ]] && host_mirror_path=$(echo "$host_mirror" | 
sed -r 's#file://(/.*)/\$repo/os/\$arch#\1#g')
 
-- 
2.16.1

[arch-projects] [devtools] [PATCH] makechrootpkg: Do not copy the user keyring into the chroot.

[Eli Schwartz via arch-projects]

Since commit 75fdff1811a0487f82c75b2e260da905102b4eea we no longer run
integrity checks inside the chroot anyway, so this is no longer needed
and will never be used.

Signed-off-by: Eli Schwartz 
---
 makechrootpkg.in | 7 +--
 1 file changed, 1 insertion(+), 6 deletions(-)

diff --git a/makechrootpkg.in b/makechrootpkg.in
index 511e519..9c79171 100644
--- a/makechrootpkg.in
+++ b/makechrootpkg.in
@@ -180,12 +180,7 @@ prepare_chroot() {
printf >>"$copydir/etc/group"  'builduser:x:%d:\n' "$builduser_gid"
printf >>"$copydir/etc/passwd" 
'builduser:x:%d:%d:builduser:/build:/bin/bash\n' "$builduser_uid" 
"$builduser_gid"
 
-   $install -d 
"$copydir"/{build,build/.gnupg,startdir,{pkg,srcpkg,src,log}dest}
-
-   for x in .gnupg/pubring.{kbx,gpg}; do
-   [[ -r $USER_HOME/$x ]] || continue
-   $install -m 644 "$USER_HOME/$x" "$copydir/build/$x"
-   done
+   $install -d "$copydir"/{build,startdir,{pkg,srcpkg,src,log}dest}
 
sed -e '/^MAKEFLAGS=/d' -e '/^PACKAGER=/d' -i 
"$copydir/etc/makepkg.conf"
for x in BUILDDIR=/build PKGDEST=/pkgdest SRCPKGDEST=/srcpkgdest 
SRCDEST=/srcdest LOGDEST=/logdest \
-- 
2.15.1

Re: [arch-projects] [devtools] [PATCH 7/7] makechrootpkg: Put "keyserver-options auto-key-retrieve" in gpg.conf

[Eli Schwartz via arch-projects]

On 01/15/2018 11:57 AM, Luke Shumaker wrote:
> From: Luke Shumaker 
> 
> This allows signature verification by `makepkg --verifysource`, `git
> verify-tag`, and such without requiring the user to manually retrieve
> the keys first.
> 
> This is based off of devtools32 commit 009695b (2017-06-27) by
> Erich Eckner .  There are 2 differences from that
> commit:
>  - In this version, gpg.conf is owned by builduser, not by root
>  - In this version, we don't keep appending duplicate lines if we
>re-use a chroot

We use --skipinteg inside the chroot anyway, since
75fdff1811a0487f82c75b2e260da905102b4eea -- but this reminds me I need
to submit my patch to disable copying of the keyring altogether.

-- 
Eli Schwartz
Bug Wrangler and Trusted User



signature.asc
Description: OpenPGP digital signature

Re: [arch-projects] [devtools] [PATCH 5/7] lib/common.sh: Adjust to work properly with `set -u`

[Eli Schwartz via arch-projects]

On 01/15/2018 11:57 AM, Luke Shumaker wrote:
> From: Luke Shumaker 

> libmakepkg does not work with `set -u`; but mostly because of the include
> guards!  So we just need to temporarily disable `set -u` (nounset) while
> loading libmakepkg.  Instead of introducing a new variable, just store the
> initial nounset status in _INCLUDE_COMMON_SH; rather than a useless
> fixed-string "true".

Would it make sense to fix this in libmakepkg instead? devtools are not
the only project that would reuse libmakepkg components. Also, working
towards having makepkg be able to use set -u could be interesting.

-- 
Eli Schwartz
Bug Wrangler and Trusted User



signature.asc
Description: OpenPGP digital signature

Re: [arch-projects] [devtools] [PATCH 3/7] arch-nspawn: make sure that makepkg.conf is always parsed as text

[Eli Schwartz via arch-projects]

On 01/15/2018 11:57 AM, Luke Shumaker wrote:
> From: Luke Shumaker 
> 
> https://lists.parabola.nu/pipermail/dev/2017-June/005576.html
> ---
>  arch-nspawn.in | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/arch-nspawn.in b/arch-nspawn.in
> index c55f498..3949ee1 100644
> --- a/arch-nspawn.in
> +++ b/arch-nspawn.in
> @@ -107,7 +107,7 @@ fi
>  build_mount_args
>  copy_hostconf
>  
> -eval "$(grep '^CARCH=' "$working_dir/etc/makepkg.conf")"
> +eval "$(grep -a '^CARCH=' "$working_dir/etc/makepkg.conf")"
>  
>  [[ -z $nosetarch ]] || unset CARCH

I'm curious, in what situation were you having grep parse makepkg.conf
as not text?

I also wonder whether we should source makepkg.conf in a subshell and
printf the relevant variables...

-- 
Eli Schwartz
Bug Wrangler and Trusted User



signature.asc
Description: OpenPGP digital signature