Re: [l10n-sync] Updating sync script on git migration

[Holger Wansing]

Hi,

Cyril Brulebois  wrote:
> Hi,
> 
> Holger Wansing  (2018-06-12):
> > Cyril Brulebois  wrote:
> > > I think it would make sense to have you in the d-i gid, so that you can
> > > access dillon, and re-enable crontab entries, maybe subscribing to cron
> > > mails? Of course I could re-enable those crontab entries myself but I
> > > won't be able to fix any possible fallouts in the upcoming days.
> > 
> > 1.
> > I just subscribed to cron mails.
> 
> I've seen your commit but this isn't live yet. ;)

Now that I have access to dillon, they _are_live_ :-)

I have also changed the number of sublevels for l10n-sync to 5, which is the
current amount of sublevels.

> > 2.
> > Above you mention "re-enable cron entries". However, I cannot find any such
> > disabled entries.
> > I assume the l10n-sync process is currently enabled, or am I missing 
> > something?
> > (I'm planning to test the modified script via some weblate commits, that's
> > why.)

I also changed some lines from svn to git
(./daily-build-aggregator and ./daily-build-manager and ./daily-build-overview)


Holger

> Modifying the git repository isn't sufficient (as we might want to tweak
> the crontab entries without a commit; at least we've been doing that
> from time to time). We would need to update the copy on dillon in d-i's
> home directory; switching it from svn to git in the process. I can't
> really commit to doing so right away though. Note there's “make check”
> and “make install” available thanks to scripts/Makefile, to list the
> diff between what's in dillon's crontab and what's in $VCS.
> 
> > 3.
> > While logged into dillon, I found an old example-preseed.txt file under
> > /srv/d-i.debian.org/www/
> > which is an outdated file from squeeze. I tried to remove it, but lacking
> > permission.
> > Adding myself to d-i gid is missing here, probably?
> 
> Right. You should get at least this:
> | User kibi may run the following commands on dillon:
> | (d-i) ALL
> 
> which will let you do things like “sudo -u d-i -s”, which means working
> as the d-i user on dillon.
> 
> For this you'll likely need to open a ticket on RT:
>   https://wiki.debian.org/Teams/DSA/RTUsage
> 
> Feel free to cc me when requesting getting added to the d-i group, I'll
> confirm with a reply (which I think was kind of needed/appreciated last
> time I've been in that process).
> 
> Thanks for joining!
> 
> 
> Cheers,
> -- 
> Cyril Brulebois (k...@debian.org)
> D-I release manager -- Release team member -- Freelance Consultant


-- 

Created with Sylpheed 3.5.1 under 
D E B I A N   L I N U X   9   " S T R E T C H " .

Registered Linux User #311290 - https://linuxcounter.net/

Re: [l10n-sync] Updating sync script on git migration

[Cyril Brulebois]

Cyril Brulebois  (2018-06-12):
> Feel free to cc me when requesting getting added to the d-i group, I'll
> confirm with a reply (which I think was kind of needed/appreciated last
> time I've been in that process).

The ticket I opened (#7319) has been resolved:

$ ssh dillon.debian.org groups holgerw
holgerw : holgerw Debian d-i

(RT wants inline signatures so your initial mail was probably discarded;
it seems my reply is what triggered a ticket number being assigned.)


Cheers,
-- 
Cyril Brulebois (k...@debian.org)
D-I release manager -- Release team member -- Freelance Consultant


signature.asc
Description: PGP signature

Re: [l10n-sync] Updating sync script on git migration

[Cyril Brulebois]

Hi,

Holger Wansing  (2018-06-12):
> Cyril Brulebois  wrote:
> > I think it would make sense to have you in the d-i gid, so that you can
> > access dillon, and re-enable crontab entries, maybe subscribing to cron
> > mails? Of course I could re-enable those crontab entries myself but I
> > won't be able to fix any possible fallouts in the upcoming days.
> 
> 1.
> I just subscribed to cron mails.

I've seen your commit but this isn't live yet. ;)

> 2.
> Above you mention "re-enable cron entries". However, I cannot find any such
> disabled entries.
> I assume the l10n-sync process is currently enabled, or am I missing 
> something?
> (I'm planning to test the modified script via some weblate commits, that's
> why.)

Modifying the git repository isn't sufficient (as we might want to tweak
the crontab entries without a commit; at least we've been doing that
from time to time). We would need to update the copy on dillon in d-i's
home directory; switching it from svn to git in the process. I can't
really commit to doing so right away though. Note there's “make check”
and “make install” available thanks to scripts/Makefile, to list the
diff between what's in dillon's crontab and what's in $VCS.

> 3.
> While logged into dillon, I found an old example-preseed.txt file under
> /srv/d-i.debian.org/www/
> which is an outdated file from squeeze. I tried to remove it, but lacking
> permission.
> Adding myself to d-i gid is missing here, probably?

Right. You should get at least this:
| User kibi may run the following commands on dillon:
| (d-i) ALL

which will let you do things like “sudo -u d-i -s”, which means working
as the d-i user on dillon.

For this you'll likely need to open a ticket on RT:
  https://wiki.debian.org/Teams/DSA/RTUsage

Feel free to cc me when requesting getting added to the d-i group, I'll
confirm with a reply (which I think was kind of needed/appreciated last
time I've been in that process).

Thanks for joining!


Cheers,
-- 
Cyril Brulebois (k...@debian.org)
D-I release manager -- Release team member -- Freelance Consultant


signature.asc
Description: PGP signature

Re: [l10n-sync] Updating sync script on git migration

[Holger Wansing]

Hi,

Cyril Brulebois  wrote:
> I think it would make sense to have you in the d-i gid, so that you can
> access dillon, and re-enable crontab entries, maybe subscribing to cron
> mails? Of course I could re-enable those crontab entries myself but I
> won't be able to fix any possible fallouts in the upcoming days.

1.
I just subscribed to cron mails.

2.
Above you mention "re-enable cron entries". However, I cannot find any such
disabled entries.
I assume the l10n-sync process is currently enabled, or am I missing something?
(I'm planning to test the modified script via some weblate commits, that's
why.)

3.
While logged into dillon, I found an old example-preseed.txt file under
/srv/d-i.debian.org/www/
which is an outdated file from squeeze. I tried to remove it, but lacking
permission.
Adding myself to d-i gid is missing here, probably?


Holger

-- 

Created with Sylpheed 3.5.1 under 
D E B I A N   L I N U X   9   " S T R E T C H " .

Registered Linux User #311290 - https://linuxcounter.net/

Re: [l10n-sync] Updating sync script on git migration

[Christian PERRIER]

Quoting Holger Wansing (li...@wansing-online.de):

> That would leave us with a gap on l10n uploads for d-i
> packages. 
> 
> So, maybe it's time for tuning my DD account to uploading DD:-)
> While being unsure, if that's doable for me, since I am not a
> programmer ...


I am not one either. I have a few skills at general script
programming, I can understand basic code and I have a few skills in
system administration.

Which was indeed enough for doing, I guess, quite decent job in
maintaining packages. The more important part is certainly having a
good knowledge of the Debian ecosystem, good knowledge of one's own
limits for technical work.and enough will to ask for help when
you're not confident in actions to take.




signature.asc
Description: PGP signature

Re: [l10n-sync] Updating sync script on git migration

[Holger Wansing]

Hi,

Am Montag, 4. Juni 2018 schrieb Christian PERRIER:
> Quoting Cyril Brulebois (k...@debian.org):
> 
> > I think it would make sense to have you in the d-i gid, so that you can
> > access dillon, and re-enable crontab entries, maybe subscribing to cron
> > mails? Of course I could re-enable those crontab entries myself but I
> > won't be able to fix any possible fallouts in the upcoming days.
> > 
> > What do you think?

Yes, seems reasonable.
But since I'm only non-uploading DD, I don't have any  
experience in the relevant processes, which means I will be
asking for help sometimes :-) 
 
> And, as explained in private mail to Cyril, it would make sense to
> have me.out of the d-i gid, given my lowered activity nowadays
> (the swith to Salsa did ring the final bell, I think).
> 
> It also makes sense to drop my account from the installer team on
> Salsa (or whatever it is called there) whihch will save me from commit
> notifications...
> 
> Dropping me from Uploaders is also a good idea (I never found time to
> do that while we were still on Alioth).

Sadly hearing that...

That would leave us with a gap on l10n uploads for d-i
packages. 

So, maybe it's time for tuning my DD account to uploading DD:-)
While being unsure, if that's doable for me, since I am not a
programmer ...

So far 
Holger

-- 
Sent from my Jolla phone
http://www.jolla.com/

Re: [l10n-sync] Updating sync script on git migration

[Christian PERRIER]

Quoting Cyril Brulebois (k...@debian.org):

> I think it would make sense to have you in the d-i gid, so that you can
> access dillon, and re-enable crontab entries, maybe subscribing to cron
> mails? Of course I could re-enable those crontab entries myself but I
> won't be able to fix any possible fallouts in the upcoming days.
> 
> What do you think?


And, as explained in private mail to Cyril, it would make sense to
have me.out of the d-i gid, given my lowered activity nowadays
(the swith to Salsa did ring the final bell, I think).

It also makes sense to drop my account from the installer team on
Salsa (or whatever it is called there) whihch will save me from commit
notifications...

Dropping me from Uploaders is also a good idea (I never found time to
do that while we were still on Alioth).

As I explained to several people already, this is part of /me
resigning from Debian one day or another, given that my priorities in
life have switched gradually.

I'll reamin subscribed to -boot so please feel free to  ask for help
and advice, particularly in this D-I l10n world, which we started to
build about 15 years ago. (I think that my first commits to D-I
SVN (or was it CVS?) go back to 2003 or so)



signature.asc
Description: PGP signature

Re: [l10n-sync] Updating sync script on git migration

[Cyril Brulebois]

Hi,

Holger Wansing  (2018-06-03):
> Hmm, maybe the whole file is even better für reviewing.
> Here it is (gzipped for file size).

Danke für das ganze File. ;)

FWIW, I suppose you could have pushed a branch with a commit or two, but
I understand that you've already put efforts into that git transition
anyway. :)


Regarding your question for the duplicated line, there use to be the
concept of big commits and atomic commits, and simplifications of those
use cases over time led to a leftover “cd” call.

The rest looks good to me so I've committed this (amending a bit) and
pushed to git master:
  
https://salsa.debian.org/installer-team/d-i/commit/cb0acca6990c60cd67aaac479372eadfa57f27e2
  
https://salsa.debian.org/installer-team/d-i/commit/3bd7ecb355795262c7db8ea62c7c52c67c4e9e29


I think it would make sense to have you in the d-i gid, so that you can
access dillon, and re-enable crontab entries, maybe subscribing to cron
mails? Of course I could re-enable those crontab entries myself but I
won't be able to fix any possible fallouts in the upcoming days.

What do you think?


Cheers,
-- 
Cyril Brulebois (k...@debian.org)
D-I release manager -- Release team member -- Freelance Consultant


signature.asc
Description: PGP signature

Re: [l10n-sync] Updating sync script on git migration

[Holger Wansing]

Hi,

Holger Wansing  wrote:
> Hi,
> 
> I have prepared the modification for the l10n-sync script, 
> please review.

Hmm, maybe the whole file is even better für reviewing.
Here it is (gzipped for file size).


Holger

-- 

Created with Sylpheed 3.5.1 under 
D E B I A N   L I N U X   9   " S T R E T C H " .

Registered Linux User #311290 - https://linuxcounter.net/



l10n-sync_workingcopy.gz
Description: application/gzip

[l10n-sync] Updating sync script on git migration

[Holger Wansing]

Hi,

I have prepared the modification for the l10n-sync script, 
please review.


Holger

-- 

Created with Sylpheed 3.5.1 under 
D E B I A N   L I N U X   9   " S T R E T C H " .

Registered Linux User #311290 - https://linuxcounter.net/

--- l10n-sync	2018-06-03 14:07:10.583896235 +0200
+++ l10n-sync_workingcopy	2018-06-03 15:35:38.039508152 +0200
@@ -7,6 +7,14 @@
 # Translators, DO NOT RUN THIS SCRIPT YOURSELVES
 #
 
+# Declaration:
+# This process has two parts:
+#   1. the packages/po directory, where translators are working in;
+#   2. the around 97 directories for the different packages in d-i.
+# In the past, part 1 was in svn, while part 2 was git.
+# Now, both are git repos, therefore I changed "svn" into "gitpo".
+#
+
 export LC_ALL=en_US.UTF-8
 
 COMMIT_MARKER="[l10n] "
@@ -20,7 +28,6 @@
 UPDATEPO=Y
 SYNCPKGS=Y
 QUIET=N
-svn=svn
 git=git
 debconfupdatepo=debconf-updatepo
 
@@ -32,7 +39,7 @@
 
 usage() {
 echo  "Usage:"
-echo  "$0 [--online] [--keep-revision lang] [--atomic] [--atomic-updates] [--commit] [--levels="number_of_levels"] [--svn="path_to_svn"] [--git="path_to_git"] [--debconf-updatepo="debconf-updatepo command"] [--quiet|--nolog] "
+echo  "$0 [--online] [--keep-revision lang] [--atomic] [--atomic-updates] [--commit] [--levels="number_of_levels"] [--git="path_to_git"] [--debconf-updatepo="debconf-updatepo command"] [--quiet|--nolog] "
 echo  "--online: Work online (will update the local copy on"
 echo  "  the fly)"
 echo  "--commit: Commit changed files (implies --online)"
@@ -46,7 +53,6 @@
 echo  "  meant for use when switching a language"
 echo  "  NOT RECOMMENDED in other situations"
 echo  "--levels: number of levels"
-echo  "--svn   : path to the svn binary"
 echo  "--git   : path to the git binary"
 echo  "--debconf-updatepo  : debconf-updatepo command line"
 echo  "  (for using hacked debconf-updatepo)"
@@ -58,18 +64,18 @@
 echo  " : path to the local copy of the D-I repository"
 }
 
-svnerr() {
-	echo "Error in a SVN operation, please investigate"
-	echo "Trying to cleanup locks..."
-	if "$DI_COPY" ;  then
-		cd $DI_COPY
-		$svn cleanup
-	fi
+gitpoerr() {
+	echo "Error in a GIT operation at packages/po dir, please investigate"
+#	echo "Trying to cleanup locks..."
+#	if "$DI_COPY" ;  then
+#		cd $DI_COPY
+#		$svn cleanup
+#	fi
 	exit 1
 }
 
 giterr() {
-	echo "Error in a GIT operation, please investigate"
+	echo "Error in a GIT operation in one of the package dirs, please investigate"
 	exit 2
 }
 
@@ -254,9 +260,6 @@
 	"--levels="*)
 		NUMLEVELS=`echo $1|cut -f2 -d=`
 		;;
-	"--svn="*)
-		svn=`echo $1|cut -f2 -d=`
-		;;
 	"--git="*)
 		git=`echo $1|cut -f2 -d=`
 		;;
@@ -320,11 +323,12 @@
 	exit 1
 fi
 
-if [ ! -d $DI_COPY/.svn ] ; then
-	error "No $DI_COPY/.svn directory found"
-	error "$DI_COPY may not be a copy of Debian Installer SVN repository"
-	exit 1
-fi
+# The packages/po dir is no longer a repository for it's own now.
+#if [ ! -d $DI_COPY/.svn ] ; then
+#	error "No $DI_COPY/.svn directory found"
+#	error "$DI_COPY may not be a copy of Debian Installer SVN repository"
+#	exit 1
+#fi
 
 if [ -n "$MERGEDIR" ] ; then
 	if [ ! -d "$MERGEDIR" ] ; then
@@ -346,8 +350,9 @@
 done
 
 # Do not accept working on an unclean copy
-if $(svn st $DI_COPY/packages/po | grep -q "^C") ; then
-	error "$DI_COPY seems to contain some SVN conflict files"
+if $($git status -s -uno $DI_COPY/packages/po) ; then
+	error "$DI_COPY seems to contain some GIT conflict files"
+	error "in packages/po."
 	error "Please fix this before launching the script again"
 	exit 1
 fi
@@ -360,7 +365,7 @@
 cd $DI_COPY/packages/po
 if [ "$ONLINE" = "Y" ] ; then
 	log_cmd -p "Synchronize $DI_COPY/packages..." \
-		$svn update || svnerr
+		$git pull || gitpoerr
 fi
 
 # Check the packages/po/run-l10n-sync file
@@ -401,22 +406,26 @@
 #cd $DI_COPY
 #if [ "$ONLINE" = "Y" ] ; then
 #	log_cmd -p "Synchronize $DI_COPY/packages..." \
-#		$svn update || svnerr
+#		$svn update || gitpoerr
 #fi
 
 # In case a merge has to be done with another directory
 # we update this directory as well
-if [ -n "$MERGEDIR" ] ; then
-	cd $MERGEDIR/packages/po
-	if [ "$ONLINE" = "Y" ] ; then
-		log_cmd -p "Synchronize the merge directory $MERGEDIR/packages/po..." \
-			$svn update || svnerr
-	fi
-fi
+#
+# !!! There is no need to do this anymore, since now we have everything in one repo.
+#
+#if [ -n "$MERGEDIR" ] ; then
+#	cd $MERGEDIR/packages/po
+#	if [ "$ONLINE" = "Y" ] ; then
+#		log_cmd -p "Synchronize the merge directory $MERGEDIR/packages/po..." \
+#			$svn update || gitpoerr
+#	fi
+#fi
 
 # Let's check the thing againceinture et bretelles as we say in French
-if $(svn st $DI_COPY/packages/po | grep -q "^C") ; then
-	error