Re: umask - default user settings?

[Max Nikulin]

On 18/07/2024 00:01, Greg Wooledge wrote:

On Wed, Jul 17, 2024 at 17:58:57 +0100, Tim Woodall wrote:

No, I'm talking about sudo, not su. I'm not a sudo user so I can't test
but my understanding is that root inherits the umask of the invoking
user (or it used to)


Looks like this is still true.

hobbit:~$ umask 077

[...]

hobbit:~$ sudo bash -c umask
0077


Partially it is the same issue as with su when pam_umask.so is missed in 
common-session. When it is present, the value from login.defs is 
respected for "sudo -i". However I am in doubts if it is proper 
behavior. For administrative tasks 022 is more reasonable default even 
if something else is configured for regular users.


sudoers(5) describes "umask_override" and "umask" settings. It seems 
changing default umask requires modification of these preferences.

Re: umask - default user settings?

[Max Nikulin]

On 19/07/2024 10:45, songbird wrote:

- Does MATE use scopes and services to run applications an components?
"ps xwf" and "systemd-cgls" trees may clarify where started applications
appear.


   neither of those show all the programs that i have
included on the panels, but there are cgroups and some
of the other programs listed.


The question is what are parents of processes started from menus, 
desktop icons, launchers, etc. in both trees. The special case is 
flatpak/snap sandboxes.


Maybe you will find missed applications in output of "ps axuwf" and 
"systemd-cgls" executed as root. Sometimes it is not easy to associate 
process names and user friendly names used in desktop environment. You 
may try to grep in /usr/share/applications for names and "Exec".

Re: umask - default user settings?

[Greg Wooledge]

On Fri, Jul 19, 2024 at 23:04:25 +0700, Max Nikulin wrote:
> On 16/07/2024 20:46, Greg Wooledge wrote:
> > On Mon, Jul 15, 2024 at 23:39:54 -0400, Greg Wooledge wrote:
> > > Now we just need for GNOME users to discover a way to configure the
> > > programs that are started as children of dbus, and then we can move
> > > forward.  Documentation would be my top priority.  If other people want
> > > to try to drum up interest in environment configuration, then there'll
> > > be documentation available for end users to follow.
> > 
> > I've added a bit of content to 
> > <https://wiki.debian.org/EnvironmentVariables>.
> 
> Isn't the following a more suitable article for umask?
> <https://wiki.debian.org/Permissions#Setting_default_umask>
> I think, it is better to drop UMask note from EnvironmentVariables.

I've made some minor changes to both pages.

Re: umask - default user settings?

[Max Nikulin]

On 16/07/2024 20:46, Greg Wooledge wrote:

On Mon, Jul 15, 2024 at 23:39:54 -0400, Greg Wooledge wrote:

Now we just need for GNOME users to discover a way to configure the
programs that are started as children of dbus, and then we can move
forward.  Documentation would be my top priority.  If other people want
to try to drum up interest in environment configuration, then there'll
be documentation available for end users to follow.


I've added a bit of content to <https://wiki.debian.org/EnvironmentVariables>.


Isn't the following a more suitable article for umask?
<https://wiki.debian.org/Permissions#Setting_default_umask>
I think, it is better to drop UMask note from EnvironmentVariables.

Re: umask - default user settings?

[songbird]

Max Nikulin wrote:
> On 19/07/2024 04:11, songbird wrote:
>>so far, agreed, i poked at it a bit the other day to see
>> if MATE would work with the roughly (user-@1000,etc) systemd
>> unit approach but that didn't accomplish anything i could tell.
>
> It would be great if those, who tried it, reported more precise what 
> they did.
>
> - Did you call "systemctl daemon-reload" for *system* instance after 
> adding a drop-in for user@1000.service?
> - Did you do logout after in the case of *user* service.d override? It 
> is not enough to execute "systemd --user daemon-reload".
> - Does MATE use scopes and services to run applications an components? 
> "ps xwf" and "systemd-cgls" trees may clarify where started applications 
> appear.

  neither of those show all the programs that i have
included on the panels, but there are cgroups and some
of the other programs listed.

  the missing programs are indeed being started since i
use them all the time.


  songbird

Re: umask - default user settings?

[Max Nikulin]

On 19/07/2024 04:11, songbird wrote:

   so far, agreed, i poked at it a bit the other day to see
if MATE would work with the roughly (user-@1000,etc) systemd
unit approach but that didn't accomplish anything i could tell.


It would be great if those, who tried it, reported more precise what 
they did.


- Did you call "systemctl daemon-reload" for *system* instance after 
adding a drop-in for user@1000.service?
- Did you do logout after in the case of *user* service.d override? It 
is not enough to execute "systemd --user daemon-reload".
- Does MATE use scopes and services to run applications an components? 
"ps xwf" and "systemd-cgls" trees may clarify where started applications 
appear.

Re: umask - default user settings?

[songbird]

Greg Wooledge wrote:
...
> It only becomes *hard* when Desktop Environments are introduced into the
> picture.

  so far, agreed, i poked at it a bit the other day to see
if MATE would work with the roughly (user-@1000,etc) systemd
unit approach but that didn't accomplish anything i could tell.

  no more time this week for such journeys and
exploritations...  but i'll continue reading along as i get
time as the topic is interesting and convoluted which are
the best sort of puzzles.  :)


  songbird

Re: umask - default user settings?

[Greg Wooledge]

On Thu, Jul 18, 2024 at 09:07:48 +0700, Max Nikulin wrote:
> Taking into account a number of bugs, perhaps it is not really bad that
> recipes how to change umask are not easily available. Documentation should
> be extensive enough to describe possible pitfalls.

That's an odd stance, especially if you consider that changing your umask
used to be literally a one-line change.  It still is, if you only use
the shell (locally or remotely).  It still is, if you use a shell login
plus startx plus a traditional window manager.

Even with a Display Manager login and a traditional WM, there are still
just two places you need to change.

It only becomes *hard* when Desktop Environments are introduced into the
picture.

Re: umask - default user settings?

[Max Nikulin]

On 17/07/2024 22:40, Greg Wooledge wrote:

hobbit:/etc/pam.d$ dpkg -S /etc/pam.d/common-session
dpkg-query: no path found matching pattern /etc/pam.d/common-session

Where does that file come from, then?


This file contains the following:


# As of pam 1.0.1-6, this file is managed by pam-auth-update by default.


The tool has a brief man page. Files in pam.d need update when packages 
with pam modules are installed or removed. It seems similar to some 
other config file only a part of it is managed by a tool.


Taking into account a number of bugs, perhaps it is not really bad that 
recipes how to change umask are not easily available. Documentation 
should be extensive enough to describe possible pitfalls.

Re: umask - default user settings?

[Greg Wooledge]

On Wed, Jul 17, 2024 at 20:51:40 +0200, Franco Martelli wrote:
> If you plan to add your contribute to the wiki page (see above) in the
> section: "Desktop Environments and systemd user services" e.g.:
> 
> - ...
> - systemctl --user daemon-reload
> - /Restart your Desktop session/
> 
> Please consider also to specify the Desktop you use, because newbies don't
> know nothing about: "Some Desktop Environments launch programs via systemd
> user services." Which Desktop?

Well, that's the problem, isn't it?  *We* don't know either, because
most of us don't use one!

I'm doing the best I can trying to document these systems that I don't
use personally.  It would be *nice* if their developers would document
them, so we wouldn't have to reverse engineer how they work and come up
with our own workarounds.  It's much, much harder when you can't even
reverse engineer one yourself, and instead have to rely on the reports
of other users.

Re: umask - default user settings?

[Franco Martelli]

On 16/07/24 at 15:46, Greg Wooledge wrote:

I've added a bit of content to .



On 17/07/24 at 04:37, Max Nikulin wrote:
daemon-reload is not enough in KDE. krunner and plasmashell services 
have been started already, so changes would not apply despite they are 
visible in "systemctl --user show-environment". Perhaps launchers just 
do not listen for a proper D-Bus signal emitted by systemd user daemon.

The same applies for environment.d.

So logout is necessary.


If you plan to add your contribute to the wiki page (see above) in the 
section: "Desktop Environments and systemd user services" e.g.:


- ...
- systemctl --user daemon-reload
- /Restart your Desktop session/

Please consider also to specify the Desktop you use, because newbies 
don't know nothing about: "Some Desktop Environments launch programs via 
systemd user services." Which Desktop? IMO it's better to add /(like 
KDE)/ and changing the statement in this way: "Some Desktop Environments 
(like KDE) launch programs via systemd user services."


Just my 2¢ tips, thanks
--
Franco Martelli

Re: umask - default user settings?

[Greg Wooledge]

On Wed, Jul 17, 2024 at 17:58:57 +0100, Tim Woodall wrote:
> No, I'm talking about sudo, not su. I'm not a sudo user so I can't test
> but my understanding is that root inherits the umask of the invoking
> user (or it used to)

Looks like this is still true.

hobbit:~$ bash
hobbit:~$ umask 077
hobbit:~$ umask
0077
hobbit:~$ sudo -s
[sudo] password for greg: 
root@hobbit:/home/greg# umask
0077
root@hobbit:/home/greg# exit
exit
hobbit:~$ sudo -i
root@hobbit:~# umask
0077
root@hobbit:~# exit
logout
hobbit:~$ sudo bash -c umask
0077
hobbit:~$ umask 022
hobbit:~$ sudo bash -c umask
0022

Re: umask - default user settings?

[Tim Woodall]

On Wed, 17 Jul 2024, Max Nikulin wrote:


On 17/07/2024 15:37, Tim Woodall wrote:

umask 077 can come with its own problems when using shared directories.


<https://wiki.debian.org/UserPrivateGroups>

Taking into account old 022 vs. 002 discussions it might be 007.


I'm not a sudo user but IIUC, root inherits the umask, which can then
cause problems when things can't read config files that should be world
readable.


Do you mean the following bug or something else?


No, I'm talking about sudo, not su. I'm not a sudo user so I can't test
but my understanding is that root inherits the umask of the invoking
user (or it used to)

Re: umask - default user settings?

[Max Nikulin]

On 15/07/2024 09:15, Alan D. Salewski wrote:
I suspect that most people /do/ change it, once they become aware of it, 
for the very reason stated in the comment above 'UMASK' in the 
/etc/login.defs file:



     # UMASK is the default umask value for pam_umask and is used by
     # useradd and newusers to set the mode of the new home directories.
     # 022 is the "historical" value in Debian for UMASK
     # 027, or even 077, could be considered better for privacy
     # There is no One True Answer here : each sysadmin must make up 
his/her

     # mind.
     ...
     UMASK   022



Have you tried to change it?

<https://bugs.debian.org/583958>
<https://bugs.debian.org/646692>
pam_umask: umask in /etc/login.defs not respected cause libpam_umask is 
not configured

Fixed in versions pam/1.5.3-1, pam/1.5.3-4
Wed, 28 Feb 2024 05:31:34 +

Not to mention systemd user sessions discussed in another branch of this 
thread.

Re: umask - default user settings?

[Greg Wooledge]

On Wed, Jul 17, 2024 at 22:10:28 +0700, Max Nikulin wrote:
> Do you mean the following bug or something else?
> <https://bugs.debian.org/711104>
> login: su - doesn't set umask
> Fixed in version pam/1.5.3-1
> Tue, 16 Jan 2024 00:19:23 +

Huh... given the age of the bug, I expected this was something already
done in bookworm, but it's not.

hobbit:/etc/pam.d$ grep -i umask *
hobbit:/etc/pam.d$ grep -i mask *
hobbit:/etc/pam.d$ 

Bookworm has PAM package version 1.5.2-6+deb12u1, not 1.5.3.  Looks like
this change was only made this year, and therefore won't appear until
Debian 13.

This makes me wonder what's setting umask *now*.  Is it still PAM, just
using a compile-time default instead of a value that's discoverable in
a conffile?

Also, this confused me:

hobbit:/etc/pam.d$ dpkg -S /etc/pam.d/common-session
dpkg-query: no path found matching pattern /etc/pam.d/common-session

Where does that file come from, then?  The installer?  Oh wait, there are
postinst scripts

hobbit:~$ grep common-session /var/lib/dpkg/info/*.postinst
/var/lib/dpkg/info/libpam-runtime.postinst: for configfile in common-auth 
common-account common-session  \
/var/lib/dpkg/info/libpam-runtime.postinst:   
"$DPKG_ROOT"/etc/pam.d/common-session.pam-old

So I guess that file comes from libpam-runtime, but it's not managed
like a regular conffile.  I suppose there was some reason for this, even
if I can't guess it.

Re: umask - default user settings?

[Max Nikulin]

On 17/07/2024 15:37, Tim Woodall wrote:

umask 077 can come with its own problems when using shared directories.


<https://wiki.debian.org/UserPrivateGroups>

Taking into account old 022 vs. 002 discussions it might be 007.


I'm not a sudo user but IIUC, root inherits the umask, which can then
cause problems when things can't read config files that should be world
readable.


Do you mean the following bug or something else?
<https://bugs.debian.org/711104>
login: su - doesn't set umask
Fixed in version pam/1.5.3-1
Tue, 16 Jan 2024 00:19:23 +


Rather than change umask, I'd suggest that the better change is to make
home directories 0700 by default.


The topic starter believes it is not enough
<https://lists.debian.org/msgid-search/860527137.0ifERbkFSE@protheus2>
Mon, 15 Jul 2024 09:04:54 +0200

However I would not mind to read more details what use cases are not 
covered. /tmp?

Re: umask - default user settings?

[Tim Woodall]

On Mon, 15 Jul 2024, Jeffrey Walton wrote:



Debian is a multi-user operating system. Decisions should be made accordingly.

I suppose umask is a moot point on phones and tablets, where
single-user is often the use case.



umask 077 can come with its own problems when using shared directories.

years ago I used to use cvs pserver specifically to finesse this
problem. Now that (almost) everybody uses a remote git server it's less
relevant there.

I'm not a sudo user but IIUC, root inherits the umask, which can then
cause problems when things can't read config files that should be world
readable.

Rather than change umask, I'd suggest that the better change is to make
home directories 0700 by default. If that is the wrong choice then it
only has to be fixed once per user. Creating 'world/group' readable
files with too restrictive permissions never goes away.

Re: umask - default user settings?

[Max Nikulin]

On 16/07/2024 10:39, Greg Wooledge wrote:

hobbit:~$ cat .config/systemd/user/service.d/env.conf
[Service]
Environment="FOO=%h/test123" "BAR=b a r"
hobbit:~$ systemctl --user daemon-reload
hobbit:~$ systemctl --user start xterm.service


daemon-reload is not enough in KDE. krunner and plasmashell services 
have been started already, so changes would not apply despite they are 
visible in "systemctl --user show-environment". Perhaps launchers just 
do not listen for a proper D-Bus signal emitted by systemd user daemon.

The same applies for environment.d.

So logout is necessary.

I have unsuccessfully tried to find a way to apply environment.d without 
logout. I do not see anything relevant in KDE system settings. 
 suggests 
$HOME/.config/plasma-workspace/env/path.sh that are autostart scripts, 
so runtime changes are not applied.


SSH login may cause starting of some systemd user services:

948 pts/0Ss 0:00  \_ -bash
   1017 pts/0R+ 0:00  \_ ps xwf
921 ?Ss 0:00 /lib/systemd/systemd --user
927 ?S  0:00  \_ (sd-pam)
942 ?Ssl0:00  \_ /usr/bin/pulseaudio --daemonize=no 
--log-target=journal
984 ?Ss 0:00  \_ /usr/bin/dbus-daemon --session 
--address=systemd: --nofork --nopidfile --systemd-activation --syslog-only


An additional consideration concerning applications started through 
D-Bus: I think, they are started through systemd nowadays. At first I 
forgot that enough .desktop files prescribe starting through D-Bus 
ignoring Exec= field.


/usr/share/applications/org.kde.kwrite.desktop:X-DBUS-StartupType=Multi
/usr/share/applications/org.kde.kwrite.desktop:X-DBUS-ServiceName=org.kde.kwrite

Re: umask - default user settings?

[Dan Purgert]

On Jul 16, 2024, Thomas Schmitt wrote:
> Hi,
> 
> to...@tuxteam.de wrote:
> > Somehow I'm glad I stayed away from DEs and systemd up to now. Perhaps I
> > just retire before the alternatives aren't viable anymore. Or perhaps, as
> > with PulseAudio, I can leapfrog that "tech".
> 
> Retirement is no solution.
> What shall we retirees do when X11 is laid to rest ?
> I am not aware of anything like fvwm running on Wayland.

Use tty1 with screen/tmux? :D

-- 
|_|O|_| 
|_|_|O| Github: https://github.com/dpurgert
|O|O|O| PGP: DDAB 23FB 19FA 7D85 1CC1  E067 6D65 70E5 4CE7 2860


signature.asc
Description: PGP signature

Re: umask - default user settings?

[debian-user]

Darac Marjal  wrote:
> I'm not saying that what you did was wrong, but systemd provides a
> few shortcuts which can make things a bit more user-friendly.
> 
> On 16/07/2024 04:39, Greg Wooledge wrote:
> > OK. Let's follow this path a bit.
> > I googled "how to create a systemd user service" and got
> > .
> >
> > Starting from there:
> >
> > hobbit:~$ cd .config
> > hobbit:~/.config$ mkdir -p systemd/user
> > hobbit:~/.config$ vi systemd/user/xterm.service
> > hobbit:~/.config$ cat systemd/user/xterm.service
> > [Unit]
> > Description=Try to run an xterm
> >
> > [Service]
> > Type=simple
> > StandardOutput=journal
> > ExecStart=/usr/bin/xterm
> >
> > [Install]
> > WantedBy=default.target  
> 
> I think the systemd shortcut here is "systemctl --user edit 
> xterm.service". Now, "systemctl edit" usually edits an "override"
> file (allowing you to override settings supplied by the OS, you can
> edit the service file directly by adding the "--full" option. And, if
> you are creating a new service you tend to have to add "--force". So,
> the most reliable command for editing a user command would be
> "systemctl --user edit --force --full xterm.service".
> 
> One advantage of using "systemctl edit" rather than simply "$EDITOR"
> is that "systemctl edit" will perform a daemon-reload upon exiting
> the editor. That is, "systemctl edit" makes systemd aware of the
> changes.
> 
> > hobbit:~/.config$ systemctl --user enable xterm.service
> > Created
> > symlink /home/greg/.config/systemd/user/default.target.wants/xterm.service
> > → /home/greg/.config/systemd/user/xterm.service. hobbit:~/.config$
> > systemctl --user start xterm.service  
> 
> You can combine "enable" and "start" into a single command by adding 
> "--now" to the enable command, thus: "systemctl --user enable --now 
> xterm.service".

Increasingly systemd reminds me of SMIT (the system management part of
AIX). Designed to simplify complicated system management scripts by
using multiple short scripts, it just resulted in thousands and
thousands of system management scripts to manage :(

Re: umask - default user settings?

[Darac Marjal]

Debian is a multi-user operating system. Decisions should be made accordingly.

I suppose umask is a moot point on phones and tablets, where
single-user is often the use case.
On the contrary, modern Android is strongly multi-user. Each "app" tends 
to be allocated its own user ID. The logic is is to provide strong 
isolation between apps.


Jeff



OpenPGP_signature.asc
Description: OpenPGP digital signature

Re: umask - default user settings?

[Darac Marjal]

I'm not saying that what you did was wrong, but systemd provides a few 
shortcuts which can make things a bit more user-friendly.


On 16/07/2024 04:39, Greg Wooledge wrote:

OK. Let's follow this path a bit.
I googled "how to create a systemd user service" and got
.

Starting from there:

hobbit:~$ cd .config
hobbit:~/.config$ mkdir -p systemd/user
hobbit:~/.config$ vi systemd/user/xterm.service
hobbit:~/.config$ cat systemd/user/xterm.service
[Unit]
Description=Try to run an xterm

[Service]
Type=simple
StandardOutput=journal
ExecStart=/usr/bin/xterm

[Install]
WantedBy=default.target


I think the systemd shortcut here is "systemctl --user edit 
xterm.service". Now, "systemctl edit" usually edits an "override" file 
(allowing you to override settings supplied by the OS, you can edit the 
service file directly by adding the "--full" option. And, if you are 
creating a new service you tend to have to add "--force". So, the most 
reliable command for editing a user command would be "systemctl --user 
edit --force --full xterm.service".


One advantage of using "systemctl edit" rather than simply "$EDITOR" is 
that "systemctl edit" will perform a daemon-reload upon exiting the 
editor. That is, "systemctl edit" makes systemd aware of the changes.



hobbit:~/.config$ systemctl --user enable xterm.service
Created symlink 
/home/greg/.config/systemd/user/default.target.wants/xterm.service → 
/home/greg/.config/systemd/user/xterm.service.
hobbit:~/.config$ systemctl --user start xterm.service


You can combine "enable" and "start" into a single command by adding 
"--now" to the enable command, thus: "systemctl --user enable --now 
xterm.service".




OpenPGP_signature.asc
Description: OpenPGP digital signature

Re: umask - default user settings?

[Jeffrey Walton]

On Tue, Jul 16, 2024 at 1:45 PM  wrote:
>[...]
>
> (The most probable outcome though is even less rosy: everything'll run in
> the browser, and Secure Boot will make sure that your hardware refuses to
> run anything else, because the chips are sponsored by the Ad Industry.

Lol...

Re: umask - default user settings?

[Thomas Schmitt]

Hi,

to...@tuxteam.de wrote:
> Somehow I'm glad I stayed away from DEs and systemd up to now. Perhaps I
> just retire before the alternatives aren't viable anymore. Or perhaps, as
> with PulseAudio, I can leapfrog that "tech".

Retirement is no solution.
What shall we retirees do when X11 is laid to rest ?
I am not aware of anything like fvwm running on Wayland.


> (The most probable outcome though is even less rosy: everything'll run in
> the browser,

Maybe we can run X11 and fvwm on a virtual outdated graphics board in a
browser window ?


> and Secure Boot will make sure that your hardware refuses to
> run anything else, because the chips are sponsored by the Ad Industry.

That would make not much difference to the end of fvwm.

"A life without pug is possible but pointless."
(Loriot: "Ein Leben ohne Mops ist moeglich, aber sinnlos.")


Have a nice day :)

Thomas

Re: umask - default user settings?

[tomas]

On Tue, Jul 16, 2024 at 11:52:29AM -0400, Greg Wooledge wrote:
> On Tue, Jul 16, 2024 at 22:21:23 +0700, Max Nikulin wrote:
> > Greg, do you have an example when Environment= in service.d works, but an
> > environment.d file does not?
> 
> Oh gods, there's MORE shit to worry about??  Of course there is.
> Bloody hell.

[...]

> For anyone reading this in the archives, out of context, please note that
> the contents of these files *do not* apply to user logins.  They only
> apply to programs started by the systemd --user daemon.

Thanks for the interested read.

It reminds me of the horror stories we told each other when 14 or so, at
the campfire.

Somehow I'm glad I stayed away from DEs and systemd up to now. Perhaps I
just retire before the alternatives aren't viable anymore. Or perhaps, as
with PulseAudio, I can leapfrog that "tech".

(The most probable outcome though is even less rosy: everything'll run in
the browser, and Secure Boot will make sure that your hardware refuses to
run anything else, because the chips are sponsored by the Ad Industry.

Oh, well).

Cheers
-- 
t


signature.asc
Description: PGP signature

Re: umask - default user settings?

[Greg Wooledge]

On Tue, Jul 16, 2024 at 22:21:23 +0700, Max Nikulin wrote:
> Greg, do you have an example when Environment= in service.d works, but an
> environment.d file does not?

Oh gods, there's MORE shit to worry about??  Of course there is.
Bloody hell.

In previous years, I remember exploring environment.d and discarding it,
because it never achieved the goals that I had in mind.  At that time, I
was trying to find a shell-agnostic and login-style-agnostic way to set
environment variables for all login sessions (console, ssh, DM).

environment.d is not that.

But in this particular investigation, we're "only" trying to find a
way to configure the user environment for programs started by Desktop
Environments.  So, back to the laboratory

Given the existence of the following files:

-rw-r--r-- 1 greg greg 66 Jul 16 11:43 .config/environment.d/foo.conf
-rw-r--r-- 1 greg greg 51 Jul 15 23:24 .config/systemd/user/service.d/env.conf
-rw-r--r-- 1 greg greg 21 Jul 15 23:39 .config/systemd/user/service.d/umask.conf

And given that "systemctl --user daemon-reload" has been performed since
they were last edited, what I'm seeing is that the contents of the
environment.d file are applied *first*, and then the contents of the
service.d files are applied *second*, overriding whatever was in the
environment.d file.

Also, and this is *highly* noteworthy, the syntaxes are different.
In particular, expansions like %h are performed in the service.d files,
but *not* in the environment.d files.

On the other hand, environment.d(5) says you can use a limited subset of
shell expansion syntax.  It might be worth investigating exactly what
variables are available for expansion at that time, but at first glance,
it looks like service.d with its %h is probably going to be more useful.

For anyone reading this in the archives, out of context, please note that
the contents of these files *do not* apply to user logins.  They only
apply to programs started by the systemd --user daemon.

Re: umask - default user settings?

[Max Nikulin]

On 16/07/2024 19:03, Greg Wooledge wrote:

On Tue, Jul 16, 2024 at 18:42:40 +0700, Max Nikulin wrote:

On 16/07/2024 10:39, Greg Wooledge wrote:

hobbit:~/.config$ cat systemd/user/xterm.service


I am a bit afraid that corner cases might exist because there are no
.service files for applications started from menus and runners


Well, I don't have "menus and runners".  I had to create *something*
I could use for testing, and that's what I came up with.


Running from a menu or a runner is similar to

systemd-run --user --scope --slice app.slice -- xterm

Accordingly to systemd-run(1), --scope causes environment inheritance 
from starting process despite new process is moved to another branch of 
cgroup tree.


However

│   │ │ ├─plasma-krunner.service (#8046)
│   │ │ │ └─2533 /usr/bin/krunner

and

│   │ │ ├─plasma-plasmashell.service (#7073)
│   │ │ │ └─2027 /usr/bin/plasmashell --no-respawn

that are parents in "ps xwf" process tree for runner and menu cases 
accordingly are *services* and it explains why service.d overrides apply.


Greg, do you have an example when Environment= in service.d works, but 
an environment.d file does not?


P.S. There is
<https://wiki.debian.org/Debate/umask>
from 2013 briefly discussing 022 vs. 002 values.

Re: umask - default user settings?

[Greg Wooledge]

On Mon, Jul 15, 2024 at 23:39:54 -0400, Greg Wooledge wrote:
> Now we just need for GNOME users to discover a way to configure the
> programs that are started as children of dbus, and then we can move
> forward.  Documentation would be my top priority.  If other people want
> to try to drum up interest in environment configuration, then there'll
> be documentation available for end users to follow.

I've added a bit of content to .

I didn't mention dbus at all.  If anyone knows about dbus configuration,
please feel free to add it, or to post something here on debian-user.

Re: umask - default user settings?

[Greg Wooledge]

On Tue, Jul 16, 2024 at 18:42:40 +0700, Max Nikulin wrote:
> On 16/07/2024 10:39, Greg Wooledge wrote:
> > hobbit:~/.config$ cat systemd/user/xterm.service
> 
> I am a bit afraid that corner cases might exist because there are no
> .service files for applications started from menus and runners

Well, I don't have "menus and runners".  I had to create *something*
I could use for testing, and that's what I came up with.

> > Now we just need for GNOME users to discover a way to configure the
> > programs that are started as children of dbus, and then we can move
> > forward.
> 
> I do not have a recent live image on my disk to verify, but I expect that
> nowadays Gnome uses systemd session as well. Desktop environments that have
> not migrated to systemd likely still relies on Xsession.

There are very few GNOME users on this mailing list.  We do not reflect
the typical Debian population very well at all.  Most of us are older,
more traditional users, who favor plain window managers (fvwm and company)
or lighter Desktop Environments.

This makes it hard to support certain parts of Debian, and GNOME is
definitely on that "hard to support" list.

Re: umask - default user settings?

[Max Nikulin]

On 16/07/2024 10:39, Greg Wooledge wrote:

On Tue, Jul 16, 2024 at 09:58:20 +0700, Max Nikulin wrote:

cat ~/.config/systemd/user/service.d/umask.conf
[Service]
UMask=0007


I googled "how to create a systemd user service" and got


The following blog posts (0pointer.de) may be a bit outdated, but still 
useful:

https://systemd.io/#the-systemd-for-administrators-blog-series


hobbit:~/.config$ cat systemd/user/xterm.service


I am a bit afraid that corner cases might exist because there are no 
.service files for applications started from menus and runners


systemd-cgls

├─user.slice (#361)
│ └─user-1000.slice (#6090)
│   ├─user@1000.service … (#6182)
│   │ ├─session.slice (#6320)
│   │ ├─app.slice (#6274)
...
│   │ │ ├─app-org.kde.konsole-dfb8….scope (#8087)
│   │ │ │ ├─ 2572 /usr/bin/konsole
...
│   │ │ ├─app-debian\x2dxterm-e15c….scope (#14192)
│   │ │ │ ├─25859 /usr/bin/xterm


systemd --user process inherits my $DISPLAY


You may use

systemctl --user show-environment


[Service]
Environment="FOO=%h/test123" "BAR=b a r"


I do not remember if environment.d(5) has some issues that may be 
avoided this way.



I'm assuming the two files in service.d/ can be merged, but I didn't
test that.


Certainly, however sometimes it is more convenient to manage independent 
drop-ins.




So, this is a highly positive result.  (And surprising.  Why isn't this
stuff documented in ALL the places??)


Man pages for systemd is so long that sometimes it is easy to miss 
something important. Sometimes systemd.index(7) and 
systemd.directives(7) are not enough to find a specific topic.


Perhaps I have noticed service.d trick in output of "sustemctl cat" for 
some service.


systemd.unit(5):

Top level per-type drop-ins can be used to change some aspect of all
units of a particular type. For example by creating the
/etc/systemd/system/service.d/ directory with a drop-in file,


Greg Wooledge wrote:

Now we just need for GNOME users to discover a way to configure the
programs that are started as children of dbus, and then we can move
forward.


I do not have a recent live image on my disk to verify, but I expect 
that nowadays Gnome uses systemd session as well. Desktop environments 
that have not migrated to systemd likely still relies on Xsession.

Re: umask - default user settings?

[Nicolas George]

Greg Wooledge (12024-07-15):
> Neither am I.  But more to the point, it appears that the default umask
> literally *cannot* be changed in any kind of universal way.  There are,
> like, half a dozen different places you'd have to apply a change in
> order to cover just the *most common* workflows.  Who knows how many
> more corner cases would be missed?

This is very true. And alas, it is not limited to umask: environment
variables, limits, etc.

The solution I chose for myself is to put it in .zshenv, zsh being my
shell, and making sure all my other startup scripts are #!/bin/zsh. But
I am sure modern display managers manage to start modern desktop
environments that manage to run user applications without ever invoking
the login shell. Yay for modernity.

Anyway, this whole discussion is moot, because:

- Experienced users can find a solution for the specific system they
  use.

- Even though the default umask is permissive, the permissions on ~ can
  be restrictive, and for lusers it is functionally equivalent, and the
  default permissions on ~ is asked by debconf.

Regards,

-- 
  Nicolas George

Re: umask - default user settings?

[Greg Wooledge]

On Tue, Jul 16, 2024 at 09:58:20 +0700, Max Nikulin wrote:
> I have naively tried
> 
> cat ~/.config/systemd/user/service.d/umask.conf
> [Service]
> UMask=0007
> 
> From xterm and konsole:
> 
> umask
> 0007

OK.  Let's follow this path a bit.

I googled "how to create a systemd user service" and got
<https://blog.victormendonca.com/2018/05/14/creating-a-simple-systemd-user-service/>.

Starting from there:

hobbit:~$ cd .config
hobbit:~/.config$ mkdir -p systemd/user
hobbit:~/.config$ vi systemd/user/xterm.service
hobbit:~/.config$ cat systemd/user/xterm.service 
[Unit]
Description=Try to run an xterm

[Service]
Type=simple
StandardOutput=journal
ExecStart=/usr/bin/xterm

[Install]
WantedBy=default.target
hobbit:~/.config$ systemctl --user enable xterm.service
Created symlink 
/home/greg/.config/systemd/user/default.target.wants/xterm.service → 
/home/greg/.config/systemd/user/xterm.service.
hobbit:~/.config$ systemctl --user start xterm.service


This caused an xterm to popup.  OK, so far so good.  Apparently the
systemd --user process inherits my $DISPLAY, so it's able to launch an
xterm.

I closed the xterm, and then:

hobbit:~$ cd ~/.config/systemd
hobbit:~/.config/systemd$ ls
user/
hobbit:~/.config/systemd$ ls user
default.target.wants/  xterm.service
hobbit:~/.config/systemd$ cd user
hobbit:~/.config/systemd/user$ mkdir service.d
hobbit:~/.config/systemd/user$ vi service.d/umask.conf
hobbit:~/.config/systemd/user$ cat service.d/umask.conf
[Service]
UMask=0007


Then:

hobbit:~$ systemctl --user start xterm.service


I ran umask within this xterm, and got 0022.  Next:

hobbit:~$ systemctl --user daemon-reload
hobbit:~$ systemctl --user start xterm.service


This time, umask gave me 0007.

So it looks like your technique works, allowing an end user to create a
systemd --user configuration file which affects all(?) of their
systemd --user services, if they have any.

I also performed this experiment:

hobbit:~$ vi .config/systemd/user/service.d/env.conf
hobbit:~$ cat .config/systemd/user/service.d/env.conf
[Service]
Environment="FOO=%h/test123" "BAR=b a r"
hobbit:~$ systemctl --user daemon-reload
hobbit:~$ systemctl --user start xterm.service


The resulting xterm had umask 0007, and the expected values in the FOO
and BAR environment variables (with %h expanded to my home directory,
as documented in systemd.unit(5)).

I'm assuming the two files in service.d/ can be merged, but I didn't
test that.

So, this is a highly positive result.  (And surprising.  Why isn't this
stuff documented in ALL the places??)  It looks like there's a chance
that some Desktop Environments can be configured by end users after all.

Now we just need for GNOME users to discover a way to configure the
programs that are started as children of dbus, and then we can move
forward.  Documentation would be my top priority.  If other people want
to try to drum up interest in environment configuration, then there'll
be documentation available for end users to follow.

Re: umask - default user settings?

[Max Nikulin]

On 16/07/2024 08:34, Greg Wooledge wrote:

On Tue, Jul 16, 2024 at 08:02:45 +0700, Max Nikulin wrote:


systemd.exec(5)


UMask=

[...]

[5] refers to <https://systemd.io/USER_RECORD>.


I do not have systemd-homed running (minimal KDE). I have no idea 
concerning default Gnome installation. My expectation that is should 
solve various issues with per-user encrypted directories and may be 
suitable for remote home directories.



https://github.com/systemd/systemd/issues/16963#issuecomment-689656886

mkdir -p /etc/systemd/system/user@1000.service.d
cat >  /etc/systemd/system/user@1000.service.d/umask.conf<

So, this way requires root privileges as well.


I have naively tried

cat ~/.config/systemd/user/service.d/umask.conf
[Service]
UMask=0007

From xterm and konsole:

umask
0007

It seem creating new files from Dolphin respect the setting as well. I 
admit I may miss some use case when it is ignored, but at first glance I 
do not see a clear reason to complain.

Re: umask - default user settings?

[Jeffrey Walton]

On Mon, Jul 15, 2024 at 9:34 PM Greg Wooledge  wrote:
>
> On Tue, Jul 16, 2024 at 08:02:45 +0700, Max Nikulin wrote:
> [...]
> > systemd.exec(5)
> >
> > > UMask=
> > > Controls the file mode creation mask. Takes an access mode in octal
> > > notation. See umask(2) for details. Defaults to 0022 for system units.
> > > For user units the default value is inherited from the per-user service
> > > manager (whose default is in turn inherited from the system service
> > > manager, and thus typically also is 0022 — unless overridden by a PAM
> > > module). In order to change the per-user mask for all user services,
> > > consider setting the UMask= setting of the user's user@.service system
> > > service instance. The per-user umask may also be set via the umask field
> > > of a user's JSON User Record[5] (for users managed by
> > > systemd-homed.service(8) this field may be controlled via homectl
> > > --umask=). It may also be set via a PAM module, such as pam_umask(8).
>
> [5] refers to <https://systemd.io/USER_RECORD>.
>
> I defy any human being to read that web page and tell me WHAT FILE TO
> EDIT, and WHAT TO PUT IN IT, to effect a change to your environment.

++

When I read that, I tried it myself:

root@raptor:~# grep -IR 'jwalton@.service' /etc
root@raptor:~# grep -IR 'jwalton@.service' /lib
root@raptor:~#

> I can't find anything concrete in there.  Just a bunch of jabber.

Welcome to the world according to Pottering.

> The only filename I could find by skimming that thing was ~/.identity,
> and that's buried *deep* inside the page.  Is that the file you're
> supposed to create and/or edit?  What do you put in it to make your
> programs have a umask of your choosing?  Is there an example?

Jeff

Re: umask - default user settings?

[Greg Wooledge]

On Tue, Jul 16, 2024 at 08:02:45 +0700, Max Nikulin wrote:
> (I am not convinced that default umask should be changed)

Neither am I.  But more to the point, it appears that the default umask
literally *cannot* be changed in any kind of universal way.  There are,
like, half a dozen different places you'd have to apply a change in
order to cover just the *most common* workflows.  Who knows how many
more corner cases would be missed?

On Tue, Jul 16, 2024 at 07:53:31 +0700, Max Nikulin wrote:
> First of all, it is up to display manager to read or to ignore ~/.profile
> and ~/.xsessionrc. SDDM reads them. However it affects only
> /usr/bin/startplasma-x11 subtree. Most user applications are started under
> "/lib/systemd/systemd --user"

So, this is a variant of the same problem we observed with GNOME a few
years ago, just with systemd --user instead of dbus.  Desktop Environments
are not the parent processes of the programs that they run.  This means
there's no way for end users to configure the environments of their
own programs.

> https://wiki.archlinux.org/title/umask#Set_umask_value_for_KDE_/_Plasma

This one gives two alternatives, both of which require editing files as
root.

> systemd.exec(5)
> 
> > UMask=
> > Controls the file mode creation mask. Takes an access mode in octal
> > notation. See umask(2) for details. Defaults to 0022 for system units.
> > For user units the default value is inherited from the per-user service
> > manager (whose default is in turn inherited from the system service
> > manager, and thus typically also is 0022 — unless overridden by a PAM
> > module). In order to change the per-user mask for all user services,
> > consider setting the UMask= setting of the user's user@.service system
> > service instance. The per-user umask may also be set via the umask field
> > of a user's JSON User Record[5] (for users managed by
> > systemd-homed.service(8) this field may be controlled via homectl
> > --umask=). It may also be set via a PAM module, such as pam_umask(8).

[5] refers to <https://systemd.io/USER_RECORD>.

I defy any human being to read that web page and tell me WHAT FILE TO
EDIT, and WHAT TO PUT IN IT, to effect a change to your environment.

I can't find anything concrete in there.  Just a bunch of jabber.

The only filename I could find by skimming that thing was ~/.identity,
and that's buried *deep* inside the page.  Is that the file you're
supposed to create and/or edit?  What do you put in it to make your
programs have a umask of your choosing?  Is there an example?

> https://github.com/systemd/systemd/issues/16963#issuecomment-689656886
> poettering commented Sep 9, 2020
> > The user@.service templated system service is instantiated for each
> > user. It's a system service like any other, hence you can extend it via
> > drop-ins, and thus configure UMask= for it, like for any other system
> > service. e.g.
> > 
> > mkdir -p /etc/systemd/system/user@1000.service.d
> > cat >  /etc/systemd/system/user@1000.service.d/umask.conf< > [Service]
> > UMask=0007
> > EOF
> > systemctl daemon-reload

So, this way requires root privileges as well.

How does an ordinary user modify *THEIR OWN ENVIRONMENT* without superuser
access?

If you don't use a Desktop Environment, then this is ridiculously simple.
You put the commands to alter your environment in your ~/.profile or
equivalent for your shell, for your shell logins.  You put the commands
in ~/.xsessionrc (on Debian), for your X sessions, if you use a Display
Manager to login.  (If you use startx, you don't even have to do that;
your X session will inherit the environment from your login shell.)

All of the programs you run, other than cron jobs, will inherit their
environment either from your login shell, or from your X session.  So
this works.

In a DE, this is just impossible to do.

Why are Desktop Environments so *stupidly broken*?  Why don't their
developers seem to *care*?

Re: umask - default user settings?

[Max Nikulin]

On 15/07/2024 20:03, Greg Wooledge wrote:

If you use a Desktop Environment, go to your DE's support mailing list,
and ask them how to set your umask so that it works as expected in all
of your programs.


(I am not convinced that default umask should be changed)

systemd.exec(5)


UMask=
Controls the file mode creation mask. Takes an access mode in octal
notation. See umask(2) for details. Defaults to 0022 for system units.
For user units the default value is inherited from the per-user service
manager (whose default is in turn inherited from the system service
manager, and thus typically also is 0022 — unless overridden by a PAM
module). In order to change the per-user mask for all user services,
consider setting the UMask= setting of the user's user@.service system
service instance. The per-user umask may also be set via the umask field
of a user's JSON User Record[5] (for users managed by
systemd-homed.service(8) this field may be controlled via homectl
--umask=). It may also be set via a PAM module, such as pam_umask(8).


https://github.com/systemd/systemd/issues/16963#issuecomment-689656886
poettering commented Sep 9, 2020

The user@.service templated system service is instantiated for each
user. It's a system service like any other, hence you can extend it via
drop-ins, and thus configure UMask= for it, like for any other system
service. e.g.

mkdir -p /etc/systemd/system/user@1000.service.d
cat >  /etc/systemd/system/user@1000.service.d/umask.conf<

Re: umask - default user settings?

[Max Nikulin]

On 15/07/2024 01:10, Greg Wooledge wrote:

On Sun, Jul 14, 2024 at 19:57:45 +0200, to...@tuxteam.de wrote:

The place to do this is the X session [1]; system-wide in
/etc/X11/Xsession.d/... and for each user in ~/.xsessionrc.


Does that work in KDE?


First of all, it is up to display manager to read or to ignore 
~/.profile and ~/.xsessionrc. SDDM reads them. However it affects only 
/usr/bin/startplasma-x11 subtree. Most user applications are started 
under "/lib/systemd/systemd --user"


https://wiki.archlinux.org/title/umask#Set_umask_value_for_KDE_/_Plasma

Setting the umask value via /etc/profile does no longer work for KDE /
Plasma sessions because these are started as systemd user units.


...and some recipes.

Re: umask - default user settings?

[Jeffrey Walton]

On Mon, Jul 15, 2024 at 12:06 PM Eduardo M KALINOWSKI
 wrote:
>
> On 14/07/2024 14:09, Hans wrote:
> > Dear list,
> >
> > I am wondering, why on a multiuser system like debian the rights for a 
> > normal
> > user are "rw- r-- r--", (owner: user and ownergroup: usergroup)
> >
> > Of course there is a reason for this, but it is not understandable for me.
> >
> >
> > First two are clear: rw for myself, and readable for all users, i am 
> > allowing
> > into my own grou.
> >
> > The last one is not clear for me. Why should I allow the rest of the world
> > read my personal documents? These are private and no one else should be able
> > to read them!
> >
> > So I would have expected a setting of "rw- r-- ---" for any files.
> >
> > Before someone argues, "you can change this by editing umask", yes, I know 
> > of
> > this of course.
> >
> > But it is not clear for me, why it is set that way by default and not as I
> > would have expected as described above.
> >
> > Sure, there is a reason for this, so I will be happy, if someone could
> > enlighten me.
>
> I kind of agree with that in principle, and I've always used an umask
> 077 myself.
>
> On the other hand, I'm the only user in my system, so it doesn't really
> matter. I expect that is the case for most users.
>
> I'm not sure if the Debian default should be changed, though.

Debian is a multi-user operating system. Decisions should be made accordingly.

I suppose umask is a moot point on phones and tablets, where
single-user is often the use case.

Jeff

Re: umask - default user settings?

[Franco Martelli]

On 14/07/24 at 20:44, to...@tuxteam.de wrote:

  so they aren't children of the GNOME top-level
process, and don't inherit the umask or environment from the session.

I'm totally willing to believe that KDE is different, but it's not
clear whether "Lists" has tried this and failed, or simply didn't know
that it should be done this way.

It would be excellent to receive confirmation from a KDE user, either
way.

I'd be curious, too.


KDE provides a special purpose directory for setting the environment 
globally: ~/.config/plasma-workspace/env/
In addition it is possible to specify environment variables for every 
application singularly by editing the "Start" menu.
In that directory must be placed executable shell scripts that configure 
the environment e.g.:


~$ ls -l ~/.config/plasma-workspace/env/kwin_env.sh
-rwxr-xr-x 1 frank frank 97 15 lug 15.38 
/home/frank/.config/plasma-workspace/env/kwin_env.sh


~$ cat ~/.config/plasma-workspace/env/kwin_env.sh
#!/bin/sh

export KWIN_TRIPLE_BUFFER=1
/usr/bin/picom -b --config ~/.config/picom.conf

I tried to add "umask 027" to the bottom of "kwin_env.sh" file but that 
change it doesn't apply to xterm.

Is there any other syntax to specify the umask? What's in your .xsessionrc?

Cheers
--
Franco Martelli

Re: umask - default user settings?

[Greg Wooledge]

If people on this mailing list are so concerned about other people's
umasks, then I would suggest a great starting point would be to start by
making it POSSIBLE for other people to set their umasks the way they want.

If you use a Desktop Environment, go to your DE's support mailing list,
and ask them how to set your umask so that it works as expected in all
of your programs.  This must include programs that are launched at login,
and programs that are launched by a menu or icon, and terminal emulators
(and not merely the shells that run inside terminal emulators).

Make sure whatever solution they give you applies regardless of whether
a program is launched as a child of a window manager, or a child of
a systemd user session, or a child of dbus.

If they can't give you a solution that works in all of these cases,
**make them fix their stupid software**.  Do not let up until they have
given their users a way to configure their environments as needed.

Once all of the Desktop Environments provide a way to do what you want,
**then** you might think about leaning on Debian to change their defaults.
That will be a separate battle, but it's not a battle you can currently
even **begin** fighting.

Also bear in mind, this is the Debian **user** mailing list, and none of
us have any power whatsoever over the Debian project.  Let alone over
the various Desktop Environments.

Re: umask - default user settings?

[Lists]

On 2024-07-15 14:30, Eduardo M KALINOWSKI wrote:


I'm not sure if the Debian default should be changed, though.


One thing to consider is that in modern software development practices 
the idea of secure/private by default is getting more and more important 
and implemented. It is good practice to protect yourself and your users 
against mistakes. I know things can be done to set a proper mask, but 
why not opt for the same stance that firewalls have adopted long ago, 
being "deny by default"? In some cases that might make a system harder 
to use, but I don't see how that could be the case in a user's home 
directory.


Just my 2 cents.

Grx HdV

Re: umask - default user settings?

[Eduardo M KALINOWSKI]

On 14/07/2024 14:09, Hans wrote:

Dear list,

I am wondering, why on a multiuser system like debian the rights for a normal
user are "rw- r-- r--", (owner: user and ownergroup: usergroup)

Of course there is a reason for this, but it is not understandable for me.


First two are clear: rw for myself, and readable for all users, i am allowing
into my own grou.

The last one is not clear for me. Why should I allow the rest of the world
read my personal documents? These are private and no one else should be able
to read them!

So I would have expected a setting of "rw- r-- ---" for any files.

Before someone argues, "you can change this by editing umask", yes, I know of
this of course.

But it is not clear for me, why it is set that way by default and not as I
would have expected as described above.

Sure, there is a reason for this, so I will be happy, if someone could
enlighten me.


I kind of agree with that in principle, and I've always used an umask 
077 myself.


On the other hand, I'm the only user in my system, so it doesn't really 
matter. I expect that is the case for most users.


I'm not sure if the Debian default should be changed, though.

--
Actually, my goal is to have a sandwich named after me.

Eduardo M KALINOWSKI
edua...@kalinowski.com.br

Re: umask - default user settings?

[Greg Wooledge]

On Mon, Jul 15, 2024 at 09:04:54 +0200, Hans wrote:
> Also, when some other applicatiions are setting correct rights. 
> Some do, some don't. 

File bug reports against the ones which don't.

Re: umask - default user settings?

[Hans]

> The door is closed by default in bookworm. User home directories are
> created with 0700 mode, see /usr/share/doc/adduser/README.gz and
> /usr/share/doc/adduser/NEWS.Debian.gz As a result, it is necessary to
> set ACLs e.g. to run unprivileged LXC containers.

That is not the point. The point us, that debian is creating a default user 
"for your daily work" at installation with umask 022. 

And we are not talking about experienced users, but of linux beginners. I 
doubt, they are aware of umask and rights and so.

Debian is made for every people, not only for experienced people.

Yes, when adduser cares about, this is one good step, but does not touch my 
argumentation. Also, when some other applicatiions are setting correct rights. 
Some do, some don't. 

That is not the point, too. The point is, should't we do it completely and 
make it standard by default - also and especially during installation to make 
debian more secure for unexprienced users and linux noobs?

Best

Hans
 

 

Re: umask - default user settings?

[Emanuel Berg]

Here is some cool ascii art to illustrate permissions
after mount.

The (x)_b notation indicates that x is in base b.

# permissions
# rwxr-xr-x dirs
local dmask=022 #   (22)_8 = (10010)_2
local fmask=133 #  (133)_8 = (  1011011)_2
# rw-r--r-- files
#

-- 
underground experts united
https://dataswamp.org/~incal

Re: umask - default user settings?

[Alan D. Salewski]

On 2024-07-14 22:15:34, "Alan D. Salewski"  spake thus:
[...]

The user's umask value would matter less if the default perms of
user $HOME directories were 077


s/were/were from a umask of/

Re: umask - default user settings?

[Greg Wooledge]

On Sun, Jul 14, 2024 at 22:15:34 -0400, Alan D. Salewski wrote:
> As it is, it
> looks[1] like default perms for $HOME are 0755.

If home directories are created with adduser, then the contents of
/etc/adduser.conf are relevant:


# The permissions mode for home directories of non-system users.
# Default: DIR_MODE=0700
#DIR_MODE=0700

# The permissions mode for home directories of system users.
# Default: SYS_DIR_MODE=0755
#SYS_DIR_MODE=0755


If they're created by something else, then it depends on what that other
creation method is.

Re: umask - default user settings?

[Alan D. Salewski]

On 2024-07-14 19:38:26, Hans  spake thus:

Hi Greg,

yes, did already change it. However, this looks like a security hole for me, 
as I believe, not many people or admins are changing this.


I suspect that most people /do/ change it, once they become aware of 
it, for the very reason stated in the comment above 'UMASK' in the 
/etc/login.defs file:



# UMASK is the default umask value for pam_umask and is used by
# useradd and newusers to set the mode of the new home directories.
# 022 is the "historical" value in Debian for UMASK
# 027, or even 077, could be considered better for privacy
# There is no One True Answer here : each sysadmin must make up his/her
# mind.
    ...
UMASK   022




IMO debian should change this in the next release, but I doubt it.


I do not feel strongly about it, but I am sympathetic with the point 
of view that having a default umask of, say, 077 would be a better 
default for many unwary Day 1 *nix users. I think it is reasonable 
for somebody to expect that that something is not accessible by 
group or others unless the user explicitly made it so.


I have no idea how such a change might affect backward 
compatibility. I'm guessing not much, since the umask value is one 
of those things that tends to get set explicitly when it matters 
(beyond the user's need to read or write their own files).


The user's umask value would matter less if the default perms of 
user $HOME directories were 077, since then even files created with 
unintentionally looser perms could not be viewed by group or 
others[0]. As it is, it looks[1] like default perms for $HOME are 
0755.




I will ask the security team for it, they will decide.

Have fun!

Hans


-Al


[0] Assuming most files created by a user are created beneath that
user's $HOME.

[1] Just did an empirical test by spinning up a Debian 12.x
("Bookworm") AWS AMI, $HOME directories have perms 0755:

$ ls -la /home
total 12
drwxr-xr-x  3 root  root  4096 Jul 14 22:32 .
drwxr-xr-x 18 root  root  4096 Jul 14 22:32 ..
drwxr-xr-x  3 admin admin 4096 Jul 14 22:32 admin

--
a l a n   d.   s a l e w s k i
ads@salewski.email
salew...@att.net
https://github.com/salewski

Re: umask - default user settings?

[Max Nikulin]

On 15/07/2024 01:32, Hans wrote:

I see itthe other way round. No, if you are in the secure area, it is the
responsibility of the owner to make it secure by design i.e with dself closing
doors where you can not look into or windows with curtains.


The door is closed by default in bookworm. User home directories are 
created with 0700 mode, see /usr/share/doc/adduser/README.gz and 
/usr/share/doc/adduser/NEWS.Debian.gz As a result, it is necessary to 
set ACLs e.g. to run unprivileged LXC containers.


RedHat likely has 0700 for $HOME for much longer time.

Put your confidential files in directories not readable by others. You 
may consider keeping these directories encrypted.


In the past it was not uncommon to have ~/public_html accessible to web 
server. Once I have faced requirement to explicitly set selinux context 
for this directory. I do not know if it was default for that linux 
distribution or configured by the administrator.

Re: umask - default user settings?

[tomas]

On Sun, Jul 14, 2024 at 08:31:23PM +0200, Me wrote:
> On 2024-07-14 19:57, to...@tuxteam.de wrote:

[...]

> > [1]  https://wiki.debian.org/Xsession
> 
> Did you actually try this? I did and it did not what I was expecting it to
> do. But maybe I should try again, maybe things have improved in the
> meanwhile.

Me? As I said, I don't use desktop environments. Just plain X and a
window manager (Fvwm). For me, .xsessionrc works beautifully. I had
to enable it in /etc/X11/Xsession.options (back then, maybe Jessie or
Stretch) and this has travelled across upgrades to now.

So no complaints from me :)

Cheers
-- 
t


signature.asc
Description: PGP signature

Re: umask - default user settings?

[Me]

On 2024-07-14 19:57, to...@tuxteam.de wrote:

On Sun, Jul 14, 2024 at 07:44:35PM +0200, Lists wrote:

On 2024-07-14 19:18, Greg Wooledge wrote:

On Sun, Jul 14, 2024 at 19:09:54 +0200, Hans wrote:

I am wondering, why on a multiuser system like debian the rights for a normal
user are "rw- r-- r--", (owner: user and ownergroup: usergroup)


Tradition, and a culture based around sharing.

The Unix culture of openness and freedom (specifically the freedom to
distribute your work to others) works best if you can say "Hey Betty,
can you take a look at my .bashrc?  I can't get my foo() function to
work."  Or "Hey friends, I've made some changes to my bar.c file that
you might want to look at."  And then they can just read the files
directly from your home directory.

If you don't like this setting, change it.



Setting umask in your shell profile isn't that hard indeed. I've doing that
for years. However, that does not mean your DE will honour that setting. I
have tried to do so for KDE (more specifically Krusader), but I ended up
nowhere. I haven't found a setting that will be honoured KDE wide or even
just in Krusader alone.


The place to do this is the X session [1]; system-wide in
/etc/X11/Xsession.d/... and for each user in ~/.xsessionrc.
You might have to set allow-user-session in the global config
/etc/X11/Xsession.options to make the second possible.

Cheers

[1]  https://wiki.debian.org/Xsession


Did you actually try this? I did and it did not what I was expecting it 
to do. But maybe I should try again, maybe things have improved in the 
meanwhile.


Grx HdV

Re: umask - default user settings?

[Lists]

On 2024-07-14 19:43, Me wrote:

Setting umask in your shell profile isn't that hard indeed. I've doing 
that for years. However, that does not mean your DE will honour that 
setting. I have tried to do so for KDE (more specifically Krusader), but 
I ended up nowhere. I haven't found a setting that will be honoured KDE 
wide or even just in Krusader alone.


Grx HdV


Twice? My apologies for that. Must have sent that without realising it.

Grx HdV

Re: umask - default user settings?

[tomas]

On Sun, Jul 14, 2024 at 02:10:46PM -0400, Greg Wooledge wrote:
> On Sun, Jul 14, 2024 at 19:57:45 +0200, to...@tuxteam.de wrote:

[...]

> Does that work in KDE?

At least The Internet (TM) (from some cursory poking) seems to
say so. I stay away from DEs for... reasons, so I can't test
it.

>  As we discovered a few years ago, it doesn't
> work in GNOME.  GNOME specifically starts up its terminal emulators
> via dbus or something,

Gah. What a bad taste. One reason more to stay away from DEs.

>  so they aren't children of the GNOME top-level
> process, and don't inherit the umask or environment from the session.
> 
> I'm totally willing to believe that KDE is different, but it's not
> clear whether "Lists" has tried this and failed, or simply didn't know
> that it should be done this way.
> 
> It would be excellent to receive confirmation from a KDE user, either
> way.

I'd be curious, too.

Cheers
-- 
t


signature.asc
Description: PGP signature

Re: umask - default user settings?

[Me]

On 2024-07-14 19:18, Greg Wooledge wrote:

On Sun, Jul 14, 2024 at 19:09:54 +0200, Hans wrote:

I am wondering, why on a multiuser system like debian the rights for a normal
user are "rw- r-- r--", (owner: user and ownergroup: usergroup)


Tradition, and a culture based around sharing.

The Unix culture of openness and freedom (specifically the freedom to
distribute your work to others) works best if you can say "Hey Betty,
can you take a look at my .bashrc?  I can't get my foo() function to
work."  Or "Hey friends, I've made some changes to my bar.c file that
you might want to look at."  And then they can just read the files
directly from your home directory.

If you don't like this setting, change it.



Setting umask in your shell profile isn't that hard indeed. I've doing 
that for years. However, that does not mean your DE will honour that 
setting. I have tried to do so for KDE (more specifically Krusader), but 
I ended up nowhere. I haven't found a setting that will be honoured KDE 
wide or even just in Krusader alone.


Grx HdV

Re: umask - default user settings?

[Teemu Likonen]

* 2024-07-14 19:44:35+0200, li...@nodatagrabbing.com wrote:

> Setting umask in your shell profile isn't that hard indeed. I've doing 
> that for years. However, that does not mean your DE will honour that 
> setting. I have tried to do so for KDE (more specifically Krusader), but 
> I ended up nowhere. I haven't found a setting that will be honoured KDE 
> wide or even just in Krusader alone.

I think nowadays systemd has to be told about new umask value. Many
things are branched from user@.service unit. So, perhaps like this:

# /etc/systemd/system/user@.service.d/my-umask.conf

    [Service]
UMask=0007

-- 
/// Teemu Likonen - .-.. https://www.iki.fi/tlikonen/
// OpenPGP: 6965F03973F0D4CA22B9410F0F2CAE0E07608462


signature.asc
Description: PGP signature

Re: umask - default user settings?

[Hans]

I see itthe other way round. No, if you are in the secure area, it is the 
responsibility of the owner to make it secure by design i.e with dself closing 
doors where you can not look into or windows with curtains. 

However, I presume, debian wants to be secure. If no one cares and all agree 
with this (in my personal opinion!) security whole, I will have to accept 
this. For myself I found a solution of course, and I just could have not told 
about it, but I cared for others and tried to put my 2 cents in it, to improve 
the security of debian.

If this is too much, then we can close this issue at once. Shall we?

Hans

  
> If you are writing something confidential, it is your responsibility to
> lock the door of your office.
> 
> Regards,

Re: umask - default user settings?

[Greg Wooledge]

On Sun, Jul 14, 2024 at 19:57:45 +0200, to...@tuxteam.de wrote:
> On Sun, Jul 14, 2024 at 07:44:35PM +0200, Lists wrote:
> > Setting umask in your shell profile isn't that hard indeed. I've doing that
> > for years. However, that does not mean your DE will honour that setting.

> The place to do this is the X session [1]; system-wide in
> /etc/X11/Xsession.d/... and for each user in ~/.xsessionrc.
> You might have to set allow-user-session in the global config
> /etc/X11/Xsession.options to make the second possible.

Does that work in KDE?  As we discovered a few years ago, it doesn't
work in GNOME.  GNOME specifically starts up its terminal emulators
via dbus or something, so they aren't children of the GNOME top-level
process, and don't inherit the umask or environment from the session.

I'm totally willing to believe that KDE is different, but it's not
clear whether "Lists" has tried this and failed, or simply didn't know
that it should be done this way.

It would be excellent to receive confirmation from a KDE user, either
way.

Re: umask - default user settings?

[Nicolas George]

Hans (12024-07-14):
> Greg, I do not agree. If I am writing a document with private content, then I 

If you are writing something confidential, it is your responsibility to
lock the door of your office.

Regards,

-- 
  Nicolas George

Re: umask - default user settings?

[Hans]

Greg, I do not agree. If I am writing a document with private content, then I 
do not want to let it be read by someone else except me. 

No one has to read any letters or cv's or maybe documents for my lawyer, my 
medic, my friends or whatever. 

And after years there are a lot of documents one is writing, many private 
things.

And i think, no one wants to sharew these with other users on the system!

At least, I won't.

Best

Hans


> hobbit:~$ ls -l .ssh
> total 72
> ...
> -rw--- 1 greg greg  1876 Sep 24  2019 id_rsa
> -rw-r--r-- 1 greg greg   394 Sep 24  2019 id_rsa.pub
> 
> The other 99.9% of your files are not secret, so they don't need to
> be hidden.

Re: umask - default user settings?

[Thomas Schmitt]

Hi,

Hans wrote:
> I am wondering, why on a multiuser system like debian the rights for a
> normal user are "rw- r-- r--", (owner: user and ownergroup: usergroup)

Because the usual umask of 0022 keeps the more credulous programs from
giving w-permission to everybody.
Any program is free to hand out restricted permission. umask just defines
what such a program shall not get done immediately when the file is
created. Afterwards a program can still widen permissions.


> First two are clear: rw for myself, and readable for all users, i am
> allowing into my own grou.

It's not necessarily your group, but rather the group to which the file
belongs. This is typically the group of the process of the program which
creates the file. (Unless it has superuser powers and can change the group
id.)

Shell command "id" can tell your current shells user id and group id
which in most cases are inherited by programs which you start.

  $ id
  id=number(name) gid=number(name) groups=number(name),...

But there are the programs which are allowed to run under a self chosen
user and group id. (See man 1 chmod permission "s" and man 2 setgid.)


Have a nice day :)

Thomas

Re: umask - default user settings?

[tomas]

On Sun, Jul 14, 2024 at 07:44:35PM +0200, Lists wrote:
> On 2024-07-14 19:18, Greg Wooledge wrote:
> > On Sun, Jul 14, 2024 at 19:09:54 +0200, Hans wrote:
> > > I am wondering, why on a multiuser system like debian the rights for a 
> > > normal
> > > user are "rw- r-- r--", (owner: user and ownergroup: usergroup)
> > 
> > Tradition, and a culture based around sharing.
> > 
> > The Unix culture of openness and freedom (specifically the freedom to
> > distribute your work to others) works best if you can say "Hey Betty,
> > can you take a look at my .bashrc?  I can't get my foo() function to
> > work."  Or "Hey friends, I've made some changes to my bar.c file that
> > you might want to look at."  And then they can just read the files
> > directly from your home directory.
> > 
> > If you don't like this setting, change it.
> > 
> 
> Setting umask in your shell profile isn't that hard indeed. I've doing that
> for years. However, that does not mean your DE will honour that setting. I
> have tried to do so for KDE (more specifically Krusader), but I ended up
> nowhere. I haven't found a setting that will be honoured KDE wide or even
> just in Krusader alone.

The place to do this is the X session [1]; system-wide in
/etc/X11/Xsession.d/... and for each user in ~/.xsessionrc.
You might have to set allow-user-session in the global config
/etc/X11/Xsession.options to make the second possible.

Cheers

[1]  https://wiki.debian.org/Xsession 
-- 
t


signature.asc
Description: PGP signature

Re: umask - default user settings?

[Greg Wooledge]

On Sun, Jul 14, 2024 at 19:44:35 +0200, Lists wrote:
> Setting umask in your shell profile isn't that hard indeed. I've doing that
> for years. However, that does not mean your DE will honour that setting. I
> have tried to do so for KDE (more specifically Krusader), but I ended up
> nowhere. I haven't found a setting that will be honoured KDE wide or even
> just in Krusader alone.

I remember someone trying to do the same thing in GNOME several years
ago, and nobody could find a solution for GNOME, either.

With a traditional window manager, it's as simple as can be.  You just
set umask in your ~/.xsession file (before exec-ing your WM) and you're
done.  I don't know why the creators of Desktop Environments have decided
to discard all common sense.

Re: umask - default user settings?

[Greg Wooledge]

On Sun, Jul 14, 2024 at 19:38:26 +0200, Hans wrote:
> Hi Greg,
> 
> yes, did already change it. However, this looks like a security hole for me, 
> as I believe, not many people or admins are changing this.
> 
> IMO debian should change this in the next release, but I doubt it.
> 
> I will ask the security team for it, they will decide.

It is NOT a security issue.  Any files that contain secret data are
protected by their individual permissions, as set by the programs
that create them.  Like your ssh private keys:

hobbit:~$ ls -l .ssh
total 72
...
-rw--- 1 greg greg  1876 Sep 24  2019 id_rsa
-rw-r--r-- 1 greg greg   394 Sep 24  2019 id_rsa.pub

The other 99.9% of your files are not secret, so they don't need to
be hidden.

Re: umask - default user settings?

[Lists]

On 2024-07-14 19:18, Greg Wooledge wrote:

On Sun, Jul 14, 2024 at 19:09:54 +0200, Hans wrote:

I am wondering, why on a multiuser system like debian the rights for a normal
user are "rw- r-- r--", (owner: user and ownergroup: usergroup)


Tradition, and a culture based around sharing.

The Unix culture of openness and freedom (specifically the freedom to
distribute your work to others) works best if you can say "Hey Betty,
can you take a look at my .bashrc?  I can't get my foo() function to
work."  Or "Hey friends, I've made some changes to my bar.c file that
you might want to look at."  And then they can just read the files
directly from your home directory.

If you don't like this setting, change it.



Setting umask in your shell profile isn't that hard indeed. I've doing 
that for years. However, that does not mean your DE will honour that 
setting. I have tried to do so for KDE (more specifically Krusader), but 
I ended up nowhere. I haven't found a setting that will be honoured KDE 
wide or even just in Krusader alone.


Grx HdV

Re: umask - default user settings?

[Hans]

Hi Greg,

yes, did already change it. However, this looks like a security hole for me, 
as I believe, not many people or admins are changing this.

IMO debian should change this in the next release, but I doubt it.

I will ask the security team for it, they will decide.

Have fun!

Hans

Am Sonntag, 14. Juli 2024, 19:18:07 CEST schrieb Greg Wooledge:
> On Sun, Jul 14, 2024 at 19:09:54 +0200, Hans wrote:
> > I am wondering, why on a multiuser system like debian the rights for a
> > normal user are "rw- r-- r--", (owner: user and ownergroup: usergroup)
> 
> Tradition, and a culture based around sharing.
> 
> The Unix culture of openness and freedom (specifically the freedom to
> distribute your work to others) works best if you can say "Hey Betty,
> can you take a look at my .bashrc?  I can't get my foo() function to
> work."  Or "Hey friends, I've made some changes to my bar.c file that
> you might want to look at."  And then they can just read the files
> directly from your home directory.
> 
> If you don't like this setting, change it.

Re: umask - default user settings?

[Greg Wooledge]

On Sun, Jul 14, 2024 at 19:09:54 +0200, Hans wrote:
> I am wondering, why on a multiuser system like debian the rights for a normal 
> user are "rw- r-- r--", (owner: user and ownergroup: usergroup)

Tradition, and a culture based around sharing.

The Unix culture of openness and freedom (specifically the freedom to
distribute your work to others) works best if you can say "Hey Betty,
can you take a look at my .bashrc?  I can't get my foo() function to
work."  Or "Hey friends, I've made some changes to my bar.c file that
you might want to look at."  And then they can just read the files
directly from your home directory.

If you don't like this setting, change it.

umask - default user settings?

[Hans]

Dear list, 

I am wondering, why on a multiuser system like debian the rights for a normal 
user are "rw- r-- r--", (owner: user and ownergroup: usergroup)

Of course there is a reason for this, but it is not understandable for me.


First two are clear: rw for myself, and readable for all users, i am allowing 
into my own grou. 

The last one is not clear for me. Why should I allow the rest of the world 
read my personal documents? These are private and no one else should be able 
to read them! 

So I would have expected a setting of "rw- r-- ---" for any files. 

Before someone argues, "you can change this by editing umask", yes, I know of 
this of course. 

But it is not clear for me, why it is set that way by default and not as I 
would have expected as described above.

Sure, there is a reason for this, so I will be happy, if someone could 
enlighten me.

Best

Hans

Re: VFAT vs. umask.

[David Wright]

On Thu 04 Aug 2022 at 13:27:34 (-0400), gene heskett wrote:
> On 8/4/22 10:47, pe...@easthope.ca wrote:
> >  From: Charles Curley 
> >  Date: Sat, 30 Jul 2022 13:39:00 -0600
> > > The preparation of any storage medium requires at least two steps. To
> > > format means ...
> > > 
> > > In another step one lays down a file system: ...
> > Understood, at a superficial level at least.  Haven't invented or
> > implemented a file system.
> > 
> > > Microsoft conflated the two, leading to much confusion over the
> > > years.
> > Even prominent software uses "format" loosely.  Eg. Gparted >
> > Partition > Format to > ... .

I searched my way through
https://gparted.org/display-doc.php?name=help-manual
and couldn't find any ambiguity. They use the term to mean what you
want to call a "high-level format", "laying down a filesystem", or
some neologism.

> > If we really want to improve terminology we need to
> > (1) understand intimately software such as gparted,
> > (2) find or invent terms more specific than "format",
> > (3) convince maintainers to fix the bugs (adopt the new terms) and
> > (4) take credit for the revolutionary improvements.  =8~)
> > 
> > > Try mounting it from the command line, and watch the output. That will
> > > show you whether it is being fscked. However, FAT has no ability to
> > > store when it was last fscked, so I would wonder why it was being
> > > fscked at all.
> > In further observations, the problem is primarily in the old Fedora
> > based OLPC XO system.  Debian Bullseye usually recognizes and mounts
> > the f.s. on the SD card with no difficulty.  Rather than troubleshoot
> > the obsolete system I'll continue an effort to make DebXO work on the
> > XO 1.5 machine.

Is this the X/Y problem's Z?

> Having worked on a couple filesystems in a past life, all I can do is
> say +100 Peter.
> "format" is one of the most overloaded words in our vocabulary.

Really?

https://en.wiktionary.org/wiki/format
https://en.wiktionary.org/wiki/set

Cheers,
David.

Re: VFAT vs. umask.

[David Wright]

On Wed 03 Aug 2022 at 08:37:24 (-0700), pe...@easthope.ca wrote:
> From: David Wright 
> Date: Sun, 31 Jul 2022 14:15:26 -0500

> > So "primary store" probably means Master Copy of Your Data. 
> 
> https://en.wiktionary.org/wiki/primary#Adjective  sense 2.
> https://en.wiktionary.org/wiki/master#Adjective  sense 2.
> https://en.wiktionary.org/wiki/store#Noun  sense 4.

Yes, that last one is no help, usually triggering a HelpDesk-type
explanation of the difference between memory and storage.
As for primary store, the primary location for my photographs is
an SD card, but the first thing I do is transfer it to the master
location on spinning rust, check it, and back it up, before deleting
the primary store to avoid a "Memory Full" (sic) incident occurring
later, in the field.

So rather than a back and forth at cross-purposes (like that with
NAT and IPv6), I stated what /I/ thought your description meant.

> > ... copy it off the SD onto a hard drive to work on it, then put it 
> > back (hopefully with care, not overwriting, and thorough flushing¹).
> 
> I never do that.  I use a file on the SD just as if it's on a spinning 
> hdd.

OK, now we know where we stand. Your working methods and priorities
are so different from mine that it's easy to end up with crossed lines.
I run one system where the main drive is an SSD, but I've only run a
system on a flash drive (it was a USB rather than an SD) once, and
just as an experiment, for the challenge of installing an EFI bootable
system. That's a far worse case, of course, because it puts / onto the
flash drive, whereas presumably yours only involves a subdirectory.

> > ... reformat the card to an ext format and you can forget about that.
> 
> According to several other messages alignment and block size are 
> concerns in reformatting.

I'm using format in the sense used by the rest of the world, which you
and Charles would call a "high-level format" judging from posts like:
https://lists.debian.org/debian-user/2022/07/msg00768.html
https://lists.debian.org/debian-user/2022/07/msg00743.html
IOW, I meant what's done by mkfs.foo, except I was assuming that
you'd make an adjustment to the Partition Type first.

So I hadn't envisaged your considering a change in the partition's
/alignment/, which appears to be perfect, both at the beginning and,
more unusually, the end.

As for block size, yes, there are advantages to matching the
filesystem parameters with the underlying firmware's allocation
and erase units, but AFAICT the only way to reliably determine
the latter is by running timing experiments on the specific type
of device.

Much of this might be moot: allegedly there are design decisions
in how the cards operate that may be more important in determining
how fast the card is in use, particularly with different
filesystems, like FAT and ext, eg using a different strategy for
handling the start of the card, assumed to be the location of the
FAT's file allocation table itself.

As I pointed out, this might not make such a huge difference for the
use I thought you were making of the card, but I haven't a clue what
the effect might be when it's the file storage being used by your
programs while they're executing. That would require knowing how
the programs used it.

So it's quite possible that sticking with FAT is a better bet.
It was never clear to me what your permission problem with using
FAT actually was in the first place. Your OP read like some sort
of riddle or paradox that we were meant to solve, which we did.
(It's the driver wotduzit.)

Cheers,
David.

Re: VFAT vs. umask.

[gene heskett]

On 8/4/22 10:47, pe...@easthope.ca wrote:

 From: Charles Curley 
 Date: Sat, 30 Jul 2022 13:39:00 -0600

The preparation of any storage medium requires at least two steps. To
format means ...

In another step one lays down a file system: ...

Understood, at a superficial level at least.  Haven't invented or
implemented a file system.


Microsoft conflated the two, leading to much confusion over the
years.

Even prominent software uses "format" loosely.  Eg. Gparted >
Partition > Format to > ... .

If we really want to improve terminology we need to
(1) understand intimately software such as gparted,
(2) find or invent terms more specific than "format",
(3) convince maintainers to fix the bugs (adopt the new terms) and
(4) take credit for the revolutionary improvements.  =8~)


Try mounting it from the command line, and watch the output. That will
show you whether it is being fscked. However, FAT has no ability to
store when it was last fscked, so I would wonder why it was being
fscked at all.

In further observations, the problem is primarily in the old Fedora
based OLPC XO system.  Debian Bullseye usually recognizes and mounts
the f.s. on the SD card with no difficulty.  Rather than troubleshoot
the obsolete system I'll continue an effort to make DebXO work on the
XO 1.5 machine.

Thx,   ... P.
Having worked on a couple filesystems in a past life, all I can do is 
say +100 Peter.

"format" is one of the most overloaded words in our vocabulary.

Take care and stay well.


Cheers, Gene Heskett.
--
"There are four boxes to be used in defense of liberty:
 soap, ballot, jury, and ammo. Please use in that order."
-Ed Howdershelt (Author, 1940)
If we desire respect for the law, we must first make the law respectable.
 - Louis D. Brandeis
Genes Web page 

Re: VFAT vs. umask.

[peter]

From: Charles Curley 
Date: Sat, 30 Jul 2022 13:39:00 -0600
> The preparation of any storage medium requires at least two steps. To 
> format means ...
> 
> In another step one lays down a file system: ...

Understood, at a superficial level at least.  Haven't invented or 
implemented a file system.

> Microsoft conflated the two, leading to much confusion over the 
> years.

Even prominent software uses "format" loosely.  Eg. Gparted > 
Partition > Format to > ... .

If we really want to improve terminology we need to 
(1) understand intimately software such as gparted, 
(2) find or invent terms more specific than "format",
(3) convince maintainers to fix the bugs (adopt the new terms) and 
(4) take credit for the revolutionary improvements.  =8~)

> Try mounting it from the command line, and watch the output. That will
> show you whether it is being fscked. However, FAT has no ability to
> store when it was last fscked, so I would wonder why it was being
> fscked at all.

In further observations, the problem is primarily in the old Fedora 
based OLPC XO system.  Debian Bullseye usually recognizes and mounts 
the f.s. on the SD card with no difficulty.  Rather than troubleshoot 
the obsolete system I'll continue an effort to make DebXO work on the 
XO 1.5 machine.

Thx,   ... P.




mobile: +1 778 951 5147
VoIP: +1 604 670 0140
https://en.wikibooks.org/wiki/User:PeterEasthope

Re: VFAT vs. umask.

[peter]

> ... reformat the card to an ext format and you can forget about that.

According to several other messages alignment and block size are 
concerns in reformatting.

> So "primary store" probably means Master Copy of Your Data. 

https://en.wiktionary.org/wiki/primary#Adjective  sense 2.
https://en.wiktionary.org/wiki/master#Adjective  sense 2.
https://en.wiktionary.org/wiki/store#Noun  sense 4.

> ... copy it off the SD onto a hard drive to work on it, then put it 
> back (hopefully with care, not overwriting, and thorough flushing [1}.

I never do that.  I use a file on the SD just as if it's on a spinning 
hdd.

To shut the machine down I don't just pull the power plug.  I use 
shutdown so that files can be closed and devices can be unmounted.

> ... when the card fails[2], you have a backup already, almost by 
> definition.

Correct.  Rsync was mentioned a few days ago.

> They're cheap; I'd use two [SD cards] anyway. 

I have two.  As mentioned at the beginning of the thread, one is about 
a decade old and still works. The other purchased a few months ago.

Thx,  ... P.


mobile: +1 778 951 5147
VoIP: +1 604 670 0140
https://en.wikibooks.org/wiki/User:PeterEasthope

Re: VFAT vs. umask.

[peter]

In previous copies of this message I omitted attribution of the quotes.

From: David Wright 
Date: Sun, 31 Jul 2022 14:15:26 -0500
> ... reformat the card to an ext format and you can forget about that.

According to several other messages alignment and block size are 
concerns in reformatting.

> So "primary store" probably means Master Copy of Your Data. 

https://en.wiktionary.org/wiki/primary#Adjective  sense 2.
https://en.wiktionary.org/wiki/master#Adjective  sense 2.
https://en.wiktionary.org/wiki/store#Noun  sense 4.

> ... copy it off the SD onto a hard drive to work on it, then put it 
> back (hopefully with care, not overwriting, and thorough flushing [1}.

I never do that.  I use a file on the SD just as if it's on a spinning 
hdd.

To shut the machine down I don't just pull the power plug.  I use 
shutdown so that files can be closed and devices can be unmounted.

> ... when the card fails[2], you have a backup already, almost by 
> definition.

Correct.  Rsync was mentioned a few days ago.

> They're cheap; I'd use two [SD cards] anyway. 

I have two.  As mentioned at the beginning of the thread, one is about 
a decade old and still works. The other purchased a few months ago.

Thx,  ... P.


mobile: +1 778 951 5147
VoIP: +1 604 670 0140
https://en.wikibooks.org/wiki/User:PeterEasthope

Re: VFAT vs. umask.

[peter]

> ... reformat the card to an ext format and you can forget about that.

According to several other messages, alignment and block size are concerns.

> So "primary store" probably means Master Copy of Your Data. 

https://en.wiktionary.org/wiki/primary#Adjective  sense 2.
https://en.wiktionary.org/wiki/master#Adjective  sense 2.
https://en.wiktionary.org/wiki/store#Noun  sense 4.

> ... copy it off the SD onto a hard drive to work on it, then put it 
> back (hopefully with care, not overwriting, and thorough flushing [1}.

I never do that.  I use a file on the SD just as if on a spinning hdd.

To shut the machine down I don't just pull the power plug.  I use 
shutdown so that files can be closed and devices can be unmounted.

> ... when the card fails[2], you have a backup already, almost by 
> definition.

Correct.  Rsync was mentioned a few days ago.

> They're cheap; I'd use two [SD cards] anyway. 

I have two.  As mentioned at the beginning of the thread, one is about 
a decade old. The other purchased recently.

Thx,  ... P.


mobile: +1 778 951 5147
VoIP: +1 604 670 0140
https://en.wikibooks.org/wiki/User:PeterEasthope

Re: VFAT vs. umask.

[peter]

From: 
Date: Wed, 3 Aug 2022 06:58:27 +0200
> AFAIU, the only critical parameter for a partitioner is the alignment,
> anyway. 
> ...
> AFAIK there is no "protocol" for the media to tell your OS about its
> preferred block size, and (USB/MMC) flash storage cheats anyway (the
> reality is just much more horrible and complex: ...

Conclusion 
Ideally alignment and block size should be preserved but J. Doe has no 
practical and reliable means for that.  So change the FAT file system 
to ext if you wish, while accepting the risk of boundary misalignment.

Thanks,  ... P.




mobile: +1 778 951 5147
VoIP: +1 604 670 0140
https://en.wikibooks.org/wiki/User:PeterEasthope

Re: VFAT vs. umask.

[tomas]

On Tue, Aug 02, 2022 at 05:07:06PM -0700, pe...@easthope.ca wrote:

[...]

> 8 x 10^9 bytes / 1.6 x 10^7 sectors 
>   ~= 8/16 x 10^3 bytes/sector 
>   ~= 512 bytes/sector.
> A familiar old number.
> 
> Gparted also shows 4.00 MiB unallocated bytes at the front of the 
> device.

This is gparted's default.

> If gparted simply does "Partition > Format to > ext2", is there any 
> chance it will change alignment or block size?

AFAIU, the only critical parameter for a partitioner is the alignment,
anyway. I have no man page for gparted, but parted does have an
option for that.

The block size is something you tell your file system. Mkfs.ext4 has
that.

AFAIK there is no "protocol" for the media to tell your OS about its
preferred block size, and (USB/MMC) flash storage cheats anyway (the
reality is just much more horrible and complex: blocks are huge [1],
and you can partially overwrite them but only delete them in bulk,
yadda, yadda, so you've a processor-cum-firmware in that lowly stick
with its own (not always good) ideas).

Cheers

[1] Trusty Wikipedia has 16..512 KiB, but I guess this keeps
   growing: https://en.wikipedia.org/wiki/Flash_memory
-- 
t


signature.asc
Description: PGP signature

Re: VFAT vs. umask.

[peter]

From: The Wanderer 
Date: Sun, 31 Jul 2022 12:29:54 -0400
> The filesystem that's on the device when it's shipped from the factory
> is almost certainly already configured in this way. My understanding is
> that that is usually what is meant by saying that the "factory format"
> of a flash storage device is optimal.

>From all I've read, that's the accepted truth, although not stated so 
clearly on a Web site of Kingston, SanDisk & etc.  

> If you know what the block sizes are and know how to specify things
> properly when creating a filesystem, however, it should be entirely
> possible to create a different filesystem on the device which is also
> properly sized and aligned in that same optimal way.

Gparted shows First sector: 8192, Last sector: 15704063, 
Total sectors: 15695872.

8 x 10^9 bytes / 1.6 x 10^7 sectors 
  ~= 8/16 x 10^3 bytes/sector 
  ~= 512 bytes/sector.
A familiar old number.

Gparted also shows 4.00 MiB unallocated bytes at the front of the 
device.

If gparted simply does "Partition > Format to > ext2", is there any 
chance it will change alignment or block size?

Otherwise, how should the new file system set up without damaging 
those crucial parameters?

Thanks! ... P.

mobile: +1 778 951 5147
VoIP: +1 604 670 0140
https://en.wikibooks.org/wiki/User:PeterEasthope

Re: VFAT vs. umask.

[David Wright]

On Sun 31 Jul 2022 at 07:51:22 (-0700), pe...@easthope.ca wrote:
> From: Linux-Fan 
> Date: Sat, 30 Jul 2022 21:37:37 +0200
> > Formatting it to ext2 should work and not cause any issues ...
> 
> Other authorities claim "factory format" is optimal and wear of flash 
> storage is a concern. A revised "format" can impose worse conditions 
> for wear?  Does any manufacturer publish about this?  What is hope?  
> What is truth?
> 
> > Modern backup tools use their own archive/data formats ...
> 
> All that's needed here is a reliable copy on the HDD, of the files on 
> the SD.  If the SD fails I mount the part of the HDD, restore to 
> another SD and continue working.  Rsync is the most effective tool 
> I've found.  More advanced archiving is not needed.

The issue that brought you here is permissions, in which case:
reformat the card to an ext format and you can forget about that.

So "primary store" probably means Master Copy of Your Data. And you
copy it off the SD onto a hard drive to work on it, then put it
back (hopefully with care, not overwriting, and thorough flushing¹).

Speed is not an overriding issue, in the sense that the computer
will wait, whereas a movie camera, say, cannot. And you're not
doing random access on myriads of small files, but just sequential
copying of your data files.

Wear is not an overriding issue, as I assume you do a reasonable
amount of work on the hard drive data before you store the outcome
back onto the SD card. And when the card fails², you have a backup
already, almost by definition.

¹ just copying to a new file, then renaming, might not be a
  guarantee that the physical data is written on the card.

² They're cheap; I'd use two anyway. That covers the case where
  the SD fails in the motel at night, and the computer you used
  yesterday is by now fifty miles away.

Cheers,
David.

Re: VFAT vs. umask.

[tomas]

On Sun, Jul 31, 2022 at 12:29:54PM -0400, The Wanderer wrote:
> On 2022-07-31 at 10:51, pe...@easthope.ca wrote:
> 
> > From: Linux-Fan 
> > Date: Sat, 30 Jul 2022 21:37:37 +0200
> 
> >> Formatting it to ext2 should work and not cause any issues ...
> > 
> > Other authorities claim "factory format" is optimal and wear of flash
> > storage is a concern. A revised "format" can impose worse conditions
> > for wear?  Does any manufacturer publish about this?  What is hope?
> > What is truth?
> 
> I would interpret a statement like that as referring to block size and
> possibly alignment.

AFAIR (it's a while ago), the FATs have their "file allocation tables"
(aka structure information) at fixed offsets in the block device.

The flash storage processor might cater to that. Then, again, it might
not.

Cheers
-- 
t


signature.asc
Description: PGP signature

Re: VFAT vs. umask.

[The Wanderer]

On 2022-07-31 at 10:51, pe...@easthope.ca wrote:

> From: Linux-Fan 
> Date: Sat, 30 Jul 2022 21:37:37 +0200

>> Formatting it to ext2 should work and not cause any issues ...
> 
> Other authorities claim "factory format" is optimal and wear of flash
> storage is a concern. A revised "format" can impose worse conditions
> for wear?  Does any manufacturer publish about this?  What is hope?
> What is truth?

I would interpret a statement like that as referring to block size and
possibly alignment.

As I understand matters, flash storage is capable of writing only
block-at-a-time; if you try to write a smaller unit than one hardware
block, the device has to read the contents of the block into memory,
replace the part of it that you want to write with your updated
contents, and write back out the entire block.

In practice - again, as I understand matters - that has multiple
consequences. (If I've got things wrong, I'm sure people will jump in
and correct me.)

One is that if you have a filesystem on the device which is trying to
use a block size that is smaller than the hardware block size, every
write will result in this read-and-write-back behavior, which is slower
than a pure write.

Another is that if you have a filesystem on the device which is trying
to use a block size that is *larger* than the hardware block size,
nearly every write will result in writing more blocks than intended -
one or more full blocks, and at least one partial block which needs to
be read in and written back out. Not only is that slower than a pure
write, it also results in more separate instances of writing a block,
which puts more wear on the flash.

Another is that if you have a filesystem on the device whose blocks are
the same size as the hardware block size, but which is not *aligned* so
that its first block *starts* at the beginning of a hardware block,
*every* write will result in writing multiple hardware blocks - the one
where the filesystem block begins, and the one where it ends. That winds
up being even slower, and putting even more wear on the flash.

The net result is that it's better for both performance and longevity to
ensure that a filesystem on flash storage is using the same block size
as the underlying hardware *and* is properly aligned to the block
boundaries of that hardware.

The filesystem that's on the device when it's shipped from the factory
is almost certainly already configured in this way. My understanding is
that that is usually what is meant by saying that the "factory format"
of a flash storage device is optimal.

If you know what the block sizes are and know how to specify things
properly when creating a filesystem, however, it should be entirely
possible to create a different filesystem on the device which is also
properly sized and aligned in that same optimal way.

-- 
   The Wanderer

The reasonable man adapts himself to the world; the unreasonable one
persists in trying to adapt the world to himself. Therefore all
progress depends on the unreasonable man. -- George Bernard Shaw



signature.asc
Description: OpenPGP digital signature

Re: VFAT vs. umask.

[Linux-Fan]

pe...@easthope.ca writes:


From: Linux-Fan 
Date: Sat, 30 Jul 2022 21:37:37 +0200
> Formatting it to ext2 should work and not cause any issues ...

Other authorities claim "factory format" is optimal and wear of flash
storage is a concern. A revised "format" can impose worse conditions
for wear?  Does any manufacturer publish about this?  What is hope?
What is truth?


Reformatting (in the sense of mkfs.ext2) does not by itself cause excessive  
wear on the flash drive. Now if the cards were optimized to handle FAT and  
somehow interpret its structures cleverly then this could in theory cause  
longer life of the drives for FAT compared to ext2. So far we got some  
rumors about that but nothing definitive. I sincerely doubt that there are  
any advantages of FAT on SD beyond the compatibility. Because: Nowdays it  
is quite common to use these cards as general purpose storage. Encrypted  
storage of Android files come to mind. Here, the card cannot optimize beyond  
the block level which is what I guess it does in any case. Here are some  
resources that you could use for further research:


- 
https://www.reddit.com/r/raspberry_pi/comments/ex7dvo/quick_reminder_that_sd_cards_with_wearleveling/
- 
https://electronics.stackexchange.com/questions/27619/is-it-true-that-a-sd-mmc-card-does-wear-levelling-with-its-own-controller

I do not know of any case where an optimization for FAT became visible.  
Especially, having used SD and µSD cards for Linux root file systems for  
longer than a year without observing any issues, I would conclude that it is  
much more related to the quality of the flash than the file system in use.



> Modern backup tools use their own archive/data formats ...

All that's needed here is a reliable copy on the HDD, of the files on
the SD.  If the SD fails I mount the part of the HDD, restore to
another SD and continue working.  Rsync is the most effective tool
I've found.  More advanced archiving is not needed.


rsync on FAT will not preserve Unix permissions and most other metadata. I  
(personally) would not consider it “reliable” since I care about these  
metadata.


There are three ways around this:

* Don't care - If your workflow does not need any Unix permissions
  whatsoever then you might as well stick with FAT if none of its other
  limits are of concern (maximum file size for FAT32 comes to mind).

* Format using an Unix-aware file system like e.g. ext2 and then
  just copy the files (rsync works).

* Use a tool that keeps the metadata for you (like the suggested
  advanced backup tools).

If rsync works for you, then by all means stick with it :)

HTH
Linux-Fan

öö

[...]


pgpMJtSUSta_V.pgp
Description: PGP signature

Re: VFAT vs. umask.

[peter]

From: Linux-Fan 
Date: Sat, 30 Jul 2022 21:37:37 +0200
> Formatting it to ext2 should work and not cause any issues ...

Other authorities claim "factory format" is optimal and wear of flash 
storage is a concern. A revised "format" can impose worse conditions 
for wear?  Does any manufacturer publish about this?  What is hope?  
What is truth?

> Modern backup tools use their own archive/data formats ...

All that's needed here is a reliable copy on the HDD, of the files on 
the SD.  If the SD fails I mount the part of the HDD, restore to 
another SD and continue working.  Rsync is the most effective tool 
I've found.  More advanced archiving is not needed.

Thanks,   ... P.

mobile: +1 778 951 5147
VoIP: +1 604 670 0140
https://en.wikibooks.org/wiki/User:PeterEasthope

Re: VFAT vs. umask.

[David Wright]

On Sat 30 Jul 2022 at 09:13:55 (-0700), pe...@easthope.ca wrote:
> From: David Wright 
> Date: Fri, 29 Jul 2022 00:00:29 -0500
> > When you copy files that have varied permissions onto the FAT, you may
> > get warnings about permissions that can't be honoured. (IIRC, copying
> > ug=r,o= would not complain, whereas u=r,go= would.)
> 
> Primary store is an SD card.  Rsync is used for backup.  Therefore 
> this dilema.

I'm not sure what you mean by the term "primary store". If you're
just storing data on the card, then you'd need to be specific about
which features you miss in FAT.

> * In Linux, an ext file system avoids those complications.  To my 
> knowledge, all SD cards are preformatted with a FAT.  Therefore ext 
> requires reformatting.
> 
> * Most advice about flash storage is to avoid reformatting.  
> Unfortunately most or all of this advice is written by software 
> people; none, that I recall, from a flash storage manufacturer.

My use for SD cards is device storage, in phones, cameras and
players. I therefore let each device format its card in the first
instance, though I might modify, say, the LABEL afterwards. (The
Serial Number can be tricky to discover, because the slot and the
adapter can hide it.) AFAICT, if a device doesn't like what it finds,
it will happily reformat the card without necessarily asking.

Some of my USB sticks are frequently reformatted, for use as
installers, live systems, data files for TV, or whatever.
I've not observed any correlation with their failure.

BTW I don't know where you read this advice and, as so often,
no reference. Who are these "software people"?

One good reason for giving this advice /in the right context/
is that naive users could easily reformat the wrong device
and lose all their data. That doesn't apply here.

> My own experience, is one SD card about a decade old, reformatted to 
> ext2 when new and still working. A second SD purchased recently with 
> factory format unchanged seems very slow in mounting.  As if running 
> fsck before every mount.  -8~/

How are you determining the time it takes to mount it? Is it just
mounting it or also reading the top-level directory?

I assume this new card is big, and probably therefore formatted as
exFAT. Which method are using to mount it, the new kernel driver,
or the older FUSE? That might make a difference. (I don't use exFAT
myself, as currently it buys me nothing.)

> Certainly tempted to reformat the new card to ext2.  Information 
> always welcome.  Knowledge even better.

Why ext2 rather than 3 or 4?

Sorry, I don't have particular knowledge, particularly when having
to guess what you have and how you're using it. (The Subject line
seemed to be a false direction.)

Cheers,
David.

Re: VFAT vs. umask.

[Tixy]

On Sat, 2022-07-30 at 22:26 +0200, to...@tuxteam.de wrote:
> 
> Rumour goes that the processor in the stick/card can cope better with
> FAT. I don't know whether it's true, though.

A long time ago I heard something along the lines that the first blocks
in the storage device (where the the FAT would go) were stored on a
higher endurance storage device. I don't know how true or widespread
that practice was.

-- 
Tixy

Re: VFAT vs. umask.

[tomas]

On Sat, Jul 30, 2022 at 09:37:37PM +0200, Linux-Fan wrote:
> pe...@easthope.ca writes:
> 
> > David,
> > thanks for the reply.
> > 
> > From: David Wright 
> > Date: Fri, 29 Jul 2022 00:00:29 -0500
> > > When you copy files that have varied permissions onto the FAT, you may
> > > get warnings about permissions that can't be honoured. (IIRC, copying
> > > ug=r,o= would not complain, whereas u=r,go= would.)
> > 
> > Primary store is an SD card.  Rsync is used for backup.  Therefore
> > this dilema.
> > 
> > * In Linux, an ext file system avoids those complications.  To my
> > knowledge, all SD cards are preformatted with a FAT.  Therefore ext
> > requires reformatting.
> 
> FAT or exFAT depending on size, yes.
> 
> > * Most advice about flash storage is to avoid reformatting.
> > Unfortunately most or all of this advice is written by software
> > people; none, that I recall, from a flash storage manufacturer.
> 
> The only reason for not chosing to format the SD card with a Linux file
> system is the lack of compatibility: Other systems (think Windows, Mac,
> etc.) will not be able to use the SD card anymore because it will appear
> „wrongly formatted” to them.

Rumour goes that the processor in the stick/card can cope better with
FAT. I don't know whether it's true, though.

Cheers
-- 
t


signature.asc
Description: PGP signature

Re: VFAT vs. umask.

[Charles Curley]

On Sat, 30 Jul 2022 09:13:55 -0700
pe...@easthope.ca wrote:

> * In Linux, an ext file system avoids those complications.  To my 
> knowledge, all SD cards are preformatted with a FAT.  Therefore ext 
> requires reformatting.

Not quite. The preparation of any storage medium requires at least two
steps. To format means to lay down sector information on each track,
including checksums, inter-record gaps, metadata, and other minutia.
Today, this is done at the factory, e.g. floppy disks and hard drives.

In another step one lays down a file system: allocate space for a root
directory, that directory's metadata, lay down a journal, and in
many cases redundant superblocks to allow for recovery in case the
main ones are clobbered, etc..

Microsoft conflated the two, leading to much confusion over the years.

So, no, replacing a FAT partition with an ext partition does not
require reformatting. It only involves laying down a new file system.

Any time you write to flash memory (SD card, USB stick, SSD, whatever)
you shorten its life, as flash memory is only good for so many writes.
When I started working with flash memory, it only stored 256 bits
(not bytes), and it was good for only a thousand writes or so. It has
gotten significantly better since. So if you replace a FAT partition,
you may want to avoid writing to every sector of the partition.

> My own experience, is one SD card about a decade old, reformatted to 
> ext2 when new and still working.

This is typical. I have SSDs in daily use of similar age. On the other
tentacle, I've had cheap USB sticks die after a dozen writes.

> A second SD purchased recently with 
> factory format unchanged seems very slow in mounting.  As if running 
> fsck before every mount.

Try mounting it from the command line, and watch the output. That will
show you whether it is being fscked. However, FAT has no ability to
store when it was last fscked, so I would wonder why it was being
fscked at all.

-- 
Does anybody read signatures any more?

https://charlescurley.com
https://charlescurley.com/blog/

Re: VFAT vs. umask.

[Linux-Fan]

pe...@easthope.ca writes:


David,
thanks for the reply.

From: David Wright 
Date: Fri, 29 Jul 2022 00:00:29 -0500
> When you copy files that have varied permissions onto the FAT, you may
> get warnings about permissions that can't be honoured. (IIRC, copying
> ug=r,o= would not complain, whereas u=r,go= would.)

Primary store is an SD card.  Rsync is used for backup.  Therefore
this dilema.

* In Linux, an ext file system avoids those complications.  To my
knowledge, all SD cards are preformatted with a FAT.  Therefore ext
requires reformatting.


FAT or exFAT depending on size, yes.


* Most advice about flash storage is to avoid reformatting.
Unfortunately most or all of this advice is written by software
people; none, that I recall, from a flash storage manufacturer.


The only reason for not chosing to format the SD card with a Linux file  
system is the lack of compatibility: Other systems (think Windows, Mac,  
etc.) will not be able to use the SD card anymore because it will appear  
„wrongly formatted” to them.



My own experience, is one SD card about a decade old, reformatted to
ext2 when new and still working. A second SD purchased recently with
factory format unchanged seems very slow in mounting.  As if running
fsck before every mount.  -8~/

Certainly tempted to reformat the new card to ext2.  Information
always welcome.  Knowledge even better.


Formatting it to ext2 should work and not cause any issues such as long as  
you remain in a “Linux world”. If the data should still be accessible from  
other systems, consider packing it into archives and storing the archives on  
the FAT file system. That is something that I used to do and that has worked  
reliably for me in the past. It does not allow for incremental updates (like  
rsync would) by default, though.


Modern backup tools use their own archive/data formats and can thus store  
Linux metadata (permissions, ownerships etc.) on file systems that are  
incapable of doing this (like FAT, cloud storage etc.). I have written my  
notes about a few such tools here:

https://masysma.lima-city.de/37/backup_tests_borg_bupstash_kopia.xhtml

HTH
Linux-Fan

öö


pgpwYqGXMmTxt.pgp
Description: PGP signature

Re: VFAT vs. umask.

[peter]

David, 
thanks for the reply.

From: David Wright 
Date: Fri, 29 Jul 2022 00:00:29 -0500
> When you copy files that have varied permissions onto the FAT, you may
> get warnings about permissions that can't be honoured. (IIRC, copying
> ug=r,o= would not complain, whereas u=r,go= would.)

Primary store is an SD card.  Rsync is used for backup.  Therefore 
this dilema.

* In Linux, an ext file system avoids those complications.  To my 
knowledge, all SD cards are preformatted with a FAT.  Therefore ext 
requires reformatting.

* Most advice about flash storage is to avoid reformatting.  
Unfortunately most or all of this advice is written by software 
people; none, that I recall, from a flash storage manufacturer.

My own experience, is one SD card about a decade old, reformatted to 
ext2 when new and still working. A second SD purchased recently with 
factory format unchanged seems very slow in mounting.  As if running 
fsck before every mount.  -8~/

Certainly tempted to reformat the new card to ext2.  Information 
always welcome.  Knowledge even better.

Thx,... P.


mobile: +1 778 951 5147
VoIP: +1 604 670 0140
https://en.wikibooks.org/wiki/User:PeterEasthope

Re: VFAT vs. umask.

[peter]

From: 
Date: Fri, 29 Jul 2022 06:22:53 +0200
> No. A FAT file system has no permissions (and no user/group ownership).
> All is faked one layer above.

Understood.  

Aren't we saying the same thing in two ways. In natural language, 777 
just means anyone can read & write & execute. 555 means anyone can 
read & execute;  none can write. Those are the possibilities with FAT.

Thx,... P.

mobile: +1 778 951 5147
VoIP: +1 604 670 0140
https://en.wikibooks.org/wiki/User:PeterEasthope

Re: VFAT vs. umask.

[David Wright]

On Thu 28 Jul 2022 at 14:49:03 (-0700), pe...@easthope.ca wrote:
> https://tldp.org/FAQ/Linux-FAQ/partitions.html has this example.
> 
>  $ mkdir /dos $
> mount -t msdos -o conv=text,umask=022,uid=100,gid=100 /dev/hda3 /dos
> 
> Therefore a new file receives permissions 755.  Correct?

With these options, that's what the driver will present to you.
But the filesystem will just write a read/write file.
(BTW I wouldn't use the parameters in that somewhat dated example.)

> But a FAT file has only all-user read-only permissions.  Either 777 or 555.  
> Correct?

If you mean the file as stored in the filesystem, yes, it either
has the readonly attribute or it doesn't.

> Explanation?

The permissions as you see them are determined by the mount parameters
that you specified with umask (or fmask/dmask), except that a readonly
attribute in the filesystem will lose you any w permissions you might
have expected from the mask.

When you copy files that have varied permissions onto the FAT, you may
get warnings about permissions that can't be honoured. (IIRC, copying
ug=r,o= would not complain, whereas u=r,go= would.)

I don't know whether/which permissions/attributes are honoured when
you're root (for you, all the time), as I only mkdosfs or dd FAT
filesystems as root.

Cheers,
David.

Re: VFAT vs. umask.

[tomas]

On Thu, Jul 28, 2022 at 02:49:03PM -0700, pe...@easthope.ca wrote:
> https://tldp.org/FAQ/Linux-FAQ/partitions.html has this example.
> 
>  $ mkdir /dos $
> mount -t msdos -o conv=text,umask=022,uid=100,gid=100 /dev/hda3 /dos
> 
> Therefore a new file receives permissions 755.  Correct?
> 
> But a FAT file has only all-user read-only permissions.  Either 777 or 555.  
> Correct?

No. A FAT file system has no permissions (and no user/group ownership).
All is faked one layer above.

Cheers
-- 
t


signature.asc
Description: PGP signature

VFAT vs. umask.

[peter]

https://tldp.org/FAQ/Linux-FAQ/partitions.html has this example.

 $ mkdir /dos $
mount -t msdos -o conv=text,umask=022,uid=100,gid=100 /dev/hda3 /dos

Therefore a new file receives permissions 755.  Correct?

But a FAT file has only all-user read-only permissions.  Either 777 or 555.  
Correct?

Explanation?

Thanks, ... P.




mobile: +1 778 951 5147
VoIP: +1 604 670 0140
https://en.wikibooks.org/wiki/User:PeterEasthope

Re: Problème avec umask dans fstab sur un sous-dossier

[Polyna-Maude Racicot-Summerside]

On 2021-11-28 3:25 p.m., lists.deb...@netc.eu wrote:
> Merci du retour Didier, 
> 
> Je vais essayer de voir tout ça pendant la semaine. 
> 
> Entre-temps ce que je n'arrive pas trop à comprendre est-ce pourquoi 2
> dossiers similaires (User1 et User2) finissent par avoir des permissions
> différentes, même si sous Windows ils ont exactement les mêmes
> paramètres de sécurité et droits d'accès... 
> 
Parce qu'ils ont pas les mêmes paramètres de correspondance sous Linux ?
Windows voit bien ses paramètres car c.est son propre système.
Mais
Linux reconnait peut-être mal l'usager 2 ou celui-ci a un umask différent.
> J'ai vraiment du mal à comprendre... 
> 
> 
> 
> Le 28/11/2021 à 21:10:05, didier gaumet  a écrit :
> 
> Le dimanche 28 novembre 2021 à 17:51 +0100, lists.deb...@netc.eu a
> écrit :
> > Merci de vos retour :)
> >
> > Je suis allé sur le site NTFS-3G site, j'ai lu toute la doc relative
> > au fichier usermap, mais j'ai eu deux soucis :
> >  - je n'ai pas réussi à trouver le fichier zip pour Windows
> 
> Il (JP Andre) a changé l'organistaion de son site perso et le site
> principal NTFS-3G sur GitHub ne propose que les sources.
> Je pense que le zip en question pour Windows version 64 bits est celui-
> là:
> 
> https://jp-andre.pagesperso-orange.fr/download-rpm.html#ntfsprogs-windows-64-2017.3.23AR.6.zip
> 
> 
> >  - une foi que j'ai essayé de lancer l'exécution sur Debian, je n'ai
> > pas trop compris les résultats ni comment les utiliser : j'ai
> > beaucoup des fichiers et sous-dossiers dans ces dossiers, il semble
> > que je dois passer sur chaque un afin de vérifier la permission et
> > faire manuellement la correspondance ?
> 
> Je pense qu'il faut que tu commences par paramétrer à la main sous
> Windows dans un fichier de config une correspondance pour faire
> connaître à Windows à quel utilisateur Windows spécifique il doit
> attrribuer les droits d'accès attribués sous Linux à un utilisateur
> Linux spécifique. Du style:
>  Marc_linux:Marc_Windows
> faire la même chose pour les groupes
> 
> Ensuite, je me suis arrêté là parce que ça m'aurait pris trop de temps
> pour te dire quoi faire exactement (vu que c'est un contexte qui est
> nouveau pour moi, il me faut comprendre de quoi il s'agit et je suis
> lent), désolé, il va falloir que tu creuses toi-même :-)
> 
> 
> 

-- 
Polyna-Maude R.-Summerside
-Be smart, Be wise, Support opensource development



OpenPGP_signature
Description: OpenPGP digital signature

Re: Problème avec umask dans fstab sur un sous-dossier

[didier gaumet]

Je pense qu tu obtiendras des éléments de réponse en lisant les docs
sur les sites, officiel de ntfs-3g, et persos de ses développeurs,
ainsi que wikipedia et autres sources comparables

Ce que je suppose c'est que NTFS étant un système de fichiers
propriétaire dont les spécifications sont fermées, ntfs-3g doit être
basé sur du retro-engineering (donc en partie divination et bidouilles)
et j'ai cru comprendre que certaines caractéristiques connues de ntfs
n'y sont pas implémentées en standard, voire même pas propsées par des
extensions.

Peut-être pourrais-tu étudier la possibilité d'utiliser un autre
système de fichiers local (le support ExFat (Microsoft a libéré les
specs ExFat il y a quelques temps)  pour Linux en espace noyau plutôt
qu'espace utilsateur semble être présent à partir du noyau Linux 5.15
(de mémoire)), ou d'utiliser un NAS physique ou un serveur de fichiers
externe. Tout ça dépend de tes besoins de sécurité, entre autres. 
Les problématiques réseau et moi ça fait 2 donc ne te fie pas
aveuglément à ce que je dis (je n'ai jamais monté de serveur NFS ou
Samba(CIFS), je n'ai même pas utilisé de clients de ces protocoles,
c'est te dire à quel point je maîtrise)

Bon courage :-) 

Re: Problème avec umask dans fstab sur un sous-dossier

[lists . debian]

Merci du retour Didier, Je vais essayer de voir tout ça pendant la semaine. 
Entre-temps ce que je n'arrive pas trop à comprendre est-ce pourquoi 2 dossiers 
similaires (User1 et User2) finissent par avoir des permissions différentes, 
même si sous Windows ils ont exactement les mêmes paramètres de sécurité et 
droits d'accès... J'ai vraiment du mal à comprendre... Le 28/11/2021 à 
21:10:05, didier gaumet  a écrit : > > Le dimanche 28 
novembre 2021 à 17:51 +0100, lists.deb...@netc.eu a écrit : > Merci de vos 
retour :) > > Je suis allé sur le site NTFS-3G site, j'ai lu toute la doc 
relative > au fichier usermap, mais j'ai eu deux soucis : > - je n'ai pas 
réussi à trouver le fichier zip pour Windows Il (JP Andre) a changé 
l'organistaion de son site perso et le site principal NTFS-3G sur GitHub ne 
propose que les sources. Je pense que le zip en question pour Windows version 
64 bits est celui- là: 
https://jp-andre.pagesperso-orange.fr/download-rpm.html#ntfsprogs-windows-64-2017.3.23AR.6.zip
 > - une foi que j'ai essayé de lancer l'exécution sur Debian, je n'ai > pas 
trop compris les résultats ni comment les utiliser : j'ai > beaucoup des 
fichiers et sous-dossiers dans ces dossiers, il semble > que je dois passer sur 
chaque un afin de vérifier la permission et > faire manuellement la 
correspondance ? Je pense qu'il faut que tu commences par paramétrer à la main 
sous Windows dans un fichier de config une correspondance pour faire connaître 
à Windows à quel utilisateur Windows spécifique il doit attrribuer les droits 
d'accès attribués sous Linux à un utilisateur Linux spécifique. Du style: 
Marc_linux:Marc_Windows faire la même chose pour les groupes Ensuite, je me 
suis arrêté là parce que ça m'aurait pris trop de temps pour te dire quoi faire 
exactement (vu que c'est un contexte qui est nouveau pour moi, il me faut 
comprendre de quoi il s'agit et je suis lent), désolé, il va falloir que tu 
creuses toi-même :-)

Re: Problème avec umask dans fstab sur un sous-dossier

[didier gaumet]

Le dimanche 28 novembre 2021 à 17:51 +0100, lists.deb...@netc.eu a
écrit :
> Merci de vos retour :)
>  
> Je suis allé sur le site NTFS-3G site, j'ai lu toute la doc relative
> au fichier usermap, mais j'ai eu deux soucis :
>  - je n'ai pas réussi à trouver le fichier zip pour Windows

Il (JP Andre) a changé l'organistaion de son site perso et le site
principal NTFS-3G sur GitHub ne propose que les sources.
Je pense que le zip en question pour Windows version 64 bits est celui-
là:
https://jp-andre.pagesperso-orange.fr/download-rpm.html#ntfsprogs-windows-64-2017.3.23AR.6.zip


>  - une foi que j'ai essayé de lancer l'exécution sur Debian, je n'ai
> pas trop compris les résultats ni comment les utiliser : j'ai
> beaucoup des fichiers et sous-dossiers dans ces dossiers, il semble
> que je dois passer sur chaque un afin de vérifier la permission et
> faire manuellement la correspondance ?

Je pense qu'il faut que tu commences par paramétrer à la main sous
Windows dans un fichier de config une correspondance pour faire
connaître à Windows à quel utilisateur Windows spécifique il doit
attrribuer les droits d'accès attribués sous Linux à un utilisateur
Linux spécifique. Du style:
 Marc_linux:Marc_Windows
faire la même chose pour les groupes

Ensuite, je me suis arrêté là parce que ça m'aurait pris trop de temps
pour te dire quoi faire exactement (vu que c'est un contexte qui est
nouveau pour moi, il me faut comprendre de quoi il s'agit et je suis
lent), désolé, il va falloir que tu creuses toi-même :-)

Re: Problème avec umask dans fstab sur un sous-dossier

[lists . debian]

Merci de vos retour :) Je suis allé sur le site NTFS-3G site, j'ai lu toute la 
doc relative au fichier usermap, mais j'ai eu deux soucis : - je n'ai pas 
réussi à trouver le fichier zip pour Windows - une foi que j'ai essayé de 
lancer l'exécution sur Debian, je n'ai pas trop compris les résultats ni 
comment les utiliser : j'ai beaucoup des fichiers et sous-dossiers dans ces 
dossiers, il semble que je dois passer sur chaque un afin de vérifier la 
permission et faire manuellement la correspondance ? Sinon j'ai aussi fait 
quelques tests, si je n'arrive pas à résoudre tout ça cette semaine, je pense 
que le plus simple serait de récréer ces dossiers et copier les fichiers. Dans 
mes tests je n'ai pas eu de soucis avec des nouveaux dossiers, qu'ils soient 
créés sous Windows ou Linux... Cordialement, Marc Le 2021-11-26 12:29, didier 
gaumet  a écrit : > > à Marc: pour compléter 
l'intervention de Hugues, tu peux regarder la > page man de ntfs-3g (par défaut 
quand tu montes du ntfs dans Debian, > c'est ntfs-3g qui est a l'oeuvre) > 
http://manpages.ubuntu.com/manpages/bionic/en/man8/mount.ntfs.8.html > > où il 
est suggéré de passer par une liste de correspondance > d'utilisateurs 
(paragraphe "user mapping"). Un tuto ici: > 
https://jp-andre.pagesperso-orange.fr/ntfsusermap.html > > ça pourrait s'avérer 
utile si par exemple tu ne te contentes pas de > consulter tes données 
partagées sous Linux mais que tu les modifies, > auquel cas tu pourrais avoir 
des problèmes d'accès une fois revenu sous > Windows > > (j'ai lu en diagonale 
dans les 2 cas et je n'ai jamais fait ça, donc > pas la peine de croire que je 
connais bien la chose) > > Je suis loin d'être expert dans le partage de 
systèmes de fichiers mais > dès que tu partages des données d'un système 
d'exploitation à un autre, > qu'il soit ou non sur le même PC, que le système 
de fichiers soit > identique ou non, du moment que chaque système 
d'exploitation gère > localement ses utilisateurs au lieu de passer par un 
serveur annuaire > d'utilisateurs, tu peux rencontrer ce genre de problème (un 
disque > externe en ext4 sur 2 PC linux différents avec le même utilisateur > 
"marc" ayant un UID différent sur chaque PC devrait représenter un > exemple 
correct). > > >

Re: Problème avec umask dans fstab sur un sous-dossier

[didier gaumet]

à Marc: pour compléter l'intervention de Hugues, tu peux regarder la
page man de ntfs-3g (par défaut quand tu montes du ntfs dans Debian,
c'est ntfs-3g qui est a l'oeuvre) 
http://manpages.ubuntu.com/manpages/bionic/en/man8/mount.ntfs.8.html

où il est suggéré de passer par une liste de correspondance
d'utilisateurs (paragraphe "user mapping"). Un tuto ici:
https://jp-andre.pagesperso-orange.fr/ntfsusermap.html

ça pourrait s'avérer utile si par exemple tu ne te contentes pas de
consulter tes données partagées sous Linux mais que tu les modifies,
auquel cas tu pourrais avoir des problèmes d'accès une fois revenu sous
Windows 

(j'ai lu en diagonale dans les 2 cas et je n'ai jamais fait ça, donc
pas la peine de croire que je connais bien la chose)

Je suis loin d'être expert dans le partage de systèmes de fichiers mais
dès que tu partages des données d'un système d'exploitation à un autre,
qu'il soit ou non sur le même PC, que le système de fichiers soit
identique ou non, du moment que chaque système d'exploitation gère
localement ses utilisateurs au lieu de passer par un serveur annuaire
d'utilisateurs, tu peux rencontrer ce genre de problème (un disque
externe en ext4 sur 2 PC linux différents avec le même utilisateur
"marc" ayant un UID différent sur chaque PC devrait représenter un
exemple correct).

Re : Problème avec umask dans fstab sur un sous-dossier

[Hugues Larrive]

‐‐‐ Original Message ‐‐‐
Le jeudi 25 novembre 2021 à 13:22, lists.deb...@netc.eu  
a écrit :

>  
> Bonjour à tous,
> 

> Je reviens encore une fois vous demander de l'aide.
> 

> J'ai un ordinateur dual boot Windows 10 / Debian 11.
> 

> Ce PC possède 2 disques, une SSD avec les 2 systèmes et un HDD où sont les 
> autres fichiers (documents, images, musique,...).
> 

> Mon idée est de partager ce HDD entre Windows et Debian. Pour le faire, dans 
> mon fichier fstab, j'ai rajouté la ligne suivante :
> 

> UUID=ACB23705B236D414   /mnt/windows    ntfs-3g defaults,umask=000    0   0
> 

> Les dossiers présents montent correctement dans le dossier /mnt/windows :
> 

> $ ls -l /mnt/windows/
> total 80
> drwxrwxrwx 1 root root 4096 14 nov. 20:20 '$RECYCLE.BIN'
> drwxrwxrwx 1 root root 4096 24 nov. 15:59 CloudStation
> drwxrwxrwx 1 root root 4096 21 nov. 11:44 Documents
> -rwxrwxrwx 1 root root 8192 25 juin 08:15 DumpStack.log.tmp
> drwxrwxrwx 1 root root 4096 22 nov. 20:41 Images
> drwxrwxrwx 1 root root 4096 24 nov. 11:53 Music
> drwxrwxrwx 1 root root 8192 23 nov. 06:21 'System Volume Information'
> drwxrwxrwx 1 root root 40960 21 nov. 22:22 Téléchargements
> drwxrwxrwx 1 root root 4096 21 nov. 19:44 Videos
>  
> Tout y est et les permissions sont bien "rwxrwxrwx".
> 

> Mon problème est dans le dossier Documents :
> 

> $ ls -l /mnt/windows/Documents/
> total 117
> drwxrwxrwx 1 root root 16384 24 nov. 15:59 User1
> -rwxrwxrwx 1 root root 0 26 nov. 2020 Default.rdp
> -rwxrwxrwx 1 root root 432 11 mars 2021 desktop.ini
> dr-xr-xr-x 1 root root 40960 24 nov. 15:59 User2
> drwxrwxrwx 1 root root 16384 24 nov. 16:00 Public
> drwxrwxrwx 1 root root 4096 24 nov. 15:59 User3
> dr-xr-xr-x 1 root root 20480 21 nov. 12:05 Scan
> -rwxrwxrwx 1 root root 18432 4 déc. 2016 Thumbs.db
> drwxrwxrwx 1 root root 0 16 nov. 23:13 'Unified Remote'
>  
> En comparant User1 avec User2, on voit bien que User1 a les mêmes permissions 
> que Documents, mas User2 ne permets pas l'écriture, ce qui m'empêche de gérer 
> ce dossier depuis Debian...
> 

> Auriez-vous une idée de pourquoi les permissions de ces sous-dossiers ne sont 
> pas les mêmes ?

Bonjour,

La page de man dit:
umask=value
  Set  the  bitmask of the file and directory permissions that are 
not present.

Donc il faudrait voir si User2 n'a pas des permissions différentes sous Windows.

> Je vous remercie par avance de vos retours,
> 

> Cordialement,
> Marc
>
Corialement,
Hugues

publickey - hlarrive@pm.me - 0xE9429B87.asc
Description: application/pgp-keys


signature.asc
Description: OpenPGP digital signature

Problème avec umask dans fstab sur un sous-dossier

[lists . debian]

Bonjour à tous, Je reviens encore une fois vous demander de l'aide. J'ai un 
ordinateur dual boot Windows 10 / Debian 11. Ce PC possède 2 disques, une SSD 
avec les 2 systèmes et un HDD où sont les autres fichiers (documents, images, 
musique,...). Mon idée est de partager ce HDD entre Windows et Debian. Pour le 
faire, dans mon fichier fstab, j'ai rajouté la ligne suivante : 
UUID=ACB23705B236D414 /mnt/windows ntfs-3g defaults,umask=000 0 0 Les dossiers 
présents montent correctement dans le dossier /mnt/windows : $ ls -l 
/mnt/windows/ total 80 drwxrwxrwx 1 root root 4096 14 nov. 20:20 '$RECYCLE.BIN' 
drwxrwxrwx 1 root root 4096 24 nov. 15:59 CloudStation drwxrwxrwx 1 root root 
4096 21 nov. 11:44 Documents -rwxrwxrwx 1 root root 8192 25 juin 08:15 
DumpStack.log.tmp drwxrwxrwx 1 root root 4096 22 nov. 20:41 Images drwxrwxrwx 1 
root root 4096 24 nov. 11:53 Music drwxrwxrwx 1 root root 8192 23 nov. 06:21 
'System Volume Information' drwxrwxrwx 1 root root 40960 21 nov. 22:22 
Téléchargements drwxrwxrwx 1 root root 4096 21 nov. 19:44 Videos Tout y est et 
les permissions sont bien "rwxrwxrwx". Mon problème est dans le dossier 
Documents : $ ls -l /mnt/windows/Documents/ total 117 drwxrwxrwx 1 root root 
16384 24 nov. 15:59 User1 -rwxrwxrwx 1 root root 0 26 nov. 2020 Default.rdp 
-rwxrwxrwx 1 root root 432 11 mars 2021 desktop.ini dr-xr-xr-x 1 root root 
40960 24 nov. 15:59 User2 drwxrwxrwx 1 root root 16384 24 nov. 16:00 Public 
drwxrwxrwx 1 root root 4096 24 nov. 15:59 User3 dr-xr-xr-x 1 root root 20480 21 
nov. 12:05 Scan -rwxrwxrwx 1 root root 18432 4 déc. 2016 Thumbs.db drwxrwxrwx 1 
root root 0 16 nov. 23:13 'Unified Remote' En comparant User1 avec User2, on 
voit bien que User1 a les mêmes permissions que Documents, mas User2 ne permets 
pas l'écriture, ce qui m'empêche de gérer ce dossier depuis Debian... 
Auriez-vous une idée de pourquoi les permissions de ces sous-dossiers ne sont 
pas les mêmes ? Je vous remercie par avance de vos retours, Cordialement, Marc