Re: Which subdirectory for a usedr-specific executable?

[David Christensen]

On 10/3/24 05:51, Richard Owlett wrote:
Is there standard/recommended location for an executable to be used by 
only a one user?

In my case it should be under /home/richard/ .
But where?

TIA




It would help if you told us about the executable and the context for 
its use -- e.g. self-contain binary for shell usage, one of several 
programs included in a large graphical user interface application suite 
that requires a specific desktop environment, etc.



David

Re: backup of backup or alternating backups?

[David Christensen]

On 9/30/24 09:39, Default User wrote:

Hi!

On a thread at another mailing list, someone mentioned that they, each
day, alternate doing backups between two external usb drives. That got
me to thinking (which is always dangerous) . . .

I have a full backup on usb external drive A, "refreshed" daily using
rsnapshot. Then, every day, I use rsync to make usb external drive B an
"exact" copy of usb external drive A. It seemed to be a good idea,
since if drive A fails, I can immediately plug in drive B to replace
it, with no down time, and nothing lost.

But of course, any errors on drive A propagate daily to drive B.

So, is there a consensus on which would be better:
1) continue to "mirror" drive A to drive B?
or,
2) alternate backups daily between drives A and B?



I migrated my data to a dedicated ZFS file server several years ago, in 
part due to advanced ZFS backup features -- snapshots, compression, 
de-duplication, replication, etc..  I used FreeBSD, but Debian has ZFS 
and should be able to do the same thing.



My live server has a ZFS pool with two striped mirrors of two 3 TB HDD's 
each and a special mirror of two 180 GB SSD's:


2024-09-30 16:44:38 toor@f5 ~
# zpool iostat -v p5
  capacity operations bandwidth
poolalloc   free   read  write   read  write
--  -  -  -  -  -  -
p5  3.19T  2.39T 49  2  28.4M  69.2K
  mirror-0  1.58T  1.14T 21  0  14.0M  10.7K
gpt/hdd1.eli-  -  8  0  6.99M  5.35K
gpt/hdd2.eli-  - 12  0  6.99M  5.35K
  mirror-1  1.58T  1.13T 20  0  14.0M  10.4K
gpt/hdd3.eli-  - 10  0  7.00M  5.20K
gpt/hdd4.eli-  -  9  0  7.00M  5.20K
special -  -  -  -  -  -
  mirror-2  29.4G   120G  7  2   408K  48.1K
gpt/ssd1.eli-  -  3  1   204K  24.1K
gpt/ssd2.eli-  -  3  1   204K  24.1K
--  -  -  -  -  -  -


The 'special' SSD mirror stores metadata, which improves overall 
performance.



I create ZFS filesystems for groups of data -- Samba users, CVS 
repository, rsync(1) backups of various non-ZFS filesystems, raw disk 
image backups, etc..



ZFS has various properties that you can tune for each filesystem.  Here 
is the filesystem for my Samba data:


2024-09-30 16:50:07 toor@f5 ~
# zfs get all p5/samba/dpchrist | sort | egrep 'NAME|inherited'
NAME   PROPERTY   VALUE  SOURCE
p5/samba/dpchrist  atime  off 
inherited from p5
p5/samba/dpchrist  com.sun:auto-snapshot  true 
inherited from p5
p5/samba/dpchrist  compressionon 
inherited from p5
p5/samba/dpchrist  dedup  verify 
inherited from p5
p5/samba/dpchrist  mountpoint /var/local/samba/dpchrist 
inherited from p5/samba
p5/samba/dpchrist  special_small_blocks   16K 
inherited from p5



'atime' is off to eliminate metadata writes when files and directories 
are read.



'com.sun:auto-snapshot' is true so that zfs-auto-snapshot(8) run via 
crontab(1) will find this filesystem, take snapshots periodically 
(daily, monthly, yearly), and manage (prune) those snapshots:


2024-09-30 16:54:00 toor@f5 ~
# crontab -l
 9 3 * * * /usr/local/sbin/zfs-auto-snapshot -k d 40
21 3 1 * * /usr/local/sbin/zfs-auto-snapshot -k m 99
27 3 1 1 * /usr/local/sbin/zfs-auto-snapshot -k y 99


I currently have 96 snapshots (e.g. backups) of the above filesystem 
going back three and a half years:


2024-09-30 16:59:48 dpchrist@f5 ~
$ ls -d /var/local/samba/dpchrist/.zfs/snapshot/zfs-auto-snap_[dmy]* | wc -l
  96

2024-09-30 17:01:12 dpchrist@f5 ~
$ ls -dt /var/local/samba/dpchrist/.zfs/snapshot/zfs-auto-snap_[dmy]* | 
tail -n 1

/var/local/samba/dpchrist/.zfs/snapshot/zfs-auto-snap_m-2020-03-01-00h21


'compression' is on so that compressible files are compressed.  (The 
default compression algorithm will skip files that are incompressible.)



'dedup' is on so that duplicate blocks are saved only once within the 
pool.  De-duplication metadata is stored on the pool 'special' SSD 
mirror, which improves de-duplication performance.



'special_small_blocks' is set to 16K so that files of size 16 KiB and 
smaller are stored on the pool 'special' SSD mirror, which improves 
small file read and write performance.



I have a backup server with matching pool construction.  I periodically 
replicate live server snapshots to the backup server (via SSH pull and 
pre-shared keys).  I would like to automate this task.



Both servers have SATA HDD mobile rack bays:

https://www.startech.com/en-us/hdd/drw150satbk


I have a pair of 6

Re: Copying installer ISO to USB Flash

[David Christensen]

On 9/20/24 07:52, Richard Owlett wrote:
Having machines with different constraints I have downloaded DVD1 and 
Netinst ISO's. I have flash drives with obsolete ISO's. For reference I 
have [ https://www.debian.org/CD/faq/#write-usb ] available.


Questions:
1. Do the flash drives require any prep?



I run a homebrew script that zeroes the dirty blocks, to prevent future 
confusion.




    [ Gparted gives warning messages on both. ]



Please post a console session if you desire comments.



2. I've casually followed recent discussion on appropriate dd options.
    What was the conclusion?
    What was the subject line {i have local copies}?



I suggest following the instructions given at the URL you cited, above.



3. Not having done a "from scratch" install recently, is there something
    I haven't thought to ask?



If the target computer has a Wi-Fi adapter, but no Ethernet adapter, 
firmware used to be an issue.  It looks like the Debian 12 installer now 
includes non-free firmware:


https://wiki.debian.org/Firmware


David

Re: BIOS unreadable at boot

[David Christensen]

On 9/16/24 03:32, Anssi Saari wrote:

Will Mengarini  writes:


ViewSonic 15E


Isn't that a CRT from the '90s? So you have some adapter for HDMI? Is
there really no other monitor you could use?




I tried using a search engine and came up empty:

https://html.duckduckgo.com/html?q=site%3AViewSonic.com%2015E


As other readers have suggested, I think the motherboard is putting out 
a video signal that the monitor does not support.



I suggest the OP test with another monitor -- one that supports up to 
FHD (1920x1024 60 Hz).



David

Re: BIOS unreadable at boot

[David Christensen]

On 9/15/24 16:12, Will Mengarini wrote:

I am trying to install Debian on a new prebuilt, but when I
boot and press , the screen is garbled.  Windows later
figures it out, but by that time it's too late to tell the
BIOS to boot from a USB stick on which I've installed netinst.

The mobo is a Gigabyte B450M DS3H WIFI (rev 1.5), manual at
.

I can see some letters on the garbled screen, but have not
yet been able to figure out how to get to the boot order.

So, either of 2 things might help me:

(1) The complete key sequence at the BIOS screen (or its 
help screen) to tell the mobo to boot from the USB stick first.

(2) Some way to boot that stick or its
contents directly from inside Windows.



Are you using the onboard graphics HDMI port?  If not, what?


What is the make and model of your monitor?


Is the display legible during the power on self-test (POST), prior to 
entering Setup?



Please describe "garbled" in more detail.  Is the image consistent or 
changing?



David

Re: Disk drive zero-fill benchmarks for various synchronization methods and block sizes

[David Christensen]

On 9/13/24 12:06, Jeffrey Walton wrote:

To add a datapoint...

My daily driver workstation is really fast with lots of RAM. It has 
3.4 GHz cpu and 64 GB of RAM. I also set swappiness to a low value

to avoid spilling out of RAM.

I use a lot of SBC's/dev boards for testing. They usually use a 
SDcard. The SDcard is really slow. The card can only provide 10 MB/s 
or 30 MB/s write speeds. Some cards I use are so cheap they are even 
slower.


If I dd an image from the workstation to the SDcard, it happens in 
under a second. dd exits, and closes its file descriptors. Something 
is obviously wrong since the image is 1 or 2 GB, and the SDcard write

speed is 10 MB/s or 30 MB/s.

The file system cache is still holding the writes. If I remove the 
SDcard and try to use it, the image is corrupt. When I say "remove",
 I mean pop the card out of the card reader since the write has 
supposedly finished.


What I found is, I have to manually call sync to ensure the image is 
written from cache to the SDcard. When I call `dd if=... of=/dev/sdd 
&& sync`, the command takes 30 seconds or so to complete. The time is

spent in sync, not dd.

Based on my experience with lots of RAM and slow media, you have to 
call sync to get the cache manager to write back to the disk.



That makes me think there is a bug in the secure digital card firmware,
the Linux secure digital card device driver, and/or the Linux file
system driver (?).


Have you seen a dd(1) invocation work correctly on a SD card without any
synchronization options and/or trailing sync(1)?


On 9/13/24 13:57, Rick Thomas wrote:

On the other hand, [running sync after dd] may not be necessary, but
it doesn't do any harm. If it is really unnecessary, it will probably
cost only a fraction of a second to do.  And if it is actually
necessary, you should do it, no matter how long it takes.



+1


David

Disk drive zero-fill benchmarks for various synchronization methods and block sizes

[David Christensen]

debian-user:

I had the need to zero-fill a Seagate Constellation ES.2 3 TB SATA 
magnetic hard disk drive using an Intel DQ67SW motherboard, Intel Core 
i7-2600S processor, 8 GB dual-channel memory, StarTech PEXSAT32 adapter, 
and Debian:


2024-09-08 08:05:02 root@bullseye-bios 
~/hardware/seagate/ST33000650NS/REDACTED

# cat /etc/debian_version ; uname -a
11.10
Linux bullseye-bios 5.10.0-32-amd64 #1 SMP Debian 5.10.223-1 
(2024-08-10) x86_64 GNU/Linux



When done, I ran some benchmarks to compare sequential write performance 
for various synchronization methods and block sizes:


1. Using dd(1) with option oflag=sync:

2024-09-08 09:19:27 dpchrist@laalaa ~/hardware/seagate/ST33000650NS/REDACTED
$ tail -n 214 log.txt | perl -pe 's/^\s+//' | egrep -A 3 'oflag=sync' | 
egrep 'time|GiB'
# time dd if=/dev/zero 
of=/dev/disk/by-id/ata-SEAGATE_ST33000650NS_REDACTED bs=1G count=1 
oflag=sync

1073741824 bytes (1.1 GB, 1.0 GiB) copied, 7.3181 s, 147 MB/s
# time dd if=/dev/zero 
of=/dev/disk/by-id/ata-SEAGATE_ST33000650NS_REDACTED bs=256M count=4 
oflag=sync

1073741824 bytes (1.1 GB, 1.0 GiB) copied, 7.39608 s, 145 MB/s
# time dd if=/dev/zero 
of=/dev/disk/by-id/ata-SEAGATE_ST33000650NS_REDACTED bs=64M count=16 
oflag=sync

1073741824 bytes (1.1 GB, 1.0 GiB) copied, 7.48021 s, 144 MB/s
# time dd if=/dev/zero 
of=/dev/disk/by-id/ata-SEAGATE_ST33000650NS_REDACTED bs=16M count=64 
oflag=sync

1073741824 bytes (1.1 GB, 1.0 GiB) copied, 7.92566 s, 135 MB/s
# time dd if=/dev/zero 
of=/dev/disk/by-id/ata-SEAGATE_ST33000650NS_REDACTED bs=4M count=256 
oflag=sync

1073741824 bytes (1.1 GB, 1.0 GiB) copied, 8.97546 s, 120 MB/s
# time dd if=/dev/zero 
of=/dev/disk/by-id/ata-SEAGATE_ST33000650NS_REDACTED bs=1M count=1K 
oflag=sync

1073741824 bytes (1.1 GB, 1.0 GiB) copied, 15.3814 s, 69.8 MB/s
# time dd if=/dev/zero 
of=/dev/disk/by-id/ata-SEAGATE_ST33000650NS_REDACTED bs=256K count=4K 
oflag=sync

1073741824 bytes (1.1 GB, 1.0 GiB) copied, 40.8931 s, 26.3 MB/s

2. Using dd(1) with sync(1):

2024-09-08 09:19:38 dpchrist@laalaa ~/hardware/seagate/ST33000650NS/REDACTED
$ tail -n 214 log.txt | perl -pe 's/^\s+//' | egrep -A 3 '&& sync' | 
egrep 'time|GiB'
# time dd if=/dev/zero 
of=/dev/disk/by-id/ata-SEAGATE_ST33000650NS_REDACTED bs=1G count=1 && sync

1073741824 bytes (1.1 GB, 1.0 GiB) copied, 7.3975 s, 145 MB/s
# time dd if=/dev/zero 
of=/dev/disk/by-id/ata-SEAGATE_ST33000650NS_REDACTED bs=256M count=4 && sync

1073741824 bytes (1.1 GB, 1.0 GiB) copied, 7.21795 s, 149 MB/s
# time dd if=/dev/zero 
of=/dev/disk/by-id/ata-SEAGATE_ST33000650NS_REDACTED bs=64M count=16 && sync

1073741824 bytes (1.1 GB, 1.0 GiB) copied, 7.21845 s, 149 MB/s
# time dd if=/dev/zero 
of=/dev/disk/by-id/ata-SEAGATE_ST33000650NS_REDACTED bs=16M count=64 && sync

1073741824 bytes (1.1 GB, 1.0 GiB) copied, 7.20049 s, 149 MB/s
# time dd if=/dev/zero 
of=/dev/disk/by-id/ata-SEAGATE_ST33000650NS_REDACTED bs=4M count=256 && sync

1073741824 bytes (1.1 GB, 1.0 GiB) copied, 7.11058 s, 151 MB/s
# time dd if=/dev/zero 
of=/dev/disk/by-id/ata-SEAGATE_ST33000650NS_REDACTED bs=1M count=1K && sync

1073741824 bytes (1.1 GB, 1.0 GiB) copied, 7.12372 s, 151 MB/s
# time dd if=/dev/zero 
of=/dev/disk/by-id/ata-SEAGATE_ST33000650NS_REDACTED bs=256K count=4K && 
sync

1073741824 bytes (1.1 GB, 1.0 GiB) copied, 7.10426 s, 151 MB/s

3. Using dd(1) without explicit synchronization:

2024-09-08 09:20:45 dpchrist@laalaa ~/hardware/seagate/ST33000650NS/REDACTED
$ tail -n 214 log.txt | perl -pe 's/^\s+//' | egrep -A 3 time | egrep 
'time|GiB' | egrep -v -A 1 '(MB\/s|sync)$' | egrep -v '^--'
# time dd if=/dev/zero 
of=/dev/disk/by-id/ata-SEAGATE_ST33000650NS_REDACTED bs=1G count=1

1073741824 bytes (1.1 GB, 1.0 GiB) copied, 7.30077 s, 147 MB/s
# time dd if=/dev/zero 
of=/dev/disk/by-id/ata-SEAGATE_ST33000650NS_REDACTED bs=256M count=4

1073741824 bytes (1.1 GB, 1.0 GiB) copied, 7.27323 s, 148 MB/s
# time dd if=/dev/zero 
of=/dev/disk/by-id/ata-SEAGATE_ST33000650NS_REDACTED bs=64M count=16

1073741824 bytes (1.1 GB, 1.0 GiB) copied, 7.21564 s, 149 MB/s
# time dd if=/dev/zero 
of=/dev/disk/by-id/ata-SEAGATE_ST33000650NS_REDACTED bs=16M count=64

1073741824 bytes (1.1 GB, 1.0 GiB) copied, 7.17308 s, 150 MB/s
# time dd if=/dev/zero 
of=/dev/disk/by-id/ata-SEAGATE_ST33000650NS_Z294MZFA bs=4M count=256

1073741824 bytes (1.1 GB, 1.0 GiB) copied, 7.20907 s, 149 MB/s
# time dd if=/dev/zero 
of=/dev/disk/by-id/ata-SEAGATE_ST33000650NS_REDACTED bs=1M count=1K

1073741824 bytes (1.1 GB, 1.0 GiB) copied, 7.11303 s, 151 MB/s
# time dd if=/dev/zero 
of=/dev/disk/by-id/ata-SEAGATE_ST33000650NS_REDACTED bs=256K count=4K

1073741824 bytes (1.1 GB, 1.0 GiB) copied, 7.13726 s, 150 MB/s


Additional observations:

1.  The Bash shell 'time' built-in command and the per-second/ 
per-transfer performance values displayed by dd(1) option 
'status=progress' can show the interaction between dd(1) option 
'oflag=sync', kernel I/O buffering, dd(1) write completion, and sync(1).


2.  dd(1) does not exit

Re: BASH reference for those who are "learning by doing"?

[David Christensen]

On 9/7/24 08:50, Richard Owlett wrote:

This started with be exploring "regular expressions".
I discovered some tutorials that were using Bash in their samples.
One {lost the reference at the moment} was almost a match for a real 
world problem I have.


But I've not used Bash in eons and have forgotten how to read a file 
into a single variable or a array variable.


I've downloaded "Bash Reference Manual"
   [https://www.gnu.org/savannah-checkouts/gnu/bash/manual/bash.html] 
for when I need fine grained details.

I've bookmarked the various links on
   [https://mywiki.wooledge.org/BashFAQ]
for quick reference.

I find neither to be search friendly.

Suggestions?

TIA



I suggest "Mastering Regular Expressions" by Friedl:

https://www.amazon.com/dp/0596528124


David

Re: Laptop screen dim on battery power

[David Christensen]

On 9/1/24 03:15, Richard Owlett wrote:

Lenovo R61 ThinkPad
Debian 10.0 MATE 1.20.4
The screen automatically dims when AC removed.
System->Hardware->Power Management->OnBatteryPower->Display
"Reduce backlight brightness" & "Dim display when idle" unchecked

Any where else to look?
TIA



Check the laptop firmware Setup utility settings.


David

Re: Usage: "debian ... amd64-netinst.iso"

[David Christensen]

On 9/1/24 06:15, Greg Wooledge wrote:

On Sun, Sep 01, 2024 at 01:19:45 -0700, David Christensen wrote:

On 8/31/24 23:16, Timothy M Butterworth wrote:

On Sat, Aug 31, 2024 at 11:15 PM David Christensen wrote:

On 8/30/24 20:48, John Conover wrote:

What does a "debian ... amd64-netinst.iso" do
with an .iso?



Yes -- but the command in the given string is "debian", which is not a valid
command.


Stop being so blindly literal.  It's pretty obvious to *most* of us
that John was referring to an .iso file, presumably one that he had
downloaded, and now was asking what to do with.



I agree that the question is broken.  Answering it literally points out 
that fact and sets the stage for the rest of my post.




So many of the replies in this thread have been disappointing.



There is a saying I heard while working in the electrical construction 
trade:


"Doing things right is a matter of experience.  Experience is a matter 
of doing things wrong."

-- Unknown



The best reply would be to point John to the Debian Installation Guide:
<https://www.debian.org/releases/bookworm/amd64/>

Specifically, Chapter 4, "Obtaining System Installation Media":
<https://www.debian.org/releases/bookworm/amd64/ch04.en.html>

This tells you what to do with the .iso file, if you're already on a
Linux system with appropriate hardware.  It also links to the Debian CD
FAQ, which contains hints on what to do on non-Linux systems.



I reviewed chapter 4, section 4.3, and subsection 4.3.1 of the "Debian 
GNU/Linux Installation Guide", and estimated that the OP needed simpler 
and more concise guidance.  So, I wrote the rest of my post.



David

Re: Usage: "debian ... amd64-netinst.iso"

[David Christensen]

On 8/31/24 23:16, Timothy M Butterworth wrote:

On Sat, Aug 31, 2024 at 11:15 PM David Christensen wrote:

On 8/30/24 20:48, John Conover wrote:

What does a "debian ... amd64-netinst.iso" do
with an .iso?


When I input that string into a computer running Debian, it produces an
error message:

2024-08-31 13:07:57 dpchrist@laalaa ~
$ echo $PS1 ; cat /etc/debian_version ; uname -a
\n\D{%Y-%m-%d %H:%M:%S} ${USER}@\h \w\n\$
11.10
Linux laalaa 5.10.0-32-amd64 #1 SMP Debian 5.10.223-1 (2024-08-10)
x86_64 GNU/Linux

2024-08-31 13:17:14 dpchrist@laalaa ~
$ debian ... amd64-netinst.iso
bash: debian: command not found



The iso file is not a command that you run on a Debian system. 



Yes -- but the command in the given string is "debian", which is not a 
valid command.




If you have
cd'd to the present working directory that the iso is located in you still
have to use "./"  ./ tells the OS that the file is located in the present
working directory pwd.



"./" is required when a command is in the current working directory and 
the current working directory is not in the PATH environment variable. 
"./" is not required for file arguments in the current working directory:


2024-09-01 01:11:28 dpchrist@laalaa ~
$ echo 'hello, world!' > hello.txt

2024-09-01 01:11:37 dpchrist@laalaa ~
$ echo 'cat $@' > foo

2024-09-01 01:11:43 dpchrist@laalaa ~
$ chmod +x foo

2024-09-01 01:11:48 dpchrist@laalaa ~
$ ./foo hello.txt
hello, world!


David

Re: Usage: "debian ... amd64-netinst.iso"

[David Christensen]

On 8/30/24 20:48, John Conover wrote:

What does a "debian ... amd64-netinst.iso" do
with an .iso?


When I input that string into a computer running Debian, it produces an 
error message:


2024-08-31 13:07:57 dpchrist@laalaa ~
$ echo $PS1 ; cat /etc/debian_version ; uname -a
\n\D{%Y-%m-%d %H:%M:%S} ${USER}@\h \w\n\$
11.10
Linux laalaa 5.10.0-32-amd64 #1 SMP Debian 5.10.223-1 (2024-08-10) 
x86_64 GNU/Linux


2024-08-31 13:17:14 dpchrist@laalaa ~
$ debian ... amd64-netinst.iso
bash: debian: command not found



Can it be coverted to a USB. How?


A Debian Installer "isohybrid" file is compatible with optical media 
(e.g. CD-R, DVD-R, or BD-R) and with USB flash drives.



If you wish to create Debian installation media, start by downloading a 
Debian Installer isohybrid file:


https://www.debian.org/distrib/


Different computer architectures require different Debian Installer 
isohybrid files.  Here is the one for 64-bit "Windows PC's":


https://cdimage.debian.org/debian-cd/current/amd64/iso-cd/debian-12.6.0-amd64-netinst.iso


Burning the Debian Installer isohybrid file onto an optical disc was 
easy when most PC's came with Windows, CD-RW drives, and burning 
software (e.g. Nero).  Today, Rufus seems to be a popular Windows 
application for copying ISO images to USB flash drives (untested):


https://www.lifewire.com/how-to-burn-an-iso-file-to-a-usb-drive-2619270


If you have a friend with Linux or BSD, they can use the dd(1) command 
to copy the Debian Installer isohybrid file to a USB flash drive:


https://wiki.debian.org/DebianInstaller/CreateUSBMedia


Alternatively, you can purchase Debian Installer media from vendors:

https://www.debian.org/CD/vendors/


Be very careful if you boot the Debian Installer on a computer that 
already has another operating system installed.  Using the Debian 
Installer Rescue Shell can be useful for troubleshooting, but requires 
advanced knowledge and skills.  Creating and maintaining a "dual boot" 
system requires even more knowledge and skills.



I suggest obtaining a used 64-bit "Windows PC" and using it only for Debian.


You can skip the installation media entirely by installing a hypervisor 
on your computer, creating a virtual machine (VM), loading the Debian 
Installer isohybrid file into the virtual optical disk drive, booting 
the VM, and installing Debian.  I like VirtualBox:


https://www.virtualbox.org/


I recommend that you buy and read "Learning the Unix Operating System" 
by Peek, et al:


https://www.amazon.com/_/dp/0596002610


Then "UNIX and Linux System Administration Handbook" by Nemeth, et al:

https://www.amazon.com/dp/0134277554


David

Re: Debian hardware: coping with Windows

[David Christensen]

On 8/25/24 20:39, Will Mengarini wrote:

I need to buy a new desktop tower, which means
it'll have Windows installed.  I haven't used
Windows since the 90s, so need some guidance.

A special complication is that I just had a computer
apocalypse in which a Power Surge From Hell nuked
*everything*, 



I suggest hiring a qualified electrician verify the electrical grounding 
system in your facility.  Ask for a recommendation and a quote to 
install surge arresters and/or lightning arresters at your electrical 
service panel or other suitable location(s).



Consider getting a good power conditioner/ UPS for your computers and/or 
electronics.




so trivial tasks like writing netinst to
a flash drive or CD-ROM are suddenly nontrivial: I need
to get Debian's netinst using Windows, with whatever
browser is there, then write it with Windows tools.  So:

(1) Will an HTTPS download in Windows
suffice to get me an uncorrupted netinst?
(Anything I need to know about "binary mode"?)

(2) What Windows tool will write that netinst to flash?
Does Windows 10 Home have that tool?  Pro?  Windows 11?
(I don't know yet what Windows I'll end up with.)

(3) What Windows tool will write that netinst to CD-ROM?
Does Windows 10 Home have that tool?  Pro?  Windows 11?
(I have ancient beige boxes that might boot from CD-ROM
but not flash; they'd be useful as failsafes.)



I suggest buying Debian installation media from a vendor:

https://www.debian.org/CD/vendors/




(4) Will the latest Mordorsoft dual-boot fsckup prevent
maintaining the Windows installation in its own partition,
so I have to nuke the only working OS before installing
Debian?  Will that be true of Windows 10 as well as 11?
(This question MIGHT be really important, because I might
have the option of buying something with Windows 10
instead of 11; maybe that'd be a major win.)

(5) If I can keep a Windows partition, how big must it be?

(6) Can/should I do the repartitioning from Windows
before installing Debian, & with what tool?



Rather than dual-boot, I put trayless mobile racks in my computers and 
put each OS on its own 2.5" SATA SSD:


https://www.startech.com/en-us/hdd/s25slotr

https://www.startech.com/en-us/hdd/hsb220sat25b

https://www.startech.com/en-us/hdd/bracketfdbk



(7) I like the strategy of having /home in a separate
partition, so I can easily upgrade by doing a fresh
install.  What's the minimum size you'd recommend
for a partition containing / but excluding /home
and intended to remain usable for the life of an
SDD, so presumably spanning many Debian releases?
(Remember I can't now look at an existing installation
for comparison; everything I had is toast.)



I put the vast majority of my data on a file server (FreeBSD, ZFS), 
which allows me to keep my OS installations small enough to fit on "16 
GB" drives (USB, SSD, HDD).  This also facilitates taking images 
regularly for disaster recovery.





(8) I've heard that the initial Windows setup process
has hair and takes an hour.  People who buy towers
from Walmart have written that they needed Walmart
customer support to get their Windows "activated",
whatever that means.  Any tips to avoid Windows
doing updates that'll bork dual-boot, or otherwise
just waste time?  Remember that this will initially
be my only working computer.  (I'm typing now on
the virtual keyboard of an ancient smartphone.)

(9) Does Windows have, or can it easily get, an
SSH client that'll let me shell in to my ISP
(Eskimo North) before I have Debian running?

I expect to read all of the Debian GNU/Linux Installation
Guide at https://www.debian.org/releases/bookworm/amd64/
eventually, but only the hardware-compatibility
stuff before making the hardware purchase.



Installing, activating, configuring, maintaining, etc., a Windows 10 
computer is non-trivial.  I found "Windows 10 Inside Out", 4th Edition, 
to be very helpful:


https://www.microsoftpressstore.com/store/windows-10-inside-out-9780136784159


That said -- I suggest buying or building a computer with no operating 
system or with some distribution of Linux.



As new computers can have chips that are too new for Debian Stable, and 
therefore unsupported, I suggest buying or building a computer with 
parts that are at least a few years old (2+ years?).  Off-lease 
corporate workstations and servers can be a very good value.



David

Re: Default partition mounts [ "Installation Guide" lacks index ]

[David Christensen]

On 8/19/24 04:19, Richard Owlett wrote:

I'm over 80 and doing first "from scratch" install since Squeeze ;}
Hardware is Lenovo R61 ThinkPad (64 bit).
I multi boot [Grub will have at least three options]:
   1. minimalist installation - primarily command line usage
   2. 64 bit Debian with maximum features
   3. 32 bit Debian - couple of applications require a 32 bit OS
   4. other installs with strong project dependencies

Today's question
At boot time, what determines which physical partition gets mounted as a 
specific directory ( /, /home, swap, and so forth )?


Please reference documentation as reading it will remind me of how and 
why I chose specific options.


TIA

P.S. - re-reading https://www.debian.org/releases/stable/amd64/ ;}!



I tried multi-boot back in the day (e.g. BIOS/MBR) -- it was not for me. 
 My solution was to buy multiple disks and put one OS on each.  You 
could do this -- the disk drive in the Lenovo R61 ThinkPad is externally 
accessible:


https://www.ifixit.com/Guide/Lenovo+ThinkPad+R61i+Hard+Drive+Replacement/118125


AIUI UEFI/GPT were designed to support multi-boot, but one-disk-per-OS 
is KISS -- each installer can have its way with the entire disk, and the 
installed OS just works (if it supports your hardware).



That said, the fundamental problem with either approach is that no 
matter which OS you are running, you want something in another OS that 
is not running.  Back in the day, I bought additional computers.  Today, 
there are several high quality hypervisors to choose from.



A key consideration is where to put your data, so that it is accessible 
from whichever OS you happen to be running.  Solutions include a file 
server, a NAS, and another drive with a lowest-common-denominator file 
system (e.g. FAT32, ExFAT, or NTFS).



David

Re: upgrade to bookworm causes breakage

[David Christensen]

On 8/19/24 15:33, Bob Mroczka wrote:

Hello,

I attempted to upgrade my system from debian 11 to 12 following the
instructions provided at
https://www.cyberciti.biz/faq/update-upgrade-debian-11-to-debian-12-bookworm.
I got to the point of executing the command at step 6 to do sudo apt
full-upgrade.  At this point my system is in a state where I'm having
difficulty recovering from.





I understand that in-place upgrading a FOSS computer over many years is 
a source of pride for many people.  I tried that, and it did not work 
out for me.  Since then, I have invested myself in fresh installs, 
minimal sysadmin changes, thorough documentation, scripting, version 
control, backup, restore, and multiple layers of redundancy.  The 
efforts are far more predictable and the results are far more reliable.



So, I suggest that you put a secure erased SSD into another computer 
with no drives other than optical, do a fresh install of Bookworm, add 
software/ configure as desired, add disks, and migrate your data.



David

Re: [Hardware support] Dell Vostro 15 3530

[David Christensen]

On 7/17/24 20:13, Erik Eduardo Alcala Salero wrote:

Good day,
I have been using Debian 12 stable in my dell vostro 15 3530, I wanted to ask 
if some one else is using this machine or having issues with the keyboard, I 
have manjaro installed also, when I boot and see the manjaro boot-loader the 
keyboard does not work, this was not the case in the past, just maybe since 3 
weeks ago when the problem started: I boot, some keys in the keyboard do not 
work and I have restart several times until I can press the enter key to log in 
into debian, any help is much appreciated.
I'm willing to troubleshoot and provide any info necessary, thanks in advance ! 
Please let me know which commands or logs are needed to check this. Help is 
much appreciated ! :)




Have you connected a known good keyboard directly to the computer to 
verify that the problem is Debian or manjaro, and not the keyboard (or a 
KVM switch, if you use one)?



David

Re: Bug?

[David Christensen]

On 7/14/24 00:57, Richard Bostrom wrote:

Executing this script halts it after the tar with the following message.

--
#!/bin/sh
tar -zcvf bak.tar.gz /home/user/Documents &&
gpg -r backup@user.local -e bak.tar.gz &&
rm -rf bak.tar.gz &&
rsync -vac --delete /home/user/Documents/bak.tar.gz.gpg /media/user/6548-2136 
&&rm -rf bak.tar.gz.gpg

tar: file changed as we read it.
--



Assuming your script is named /home/user/script and your current working 
directory is /home/user, please run the following commands in a terminal 
and post the complete console session -- prompts, commands entered, 
output displayed:


$ cat /etc/debian_version; uname -a

$ cd

$ pwd

$ ls -l bak.tar.gz*

$ ls -l /media/user/6548-2136

$ cat script

$ sh -x script

$ ls -l bak.tar.gz*

$ ls -l /media/user/6548-2136


David

Re: Security Flaw:

[David Christensen]

On 7/9/24 23:34, Richard Bostrom wrote:

I cannot update my passphrase in crypttab although the passphrase is updated in 
the OS I cannot enter my OS without using the latest passphrase.

Yours sincerely
Richardh Bostrom



Passphrases in crypttab(5) are for disks, disk partitions, virtual 
devices, etc., and are unrelated to user passwords.  Changing one does 
not affect the other, and vice versa.



If you are using LUKS to encrypt a partition, LUKS supports more than 
one passphrase.  See cryptsetup(8).  To change the passphrase, I would 
add the new passphrase, reboot, and enter the new passphrase to verify. 
When you are confident the new passphrase works, delete the old passphrase.



David

Re: Debian 11 and IPv4 static IP address

[David Christensen]

On 7/5/24 19:06, Max Nikulin wrote:


On 7/5/24 21:14, George at Clug wrote:



Thank you for your replies.


The underlying issue appears to be that my old-school Linux console 
network administration skills have been rendered obsolete by systemd and 
NetworkManager.  I typically install Xfce when installing Debian, and 
the panel NetworkManager Applet has made it too easy to "monkey see, 
monkey do" without understanding the details.



What I really need is a good book or document that explains the design 
and implementation of networking with systemd and Network Manager on 
modern Debian GNU/Linux systems.  Recommendations?



David

Re: Debian 11 and IPv4 static IP address

[David Christensen]

On 7/4/24 21:08, Felix Miata wrote:


On 7/4/24 21:10, jeremy ardley wrote:


On 7/5/24 11:44, Franco Martelli wrote:



Thank you all for the replies.


I can find no statement in The Debian Administrator's Handbook regarding 
disabling DHCP when using a static IP:


https://www.debian.org/doc/manuals/debian-handbook/sect.network-config.en.html#sect.interface-ethernet


Similarly, when STFW "Debian 11 static IP" and reading articles.


Browsing the Debian Wiki page "NetworkConfiguration" -> "Info" link:

https://wiki.debian.org/NetworkConfiguration?action=info

The following sentence:

"Make sure to disable all DHCP services, e.g. dhcpcd."

Was added at revision 97:

97 	2017-12-27 10:50:07 	25741 	to previous 	?mrsgreen 	note for static 
ip's: disable dhcp services 	view



Who is "mrsgreen"?


David

Re: Debian 11 and IPv4 static IP address

[David Christensen]

On 7/4/24 19:30, Felix Miata wrote:

David Christensen composed on 2024-07-04 19:06 (UTC-0700):

I have built a VirtualBox virtual machine and installed Debian 11 with
SSH server and standard system utilities only.  I plan to use the VM to
run the UniFi Network Controller to manage the UniFi equipment on my
SOHO LAN:

The VM currently has a DHCP address.  I would like to set a static IPv4
address.  The Debian wiki tells me to edit /etc/network/interfaces and
provides some example settings that I believe I can adapt to my use-case:

The Debian wiki also says:
"Make sure to disable all DHCP services, e.g. dhcpcd."
What is the correct method to "disable all DHCP services"?


I think there are more than one. One thing is to check what is enabled, then
disable or uninstall whatever owns the unit(s):

systemctl list-unit-files | egrep 'net|dhcp'



2024-07-04 20:26:35 root@unifi ~
# systemctl list-unit-files | egrep 'net|dhcp'
networking.service enabled enabled
systemd-network-generator.service  disableddisabled
systemd-networkd-wait-online.service   disableddisabled
systemd-networkd.service   disabledenabled
systemd-networkd.socketdisabledenabled
network-online.target  static  -
network-pre.target static  -
network.target static  -


Do the same with whatever "manages" /etc/resolv.conf, 



How do I determine what manages /etc/resolv.conf?



and create a regular file of your own design.

I don't use VMs, but I did several hours ago instruct several hours ago using
systemd-network to setup static IP service for a *buntu user who couldn't get
netplan sorted:
<https://www.linuxquestions.org/questions/ubuntu-63/netplan-apply-no-longer-setting-static-ip-address-4175738702/#post6512274>

I've been using systemd-network static IPs on everything for several years,
including Bullseye.



2024-07-04 20:15:43 root@laalaa ~
# man systemd-network
No manual entry for systemd-network


How do you use systemd-network to configure an interface with a static 
IP address?



David

Debian 11 and IPv4 static IP address

[David Christensen]

debian-user:

I have built a VirtualBox virtual machine and installed Debian 11 with 
SSH server and standard system utilities only.  I plan to use the VM to 
run the UniFi Network Controller to manage the UniFi equipment on my 
SOHO LAN:


root@unifi:~# cat /etc/debian_version ; uname -a
11.10
Linux unifi 5.10.0-30-amd64 #1 SMP Debian 5.10.218-1 (2024-06-01) x86_64 
GNU/Linux



The VM currently has a DHCP address.  I would like to set a static IPv4 
address.  The Debian wiki tells me to edit /etc/network/interfaces and 
provides some example settings that I believe I can adapt to my use-case:


https://wiki.debian.org/NetworkConfiguration#Configuring_the_interface_manually

auto eth0
iface eth0 inet static
address 192.0.2.7/24
gateway 192.0.2.254


The Debian wiki also says:

"Make sure to disable all DHCP services, e.g. dhcpcd."


What is the correct method to "disable all DHCP services"?


David

Re: Re (3): Backup.

[David Christensen]

On 6/30/24 08:37, Andy Smith wrote:



Thank you for that informative discussion of rsnapshot(1) and related.  :-)


My initial reaction to this thread was to recommend Preston [1].  I 
still think that is decent advice; both for noobs and for experienced 
people who missed it.



David


[1]  Preston, W., 2007, "Backup & Recovery", O'Reilly Media, Inc.
ISBN: 9780596102463, 
https://www.oreilly.com/library/view/backup-recovery/0596102461/

Re: Strange Problem

[David Christensen]

On 6/29/24 10:01, Stephen P. Molnar wrote:



On 06/29/2024 12:28 PM, Darac Marjal wrote:


On 29/06/2024 15:13, Stephen P. Molnar wrote:
I have just restated by Xfce4 user on my Bookworm system and find 
that I can no longer resize some of the apps on the desktop and the 
icons in the upper right corner of the tool bar are missing.


Often, this is an indication that the window manager isn't running. 
For XFCE4, the window manager is called "xfwm4". Try pressing "Alt+F2" 
to bring up the launch dialog, enter "xfwm4" at the prompt and press 
enter.



This is only om my user directory, the root directory is still normal.

Don't run X sessions (read: don't run XFCE) as root.


I have not the faintest idea as to what might be going on and would 
greatly appreciate assistance.


Thanks in advance.


Well, the problem is back. I logged out and back in, and there it was!

I tried "Alt+F2" then "xfwm4" (without the quotes, of course) and got 
nothing.



Please power off your computer.  Then power on your computer.  Then 
document every command or value you enter into the computer and every 
response by the computer.  If and when the computer does something other 
than what you expect, document what the computer did and what you 
expected it to do.  Research the proper terminology and use it.  Then 
power off, power on, follow your document, and see if the issues are 
repeatable.  Then post your document.



David

Re: installation

[David Christensen]

On 6/28/24 10:20, dewey rahn wrote:

When I used to use Debian when a new release came out (like from 10 to 11) you 
had to completely reinstall the operation system. Is that the case now?



I have invested myself in backup, recovery, and version control/ 
configuration management.  So, a major version upgrade for me consists 
of backing up data on Debian version X, verifying that all of the system 
configuration files are checked in to the version control system (CVS), 
pulling the old system drive, installing a wiped disk drive, doing a 
fresh install of Debian version X+1, checking out the system 
configuration files to a side directory, merging system configuration 
files by hand, and restoring data.



I suggest that you start by implementing a version control system that 
you can use over the network.



David

Re: Current best practices for system configuration management?

[David Christensen]

On 6/22/24 11:33, Dmitrii Odintcov wrote:

Hi all,


Sorry to resurrect an old-ish thread, but I am facing the exact same
task, minus the know-how.

Basically I am looking to pre-configure a number of Debian setups -
let's say, "server", "laptop" and "PC" - that would contain sets of
packages to install (or uninstall), configuration files (including but
not limited to /etc/), and possibly arbitrary scripts to execute upon
installation (e.g. to make more precise edits to configs).

I would like to store these in some central, git-controlled location
where I could pull them from to my target machines. This kind of setup
could be rather low-level, such as configuring APT sources and
preferences, installing and configuring systemd-networkd, etc.
Ideally, I would also be able to incorporate these into a Debian
installer so I could get a new machine "up and running" with my
defaults in one go.

It is worth adding that I am not looking to build an "infrastructure",
and no setup *between* these machines (networking, file sharing, etc.)
is necessary.

Generally, I would rather avoid complicated tools like Ansible and
those with large dependencies such as interpreted languages (except
Perl).

So far, equivs and config-package-dev appear most relevant, but the
former is somewhat lacking in documentation (or I am in
documentation-finding ability), and the latter seems to be focused on
config editing/deployment rather than package installation.


Would be grateful for some advice!



I think the "best" answer depends upon the scale of your installation.


I have a SOHO network with a dozen or so Debian, Windows, macOS, and iOS 
clients, a FreeBSD/ZFS CVS, SSH, and Samba server, and a FreeBSD/ZFS 
backup server.  For system administration, including configuration 
management, I have gone down the do-it-yourself (DIY) path using 
lowest-common-denominator FOSS command line tools.



After the network, ssh(1), and rsync(1), the most valuable tool for 
system administration (including configuration management) has been a 
version control system.  I prefer CVS over Git because CVS provides 
monotonically-increasing MAJOR.MINOR version numbers via keywords (e.g. 
$Revision$) that can be included inside managed plaintext files.  I 
create a CVS project for everything I want to manage.  For OS's, I check 
in a sysadmin log file with my notes and console sessions, a list of 
packages installed, various reports that I have run, and any system 
configuration files that I have modified, added, or deleted.



After version control, the next most valuable tool has been scripting. 
Over the years, I have written numerous scripts to automate repetitive 
chores.  When the needs are simple, I write Bourne shell scripts.  When 
I want more power, I upgrade to Perl.  All of the scripts are checked in 
to CVS.



I expect Ansible, Puppet, etc., would work at my scale, but are designed 
for large installations.



David

Re: mounting external hard drive from rescue mode shell?

[David Christensen]

On 6/22/24 10:37, Richard Owlett wrote:
I ask about i386 Debian Live because I have a fine operational Sony 
laptop that currently runs Debian 9.0 and has a $20 price tag on its 
bottom.


This machine has option to boot Debian 11 with an AMD64 kernel.
I routinely run Debian 9.13 because its configuration is comfortable 
(i.e. useful).


I have 2 other laptops which will have something >= Debian 12 before I 
abandon this machine.



On 6/22/24 10:49, Richard Owlett wrote:
> On 06/22/2024 12:13 PM, Stefan Monnier wrote:
>> On 6/22/24 09:57, David Christensen wrote:
>>> Rather than creating a customized Debian Live image, I install
>>> Debian onto a USB flash drive or onto a 2.5" SATA SSD connected via
>>> a USB-SATA adapter cable:
>>
>> +1
>>
>> It's pretty easy to make a simple Debian install on some old USB key
>> you have lying around and it comes really handy.
>
> Relevant laptop is so old I don't know if it can boot from a physical
> USB device. I was suspecting that simplest thing would be copying
> suitable image to hard drive and let GRUB earn its keep ;}


So, the Sony laptop has Debian 9.13 i386 installed on the HDD/SSD?


Debian 9 LTS support ended on 01 Jul 2022:

https://endoflife.date/debian


If the Sony laptop can boot Debian 11 amd64 via CD, it should be able to 
boot an amd64 OS via USB.  I suggest copying 
debian-12.5.0-amd64-netinst.iso to a USB flash drive and trying to boot 
it.  If it works, you can decide if you want to use d-i, if you want to 
burn a live distribution to a USB device, and/or if you want to install 
Debian onto a USB device.



David

Re: mounting external hard drive from rescue mode shell?

[David Christensen]

On 6/22/24 04:43, Richard Owlett wrote:

Thank you for reminding me of live images just now. Perfect timing.
I have an i386 machine with some atypical constraints.
https://www.debian.org/CD/live/ states only amd64 images are currently 
available.


Questions:
   1. What is latest i386 live image available in some archive?
   2. I have a working machine that will take a current full install
  of an i386 system. Can an average user create his own i386 live
  install image?



Rather than creating a customized Debian Live image, I install Debian 
onto a USB flash drive or onto a 2.5" SATA SSD connected via a USB-SATA 
adapter cable:


https://www.startech.com/en-us/hdd/usb3s2sat3cb


Installing Debian onto a USB device allows me to choose the tasks I want 
at install time and then to add, remove, update, upgrade, etc., the 
packages I want later.  I load the USB Debian instances with all of my 
favorite Unix/ Linux/ Debian trouble-shooting tools, plus my own scripts.



My Debian 11 amd64 BIOS/MBR USB instance works on all of my amd64 
BIOS/MBR computers and works on most of my EUFI/GPT computers when set 
to BIOS/MBR mode.



My Debian 11 am64 UEFI/GPT USB instance was created on a Windows 10 era 
machine with UEFI/GPT and only works on similar machines with similar 
settings.



David

Re: mounting external hard drive from rescue mode shell?

[David Christensen]

On 6/20/24 19:10, Max Nikulin wrote:

On 20/06/2024 12:06, David Christensen wrote:

You can use the fdisk(8) command to list the partitions on a drive.


lsblk --fs

perhaps with "-o +SIZE" may be more convenient to get overview of drives.



The debian-11.9.0-amd64-netinst rescue shell does not include lsblk(8):

~ # lsblk
/bin/sh: lsblk: not found


David

Re: How to recover when monitor goes blank.

[David Christensen]

On 6/19/24 13:45, Ram Ramesh wrote:

Hi,

   I have my monitor, keyboard and mouse shared through a KVM switch. 
One host is Linux Debian bookworm 12.5 and another is laptop running 
Windows 11. When I leave KVM on the laptop side for extended period I 
have issues switching back to Debian side. When I switch, the screen is 
blank and KB does not respond as if Debian is running headless. I had to 
remote login and reboot Debian side with KVM locked on this side to get 
back the monitor/KB. This happens regardless of whether I am in Xorg or 
VT.  I do not know how to force Debian/Linux to check for monitor/KB 
again after extended period of disconnect when it has assumed it is 
running headless. Any solutions?


Regards
Ramesh



My Debian machines have Xfce.  I configure Applications Menu -> Settings 
-> Power Manager -> Display -> Display power management -> Off.



David

Re: mounting external hard drive from rescue mode shell?

[David Christensen]

On 6/19/24 12:23, Heriberto Avelino wrote:

On Wed, Jun 19, 2024 at 9:04 AM Heriberto Avelino wrote:

Is it possible to mount an external hard drive while running Debian in
rescue mode?
Furthermore, the ultimate question is how could I copy folders from the
computer's hard drive to the external one while in rescue mode?



> Thanks Eben and David!
> I am now on a shell (BusyBox v.35.0 Debian 1:1.35.0-4+b3)
> I don't see the mounting points to execute cp.
> There is nothing under media nor root; under usr I can see only: bin
> lib local sbin share.
> Where are the internal h-drive and the external?
> I would very much appreciate your further help.
> Who could have envisioned a community like this back in the 80's? This
> is great!
> Heriberto


We are glad to help.  :-)


On Debian GNU/Linux, SATA drives (internal and external) should have 
special files (device nodes) under /dev.  You can use the ls(1) command 
and a glob pattern to find them:


~ # ls /dev/sd?

/dev/sda


You can use the fdisk(8) command to list the partitions on a drive.  For 
example, here is the drive containing Debian in the computer I am using now:


~ # fdisk -l /dev/sda
Disk /dev/sda: 55.9 GiB, 60022480896 bytes, 117231408 sectors
Disk model: INTEL SSDSC2CW06
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0x544032f5

Device BootStart   End  Sectors  Size Id Type
/dev/sda1  *2048   1953791  1951744  953M 83 Linux
/dev/sda21953792   3907583  1953792  954M 83 Linux
/dev/sda33907584  29298687 25391104 12.1G 83 Linux
/dev/sda4   29298688 117229567 87930880 41.9G 83 Linux


Partitions can contain one of several things.  One possibility is a file 
system.  I keep detailed records on all of my OS's and drives, so I know 
what is what.  Figuring out the contents of an unknown drive using the 
Debian rescue shell is possible, but requires more knowledge and effort. 
 Using a live Linux distribution instead of the Debian rescue shell can 
make such tasks easier.  I install Debian onto a SATA SSD via a USB-SATA 
adapter cable for this purpose.



Once you have identified the device node of a partition that contains 
the file system that you want to read and/or write, you must mount the 
file system.  The first step is to create a mount point with mkdir(1):


~ # mkdir /scratch


The second step is to mount the file system with mount(8):

~ # mount /dev/sda4 /scratch


Repeat the above process to mount any additional file systems that you 
want to read and/or write.



One the file system(s) are mounted, you can read and/or write files 
and/or directories.  For example, you can use cp(1) to copy files from 
an internal drive to an external drive.



"Learning the Unix Operating System" is a good book for learning how to 
use Unix/ Linux from the command line:


https://www.oreilly.com/library/view/learning-the-unix/0596002610/


"UNIX and Linux System Administration Handbook" is a good book for 
learning how to administer Unix/ Linux from the command line:


https://www.oreilly.com/library/view/unix-and-linux/9780134278308/


"Design of the UNIX Operating System" is a good book for understanding 
how Unix worked around the time Linux Torvalds wrote Linux:


https://www.pearson.com/en-us/subject-catalog/p/design-of-the-unix-operating-system/P20009243/9780132017992

https://en.wikipedia.org/wiki/Linus_Torvalds


David

Re: mounting external hard drive from rescue mode shell?

[David Christensen]

On 6/19/24 08:04, Heriberto Avelino wrote:

Dear all:
Is it possible to mount an external hard drive while running Debian in
rescue mode?



Yes.



Furthermore, the ultimate question is how could I copy folders from the
computer's hard drive to the external one while in rescue mode?
Many thanks!!
Heriberto



This is how I start a rescue shell using debian-11.9.0-amd64-netboot media:

Debian GNU/Linux installer menu (BIOS mode) -> Advanced options
Advanced options -> Rescue mode
Language -> C
Continent or region -> North America
Country, territory or area -> United States
Keymap -> American English
Hostname -> debianrescue
Domain name -> tracy.holgerdanske.com
Select your time zone -> Pacific
Passphrase for /dev/sda3 ->  -> Continue
Device to use as a root file system -> Do not use a root file system
Rescue operations -> Execute a shell in the installer environment
Executing a shell -> Continue


Once busybox(1) is running, you can issue mount(8) and cp(1) commands 
similar to using a shell.



David

Re: can't connect to server from outside LAN

[David Christensen]

On 6/12/24 15:54, Greg Marks wrote:

I'm running a Debian server from my home with a static IP address,
with ssh configured to use key-based authentication rather than
password-based.  As of a couple weeks ago, I have been unable to ssh to
my server from external locations.  When I ssh from a laptop connected
to the wireless network on the same router as my home server, I do
successfully connect to the server.  But when I ssh from an external
location, I get this error:

OpenSSH_8.4p1 Debian-5+deb11u3, OpenSSL 1.1.1w  11 Sep 2023
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: /etc/ssh/ssh_config line 19: include /etc/ssh/ssh_config.d/*.conf 
matched no files
debug1: /etc/ssh/ssh_config line 21: Applying options for *
debug2: resolve_canonicalize: hostname xxx.xxx.xxx.xxx is address
debug3: expanded UserKnownHostsFile '~/.ssh/known_hosts' -> 
'/home/user/.ssh/known_hosts'
debug3: expanded UserKnownHostsFile '~/.ssh/known_hosts2' -> 
'/home/user/.ssh/known_hosts2'
debug2: ssh_connect_direct
debug1: Connecting to xxx.xxx.xxx.xxx [xxx.xxx.xxx.xxx] port 22.
debug1: Connection established.
debug1: identity file /home/user/.ssh/id_rsa type -1
debug1: identity file /home/user/.ssh/id_rsa-cert type -1
debug1: identity file /home/user/.ssh/id_dsa type -1
debug1: identity file /home/user/.ssh/id_dsa-cert type -1
debug1: identity file /home/user/.ssh/id_ecdsa type -1
debug1: identity file /home/user/.ssh/id_ecdsa-cert type -1
debug1: identity file /home/user/.ssh/id_ecdsa_sk type -1
debug1: identity file /home/user/.ssh/id_ecdsa_sk-cert type -1
debug1: identity file /home/user/.ssh/id_ed25519 type -1
debug1: identity file /home/user/.ssh/id_ed25519-cert type -1
debug1: identity file /home/user/.ssh/id_ed25519_sk type -1
debug1: identity file /home/user/.ssh/id_ed25519_sk-cert type -1
debug1: identity file /home/user/.ssh/id_xmss type -1
debug1: identity file /home/user/.ssh/id_xmss-cert type -1
debug1: Local version string SSH-2.0-OpenSSH_8.4p1 Debian-5+deb11u3
kex_exchange_identification: read: Connection timed out
banner exchange: Connection to xxx.xxx.xxx.xxx port 22: Connection timed out

When I ping the server from external locations, I get 100% packet loss;
whereas when I ping the server from my local wireless network, there
is 0% packet loss.  (I do have nftables set to drop connections from
numerous IP addresses that have attempted hacks in the past; however,
the problem persists after flushing nftables, and at any rate, using
check-host.net and www.site24x7.com to ping my server from various
worldwide locations also results in 100% packet loss.)  Port 22 is open.
The package ufw is not installed on my server.

The apache2 Web server running on my home server is correctly hosting
my Web pages: from external locations, my Web page gmarks.org will
open in a Web browser (even though "ping -c 10 gmarks.org" shows 100%
packet loss).  Running "traceroute xxx.xxx.xxx.xxx" from external
locations reported four successful steps, not reaching my server IP,
followed by a series of "* * *" lines.

Running "sudo service sshd status" on my server shows ssh.service
is active and running.  Running "ip address show" on my server shows
nothing unusual.  I've restarted my router, and I've restarted my server;
neither helped.

The problem began a couple weeks ago; previously (and for many years)
I had been able to ssh to my server without issue.  The first time it
failed, I was using free wireless at an airport; I was able to ssh to my
server from the hotel that morning, and maybe, the first time I tried,
from the airport, but then subsequent ssh attempts from the airport
failed to connect.  I mention this only because nothing had changed in
my server's configuration when this problem began.

This is a real problem for me, as a lot of my work involves sending
files via scp between work and home.  Any suggestions about how to
troubleshoot and hopefully fix the problem will be greatly appreciated.

Best regards,
Greg Marks



Ping:

2024-06-12 16:19:25 dpchrist@laalaa ~
$ ping -c 1 -v gmarks.org
PING gmarks.org (76.235.90.201) 56(84) bytes of data.

--- gmarks.org ping statistics ---
1 packets transmitted, 0 received, 100% packet loss, time 0ms


HTTPS works:

https://gmarks.org/

Greg Marks
Office: 313 Ritter Hall
Phone: (314)977-7206
E-mail address:  local-part consists of surname, domain name identical 
to that of the present Web page

PGP encryption public key ID: 0x53F269E8

Postal Address:
Department of Mathematics and Statistics
St. Louis University
221 N. Grand Blvd.
St. Louis, MO 63103-2006
U.S.A.

Resources for students

Resources for mathematicians

Research papers


It appears that someone who controls one or more of the routers between 
the Internet and your router has blocked ICMP echo requests, ICMP echo 
replies, and TCP port 22 (SSH).



I suggest that you 

Re: about 10th new install of bullseye

[David Christensen]

On 6/8/24 12:13, gene heskett wrote:

On 6/8/24 03:22, David Christensen wrote:
If you installed VirtualBox on your Debian primary workstation, you 
could create one Debian VM for each of your engineering/ manufacturing 
apps.  This would give each app a clean Debian VM for installation, 
prevent apps from fighting each other, and prevent apps from modifying 
your base Debian installation.


It is quite rare that a snap, appimage, or venv needs anything from the 
system. Memory or storage is generally done at whatever venv is started 
as the user. That venv equ is generally what they all claim to do. I see 
your reticence to make use of them as a restriction.



My suggestion is a variation of the "divide and conquer" troubleshooting 
strategy.



I am not familiar with snap, appimage, or venv.  Regardless of the 
software distribution mechanism, I expect that each app is developed and 
tested against a list of supported OS's and releases using VM's.  If you 
provide each app with its own VM containing a supported OS and release, 
the app should install and work correctly.  And, your base Debian 
installation should remain stable.



David

Re: about 10th new install of bullseye

[David Christensen]

On 6/7/24 22:41, gene heskett wrote:
I OTOH, have found AppImages a good way to get uptodate, and keep 
uptodate, packages like OpenSCAD, FreeCAD and the miriad 3d slicers, 
most of which do a new AppImage in the first week of the month. So the 
OpenSCAD I'm running is nearly 4 years newer than the repo version, and 
probably 20x faster.



I have found that installing software on Debian by any means other than 
official Debian packages is a recipe for disaster.



I sometimes write Perl code that runs as root.  I use VirtualBox and do 
my development and testing on virtual machines.  Oracle provides Debian 
packages and integrates with sources.list(5) and apt-get(8).  See 
"Debian-based Linux distributions":


https://www.virtualbox.org/wiki/Linux_Downloads


If you installed VirtualBox on your Debian primary workstation, you 
could create one Debian VM for each of your engineering/ manufacturing 
apps.  This would give each app a clean Debian VM for installation, 
prevent apps from fighting each other, and prevent apps from modifying 
your base Debian installation.



David

Re: about 10th new install of bullseye

[David Christensen]

On 6/6/24 22:14, gene heskett wrote:
In experimenting I've found a name clash, there are appprently two 
orca's. one is a speech synth, one is a slicer for 3d printers I don't 
use. Typing orca in a shell locks the shell wo any output, for several 
minutes but comes back to a prompt with a ctl-c, so I've NDC which was 
being executed. Whatevver, the installation is quite voluminous:

gene@coyote:~/AppImages$ locate orca |wc -l
1560

So I took orca out, which took gnome out. But now gnomes dependencies 
will put orca back in. So now I can't run autoremove. So one more time 
this broken damned bookworm install has bit me in a rear.


No Cheers, Gene Heskett, CET.



Here are my installation notes from when I migrated my daily driver from 
Debian 9 to Debian 11.  It has orca, and orca has never bothered me:


January 9, 2022

1.  Wipe Intel SSD 520 Series 60 GB drive in Intel DQ67SW.  Insert
debian-11.2.0-amd64-netinst USB flash drive into USB 3.0 port
adjacent Gigabit port. Boot:

Debian GNU/Linux installer menu (BIOS mode)
install
LanguageC
Continent or region North America
Country, territory or area  United States
Keymap to use   American English
Hostnamelaalaa
Domain name tracy.holgerdanske.com
Root password   
Re-enter password   
Full name for new user  debian
Username for your account   debian
Choose a password   
Re-enter password   
Select your time zone   Pacific
Partitioning method Manual
  Select a partition... SCSI1 (0,0,0) (sda) - 60.0 GB ATA INTEL 
SSDSC2CW06
Create partition table  Yes
  Select a partition... pri/log 60.0 GB FREE SPACE
Create a new partition
  New partition size1 GB
  Type  Primary
  Location  Beginning
  Partition settings
Use as  Ext4 journaling file system
Mount point /boot
Mount options   defaults
Label   laalaa_boot
Reserved blocks 5%
Typical usage   standard
Bootable flag   on
Done setting up the partition
  Select a partition... pri/log 59.0 GB FREE SPACE
Create a new partition
  New partition size1 GB
  Type  Primary
  Location  Beginning
  Partition settings
Use as  physical volume for encryption
Encryption method   Device-mapper (dm-crypt)
Encryption  aes
Key size256
IV algorithmxts-plain64
Encryption key  Random key
Erase data  no
Bootable flag   off
Done setting up the partition
  Select a partition... pri/log 58.0 GB FREE SPACE
Create a new partition
  New partition size13 GB
  Type  Primary
  Location  Beginning
  Partition settings
Use as  physical volume for encryption
Encryption method   Device-mapper (dm-crypt)
Encryption  aes
Key size256
IV algorithmxts-plain64
Encryption key  Passphrase
Erase data  no
Bootable flag   off
Done setting up the partition
  Configure encrypted volumes
Write the changes to disk   Yes
Encryption configurationCreate encrypted volumes
Devices to encrypt  
  [*] /dev/sda2 (1000MB; crypto)
  [*] /dev/sda3 (13000MB; crypt)
Continue
Encryption configurationFinish
Encryption passphrase   
Re-enter passphrase 
  Select a partition... #1 13.0 GB f ext4
Partition settings
  Use asExt4 journaling file system
  Mount point   /
  Mount options defaults
  Label laalaa_root
  Reserved blocks   5%
  Typical usage standard
  Done setting up the partition
  Finish partitioning and w

Re: about 10th new install of bullseye BUT its not Bullseye, its bookworm!

[David Christensen]

On 6/5/24 08:21, gene heskett wrote:> 
But in asking how to get rid of [orca], the subject 
is always changed and I always get re-install instructions.  



Because that is the most practical and correct answer for your 
situation; especially given the disk access issues.



AIUI assistive technologies have been standard on FOSS graphical 
workstations for years.  It should be possible to turn assistance off, 
but it might not be possible to eliminate the machine code throughout 
the entire software stack.



I install Debian with the Xfce desktop, SSH server, and standard system 
utilities onto minimal hardware.  It takes a known amount of time and 
usually works.  I have successfully ignored assistive technologies for 
years (decades?).  Yes, the assistive technologies are wasting storage, 
memory, and cycles, and they create a larger threat surface, but those 
risks and costs are cheaper than me trying to understand and control all 
of the details.



Succeeding with software requires that you devise strategies to work 
within the limitations of the software.  Alternatively with FOSS, you 
can change the software.



David

Re: about 10th new install of bullseye

[David Christensen]

https://www.mail-archive.com/debian-user%40lists.debian.org/msg779582.html

Gene Heskett Fri, 18 Feb 2022 09:14:03 -0800



> On 2/19/22 06:31, Andrew M.A. Cater wrote:



On 6/4/24 03:26, gene heskett wrote:
How much longer till trixie is officially out??  What you are proposing 
sounds like several days work, and i have other irons in the fire. This 
release has been such a disaster for me because the install insists on 
installing and configuring orca and brltty w/o asking. I've done 40 some 
installs now, trying to stop it from wasting about a second while its 
yelling every keystroke at me because it thinks I'm blind. I finally 
have orca disabled and the computer is useful. The delays are a pain in 
the a$$ but i can do work now. It is not useful when orca is using 90% 
of a 6 core I5 yelling at me loud enough to announce and pronounce every 
keystroke or mouse motion/click loud enough to wake the neighbors.  The 
first 23 installs never asked me if I wanted that crap. And if you nuked 
the orca executable it would not reboot but hung forever waiting for 
orca to start. I have it usable, the installer AFAIAC is broken and I 
don't want to have to go through all that again. Until the installer 
ASKS me if I want it because it thinks I am blind, I have only one nerve 
left and and the suggestion that I do yet another install, is standing 
on it. Trying to remove it now, it insists on removing gnome and every 
dependency. I just checked again with synaptic, removing either orca or 
brltty still wants to destroy the system, Yet all I get when I fuss 
about the broken installer is "won't fix, not broken'.



I suggest:

1.  Back up the system configuration and data.

2.  Disconnect everything internal to the chassis except for the 
motherboard, power supply, front panel, fans, processor, memory, and one 
disk drive for the OS connected to the first IDE, SATA, or NVMe port.


3.  Disconnect everything external to the chassis except AC power, wired 
keyboard, wired mouse, wired monitor, and Ethernet.


4.  Boot into Setup and reset settings to factory defaults.  Choose 
between BIOS/Legacy and UEFI, if there is a choice.  Set the disk 
controller mode to AHCI.  Set the clock to UTC.


5.  Boot the disk manufacturer toolkit and wipe the OS drive -- secure 
erase for SSD's and zero-fill for HDD's.


I seem to recall that you have a 1 TB WD Black.  WD does not appear to 
offer a bootable disk drive toolkit (?):


https://support-en.wd.com/app/products/downloads/softwaredownloads

If you can find a FOSS toolkit to do a secure erase, that would be best. 
 Alternatively, find the non-zero blocks and zero them (a good job for 
a script).


6.  Boot debian-12.5.0-amd64-netinst.iso and install Debian onto the OS 
disk.  I partition manually with 1 GB EFI system partition, 1 GB boot 
partition, 1 GB random encrypted swap partition, and a small passphrase 
encrypted root partition (twice your current root partition usage?). 
Save the remaining free space for CAD, CNC, 3-D, etc., working/ scratch 
files and over-provisioning, to be configured after installation.  If 
your desktop environment of choice is not offered by d-i, do not install 
a desktop environment.


7.  At the end of installation, reboot.  Remove d-i media during POST. 
Verify the system boots from the OS disk.  Login and check vitals, but 
do not change anything.  Power off.


8.  Boot your FOSS toolkit of choice, or d-i rescue shell, and take a 
compressed image of the OS disk to a file on a USB HDD.



I use a version control system (CVS over SSH) for software development, 
but also find it to be very useful for system administration.



David

Re: advanced scripting problems - or wrong approach?

[David Christensen]

On 6/2/24 21:35, DdB wrote:

Am 02.06.2024 um 02:41 schrieb DdB:

Will share my findings, once i made more progress...


Here is what i've got before utilizing it:



datakanja@PBuster-NFox:/mnt/tmp$ cat test
#!/bin/bash -e
# testing usefulness of coprocess to control host and backup machine from a 
single script.
# beware: do not use subprocesses or pipes, as that will confuse the pipes 
setup by coproc!
# At this point, this interface may not be very flexible
# but trying to follow best practices for using coproc in bash scripts
# todo (deferred): how to handle stderr inside coproc?
# todo (deferred): what, if coproc dies unexpectedly?

# setting up the coprocess:
stdout_to_ssh_stdin=5 # arbitrary choice outside the range of used file 
desciptors
stdin_from_ssh_stdout=6

coproc SSH { bash; } # for testing purposes, i refrain from really 
involving ssh just yet and replace it with bash:

# save filedescriptors by duplicating them:
eval "exec ${stdin_from_ssh_stdout}<&${SSH[0]} 
${stdout_to_ssh_stdin}>&${SSH[1]}"
echo The PID of the coproc is: $SSH_PID # possibly useful for inspection

unique_eof_delimirer=""
line=""
# collect the output available and print it locally (synchonous):
function print-immediate-output () {
while IFS= read -r -u "${stdin_from_ssh_stdout}" line
do
if [[ "${line:0-5:5}" == "$unique_eof_delimirer" ]] # currently, 
the length is fixed
then
line="${line%}"
if [[ ! -z $line ]]
then
printf '%s\n' "$line"
fi
break
fi
printf '%s\n' "$line"
done
}

# send a single command via ssh and print output locally
function send-single-ssh-command () {
printf '%s\n' "$@" >&"${stdout_to_ssh_stdin}"
printf '%s\n' "echo '"$unique_eof_delimirer"'" 
>&"${stdout_to_ssh_stdin}"
print-immediate-output
}


send-single-ssh-command "find . -maxdepth 1 -name [a-z]\*" # more or less a 
standard command, that succeeds
send-single-ssh-command "ls nothin" # more or less a standard command, that 
fails

# tearing down the coprocess:
printf "%s\n" "exit" >&"${stdout_to_ssh_stdin}" # not interested in any 
more output (probably none)
wait
# Descriptors must be closed to prevent leaking.
eval "exec ${stdin_from_ssh_stdout}<&- ${stdout_to_ssh_stdin}>-"

echo "waited for the coproc to end gracefully, done"

datakanja@PBuster-NFox:/mnt/tmp$ ./test
The PID of the coproc is: 28154
./test
./out
ls: Zugriff auf 'nothin' nicht möglich: Datei oder Verzeichnis nicht gefunden
waited for the coproc to end gracefully, done
datakanja@PBuster-NFox:/mnt/tmp$



"test" is both a program and a shell builtin.  I suggest that you pick 
another, non-keyword, name for your script.



I suggest adding the Bash option "-u' (nounset).


Your file descriptor duplication, redirection, etc., seems overly 
complex.  Would not it be easier to use the coproc handles directly?


2024-06-03 08:49:41 dpchrist@laalaa ~/sandbox/bash
$ nl coproc-demo
 1  #!/usr/bin/env bash
 2  # $Id: coproc-demo,v 1.3 2024/06/03 15:49:36 dpchrist Exp $
 3  set -e
 4  set -u
 5  coproc COPROC { bash ; }
 6  echo 'echo "hello, world!"' >&"${COPROC[1]}"
 7  read -r reply <&"${COPROC[0]}"
 8  echo $reply
 9  echo "exit" >&"${COPROC[1]}"
10  wait $COPROC_PID

2024-06-03 08:49:44 dpchrist@laalaa ~/sandbox/bash
$ bash -x coproc-demo
+ set -e
+ set -u
+ echo 'echo "hello, world!"'
+ bash
+ read -r reply
+ echo hello, 'world!'
hello, world!
+ echo exit
+ wait 4229


David

Re: advanced scripting problems - or wrong approach?

[David Christensen]

On 6/1/24 00:20, DdB wrote:

Hello,

for years have i been using a self-made backup script, that did mount a
drive via USB, performed all kinds of plausibility checks, before
actually backing up incrementally. Finally verifying success and logging
the activities while kicking the ISB drive out.

Since a few months, i do have a real backup server instead, connecting
to it via ssh i was able to have 2 terminals open and back up manually.
Last time, i introduced a mistake by accident and since, i am trying to
automate the whole thing once again, but that is difficult, as the load
on the net is huge, mbuffer is useful in that regard. So i was intending
to have just one script for all the operations using coproc to
coordinate the 2 servers.

But weird things are going on, i cant reliably communicate between host
and backup server, at least not automatically.

Searching the web, i found:
https://github.com/reconquest/coproc.bash/blob/master/REFERENCE.md
But i was unable to get this to work, which seems to indicate, that i am
misunderstanding something.
The only success i had was to "talk" to a chess engine in a coprocess,
which did go well. But neither bash nor ssh are cooperating, i may have
timing issues with the pipes or some other side effects.

How can i describe? For example if i start this:

#!/bin/bash -e

coproc { bash; }
exec 5<&${COPROC[0]} 6>&${COPROC[1]}
fd=5

echo "ls" >&6
while IFS= read -ru $fd line
do
 printf '%s\n' "$line"
done

printf "%s\n" "sleep 3;exit" >&6
while IFS= read -ru $fd line
do
 printf '%s\n' "$line"
done

exec 5<&- 6>&-

wait
echo waited, done


i get the output from ls, but then the thing is hanging indefinitely,
apparently not reaching the exit line. :(

Anyone who can share his experience to advance my experimenting?
DdB




https://en.wikipedia.org/wiki/XY_problem


Please define the root problem you are trying to solve.


David

Re: "Repeaters", etc.

[David Christensen]

On 5/30/24 03:14, Roger Price wrote:

On Wed, 29 May 2024, David Christensen wrote:

On 5/29/24 03:36, Roger Price wrote:

On Tue, 28 May 2024, David Christensen wrote:

On 5/28/24 00:28, Roger Price wrote:
I wired my place Cat5. A lot of work, and I regretted it.  I live 
in the hills behind Nice, an area with a lot of lightning.  The 
overhead line to my place took a hit and thanks to the Cat5 
conductivity I lost equipment.


How do you know that the damage your equipment suffered was due to 
the Cat 5e wiring and not due to the electrical power conductors?


Electrical power to my computers comes through 30mA differential 
circuit breakers to Eaton Ellipse 1600 UPS units. I had no such 
protection for the telephone signal, and I saw flashes at the 
telephone junction box.  So I summise that the Cat5 wiring did the 
damage.


Those UPS's should be able to protect telephone and Ethernet, in 
addition to electrical power.  Have you applied the UPS's to the 
former two?


The UPS's stand next to the workstations and well away from the place 
where the telephone line arrives, so I didn't use the UPS's to protect 
the telephone line. My fault.  Later I added a surge protector to the 
copper telephone line.  I am now in the process of migrating from copper 
to fiber so I will need an extra UPS next the fiber terminator.


Roger

===

https://www.eaton.com/sg/en-us/catalog/backup-power-ups-surge-it-power-distribution/eaton-ellipse-pro-ups.html


PS: I once had a lightning strike direct to the house.  Frightening.  
Although every differential circuit breaker in the house tripped, the 
circuit board in the UPS melted.  But even when melting, it protected 
the Dell T7500. No damage to the T7500, no data lost. I took a photo of 
the melt, sent it to Eaton, and they replaced the UPS.



Have you consider applying lightning protection to your house?

1.  Lightning rods, down lines, ground rods, perimeter ground loop, etc..

2. Lightning arresters at the electrical, telephone, CATV, etc. service 
entrance points.



David

Re: "Repeaters", etc.

[David Christensen]

On 5/29/24 03:36, Roger Price wrote:

On Tue, 28 May 2024, David Christensen wrote:

On 5/28/24 00:28, Roger Price wrote:
I wired my place Cat5. A lot of work, and I regretted it.  I live in 
the hills behind Nice, an area with a lot of lightning.  The overhead 
line to my place took a hit and thanks to the Cat5 conductivity I 
lost equipment.


How do you know that the damage your equipment suffered was due to the 
Cat 5e wiring and not due to the electrical power conductors?


Electrical power to my computers comes through 30mA differential circuit 
breakers to Eaton Ellipse 1600 UPS units. I had no such protection for 
the telephone signal, and I saw flashes at the telephone junction box.  
So I summise that the Cat5 wiring did the damage.


Roger



https://www.eaton.com/sg/en-us/catalog/backup-power-ups-surge-it-power-distribution/eaton-ellipse-pro-ups.html

https://standards.globalspec.com/std/104626/iec-61643-1


Those UPS's should be able to protect telephone and Ethernet, in 
addition to electrical power.  Have you applied the UPS's to the former two?



David

Re: "Repeaters", etc.

[David Christensen]

On 5/28/24 17:10, John Hasler wrote:

David writes:

AIUI in the USA for residential 120/240V single-phase three-wire service
drops, electrical utilities either run all three phases along the
distribution line or they run two phases.  Running one phase and a neutral
instead of two phases would reduce the power by the square root of 3


Here in rural Wisconsin the 7200V distribution line leaves the
substation as three phases and a grounded neutral.  This eventually
branches out into three single phase lines consisting of a phase and a
grounded neutral.  The pole pigs are connected phase to neutral.



Interesting.  STFW I found an article and a web site that clarifies the 
above arrangements and more:


https://electrical-engineering-portal.com/primary-distribution-circuits


David

Re: "Repeaters", etc.

[David Christensen]

On 5/28/24 12:47, gene heskett wrote:

On 5/28/24 15:29, Roy J. Tellason, Sr. wrote:

On Tuesday 28 May 2024 01:49:52 pm Paul M Foster wrote:

I've never see a 3 phase in a house. Common in commercial/industrial,
though.
Residential installations (talking in the US here) typically involve 
*one* transformer tapping a single phase out of the three that are up 
there on the pole.  The secondary is center-tapped,  and it's that 
point which is grounded at the service entrance.  Running 3-phase 
power requires *three* transformers up on the pole,  much more in the 
way of expense if you want that for some reason,  and I don't know of 
anybody that does that.  Even those who are into having some 
nontrivial machinery around seem these days to use a VFD to give them 
multiple phases at the machine,  rather than going through the expense 
of having it run in from the pole...


And here you have it from another CET.

Cheers, Gene Heskett, CET.



AIUI in the USA for residential 120/240V single-phase three-wire service 
drops, electrical utilities either run all three phases along the 
distribution line or they run two phases.  Running one phase and a 
neutral instead of two phases would reduce the power by the square root 
of 3.



Running one phase and using the Earth as the return conductor is very 
dangerous and not modern practice.



David

Re: "Repeaters", etc.

[David Christensen]

On 5/28/24 00:28, Roger Price wrote:
I wired my place Cat5. A lot of work, and I regretted it.  I live in the 
hills behind Nice, an area with a lot of lightning.  The overhead line 
to my place took a hit and thanks to the Cat5 conductivity I lost 
equipment.



If your electrical utility uses pole-mounted distribution lines, 
transformers, service drops, etc., and lightning strikes the 
high-voltage conductors, there will be a surge on the customer service 
conductors that places persons and property at risk.  If lightning jumps 
to the customer service conductors, then the risk to persons and 
property can be extreme.



How do you know that the damage your equipment suffered was due to the 
Cat 5e wiring and not due to the electrical power conductors?



David

Re: "Repeaters", etc.

[David Christensen]

On 5/27/24 19:05, Paul M Foster wrote:

I did some more research, and it looks like I must have misstated the
problem.

Let's assume I can't get in the attic and wire the place. Let's 
assume that I've got a wireless router/modem in, say, the garage. 
Let's say I have three rooms with devices I want to connect (one way 
or another) to my router/modem.


It appears there are two solutions. One is wifi extenders, and one
is a mesh network. In both cases, the device sits in the room and 
communicates via wifi to the modem/router. The devices in the room 
connect to the device via ethernet cable.


How does that sound? Any dissenting opinions? Any brand 
recommendations?



On 5/27/24 20:14, Paul M Foster wrote:
Coincidentally, I used to be an electrician too, but we almost never 
ran low voltage except for doorbells.


The house in question appears to have a generous attic, but they've 
blown in two feet of insulation I'd rather not disturb. And that

much insulation makes the headers of walls very hard to find. Also,
I'm not in my 20s anymore, and crawling around in attics is
difficult.

In the house I'm living in now, I did go into the attic years ago 
with cat 5e and wired up the living room.


FWIW, in the house we're buying, I need internet (wired) in the 
living room, bedroom 2 and bedroom 4. Also, it's concrete block 
construction (outer walls).



On 5/27/24 21:50, Paul M Foster wrote:

Well, if I'm gonna run cat 5, I might as well just put a jack in each
room.> No POE needed. The only reason for wifi at all in this case is
so I don't *have*  to run cat 5.



From what I've read, TP-Link gets good reviews.


FYI my previous coax cable suggestion will likely involve MoCA:

https://en.wikipedia.org/wiki/Multimedia_over_Coax_Alliance


Another idea is power-line communication.  I seem to recall reading that 
an RF choke should be installed on the incoming electrical service to 
prevent interference to/from the neighbors, but STFW I do not see any 
mention of that today:


https://en.wikipedia.org/wiki/Powerline_Ethernet


David

Re: "Repeaters", etc.

[David Christensen]

On 5/27/24 14:09, Paul M Foster wrote:

Folks:

At some point this year, I'm moving into a new house, and it is not wired
for internet (WHY aren't new houses wired with Cat5/6/7?). The local
internet provider will likely provide a wireless router, as they all do. My
idea is to put a device which receives wireless signal from the
router/modem, and has an RJ45 jack in it in each room. So each room would
have one of these, and the devices in it would be hooked to that device via
cat 5e. I hope that's clear.

I'd like to shop for such a device, but I don't know what it's called. Can
anyone provide advice, and possibly preferred brand names? I'd appreciate
it.

Paul



Is the house wired for cable television (RG-6 coaxial cable)?  If so, 
and you choose the right Internet provider, you might be able to get a 
"main box" with 1+ type F connectors, 4 @ RJ-45 Gigabit ports,  Wi-Fi 
access point, etc., and "satellite boxes" with 1+ type F connector, 1+ 
RJ-45 Gigabit ports, Wi-Fi access point, etc..  Ask you cable 
television/ Internet provider.



An alternative to running Ethernet cables inside walls is to run the 
cables on the surface -- e.g. staple to wall along floor molding, drill 
and pull through walls as required, paint to match, etc..  Cat 5e is 
smaller diameter and easier to work with than Cat 6a.  I surface wired 
my house with Cat 5e ~20 years ago and have 1000BASE-T (Gigabit 
Ethernet) switches and NIC's.  All of the cable runs are under ~20 
meters, so I should be able to upgrade to 2.5GBASE-T.



David

Re: HDD long-term data storage with ensured integrity

[David Christensen]

On 5/3/24 04:26, Marc SCHAEFER wrote:

On Mon, Apr 08, 2024 at 10:04:01PM +0200, Marc SCHAEFER wrote:

For off-site long-term offline archiving, no, I am not using RAID.


Now, as I had to think a bit about ONLINE integrity, I found this
comparison:

https://github.com/t13a/dm-integrity-benchmarks

Contenders are btrfs, zfs, and notably ext4+dm-integrity+dm-raid

I tend to have a biais favoring UNIX layered solutions against
"all-into-one" solutions, and it seems that performance-wise,
it's also quite good.

I wrote this script to convince myself of auto-correction
of the ext4+dm-integrity+dm-raid layered approach.



Thank you for devising a benchmark and posting some data.  :-)


FreeBSD also offers a layered solution.  From the top down:

* UFS2 file system, which supports snapshots (requires partitions with 
soft updates enabled).


* gpart(8) for partitions (volumes).

* graid(8) for redundancy and self-healing.

* geli(8) providers with continuous integrity checking.


AFAICT the FreeBSD stack is mature and production quality, which I find 
very appealing.  But the feature set is not as sophisticated as ZFS, 
which leaves me wanting.  Notably, I have not found a way to replicate 
UFS snapshots directly -- the best I can dream up is synchronizing a 
snapshot to a backup UFS2 filesystem and then taking a snapshot with the 
same name.



I am coming to the conclusion that the long-term survivability of data 
requires several components -- good live file system, good backups, good 
archives, continuous internal integrity checking with self-healing, 
periodic external integrity checking (e.g. mtree(1)) with some form of 
recovery (e.g. manual), etc.. If I get the other pieces right, I could 
go with OpenZFS for the live and backup systems, and worry less about 
data corruption bugs.



David

Re: realpath quoting

[David Christensen]

On 5/3/24 04:34, jeremy ardley wrote:


On 3/5/24 19:06, Greg Wooledge wrote:

I would suggest that if you need to use a debugger to track down a bug
in your program, you should use filenames that don't require quoting
when you set up your tests.


1970's style static test cases are not relevant here.

In the real world...  I download files generated by another system that 
are constantly changing content and with names I don't control.


My workflow is to download a new file from a remote source and then run 
my processor over it.


As a necessary consequence I need the fully quoted or escaped file name 
of the new file to feed to the processor/debugger.


I can obviously add an extra step to the process to convert the new file 
name to something acceptable before processing. However, my question was 
how to avoid that extra step by getting fully quoted filenames to process.





So, you are copying and pasting file names via some clipboard?  emacs(1) 
might have a way to put a filter into that process, but I am unaware of 
a similar feature using Xfce and Terminal (my platform).



I have tried renaming files in similar situations, but you will want to 
rename them everywhere if you use rsync(1).



What if you downloaded files to a directory with a well-formed name and 
added a feature to your script to process files that appear in that 
directory?



David

Re: realpath quoting

[David Christensen]

On 5/3/24 04:09, Greg Wooledge wrote:

On Thu, May 02, 2024 at 10:18:03PM -0700, David Christensen wrote:

I am unable to find $'string' in the dash(1) man page (?).  As I typically
write "#!/bin/sh" shell scripts, writing such to deal with file names
containing non-printing characters is going to baffle me.


Currently, $' quoting is a bash extension.  It's supposed to appear in
some future edition of POSIX, at which point shells like dash will be
required to adopt it (whenever they get around to it).  For now, though,
you should consider it bash only.



Thank you for the clarification.  :-)


David

Re: realpath quoting

[David Christensen]

On 5/2/24 19:56, Max Nikulin wrote:

On 03/05/2024 09:19, Greg Wooledge wrote:

I still insist that this is a workaround that should *not*  be used
to try to cancel out quoting bugs in one's shell scripts.


There are still specific cases when quoting is necessary, e.g. ssh 
remote command 



+1


(however you have to be sure concerning shell on the 
remote host).



+1


In BASH printf has %q format. GNU coreutils supports it as well, but 
dash does not, so be careful.



My practice is to start with '#!/bin/sh' and migrate to '#!/usr/bin/env 
perl' as complexity increases.




Likely Jeremy's case does not really require this kind of quoting.



We need to see the full range of file names the OP is trying to deal with.


While "ls -l" output is for humans, realpath is often used in scripts. 
Certainly it should nor return quoted output by default. I am in doubts 
if a dedicated option should be added to realpath.



Thank you for helping me realize that my solution fails to print the 
resolved absolute file name.  Here is the updated solution:


2024-05-02 21:57:56 dpchrist@laalaa ~
$ touch 'name with spaces'

2024-05-02 22:23:01 dpchrist@laalaa ~
$ touch 'name with
> newline'

2024-05-02 22:28:36 dpchrist@laalaa ~
$ perl -MString::ShellQuote '-MFile::Spec::Functions qw(rel2abs)' -e 
'print shell_quote(map { rel2abs $_ } @ARGV), "\n"' name*

'/home/dpchrist/name with
newline' '/home/dpchrist/name with spaces'


David

Re: realpath quoting

[David Christensen]

On 5/2/24 19:19, Greg Wooledge wrote:

On Thu, May 02, 2024 at 07:11:46PM -0700, David Christensen wrote:

Perhaps Perl and the module String::ShellQuote ?

2024-05-02 18:50:28 dpchrist@laalaa ~
$ touch "name with spaces"

2024-05-02 18:50:45 dpchrist@laalaa ~
$ touch "name with\nnewline"


You didn't create a name with a newline in it here.  You created a name
with a backslash in it.  If you wanted a newline, you would have to use
the $'...' quoting form (in bash).

 touch $'name with\nnewline'



Thank you for the clarification.


RTFM bash(1):

QUOTING
...
Enclosing  characters  in  double quotes preserves the literal
value of all characters within the quotes, with the exception of
$, `,  \,  and, when history expansion is enabled, !. ...
The backslash retains its special  meaning  only when followed
by one of the following characters: $, `, ", \, or .
...
Words of the form $'string' are treated specially.  The word
expands to string,  with backslash-escaped characters replaced
as specified by the ANSI C standard.


I found another way to obtain a file name containing a newline -- by 
pressing  when typing a double-quoted string literal:


2024-05-02 21:52:23 dpchrist@laalaa ~
$ touch "foo
> bar"

2024-05-02 21:52:29 dpchrist@laalaa ~
$ ls -l foo*
-rw-r--r-- 1 dpchrist dpchrist 0 May  2 21:52 'foo'$'\n''bar'

2024-05-02 21:52:36 dpchrist@laalaa ~
$ perl -MString::ShellQuote -e 'print shell_quote(@ARGV), "\n"' foo*
'foo
bar'


It also seems to work for single-quoted string literals:

2024-05-02 21:57:08 dpchrist@laalaa ~
$ touch 'foo
> bar'

2024-05-02 21:57:14 dpchrist@laalaa ~
$ ls -l foo*
-rw-r--r-- 1 dpchrist dpchrist 0 May  2 21:57 'foo'$'\n''bar'

2024-05-02 21:57:18 dpchrist@laalaa ~
$ perl -MString::ShellQuote -e 'print shell_quote(@ARGV), "\n"' foo*
'foo
bar'


I am unable to find $'string' in the dash(1) man page (?).  As I 
typically write "#!/bin/sh" shell scripts, writing such to deal with 
file names containing non-printing characters is going to baffle me.




I still insist that this is a workaround that should *not* be used
to try to cancel out quoting bugs in one's shell scripts.  Just write
the shell scripts correctly in the first place.



I would if I could.


While I am also unable to write Perl scripts correctly in the first 
place, the quoting rules are easier.



David

Re: realpath quoting

[David Christensen]

On 5/2/24 15:59, jeremy ardley wrote:
I have a need  to get the full path of a file that has spaces in its 
name to use as a program argument


e.g.

jeremy@client:~$ ls -l name\ with\ spaces
-rw-r--r-- 1 jeremy jeremy 0 May  3 06:51 'name with spaces'
jeremy@client:~$ realpath name\ with\ spaces
/home/jeremy/name with spaces


The spaces without quotes cause problems with subsequent processing.

Can realpath or other utility return a quoted pathname?



Perhaps Perl and the module String::ShellQuote ?

2024-05-02 18:50:28 dpchrist@laalaa ~
$ touch "name with spaces"

2024-05-02 18:50:45 dpchrist@laalaa ~
$ touch "name with\nnewline"

2024-05-02 19:06:01 dpchrist@laalaa ~
$ perl -MString::ShellQuote -e 'print shell_quote(@ARGV), "\n"' name*
'name with spaces' 'name with\nnewline'


David

Re: Debian@IBMx3550

[David Christensen]

On 4/23/24 14:35, Greg wrote:

Hi there,

I got refurb IBM x3550 M3 7944 server and I'm a bit lost. Is there any 
Linux/Debian software (some gui would be nice) to monitor fan speed, 
temperatures, voltages, disks.. ?


Thanks in advance for any help
Greg




If you installed the Xfce desktop, add the "Sensor" plug-in/ applet to 
the Panel.  Panel can display various temperatures, fan speeds, etc..



Note that you may to set the SUID bit on /usr/sbin/hddtemp for disk 
drive readings to be available:


# chmod u+s /usr/sbin/hddtemp


David

Re: Subject: Glitchy sound in Steam games after hard drive upgrade

[David Christensen]

On 4/23/24 09:02, debian-u...@howorth.org.uk wrote:

Charlie Gibbs  wrote:

On 2024-04-22 16:50, Jeffrey Walton wrote:


What are the old and new hard drive model numbers and specs?


The old drive is a Western Digital WD5000YS (500GB SATA).
The new drive is a Western Digital Red, WF40EFPX (4TB SATA).


According to my searches, there's no such disk as a WF40EFPX. Are you
sure that's what it is? If by any chance it is a WD40EFRX then that is
certainly slower than your old drive, so may cause some problems as
suggested.



I doubt the new drive is slower than the old drive:

- https://www.harddrivebenchmark.net/hdd.php?hdd=WDC%20WD5000YS

WDC WD5000YS  425

- https://www.harddrivebenchmark.net/hdd.php?hdd=WDC%20WD40EFRX

WDC WD40EFRX1,943


David

Re: Subject: Glitchy sound in Steam games after hard drive upgrade

[David Christensen]

On 4/22/24 21:26, Charlie Gibbs wrote:

On 2024-04-22 16:50, Jeffrey Walton wrote:

What are the old and new hard drive model numbers and specs?


The old drive is a Western Digital WD5000YS (500GB SATA).



https://www.newegg.com/western-digital-re2-wd5000ys-500gb/p/N82E16822136032?Item=N82E16822136032



The new drive is a Western Digital Red, WF40EFPX (4TB SATA).



https://www.westerndigital.com/products/internal-drives/wd-red-plus-sata-3-5-hdd?sku=WD40EFPX


Both drives are spinning rust.  I'm upgrading for the increased 
capacity, i.e. to store more MP3s and videos.


Many thanks to all who have replied.  When my schedule permits me to 
continue experimenting, I'm going to try copying /etc from the old drive 
to the new one.   I've already learned how _not_ to do this:


Boot from the new drive
$ su root
# cd /
# mv etc etc.ori
# rsync -av /mnt/backup/etc .

The second line makes the system fall over and makes logins impossible. 
It took a boot from the rescue CD to undo the damage, which fortunately 
was easy since the deadly step at least succeeded in backing up /etc.


Next time I'll do it while booted from the old drive.



Copying an entire /etc directory from one machine to another requires a 
highly controlled environment and lot of engineering.  I have always 
migrated /etc settings from one OS instance to another OS instance by 
hand, one service/ configuration file at a time.



Can you leave the 500 GB HDD operational and use the 4 TB HDD for data?


David

Re: Strange New Installation Behavior

[David Christensen]

On 4/22/24 06:00, Stephen P. Molnar wrote:
I am running Bookworm and cleaned up a couple of files too many 
resulting in a messed up Xfce Desktop. I decided that this would be a

 good time to reinstall the Bullseye.

I made a backup of my /home/comp directory using Deja-dup.

I downloaded and ran the 512 check sum on a copy of 
Debian-12.5.0-amd64-DVD-1.iso and ran the Graphical Install mode on

the 1.0 TD SSD on my Computer. The installation went smoothly without
any warning or error messages.

I logged in as root to set up the Desktop and, much to my surprise, 
found that my previous Desktop configuration was still there!!???

This was also the case when I logged in user!!!???

I have been using computers in my work since the 1960, the era of the
 Hollerith Card and tape drives and Linux since early days of
Slackware and the Red Hat Mother's Day Edition. Now I am not a
computer expert but a Research Chemist.  I have installed Linux OS's
many times and consider Linux my primary computational platform. I
have never encountered the situation and have no ideas as to what is
going on.

I have been runnind Debian since Etch.

I would appreciate some insight into what might be going on.

Thanks in advance.

Stephen P. Molnar, Ph.D. https://insilicochemistry.net (614)312-7528
(c) Skype: smolnar1



On 4/22/24 09:34, Stephen P. Molnar wrote:

I did not want to revert to Bullseye, but to reinstall to Bookworm.


I suggest that you buy a good 16 GB USB flash drive and install Debian 
12 with Xfce onto it.  Having a working live USB stick is very useful 
for low-level disk drive chores such as examining, backing up, testing, 
repairing, restoring, wiping, etc..  Use it to:


1.  Ensure that you have a good backup of your 1 TB SSD.

2.  Make additional backups or archives of all or part of your 1 TB SSD. 
 Note the mantra: "Data does not exist unless it exists in three places".


3.  Wipe the SSD so that the Debian installer will see a blank disk and 
respond accordingly when you later install Debian onto the SSD.



Regarding copying a home directory from one OS installation to another 
OS installation, please see my comments on another thread:


https://lists.debian.org/debian-user/2024/04/msg00336.html


Once you have logged in to your new account on your fresh install, I 
suggest that you restore your /home/comp backup to a subdirectory and 
manually copy/ move/ edit/ merge files and directories from the restore 
subdirectory into your fresh home directory.  Be very careful not to 
damage or delete anything needed by your fresh desktop or applications.



David

Re: Subject: Glitchy sound in Steam games after hard drive upgrade

[David Christensen]

On 4/21/24 22:33, Charlie Gibbs wrote:

I should probably be posting this to the Steam forums, but
most of the denizens there are Windows people so I might be
better off letting you Debian gurus have a go at it first.

TL;DR: Copying an existing /home into a fresh Debian installation
causes audio in Steam games to glitch - but all other sound is OK.

Full description:

I have a machine in the living room that stores MP3s and videos
and serves them to other machines on our network as well as playing
them locally on our TV's big screen.  I also play a few Steam games
(e.g. Portal) on it.  It's a 2007-vintage machine, but it has 8GB
of RAM and enough CPU power to do the job, and runs the latest
version of Bookworm.

Recently I decided to upgrade its storage capacity, and replaced
its 500GB hard drive (which was pretty large at the time I bought
it) with a 4TB drive.  I did an install from scratch using a
network install CD, then copied my /home partition (using rsync)
from the old drive.  Everything works great with one exception:
when I fire up Portal the sound gets glitches about once a second.
This only happens with Steam games; I can play MP3s and videos
with mpv and the sound is perfect, as it is when watching YouTube
videos.  If I swap the old drive back in everything is fine.

Obviously my Steam programs and configuration files are in my
home directory, since the updated system comes up icons and all
without re-installing Steam, and can find everything it needs to
run the games.  But perhaps there are a few files somewhere else
(/usr?) containing information critical to audio for Steam.

Any ideas?

(Side question: is this an acceptable way to upgrade a hard drive?)



Copying a home directory from one OS instance to another OS instance 
sounds risky, especially as I run various OS's.  I have several 
instances of Debian 11, and would not consider them to be identical 
enough to try it.  I only touch the content I create or have learned how 
to manage.



I put my OS on a small SSD and the vast majority of my data on HDD RAID 
in a file server.



As I am the only user on my Debian daily driver, I leave the /home 
directory on the root file system and keep as little as possible in it.



I mount the file server shares under /mnt, and create symlinks in my 
home directory that point into the mounted file system.



I use CVS for project working directories.  To migrate to a new home 
directory, I check in the projects in the old home and check out the 
project in the new home.



I use Firefox and its sync feature.  To migrate to a new home, I start 
Firefox, log in, wait for my settings to sync, and then check all of the 
settings by hand.



I use Thunderbird.  To migrate to a new home, I create a tarball of my 
Thunderbird profile directory on the old machine, expand the tarball on 
the new machine, and configure Thunderbird to use that profile.



I do not attempt to migrate any of the various home directory 
configuration directories; I let the installer and/or package manager 
create them, and let the desktop, apps, etc., manage them.



David

Re: Debian 11 Xfce panel Network Manager applet has disappeared

[David Christensen]

On 4/14/24 05:29, David Christensen wrote:

debian-user:

I have a Dell Latitude E6520:

2024-04-14 04:28:39 dpchrist@laalaa ~
$ cat /etc/debian_version ; uname -a
11.9
Linux laalaa 5.10.0-28-amd64 #1 SMP Debian 5.10.209-2 (2024-01-31) 
x86_64 GNU/Linux


2024-04-14 04:34:40 dpchrist@laalaa ~
$ dpkg-query -l xfce4 network-manager network-manager-gnome
Desired=Unknown/Install/Remove/Purge/Hold
| 
Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend

|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
||/ Name  Version  Architecture Description
+++-=---=
ii  network-manager   1.30.6-1+deb11u1 amd64    network 
management framework (daemon and userspace tools)
ii  network-manager-gnome 1.20.0-3 amd64    network 
management framework (GNOME frontend)
ii  xfce4 4.16 all  Meta-package for 
the Xfce Lightweight Desktop Environment



I have used the Xfce panel Network Manager applet for many years. 
Tonight, I noticed that it has disappeared (!).



I compared the problem machine against another with a working Xfce panel 
Network Manager applet, and discovered that the Status Tray Plugin was 
missing.  I may have deleted Status Tray Plugin while cleaning, but did 
not notice the change immediately (?).



So, I added Status Tray Plugin and now Network Manager has returned.


David

Re: Fwd: Debian 11 Xfce panel Network Manager applet has disappeared

[David Christensen]

On 4/19/24 00:16, Florent Rougon wrote:

Another thing: did you look into ~/.xsession-errors?
(Sorry if this was already mentioned and I missed it.)



Please see attached copy of ~/.xsession-errors, taken immediately after 
system restart and login.



"nm-applet" does not appear in .xsession-errors, but there are plenty of 
other warnings and error messages.  Perhaps the failure of nm-applet is 
a symptom of a more fundamental failure (?).




More involved: if you can't find any trace of the applet doing
something, maybe rebuilding the package after adding a few fprintf()
calls would help.



I think it is time for a bug report.


David


.xsession-errors-20240419-121605.gz
Description: application/gzip

Re: Fwd: Debian 11 Xfce panel Network Manager applet has disappeared

[David Christensen]

On 4/18/24 09:46, Gareth Evans wrote:

On Thu 18/04/2024 at 11:05, David Christensen  wrote:


Move aside the ~/.config/xfce4 directory:
...
Restart -- screen with wallpaper alone.
...


Hi David,

Starting from Mate DE only and some old (bookworm) XFCE config files, if I:

$ sudo apt install task-xfce-desktop

then log out and into XFCE, I get an XFCE desktop with wallpaper, desktop 
icons, panel, main menu, launchers, window buttons, notification area.

Then I:

$ mv ~/.config/xfce4 ~/.config/xfce4.old

then logout and into XFCE again, I get wallpaper, desktop icons, panel, main 
menu, window buttons, notification area, but only non-functional launcher 
placeholder icons.

$ diff ~/.config/xfce4 ~/.config/xfce4.old
Common subdirectories: /home/user/.config/xfce4/desktop and 
/home/user/.config/xfce4.old/desktop
Only in /home/user/.config/xfce4.old: panel   }
Only in /home/user/.config/xfce4.old: src }  <-- these are 
subdirectories
Only in /home/user/.config/xfce4.old: terminal}
Common subdirectories: /home/user/.config/xfce4/xfconf and 
/home/user/.config/xfce4.old/xfconf
Common subdirectories: /home/user/.config/xfce4/xfwm4 and 
/home/user/.config/xfce4.old/xfwm4
$

(though there probably wasn't much there to start with)

I'm not sure what would cause this difference in behaviour (though wonder if 
this might suggest more amiss with your XFCE installation) and I will watch 
this thread with interest.

Also I just "rediscovered" that reinstalling task-xfce-desktop doesn't 
reinstall those packages which it brings in in the first place, though I'm sure you knew 
that :)

Sorry I couldn't be of more help.



Thanks for the suggestions.  We have more information now.  :-)


David

Re: Fwd: Debian 11 Xfce panel Network Manager applet has disappeared

[David Christensen]

On 4/18/24 07:28, Max Nikulin wrote:

On 18/04/2024 17:05, David Christensen wrote:

$ mv .config/xfce4-20240418-180045/ .config/xfce4

Restart -- back to Xfce panel with no Network Manager.


Try to create a new system user and log in. Is nm-applet present?



Logging in using another previously working account produces the same 
result -- Xfce Panel displays Notification Plugin (bell icon), but no 
Network Manager icon.



Creating a new account and logging in produces the same result.


David

Re: Fwd: Debian 11 Xfce panel Network Manager applet has disappeared

[David Christensen]

On 4/18/24 05:34, e...@gmx.us wrote:

On 4/18/24 05:27, David Christensen wrote:

On 4/17/24 12:37, Richmond wrote:

What are the permissions on the nm-applet binary? maybe it doesn't have
permission to execute, or the process which starts it doesn't have
permission.



2024-04-18 02:24:20 root@laalaa ~
# ls -l `which nm-applet`
-rwxr-xr-x 1 root root 250784 Feb 27  2021 /usr/bin/nm-applet


I do not know what binary starts nm-applet, but here is a WAG:

2024-04-18 02:27:13 root@laalaa ~
# ll -l `which xfce4-panel`
-rwxr-xr-x 1 root root 363328 2021-02-27 08:29:44 /usr/bin/xfce4-panel


Can PPID tell you?



Yes.  xfce4-session is the parent of nm-applet:

2024-04-18 10:27:32 dpchrist@laalaa ~
$ ps -Ao "%p %P %a" | grep nm-applet
   15301379 nm-applet
   39373865 grep nm-applet

2024-04-18 10:27:44 dpchrist@laalaa ~
$ ps -Ao "%p %P %a" | grep 1379
   13791344 xfce4-session
   14211379 /usr/bin/ssh-agent x-session-manager
   14581379 xfwm4 --display :0.0 --sm-client-id 
2d1bdc0a5-06f2-4835-944e-7e402203bd9f
   14711379 xfsettingsd --display :0.0 --sm-client-id 
29896f657-477c-498a-8156-db9d51f74bcf
   14971379 xfce4-panel --display :0.0 --sm-client-id 
23fd394e3-a4ee-485b-b43c-38bf845d6699
   15011379 xfdesktop --display :0.0 --sm-client-id 
2c913de99-acac-4d70-b6e9-8eda44b6f6da
   15061379 xfce4-power-manager --restart --sm-client-id 
24cebdb39-bdc2-496f-aab0-7b11397b48d3

   15131379 /usr/lib/x86_64-linux-gnu/xfce4/notifyd/xfce4-notifyd
   15151379 light-locker
   15161379 /usr/bin/python3 /usr/share/system-config-printer/applet.py
   15171379 
/usr/lib/policykit-1-gnome/polkit-gnome-authentication-agent-1

   15241379 xiccd
   15301379 nm-applet
   39403865 grep 1379

2024-04-18 10:29:20 dpchrist@laalaa ~
$ ls -l `which xfce4-session`
-rwxr-xr-x 1 root root 260496 Dec 23  2020 /usr/bin/xfce4-session


David

Re: Debian 11 Xfce panel Network Manager applet has disappeared

[David Christensen]

On 4/17/24 12:07, Charles Curley wrote:

On Wed, 17 Apr 2024 11:41:24 -0700
David Christensen  wrote:


My WAG is that nm-applet is failing to start, but I have been unable
to find if and where any error message is reported.


My instance of nm-applet does run, and I see this as part of the boot
process:

root@hawk:~# journalctl -b | grep nm-app
Apr 15 11:27:42 hawk NetworkManager[1354]:   [1713202062.7737] 
agent-manager: agent[108f011a1115d508,:1.131/org.freedesktop.nm-applet/1000]: agent 
registered
root@hawk:~#

I suspect that if nm-applet doesn't start, you won't see any output
from that command.



2024-04-18 03:07:20 root@laalaa ~
# journalctl -b | grep nm-app
Apr 18 03:01:39 laalaa NetworkManager[833]:   [1713434499.3660] 
agent-manager: 
agent[2a08cda7fa35849b,:1.48/org.freedesktop.nm-applet/13250]: agent 
registered



David

Re: Fwd: Debian 11 Xfce panel Network Manager applet has disappeared

[David Christensen]

On 4/17/24 19:41, Gareth Evans wrote:

On Wed 17/04/2024 at 19:41, David Christensen  wrote:

On 4/17/24 03:47, Gareth Evans wrote:

On Wed 17/04/2024 at 09:18, David Christensen  wrote:

On 4/16/24 08:56, Gareth Evans wrote:

On 16 Apr 2024, at 00:18, David Christensen  wrote:

On 4/15/24 09:21, Gareth Evans wrote:

On Sun 14/04/2024 at 13:29, David Christensen wrote:
...
I have used the Xfce panel Network Manager applet for many years.
Tonight, I noticed that it has disappeared (!).
...


There is apparently a long history of nm-applet/XFCE panel-related issues (and 
not many great answers), as evidenced by such as

https://bbs.archlinux.org/viewtopic.php?id=161998

https://forum.xfce.org/viewtopic.php?id=6105

https://superuser.com/questions/900490/networkmanager-icon-on-notification-area-is-not-present



I tried:

1.  Remove Notification applet, restart, add Notification applet, no 
Network Manager, restart -- no Network Manager.


2.  Look at /etc/NetworkManager/NetworkManager.conf

2024-04-18 02:40:49 dpchrist@laalaa ~
$ cat /etc/NetworkManager/NetworkManager.conf
[main]
plugins=ifupdown,keyfile

[ifupdown]
managed=false

Change it to:

2024-04-18 02:43:35 root@laalaa ~
# cat /etc/NetworkManager/NetworkManager.conf
[main]
plugins=ifupdown,keyfile

[ifupdown]
managed=true

Restart -- no Network Manager.

Verify /etc//NetworkManager/NetworkManager.conf:

2024-04-18 02:45:05 dpchrist@laalaa ~
$ cat /etc/NetworkManager/NetworkManager.conf
[main]
plugins=ifupdown,keyfile

[ifupdown]
managed=true



If you get ps output like this directly after a reboot:


$ ps aux | grep nm-applet
dpchrist1518  0.1  0.2 426500 35380 ?Sl   16:06   0:00 nm-applet



2024-04-18 02:45:10 dpchrist@laalaa ~
$ ps aux | grep nm-applet
dpchrist1526  0.1  0.2 426484 35256 ?Sl   02:44   0:00 nm-applet
dpchrist1869  0.0  0.0   3240   712 pts/0S+   02:47   0:00 grep 
nm-applet




then I don't think the issue is with the starting/running of nm-applet itself, 
but rather some issue with the notification plugin, which I'm not sure how to 
begin troubleshooting.  I reluctantly abandoned XFCE partly due to panel 
instability some time ago.

The advice in the final link to rm -rf  ~/.config/xfce* might be a bit extreme, 
but I might try renaming it to force a rebuild on next login, and possibly 
reinstall task-xfce-desktop (or selected packages)



Move aside the ~/.config/xfce4 directory:


2024-04-18 02:50:20 dpchrist@laalaa ~
$ ls -ld .config/xfce4/
drwxr-xr-x 8 dpchrist dpchrist 4096 Apr 18 00:45 .config/xfce4/

2024-04-18 02:50:23 dpchrist@laalaa ~
$ mv .config/xfce4/ .config/xfce4-20240418-180045


Restart -- screen with wallpaper alone.


Right-click on desktop -> Applications -> Settings -> Panel brings up 
the Panel Preferences application.  The Items tab shows my previous 
settings.  Fumbling around, I am unable to get that panel to display 
(?).  If I create a new panel, I am unable to get it to display (?).



Open a Terminal and look if Xfce created a new configuration directory:

2024-04-18 02:57:31 dpchrist@laalaa ~
$ ls -ld .config/xfce4*
drwxr-xr-x 7 dpchrist dpchrist 4096 Apr 18 02:57 .config/xfce4
drwxr-xr-x 8 dpchrist dpchrist 4096 Apr 18 00:45 
.config/xfce4-20240418-180045



Yes, it did.  Compare the old directory to the new directory:

2024-04-18 02:58:11 dpchrist@laalaa ~
$ diff -rq .config/xfce4-20240418-180045/ .config/xfce4
diff: .config/xfce4-20240418-180045/desktop/icons.screen.latest.rc: No 
such file or directory

Only in .config/xfce4-20240418-180045/desktop: icons.screen0-1008x725.rc
Only in .config/xfce4-20240418-180045/desktop: icons.screen0-1424x857.rc
Only in .config/xfce4-20240418-180045/desktop: icons.screen0-1664x1007.rc
Only in .config/xfce4-20240418-180045/desktop: icons.screen0-1904x1037.rc
Only in .config/xfce4-20240418-180045/desktop: icons.screen0-1904x1064.rc
Only in .config/xfce4-20240418-180045/desktop: icons.screen0-2928x1037.rc
Files .config/xfce4-20240418-180045/desktop/icons.screen0-3824x1037.rc 
and .config/xfce4/desktop/icons.screen0-3824x1037.rc differ

Only in .config/xfce4-20240418-180045/: help.rc
Only in .config/xfce4-20240418-180045/panel: launcher-15
Only in .config/xfce4/panel: launcher-17
Only in .config/xfce4/panel: launcher-18
Only in .config/xfce4/panel: launcher-19
Only in .config/xfce4/panel: launcher-20
Only in .config/xfce4-20240418-180045/panel: launcher-24
Only in .config/xfce4-20240418-180045/panel: launcher-28
Only in .config/xfce4-20240418-180045/panel: launcher-7
Only in .config/xfce4-20240418-180045/: src
Files .config/xfce4-20240418-180045/terminal/accels.scm and 
.config/xfce4/terminal/accels.scm differ

Only in .config/xfce4-20240418-180045/terminal: terminalrc
Only in .config/xfce4-20240418-180045/: xfce4-screenshooter
Only in .config/xfce4-2024

Re: Fwd: Debian 11 Xfce panel Network Manager applet has disappeared

[David Christensen]

On 4/17/24 13:56, e...@gmx.us wrote:

On 4/17/24 15:37, Richmond wrote:

David Christensen writes:

My WAG is that nm-applet is failing to start, but I have been unable to
find if and where any error message is reported.


What are the permissions on the nm-applet binary?


And is its filesystem mounted with noexec?



2024-04-18 02:27:18 root@laalaa ~
# df `which nm-applet`
Filesystem 1M-blocks  Used Available Use% Mounted on
/dev/mapper/sdb3_crypt12084M 8927M 2522M  78% /

2024-04-18 02:28:57 root@laalaa ~
# mount | grep sdb3
/dev/mapper/sdb3_crypt on / type ext4 (rw,relatime,errors=remount-ro)


David

Re: Fwd: Debian 11 Xfce panel Network Manager applet has disappeared

[David Christensen]

On 4/17/24 12:37, Richmond wrote:

David Christensen  writes:

On Sun 14/04/2024 at 13:29, David Christensen wrote:
...
I have used the Xfce panel Network Manager applet for many years.
Tonight, I noticed that it has disappeared (!).
...

What are the permissions on the nm-applet binary? maybe it doesn't have
permission to execute, or the process which starts it doesn't have
permission.



2024-04-18 02:24:20 root@laalaa ~
# ls -l `which nm-applet`
-rwxr-xr-x 1 root root 250784 Feb 27  2021 /usr/bin/nm-applet


I do not know what binary starts nm-applet, but here is a WAG:

2024-04-18 02:27:13 root@laalaa ~
# ll -l `which xfce4-panel`
-rwxr-xr-x 1 root root 363328 2021-02-27 08:29:44 /usr/bin/xfce4-panel


David

Fwd: Debian 11 Xfce panel Network Manager applet has disappeared

[David Christensen]

 Forwarded Message 
Subject: Re: Debian 11 Xfce panel Network Manager applet has disappeared
Date: Wed, 17 Apr 2024 01:18:34 -0700
From: David Christensen 
To: Gareth Evans 

On 4/16/24 08:56, Gareth Evans wrote:

On 16 Apr 2024, at 00:18, David Christensen  wrote:

On 4/15/24 09:21, Gareth Evans wrote:

On Sun 14/04/2024 at 13:29, David Christensen wrote:
...
I have used the Xfce panel Network Manager applet for many years.
Tonight, I noticed that it has disappeared (!).
...

2024-04-15 16:08:23 dpchrist@laalaa ~
$ ps aux | grep nm-applet
dpchrist1518  0.1  0.2 426500 35380 ?Sl   16:06   0:00 nm-applet
dpchrist1940  0.0  0.0   3240   644 pts/0S+   16:15   0:00 grep 
nm-applet

2024-04-15 16:15:12 dpchrist@laalaa ~
$ nm-applet

2024-04-15 16:15:31 dpchrist@laalaa ~
$ ps aux | grep nm-applet
dpchrist1518  0.1  0.2 426500 35380 ?Sl   16:06   0:00 nm-applet
dpchrist1952  0.0  0.0   3240   644 pts/0S+   16:15   0:00 grep 
nm-applet


That seems to show it's running from the outset, just not being displayed on 
the panel.

Does rebooting (or logging out and in again) bring it back?



No.


David

Fwd: Debian 11 Xfce panel Network Manager applet has disappeared

[David Christensen]

 Forwarded Message 
Subject: Re: Debian 11 Xfce panel Network Manager applet has disappeared
Date: Wed, 17 Apr 2024 11:38:49 -0700
From: David Christensen 
To: Gareth Evans 

On 4/17/24 03:47, Gareth Evans wrote:

On Wed 17/04/2024 at 09:18, David Christensen  wrote:

On 4/16/24 08:56, Gareth Evans wrote:

On 16 Apr 2024, at 00:18, David Christensen  wrote:

On 4/15/24 09:21, Gareth Evans wrote:

On Sun 14/04/2024 at 13:29, David Christensen wrote:
...
I have used the Xfce panel Network Manager applet for many years.
Tonight, I noticed that it has disappeared (!).
...

2024-04-15 16:08:23 dpchrist@laalaa ~
$ ps aux | grep nm-applet
dpchrist1518  0.1  0.2 426500 35380 ?Sl   16:06   0:00 nm-applet
dpchrist1940  0.0  0.0   3240   644 pts/0S+   16:15   0:00 grep 
nm-applet

2024-04-15 16:15:12 dpchrist@laalaa ~
$ nm-applet

2024-04-15 16:15:31 dpchrist@laalaa ~
$ ps aux | grep nm-applet
dpchrist1518  0.1  0.2 426500 35380 ?Sl   16:06   0:00 nm-applet
dpchrist1952  0.0  0.0   3240   644 pts/0S+   16:15   0:00 grep 
nm-applet


That seems to show it's running from the outset, just not being displayed on 
the panel.

Does rebooting (or logging out and in again) bring it back?



No.


OK.  You may have checked this already, but in case not, if I install XFCE and 
go to

Settings > Session and Startup > Application Autostart

there is an entry in the list called

"Network (Manage your network connections)"



It is checked.



which shows a tooltip of "command: nm-applet"



Command: nm-applet



Might this somehow have become unset?

I'm not sure if it's possible for GUI config helpers to become detached from 
actual settings - this seems to describe the relevant locations:

https://unix.stackexchange.com/questions/669372/xfce4-session-and-startup-where-are-autostart-items-saved



2024-04-17 11:21:06 dpchrist@laalaa ~
$ grep nm-applet ~/.config/autostart
grep: /home/dpchrist/.config/autostart: No such file or directory

2024-04-17 11:33:11 dpchrist@laalaa ~
$ find .config -name autostart

2024-04-17 11:33:22 dpchrist@laalaa ~
$


2024-04-17 11:34:14 dpchrist@laalaa ~
$ grep -r nm-applet /etc/xdg/autostart
/etc/xdg/autostart/nm-applet.desktop:Exec=nm-applet
/etc/xdg/autostart/nm-applet.desktop:X-GNOME-Bugzilla-Component=nm-applet


My WAG is that nm-applet is failing to start, but I have been unable to 
find if and where any error message is reported.



David

Re: Debian 11 Xfce panel Network Manager applet has disappeared

[David Christensen]

On 4/15/24 09:21, Gareth Evans wrote:

On Sun 14/04/2024 at 13:29, David Christensen wrote:

...
I have used the Xfce panel Network Manager applet for many years.
Tonight, I noticed that it has disappeared (!).
...

Hi David,

I can't speak for XFCE, but certainly for Mate there was a time when multiple 
notification area panel widgets were available, not all of which would show 
everything to be expected.  Is that a possibility?

Does

$ ps aux |grep nm-applet

show anything?  Before or after

$ nm-applet

?



2024-04-15 16:08:23 dpchrist@laalaa ~
$ ps aux | grep nm-applet
dpchrist1518  0.1  0.2 426500 35380 ?Sl   16:06   0:00 nm-applet
dpchrist1940  0.0  0.0   3240   644 pts/0S+   16:15   0:00 grep 
nm-applet


2024-04-15 16:15:12 dpchrist@laalaa ~
$ nm-applet

2024-04-15 16:15:31 dpchrist@laalaa ~
$ ps aux | grep nm-applet
dpchrist1518  0.1  0.2 426500 35380 ?Sl   16:06   0:00 nm-applet
dpchrist1952  0.0  0.0   3240   644 pts/0S+   16:15   0:00 grep 
nm-applet



David

Debian 11 Xfce panel Network Manager applet has disappeared

[David Christensen]

debian-user:

I have a Dell Latitude E6520:

2024-04-14 04:28:39 dpchrist@laalaa ~
$ cat /etc/debian_version ; uname -a
11.9
Linux laalaa 5.10.0-28-amd64 #1 SMP Debian 5.10.209-2 (2024-01-31) 
x86_64 GNU/Linux


2024-04-14 04:34:40 dpchrist@laalaa ~
$ dpkg-query -l xfce4 network-manager network-manager-gnome
Desired=Unknown/Install/Remove/Purge/Hold
| 
Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend

|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
||/ Name  Version  Architecture Description
+++-=---=
ii  network-manager   1.30.6-1+deb11u1 amd64network 
management framework (daemon and userspace tools)
ii  network-manager-gnome 1.20.0-3 amd64network 
management framework (GNOME frontend)
ii  xfce4 4.16 all  Meta-package for 
the Xfce Lightweight Desktop Environment



I have used the Xfce panel Network Manager applet for many years. 
Tonight, I noticed that it has disappeared (!).



But, the machine is connected to my LAN:

2024-04-14 05:24:10 root@laalaa ~
# ifconfig wlp3s0
wlp3s0: flags=4163  mtu 1500
inet 192.168.REDACTED  netmask 255.255.255.0  broadcast 
192.168.REDACTED

inet6 REDACTED  prefixlen 64  scopeid 0x20
ether REDACTED  txqueuelen 1000  (Ethernet)
RX packets 5786  bytes 2830592 (2.6 MiB)
RX errors 0  dropped 119  overruns 0  frame 0
TX packets 3897  bytes 518278 (506.1 KiB)
TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0


Looking in the the Xfce panel Application Menu, I am unable to find 
Network Manager.



Looking at the Debian WIKI page "NetworkManager":

https://wiki.debian.org/NetworkManager


It looks like the Network Manager daemon is running:

2024-04-14 04:32:49 dpchrist@laalaa ~
$ ps -A | grep -i network
828 ?00:00:00 NetworkManager


nm-applet(1) looks like the program I want (?).  Attempting to start it 
via a terminal has no effect:


2024-04-14 04:40:25 dpchrist@laalaa ~
$ which nm-applet
/usr/bin/nm-applet

2024-04-14 05:27:05 dpchrist@laalaa ~
$ nm-applet

2024-04-14 05:27:08 dpchrist@laalaa ~
$


RTFM nm-applet(1), it seems the desktop session manager is failing to 
start nm-applet(1) (?):


2024-04-14 04:58:49 dpchrist@laalaa ~
$ man nm-applet | cat
...
DESCRIPTION
   nm-applet  is  a GTK-based GUI applet to monitor network status
   and devices and to start and stop network  connections  managed
   by  NetworkManager.   nm-applet is normally started at login by
   the desktop session manager and does not need to be  run  manu-
   ally.   nm-applet conforms to the XDG System Tray specification
   and requires that the desktop environment provide a System Tray
   implementation in which the applet will be embedded.


I am unable to find relevant error messages under /var/log.


The network Connection Editor can be run via a terminal:

2024-04-14 04:55:29 dpchrist@laalaa ~
$ which nm-connection-editor
/usr/bin/nm-connection-editor

2024-04-14 04:55:38 dpchrist@laalaa ~
$ nm-connection-editor


Does anyone know why the Network Manager Xfce panel applet is missing, 
how to get it back, and/or how to start it some other way?



David

Re: HDD long-term data storage with ensured integrity

[David Christensen]

On 4/12/24 08:14, piorunz wrote:

On 10/04/2024 12:10, David Christensen wrote:

Those sound like some compelling features.


I believe the last time I tried Btrfs was Debian 9 (?).  I ran into
problems because I did not do the required manual maintenance
(rebalancing).  Does the Btrfs in Debian 11 or Debian 12 still require
manual maintenance?  If so, what and how often?


I don't do balance at all, it's not required.

Scrub is recommended, because it will detect any bit-rot due to hardware
errors on HDD media. It scans the entire surface of allocated sectors on
all drives. I do scrub usually monthly.



Thank you for the information.


David

Re: HDD long-term data storage with ensured integrity

[David Christensen]

On 4/10/24 08:49, Paul Leiber wrote:

Am 10.04.2024 um 13:10 schrieb David Christensen:
Does the Btrfs in Debian 11 or Debian 12 still require 
manual maintenance?  If so, what and how often?


Scrub and balance are actions which have been recommended. I am using 
btrfsmaintenance scripts [1][2] to automate this. I am doing a weekly 
balance and a monthly scrub. After some reading today, I am getting 
unsure if this is approach is correct, especially if balance is 
necessary anymore (it usually doesn't find anything to do anyway), so 
please take these periods with caution. My main message is that such 
operations can be automated using the linked scripts.


Best regards,

Paul

[1] https://packages.debian.org/bookworm/btrfsmaintenance
[2] https://github.com/kdave/btrfsmaintenance



Thank you.  Those scripts should be useful.


David

Re: HDD long-term data storage with ensured integrity

[David Christensen]

On 4/9/24 17:08, piorunz wrote:

On 02/04/2024 13:53, David Christensen wrote:


Does anyone have any comments or suggestions regarding how to use
magnetic hard disk drives, commodity x86 computers, and Debian for
long-term data storage with ensured integrity?


I use Btrfs, on all my systems, including some servers, with soft Raid1
and Raid10 modes (because these modes are considered stable and
production ready). I decided on Btrfs not ZFS, because Btrfs allows to
migrate drives on the fly while partition is live and heavily used,
replace them with different sizes and types, mixed capacities, change
Raid levels, change amount of drives too. I could go from single drive
to Raid10 on 4 drives and back while my data is 100% available at all 
times.

It saved my bacon many times, including hard checksum corruption on NVMe
drive which otherwise I would never know about. Thanks to Btrfs I
located the corrupted files, fixed them, got hardware replaced under
warranty.
Also helped with corrupted RAM: Btrfs just refused to save file because
saved copy couldn't match read checksum from the source due to RAM bit
flips. Diagnosed, then replaced memory, all good.
I like a lot when one of the drives get ATA reset for whatever reason,
and all other drives continue to read and write, I can continue using
the system for hours, if I even notice. Not possible in normal
circumstances without Raid. Once the problematic drive is back, or after
reboot if it's more serious, then I do "scrub" command and everything is
resynced again. If I don't do that, then Btrfs dynamically correct
checksum errors on the fly anyway.
And list goes on - I've been using Btrfs for last 5 years, not a single
problem to date, it survived hard resets, power losses, drive failures,
countless migrations.



Those sound like some compelling features.


I believe the last time I tried Btrfs was Debian 9 (?).  I ran into 
problems because I did not do the required manual maintenance 
(rebalancing).  Does the Btrfs in Debian 11 or Debian 12 still require 
manual maintenance?  If so, what and how often?




[1] https://github.com/openzfs/zfs/issues/15526

[2] https://github.com/openzfs/zfs/issues/15933


Problems reported here are from Linux kernel 6.5 and 6.7 on Gentoo
system. Does this even affects Debian Stable with 6.1 LTS?



I do not know.



--
With kindest regards, Piotr.

⢀⣴⠾⠻⢶⣦⠀
⣾⠁⢠⠒⠀⣿⡁ Debian - The universal operating system
⢿⡄⠘⠷⠚⠋⠀ https://www.debian.org/
⠈⠳⣄



David

Re: Why LVM

[David Christensen]

On 4/8/24 16:54, Stefan Monnier wrote:

If I have a hot-pluggable device (SD card, USB drive, hot-plug SATA/SAS
drive and rack, etc.), can I put LVM on it such that when the device is
connected to a Debian system with a graphical desktop (I use Xfce) an icon
is displayed on the desktop that I can interact with to display the file
systems in my file manager (Thunar)?


In the past: definitely not.  Currently: no idea.
I suspect not, because I think the behavior on disconnection is still
poor (you want to be extra careful to deactivate all the volumes on the
drive *before* removing it, otherwise they tend to linger "for ever").

I guess that's one area where partitions are still significantly better
than LVM.


 Stefan "who doesn't use much hot-plugging of mass storage"



Thank you for the clarification.  :-)


David

Re: Why LVM

[David Christensen]

On 4/8/24 14:08, Stefan Monnier wrote:

David Christensen [2024-04-08 11:28:04] wrote:

Why LVM?


Personally, I've been using LVM everywhere I can (i.e. everywhere
except on my OpenWRT router, tho I've also used LVM there back when my
router had an HDD.  I also use LVM on my 2GB USB rescue image).

To me the question is rather the reverse: why not?
I basically see it as a more flexible form of partitioning.

Even in the worst cases where I have a single LV volume, I appreciate
the fact that it forces me to name things, isolating me from issue
linked to predicting the name of the device and the issues that plague
UUIDs (the fact they're hard to remember, and that they're a bit too
magical/hidden for my taste, so they sometimes change when I don't want
them to and vice versa).


 Stefan



If I have a hot-pluggable device (SD card, USB drive, hot-plug SATA/SAS 
drive and rack, etc.), can I put LVM on it such that when the device is 
connected to a Debian system with a graphical desktop (I use Xfce) an 
icon is displayed on the desktop that I can interact with to display the 
file systems in my file manager (Thunar)?



David

Re: HDD long-term data storage with ensured integrity

[David Christensen]

On 4/8/24 13:04, Marc SCHAEFER wrote:

Hello,

On Mon, Apr 08, 2024 at 11:28:04AM -0700, David Christensen wrote:

So, an ext4 file system on an LVM logical volume?

Why LVM?  Are you implementing redundancy (RAID)?  Is your data larger than
a single disk (concatenation/ JBOD)?  Something else?


For off-site long-term offline archiving, no, I am not using RAID.

No, it's not LVM+md, just plain LVM for flexibility.

Typically I use 16 TB hard drives, and I tend to use one LV per data
source, the LV name being the data source and the date of the copy.
Or sometimes I just copy a raw volume (ext4 or something else)
to a LV.

With smaller drives (4 TB) I tend to not use LVM, just plain ext4 on the
raw disk.

I almost never use partitionning.

However, I tend to use luks encryption (per ext4 filesystem) when the
drives are stored off-site.  So it's either LVM -> LV -> LUKS -> ext4
or raw disk -> LUKS -> ext4.

You can find some of the scripts I use to automate this off-site
long-term archiving here:

https://git.alphanet.ch/gitweb/?p=various;a=tree;f=offsite-archival/LVM-LUKS



Thank you for the clarification.  :-)


David

Re: HDD long-term data storage with ensured integrity

[David Christensen]

On 4/8/24 02:38, Marc SCHAEFER wrote:

For offline storage:

On Tue, Apr 02, 2024 at 05:53:15AM -0700, David Christensen wrote:

Does anyone have any comments or suggestions regarding how to use magnetic
hard disk drives, commodity x86 computers, and Debian for long-term data
storage with ensured integrity?


I use LVM on ext4, and I add a MD5SUMS file at the root.

I then power up the drives at least once a year and check the MD5SUMS.

A simple CRC could also work, obviously.

So far, I have not detected MORE corruption with this method than the
drive ECC itself (current drives & buses are much better than they
used to be).  When I have errors detected, I replace the file with
another copy (I usually have multiple off-site copies, and sometimes
even on-site online copies, but not always).  When the errors add
up, it is time to buy another drive, usually after 5+ years or
even sometimes 10+ years.

So, just re-reading the content might be enough, once a year or so.

This is for HDD (for SDD I have no offline storage experience, it
could be shorter).



Thank you for the reply.


So, an ext4 file system on an LVM logical volume?


Why LVM?  Are you implementing redundancy (RAID)?  Is your data larger 
than a single disk (concatenation/ JBOD)?  Something else?



David

Re: readonly installer, (SOLVED)

[David Christensen]

On 4/3/24 19:05, Stefan Monnier wrote:

I have a 128 MB USB flash drive from back in the day that includes a write
protect switch.  There are few products today that offer that feature.


Side note: AFAIK this "write protect switch" doesn't prevent writing.
It just tells your card reader that you'd like to avoid writing to it.
Whether it ends up doing what you want depends on the hardware exposing
that info to the driver and the driver paying attention to it.


 Stefan



No, it is a USB flash drive with a write protect switch.  Here is a 
modern example:


https://www.amazon.com/dp/B00JJIE95G


David

Re: Debian ISOs on USB stick

[David Christensen]

On 4/3/24 05:56, Thomas Schmitt wrote:

Hi,

i read from bytes 2085412 to 2085479:
   "Info rrmation Syste rm VolumeSYSTEM~"
which is similar to the alterations of one of the USB sticks shown in
   https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1056998#35

The web knows about a Microsoft folder named "System Volume Information".
   
https://answers.microsoft.com/en-us/windows/forum/all/system-volume-information-what-is-it-and-what-is/3bc81844-0baa-46bd-9949-4efb4678b677
   "whenever I put my flash-drive or my micro sd adapter and sd card into
my windows 8.1 something called "System Volume Information" is always
getting added on."

So did you perhaps show this USB stick to a running MS-Windows system ?


Have a nice day :)

Thomas



It is possible the drive was inserted into a Windows computer.


If and when I need a newer d-i, perhaps I will put the ISO onto a USB 
flash drive, conduct more experiments, and post the results.



I apologize for blaming d-i for what might be Dell, Intel, BIOS/UEFI, 
Microsoft, and/or other bugs.



David

Re: readonly installer, (SOLVED)

[David Christensen]

On 4/3/24 08:16, David Wright wrote:

On Tue 02 Apr 2024 at 05:54:06 (-0700), David Christensen wrote:

On 4/1/24 11:35, DdB wrote:

Am 01.04.2024 um 18:52 schrieb David Christensen:

A bad USB flash drive would explain why you cannot boot the Debian
installer.  Please buy a good quality USB 3.0+ flash drive and try again.


A friend of mine just let me use an external CD-Drive with the netboot
image.


I thought about suggesting that in my last post, but did not want to
complicate things.  A key advantage of using a CD-R disc is that you
can verify the disc contents and/or checksum against the ISO and/or
checksum now and in the future.  This is not true for a USB flash
drive, because the Debian installer modifies the contents of the USB
flash drive when it runs.


If this troubles you, you can also use an SD card with a write-lock,
or a µSD card with a lock on the SD adaptor.

Check that the write-lock works with the logs when you plug it in,
or run fdisk/gdisk and immediately quit.

Cheers,
David.



I have a 128 MB USB flash drive from back in the day that includes a 
write protect switch.  There are few products today that offer that feature.



David

Re: HDD long-term data storage with ensured integrity

[David Christensen]

On 4/2/24 14:57, David Christensen wrote:
AIUI neither LVM nor ext4 have data and metadata checksum and correction 
features.  But, it should be possible to achieve such by including 
dm-integrity (for checksumming) and some form of RAID (for correction) 
in the storage stack.  I need to explore that possibility further.



I have RTFM dm-integrity before and it is still experimental.  I need 
something that is production ready:


https://manpages.debian.org/bookworm/cryptsetup-bin/cryptsetup.8.en.html

Authenticated disk encryption (EXPERIMENTAL)


David

Re: Debian ISOs on USB stick, was: SOLVED

[David Christensen]

On 4/3/24 03:36, David Christensen wrote:

On 4/3/24 00:30, Thomas Schmitt wrote:

Hi,

David Christensen wrote:
It's a relatively simple experiment to confirm that a USB flash drive 
with

d-i changes after the first boot.


This could still be
   https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1056998
where Lenovo BIOS and/or MS-Windows altered the USB stick.



Same for finding which bytes change.


I fail to find this particular info in
   Date: Tue, 2 Apr 2024 14:46:42 -0700
   From: David Christensen 
   Message-ID: 

If we have the exact ISO name (i.e. URL from where it stems) and the
byte address of the alteration, xorriso can find the affected file, if
any.

In case of bug #1056998 it was the EFI partition image 
/boot/grub/efi.img.

Mounting the altered and unaltered image files showed changes in the
FAT filesystem which point to the culprits Lenovo and Microsoft.


The other plausible way of altering the ISO image on the stick would be
adding a new partition.
The MBR partition table is part of the Debian ISO and thus part of the
checksummed area. Even if all other alterations happen after the end
of the checksummed ISO image, the changed partition table will cause the
Debian checksum to become invalid.

(I am not aware that Debian installer changes the table. If it does 
indeed

then this might be worth a new bug discussion.)


Have a nice day :)

Thomas



2024-04-03 03:29:18 root@laalaa /samba/dpchrist/iso/debian/11.3.0
# cmp --verbose debian-11.3.0-amd64-netinst.iso /dev/sdb
   2083201   0 377
   2083202   0 377
   2083203   0 377
   2085249   0 377
   2085250   0 377
   2085251   0 377
   2085409   0 102
   2085410   0  40
   2085412   0 111
   2085414   0 156
   2085416   0 146
   2085418   0 157
   2085420   0  17
   2085422   0 162
   2085423   0 162
   2085425   0 155
   2085427   0 141
   2085429   0 164
   2085431   0 151
   2085433   0 157
   2085437   0 156
   2085441   0   1
   2085442   0 123
   2085444   0 171
   2085446   0 163
   2085448   0 164
   2085450   0 145
   2085452   0  17
   2085454   0 162
   2085455   0 155
   2085457   0  40
   2085459   0 126
   2085461   0 157
   2085463   0 154
   2085465   0 165
   2085469   0 155
   2085471   0 145
   2085473   0 123
   2085474   0 131
   2085475   0 123
   2085476   0 124
   2085477   0 105
   2085478   0 115
   2085479   0 176
   2085480   0  61
   2085481   0  40
   2085482   0  40
   2085483   0  40
   2085484   0  26
   2085486   0 167
   2085487   0 174
   2085488   0 277
   2085489   0 235
   2085490   0 124
   2085491   0 235
   2085492   0 124
   2085495   0 175
   2085496   0 277
   2085497   0 235
   2085498   0 124
   2085500   0   5
   4719105   0  56
   4719106   0  40
   4719107   0  40
   4719108   0  40
   4719109   0  40
   4719110   0  40
   4719111   0  40
   4719112   0  40
   4719113   0  40
   4719114   0  40
   4719115   0  40
   4719116   0  20
   4719118   0 167
   4719119   0 174
   4719120   0 277
   4719121   0 235
   4719122   0 124
   4719123   0 235
   4719124   0 124
   4719127   0 175
   4719128   0 277
   4719129   0 235
   4719130   0 124
   4719132   0   5
   4719137   0  56
   4719138   0  56
   4719139   0  40
   4719140   0  40
   4719141   0  40
   4719142   0  40
   4719143   0  40
   4719144   0  40
   4719145   0  40
   4719146   0  40
   4719147   0  40
   4719148   0  20
   4719150   0 167
   4719151   0 174
   4719152   0 277
   4719153   0 235
   4719154   0 124
   4719155   0 235
   4719156   0 124
   4719159   0 175
   4719160   0 277
   4719161   0 235
   4719162   0 124
   4719169   0 102
   4719170   0 107
   4719172   0 165
   4719174   0 151
   4719176   0 144
   4719180   0  17
   4719182   0 377
   4719183   0 377
   4719184   0 377
   4719185   0 377
   4719186   0 377
   4719187   0 377
   4719188   0 377
   4719189   0 377
   4719190   0 377
   4719191   0 377
   4719192   0 377
   4719193   0 377
   4719194   0 377
   4719197   0 377
   4719198   0 377
   4719199   0 377
   4719200   0 377
   4719201   0   1
   4719202   0 111
   4719204   0 156
   4719206   0 144
   4719208   0 145
   4719210   0 170
   4719212   0  17
   4719214   0 377
   4719215   0 145
   4719217   0 162
   4719219   0 126
   4719221   0 157
   4719223   0 154
   4719225   0 165
   4719229   0 155
   4719231   0 145
   4719233   0 111
   4719234   0 116
   4719235   0 104
   4719236   0 105
   4719237   0 130
   4719238   0 105
   4719239   0 176
   4719240   0  61
   4719241   0  40
   4719242   0  40
   4719243   0  40
   4719244   0  40
   4719246   0 171
   4719247   0 174
   4719248   0 277
   4719249   0 235
   4719250   0 124
   4719251   0 235
   4719252   0 124
   4719255   0 175
   4719256   0 277
   4719257   0 235
   4719258   0 124
   4719259   0   1
   4719260   0   5
   4719261   0 114
   4721153   0 173
   4721155   0  71
   4721157   0 101
   4721159   0  65
   4721161   0 104
   4721163   0 101
   4721165   0 106
   4721167   0  65
   4721169   0  67
   4721171   

Re: Debian ISOs on USB stick, was: SOLVED

[David Christensen]

On 4/3/24 00:30, Thomas Schmitt wrote:

Hi,

David Christensen wrote:

It's a relatively simple experiment to confirm that a USB flash drive with
d-i changes after the first boot.


This could still be
   https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1056998
where Lenovo BIOS and/or MS-Windows altered the USB stick.



Same for finding which bytes change.


I fail to find this particular info in
   Date: Tue, 2 Apr 2024 14:46:42 -0700
   From: David Christensen 
   Message-ID: 

If we have the exact ISO name (i.e. URL from where it stems) and the
byte address of the alteration, xorriso can find the affected file, if
any.

In case of bug #1056998 it was the EFI partition image /boot/grub/efi.img.
Mounting the altered and unaltered image files showed changes in the
FAT filesystem which point to the culprits Lenovo and Microsoft.


The other plausible way of altering the ISO image on the stick would be
adding a new partition.
The MBR partition table is part of the Debian ISO and thus part of the
checksummed area. Even if all other alterations happen after the end
of the checksummed ISO image, the changed partition table will cause the
Debian checksum to become invalid.

(I am not aware that Debian installer changes the table. If it does indeed
then this might be worth a new bug discussion.)


Have a nice day :)

Thomas



2024-04-03 03:29:18 root@laalaa /samba/dpchrist/iso/debian/11.3.0
# cmp --verbose debian-11.3.0-amd64-netinst.iso /dev/sdb
  2083201   0 377
  2083202   0 377
  2083203   0 377
  2085249   0 377
  2085250   0 377
  2085251   0 377
  2085409   0 102
  2085410   0  40
  2085412   0 111
  2085414   0 156
  2085416   0 146
  2085418   0 157
  2085420   0  17
  2085422   0 162
  2085423   0 162
  2085425   0 155
  2085427   0 141
  2085429   0 164
  2085431   0 151
  2085433   0 157
  2085437   0 156
  2085441   0   1
  2085442   0 123
  2085444   0 171
  2085446   0 163
  2085448   0 164
  2085450   0 145
  2085452   0  17
  2085454   0 162
  2085455   0 155
  2085457   0  40
  2085459   0 126
  2085461   0 157
  2085463   0 154
  2085465   0 165
  2085469   0 155
  2085471   0 145
  2085473   0 123
  2085474   0 131
  2085475   0 123
  2085476   0 124
  2085477   0 105
  2085478   0 115
  2085479   0 176
  2085480   0  61
  2085481   0  40
  2085482   0  40
  2085483   0  40
  2085484   0  26
  2085486   0 167
  2085487   0 174
  2085488   0 277
  2085489   0 235
  2085490   0 124
  2085491   0 235
  2085492   0 124
  2085495   0 175
  2085496   0 277
  2085497   0 235
  2085498   0 124
  2085500   0   5
  4719105   0  56
  4719106   0  40
  4719107   0  40
  4719108   0  40
  4719109   0  40
  4719110   0  40
  4719111   0  40
  4719112   0  40
  4719113   0  40
  4719114   0  40
  4719115   0  40
  4719116   0  20
  4719118   0 167
  4719119   0 174
  4719120   0 277
  4719121   0 235
  4719122   0 124
  4719123   0 235
  4719124   0 124
  4719127   0 175
  4719128   0 277
  4719129   0 235
  4719130   0 124
  4719132   0   5
  4719137   0  56
  4719138   0  56
  4719139   0  40
  4719140   0  40
  4719141   0  40
  4719142   0  40
  4719143   0  40
  4719144   0  40
  4719145   0  40
  4719146   0  40
  4719147   0  40
  4719148   0  20
  4719150   0 167
  4719151   0 174
  4719152   0 277
  4719153   0 235
  4719154   0 124
  4719155   0 235
  4719156   0 124
  4719159   0 175
  4719160   0 277
  4719161   0 235
  4719162   0 124
  4719169   0 102
  4719170   0 107
  4719172   0 165
  4719174   0 151
  4719176   0 144
  4719180   0  17
  4719182   0 377
  4719183   0 377
  4719184   0 377
  4719185   0 377
  4719186   0 377
  4719187   0 377
  4719188   0 377
  4719189   0 377
  4719190   0 377
  4719191   0 377
  4719192   0 377
  4719193   0 377
  4719194   0 377
  4719197   0 377
  4719198   0 377
  4719199   0 377
  4719200   0 377
  4719201   0   1
  4719202   0 111
  4719204   0 156
  4719206   0 144
  4719208   0 145
  4719210   0 170
  4719212   0  17
  4719214   0 377
  4719215   0 145
  4719217   0 162
  4719219   0 126
  4719221   0 157
  4719223   0 154
  4719225   0 165
  4719229   0 155
  4719231   0 145
  4719233   0 111
  4719234   0 116
  4719235   0 104
  4719236   0 105
  4719237   0 130
  4719238   0 105
  4719239   0 176
  4719240   0  61
  4719241   0  40
  4719242   0  40
  4719243   0  40
  4719244   0  40
  4719246   0 171
  4719247   0 174
  4719248   0 277
  4719249   0 235
  4719250   0 124
  4719251   0 235
  4719252   0 124
  4719255   0 175
  4719256   0 277
  4719257   0 235
  4719258   0 124
  4719259   0   1
  4719260   0   5
  4719261   0 114
  4721153   0 173
  4721155   0  71
  4721157   0 101
  4721159   0  65
  4721161   0 104
  4721163   0 101
  4721165   0 106
  4721167   0  65
  4721169   0  67
  4721171   0  55
  4721173   0  71
  4721175   0 102
  4721177   0  63
  4721179   0  61
  4721181   0  55
  4721183   0  64
  4721185   0  71
  4721187   0  62
  4721189   0 105
  4721191   0  55
  4721193   0 102
  4721195   0  66
  4721197   

Re: Debian ISOs on USB stick, was: SOLVED

[David Christensen]

On 4/2/24 08:56, Thomas Schmitt wrote:

Hi,

David Christensen wrote:

the Debian installer modifies the contents of the USB flash drive when
it runs.


Do you mean inside the range of the ISO image or outside by creating a
new partition ?


songbird wrote:

if it is an iso image copied to the USB stick it should not
be modified if you haven't somehow told the installer to
install the system to that USB stick (somehow).


There are other parties which feel entitled to operate on the EFI System
Partition of a USB stick.
In
   https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1056998
we found that Lenovo Thinkpad firmware created directories for storing
an empty file named "/efi/Lenovo/BIOS/SelfHealing.fd" and that MS-Windows
created a 12-byte file named "/System Volume Information/WPSettings.dat"
when it had contact with the USB stick.



i guess if you wanted to be really sure you could mount it read-only.


I think it's the installer which mounts the ISO 9660 filesystem.
Whatever, the Linux kernel has no regular means to alter an ISO 9660
filesystem. Neither kernel nor Debain installer will be so daring to
operate with byte level commands on that filesystem.

But the FAT filesystem in file /boot/grub/efi.img of the ISO 9660
filesystem in debian-12.*-amd64-netinst.iso is advertised by the partition
table of the image and thus attracts vermin.


Have a nice day :)

Thomas



Please see my reply to songbird.


It's a relatively simple experiment to confirm that a USB flash drive 
with d-i changes after the first boot.  Same for finding which bytes 
change.  The challenge is figuring out what performed the change(s) and 
why.  I assumed it was d-i, but no longer own 64-bit BIOS-only computers 
to confirm.



David

Re: HDD long-term data storage with ensured integrity

[David Christensen]

On 4/2/24 06:55, Stefan Monnier wrote:

The most obvious alternative to ZFS on Debian would be Btrfs.  Does anyone
have any comments or suggestions regarding Btrfs and data corruption bugs,
concurrency, CMM level, PSP, etc.?


If you're worried about such things, I'd think "the most obvious
alternative" is LVM+ext4.  Both Btrfs and ZFS share the same underlying
problem: more features => more code => more bugs.


 Stefan



AIUI neither LVM nor ext4 have data and metadata checksum and correction 
features.  But, it should be possible to achieve such by including 
dm-integrity (for checksumming) and some form of RAID (for correction) 
in the storage stack.  I need to explore that possibility further.



David

Re: SOLVED

[David Christensen]

On 4/2/24 07:55, songbird wrote:

David Christensen wrote:

I thought about suggesting that in my last post, but did not want to
complicate things.  A key advantage of using a CD-R disc is that you can
verify the disc contents and/or checksum against the ISO and/or checksum
now and in the future.  This is not true for a USB flash drive, because
the Debian installer modifies the contents of the USB flash drive when
it runs.


   if it is an iso image copied to the USB stick it should not
be modified if you haven't somehow told the installer to
install the system to that USB stick (somehow).

   i guess if you wanted to be really sure you could mount it
read-only.


   songbird



I used to think that the d-i ran in memory when booted from read-write 
media, but discovered otherwise several years ago.



I previously downloaded debian-11.3.0-amd64-netinst.iso and SHA512SUMS:

2022-04-29 22:16:19 dpchrist@tinkywinky 
~/samba/dpchrist/iso/debian/11.3.0

$ ls -l
total 380414
-rwxr-xr-x 1 dpchrist dpchrist   494 Apr 28 21:04 SHA512SUMS
-rwxr-xr-x 1 dpchrist dpchrist   833 Apr 28 21:04 
SHA512SUMS.sign
-rwxr-xr-x 1 dpchrist dpchrist 396361728 Apr 28 21:05 
debian-11.3.0-amd64-netinst.iso



I verified the checksum of the ISO file:

2022-04-29 22:17:17 dpchrist@tinkywinky 
~/samba/dpchrist/iso/debian/11.3.0

$ grep debian-11.3.0-amd64-netinst.iso SHA512SUMS | sha512sum -c
debian-11.3.0-amd64-netinst.iso: OK


I burned the ISO to a zeroed USB flash drive:

2022-04-29 22:39:25 root@tinkywinky 
~/hardware/adata/usb-flash-drive/REDACTED
# time dd 
if=/home/dpchrist/samba/dpchrist/iso/debian/11.3.0/debian-11.3.0-amd64-netinst.iso 
of=/dev/disk/by-id/usb-ADATA_USB_Flash_Drive_1392303332110024-0\:0 bs=1M 
iflag=fullblock oflag=sync,noatime status=progress

394264576 bytes (394 MB, 376 MiB) copied, 76.1204 s, 5.2 MB/s
378+0 records in
378+0 records out
396361728 bytes (396 MB, 378 MiB) copied, 76.5701 s, 5.2 MB/s

real1m16.582s
user0m0.012s
sys 0m0.584s


I computed the checksum of the relevant blocks of the USB flash drive:

2022-04-29 22:43:56 root@tinkywinky 
~/hardware/adata/usb-flash-drive/REDACTED
# time dd 
if=/dev/disk/by-id/usb-ADATA_USB_Flash_Drive_REDACTED-0\:0 bs=1M 
count=378 iflag=fullblock | sha512sum

378+0 records in
378+0 records out
396361728 bytes (396 MB, 378 MiB) copied, 25.0641 s, 15.8 MB/s

2810f894afab9ac2631ddd097599761c1481b85e629d6a3197fe1488713af048d37241eb85def681ba86e62b406dd9b891ee1ae7915416335b6bb000d57c1e53 
 -


real0m25.068s
user0m3.468s
sys 0m0.720s


The USB flash drive checksum matched the value stored in SHA512SUMS:

2022-04-29 22:44:58 root@tinkywinky 
~/hardware/adata/usb-flash-drive/REDACTED
# grep 
2810f894afab9ac2631ddd097599761c1481b85e629d6a3197fe1488713af048d37241eb85def681ba86e62b406dd9b891ee1ae7915416335b6bb000d57c1e53 
/home/dpchrist/samba/dpchrist/iso/debian/11.3.0/SHA512SUMS


2810f894afab9ac2631ddd097599761c1481b85e629d6a3197fe1488713af048d37241eb85def681ba86e62b406dd9b891ee1ae7915416335b6bb000d57c1e53 
 debian-11.3.0-amd64-netinst.iso



I have since used the USB flash drive to install Debian onto one or more 
computers.



If I compute the checksum of the relevant blocks of the USB flash drive 
today:


2024-04-02 14:32:43 root@laalaa ~
# time dd if=/dev/disk/by-id/usb-ADATA_USB_Flash_Drive_REDACTED-0\:0 
bs=1M count=378 iflag=fullblock | sha512sum

378+0 records in
378+0 records out
6cbb7e54fccdf550cbb5535d7dd9357513b36b767f0aaa550b1d58b37ef827c881036dcf47b3c377d196c4e77d14c786a6dd975aa558a11a81cf9ae107062abc 
 -

396361728 bytes (396 MB, 378 MiB) copied, 27.7592 s, 14.3 MB/s

real0m27.766s
user0m4.134s
sys 0m1.380s


The checksum has changed because d-i modified the USB flash drive.


I previously confirmed this behavior years ago after booting a d-i USB 
flash drive only once.



You must use read-only media (e.g. CD-R) if you want to be able to 
verify d-i after use.



David

Re: SOLVED

[David Christensen]

On 4/1/24 11:35, DdB wrote:

Am 01.04.2024 um 18:52 schrieb David Christensen:

A bad USB flash drive would explain why you cannot boot the Debian
installer.  Please buy a good quality USB 3.0+ flash drive and try again.


A friend of mine just let me use an external CD-Drive with the netboot
image. 



I thought about suggesting that in my last post, but did not want to 
complicate things.  A key advantage of using a CD-R disc is that you can 
verify the disc contents and/or checksum against the ISO and/or checksum 
now and in the future.  This is not true for a USB flash drive, because 
the Debian installer modifies the contents of the USB flash drive when 
it runs.




This is already the third time, i am restarting the installation
process, due to my false assumptions about the intelligence within the
installer.

The last time, i was quite happy until i came to notice, that partitions
were not aligned with physical sector boundaries, which i assumed would
be elementary best practice.



I chose manual partitioning and the Debian installer aligned the 
partitions to 2**20 byte boundaries:


2024-04-02 04:07:16 root@laalaa ~
# cat /etc/debian_version ; uname -a
11.9
Linux laalaa 5.10.0-28-amd64 #1 SMP Debian 5.10.209-2 (2024-01-31) 
x86_64 GNU/Linux


2024-04-02 04:08:18 root@laalaa ~
# fdisk -l /dev/sda
Disk /dev/sda: 55.9 GiB, 60022480896 bytes, 117231408 sectors
Disk model: INTEL SSDSC2CW06
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0x544032f5

Device BootStart   End  Sectors  Size Id Type
/dev/sda1  *2048   1953791  1951744  953M 83 Linux
/dev/sda21953792   3907583  1953792  954M 83 Linux
/dev/sda33907584  29298687 25391104 12.1G 83 Linux
/dev/sda4   29298688 117229567 87930880 41.9G 83 Linux




But apart from losing some of my illusions the hard way, all is well.
A big thank you to all the crowd offering suggestions and encouragement.

so long, DdB



I'm glad you were able to install Debian.  :-)


David

HDD long-term data storage with ensured integrity

[David Christensen]

On 3/31/24 02:18, DdB wrote:
> i intend to create a huge backup server from some oldish hardware.
> Hardware has been partly refurbished and offers 1 SSD + 8 HDD on a
> 6core Intel with 64 GB RAM. ... the [Debian] installer ... aborts.

On 4/1/24 11:35, DdB wrote:
> A friend of mine just let me use an external CD-Drive with the netboot
> image. ... all is well.


Now you get to solve the same problem I have been stuck on since last 
November -- how to use those HDD's.



ZFS has been my bulk storage solution of choice for the past ~4 years, 
but the recent data corruption bugs [1, 2] have me worried.  From a 
technical perspective, it's about incorrect concurrent execution of GNU 
cp(1), Linux, and/or OpenZFS.  From a management perspective, it's about 
Capability Maturity Model (CMM) [3] and Programming Systems Product 
(PSP) [4].



The most obvious alternative to ZFS on Debian would be Btrfs.  Does 
anyone have any comments or suggestions regarding Btrfs and data 
corruption bugs, concurrency, CMM level, PSP, etc.?



Does anyone have any comments or suggestions regarding how to use 
magnetic hard disk drives, commodity x86 computers, and Debian for 
long-term data storage with ensured integrity?



David


[1] https://github.com/openzfs/zfs/issues/15526

[2] https://github.com/openzfs/zfs/issues/15933

[3] https://en.wikipedia.org/wiki/Capability_maturity_model

[4] https://en.wikipedia.org/wiki/The_Mythical_Man-Month

Re: help needed to get a bookworm install to succeed

[David Christensen]

On 4/1/24 03:10, DdB wrote:

Am 01.04.2024 um 07:44 schrieb David Christensen:

Please post a console session that identifies the ISO you are using,
verifies the checksum, burns the ISO to a USB flash drive, and compares
the ISO against the flash drive.


Ok, in the meantime, i came to similar conclusions and found that the
USB-stick i was using, had consistent read errors at the first 2
gigabytes after having been used for years as memory extension in my
router. Fixed that and will replace the stick.



A bad USB flash drive would explain why you cannot boot the Debian 
installer.  Please buy a good quality USB 3.0+ flash drive and try again.



David

Re: help needed to get a bookworm install to succeed

[David Christensen]

On 3/31/24 02:18, DdB wrote:

Hello list,

i intend to create a huge backup server from some oldish hardware.
Hardware has been partly refurbished and offers 1 SSD + 8 HDD on a 6core
Intel with 64 GB RAM.
Already before assembling the hardware, grub was working from the SSD,
which got lvm partitioning and is basically empty. As i have no working
CD drive nor can this old machine boot from USB, i put an ISO for
bookworm onto an lvm-LV. Using grub, i can manually boot from that ISO
and see the first installer screens. But after asking some questions,
the installer wants to mount the external media (ISO), and does not find
it on sd[a-z], then aborts.
By switching to Desktop 4, i can see the attempt to search for the
"CD"-drive, which is bound to fail.
I am not familiar with the very restricted shell, that is available from
the installer (busybox) and have not yet found an approach to circumvent
my problems. i would like to use the installer, as debootstrapping would
necessitate alot more knowledge than mine.

Suggestions are welcome :-)
DdB




A computer with a 6-core processor, 64 GB memory, and 9 drive bays/ 
ports that cannot boot USB?  That does not make sense.



Please post a console session that identifies the ISO you are using, 
verifies the checksum, burns the ISO to a USB flash drive, and compares 
the ISO against the flash drive.



Then insert the USB flash drive into a USB port on the the target 
computer, power up and enter Setup, reset the settings to factory 
defaults, enable USB booting, set the USB flash drive as the first boot 
device, save, and exit.  The Debian installer should then boot.



David

Re: Debian 12.5 up-to-date Xfce, Firefox clings to USB stick

[David Christensen]

On 3/30/24 08:17, Antti-Pekka Känsälä wrote:

What could be the deal, when Firefox tries to stop me from unmounting a
stick, after I've accessed files on it through Firefox?  I worry about my
stick security.  Thanks.



Linux knows what files are open on each file system.  If you try to 
unmount a file system with open files or eject a mounted USB drive with 
open files, Linux will refuse and your desktop environment will display 
a suitable error dialog.  This is a feature, not a bug.



The solution is to close all the files on the file system, and then 
unmount it.



David

Re: Debian 12.5.0 amd64 and OpenZFS bug #15526

[David Christensen]

On 3/25/24 15:05, Gareth Evans wrote:

On Fri 22/03/2024 at 21:01, Gareth Evans  wrote:

As anyone interested can see from the ref to #15933 in the below, there seems 
to have been considerable effort in getting to grips with this bug (actually 
multiple bugs), and it looks like a fix may be forthcoming, though not sure at 
the time of writing if there may be some further polishing first

https://github.com/openzfs/zfs/pull/16019


https://github.com/openzfs/zfs/issues/15933

is now closed as completed with fix

https://github.com/openzfs/zfs/commit/102b468b5e190973fbaee6fe682727eb33079811

which for the moment necessarily adds synchronous writes.

FYI.
Gareth



Thank you for keeping an eye on this.


Looking at the github commit, the C code makes me worry -- it does not 
appear to use traditional C/C++ thread-safe programming techniques such 
as I learned in CS and used when I did systems programming (e.g. guard 
functions, critical sections, locks, semaphores, etc.).  Do I need to 
look at more enclosing code to see such, are those techniques missing, 
are there some newer techniques I do not understand, or something else?



David

Re: trying to parse lines from an awkwardly formatted HAR file ...

[David Christensen]

On 3/22/24 22:53, Albretch Mueller wrote:

out of a HAR file containing lots of obfuscating js cr@p and all kinds of
nonsense I was able to extract line looking like:

var00='{\"index\":\"prod-h-006\",\"fields\":{\"identifier\":\"bub_gb_O2EAMAAJ\",\"title\":\"Die
Wissenschaft vom subjectiven Geist\",\"creator\":[\"Karl Rosenkranz\",
\"Mr. ABC123\"],\"collection\":[\"europeanlibraries\",
\"americana\"],\"year\":1843,\"language\":[\"German\"],\"item_size\":797368506},\"_score\":[50.629513]}'
echo "// __ \$var00: |$var00|"

The final result that I need would look like:
o
var02='bub_gb_O2EAMAAJ|Die Wissenschaft vom subjectiven Geist|["Karl
Rosenkranz", "Mr. ABC123"]|["europeanlibraries",
"americana"]|1843|["German"]|797368506|[50.629513]'
echo "// __ \$var02: |$var02|"

I have tried substring substitution, sed et tr to no avail.

lbrtchx



My daily driver:

2024-03-23 04:02:27 dpchrist@laalaa 
~/sandbox/perl/debian-users/20240322-2253-albretch-mueller

$ cat /etc/debian_version; uname -a; perl -v | head -n 2 | grep .
11.9
Linux laalaa 5.10.0-28-amd64 #1 SMP Debian 5.10.209-2 (2024-01-31) 
x86_64 GNU/Linux
This is perl 5, version 32, subversion 1 (v5.32.1) built for 
x86_64-linux-gnu-thread-multi



Put the JSON into a data file, one record per line (my mailer is 
line-wrapping data.json -- it contains two lines):


2024-03-23 04:22:20 dpchrist@laalaa 
~/sandbox/perl/debian-users/20240322-2253-albretch-mueller

$ cat data.json
{"index":"prod-h-006","fields":{"identifier":"bub_gb_O2EAMAAJ","title":"Die 
Wissenschaft vom subjectiven Geist","creator":["Karl Rosenkranz", "Mr. 
ABC123"],"collection":["europeanlibraries", 
"americana"],"year":1843,"language":["German"],"item_size":797368506},"_score":[50.629513]}
{"index":"prod-h-007","fields":{"identifier":"abc_de_12FGHIJKLMNO","title":"My 
Title","creator":["Some Body", "Somebody 
Else"],"collection":["europeanlibraries", 
"americana"],"year":2024,"language":["English"],"item_size":1234567890},"_score":[12.345678]}



A Perl script to read newline-delimited JSON records and pretty print each:

2024-03-23 04:28:59 dpchrist@laalaa 
~/sandbox/perl/debian-users/20240322-2253-albretch-mueller

$ cat munge-json
#!/usr/bin/perl
# $Id: munge-json,v 1.3 2024/03/23 11:28:58 dpchrist Exp $
# Refer to debian-user 3/22/24 22:53 Albretch Mueller
# "trying to parse lines from an awkwardly formatted HAR file"
# by David Paul Christensen dpchr...@holgerdanske.com
# Public Domain
use strict;
use warnings;
use Data::Dumper;
use JSON;
use Getopt::Long;
$Data::Dumper::Sortkeys = 1;
my $debug;
GetOptions('debug|d' => \$debug) or die;
while (<>) {
my $rh = decode_json $_;
print Data::Dumper->Dump([$rh], [qw(rh)]) if $debug;
print
join('|',
$rh->{fields}{identifier},
$rh->{fields}{title},
'["' .  join('", "', @{$rh->{fields}{creator}}) . '"]',
'["' .  join('", "', @{$rh->{fields}{collection}}) . '"]',
$rh->{fields}{year},
'["' .  join('", "', @{$rh->{fields}{language}}) . '"]',
$rh->{fields}{item_size},
'[' .  join(', ', @{$rh->{_score}}) . ']',
), "\n";
}   


Run the script as a Unix filter:

2024-03-23 04:30:16 dpchrist@laalaa 
~/sandbox/perl/debian-users/20240322-2253-albretch-mueller

$ ./munge-json data.json
bub_gb_O2EAMAAJ|Die Wissenschaft vom subjectiven Geist|["Karl 
Rosenkranz", "Mr. ABC123"]|["europeanlibraries", 
"americana"]|1843|["German"]|797368506|[50.629513]
abc_de_12FGHIJKLMNO|My Title|["Some Body", "Somebody 
Else"]|["europeanlibraries", 
"americana"]|2024|["English"]|1234567890|[12.345678]


2024-03-23 04:30:18 dpchrist@laalaa 
~/sandbox/perl/debian-users/20240322-2253-albretch-mueller

$ cat data.json | ./munge-json
bub_gb_O2EAMAAJ|Die Wissenschaft vom subjectiven Geist|["Karl 
Rosenkranz", "Mr. ABC123"]|["europeanlibraries", 
"americana"]|1843|["German"]|797368506|[50.629513]
abc_de_12FGHIJKLMNO|My Title|["Some Body", "Somebody 
Else"]|["europeanlibraries", 
"americana"]|2024|["English"]|1234567890|[12.345678]



David

Re: electrons/the Internet doesn't like … that I like to eat raw garlic, ...

[David Christensen]

On 3/4/24 16:06, David Wright wrote:

On Mon 04 Mar 2024 at 12:36:54 (-0800), David Christensen wrote:

On 3/4/24 08:37, Albretch Mueller wrote:



_LINK="https://christuniversity.in/uploads/course/E&Comp_21-25_Lateral
Entry(1)_20210618043317.pdf"


I ignored the filename, and pasted https://christuniversity.in/uploads/course/
into FF. Here's the text copy/pasted off the page that was displayed.
It was accompanied by the image that is displayed at
https://christuniversity.in/images/cour-btch-bnnr.jpg
...
Just a data point.



Testing ping again:

2024-03-04 17:31:14 dpchrist@laalaa ~
$ ping -c 3 christuniversity.in
PING christuniversity.in (111.93.136.229) 56(84) bytes of data.
64 bytes from 111.93.136.229 (111.93.136.229): icmp_seq=1 ttl=49 time=273 ms
64 bytes from 111.93.136.229 (111.93.136.229): icmp_seq=2 ttl=49 time=272 ms
64 bytes from 111.93.136.229 (111.93.136.229): icmp_seq=3 ttl=49 time=273 ms

--- christuniversity.in ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2002ms
rtt min/avg/max/mdev = 271.914/272.406/272.725/0.352 ms

2024-03-04 17:31:23 dpchrist@laalaa ~
$ ping -c 3 103.105.225.131
PING 103.105.225.131 (103.105.225.131) 56(84) bytes of data.

--- 103.105.225.131 ping statistics ---
3 packets transmitted, 0 received, 100% packet loss, time 2060ms

2024-03-04 17:31:41 dpchrist@laalaa ~
$ ping -c 3 111.93.136.229
PING 111.93.136.229 (111.93.136.229) 56(84) bytes of data.
64 bytes from 111.93.136.229: icmp_seq=1 ttl=49 time=277 ms
64 bytes from 111.93.136.229: icmp_seq=2 ttl=49 time=280 ms
64 bytes from 111.93.136.229: icmp_seq=3 ttl=49 time=276 ms

--- 111.93.136.229 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2003ms
rtt min/avg/max/mdev = 276.118/277.636/280.041/1.719 ms


So, same results as before -- one IP works, the other does not, and I 
got lucky with the FQDN (a previous test got the wrong IPv4 address and 
timed out).



Testing Firefox:

https://christuniversity.in/uploads/course/

https://103.105.225.131/uploads/course/

https://111.93.136.229/uploads/course/


All three time out.


Doing whois searches on the A record IP addresses:

1.  https://www.whois.com/whois/103.105.225.131


The IPv4 address holder appears to be a small ISP with 4 @ IPv4 class C 
ranges (1,024 addresses).  It appears nothing is connected to the 
christuniversity.in IPv4 address.



2. https://www.whois.com/whois/111.93.136.229


The IPv4 address holder appears to be a larger ISP with 1 @ IPv4 class B 
range (65,535 addresses).  It appears there is a host connected to the 
christuniversity.in IPv4 address, but I cannot connect to its web server.



STFW for information about DNS A (Address) records, I see:

https://www.cloudflare.com/learning/dns/dns-records/dns-a-record/

What is a DNS A record?
...
The vast majority of websites only have one A record, but it is possible 
to have several. Some higher profile websites will have several 
different A records as part of a technique called round robin load 
balancing, which can distribute request traffic to one of several IP 
addresses, each hosting identical content.



So, two DNS A records for the same FQDN is allowed and can be useful.


Searching for all DNS records for christuniversity.in :

https://www.whatsmydns.net/dns-lookup?query=christuniversity.in&server=opendns


I see the two A (address) records that we have been discussing:

id 21430, opcode QUERY, rcode NOERROR, flags QR RD RA
;QUESTION
christuniversity.in. IN A
;ANSWER
christuniversity.in. 60 IN A 111.93.136.229
christuniversity.in. 60 IN A 103.105.225.131
;AUTHORITY
;ADDITIONAL


I see five MX (mail exchanger) records:

id 52477, opcode QUERY, rcode NOERROR, flags QR RD RA
;QUESTION
christuniversity.in. IN MX
;ANSWER
christuniversity.in. 60 IN MX 1 aspmx.l.google.com.
christuniversity.in. 60 IN MX 5 alt1.aspmx.l.google.com.
christuniversity.in. 60 IN MX 5 alt2.aspmx.l.google.com.
christuniversity.in. 60 IN MX 10 alt3.aspmx.l.google.com.
christuniversity.in. 60 IN MX 10 alt4.aspmx.l.google.com.
;AUTHORITY
;ADDITIONAL


I see two NS (nameserver) records:

id 57399, opcode QUERY, rcode NOERROR, flags QR RD RA
;QUESTION
christuniversity.in. IN NS
;ANSWER
christuniversity.in. 60 IN NS ns1.christuniversity.in.
christuniversity.in. 60 IN NS ns2.christuniversity.in.
;AUTHORITY
;ADDITIONAL


I find it strange that there are no A records for:

ns1.christuniversity.in
ns2.christuniversity.in


And yet dig(1) can find them:

2024-03-04 17:56:59 dpchrist@laalaa ~
$ dig @9.9.9.9 ns1.christuniversity.in

; <<>> DiG 9.16.48-Debian <<>> @9.9.9.9 ns1.christuniversity.in
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 40331
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 512
;; QUESTION SECTION:
;ns1.christuniversity.in.   IN  A

;

Re: resolv.conf (was Re: electrons/the Internet [racism redacted])

[David Christensen]

On 3/4/24 13:11, Greg Wooledge wrote:

On Mon, Mar 04, 2024 at 12:36:54PM -0800, David Christensen wrote:

I believe Debian rewrites /etc/resolv.conf on every boot.


This is not correct.  It's *partly* correct if you ignore a lot of
complicating factors.

Short version: read <https://wiki.debian.org/resolv.conf>.

Long version follows:

If you have a static network interface configuration, defined in
/etc/network/interfaces, with no DHCP client, no VPN stuff going on,
etc. then your /etc/resolv.conf will not be changed.  You can edit the
file and put whatever you want in it, and it'll remain as you wish it
to be.

Unfortunately, almost *nobody* has a setup like this any more.

In a more typical environment, you get your IP address via DHCP, which
means you're running a DHCP client daemon.  Most DHCP client daemons
will rewrite the /etc/resolv.conf file every time they refresh their
DHCP lease.  This may indeed happen at boot time, but it'll also happen
a couple times a day during normal operations.

So, a simple instruction like "edit /etc/resolv.conf" is no longer
possible.  Even worse, there's no single *alternative* either.  You can't
even say "do ___ instead".

To put the correct values into your /etc/resolv.conf file nowdays, you
have to select a *strategy*.  You need to find an indirect way to put
the right content into some *other* place, in such a way that it will
eventually find its way into /etc/resolv.conf every time the file is
rewritten.  And there are *lots* of strategies that will work, so you
can't even say "obviously this one is best".  Life is not that simple.

Or, you could use chattr +i to make the /etc/resolv.conf file immutable,
so DHCP clients and other programs cannot overwrite it.

Either way, you take ownership of whatever strategy you decide to use,
together with its pros and cons.  You'll have to understand that on *this*
system, you went with *this* strategy, and remember where to put your
changes, and how to make them.  Or at the very least, you'll need to be
*aware* of all the strategies you've got in play on all of your systems,
and know how to identify which one is in use on any given system.



Thank you for the clarification.  Thankfully, my gateway DHCP server and 
my Debian instances work together.



David

Re: electrons/the Internet doesn't like question authority niggahs?, or is it that I like to eat raw garlic, ...

[David Christensen]

On 3/4/24 08:37, Albretch Mueller wrote:





Yes, networking problems are infuriating.



Something that shouldn't be happening at all is that after I use
traceroute once, it doesn't work again and my Internet access speed
describes like a sinus curve which amplitude remains for the most part
under 16KiB per second and for more than one second as 0B per second.

_LINK="https://christuniversity.in/uploads/course/E&Comp_21-25_Lateral
Entry(1)_20210618043317.pdf"



I have AT&T Internet service in Tracy, California.


My daily driver is:

2024-03-04 10:47:12 dpchrist@laalaa ~
$ cat /etc/debian_version ; uname -a
11.9
Linux laalaa 5.10.0-28-amd64 #1 SMP Debian 5.10.209-2 (2024-01-31) 
x86_64 GNU/Linux



When I click the above link in my mail client (Thunderbird), my browser 
(Firefox) attempts to open the URL.  But, the URL is mangled by mail 
client line wrap and/or indentation (?), and the connection times out:


https://christuniversity.in/uploads/course/E&Comp_21-25_Lateral

An error occurred during a connection to christuniversity.in.

The site could be temporarily unavailable or too busy. Try again in 
a few moments.
If you are unable to load any pages, check your computer’s network 
connection.
If your computer or network is protected by a firewall or proxy, 
make sure that Firefox is permitted to access the web.



The following URL's also time out (see DNS comments, below):

http://christuniversity.in/
http://103.105.225.131/
http://111.93.136.229/

https://christuniversity.in/
https://103.105.225.131/
https://111.93.136.229/



  1) is the file actually there?:

wget -q --spider "${_LINK}"; _WGETQ=$?



I refrain from spidering web sites -- being blackholed is not good.



$ ls -l /etc/resolv.conf
-rw-r--r-- 1 root root 204 Mar  3 18:59 /etc/resolv.conf



2024-03-04 10:50:49 dpchrist@laalaa ~
$ ls -l /etc/resolv.conf
-rw-r--r-- 1 root root 83 Mar  4 09:50 /etc/resolv.conf



$ cat /etc/resolv.conf
# Generated by NetworkManager
# nameserver 192.168.1.254
# nameserver 192.168.68.1

# https://serverfault.com/questions/76421/wget-cant-resolve-host
# RED 2013-03-31
nameserver 8.8.8.8
nameserver 8.8.4.4



2024-03-04 10:51:32 dpchrist@laalaa ~
$ cat /etc/resolv.conf
# Generated by NetworkManager
search tracy.holgerdanske.com
nameserver 192.168.5.1


I believe Debian rewrites /etc/resolv.conf on every boot.


Hard coding Google Public DNS servers should work, but letting your 
gateway do it for your LAN is easier to manage, is faster, and conserves 
WAN bandwidth.  I would revert your changes.



And, Google is watching you.


STFW for DNS privacy:

https://avoidthehack.com/best-dns-privacy


I think I will configure my gateway to use Quad9:

https://www.quad9.net/service/locations/


The next level up would be DNS over TLS (DoT), DNS over HTTPS (DoH), 
DNSCrypt, etc.:


https://en.wikipedia.org/wiki/DNS_over_TLS



$ ls -l /etc/nsswitch.conf
-rw-r--r-- 1 root root 613 Mar  3 12:57 /etc/nsswitch.conf



2024-03-04 10:52:04 dpchrist@laalaa ~
$ ls -l /etc/nsswitch.conf
-rw-r--r-- 1 root root 542 Jan  9  2022 /etc/nsswitch.conf



$ cat /etc/nsswitch.conf
# /etc/nsswitch.conf
#
# Example configuration of GNU Name Service Switch functionality.
# If you have the `glibc-doc-reference' and `info' packages installed, try:
# `info libc "Name Service Switch"' for information about this file.

passwd: files
group:  files
shadow: files
gshadow:files

#hosts:  files mdns4_minimal [NOTFOUND=return] dns myhostname

hosts:  files dns mdns4_minimal [NOTFOUND=return] dns myhostname

networks:   files

protocols:  db files
services:   db files
ethers: db files
rpc:db files

netgroup:   nis



2024-03-04 10:56:37 dpchrist@laalaa ~
$ cat /etc/nsswitch.conf
# /etc/nsswitch.conf
#
# Example configuration of GNU Name Service Switch functionality.
# If you have the `glibc-doc-reference' and `info' packages installed, try:
# `info libc "Name Service Switch"' for information about this file.

passwd: files systemd
group:  files systemd
shadow: files
gshadow:files

hosts:  files mdns4_minimal [NOTFOUND=return] dns
networks:   files

protocols:  db files
services:   db files
ethers: db files
rpc:db files

netgroup:   nis


It appears my /etc/nsswitch.conf has not been touched since 
installation.  I would revert your changes.




; <<>> DiG 9.18.19-1~deb12u1-Debian <<>> +time christuniversity.in
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 49715
;; flags: qr rd ra; QUERY: 1, ANSWER: 2, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 512
;; QUESTION SECTION:
;christuniversity.in.   IN  A

;; ANSWER SECTION:
christuniversity.in.60  IN  A   103.105.225.131
christuniversity.in.60  IN  A   111.93.136.229

;; Query time: 327 msec
;; SERVER: 8.8.

Re: Debian 12.5.0 amd64 and OpenZFS bug #15526

[David Christensen]

On 2/26/24 20:52, Gareth Evans wrote:

Replied to OP by mistake, reposting to list.

On Sun 25/02/2024 at 05:34, David Christensen  wrote:

debian-user:

Is Debian 12.5.0 amd64 affected by OpenZFS bug #15526?

https://cdimage.debian.org/debian-cd/current/amd64/iso-cd/debian-12.5.0-amd64-netinst.iso

https://packages.debian.org/bookworm/zfs-dkms

https://github.com/openzfs/zfs/issues/15526


Hi David,

Given the complexity of the issues, I'm not sure if this truly answers your 
question, but

https://github.com/openzfs/zfs/issues/15933

seems to suggest that or a similar issue is still ongoing with Open ZFS 2.2.3, 
which is later than the version currently available from bookworm or 
bookworm-backports.

It seems bookworm-backports might eventually provide the solution, if at all, 
per the Debian wiki on ZFS:

"it is recommended by Debian ZFS on Linux Team to install ZFS related packages from 
Backports archive. Upstream stable patches will be tracked and compatibility is always 
maintained."
https://wiki.debian.org/ZFS

Currently:

$ apt policy zfs-dkms
zfs-dkms:
   Installed: 2.2.2-4~bpo12+1
   Candidate: 2.2.2-4~bpo12+1
   Version table:
  *** 2.2.2-4~bpo12+1 100
 100 https://deb.debian.org/debian bookworm-backports/contrib amd64 
Packages
 100 https://deb.debian.org/debian bookworm-backports/contrib i386 
Packages
 100 /var/lib/dpkg/status
  2.1.11-1 500
 500 https://deb.debian.org/debian bookworm/contrib amd64 Packages
 500 https://deb.debian.org/debian bookworm/contrib i386 Packages

Hope that helps.
Gareth



That you for citing OpenZFS bug #15933.


These appear to be the ZFS packages for the available Debian releases:

https://packages.debian.org/buster/zfs-dkms

buster  zfs-dkms (0.7.12-2+deb10u2)
buster-backportszfs-dkms (2.0.3-9~bpo10+1)
bullseyezfs-dkms (2.0.3-9+deb11u1)
bullseye-backports  zfs-dkms (2.1.11-1~bpo11+1)
bookwormzfs-dkms (2.1.11-1)
bookworm-backports  zfs-dkms (2.2.2-4~bpo12+1)
trixie  zfs-dkms (2.2.2-4)


The question is, how far back to go?  Is OpenZFS 2.1.x buggy?  OpenZFS 
2.0.x?  What is 0.7.12 -- OpenZFS, ZFS-on-Linux, or something else -- 
and is it buggy?



David

Debian 12.5.0 amd64 and OpenZFS bug #15526

[David Christensen]

debian-user:

Is Debian 12.5.0 amd64 affected by OpenZFS bug #15526?

https://cdimage.debian.org/debian-cd/current/amd64/iso-cd/debian-12.5.0-amd64-netinst.iso

https://packages.debian.org/bookworm/zfs-dkms

https://github.com/openzfs/zfs/issues/15526


David

Re: Orphaned Inode Problem

[David Christensen]

On 2/21/24 03:00, Jörg-Volker Peetz wrote:

Hi,

did you take a look at the smartctl output?

Somewhere I read, for maintainance of an SSD all it's cells should be 
read from time to time like this


sudo dd if=/dev/DEVICE of=/dev/null bs=8M status=progress

where device is something like sda or nvme0n1, especially if it was 
switched off for a longer period. At least, it shows the current read 
performance of the device.
An SSD should regularly be trimmed, if in use. This is to assist it's 
wear leveling process.


What's your opinion?

Regards,
Jörg.



I prefer to run a SMART long test periodically.  This should read every 
cell, including those that are reserved and not visible to the OS.



AIUI So long as the SSD can maintain a supply of erased cells via 
manufacturer over-provisioning, trim is not required to maintain 
performance.  If you have workload that does a lot of writes in a short 
period of time and exhausts the manufacturer over-provisioning, leaving 
free space on the SSD and trimming can be a work-around.



If you are using strong encryption, not trimming will leave crypttext on 
disk that creates more work for an attacker.  If you are using weak 
encryption, not trimming will leave crypttext on disk that an attacker 
can recover.



For imaging/ cloning, trimming will zero blocks freed by the OS and 
facilitate compression of the image file.



I have a SOHO network with about two dozen disks.  Running smartctl by 
hand is a PITA.  Running fstrim(8) by hand is easy enough.  I try to do 
both once a month.  I need to figure out smartd(8).



David