Re: Network bridge and MAC address exposure

[Igor Cicimov]

On Sun, Sep 4, 2022 at 4:40 PM Rand Pritelrohm 
wrote:

> Hello,
>
> I am not a network specialist and despite a lot of documentation
> readings and searchs on the net I haven't get a simple and clear answer
> to my question.
>
> Consider this simple schematic:
>
>
> | VM | -> | HOST | -> | GW | -> ISP
>
>
> Lets say the physical interface name on the 'host' is eth0 and the LAN
> subnet is 192.168.0.0.
>
> I want to configure the network on the 'host' in order for the VM to
> access the Internet.
>
> Thus I consider 2 scenarios to setup the 'host' network.
>
>
> 1. Bridge using routed subnet:
>
> ip link add dev br0 type bridge
> ip addr add 192.168.222.1/24 dev br0
> ip link set dev br0 up
>
> ip tuntap add tap0 mode tap
> ip link set dev tap0 up
> ip link set dev tap0 master br0
>
> #Then I have to enable routing
> echo '1' > /proc/sys/net/ipv4/ip_forward
> iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE
>
>
> 2. Bridge on the same subnet as the LAN:
>
> ip link add dev br0 type bridge
> ip link set dev br0 up
>
> ip link set dev eth0 master br0
> ip link set dev eth0 up
> ip addr add 192.168.0.200/24 dev br0
> ip route add default via 192.168.0.1
>
> ip tuntap add tap0 mode tap
> ip link set dev tap0 up
> ip link set dev tap0 master br0
>
>
> For both scenarios the VM is then setup with it's own MAC address and
> it's IP on the configured subnet of the bridge.
>
>
> Here is my question:
> For both scenarios, what is the effectively seen MAC address by the
> GW when the VM access the Internet (host or VM MAC address)?
>
> Regards,
> Rand.
>
>
MAC is used in L2 networking level and GW is L3 routing hence the MAC is
irrelevant. If you run `ip route show` locally all you see is IPs and
network interfaces no MACs in there at all.

Re: failing HDD, ddrescue says remaning time is 7104d

[David Wright]

On Sat 03 Sep 2022 at 17:46:39 (+0200), ppr wrote:
> Le 01/09/2022 à 00:24, David Wright a écrit :
> 
> > -R --reverse will start an attempt from the end of the disk, and if
> > you're extremely lucky, it might copy most of the remaining 960-odd GB
> > of data. OTOH it might only confirm that the disk is closer to meeting
> > its maker than it was when you started the rescue.
> 
> I am considering using -R option (as suggested by David Wright) to see
> if ddrescue can have better transfert rate starting from the other
> side of the HDD. My concern is whether or not ddrescue can resume a
> job started, stopped, and relaunched with -R option added. ddrescue's
> manual says only:
> 
> --reverse
> Reverse the direction of all passes (copying, trimming, scraping,
> and retrying). Every pass that is normally run forwards will now be
> run backwards, and vice versa. '--reverse' does not modify the size of
> the blocks copied during each phase, just the order in which they are
> tried.
> 
> If I stop ddrescue and add -R option, would the mapfile be able to
> handle this (ddrescue started from the beginning and the end the the
> drive)?

I don't know the answer to that. For safety, I would copy the mapfile
so far made before changing direction, just in case. After changing
direction, I would compare the backed up mapfile with the one that's
being written to see what's happening to it.

Note that the size of the output image may suddenly increase to the
total size of the disk being rescued, because ddrescue needs to start
writing to the end of it.

> I also add some information related to other answers:
> 
> - The destination HDD is an external USB3 one. It is a physical RAID 1
> with two 8To HDD 7200rpm. The commercial name is Lacie 2big raid. The
> fact is image is growing and the speed seems to vary (even if never
> fast) lets me think it is perhaps the HDD condition which is the cause
> of the slow transfer rate.

So your diagnostic (image is growing) won't be usable after reversing.
But running, say, tail -f on the mapfile might show lines being
appended to it. Or repeated diffs between the mapfile and its backup
as ddrescue runs. (Your experience with ddrescue is running ahead of mine.)

> - I commented out the failing HDD in /etc/fstab and disable
> smartmontools tests for this drive. I did not try to mount the HDD.

Cheers,
David.

Re: Seeing progross during fsck on boot

[Greg Wooledge]

On Mon, Sep 05, 2022 at 01:19:29AM +1000, David wrote:
> 6) e2fsck is capable of checking multiple filesystems at the same time, for
>faster completion when there are multiple drives with the same number in
>the 'passno' field of /etc/fstab.
> 
>But, as noted by David Wright, e2fsck is only capable of displaying one
>progress bar at a time.

On some old Red Hat systems at work, I have observed e2fsck performing
parallel scans of file systems on multiple disks.  On the systems in
question, it showed one progress bar *at a time* in a cyclic rotation.
That is, it would show the progress bar for file system number one, for
a few seconds, and then the progress bar for file system number two, for
a few seconds, and so on.

I don't know whether the e2fsck on Debian does this.  Or even whether
modern Red Hat descendant systems do it.  It's just what I saw in
the past.

Re: Seeing progross during fsck on boot

[David]

On Sun, 4 Sept 2022 at 22:15, Mike  wrote:

> Thanks for the very detailed description.  This was just what I was
> after.  I'd kind of figured a few things, that it likely needed some
> kind of switch to fsck to produce output and likely systemd was either
> not passing that flag or swallowing the output.  I've never delved into
> how disks get fscked on boot, either with systemd or sysv, so I wasn't
> really sure where to start looking.
>
> Your explanation was very helpful and I think also the last point was a
> good one.  I've converted the remaining ext3 filesystems to ext4 and
> will see how that goes.

Yes, I feel confident that will give you a huge benefit. You can test it,
see below.

I spent a bit more time looking at this and I am confident now about the
conclusions I have reached.

My previous message was "thinking out load" and somewhat inconclusive, the
contents of this message are more useful.

I have been curious in the past about how all this works, so I had an
interest in looking into it, and am happy to share the results.

1) At boot, the filesystems which have 1 in the 'passno' (sixth) field of
   /etc/fstab are checked by /sbin/fsck in the initrd.

   I believe systemd is irrelevant, because it is not running at that time.

2) After systemd starts, as explained by 'man 8 systemd-fsck', systemd only
   checks the root filesystem if that was not already done by the initrd.

   Having systemd check the root file system is something that we want to
   avoid, because it would need to unmount it.

   When running, systemd also checks the filesystems which have values of
   2 or higher in the 'passno' field of etc/fstab, because these are not
   checked by the initrd processes.

3) The command 'fsck' works on many filesystems. It is provided by the
   'util-linux' package.

   In the case of ext2/3/4 filesystems, 'fsck' invokes 'e2fsck' also known
   as 'fsck.ext4' to do the work. Those are provided by the 'e2fsprogs'
   package.

4) systemd-fsck also uses 'e2fsck' to do the work on ext2/3/4 systems.

5) Not important, but the '-C' argument of fsck and e2fsck are slightly
   different.

   Giving '-C' to fsck causes it to invoke e2fsck with '-C0'.

   Evidence:
 # fsck -C -N /dev/sda1
 fsck from util-linux 2.36.1
 [/usr/sbin/fsck.ext4 (1) -- /mnt/Z] fsck.ext4 -C0 /dev/sda1
 #

   '-N' invokes a "dry-run", which just prints the command that fsck
   passes to its helper program, instead of executing it

   Note that e2fsck and fsck.ext4 are the same program.

6) e2fsck is capable of checking multiple filesystems at the same time, for
   faster completion when there are multiple drives with the same number in
   the 'passno' field of /etc/fstab.

   But, as noted by David Wright, e2fsck is only capable of displaying one
   progress bar at a time.

7) Looking at the source of fsck
 https://sources.debian.org/src/e2fsprogs/1.46.5-2/e2fsck/unix.c
   around line 445 is interesting.

   It confirms that when fsck gives output like this, often seen at boot:

 : clean, 123/456 files, 789/1011 blocks

   that the keyword "clean" is fsck telling us "I looked briefly at the
and I decided that it did not need to be scanned".

   The code shows tht when this occurs, no progress bar is displayed.
   A progress bar is only displayed when fsck actually scans the device.

8) e2fsck can be forced to scan a device by giving it the '-f' option. This
   will be done by the initrd code if 'forcefsck' is specified on the
   kernel boot command. That is quite an easy test for anyone to do.

9) This kernel parameter 'forcefsck' is detected by
 /usr/share/initramfs-tools/init
   and slightly modified before being passed to the _checkfs_once()
   function in
 /usr/share/initramfs-tools/functions

10) I have confirmed that adding 'forcefsck' does cause a progress bar to
be displayed. I tested here on Debian 10 i386 because I needed a slow
machine to see it, because ext4 checking is so fast.

11) I also confirmed that the fsck progress bar is displayed even if
'quiet' is also specified on the kernel command line.

Re: Network bridge and MAC address exposure

[tomas]

On Sun, Sep 04, 2022 at 09:42:34AM +0200, john doe wrote:
> On 9/4/2022 8:39 AM, Rand Pritelrohm wrote:

[...]

> >  #Then I have to enable routing
> >  echo '1' > /proc/sys/net/ipv4/ip_forward
> >  iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE
> > 
> 
> You are answering your own question, if you masquerade the MAC addr from
> the VM will not be seen upstream.

Only that NAT masquerades the IP address. It doesn't know anything
about MACs (as far as it is concerned there could be no MAC at all,
it wouldn't care).

The MAC disappears as soon as there is routing. You see the router's
MAC at the other side. No need to NAT.

Cheers
-- 
t


signature.asc
Description: PGP signature

Re: Seeing progross during fsck on boot

[David Wright]

On Sun 04 Sep 2022 at 13:15:24 (+0100), Mike wrote:
> On Sun, Sep 04, 2022 at 01:01:03AM +1000, David wrote:
> > On Sun, 4 Sept 2022 at 00:18, Charles Curley
> >  wrote:
> > > On Sat, 3 Sep 2022 22:57:19 +1000 David  wrote:
> > 
> > > > I imagine that could be overcome by copying the above service file to
> > > >   /etc/lib/systemd/system/systemd-fsck-root.service and editing the
> > > > above ExecStart line to use /sbin/fsck instead.
> > >
> > > I believe on Debian that should be
> > > /etc/systemd/system/systemd-fsck-root.service
> > 
> > Hi Charles, yes thanks for picking up that edit mistake, it was supposed to
> > be a simple change (lib --> etc) but I neglected to delete the 'lib'.
> > 
> > Also I noticed another error in my transcription of the console message,
> > I missed the hyphen in the package name, it should be:
> > Begin: Will now check root file system ... fsck from util-linux 2.36.1
> >
> Thanks for the very detailed description.  This was just what I was
> after.  I'd kind of figured a few things, that it likely needed some
> kind of switch to fsck to produce output and likely systemd was either
> not passing that flag or swallowing the output.  I've never delved into
> how disks get fscked on boot, either with systemd or sysv, so I wasn't
> really sure where to start looking.

As I said, we don't know the disposition of your disks. The root
partition isn't a problem: I've made no changes like the above, but
I can see it's being checked. But if you also need to check more than
one other partition at boot time, then the penalty for obtaining a
progress indication may be serialisation, which sounds undesirable
in your case (parallel takes four hours).

> Your explanation was very helpful and I think also the last point was a
> good one.  I've converted the remaining ext3 filesystems to ext4 and
> will see how that goes.

That might well be the most productive change made.

> It was an interesting point too that someone suggested about
> ShowStatus=auto.  That sounds helpful, athough when I look in
> system.conf I notice two things:
> 
> The comment at the top:
> 
> # Entries in this file show the compile time defaults.
> 
> And the entry itself:
> 
> #ShowStatus=yes
> 
> This suggests that it should be printing status stuff.  I checked my
> config on my desktop and saw the same.  That I recall only prints one
> line during startup for a service that fails to start and that I've
> never bothered to fix.

No, the documentation says: "Defaults to enabled, unless quiet
is passed as kernel command line option, in which case it defaults
to error." AIUI the Debian default /is/ quiet, and you can see it
in my kernel command line that I posted. (IIRC, not including quiet
can be somewhat overwhelming in the level of detail it spews out.)
So you need, as I do, to use the kernel command line to set it,
rather than system.conf.

> I'll do some more digging on my desktop to understand the ShowStatus
> thing but for now I'll be happy to see how I get along with ext4.
> 
> Thanks again to everyone who offered their input into this.  It's been
> very helpful for me.

Cheers,
David.

Re: booting 5.10 686-pae kernel on domU

[Tim Woodall]

On Sun, 4 Sep 2022, Andy Smith wrote:


Hello,

On Sun, Sep 04, 2022 at 11:07:35AM +0100, Tim Woodall wrote:

Should I be able to boot a 686-pae i686 kernel on a xen domU (bullseye -
but bookworm fails too)


The Linux kernel removed PV mode 32-bit Xen domU support at version
5.9. What type of virtualisation are you using? (show us your domain
config file from /etc/xen)

type=pvh should work. type=pv is not expected to work. I don't know
about type=hvm.


kernel="/usr/lib/grub-xen/grub-i386-xen.bin"
memory=256
name="test17"
type="pvh"
pvshim=1
disk=[ 
'script=block-iscsi,vdev=xvda,target=portal=xen17:3260,iqn=iqn.xen17:test17,w',
]
vif=[ 'bridge=xenbr0,mac=00:16:3e:e0:70:50',
]

But you've given me the clue I needed to fix it:
kernel="/usr/lib/grub-xen/grub-i386-xen_pvh.bin"

and remove pvshim=1


Thanks!


Making the domU multiarch and booting it with an amd64 kernel while
retaining the i686 userland should work.


Good idea.

Tim.

Re: Seeing progross during fsck on boot

[Mike]

On Sun, Sep 04, 2022 at 01:01:03AM +1000, David wrote:
> On Sun, 4 Sept 2022 at 00:18, Charles Curley
>  wrote:
> > On Sat, 3 Sep 2022 22:57:19 +1000 David  wrote:
> 
> > > I imagine that could be overcome by copying the above service file to
> > >   /etc/lib/systemd/system/systemd-fsck-root.service and editing the
> > > above ExecStart line to use /sbin/fsck instead.
> >
> > I believe on Debian that should be
> > /etc/systemd/system/systemd-fsck-root.service
> 
> Hi Charles, yes thanks for picking up that edit mistake, it was supposed to
> be a simple change (lib --> etc) but I neglected to delete the 'lib'.
> 
> Also I noticed another error in my transcription of the console message,
> I missed the hyphen in the package name, it should be:
> Begin: Will now check root file system ... fsck from util-linux 2.36.1
>

Hi David,

Thanks for the very detailed description.  This was just what I was
after.  I'd kind of figured a few things, that it likely needed some
kind of switch to fsck to produce output and likely systemd was either
not passing that flag or swallowing the output.  I've never delved into
how disks get fscked on boot, either with systemd or sysv, so I wasn't
really sure where to start looking.

Your explanation was very helpful and I think also the last point was a
good one.  I've converted the remaining ext3 filesystems to ext4 and
will see how that goes.

It was an interesting point too that someone suggested about
ShowStatus=auto.  That sounds helpful, athough when I look in
system.conf I notice two things:

The comment at the top:

# Entries in this file show the compile time defaults.

And the entry itself:

#ShowStatus=yes

This suggests that it should be printing status stuff.  I checked my
config on my desktop and saw the same.  That I recall only prints one
line during startup for a service that fails to start and that I've
never bothered to fix.

I'll do some more digging on my desktop to understand the ShowStatus
thing but for now I'll be happy to see how I get along with ext4.

Thanks again to everyone who offered their input into this.  It's been
very helpful for me.

Regards,
Mike. 


signature.asc
Description: PGP signature

Re: Want to contibute to debian

[The Wanderer]

On 2022-09-04 at 07:19, piorunz wrote:

> On 03/09/2022 08:44, 5HROUD wrote:
>
>> Hi there,
>> I am a student who develops software application for Linux. Because all
>> my project are opensource I wish I could work as a debian developer.
>> Since I love using debian and also wanted to contribute to it.
>> Thanks,
>> Shroud
>>
>> Sent with Proton Mail  secure email.
> 
> Great. Best way to start would be to check:
> https://wiki.debian.org/DebianDeveloper
> https://www.debian.org/doc/manuals/maint-guide/
> 
> And so on, everything is on Debian Wiki and other pages. Good luck :)

Also consider looking at the 'how-can-i-help' package:

$ apt-cache show how-can-i-help

-- 
   The Wanderer

The reasonable man adapts himself to the world; the unreasonable one
persists in trying to adapt the world to himself. Therefore all
progress depends on the unreasonable man. -- George Bernard Shaw



signature.asc
Description: OpenPGP digital signature

Re: Want to contibute to debian

[piorunz]

On 03/09/2022 08:44, 5HROUD wrote:

Hi there,
I am a student who develops software application for Linux. Because all
my project are opensource I wish I could work as a debian developer.
Since I love using debian and also wanted to contribute to it.
Thanks,
Shroud

Sent with Proton Mail  secure email.


Great. Best way to start would be to check:
https://wiki.debian.org/DebianDeveloper
https://www.debian.org/doc/manuals/maint-guide/

And so on, everything is on Debian Wiki and other pages. Good luck :)

--
With kindest regards, Piotr.

⢀⣴⠾⠻⢶⣦⠀
⣾⠁⢠⠒⠀⣿⡁ Debian - The universal operating system
⢿⡄⠘⠷⠚⠋⠀ https://www.debian.org/
⠈⠳⣄

Re: booting 5.10 686-pae kernel on domU

[Andy Smith]

Hello,

On Sun, Sep 04, 2022 at 11:07:35AM +0100, Tim Woodall wrote:
> Should I be able to boot a 686-pae i686 kernel on a xen domU (bullseye -
> but bookworm fails too)

The Linux kernel removed PV mode 32-bit Xen domU support at version
5.9. What type of virtualisation are you using? (show us your domain
config file from /etc/xen)

type=pvh should work. type=pv is not expected to work. I don't know
about type=hvm.

Making the domU multiarch and booting it with an amd64 kernel while
retaining the i686 userland should work.

Cheers,
Andy

-- 
https://bitfolk.com/ -- No-nonsense VPS hosting

booting 5.10 686-pae kernel on domU

[Tim Woodall]

Hi,

Should I be able to boot a 686-pae i686 kernel on a xen domU (bullseye -
but bookworm fails too)

(XEN) d46v0 Unhandled page fault fault/trap [#14, ec=0010]
(XEN) Pagetable walk from 0100:
(XEN)  L4[0x000] = 09790027 1f90
(XEN)  L3[0x000] = 097a4027 1fa4
(XEN)  L2[0x008] =  
(XEN) domain_crash_sync called from entry.S: fault at 82d040271ed8 
x86_64/compat/entry.S#compat_create_bounce_frame+0xd9/0xf6
(XEN) Domain 46 (vcpu#0) crashed on cpu#0:
(XEN) [ Xen-4.14.5  x86_64  debug=n   Not tainted ]
(XEN) CPU:0
(XEN) RIP:e019:[<0100>]
(XEN) RFLAGS: 0246   EM: 1   CONTEXT: pv guest (d46v0)
(XEN) rax: 0100   rbx:    rcx: 
(XEN) rdx:    rsi: c1f8d000   rdi: 0001
(XEN) rbp:    rsp: c20a5ff0   r8:  
(XEN) r9:     r10:    r11: 
(XEN) r12:    r13:    r14: 
(XEN) r15:    cr0: 8005003b   cr4: 001006e4
(XEN) cr3: 0f25b000   cr2: 0100
(XEN) fsb:    gsb:    gss: 
(XEN) ds: e021   es: e021   fs: e021   gs: e021   ss: e021   cs: e019
(XEN) Guest stack trace from esp=c20a5ff0:
(XEN)   0010 0100 0001e019 00010046    
(XEN)          
(XEN)          
(XEN)          
(XEN)          
(XEN)          
(XEN)          
(XEN)          
(XEN)          
(XEN)          
(XEN)          
(XEN)          
(XEN)          
(XEN)          
(XEN)          
(XEN)          
(XEN)          
(XEN)          
(XEN)          
(XEN)          

This is a minimal install - I've now tried both as a fresh install and
upgrading from buster but I always get this same failure. The 4.19
buster kernel boots bullseye fine. Dom0 is bullseye.


I cannot find anybody else reporting this so either I'm doing something
wrong, doing something that is no longer supposed to work, or nobody is
using xen any more...


Assuming there isn't a simple solution, what do people think is the best
way forwards:

1. Convert to using KVM instead - this seems to be more popular now -
when I started using xen I don't think it even existed.

2. Convert my few remaining i686-pae images into amd64 images.

Will KVM boot a 686-pae kernel? If not then it seems I need to do 2
first anyway.


My processor says it supports VT-x although I don't see that in flags -
perhaps because I'm looking in a dom0? So changing to KVM might need new
hardware...


Tim.

Re: Want to contibute to debian

[john doe]

On 9/3/2022 9:44 AM, 5HROUD wrote:

Hi there,
I am a student who develops software application for Linux. Because all my 
project are opensource I wish I could work as a debian developer. Since I love 
using debian and also wanted to contribute to it.
Sent with [Proton Mail](https://proton.me/) secure email.

>

The best way is to read the doc.

--
John Doe

Re: Network bridge and MAC address exposure

[john doe]

On 9/4/2022 8:39 AM, Rand Pritelrohm wrote:

Consider this simple schematic:


 | VM | -> | HOST | -> | GW | -> ISP


Lets say the physical interface name on the 'host' is eth0 and the LAN
subnet is 192.168.0.0.

I want to configure the network on the 'host' in order for the VM to
access the Internet.

Thus I consider 2 scenarios to setup the 'host' network.


1. Bridge using routed subnet:

 ip link add dev br0 type bridge
 ip addr add 192.168.222.1/24 dev br0
 ip link set dev br0 up

 ip tuntap add tap0 mode tap
 ip link set dev tap0 up
 ip link set dev tap0 master br0

 #Then I have to enable routing
 echo '1' > /proc/sys/net/ipv4/ip_forward
 iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE



You are answering your own question, if you masquerade the MAC addr from
the VM will not be seen upstream.



2. Bridge on the same subnet as the LAN:

 ip link add dev br0 type bridge
 ip link set dev br0 up

 ip link set dev eth0 master br0
 ip link set dev eth0 up
 ip addr add 192.168.0.200/24 dev br0
 ip route add default via 192.168.0.1

 ip tuntap add tap0 mode tap
 ip link set dev tap0 up
 ip link set dev tap0 master br0




The MAC addr of the VM will be seen upstream.


For both scenarios the VM is then setup with it's own MAC address and
it's IP on the configured subnet of the bridge.


Here is my question:
 For both scenarios, what is the effectively seen MAC address by the
 GW when the VM access the Internet (host or VM MAC address)?



The best way to answer your question is to get dirty!!! :)

--
John Doe

Re: Network bridge and MAC address exposure

[Jeremy Ardley]

On 4/9/22 2:39 pm, Rand Pritelrohm wrote:

1. Bridge using routed subnet:

 ip link add dev br0 type bridge
 ip addr add 192.168.222.1/24 dev br0
 ip link set dev br0 up

 ip tuntap add tap0 mode tap
 ip link set dev tap0 up
 ip link set dev tap0 master br0

 #Then I have to enable routing
 echo '1' > /proc/sys/net/ipv4/ip_forward
 iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE


2. Bridge on the same subnet as the LAN:

 ip link add dev br0 type bridge
 ip link set dev br0 up

 ip link set dev eth0 master br0
 ip link set dev eth0 up
 ip addr add 192.168.0.200/24 dev br0
 ip route add default via 192.168.0.1

 ip tuntap add tap0 mode tap
 ip link set dev tap0 up
 ip link set dev tap0 master br0


Here is my question:
 For both scenarios, what is the effectively seen MAC address by the
 GW when the VM access the Internet (host or VM MAC address)?


Intuitively (to me at least) NAT/routing (layer 3) replaces the client MAC 
address with the host MAC, while bridging (layer 2) passes the client MAC 
address.

However to be certain, put wireshark on and look at the packet contents in both 
scenarios.


--
Jeremy



OpenPGP_signature
Description: OpenPGP digital signature