[gentoo-user] Netgear AC1750 C7 V2 and IPv6

[Dale]

Howdy,

Starting a new thread.  I got my modem and router all hooked up. 
Networking works except IPv6 isn't quite there yet.  I even got my cell
phone and printer working again, IP network address changed.  I'm almost
certain the modem and my puter has a working IPv6.  Going to give some
info but some numbers may be changed, for obvious reasons since I don't
want some weirdo tracking me down.  Modem first: 



Internet IPv4 Address    74.999.999.2
Internet Gateway IP Address    74.999.999.1
Internet IPv6 Prefix    2602:999:defg:9029::


Since the modem is basically fully automatic, there isn't much info to
share here.  It shows it has IPv6 addresses tho.  Now for the router:


IPv6 Status
WAN
Connection Type:    
DHCPv6
IPv6 Address:    
::
IPv6 Default Gateway:    
::
Primary IPv6 DNS:    
::
Secondary IPv6 DNS:    
::
LAN
IPv6 Address Assign Type:    
SLAAC
IPv6 Address:    
2602:999:defg:9029:d66e:eff:fe42:55cf/64
Link-local Address:    
fe80::d99e:eff:fe42:55cf/64


Now for the puter from ifconfig:


root@fireball / # ifconfig
eth1: flags=4163  mtu 1500
    inet 192.168.0.100  netmask 255.255.255.0  broadcast 192.168.0.255
    inet6 fe80::9dbe:4ba5:f194:2063  prefixlen 64  scopeid 0x20
    inet6 2602:999:defg:9029:2b86:428b:95ee:ba9d  prefixlen 64 
scopeid 0x0
    ether 00:01:53:80:dc:35  txqueuelen 1000  (Ethernet)


I've looked at the manual, googled it for some tips to try and tried
several other things such as rebooting the router, even resetting it and
starting from scratch.  I also updated the firmware, one of the first
things I did but it wasn't to out of date.  It seems the modem is ready
and that the router should either get the info from the modem or just go
straight through and get it from my ISP, I'd suspect getting it from the
modem myself.  It seems for some reason, the router is not getting the
info it needs to make the IPv6 part work.  I'm not finding anything
settings wise that changes this.  It seems it should be pretty much
automatic just like IPv4 does.

I visited a IPv6 test site.  It shows IPv6 not working.  The modem has a
test for connections that makes me think it could be the ISP with the
problem.  It tests the ethernet port, DNS, DSL etc but on the IP test,
it fails and shows this for the details:


Continuity check to 6rd BR    Fail


The other tests passed.  Since that is the modem itself, it seems to me
the failure is upstream, not me this time.  But the modem shows it
received IPv6 info, which is confusing.  I googled the 6rd BR thing and
it even had a nice little picture of how things connect.  If I
understand it correctly, the problem is upstream but I'm not sure. 

Does this look like a upstream problem with my ISP or am I missing some
setting somewhere?  If you need additional info, let me know.  The
changes I made in IP numbers should be the same whether it's the modem,
router or puter.  I tried to match them up and only change enough to
make tracking hard.

Thanks much.

Dale

:-)  :-)

Re: [gentoo-user] Stable Python package changes USE flags with ~amd64

[Franz Fellner]

I assume those emerge commands weren't done on one machine but come from
those two different machines.
This change in USE Flags can't come from that line in
package.accept_keywords.
This is a change in PYTHON_TARGETS in make.conf, package.use or package.env.
Carefully go through those config files/directories, I am sure you will
find the offending line.

Regards
Franz

Am Fr., 3. Jan. 2020 um 11:44 Uhr schrieb Mickaël Bucas :

> Hello
>
> For some time I've been wondering why I had a difference on
> dev-python/olefile-0.46 between 2 machines : one was installed with
> python_targets_python3_7, the other wasn't.
> And I finally pinpointed it to package.accept_keywords containing
> "dev-python/olefile ~amd64" on one of the machines only
>
> At the time of writing, dev-python/olefile-0.46 is the stable version, and
> KEYWORDS contains "amd64" (no tilde) among others.
>
> When package.accept_keywords doesn't contain "dev-python/olefile ~amd64",
> I get :
> emerge -pv1 --verbose-conflicts olefile
> These are the packages that would be merged, in order:
> Calculating dependencies... done!
> [ebuild   R] dev-python/olefile-0.46::gentoo  USE="-doc"
> PYTHON_TARGETS="python2_7 python3_6 (-pypy3) (-python3_7) (-python3_8)" 0
> KiB
> Total: 1 package (1 reinstall), Size of downloads: 0 KiB
>
> => Python 3.7 is disabled
>
> When package.accept_keywords contains "dev-python/olefile ~amd64", I get :
> emerge -pv1 olefile
> These are the packages that would be merged, in order:
> Calculating dependencies... done!
> [ebuild   R] dev-python/olefile-0.46::gentoo  USE="-doc"
> PYTHON_TARGETS="python2_7 python3_6 python3_7* -pypy3 -python3_8" 0 KiB
> Total: 1 package (1 reinstall), Size of downloads: 0 KiB
>
> => Python 3.7 is enabled
>
> It seems really really strange to me for the same version of a stable
> package to be "influenced" by keywording.
> Is it a bug or a feature ?
> Did I do something wrong ?
>
> Thanks
> Best regards
> Mickaël Bucas
>

Re: [gentoo-user] Guidance on using Gentoo to clone a Win 10 system drive

[Michael Jones]

I would be extremely surprised if it activated after not having done so
within an hour.

You can manually trigger activation in the Windows 10 settings menu
somewhere, and get an answer immediately.

You *do* need to make sure the version of Windows is 100% identical to what
was previously installed though, down to the various international version
that have different language packs and misc media features enabled /
disabled.


On Mon, Jan 6, 2020 at 6:39 PM Mark Knecht  wrote:

>
>
> On Mon, Jan 6, 2020 at 5:19 PM Bill Kenworthy  wrote:
> >
> > Hi Mark,
> >  was your old version Win10 PRO" as well? - as far as I know a
> > reinstall will only validate if the hardware as recorded at MS mostly
> > matches and its the same version.  Cloning via dd, then running through
> > the re-validation checks, then making changes in small steps is the only
> > way I have been able to make it work despite what is written in the link
> > below.
> >
> > Also check out:
> >
> https://www.techrepublic.com/article/windows-10-will-microsoft-charge-you-if-you-need-to-reinstall/
> >
> > BillK
> >
>
> Bill,
>It's a great question that I cannot answer with certainty unless I put
> the old drive back in. I thought it was when I did the install this
> afternoon but I wasn't sure.
>
>I'm going to let the machine sit overnight and see if it activates
> automatically. If it doesn't I'll go back to the old drive and if needed
> will do a new reinstall with the right version. If I can get away with this
> path I will. If not I'll go with something like Mick suggested.
>
> thanks,
> Mark
>

Re: [gentoo-user] Guidance on using Gentoo to clone a Win 10 system drive

[Mark Knecht]

On Mon, Jan 6, 2020 at 5:19 PM Bill Kenworthy  wrote:
>
> Hi Mark,
>  was your old version Win10 PRO" as well? - as far as I know a
> reinstall will only validate if the hardware as recorded at MS mostly
> matches and its the same version.  Cloning via dd, then running through
> the re-validation checks, then making changes in small steps is the only
> way I have been able to make it work despite what is written in the link
> below.
>
> Also check out:
>
https://www.techrepublic.com/article/windows-10-will-microsoft-charge-you-if-you-need-to-reinstall/
>
> BillK
>

Bill,
   It's a great question that I cannot answer with certainty unless I put
the old drive back in. I thought it was when I did the install this
afternoon but I wasn't sure.

   I'm going to let the machine sit overnight and see if it activates
automatically. If it doesn't I'll go back to the old drive and if needed
will do a new reinstall with the right version. If I can get away with this
path I will. If not I'll go with something like Mick suggested.

thanks,
Mark

Re: [gentoo-user] Guidance on using Gentoo to clone a Win 10 system drive

[Bill Kenworthy]

Hi Mark,
    was your old version Win10 PRO" as well? - as far as I know a 
reinstall will only validate if the hardware as recorded at MS mostly 
matches and its the same version.  Cloning via dd, then running through 
the re-validation checks, then making changes in small steps is the only 
way I have been able to make it work despite what is written in the link 
below.


Also check out: 
https://www.techrepublic.com/article/windows-10-will-microsoft-charge-you-if-you-need-to-reinstall/


BillK


On 7/1/20 7:37 am, Mark Knecht wrote:

Michael,
   I got Win 10 Pro installed via the M$ tool that creates USB install 
devices. It worked fine. Reading online it seems that if M$ sees the 
new disk as still the same 'hardware' then it's supposed to 
automatically validate and I'd be good to go. so far, after 2 hours it 
hasn't done that but I'll give it awhile and see what happens. As it 
only took an hour I might still try the disk copy path and see if that 
comes up validated as that would also transfer the couple of 
applications I have on the original hard drive.


   Anyway, thanks for the ideas.

Cheers,
Mark

On Mon, Jan 6, 2020 at 3:01 PM Michael Jones > wrote:


You can use the Windows 10 Download Tool (Or similarly named
thing, sorry, I can't find the details of it at this time) to
download an ISO image

Combine that with the rufus program https://rufus.ie/ (I use the
portable one, personally) to create a Windows 10 USB installer stick.

On Mon, Jan 6, 2020 at 2:39 PM Mark Knecht mailto:markkne...@gmail.com>> wrote:

Hi Michael,
   Thanks for the response. Great info.

   The install Win 10 clean sounds wonderful if it works. With
no DVD in this machine it sounds like I should investigate an
install from USB if the machine supports it. It's an Asus
gaming laptop circa 2008 so hopefully that works but I've
never done it on this machine.

Cheers,
Mark

On Mon, Jan 6, 2020 at 12:56 PM Michael Jones
mailto:gen...@jonesmz.com>> wrote:

Generally the way I've handled this situation in the past
is like so (this is written from memory, so expect
gratuitous problems).

On the machine with the drive attached
mbuffer -i /dev/mydrive | xz -e -9 | mbuffer -O hostname:port

On a machine with storage space
mbuffer -I port -o /path/to/storage.xz

To make a backup.


In terms of cloning windows to another harddrive in
general, as long as the destination harddrive is large
enough to fit the original drive without issues, simply
running:

dd if=/dev/original of=/dev/destination
(I prefer dcfldd, personally)

Is enough. Run gparted (the graphical version, for nice
wizards) after, and it'll fixup your partition table for
you to match the new size, and you can re-size any
partitions you have to make them match as well. I do
exactly this all the time and have yet to have a problem.

As for windows 10 licensing, don't trust me on this
blindly, but your license should be tied to the hardware
fingerprint of the laptop. So even installing windows
fresh on your new SSD should result in Windows activating
automatically. In fact, you might want to take this
opportunity to try that out, to get a completely fresh
installation without the decade of old cruft built up by
window's lack of a package manager.

If it doesn't activate as soon as you plug in an ethernet
cable, you can just wipe your SSD and copy your old
installation as discussed already.



On Mon, Jan 6, 2020 at 1:11 PM Mark Knecht
mailto:markkne...@gmail.com>> wrote:

Hi all,
   I haven't been here in a couple of years. IT's
great to see some familiar names posting. Cheers to all.

   I have a laptop running Win 10 with no (working)
DVD/CDROM. For various reasons I want to move from a
10 year old laptop drive to a new SSD and am looking
for guidance on I might do that. Win 10 is properly
licensed but through a weird channel - it was Win 7
that M$ allowed to convert to Win 10 for free and I'm
nervous that if the hard drive died I'd have to
purchase a new license as the free conversion path
likely doesn't exist anymore.

   Both drives are nominally 500GB.

   The older hard drive fdisk info shows:

root@science:~# fdisk --list /dev/sde
Disk /dev/sde: 465.8 GiB, 500107862016 bytes,
976773168 

Re: [gentoo-user] Guidance on using Gentoo to clone a Win 10 system drive

[Mark Knecht]

On Mon, Jan 6, 2020 at 4:38 PM Mick  wrote:
>
> Hi Mark, welcome back!  :-)
>

Hey Mick. Thanks for the welcome.


> This will take for-ever on larger disks as it will be copying all empty
bits
> and bytes.  Instead you may wish to try clonezilla, or partclone.
>
> https://clonezilla.org/
>
> Clonezilla Live will copy the whole disk or selected partitions along with
> their UUIDs, so Win10 should have no idea it was just migrated.  ;-)
>
> You'll need a USB/eSATA caddy to put your new drive in and connect it to
the
> candidate laptop, or fit both drives in your desktop and perform the
cloning
> there.  Here's the step-by-step instructions you asked for:
>
> https://clonezilla.org/clonezilla-live-doc.php
>
Good point about the amount of time. I'll investigate. If the SSD came up
as activated that would be a big win as it hasn't happened on it's own with
the new install. Thanks!

Cheers,
Mark

Re: [gentoo-user] Guidance on using Gentoo to clone a Win 10 system drive

[Mick]

Hi Mark, welcome back!  :-)

On Monday, 6 January 2020 19:55:44 GMT Michael Jones wrote:
> Generally the way I've handled this situation in the past is like so (this
> is written from memory, so expect gratuitous problems).
> 
> On the machine with the drive attached
> mbuffer -i /dev/mydrive | xz -e -9 | mbuffer -O hostname:port
> 
> On a machine with storage space
> mbuffer -I port -o /path/to/storage.xz
> 
> To make a backup.

Useful for creating a compressed backup image over the network, but not for 
cloning.


> In terms of cloning windows to another harddrive in general, as long as the
> destination harddrive is large enough to fit the original drive without
> issues, simply running:
> 
> dd if=/dev/original of=/dev/destination
> (I prefer dcfldd, personally)

This will take for-ever on larger disks as it will be copying all empty bits 
and bytes.  Instead you may wish to try clonezilla, or partclone.

https://clonezilla.org/

Clonezilla Live will copy the whole disk or selected partitions along with 
their UUIDs, so Win10 should have no idea it was just migrated.  ;-)

You'll need a USB/eSATA caddy to put your new drive in and connect it to the 
candidate laptop, or fit both drives in your desktop and perform the cloning 
there.  Here's the step-by-step instructions you asked for:

https://clonezilla.org/clonezilla-live-doc.php

-- 
Regards,
Mick

signature.asc
Description: This is a digitally signed message part.

Re: [gentoo-user] Guidance on using Gentoo to clone a Win 10 system drive

[Mark Knecht]

Michael,
   I got Win 10 Pro installed via the M$ tool that creates USB install
devices. It worked fine. Reading online it seems that if M$ sees the new
disk as still the same 'hardware' then it's supposed to automatically
validate and I'd be good to go. so far, after 2 hours it hasn't done that
but I'll give it awhile and see what happens. As it only took an hour I
might still try the disk copy path and see if that comes up validated as
that would also transfer the couple of applications I have on the original
hard drive.

   Anyway, thanks for the ideas.

Cheers,
Mark

On Mon, Jan 6, 2020 at 3:01 PM Michael Jones  wrote:

> You can use the Windows 10 Download Tool (Or similarly named thing, sorry,
> I can't find the details of it at this time) to download an ISO image
>
> Combine that with the rufus program https://rufus.ie/ (I use the portable
> one, personally) to create a Windows 10 USB installer stick.
>
> On Mon, Jan 6, 2020 at 2:39 PM Mark Knecht  wrote:
>
>> Hi Michael,
>>Thanks for the response. Great info.
>>
>>The install Win 10 clean sounds wonderful if it works. With no DVD in
>> this machine it sounds like I should investigate an install from USB if the
>> machine supports it. It's an Asus gaming laptop circa 2008 so hopefully
>> that works but I've never done it on this machine.
>>
>> Cheers,
>> Mark
>>
>> On Mon, Jan 6, 2020 at 12:56 PM Michael Jones  wrote:
>>
>>> Generally the way I've handled this situation in the past is like so
>>> (this is written from memory, so expect gratuitous problems).
>>>
>>> On the machine with the drive attached
>>> mbuffer -i /dev/mydrive | xz -e -9 | mbuffer -O hostname:port
>>>
>>> On a machine with storage space
>>> mbuffer -I port -o /path/to/storage.xz
>>>
>>> To make a backup.
>>>
>>>
>>> In terms of cloning windows to another harddrive in general, as long as
>>> the destination harddrive is large enough to fit the original drive without
>>> issues, simply running:
>>>
>>> dd if=/dev/original of=/dev/destination
>>> (I prefer dcfldd, personally)
>>>
>>> Is enough. Run gparted (the graphical version, for nice wizards) after,
>>> and it'll fixup your partition table for you to match the new size, and you
>>> can re-size any partitions you have to make them match as well. I do
>>> exactly this all the time and have yet to have a problem.
>>>
>>> As for windows 10 licensing, don't trust me on this blindly, but your
>>> license should be tied to the hardware fingerprint of the laptop. So even
>>> installing windows fresh on your new SSD should result in Windows
>>> activating automatically. In fact, you might want to take this opportunity
>>> to try that out, to get a completely fresh installation without the decade
>>> of old cruft built up by window's lack of a package manager.
>>>
>>> If it doesn't activate as soon as you plug in an ethernet cable, you can
>>> just wipe your SSD and copy your old installation as discussed already.
>>>
>>>
>>>
>>> On Mon, Jan 6, 2020 at 1:11 PM Mark Knecht  wrote:
>>>
 Hi all,
I haven't been here in a couple of years. IT's great to see some
 familiar names posting. Cheers to all.

I have a laptop running Win 10 with no (working) DVD/CDROM. For
 various reasons I want to move from a 10 year old laptop drive to a new SSD
 and am looking for guidance on I might do that. Win 10 is properly licensed
 but through a weird channel - it was Win 7 that M$ allowed to convert to
 Win 10 for free and I'm nervous that if the hard drive died I'd have to
 purchase a new license as the free conversion path likely doesn't exist
 anymore.

Both drives are nominally 500GB.

The older hard drive fdisk info shows:

 root@science:~# fdisk --list /dev/sde
 Disk /dev/sde: 465.8 GiB, 500107862016 bytes, 976773168 sectors
 Disk model: ASM1053E
 Units: sectors of 1 * 512 = 512 bytes
 Sector size (logical/physical): 512 bytes / 512 bytes
 I/O size (minimum/optimal): 512 bytes / 512 bytes
 Disklabel type: dos
 Disk identifier: 0xe0c5913d

 Device Boot Start   End   Sectors   Size Id Type
 /dev/sde1  63  45062324  45062262  21.5G 1c Hidden W95
 FAT32 (LBA)
 /dev/sde2  * 45062325 288063133 243000809 115.9G  7 HPFS/NTFS/exFAT
 /dev/sde3   288063488 289247231   1183744   578M 27 Hidden NTFS
 WinRE
 /dev/sde4   289249254 976768064 687518811 327.9G fd Linux raid
 autodetect
 root@science:~#

 The Linux RAID autodetect is from running Gentoo at some earlier time
 and probably doesn't need to be copied. I'm not at all sure what /dev/sde3
 is or whether it's required to make M$ happy.

The new SSD is unused and shows:

 root@science:~# fdisk --list /dev/sdf
 Disk /dev/sdf: 465.8 GiB, 500107862016 bytes, 976773168 sectors
 Disk model: ASM1053E
 Units: sectors of 1 * 512 = 512 bytes
 Sector size (logical/physical): 

Re: [gentoo-user] Guidance on using Gentoo to clone a Win 10 system drive

[Michael Jones]

You can use the Windows 10 Download Tool (Or similarly named thing, sorry,
I can't find the details of it at this time) to download an ISO image

Combine that with the rufus program https://rufus.ie/ (I use the portable
one, personally) to create a Windows 10 USB installer stick.

On Mon, Jan 6, 2020 at 2:39 PM Mark Knecht  wrote:

> Hi Michael,
>Thanks for the response. Great info.
>
>The install Win 10 clean sounds wonderful if it works. With no DVD in
> this machine it sounds like I should investigate an install from USB if the
> machine supports it. It's an Asus gaming laptop circa 2008 so hopefully
> that works but I've never done it on this machine.
>
> Cheers,
> Mark
>
> On Mon, Jan 6, 2020 at 12:56 PM Michael Jones  wrote:
>
>> Generally the way I've handled this situation in the past is like so
>> (this is written from memory, so expect gratuitous problems).
>>
>> On the machine with the drive attached
>> mbuffer -i /dev/mydrive | xz -e -9 | mbuffer -O hostname:port
>>
>> On a machine with storage space
>> mbuffer -I port -o /path/to/storage.xz
>>
>> To make a backup.
>>
>>
>> In terms of cloning windows to another harddrive in general, as long as
>> the destination harddrive is large enough to fit the original drive without
>> issues, simply running:
>>
>> dd if=/dev/original of=/dev/destination
>> (I prefer dcfldd, personally)
>>
>> Is enough. Run gparted (the graphical version, for nice wizards) after,
>> and it'll fixup your partition table for you to match the new size, and you
>> can re-size any partitions you have to make them match as well. I do
>> exactly this all the time and have yet to have a problem.
>>
>> As for windows 10 licensing, don't trust me on this blindly, but your
>> license should be tied to the hardware fingerprint of the laptop. So even
>> installing windows fresh on your new SSD should result in Windows
>> activating automatically. In fact, you might want to take this opportunity
>> to try that out, to get a completely fresh installation without the decade
>> of old cruft built up by window's lack of a package manager.
>>
>> If it doesn't activate as soon as you plug in an ethernet cable, you can
>> just wipe your SSD and copy your old installation as discussed already.
>>
>>
>>
>> On Mon, Jan 6, 2020 at 1:11 PM Mark Knecht  wrote:
>>
>>> Hi all,
>>>I haven't been here in a couple of years. IT's great to see some
>>> familiar names posting. Cheers to all.
>>>
>>>I have a laptop running Win 10 with no (working) DVD/CDROM. For
>>> various reasons I want to move from a 10 year old laptop drive to a new SSD
>>> and am looking for guidance on I might do that. Win 10 is properly licensed
>>> but through a weird channel - it was Win 7 that M$ allowed to convert to
>>> Win 10 for free and I'm nervous that if the hard drive died I'd have to
>>> purchase a new license as the free conversion path likely doesn't exist
>>> anymore.
>>>
>>>Both drives are nominally 500GB.
>>>
>>>The older hard drive fdisk info shows:
>>>
>>> root@science:~# fdisk --list /dev/sde
>>> Disk /dev/sde: 465.8 GiB, 500107862016 bytes, 976773168 sectors
>>> Disk model: ASM1053E
>>> Units: sectors of 1 * 512 = 512 bytes
>>> Sector size (logical/physical): 512 bytes / 512 bytes
>>> I/O size (minimum/optimal): 512 bytes / 512 bytes
>>> Disklabel type: dos
>>> Disk identifier: 0xe0c5913d
>>>
>>> Device Boot Start   End   Sectors   Size Id Type
>>> /dev/sde1  63  45062324  45062262  21.5G 1c Hidden W95 FAT32
>>> (LBA)
>>> /dev/sde2  * 45062325 288063133 243000809 115.9G  7 HPFS/NTFS/exFAT
>>> /dev/sde3   288063488 289247231   1183744   578M 27 Hidden NTFS
>>> WinRE
>>> /dev/sde4   289249254 976768064 687518811 327.9G fd Linux raid
>>> autodetect
>>> root@science:~#
>>>
>>> The Linux RAID autodetect is from running Gentoo at some earlier time
>>> and probably doesn't need to be copied. I'm not at all sure what /dev/sde3
>>> is or whether it's required to make M$ happy.
>>>
>>>The new SSD is unused and shows:
>>>
>>> root@science:~# fdisk --list /dev/sdf
>>> Disk /dev/sdf: 465.8 GiB, 500107862016 bytes, 976773168 sectors
>>> Disk model: ASM1053E
>>> Units: sectors of 1 * 512 = 512 bytes
>>> Sector size (logical/physical): 512 bytes / 4096 bytes
>>> I/O size (minimum/optimal): 4096 bytes / 4096 bytes
>>> root@science:~#
>>>
>>>The appear to have the same sector count and overall size.
>>>
>>>I can make a 1TB drive available in my big machine and work over USB
>>> (which is what I'm doing to get the info above) but I'm unclear how much of
>>> this can be done automatically and how much I might need to do by hand.
>>>
>>>As long as I don't hurt the old drive I can put data on the SSD
>>> multiple times to get through the process in case I have trouble.
>>>
>>>Does anyone have experience with this sort of issue and can you point
>>> me toward some instructions I might try?
>>>
>>> Thanks,
>>> Mark
>>>
>>>
>>>

Re: [gentoo-user] Guidance on using Gentoo to clone a Win 10 system drive

[Mark Knecht]

Hi Michael,
   Thanks for the response. Great info.

   The install Win 10 clean sounds wonderful if it works. With no DVD in
this machine it sounds like I should investigate an install from USB if the
machine supports it. It's an Asus gaming laptop circa 2008 so hopefully
that works but I've never done it on this machine.

Cheers,
Mark

On Mon, Jan 6, 2020 at 12:56 PM Michael Jones  wrote:

> Generally the way I've handled this situation in the past is like so (this
> is written from memory, so expect gratuitous problems).
>
> On the machine with the drive attached
> mbuffer -i /dev/mydrive | xz -e -9 | mbuffer -O hostname:port
>
> On a machine with storage space
> mbuffer -I port -o /path/to/storage.xz
>
> To make a backup.
>
>
> In terms of cloning windows to another harddrive in general, as long as
> the destination harddrive is large enough to fit the original drive without
> issues, simply running:
>
> dd if=/dev/original of=/dev/destination
> (I prefer dcfldd, personally)
>
> Is enough. Run gparted (the graphical version, for nice wizards) after,
> and it'll fixup your partition table for you to match the new size, and you
> can re-size any partitions you have to make them match as well. I do
> exactly this all the time and have yet to have a problem.
>
> As for windows 10 licensing, don't trust me on this blindly, but your
> license should be tied to the hardware fingerprint of the laptop. So even
> installing windows fresh on your new SSD should result in Windows
> activating automatically. In fact, you might want to take this opportunity
> to try that out, to get a completely fresh installation without the decade
> of old cruft built up by window's lack of a package manager.
>
> If it doesn't activate as soon as you plug in an ethernet cable, you can
> just wipe your SSD and copy your old installation as discussed already.
>
>
>
> On Mon, Jan 6, 2020 at 1:11 PM Mark Knecht  wrote:
>
>> Hi all,
>>I haven't been here in a couple of years. IT's great to see some
>> familiar names posting. Cheers to all.
>>
>>I have a laptop running Win 10 with no (working) DVD/CDROM. For
>> various reasons I want to move from a 10 year old laptop drive to a new SSD
>> and am looking for guidance on I might do that. Win 10 is properly licensed
>> but through a weird channel - it was Win 7 that M$ allowed to convert to
>> Win 10 for free and I'm nervous that if the hard drive died I'd have to
>> purchase a new license as the free conversion path likely doesn't exist
>> anymore.
>>
>>Both drives are nominally 500GB.
>>
>>The older hard drive fdisk info shows:
>>
>> root@science:~# fdisk --list /dev/sde
>> Disk /dev/sde: 465.8 GiB, 500107862016 bytes, 976773168 sectors
>> Disk model: ASM1053E
>> Units: sectors of 1 * 512 = 512 bytes
>> Sector size (logical/physical): 512 bytes / 512 bytes
>> I/O size (minimum/optimal): 512 bytes / 512 bytes
>> Disklabel type: dos
>> Disk identifier: 0xe0c5913d
>>
>> Device Boot Start   End   Sectors   Size Id Type
>> /dev/sde1  63  45062324  45062262  21.5G 1c Hidden W95 FAT32
>> (LBA)
>> /dev/sde2  * 45062325 288063133 243000809 115.9G  7 HPFS/NTFS/exFAT
>> /dev/sde3   288063488 289247231   1183744   578M 27 Hidden NTFS WinRE
>> /dev/sde4   289249254 976768064 687518811 327.9G fd Linux raid
>> autodetect
>> root@science:~#
>>
>> The Linux RAID autodetect is from running Gentoo at some earlier time and
>> probably doesn't need to be copied. I'm not at all sure what /dev/sde3 is
>> or whether it's required to make M$ happy.
>>
>>The new SSD is unused and shows:
>>
>> root@science:~# fdisk --list /dev/sdf
>> Disk /dev/sdf: 465.8 GiB, 500107862016 bytes, 976773168 sectors
>> Disk model: ASM1053E
>> Units: sectors of 1 * 512 = 512 bytes
>> Sector size (logical/physical): 512 bytes / 4096 bytes
>> I/O size (minimum/optimal): 4096 bytes / 4096 bytes
>> root@science:~#
>>
>>The appear to have the same sector count and overall size.
>>
>>I can make a 1TB drive available in my big machine and work over USB
>> (which is what I'm doing to get the info above) but I'm unclear how much of
>> this can be done automatically and how much I might need to do by hand.
>>
>>As long as I don't hurt the old drive I can put data on the SSD
>> multiple times to get through the process in case I have trouble.
>>
>>Does anyone have experience with this sort of issue and can you point
>> me toward some instructions I might try?
>>
>> Thanks,
>> Mark
>>
>>
>>

Re: [gentoo-user] Guidance on using Gentoo to clone a Win 10 system drive

[Michael Jones]

Generally the way I've handled this situation in the past is like so (this
is written from memory, so expect gratuitous problems).

On the machine with the drive attached
mbuffer -i /dev/mydrive | xz -e -9 | mbuffer -O hostname:port

On a machine with storage space
mbuffer -I port -o /path/to/storage.xz

To make a backup.


In terms of cloning windows to another harddrive in general, as long as the
destination harddrive is large enough to fit the original drive without
issues, simply running:

dd if=/dev/original of=/dev/destination
(I prefer dcfldd, personally)

Is enough. Run gparted (the graphical version, for nice wizards) after, and
it'll fixup your partition table for you to match the new size, and you can
re-size any partitions you have to make them match as well. I do exactly
this all the time and have yet to have a problem.

As for windows 10 licensing, don't trust me on this blindly, but your
license should be tied to the hardware fingerprint of the laptop. So even
installing windows fresh on your new SSD should result in Windows
activating automatically. In fact, you might want to take this opportunity
to try that out, to get a completely fresh installation without the decade
of old cruft built up by window's lack of a package manager.

If it doesn't activate as soon as you plug in an ethernet cable, you can
just wipe your SSD and copy your old installation as discussed already.



On Mon, Jan 6, 2020 at 1:11 PM Mark Knecht  wrote:

> Hi all,
>I haven't been here in a couple of years. IT's great to see some
> familiar names posting. Cheers to all.
>
>I have a laptop running Win 10 with no (working) DVD/CDROM. For various
> reasons I want to move from a 10 year old laptop drive to a new SSD and am
> looking for guidance on I might do that. Win 10 is properly licensed but
> through a weird channel - it was Win 7 that M$ allowed to convert to Win 10
> for free and I'm nervous that if the hard drive died I'd have to purchase a
> new license as the free conversion path likely doesn't exist anymore.
>
>Both drives are nominally 500GB.
>
>The older hard drive fdisk info shows:
>
> root@science:~# fdisk --list /dev/sde
> Disk /dev/sde: 465.8 GiB, 500107862016 bytes, 976773168 sectors
> Disk model: ASM1053E
> Units: sectors of 1 * 512 = 512 bytes
> Sector size (logical/physical): 512 bytes / 512 bytes
> I/O size (minimum/optimal): 512 bytes / 512 bytes
> Disklabel type: dos
> Disk identifier: 0xe0c5913d
>
> Device Boot Start   End   Sectors   Size Id Type
> /dev/sde1  63  45062324  45062262  21.5G 1c Hidden W95 FAT32
> (LBA)
> /dev/sde2  * 45062325 288063133 243000809 115.9G  7 HPFS/NTFS/exFAT
> /dev/sde3   288063488 289247231   1183744   578M 27 Hidden NTFS WinRE
> /dev/sde4   289249254 976768064 687518811 327.9G fd Linux raid
> autodetect
> root@science:~#
>
> The Linux RAID autodetect is from running Gentoo at some earlier time and
> probably doesn't need to be copied. I'm not at all sure what /dev/sde3 is
> or whether it's required to make M$ happy.
>
>The new SSD is unused and shows:
>
> root@science:~# fdisk --list /dev/sdf
> Disk /dev/sdf: 465.8 GiB, 500107862016 bytes, 976773168 sectors
> Disk model: ASM1053E
> Units: sectors of 1 * 512 = 512 bytes
> Sector size (logical/physical): 512 bytes / 4096 bytes
> I/O size (minimum/optimal): 4096 bytes / 4096 bytes
> root@science:~#
>
>The appear to have the same sector count and overall size.
>
>I can make a 1TB drive available in my big machine and work over USB
> (which is what I'm doing to get the info above) but I'm unclear how much of
> this can be done automatically and how much I might need to do by hand.
>
>As long as I don't hurt the old drive I can put data on the SSD
> multiple times to get through the process in case I have trouble.
>
>Does anyone have experience with this sort of issue and can you point
> me toward some instructions I might try?
>
> Thanks,
> Mark
>
>
>

[gentoo-user] Guidance on using Gentoo to clone a Win 10 system drive

[Mark Knecht]

Hi all,
   I haven't been here in a couple of years. IT's great to see some
familiar names posting. Cheers to all.

   I have a laptop running Win 10 with no (working) DVD/CDROM. For various
reasons I want to move from a 10 year old laptop drive to a new SSD and am
looking for guidance on I might do that. Win 10 is properly licensed but
through a weird channel - it was Win 7 that M$ allowed to convert to Win 10
for free and I'm nervous that if the hard drive died I'd have to purchase a
new license as the free conversion path likely doesn't exist anymore.

   Both drives are nominally 500GB.

   The older hard drive fdisk info shows:

root@science:~# fdisk --list /dev/sde
Disk /dev/sde: 465.8 GiB, 500107862016 bytes, 976773168 sectors
Disk model: ASM1053E
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0xe0c5913d

Device Boot Start   End   Sectors   Size Id Type
/dev/sde1  63  45062324  45062262  21.5G 1c Hidden W95 FAT32
(LBA)
/dev/sde2  * 45062325 288063133 243000809 115.9G  7 HPFS/NTFS/exFAT
/dev/sde3   288063488 289247231   1183744   578M 27 Hidden NTFS WinRE
/dev/sde4   289249254 976768064 687518811 327.9G fd Linux raid
autodetect
root@science:~#

The Linux RAID autodetect is from running Gentoo at some earlier time and
probably doesn't need to be copied. I'm not at all sure what /dev/sde3 is
or whether it's required to make M$ happy.

   The new SSD is unused and shows:

root@science:~# fdisk --list /dev/sdf
Disk /dev/sdf: 465.8 GiB, 500107862016 bytes, 976773168 sectors
Disk model: ASM1053E
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
root@science:~#

   The appear to have the same sector count and overall size.

   I can make a 1TB drive available in my big machine and work over USB
(which is what I'm doing to get the info above) but I'm unclear how much of
this can be done automatically and how much I might need to do by hand.

   As long as I don't hurt the old drive I can put data on the SSD multiple
times to get through the process in case I have trouble.

   Does anyone have experience with this sort of issue and can you point me
toward some instructions I might try?

Thanks,
Mark

Re: [gentoo-user] External hard drive and idle activity

[Mick]

On Monday, 6 January 2020 13:53:41 GMT Rich Freeman wrote:
> On Mon, Jan 6, 2020 at 8:25 AM Mick  wrote:
> > If they are used as normal PC drives for regular writing
> > of data, or with back up commands which use rsync, cp, etc. then the disk
> > will fail much sooner than expected because of repeated multiple areas
> > being deleted, before each smaller write.  I recall reading about how
> > short the life of SMR drives was shown to be when used in NAS devices -
> > check google or youtube if you're interested in the specifics.
> 
> Can you give a link - I'm not finding anything, and I'm a bit dubious
> of this claim, because they still are just hard drives.  These aren't
> SSDs and hard drives should not have any kind of erasure limit.

This (random) link strongly recommends against usage in NAS, but gives no 
reliability data:

https://www.storagereview.com/seagate_archive_hdd_review_8tb

This is a youtube video where someone was comparing SMR failures on a NAS: 

https://www.youtube.com/watch?v=CR_bfbOTY1o


> Now, an SMR used for random writes is going to be a REALLY busy drive,
> so I could see the drive being subject to a lot more wear and tear.
> I'm just not aware of any kind of serious study.  And of course any
> particular model of hard drive can have reliability issues (just look
> up the various reliability studies).

Right, I haven't seen any lab reliability studies published.  I would think 
more information could be sourced in IRC/ML where datacenter sysadmins hide to 
compare their ... hardware.  :-)

Reading another random link it seems Dale's 8TB SMR drive has a 20GB 
conventional PMR platter/area in it to catch and cache any small writes.  The 
firmware will subsequently transfer the cached data on the SMR area of the 
drive in due course, after it deletes the requisite adjacent overlapping 
tracks.  This means up to 20GB of initial writes will be normal, dropping to 
lower speeds thereafter as the PMR cache needs to be flushed:

https://www.ixsystems.com/community/threads/smr-hard-drives-do-you-think-they-are-proper-nas-drives.35805/

If this is so, it explains Dale's observation of a hyperactive disk, well 
after it was dismounted.  Its firmware's been busy!

[snip ...]

> Granted, I don't rewrite it often but unless zfs is
> SMR-aware it is still going to be writing lots of modest-sized files
> as the original files get chunked up and distributed across the nodes.
> On the disk lizardfs data just looks like a browser cache, with
> everything in numbered files about 60MB in size in my case.  The files
> also appear to turn over a bit during rebalancing.

I would think bit flipping between the 20GB PMR cache and the 8TB SMR tracks 
represents an increased risk, vis A vis a single-step data transfer.  Data 
scrubbing well after the write has completed and committed to the SMR tracks 
would reveal any anomalies.

What would seriously mess things up is creating a raid with mixed PMR and SMR 
disks and running big (bigger than the internal cache) data writes.  Some PMR 
disks will complete well before the SMR.  I/O blocking and timeouts could 
ensue and the applications performing the writing could hang/fail.

Anyway, write once - read often, fits well the use case for these disks.  They 
should be right at home for long term video and media storage.
-- 
Regards,
Mick

signature.asc
Description: This is a digitally signed message part.

Re: [gentoo-user] External hard drive and idle activity

[Rich Freeman]

On Mon, Jan 6, 2020 at 10:05 AM Dale  wrote:
>
> The drive I have is likely
> done on the drive itself, device managed, which is good for me.

Really the ideal situation are the Host Aware drives.  I have no idea
what percentage of the markets they make.  They fall back to being
device managed if the host doesn't do anything to manage them.

Your drive is device managed.  See the link I posted earlier to the mfr info.

> I've been noticing that too.  Only bad thing is, I can't always tell
> what is in the enclosure.  Sometimes the info is given but sometimes
> not.  I've also seen a few people complain that what they got was not
> the model of drive they thought.

Yeah, there are no guarantees as to what you'll get if you go the
shucking route.  If you absolutely need a certain amount of cache or a
red firmware then you're just going to have to pay double to get that
guarantee.

For my application I'd definitely prefer the red firmware, but it
isn't really the end of the world if the drive takes a few seconds to
timeout on that one failure every 5 years.  I'm not going to pay
double just to guarantee a particular model.  If it were $20 more that
would be another matter, but we're talking $180 vs $350 here.

The other gotcha is that if you want to do a warranty replacement at
some point you're going to have to put it back in the enclosure to do
so.  That means hanging onto enclosures, and is of course a bit of a
pain besides.  Again, with a 50% reduction in cost you'd need to have
a lot of drive failures to be worth worrying about, especially since I
believe drive warranties are getting shorter anyway.

If you're in a typical enterprise situation then you're going to want
to just buy the bare drives with the standard warranty/etc.  If you
buy in bulk chances are you're getting a discount anyway (and then
maybe you can get them in caddies or whatever).  The Best Buy deals
are sporadic anyway and limited in quantity - you could never run a
data center that way.  That's why they're priced the way they are...

-- 
Rich

[gentoo-user] Re: External hard drive and idle activity

[Grant Edwards]

On 2020-01-06, Dale  wrote:

> Finally got around to rebooting.  It was a experience for sure.  X
> wouldn't come up, the plasma thingy doing its thing.  Jeepers.  No
> wonder I hate rebooting.

I try to remember to reboot all of my machines once a month or so when
I've got some spare time (especially after significant updates).
Otherwise, something will demand/cause a reboot in the middle of
something urgent and then you know what happens...

-- 
Grant Edwards   grant.b.edwardsYow! for ARTIFICIAL
  at   FLAVORING!!
  gmail.com

Re: [gentoo-user] External hard drive and idle activity

[Dale]

Rich Freeman wrote:
> On Mon, Jan 6, 2020 at 9:18 AM Dale  wrote:
>> Rich Freeman wrote:
>>> On Mon, Jan 6, 2020 at 8:25 AM Mick  wrote:
 If they are used as normal PC drives for regular writing
 of data, or with back up commands which use rsync, cp, etc. then the disk 
 will
 fail much sooner than expected because of repeated multiple areas being
 deleted, before each smaller write.  I recall reading about how short the 
 life
 of SMR drives was shown to be when used in NAS devices - check google or
 youtube if you're interested in the specifics.
>>> Can you give a link - I'm not finding anything, and I'm a bit dubious
>>> of this claim, because they still are just hard drives.  These aren't
>>> SSDs and hard drives should not have any kind of erasure limit.
>>>
>>> Now, an SMR used for random writes is going to be a REALLY busy drive,
>>> so I could see the drive being subject to a lot more wear and tear.
>>> I'm just not aware of any kind of serious study.  And of course any
>>> particular model of hard drive can have reliability issues (just look
>>> up the various reliability studies).
>>>
>> I ran up on this article however, it is a short time frame.  Still might
>> be a interesting read tho.
>>
>> https://blogs.dropbox.com/tech/2019/07/smr-what-we-learned-in-our-first-year/
> That article makes no mention of reliability issues with SMR.  In
> fact, they mention that they want 40% of their storage to be on SMR by
> now.  Clearly they wouldn't be doing that if the drives failed
> frequently.
>
> Note that they did modify their software to have write patterns
> suitable for SMR.  That is the key here.  You absolutely have to
> engineer your application to be suitable for SMR, or only choose SMR
> if your application is already suitable.  You can't just expect these
> drives to perform remotely acceptably if you just throw random writes
> at them.

True but they likely have the drives that have it handled in software,
host managed I think they call it.  Another article I read was talking
about three different approaches to SMR.  The drive I have is likely
done on the drive itself, device managed, which is good for me.  The
ones in the article appear to manage the data transfer in software.  I
noticed they also use SSDs as sort of a temporary storage, if I
understood that correctly.  I think they did that to speed things up a bit.


>> I'm still a bit curious and somewhat untrusting of those things tho.
>> Regular hard drives go bad often enough as it is.  We don't need some
>> fancy unknown thing inserted just to add more issues.  Sort of reminds
>> me of the init thingy.  Each thing added is another failure point.
> Obviously they're relatively new, but they seem reliable enough.
> They're just not suitable for general purpose use.
>
>> I'm going to test my ebay skills and see if I can find some non-SMR
>> drives.  It sounds like some require some research to know if they are
>> or not.  :/
> That's pretty simple.  Find a drive that looks reasonable
> price/capacity/etc-wise.  Then just google the model number to confirm
> it isn't SMR.
>
> If you're in the US though you're probably best off shucking drives
> from Best Buy these days.  A drive that costs $350 as a bare drive
> will get sold for $180 in a USB enclosure.  I think it is just market
> segmentation.  They want to get top dollar from enterprise users, and
> they aren't going to be shucking drives from Best Buy bought on "limit
> 1 item per customer" sales.  By shucking I'm getting 12TB red drives
> for less than the cost of a 6TB green drive.  Just be aware that if
> your PSU is old you'll need to tape over some of the SATA power pins.
> New PSUs - even cheap ones - haven't given me any trouble.
>
> I'm sure there are more up-to-date guides as these days the drives are
> 12TB, but here is the gist of it:
> https://www.reddit.com/r/DataHoarder/comments/7fx0i0/wd_easystore_8tb_compendium/
>
> If you aren't in the US I have no idea whether equivalent deals are
> available.  That subreddit is a good place to go for info on cheap
> hard drives though.
>


I've been noticing that too.  Only bad thing is, I can't always tell
what is in the enclosure.  Sometimes the info is given but sometimes
not.  I've also seen a few people complain that what they got was not
the model of drive they thought. 

I suspect one can get a adapter for that P/S connector.  I can't recall
when I got the P/S I currently have but it is a few years old.  I think
it's a ThermalTake or something like that.  I got to overclockers forum
where they list good ones.  I'm almost certain it has standard
connectors which may be a problem.  I've read about having to cover up a
pin or something but never seen one in person. 

This is a educational thread. I didn't even know SMR was a thing until
this thread came along. 

Dale

:-)  :-) 

Re: [gentoo-user] External hard drive and idle activity

[Rich Freeman]

On Mon, Jan 6, 2020 at 9:18 AM Dale  wrote:
>
> Rich Freeman wrote:
> > On Mon, Jan 6, 2020 at 8:25 AM Mick  wrote:
> >> If they are used as normal PC drives for regular writing
> >> of data, or with back up commands which use rsync, cp, etc. then the disk 
> >> will
> >> fail much sooner than expected because of repeated multiple areas being
> >> deleted, before each smaller write.  I recall reading about how short the 
> >> life
> >> of SMR drives was shown to be when used in NAS devices - check google or
> >> youtube if you're interested in the specifics.
> > Can you give a link - I'm not finding anything, and I'm a bit dubious
> > of this claim, because they still are just hard drives.  These aren't
> > SSDs and hard drives should not have any kind of erasure limit.
> >
> > Now, an SMR used for random writes is going to be a REALLY busy drive,
> > so I could see the drive being subject to a lot more wear and tear.
> > I'm just not aware of any kind of serious study.  And of course any
> > particular model of hard drive can have reliability issues (just look
> > up the various reliability studies).
> >
>
> I ran up on this article however, it is a short time frame.  Still might
> be a interesting read tho.
>
> https://blogs.dropbox.com/tech/2019/07/smr-what-we-learned-in-our-first-year/

That article makes no mention of reliability issues with SMR.  In
fact, they mention that they want 40% of their storage to be on SMR by
now.  Clearly they wouldn't be doing that if the drives failed
frequently.

Note that they did modify their software to have write patterns
suitable for SMR.  That is the key here.  You absolutely have to
engineer your application to be suitable for SMR, or only choose SMR
if your application is already suitable.  You can't just expect these
drives to perform remotely acceptably if you just throw random writes
at them.

> I'm still a bit curious and somewhat untrusting of those things tho.
> Regular hard drives go bad often enough as it is.  We don't need some
> fancy unknown thing inserted just to add more issues.  Sort of reminds
> me of the init thingy.  Each thing added is another failure point.

Obviously they're relatively new, but they seem reliable enough.
They're just not suitable for general purpose use.

> I'm going to test my ebay skills and see if I can find some non-SMR
> drives.  It sounds like some require some research to know if they are
> or not.  :/

That's pretty simple.  Find a drive that looks reasonable
price/capacity/etc-wise.  Then just google the model number to confirm
it isn't SMR.

If you're in the US though you're probably best off shucking drives
from Best Buy these days.  A drive that costs $350 as a bare drive
will get sold for $180 in a USB enclosure.  I think it is just market
segmentation.  They want to get top dollar from enterprise users, and
they aren't going to be shucking drives from Best Buy bought on "limit
1 item per customer" sales.  By shucking I'm getting 12TB red drives
for less than the cost of a 6TB green drive.  Just be aware that if
your PSU is old you'll need to tape over some of the SATA power pins.
New PSUs - even cheap ones - haven't given me any trouble.

I'm sure there are more up-to-date guides as these days the drives are
12TB, but here is the gist of it:
https://www.reddit.com/r/DataHoarder/comments/7fx0i0/wd_easystore_8tb_compendium/

If you aren't in the US I have no idea whether equivalent deals are
available.  That subreddit is a good place to go for info on cheap
hard drives though.

-- 
Rich

Re: [gentoo-user] External hard drive and idle activity

[Dale]

Rich Freeman wrote:
> On Mon, Jan 6, 2020 at 8:25 AM Mick  wrote:
>> If they are used as normal PC drives for regular writing
>> of data, or with back up commands which use rsync, cp, etc. then the disk 
>> will
>> fail much sooner than expected because of repeated multiple areas being
>> deleted, before each smaller write.  I recall reading about how short the 
>> life
>> of SMR drives was shown to be when used in NAS devices - check google or
>> youtube if you're interested in the specifics.
> Can you give a link - I'm not finding anything, and I'm a bit dubious
> of this claim, because they still are just hard drives.  These aren't
> SSDs and hard drives should not have any kind of erasure limit.
>
> Now, an SMR used for random writes is going to be a REALLY busy drive,
> so I could see the drive being subject to a lot more wear and tear.
> I'm just not aware of any kind of serious study.  And of course any
> particular model of hard drive can have reliability issues (just look
> up the various reliability studies).
>

I ran up on this article however, it is a short time frame.  Still might
be a interesting read tho.

https://blogs.dropbox.com/tech/2019/07/smr-what-we-learned-in-our-first-year/

I'm still a bit curious and somewhat untrusting of those things tho. 
Regular hard drives go bad often enough as it is.  We don't need some
fancy unknown thing inserted just to add more issues.  Sort of reminds
me of the init thingy.  Each thing added is another failure point. 

I'm going to test my ebay skills and see if I can find some non-SMR
drives.  It sounds like some require some research to know if they are
or not.  :/

Dale

:-)  :-) 

Re: [gentoo-user] qtwebkit problem (solved)

[Philip Webb]

200105 Jack wrote:
> On 1/5/20 5:01 AM, Philip Webb wrote:
>> In order to update to the latest stable PyQt5-5.13.2 ,
>> I was required to remerge qtwebkit-5.212.0_pre20190629 ;
>> Towards the end of the latter compile, I suddenly found myself
>> looking at a completely dark screen : the whole system had crashed.
> There have been other good comments, but I'd ask what are your --jobs 
> and --load-average settings?  Does it crash using -j1 ?
> I realize this may be moot, since you have already completed the emerge
> and may not want to risk another crash, but it's a thought.

Thanks for this + the other comments.

I'm not interested enough to make any further tests.
I doubt if the problem is hardware, which is  4 yr old  & very reliable ;
the usual rule-of-thumb is 'random = hardware, predictable = software'
& this phenomenon was predictable after the 1st occurence.

Since the previous emerge(s) of this version didn't fail,
the cause has to be some extra demand imposed by the new requirements
eg of the latest PyQt5.  This seems quite likely :
Python + Qt both seem to be projects with little discipline or care,
which keep on bloating & demanding more-more from users.
The Python 2/3 fiasco is a big pain in the neck
& whenever Qt needs updating, you have to delete everything first
before re-installing it, otherwise Portage refuses to handle it.

Back in the days when software had to work on an XT,
programmers had to exercise rigorous self-discipline ;
these days of unlimited memory/storage/processing have made them lazy.
On Gentoo, we get all the advantages of controlling our own systems,
but the downside is that we have to attend to some problems
which are handled by the devs on binary distros.

so back to real life ... (grin)

-- 
,,
SUPPORT ___//___,   Philip Webb
ELECTRIC   /] [] [] [] [] []|   Cities Centre, University of Toronto
TRANSIT`-O--O---'   purslowatchassdotutorontodotca

Re: [gentoo-user] External hard drive and idle activity

[Rich Freeman]

On Mon, Jan 6, 2020 at 8:25 AM Mick  wrote:
>
> If they are used as normal PC drives for regular writing
> of data, or with back up commands which use rsync, cp, etc. then the disk will
> fail much sooner than expected because of repeated multiple areas being
> deleted, before each smaller write.  I recall reading about how short the life
> of SMR drives was shown to be when used in NAS devices - check google or
> youtube if you're interested in the specifics.

Can you give a link - I'm not finding anything, and I'm a bit dubious
of this claim, because they still are just hard drives.  These aren't
SSDs and hard drives should not have any kind of erasure limit.

Now, an SMR used for random writes is going to be a REALLY busy drive,
so I could see the drive being subject to a lot more wear and tear.
I'm just not aware of any kind of serious study.  And of course any
particular model of hard drive can have reliability issues (just look
up the various reliability studies).

> Personally, I would only use such a drive for 'keepers'.  Say, films I intend
> to write once and watch many times, ripped music albums, family photos, etc.
> For OS files and other temporary backups I would use a normal PC drive.

Certainly I would never use an SMR for an OS or /home.  Backups should
be fine, as long as you're using a sequential backup file format.
tar/duplicity should be fine.  Dar is probably fine but I'd need to
check (I think it just writes the index to the end, so the seeking
issues are on reading and not writing).  Even zip/etc is going to be
fine.  What is going to be a problem is anything that just replicates
the original data as all the separate files/directories that exist on
the original drive, like rsync/rsnapshot/etc.  Those formats are of
course attractive because the backup is just a replica of the
original, but they involve random writes.  Most formats that just
create a bunch of files named archive-001.foo that need a special
command to restore are going to be fine.

I personally haven't encountered a need to consider an SMR drive as
you can shuck those 12TB Easystore drives for something like $180 on
sale, at least in the US.  Those are just standard drives (often with
red firmware).  I couldn't even use them for my multimedia as I'm
storing that stuff on lizardfs right now and that breaks everything
into chunks.  Granted, I don't rewrite it often but unless zfs is
SMR-aware it is still going to be writing lots of modest-sized files
as the original files get chunked up and distributed across the nodes.
On the disk lizardfs data just looks like a browser cache, with
everything in numbered files about 60MB in size in my case.  The files
also appear to turn over a bit during rebalancing.

-- 
Rich

Re: [gentoo-user] External hard drive and idle activity

[Mick]

On Monday, 6 January 2020 08:48:13 GMT Stefan Schmiedl wrote:
> Dale,
> 
> "Dale" , 06.01.2020, 09:29:
> > Also, when looking for a drive to buy, what should one look at to see if
> > it is a SMR drive?

You will need to visit the OEMs website and dig into the documentation they 
provide.  Keywords like "archive drive/disk/format", "shingled magnetic 
recording" and "SMR", would be a giveaway this is not a normal PC drive.


> > While it may be OK for my backups, I'd like to avoid
> > them on the drives inside my rig that are used for the OS or /home.  I
> > dunno, just a gut thing.
> 
> it's not "just a gut thing". SMR drives are not meant for random
> access writing; they write like a tape and read like a disk.
> 
> A while ago, one of my clients bought one of those things
> to replace an older failing backup drive. The next night, the
> backup took hours instead of minutes. No knowing what was inside
> the box, I did some measurements and discovered that the first
> few files were written quickly, then things got really slow,
> with the rsync process waiting (state "D") for the drive to
> finish.
> 
> tar-based backups went much quicker, though, which matches the
> expected behaviour of SMR drives; the drive did not need to rewrite
> many large areas due to many small changes, instead it only had to
> write one large area due to one large change.
> 
> s.

Stefan reinforced a point made earlier by Richard (I think).  These drives are 
only good for linear backups, like tar performs when it appends newer files to 
an existing tarball.  If they are used as normal PC drives for regular writing 
of data, or with back up commands which use rsync, cp, etc. then the disk will 
fail much sooner than expected because of repeated multiple areas being 
deleted, before each smaller write.  I recall reading about how short the life 
of SMR drives was shown to be when used in NAS devices - check google or 
youtube if you're interested in the specifics.

Personally, I would only use such a drive for 'keepers'.  Say, films I intend 
to write once and watch many times, ripped music albums, family photos, etc.  
For OS files and other temporary backups I would use a normal PC drive.

PS. When you put together a tar script do not forget to add --xattrs.  If not, 
you'll find some commands break when you run them from a restored fs.
-- 
Regards,
Mick

signature.asc
Description: This is a digitally signed message part.

Re: [gentoo-user] External hard drive and idle activity

[Stefan Schmiedl]

Dale,

"Dale" , 06.01.2020, 09:29:

> Also, when looking for a drive to buy, what should one look at to see if
> it is a SMR drive?  While it may be OK for my backups, I'd like to avoid
> them on the drives inside my rig that are used for the OS or /home.  I
> dunno, just a gut thing. 

it's not "just a gut thing". SMR drives are not meant for random 
access writing; they write like a tape and read like a disk.

A while ago, one of my clients bought one of those things 
to replace an older failing backup drive. The next night, the
backup took hours instead of minutes. No knowing what was inside
the box, I did some measurements and discovered that the first
few files were written quickly, then things got really slow,
with the rsync process waiting (state "D") for the drive to
finish.

tar-based backups went much quicker, though, which matches the
expected behaviour of SMR drives; the drive did not need to rewrite
many large areas due to many small changes, instead it only had to
write one large area due to one large change.

s.

Re: [gentoo-user] External hard drive and idle activity

[Dale]

Dale wrote:
> <<< SNIP >>>
> Should I change the mounting options for this drive?  I've read some
> people use certain options for SSD and they are needed.  This is the
> options being used according to mount:
>
> /dev/sdj1 on /run/media/dale/8tb-backup type ext4
> (rw,nosuid,nodev,relatime,uhelper=udisks2)
>
> That I guess is the default way KDE/udisks/etc does it. 
>
> One other question, what should I look for to avoid these types of
> drives?  I'll go back and look but I don't recall it saying anything
> about this.  I'm also trying to figure out if this is a good thing or not. 
>
> Thanks much for the info.  I'll read this one again shortly. Lot of info
> to absorb. 
>
> Dale
>
> :-)  :-) 
>


Finally got around to rebooting.  It was a experience for sure.  X
wouldn't come up, the plasma thingy doing its thing.  Jeepers.  No
wonder I hate rebooting.  ROFL  I enabled the options mentioned in the
kernel and finally rebooted using it. This is the option and the current
mount info:


root@fireball / # zcat /proc/config.gz | grep DM_ZONED
CONFIG_DM_ZONED=y
root@fireball / #

/dev/sdj1 on /run/media/dale/8tb-backup type ext4
(rw,nosuid,nodev,relatime,uhelper=udisks2)


It looks the same to me.  While it seems safe as is, should I be
changing something to make it use either additional or other options? 

While it is a backup, I'd like to make sure they are worth something if
the need should arise. 

Also, when looking for a drive to buy, what should one look at to see if
it is a SMR drive?  While it may be OK for my backups, I'd like to avoid
them on the drives inside my rig that are used for the OS or /home.  I
dunno, just a gut thing. 

Thanks.

Dale

:-)  :-) 

Re: [gentoo-user] KDE plasmashell and wallpapers.

[Dale]

Dale wrote:
>
> OK.  This is what it spits out, one after another:
>
>
> kf5.kpackage: No metadata file in the package, expected it at:
>
>
> After that, it repeats the same thing with the path and name of each
> image on the end of the above.  The only thing that changes is the file
> name.
>
> With that, I googled and found this.
>
>
> https://forum.manjaro.org/t/plasmashell-freezing-after-update/76302/7
>
> Which had this fix:
>
> "All good, I found a solution.
> I just cleaned all plasma configuration with rm ~/.config/plasma* and
> logged in my session again."
>
> I have these files located there:
>
>
> root@fireball / # ls -al /home/dale/.config/plasma*
> -rw-rw-r-- 1 dale users    35 Oct 16  2017
> /home/dale/.config/plasma_calendar_holiday_regions
> -rw-rw-r-- 1 dale users    26 Oct 16  2017
> /home/dale/.config/plasma-localerc
> -rw--- 1 dale users    34 Oct 10 08:16 /home/dale/.config/plasmanotifyrc
> -rw-rw-r-- 1 dale users 11262 Dec 24 13:01
> /home/dale/.config/plasma-org.kde.plasma.desktop-appletsrc
> -rw-rw-r-- 1 dale users    68 May  3  2018 /home/dale/.config/plasmarc
> -rw-rw-r-- 1 dale users   974 Dec 23 10:40 /home/dale/.config/plasmashellrc
> -rw--- 1 dale users   207 Mar 24  2018
> /home/dale/.config/plasma_workspace.notifyrc
> root@fireball / #
>
>
> I'd think the 1st, 4th and last one wouldn't be the ones, but what do I
> know.  I'd think the others could be something.  Still, I went digging
> through them all.  The file plasma-org.kde.plasma.desktop-appletsrc has
> many mentions of wallpapers and lists directories of where they are.  So
> far, #1 suspect.  After checking the other files, no mention of
> wallpapers found.  I know, I could have used grep for this but . . .
>
> OK.  I renamed the file with .old.  Logged out and back in again and
> lost pretty much every setting I had.  No big surprise just slightly
> annoying.  Anyway, after getting things back to being usable, I added
> the wallpaper directory back, the monster directory, and guess what,
> same thing.  CPU went to 100% on one core and stayed there.  Now it just
> so happens that a friend came up to visit and we ended up talking for a
> good hour or so.  When I came back, it was still banging away doing
> whatever silliness it is doing.  So, we up to about a hour without it
> completing whatever it is doing.  Do I really want to go through this
> with every login??? 
>
> So, it seem that when they changed the random only part of this to being
> able to do them in order, they added some indexing system or something
> without thinking about people who may have quite a few of wallpapers.  I
> guess in the meantime, I'm going to have to whittle down the directories
> or something.  Of course, I could add my directory for my camera pics
> and see if it just plain blows up or something.  That would add another
> 50,000 images and push the total over 200,000 at that point.  ROFLMBO
>
> It sucks when they improve one thing but then basically break the whole
> thing in the process.  ROFL 
>
> Now to see what other replies I have. 
>
> Thanks much.
>
> Dale
>
> :-)  :-) 
>


OK.  I finally got around to rebooting.  Took me a while but anyway. 
When I logged into KDE, nothing plasma was working.  There was no
background at all, no panel thingy at the bottom, just nothingness.  It
was ugly.  I used the ctrl F* key to get where I could see gkrellm.  One
core of the CPU was banging away at something.  I had a suspect too. 
;-)  I let it sit for a bit and sure enough, the panel thingy showed up,
my background popped up and the output for the TVs also kicked in. 

So, the "fix" for making things not random is to bog plasma down with
some index/db/whatever thingy.  I guess until there is a proper fix,
I'll just have to whittle this thing down some more.  It took a couple
minutes or so to do whatever it is doing too.  Sort of had me worried
for a minute there.  My first problem was X itself not coming up.  After
trying the usual suspects, restarting udev fixed that.  I'm not sure
what to think about that.  :/

Thanks to all.  Just wanted to give a update. 

Dale

:-)  :-)