Re: [gentoo-user] electron and sslv3

[Damo Brisbane]

FYI this info and use flags worked for me to get a clean *electron* build
(with also getting eg *nodejs *dependency). I think also works for
chromium. I doubt "static-libs" is making any difference:

*/etc/portage>* emerge --info electron

dev-libs/openssl-1.0.2l::gentoo was built with the following:
USE="asm sslv2 sslv3 static-libs tls-heartbeat zlib -bindist -gmp -kerberos
-rfc3779 -sctp -test -vanilla"

*/etc/portage>* emerge --info electron

dev-libs/openssl-1.0.2l::gentoo was built with the following:
USE="asm sslv2 sslv3 static-libs tls-heartbeat zlib -bindist -gmp -kerberos
-rfc3779 -sctp -test -vanilla"

*/etc/portage>* egrep -r "(ssl|electron|nodejs)" pack*

package.accept_keywords:>=dev-libs/openssl-1.0.2l ~amd64
package.accept_keywords:=dev-util/electron-1.3.13-r1 ~amd64

package.mask/mask:=dev-libs/openssl-1.0.1
package.mask/mask:=dev-libs/openssl-1.0.2g
package.mask/mask:=dev-libs/openssl-1.1.0f
package.use/use:>=dev-libs/openssl-1.0.2l ssl sslv2 sslv3 static-libs


On Mon, Sep 4, 2017 at 4:23 PM, Damo Brisbane  wrote:

> Emerge -pv openssl:
>
> [ebuild   R] dev-libs/openssl-1.0.2l::gentoo  USE="asm sslv3
> tls-heartbeat zlib -bindist -gmp -kerberos -rfc3779 -sctp -sslv2
> -static-libs {-test} -vanilla"...
>
> I figured ssl better off without it; I think the issue with this package
> is it builds it's own version of chromium as part of the emerge, and I
> think this is where the ssl dependency comes in. Right though, I think
> package maintainer is where I need to head to next.
>
> Thanks
>
> On Sat, Sep 2, 2017 at 11:40 AM, Adam Carter 
> wrote:
>
>> On Sat, Sep 2, 2017 at 6:26 AM, Damo Brisbane 
>> wrote:
>>
>>> Hello,
>>>
>>> I am having troubles installing dev-util/electron, related to linking in
>>> "ssl3" in the final step of the ebuild, from build log:
>>>
>>> /usr/lib/gcc/x86_64-pc-linux-gnu/6.3.0/../../../../x86_64-pc-linux-gnu/bin/ld:
>>> cannot find -lssl3
>>>
>>>
>>> FYI on ssl, I only want a "working/current" ssl and/or tls installation
>>> and I don't care for the details around the installation other than I would
>>> like - as much as possible - "ssl" to be future proof and compatible with
>>> current and new installs; in this case I just want electron, and I can't
>>> install the package because of this linking error. I can successfully build
>>> by hacking the final link step and simply remove the reference to "-lssl",
>>> below:
>>>
>>>
>>> > cd $PORTAGE_TMPDIR/dev-util/electron-1.3.13-r1/work/chromium-52
>>> .0.2743.82/out/R
>>> > x86_64-pc-linux-gnu-g++ -Wl,-O1 -Wl,--a 
>>> > obj/atom/app/electron.atom_main.o  obj/libelectron_lib.a
>>> o... lib/libnode.so lib/libv8.so -lz -lhttp_parser -lssl -lcrypto -
>>>
>>>
>>> and compiles fine.
>>>
>>> There are no "ssl" use flags on electron?:
>>>
>>
>> My first guess would be that your openssl is not compiled with sslv3. The
>> ebuild for electron only asks for >=dev-libs/openssl-1.0.2g:0=[-bindist]
>> not openssl[sslv3]. If that's the problem then there's a bug in electrons
>> ebuild.
>>
>> What does emerge -pv openssl show for use flags?
>>
>> However, ssl is pretty much deprecated these days due to security issues,
>> so unless you have a need to support something that cant do TLS, you're
>> better off leaving it out. Another issue may be that -lssl may be a loose
>> term for SSL+TLS...
>>
>
>

Re: [gentoo-user] Dual booting with Windows 10

[R0b0t1]

On Sun, Sep 17, 2017 at 9:12 AM, Peter Humphrey  wrote:
> On Thursday, 14 September 2017 19:51:37 BST R0b0t1 wrote:
>> On Thu, Sep 14, 2017 at 3:20 AM, Peter Humphrey 
> wrote:
>> > On Thursday, 14 September 2017 05:09:14 BST R0b0t1 wrote:
>> >> The trickiest part is still the same - going from GRUB or, now, your
>> >> EFI shell, to Window's bootloader. See here:
>> >> https://wiki.archlinux.org/index.php/GRUB#Chainloading_Windows.2FLinux_
>> >> ins talled_in_UEFI_mode.
>> >
>> > That advice, though helpful, is about Grub, which isn't installed on
>> > this box. I did try at first to get it to work here, but failed, so I
>> > removed it and went for bootctl. It's a fiddle to keep up to date with
>> > kernel upgrades, but at least it works.
>>
>> In that case it seems like systemd-boot will check for the Windows
>> loader and add it to its menu automatically
>> (https://wiki.archlinux.org/index.php/systemd-boot#Adding_boot_entries).
>> As above, you may need to reinstall it if the Windows bootloader
>> installs itself on top of systemd-boot.
>>
>> I originally thought you were just booting an EFI stub kernel, in
>> which case you would have needed some kind of boot manager.
>
> I have three questions now:
>
> 1.  Will Windows 10 install itself in the unpartitioned space? I've 
> attached
> a screen shot of gparted to show the current layout.
>

Yes. It will split the free space into a number of partitions if you
give the installer no further instruction besides selecting the
unallocated area.

To force Windows to use one partition delete the ones it creates
automatically. You will need to select "custom" or "advanced" in every
place it is offered as an option.

> 2.  What will happen to the UEFI kernel entries in /dev/nvme0n1p1?
>

When people say "entries" they are usually referring to settings in
the nonvolatile memory used by a motherboard's EFI firmware. An entry
associates with an ID a path, priority, and name which is used to
start the corresponding EFI executable.

The actual kernels on /dev/nvme0np1 will remain there because Windows
won't touch that partition unless you tell it to.

> 3.  Those entries include some left over from experimenting with other
> distros. How can I manage the entries and purge the ones I don't need?
> "Bootctl remove" ignores them.
>

If you are referring to the kernels left in your /boot then simply
delete them. "Bootctl remove" and other EFI boot managers I have seen
refuse to touch your disk. They operate on the EFI configuration
memory.

> Thanks everyone for your help so far.
>
> I don't want to install into a VM, because my main reason for installing
> Win10 is to be able to run an occasional firmware update program, none of
> which, it seems, run on Linux. Of course, it should also help me get up to
> speed with the M$ world.
>

If you pass an entire hard disk to the VM you can then take it out and
put it in another computer and boot it (or boot it in the same
computer sans hypervisor).

With Linux you can pass partitions in individually and use what the
guest thinks is a raw character device as a disk, so that if you
wanted to boot that installation from outside of the hypervisor you
could. This might not be possible with Windows.

If you install into a VM you can pass almost everything to the VM
directly. I suppose the only thing that may not work extremely well
would be motherboard firmware updates, but if you look QEMU has
options to pass almost everything in a computer to a VM. Admittedly
this isn't a very plug-and-play solution.

Aside from firmware updates (realize though that almost everything -
barring some low level interfaces like I2C - can be passed to a VM) I
would invite you to use Windows only in a VM. I find it easier to get
work done in this way while using Windows programs. Xfreerdp is a good
way to interact with a Windows guest and can provide better desktop
integration than QEMU or libvirtd.

Cheers,
 R0b0t1

Re: [gentoo-user] Searching something like a merge between gnu-screen and cutecom...

[R0b0t1]

Hello!

On Sun, Sep 17, 2017 at 10:41 AM,   wrote:
> Hi,
>
> this is complicate for me to explain...let' try it nvertheless ;)
>
> I am experimenting with FORTH (punyforth) on an ESP8266.
> FORTH has a REPL, which -- especially in the beginning -- is
> very helpful to try things out.
>
> With cutecom I can connect to the ESP8266 and get a response.
> Unfortunately - in this particular case - cutecom separates the
> input line from what the ESP8266 i sending back.
>

Like this 
(https://arduino.stackexchange.com/questions/24578/how-to-filter-a-blank-line-received-over-serial-esp8266)?

It sounds like the firmware on the chip is written incorrectly. If it
is doing something like echoing lines back with \r\n at the end you
can usually configure your terminal software to translate them to \n,
but if it's doing it sporadically you may need to put a layer in
between you and the terminal.

> With Arduino devices I had no problem to do the same thing with
> gnu-screen like this
>
> screen /dev/ttyUSB1 115200
>
> ...with the ESP8266 (using the same baudrate as with cutecom
> before) the cursor get glued in the upper left corner.
> Hitting a key, CTRL-D. CTRL-C,  and other obvious
> candidates does not help. The only waty out is killall.
>

I think screen can support dumb terminals, but this behavior makes it
sound like it is expecting smart hardware terminal feedback. Check the
manual.

> Now I am in search of a "real terminal"-like thing, which
> allows me to interactively connect to the ESP8266 while
> maintaining the chronological sequence of my inputs and
> the returns of the ESP8266.
>
> I tried minicom, but I always dislike the way that beast
> is configured,
>
> Is there anything reliable available...may be even without a gui?
>

http://pyserial.readthedocs.io/en/latest/tools.html#module-serial.tools.miniterm

pySerial ships with a minimal terminal emulator which is perfect for
reading device output. I tend to use it above anything else because it
is sufficient and I am generally working with Python. There is no
readline support, but you didn't ask about that.

Importantly it doesn't cook the input in any way or expect the
terminal to do anything, so you can use it to better figure out what
the ESP8266 firmware is doing.

Cheers,
 R0b0t1

[gentoo-user] Re: [offtopic] Copy-On-Write ?

[Kai Krakow]

Am Sun, 17 Sep 2017 08:20:50 -0500
schrieb Dan Douglas :

> On 09/17/2017 04:17 AM, Kai Krakow wrote:
> > Am Sun, 17 Sep 2017 01:20:45 -0500
> > schrieb Dan Douglas :
> >   
> >> On 09/16/2017 07:06 AM, Kai Krakow wrote:  
>  [...]  
>  [...]  
> >>  [...]
>  [...]  
>  [...]  
> >>
> >> According to btrfs-filesystem(8), defragmentation breaks reflinks,
> >> in all but a few old kernel versions where I guess they tried to
> >> fix the problem and apparently failed.  
> > 
> > It was splitting and splicing all the reflinks which is actually a
> > tree walk with more and more extents coming into the equation, and
> > ended up doing a lot of small IO and needing a lot of memory. I
> > think you really cannot fix this when working with extents.  
> 
> I figured by "break up" they meant it eliminates the reflink by making
> a full copy... so the increased space they're talking about isn't
> really double that of the original data in other words.
> 
> >   
> >> This really makes much of what btrfs
> >> does altogether pointless if you ever defragment manually or have
> >> autodefrag enabled. Deduplication is broken for the same reason.  
> > 
> > It's much easier to fix this for deduplication: Just write your
> > common denominator of an extent to a tmp file, then walk all the
> > reflinks and share them with parts of this extent.
> > 
> > If you carefully select what to defragment, there should be no
> > problem. A defrag tool could simply skip all the shared extents. A
> > few fragments do not hurt performance at all, but what's important
> > is spatial locality. A lot small fragments may hurt performance a
> > lot, so one could give the defragger a hint when to ignore the rule
> > and still defragment the extent. Also, when your deduplication
> > window is 1M you could probably safely defrag all extents smaller
> > than 1M.  
> 
> Yeah this sort of hurts with the way I deal wtih KVM image snapshots.
> I have raw base images as backing files with lots of shared and null
> data, so I run `fallocate --dig-holes' followed by `duperemove
> --dedupe-options=same' on the cow-enabled base images and hope that
> btrfs defrag can clean up the resulting fragmented mess, but it's a
> slow process and doesn't seem to do a good job.

I would be interested about your results if you try bees[1] to
deduplicate your KVM images. It should be able to dig holes and merge
blocks by reflinking. I'm not sure if it would merge continuous extents
back into one single extent, I think that's on a todo list. It could
act as a reflink-aware defragger then.

It currently does not work well for mixed datasum/nodatasum workloads,
so I made a PR[2] to ignore nocow files. A more elaborated patch would
not try to reflink datasum and nodatasum extents (nocow implies
nodatasum).

[1]: https://github.com/Zygo/bees
[2]: https://github.com/Zygo/bees/pull/21


-- 
Regards,
Kai

Replies to list-only preferred.


pgpb6FiJolG_M.pgp
Description: Digitale Signatur von OpenPGP

[gentoo-user] Searching something like a merge between gnu-screen and cutecom...

[tuxic]

Hi,

this is complicate for me to explain...let' try it nvertheless ;)

I am experimenting with FORTH (punyforth) on an ESP8266.
FORTH has a REPL, which -- especially in the beginning -- is
very helpful to try things out.

With cutecom I can connect to the ESP8266 and get a response.
Unfortunately - in this particular case - cutecom separates the
input line from what the ESP8266 i sending back.

With Arduino devices I had no problem to do the same thing with
gnu-screen like this 

screen /dev/ttyUSB1 115200

...with the ESP8266 (using the same baudrate as with cutecom
before) the cursor get glued in the upper left corner.
Hitting a key, CTRL-D. CTRL-C,  and other obvious
candidates does not help. The only waty out is killall.

Now I am in search of a "real terminal"-like thing, which 
allows me to interactively connect to the ESP8266 while
maintaining the chronological sequence of my inputs and
the returns of the ESP8266.

I tried minicom, but I always dislike the way that beast 
is configured,

Is there anything reliable available...may be even without a gui?

Thanks fpr any help in advance!
Cheers
Meino

Re: [gentoo-user] Re: [offtopic] Copy-On-Write ?

[Dan Douglas]

On 09/17/2017 04:17 AM, Kai Krakow wrote:
> Am Sun, 17 Sep 2017 01:20:45 -0500
> schrieb Dan Douglas :
> 
>> On 09/16/2017 07:06 AM, Kai Krakow wrote:
>>> Am Fri, 15 Sep 2017 14:28:49 -0400
>>> schrieb Rich Freeman :
>>>   
 On Fri, Sep 8, 2017 at 3:16 PM, Kai Krakow 
 wrote:  
>>  [...]  

 True, but keep in mind that this applies in general in btrfs to any
 kind of modification to a file.  If you modify 1MB in the middle
 of a 10GB file on ext4 you end up it taking up 10GB of space.  If
 you do the same thing in btrfs you'll probably end up with the
 file taking up 10.001GB.  Since btrfs doesn't overwrite files
 in-place it will typically allocate a new extent for the
 additional 1MB, and the original content at that position within
 the file is still on disk in the original extent.  It works a bit
 like a log-based filesystem in this regard (which is also
 effectively copy on write).  
>>>
>>> Good point, this makes sense. I never thought about that.
>>>
>>> But I guess that btrfs doesn't use 10G sized extents? And I also
>>> guess, this is where autodefrag jumps in.  
>>
>> According to btrfs-filesystem(8), defragmentation breaks reflinks, in
>> all but a few old kernel versions where I guess they tried to fix the
>> problem and apparently failed.
> 
> It was splitting and splicing all the reflinks which is actually a tree
> walk with more and more extents coming into the equation, and ended up
> doing a lot of small IO and needing a lot of memory. I think you really
> cannot fix this when working with extents.

I figured by "break up" they meant it eliminates the reflink by making
a full copy... so the increased space they're talking about isn't really
double that of the original data in other words.

> 
>> This really makes much of what btrfs
>> does altogether pointless if you ever defragment manually or have
>> autodefrag enabled. Deduplication is broken for the same reason.
> 
> It's much easier to fix this for deduplication: Just write your common
> denominator of an extent to a tmp file, then walk all the reflinks and
> share them with parts of this extent.
> 
> If you carefully select what to defragment, there should be no problem.
> A defrag tool could simply skip all the shared extents. A few fragments
> do not hurt performance at all, but what's important is spatial
> locality. A lot small fragments may hurt performance a lot, so one
> could give the defragger a hint when to ignore the rule and still
> defragment the extent. Also, when your deduplication window is 1M you
> could probably safely defrag all extents smaller than 1M.

Yeah this sort of hurts with the way I deal wtih KVM image snapshots. I
have raw base images as backing files with lots of shared and null
data, so I run `fallocate --dig-holes' followed by `duperemove
--dedupe-options=same' on the cow-enabled base images and hope that
btrfs defrag can clean up the resulting fragmented mess, but it's a slow
process and doesn't seem to do a good job.



signature.asc
Description: OpenPGP digital signature

[gentoo-user] Re: [offtopic] Copy-On-Write ?

[Kai Krakow]

Am Sun, 17 Sep 2017 01:20:45 -0500
schrieb Dan Douglas :

> On 09/16/2017 07:06 AM, Kai Krakow wrote:
> > Am Fri, 15 Sep 2017 14:28:49 -0400
> > schrieb Rich Freeman :
> >   
> >> On Fri, Sep 8, 2017 at 3:16 PM, Kai Krakow 
> >> wrote:  
>  [...]  
> >>
> >> True, but keep in mind that this applies in general in btrfs to any
> >> kind of modification to a file.  If you modify 1MB in the middle
> >> of a 10GB file on ext4 you end up it taking up 10GB of space.  If
> >> you do the same thing in btrfs you'll probably end up with the
> >> file taking up 10.001GB.  Since btrfs doesn't overwrite files
> >> in-place it will typically allocate a new extent for the
> >> additional 1MB, and the original content at that position within
> >> the file is still on disk in the original extent.  It works a bit
> >> like a log-based filesystem in this regard (which is also
> >> effectively copy on write).  
> > 
> > Good point, this makes sense. I never thought about that.
> > 
> > But I guess that btrfs doesn't use 10G sized extents? And I also
> > guess, this is where autodefrag jumps in.  
> 
> According to btrfs-filesystem(8), defragmentation breaks reflinks, in
> all but a few old kernel versions where I guess they tried to fix the
> problem and apparently failed.

It was splitting and splicing all the reflinks which is actually a tree
walk with more and more extents coming into the equation, and ended up
doing a lot of small IO and needing a lot of memory. I think you really
cannot fix this when working with extents.


> This really makes much of what btrfs
> does altogether pointless if you ever defragment manually or have
> autodefrag enabled. Deduplication is broken for the same reason.

It's much easier to fix this for deduplication: Just write your common
denominator of an extent to a tmp file, then walk all the reflinks and
share them with parts of this extent.

If you carefully select what to defragment, there should be no problem.
A defrag tool could simply skip all the shared extents. A few fragments
do not hurt performance at all, but what's important is spatial
locality. A lot small fragments may hurt performance a lot, so one
could give the defragger a hint when to ignore the rule and still
defragment the extent. Also, when your deduplication window is 1M you
could probably safely defrag all extents smaller than 1M.


-- 
Regards,
Kai

Replies to list-only preferred.


pgpv49kJ6PHj8.pgp
Description: Digitale Signatur von OpenPGP

Re: [gentoo-user] Re: [offtopic] Copy-On-Write ?

[Dan Douglas]

On 09/16/2017 07:06 AM, Kai Krakow wrote:
> Am Fri, 15 Sep 2017 14:28:49 -0400
> schrieb Rich Freeman :
> 
>> On Fri, Sep 8, 2017 at 3:16 PM, Kai Krakow 
>> wrote:
>>>
>>> At least in btrfs there's also a caveat that the original extents
>>> may not actually be split and the split extents share parts of the
>>> original extent. That means, if you delete the original later, the
>>> copy will occupy more space than expected until you defragment the
>>> file: 
>>
>> True, but keep in mind that this applies in general in btrfs to any
>> kind of modification to a file.  If you modify 1MB in the middle of a
>> 10GB file on ext4 you end up it taking up 10GB of space.  If you do
>> the same thing in btrfs you'll probably end up with the file taking up
>> 10.001GB.  Since btrfs doesn't overwrite files in-place it will
>> typically allocate a new extent for the additional 1MB, and the
>> original content at that position within the file is still on disk in
>> the original extent.  It works a bit like a log-based filesystem in
>> this regard (which is also effectively copy on write).
> 
> Good point, this makes sense. I never thought about that.
> 
> But I guess that btrfs doesn't use 10G sized extents? And I also guess,
> this is where autodefrag jumps in.

According to btrfs-filesystem(8), defragmentation breaks reflinks, in
all but a few old kernel versions where I guess they tried to fix the
problem and apparently failed. This really makes much of what btrfs does
altogether pointless if you ever defragment manually or have autodefrag
enabled. Deduplication is broken for the same reason.



signature.asc
Description: OpenPGP digital signature