Re: When will OpenBSD become a friendly place for bug reporters?

[NilsOla Nilsson]

On Sat, Jul 13, 2019 at 03:47:53PM +0200, Ingo Schwarze wrote:
> Hi NilsOla,
> 
>  On 2019-07-11, Ingo Schwarze  wrote:
> 
> >>> I just did
> >>>   cd /usr/xenocara/app/xterm/
> >>>   cvs up -dP
> >>>   make clean
> >>>   make obj
> >>>   make
> >>>   doas make install
> >>> and now compose works again for me in xterm, too.
> >>> 
> >>> I have no idea what was going on.  I often have patches lying around
> >>> in my tree for diverse testing purposes, though i fail to remeber
> >>> anything in particular in xterm(1) lately, and there are no patches
> >>> there now.  Anyway, sorry for the noise.
> >>> 
> >>> To the other poster who reported problems with compose in xterm:
> >>> Can you try updating and recompiling xterm as shown above,
> >>> and does that help for you, too?
> 
> >> It works for mee too. 
> >> So, the xterm distributed in current is not the same
> >> as in the source tree? Just asking.
> 
> I don't know.  It sounds as if this aspect of xterm(1) was briefly
> broken in at least one snapshot about three weeks ago.  But i don't
> know why, i see no plausible reason.  Nothing was committed to xterm
> for four months, and i don't recall hearing about uncommitted xterm
> patches in snapshots either.
> 
> > It works also for one xterm that was started before the
> > reinstall.
> 
> Now that's very weird.  It doesn't for me.  I still have some old
> xterm binaries running, and compose still doesn't work in those,
> in the way i described, while it does work in xterms that i start
> now using the freshly recompiled /usr/X11R6/bin/xterm.

Of the xterms started before the reinstall it works
only for the one were i run the reinstallation. 

BTW I have one other machine with openbsd 6.5 stable, and
there it seems to work from the beginning

Yours 
NilsOla

-- 
Nils Ola Nilsson, 🐞 email nils...@abc.se, tel +46-70-374 69 89


signature.asc
Description: PGP signature

Re: When will OpenBSD become a friendly place for bug reporters?

[Ingo Schwarze]

Hi Ian,

ropers wrote on Sun, Jul 14, 2019 at 01:11:45AM +0200:

> When you say Latin-1, are you referring to ISO-8859-1,

Yes.

> which is what OpenBSD uses in many places,

I would say *used* not so much *uses*.
I'm not aware of many that are left.

> including on console?

Yes, that is one of the few places still using ISO-latin-1,
mostly because console UTF-8 support involves non-trivial
kernel work and the developers who started that work many years ago
got distracted and nothing ever materialized that could be committed.

If there are other areas where OpenBSD is still using ISO-latin-1,
in particular in userland, i'd love to learn about that and might
consider fixing it, if i'm able to.

Yours,
  Ingo

Re: When will OpenBSD become a friendly place for bug reporters?

[ropers]

On 14/07/2019, ropers  wrote:
> On 13/07/2019, Ingo Schwarze  wrote:
>> Hi Ian,
>>
>> ropers wrote on Sat, Jul 13, 2019 at 02:53:28AM +0200:
>>
>>> altnumd with e.g. CP437 support
>>
>> Over my dead body.  We won't add support for legacy character encodings.
>
> I understand the sentiment. The way I'm thinking about it though
> wouldn't necessarily force CP437 on anybody; that was just my example.
> If there gets to be a default .altnumrc, it could also be one that
> supports ISO-8859-1 or Windows CP1252 (the latter making more sense,
> because nobody has ISO-8859-1 Alt codes in muscle memory). And it
> wouldn't really even be legacy encoding "support" beyond merely
> offering the plumbing for people to type the key combos they still
> remember or still find on the web.  The plan is to actually make those
> produce nice, proper UTF-8, at least once things get beyond scan
> codes, etc.
>
>>> the more fundamental idea is to have this be completely configurable,
>>
>> The OpenBSD philosophy is to limit configurability to the utter minimum
>> needed.
>
> I'm not sure I would entirely agree.  Does pf.conf offer the utter
> minimum configurability needed?  It depends on your point of view I
> suppose.  Don't get me wrong, I can see the merit of your sentiment
> too, and it's a suckless kind of thought, perhaps.
> Either way, I don't think I'd dare dream of getting to the level where
> anything substantially by me would actually make its way into OpenBSD;
> just a port would be plenty of an achievement for me.  I appreciate
> that altnumd might require deep hooks, and in that sense perhaps even
> some helping hand from the the system at some stage -- but I'd worry
> about crossing that bridge when I get to it.  If I get to it.
>
>> Take mandoc as an example.  It supports as input: ASCII, UTF-8, latin-1,
>
> When you say Latin-1, are you referring to ISO-8859-1, which is what
> OpenBSD uses in many places, including on console? (My thoughts on all
> of that, and I have learnt a lot about it all, are for another email,
> Real Soon Now.)
>
>> and the latter only because legacy manual pages encoded that way are
>> still somewhat common in the wild.  It's all fully transparent and
>> just works.  Yes, the -K option exists, but only for compatibility,
>> you almost never need it.  No configuration or customization is ever
>> required, and no helper programs like preconv(1) with groff.
>> It supports as output: ASCII and UTF-8.  Again, almost nothing to
>> configure, just LC_CTYPE is enough, or -Tascii / -Tutf8 to override
>> that.  Though the final code achieving all that is relatively simple,
>> getting there took years of thought on how to simplify.
>>
>> Take any other base userland program; likewise optimized for KISS.
>> Examples of some that i contributed to for that purpose include
>> ksh(1), xterm(1), ls(1), ssh(1), sftp(1), though these are by now
>> means perfect yet; there is still much that could be taken away.
>>
>> The part that is still really complicated, difficult to use, and
>> fragile is keyboard encoding and low-level input handling.
>
> And that's the part, well, that and injecting things back into some
> buffer there, those are the parts I really would like some advice,
> from absolutely anyone, on where to even start, i.e. where to look,
> and what to read and find out more about.
>
> Thanks for your time,
> Ian
>

Re: When will OpenBSD become a friendly place for bug reporters?

[Ingo Schwarze]

Hi Ian,

ropers wrote on Sat, Jul 13, 2019 at 02:53:28AM +0200:

> altnumd with e.g. CP437 support

Over my dead body.  We won't add support for legacy character encodings.

> the more fundamental idea is to have this be completely configurable,

The OpenBSD philosophy is to limit configurability to the utter minimum
needed.

Take mandoc as an example.  It supports as input: ASCII, UTF-8, latin-1,
and the latter only because legacy manual pages encoded that way are
still somewhat common in the wild.  It's all fully transparent and
just works.  Yes, the -K option exists, but only for compatibility,
you almost never need it.  No configuration or customization is ever
required, and no helper programs like preconv(1) with groff.
It supports as output: ASCII and UTF-8.  Again, almost nothing to
configure, just LC_CTYPE is enough, or -Tascii / -Tutf8 to override
that.  Though the final code achieving all that is relatively simple,
getting there took years of thought on how to simplify.

Take any other base userland program; likewise optimized for KISS.
Examples of some that i contributed to for that purpose include
ksh(1), xterm(1), ls(1), ssh(1), sftp(1), though these are by now
means perfect yet; there is still much that could be taken away.

The part that is still really complicated, difficult to use, and
fragile is keyboard encoding and low-level input handling.

Yours,
  Ingo

Re: When will OpenBSD become a friendly place for bug reporters?

[Ingo Schwarze]

Hi NilsOla,

NilsOla Nilsson wrote on Sat, Jul 13, 2019 at 08:08:01AM +0200:
> On Sat, Jul 13, 2019 at 07:51:33AM +0200, NilsOla Nilsson wrote:
>> On Fri, Jul 12, 2019 at 08:14:06PM +0200, Ingo Schwarze wrote:
>>> Christian Weisgerber wrote on Fri, Jul 12, 2019 at 05:48:37PM -:
 On 2019-07-11, Ingo Schwarze  wrote:

> Quite likely.  I'm so clueless that right now, i can't even seem to get
> Compose to work even though i'm sure i had it working in the past.

 I use "setxkbmap -option compose:ralt" and compose works as expected
 for me in xterm.

>>> I just did
>>>   cd /usr/xenocara/app/xterm/
>>>   cvs up -dP
>>>   make clean
>>>   make obj
>>>   make
>>>   doas make install
>>> and now compose works again for me in xterm, too.
>>> 
>>> I have no idea what was going on.  I often have patches lying around
>>> in my tree for diverse testing purposes, though i fail to remeber
>>> anything in particular in xterm(1) lately, and there are no patches
>>> there now.  Anyway, sorry for the noise.
>>> 
>>> To the other poster who reported problems with compose in xterm:
>>> Can you try updating and recompiling xterm as shown above,
>>> and does that help for you, too?

>> It works for mee too. 
>> So, the xterm distributed in current is not the same
>> as in the source tree? Just asking.

I don't know.  It sounds as if this aspect of xterm(1) was briefly
broken in at least one snapshot about three weeks ago.  But i don't
know why, i see no plausible reason.  Nothing was committed to xterm
for four months, and i don't recall hearing about uncommitted xterm
patches in snapshots either.

> It works also for one xterm that was started before the
> reinstall.

Now that's very weird.  It doesn't for me.  I still have some old
xterm binaries running, and compose still doesn't work in those,
in the way i described, while it does work in xterms that i start
now using the freshly recompiled /usr/X11R6/bin/xterm.

Yours,
  Ingo

Re: When will OpenBSD become a friendly place for bug reporters?

[NilsOla Nilsson]

On Sat, Jul 13, 2019 at 07:51:33AM +0200, NilsOla Nilsson wrote:
> On Fri, Jul 12, 2019 at 08:14:06PM +0200, Ingo Schwarze wrote:
> > Hi Christian,
> > 
> > Christian Weisgerber wrote on Fri, Jul 12, 2019 at 05:48:37PM -:
> > > On 2019-07-11, Ingo Schwarze  wrote:
> > 
> > >> Quite likely.  I'm so clueless that right now, i can't even seem to get
> > >> Compose to work even though i'm sure i had it working in the past.
> >  
> > > I use "setxkbmap -option compose:ralt" and compose works as expected
> > > for me in xterm.
> > 
> > Thanks for mentioning that.
> > 
> > I just did
> > 
> >   cd /usr/xenocara/app/xterm/
> >   cvs up -dP
> >   make clean
> >   make obj
> >   make
> >   doas make install
> > 
> > and now compose works again for me in xterm, too.
> > 
> > I have no idea what was going on.  I often have patches lying around
> > in my tree for diverse testing purposes, though i fail to remeber
> > anything in particular in xterm(1) lately, and there are no patches
> > there now.  Anyway, sorry for the noise.
> > 
> > To the other poster who reported problems with compose in xterm:
> > Can you try updating and recompiling xterm as shown above,
> > and does that help for you, too?
> 
> It works for mee too. 
> So, the xterm distributed in current is not the same
> as in the source tree? Just asking.

It works also for one xterm that was started before the
reinstall.
> 
> /NilsOla
> -- 
> Nils Ola Nilsson, 🐞 email nils...@abc.se, tel +46-70-374 69 89



-- 
Nils Ola Nilsson, 🐞 email nils...@abc.se, tel +46-70-374 69 89


signature.asc
Description: PGP signature

Re: When will OpenBSD become a friendly place for bug reporters?

[NilsOla Nilsson]

On Fri, Jul 12, 2019 at 08:14:06PM +0200, Ingo Schwarze wrote:
> Hi Christian,
> 
> Christian Weisgerber wrote on Fri, Jul 12, 2019 at 05:48:37PM -:
> > On 2019-07-11, Ingo Schwarze  wrote:
> 
> >> Quite likely.  I'm so clueless that right now, i can't even seem to get
> >> Compose to work even though i'm sure i had it working in the past.
>  
> > I use "setxkbmap -option compose:ralt" and compose works as expected
> > for me in xterm.
> 
> Thanks for mentioning that.
> 
> I just did
> 
>   cd /usr/xenocara/app/xterm/
>   cvs up -dP
>   make clean
>   make obj
>   make
>   doas make install
> 
> and now compose works again for me in xterm, too.
> 
> I have no idea what was going on.  I often have patches lying around
> in my tree for diverse testing purposes, though i fail to remeber
> anything in particular in xterm(1) lately, and there are no patches
> there now.  Anyway, sorry for the noise.
> 
> To the other poster who reported problems with compose in xterm:
> Can you try updating and recompiling xterm as shown above,
> and does that help for you, too?

It works for mee too. 
So, the xterm distributed in current is not the same
as in the source tree? Just asking.

Thanks for the help!

/NilsOla
-- 
Nils Ola Nilsson, 🐞 email nils...@abc.se, tel +46-70-374 69 89


signature.asc
Description: PGP signature

Re: When will OpenBSD become a friendly place for bug reporters?

[ropers]

On 12/07/2019, Ingo Schwarze  wrote:
> Hi Ian,
>
> ropers wrote on Fri, Jul 12, 2019 at 01:37:16AM +0200:
>> On 11/07/2019, Ingo Schwarze  wrote:
>
>>> There is no reason to make it different.  ASCII is a subset of Unicode,
>>> with the same numbering.  So the "U" looks redundant to me.
>
>> There are several reasons why it isn't redundant:
>
> Your reasons are not part of the solution but part of the problem.
>
> Logically, the task is very simple:
>
>  1. Only UTF-8 input is needed because ASCII is a subset of that,
> and no other character set or encoding must be supported.
> (Of course, a method to input arbitrary bytes that do not form
> characters is also needed but that is rather tangential to this
> discussion).

Oh, no, nono. Just blindly throwing arbitrary eight-bit bytes at the
system isn't what I had in mind with altnumd.  That hasn't been what I
had in mind ever since I started reading up on UTF-8 and Unicode in
general.[0]

However, one needs to be careful here: UTF-8-encoded characters are
not strictly the same thing as scan codes or whatever is in the
keyboard buffer
{
   and that's the part I'm reeeally asking for help with; to wit:
>> And again, to me the hardest part of that is figuring out
>> how or where or what exactly to insert into the, or what, keyboard buffer.
>> (...)
>> Any hints on how to even start with that hardest part, or **what to read**
>> or **where to look** would be MORE than welcome
}.
Grabbing key combos and inserting something back into some keyboard
buffer is still input, rendering the correct UTF-8 character is
output.

altnumd with e.g. CP437 support would allow users to type those
decimals (with Alt+numpad), but it should actually act just the same
as if the correct corresponding U+ code point had been input in
some other way.
  Wikipedia's CP437 code page table currently, for convenience, also
lists the corresponding Unicode code points for each character[1]:

  The idea is to let people type like they know how to, and it would
create the same character, even though we're now on Unicode.
  HOWEVER, while a CP437 .altnumrc supporting Alt+0 thru 255 might be
the default configuration, the more fundamental idea is to have this
be completely configurable, which is also how different code pages
would be supported:
.altnumrc would contain key-value pairs. On the left, any decimal is
possible, far beyond 255 actually, for I-need-this-for-the-easter-egg
reasons. Thus, on the left: any long long unsigned int decimal. On the
right: any U+ code point (or perhaps comma-separated list of code
points?):

#.altnumrc for CP437:
#
000 U+
001 U+263A
002 U+263B
003 U+2665
004 U+2666
005 U+2663
006 U+2660
007 U+2022
008 U+25D8
009 U+25CB
010 U+25D9
011 U+2642
012 U+2640
013 U+266A
014 U+266B
015 U+263C
(...)
032 U+0020
033 U+0021
...and so on.
(Dropping leading zeroes would be legal for Alt+0-255; maybe throughout.)

Want Windows Alt codes instead? No problem, just swap out this
.altnumrc with a version for CP1252[2] and restart altnumd. Want
ISO-8859-1[3]? Likewise. Not all numbers would have to be defined, and
in fact both CP1252 and ISO-8859-1 leave some undefined. Heck, if you
wanted, you could roll your own .altnumrc with only two entries, for
the c-cedilla and a-umlaut you mentioned, set at any numbers you
please.

OR at least that's the idea. You're correct that it's all talk so far.
Figuring out if I really can code this? That's where I asked for help
on what to read, see above.

>  2. Physical keys must produce the characters printed on them.
>
>  3. One method is needed to input codepoints numerically, but not
> more than one.

I kind of agree -- except, the Alt code method already exists, and has
existed for a long time, and is still widespread.[4]
In some sense, any other new entry method has a higher burden of proof.
However, precisely because altnumd as envisaged already requires most
of the U+ code point-savvy plumbing, I realise that if things
actually progressed that far, then adding on some actually universal
Alt+u entry support would be marching downhill.

>  4. One method may be convenient to enter often-needed characters
> quickly (like Compose in X) and likely one mathod for languages
> that need very large numbers of characters (i don't know much
> about those).
>
> Items 1 to 3 are really the meat of the matter.  Item 4 is more like
> an add-on for convenience.

The way I read that, you mentioned two things in 4.: the shortcut
quick method and the comprehensive method. altnumd as envisaged
*could* do both, thanks to the crazy plan to make the left decimal a
ridiculously long int; however it would be much more practical to use
Alt+ for something short or already-known like CP437, and to
have actual code point support involve the literal entry of Unicode
code points.
Neither of those are substitutes for normal CJK input[5], which I know
a lit

Re: When will OpenBSD become a friendly place for bug reporters?

[Ingo Schwarze]

Hi Christian,

Christian Weisgerber wrote on Fri, Jul 12, 2019 at 05:48:37PM -:
> On 2019-07-11, Ingo Schwarze  wrote:

>> Quite likely.  I'm so clueless that right now, i can't even seem to get
>> Compose to work even though i'm sure i had it working in the past.
 
> I use "setxkbmap -option compose:ralt" and compose works as expected
> for me in xterm.

Thanks for mentioning that.

I just did

  cd /usr/xenocara/app/xterm/
  cvs up -dP
  make clean
  make obj
  make
  doas make install

and now compose works again for me in xterm, too.

I have no idea what was going on.  I often have patches lying around
in my tree for diverse testing purposes, though i fail to remeber
anything in particular in xterm(1) lately, and there are no patches
there now.  Anyway, sorry for the noise.

To the other poster who reported problems with compose in xterm:
Can you try updating and recompiling xterm as shown above,
and does that help for you, too?

Yours,
  Ingo

Re: When will OpenBSD become a friendly place for bug reporters?

[Raul Miller]

On Thu, Jul 11, 2019 at 7:39 PM ropers  wrote:
> I've just noticed yet another false positive where Gmail has
> classified your email as spam here for the n-th time.  I'm not sure if
> that's just happening to my mailbox, or if it's Gmail-wide or, worse,

My gmail spam bucket contains some openbsd mailing list traffic, but
nothing from Ingo Schwarze. For example, nothing from this thread has
been classified as spam, for me.

(It may be that gmail spam countermeasures are partially personalized?)

((My apologies for contributing nothing relevant to the central topics
of this thread, but this tangent seemed worth a small comment.))

Thanks,

-- 
Raul

Re: When will OpenBSD become a friendly place for bug reporters?

[Christian Weisgerber]

On 2019-07-11, Ingo Schwarze  wrote:

> Quite likely.  I'm so clueless that right now, i can't even seem to get
> Compose to work even though i'm sure i had it working in the past.

I use "setxkbmap -option compose:ralt" and compose works as expected
for me in xterm.

Zwölf Boxkämpfer jagen Viktor quer über den großen Sylter Deich.

Dès Noël où un zéphyr haï me vêt de glaçons würmiens je dîne d'exquis
rôtis de bœuf au kir à l'aÿ d'age mûr & cætera !

Pójdźże, kiń tę chmurność w głąb flaszy!

(Yes, I entered those in an OpenBSD xterm.)

-- 
Christian "naddy" Weisgerber  na...@mips.inka.de

Re: When will OpenBSD become a friendly place for bug reporters?

[Roderick]

On Fri, 12 Jul 2019, Ingo Schwarze wrote:


(There seems to be a trend where big corps are quite happy to
discourage people from running their own MTAs


Not just running their own MTAs, also using non-commercial .edu
infrastructure.


Not only that.

Here you see the developer of cyrus-imap moving to MS Exchange
and G Suite:

https://www.cmu.edu/computing/services/comm-collab/email-calendar/cyrus/decommission.html

The same for the developer of UW imap:

https://itconnect.uw.edu/connect/email/google-apps/
https://itconnect.uw.edu/connect/email/exchange-online/

And the same from the house of exim:

https://help.uis.cam.ac.uk/service/email

And sure they are not an exception, it seems to be a trend.

Rod.

Re: When will OpenBSD become a friendly place for bug reporters?

[Ingo Schwarze]

Hi Ian,

ropers wrote on Fri, Jul 12, 2019 at 01:37:16AM +0200:
> On 11/07/2019, Ingo Schwarze  wrote:

>> There is no reason to make it different.  ASCII is a subset of Unicode,
>> with the same numbering.  So the "U" looks redundant to me.

> There are several reasons why it isn't redundant:

Your reasons are not part of the solution but part of the problem.

Logically, the task is very simple:

 1. Only UTF-8 input is needed because ASCII is a subset of that,
and no other character set or encoding must be supported.
(Of course, a method to input arbitrary bytes that do not form
characters is also needed but that is rather tangential to this
discussion).

 2. Physical keys must produce the characters printed on them.

 3. One method is needed to input codepoints numerically, but not
more than one.

 4. One method may be convenient to enter often-needed characters
quickly (like Compose in X) and likely one mathod for languages
that need very large numbers of characters (i don't know much
about those).

Items 1 to 3 are really the meat of the matter.  Item 4 is more like
an add-on for convenience.

That said, i think i'll retire from this thread because we are just
talking.  Besides, i have a strong suspicion that you shold pick a
simpler project, in particular as your first project.  This one
seems seriously difficult conceptionally, exceedingly difficult
technically, in particular regarding the complex kernel-xenocara-userland
interactions, and *terrifyingly* complicated from a system integration
perspective - and you know, when the goal is to make something fit
for practical use (and commit), the system intergration part is
often the most dangerous obstacle in the first place, often challenging
even for seasoned developers.  Nothing wrong with picking a project
that is *technically* difficult if you feel adventurous (as long
as it is cleanly self-contained), but do try to start with projects
where system integration is easy, or expect almost certain eventual
failure - quite likely after already having invested lots of work.

Yours,
  Ingo


P.S. about broken spam filters:

> I've just noticed yet another false positive where Gmail has
> classified your email as spam here for the n-th time.  I'm not sure if
> that's just happening to my mailbox, or if it's Gmail-wide or, worse,
> if lots of MTAs out there treat your emails as spam.

So far, i have heard about outlook.com (which obviously nobody
should use anyway) occasionally classifying all mail coming from
the University of Karlsruhe (kit.edu) as spam, and about gmail.com
doing the same in rare cases.  Both of these appear to sometimes
consider that university - which is among the dozen or so most
important technical research universities in Germany - as a spam
site.  There is nothing much that i can do about that.

> (There seems to be a trend where big corps are quite happy to
> discourage people from running their own MTAs

Not just running their own MTAs, also using non-commercial .edu
infrastructure.

> and increasingly throw their weight around rejecting anything that
> isn't credentialled up the wazoo with SPF, DKIM, DMARC or whatever,

Of course large advertising corporations do what it takes to grab
market share, and vendor-lock in by breaking compatibility is a classic
method for doing that.

If your spam filter is broken, fix it.  I can hardly help with that.
If your ISP won't let you fix it, get a better provider.

Even if i wanted to contact the kit.edu postmasters to ask whether
they can do anything about your problem, you didn't provide any
information whatsoever - like for which exact reason which receiving
Google mail server classified which sending kit.edu mailserver as
a spam site, and at which exact time.  Such information should be
sent privately, not to public lists.

Re: When will OpenBSD become a friendly place for bug reporters?

[ropers]

On 12/07/2019, Ingo Schwarze  wrote:
> Hi NilsOla,
>
> NilsOla Nilsson wrote on Thu, Jul 11, 2019 at 11:24:24PM +0200:
>
>> I am using cwm and ksh and at present my compose key
>> work in st and in gvim, but not in xterm.
>> I am on current updated a few weeks ago.
>
> Oh.  Thank you for the hint.  Given my clumsiness with X keyboard
> configuration, i assumed it was just me, but your additional data
> indicates there may actually be a regression in xterm(1).
>
> Who knows what a GTK leviathan like gvim is doing.  But suckless
> software is about the opposite of bloatware and i would be quite
> surprised if it internally re-implemented the Compose mechanism.
> So the fact that Compose works in the x11/st(1) port (which i
> can confirm) means that i probably do have Compose correctly
> configured and that xterm(1) is somehow breaking the characters
> it receives from X - not all of them, but those where Compose
> was involved.
>
> The above suspicion is supported by the fact that Compose also
> works in xedit(1).  It even works in xclipboard(1), xman(1),
> nedit(1), except that these programs don't support UTF-8 but
> appear to be hardcoded to Latin-1, such that the entered UTF-8
> characters appear as garbage two-character sequences.
>
> To me, this looks suspiciously like a bug now.  :-(

Oh -- thank you both for that. I hadn't noticed, but that could have
really bitten me, as I'm currently using xterm more, chiefly for its
UTF-8 support that the console doesn't have, especially ever since
I've belatedly figured out Ctrl+right-click and Ctrl+left-click in
xterm. Look at all these choices I didn't know I had. :)

Ian

Re: When will OpenBSD become a friendly place for bug reporters?

[ropers]

On 11/07/2019, Ingo Schwarze  wrote:
> Hi Ian,

Hi Ingo,

I've just noticed yet another false positive where Gmail has
classified your email as spam here for the n-th time.  I'm not sure if
that's just happening to my mailbox, or if it's Gmail-wide or, worse,
if lots of MTAs out there treat your emails as spam.
(There seems to be a trend where big corps are quite happy to
discourage people from running their own MTAs and increasingly throw
their weight around rejecting anything that isn't credentialled up the
wazoo with SPF, DKIM, DMARC or whatever, and of course there are
powerful interests who want to deanonymise the Net, which may be
related.)
Either way, maybe this is something you'll want to look into from your end?

> ropers wrote on Thu, Jul 11, 2019 at 12:41:45AM +0200:
>
>> While I'm personally only or mainly interested in Alt+numeric input,
>> if altnumd existed, it would probably be comparatively easy to then
>> extend it and add support for Alt+u thru Alt+u10, with the U
>> becoming a reserved keyword unambiguously signifying that what follows
>> will be a Unicode code point between U+ and U+10.
>
> There is no reason to make it different.  ASCII is a subset of Unicode,
> with the same numbering.  So the "U" looks redundant to me.

There are several reasons why it isn't redundant:

1. Alt codes are decimal, but Unicode code points are hexadecimal.
Alt+73, Alt+78, Alt+71, Alt+79 "INGO" would become "sxqy" if treated as hex.

2. Unicode code points (format: U+, mostly, though it goes up to U+10)
are NOT character bytes. I quoted Wikipedia on this in my email two days ago:
>> [4] :
>> "The compromise solution that was eventually found and developed into
>> Unicode was to break the assumption (dating back to telegraph codes)
>> that each character should always directly correspond to a particular
>> sequence of bits. Instead, characters would first be mapped to a
>> universal intermediate representation in the form of abstract numbers
>> called __code points__. Code points would then be represented in a
>> variety of ways and with various default numbers of bits per character
>> (__code units__) depending on context. To encode code points higher
>> than the length of the code unit, such as above 256 for 8-bit units,
>> the solution was to implement variable-width encodings where an escape
>> sequence would signal that subsequent bits should be parsed as a
>> higher code point."
You are correct that in the case of the variable-length UTF-8 and for
the 128 (non-extended/7-bit) ASCII characters only, this isn't a
problem, because with them, code points (U+) and code units
(bytes) actually ARE still substantially identical. That saving grace
pretty much does not exist with other, non-UTF-8 Unicode encodings.
Okay, maybe it still does if you drop all the leading zeroes over
multiple bytes. However:

3. I would be wary of dropping leading zeroes in the case of Unicode
code point support. With Alt codes, the precedent of optionally
allowing the dropping of leading zeroes has been set, but pretty much
all Unicode documentation I'm aware of consistently prints code points
in the U+ format (or longer, up to U+10 where applicable).
There's a good argument for supporting code point entry exactly as
written, and nobody writes U+0 through U+FFF.
If you install the gucharmap package
, it has a Character Details tab
where you can not only see how much UTF-8 code units (bytes) can
differ from code points (U+), but you can also see that even for
those low code points where both match, the U+ is still not
printed with leading zeroes omitted.

4. One also should be as restrained and conservative as is practical
in terms of "claiming" key combinations, especially claiming them
system-wide. Yes, users could set up some hotkey somewhere that kills
and relaunches altnumd (I'm not even sure if that belongs in altnumd
itself), but you don't want to do that all the time just to type a key
combo that collides with altnumd. "Hold down Alt while typing U,
 then release Alt" is quite specific, and could reduce
cases where a sequence in .altnumrc collides with something else.
"Hold down Alt while typing up to three digits on the number pad, then
release Alt" is also relatively specific, though perhaps one might
accept non-numpad digit entry too, or make that choice configurable.
Alt+ single digit is more likely to collide with something, though
the long-standing precedent of Alt codes existing at least on some
platforms may make that less likely.

5. Perhaps there could be an opportunity for simplifying and unifying
Alt+U and existing iffy Ctrl+Shift U+ support? OTOH, maybe
it's better to deliberately not collide with that other method and
maybe that's a good reason for a universal Unicode code point method
to reside at its own key combo. Remember, my actual goal is Alt code
suppor

Re: When will OpenBSD become a friendly place for bug reporters?

[Ingo Schwarze]

Hi NilsOla,

NilsOla Nilsson wrote on Thu, Jul 11, 2019 at 11:24:24PM +0200:

> I am using cwm and ksh and at present my compose key
> work in st and in gvim, but not in xterm.
> I am on current updated a few weeks ago.

Oh.  Thank you for the hint.  Given my clumsiness with X keyboard
configuration, i assumed it was just me, but your additional data
indicates there may actually be a regression in xterm(1).

Who knows what a GTK leviathan like gvim is doing.  But suckless
software is about the opposite of bloatware and i would be quite
surprised if it internally re-implemented the Compose mechanism.
So the fact that Compose works in the x11/st(1) port (which i
can confirm) means that i probably do have Compose correctly
configured and that xterm(1) is somehow breaking the characters
it receives from X - not all of them, but those where Compose
was involved.

The above suspicion is supported by the fact that Compose also
works in xedit(1).  It even works in xclipboard(1), xman(1),
nedit(1), except that these programs don't support UTF-8 but
appear to be hardcoded to Latin-1, such that the entered UTF-8
characters appear as garbage two-character sequences.

To me, this looks suspiciously like a bug now.  :-(

Yours,
  Ingo


> On Thu, Jul 11, 2019 at 09:18:41PM +0200, Ingo Schwarze wrote:
> > Hi Ian,
> > 
> > ropers wrote on Thu, Jul 11, 2019 at 12:41:45AM +0200:
> > 
> > > While I'm personally only or mainly interested in Alt+numeric input,
> > > if altnumd existed, it would probably be comparatively easy to then
> > > extend it and add support for Alt+u thru Alt+u10, with the U
> > > becoming a reserved keyword unambiguously signifying that what follows
> > > will be a Unicode code point between U+ and U+10.
> > 
> > There is no reason to make it different.  ASCII is a subset of Unicode,
> > with the same numbering.  So the "U" looks redundant to me.
> > 
> > > There's a huge competence gap between us,
> > 
> > Quite likely.  I'm so clueless that right now, i can't even seem to get
> > Compose to work even though i'm sure i had it working in the past.
> > This is on amd64-current, inside xterm(1) and ksh(1):
> > 
> >$ locale
> >   LANG=
> >   LC_COLLATE="en_US.UTF-8"
> >   LC_CTYPE="en_US.UTF-8"
> >   LC_MONETARY="en_US.UTF-8"
> >   LC_NUMERIC="en_US.UTF-8"
> >   LC_TIME="en_US.UTF-8"
> >   LC_MESSAGES="en_US.UTF-8"
> >   LC_ALL=en_US.UTF-8
> >$ setxkbmap -query -v -v -v 
> >   Setting verbose level to 8
> >   locale is en_US.UTF-8
> >   Trying to load rules file ./rules/base...
> >   Trying to load rules file /usr/X11R6/share/X11/xkb/rules/base...
> >   Success.
> >   Applied rules from base:
> >   rules:  base
> >   model:  pc105
> >   layout: de
> >   Trying to build keymap using the following components:
> >   keycodes:   xfree86+aliases(qwertz)
> >   types:  complete
> >   compat: complete
> >   symbols:pc+de+inet(pc105)+terminate(ctrl_alt_bksp)
> >   geometry:   pc(pc105)
> >   rules:  base
> >   model:  pc105
> >   layout: de
> > 
> > At this point, the caps key toggles caps lock, i.e. pressing
> > 
> >   caps a caps a
> > 
> > results in the input "Aa".
> > 
> >$ setxkbmap -option compose:caps -v -v -v   
> >   Setting verbose level to 8
> >   locale is en_US.UTF-8
> >   Trying to load rules file ./rules/base...
> >   Trying to load rules file /usr/X11R6/share/X11/xkb/rules/base...
> >   Success.
> >   Applied rules from base:
> >   rules:  base
> >   model:  pc105
> >   layout: de
> >   options:compose:caps
> >   Trying to build keymap using the following components:
> >   keycodes:   xfree86+aliases(qwertz)
> >   types:  complete
> >   compat: complete
> >   symbols:pc+de+inet(pc105)+terminate(ctrl_alt_bksp)+compose(caps)
> >   geometry:   pc(pc105)
> > 
> > Now, the caps key no longer toggles caps lock and becomes a dead key,
> > i.e. pressing
> > 
> >   caps , c caps " a
> > 
> > results in the input "ca".  However, the resulting input is really
> > ASCII-c ASCII-a rather than the expected c-cedille a-umlaut.
> > It looks like Compose works well enough to discard the , and ",
> > but not well enough to actually generate non-ASCII characters.
> > 
> > Somewhat grumpy today,
> >   Ingo
> 
> -- 
> Nils Ola Nilsson, 🐞 email nils...@abc.se, tel +46-70-374 69 89

Re: When will OpenBSD become a friendly place for bug reporters?

[NilsOla Nilsson]

I am using cwm and ksh and at present my compose key
work in st and in gvim, but not in xterm.
I am on current updated a few weeks ago.

/NilsOla

On Thu, Jul 11, 2019 at 09:18:41PM +0200, Ingo Schwarze wrote:
> Hi Ian,
> 
> ropers wrote on Thu, Jul 11, 2019 at 12:41:45AM +0200:
> 
> > While I'm personally only or mainly interested in Alt+numeric input,
> > if altnumd existed, it would probably be comparatively easy to then
> > extend it and add support for Alt+u thru Alt+u10, with the U
> > becoming a reserved keyword unambiguously signifying that what follows
> > will be a Unicode code point between U+ and U+10.
> 
> There is no reason to make it different.  ASCII is a subset of Unicode,
> with the same numbering.  So the "U" looks redundant to me.
> 
> > There's a huge competence gap between us,
> 
> Quite likely.  I'm so clueless that right now, i can't even seem to get
> Compose to work even though i'm sure i had it working in the past.
> This is on amd64-current, inside xterm(1) and ksh(1):
> 
>$ locale
>   LANG=
>   LC_COLLATE="en_US.UTF-8"
>   LC_CTYPE="en_US.UTF-8"
>   LC_MONETARY="en_US.UTF-8"
>   LC_NUMERIC="en_US.UTF-8"
>   LC_TIME="en_US.UTF-8"
>   LC_MESSAGES="en_US.UTF-8"
>   LC_ALL=en_US.UTF-8
>$ setxkbmap -query -v -v -v 
>   Setting verbose level to 8
>   locale is en_US.UTF-8
>   Trying to load rules file ./rules/base...
>   Trying to load rules file /usr/X11R6/share/X11/xkb/rules/base...
>   Success.
>   Applied rules from base:
>   rules:  base
>   model:  pc105
>   layout: de
>   Trying to build keymap using the following components:
>   keycodes:   xfree86+aliases(qwertz)
>   types:  complete
>   compat: complete
>   symbols:pc+de+inet(pc105)+terminate(ctrl_alt_bksp)
>   geometry:   pc(pc105)
>   rules:  base
>   model:  pc105
>   layout: de
> 
> At this point, the caps key toggles caps lock, i.e. pressing
> 
>   caps a caps a
> 
> results in the input "Aa".
> 
>$ setxkbmap -option compose:caps -v -v -v   
>   Setting verbose level to 8
>   locale is en_US.UTF-8
>   Trying to load rules file ./rules/base...
>   Trying to load rules file /usr/X11R6/share/X11/xkb/rules/base...
>   Success.
>   Applied rules from base:
>   rules:  base
>   model:  pc105
>   layout: de
>   options:compose:caps
>   Trying to build keymap using the following components:
>   keycodes:   xfree86+aliases(qwertz)
>   types:  complete
>   compat: complete
>   symbols:pc+de+inet(pc105)+terminate(ctrl_alt_bksp)+compose(caps)
>   geometry:   pc(pc105)
> 
> Now, the caps key no longer toggles caps lock and becomes a dead key,
> i.e. pressing
> 
>   caps , c caps " a
> 
> results in the input "ca".  However, the resulting input is really
> ASCII-c ASCII-a rather than the expected c-cedille a-umlaut.
> It looks like Compose works well enough to discard the , and ",
> but not well enough to actually generate non-ASCII characters.
> 
> Somewhat grumpy today,
>   Ingo

-- 
Nils Ola Nilsson, 🐞 email nils...@abc.se, tel +46-70-374 69 89


signature.asc
Description: PGP signature

Re: When will OpenBSD become a friendly place for bug reporters?

[Ingo Schwarze]

Hi Ian,

ropers wrote on Thu, Jul 11, 2019 at 12:41:45AM +0200:

> While I'm personally only or mainly interested in Alt+numeric input,
> if altnumd existed, it would probably be comparatively easy to then
> extend it and add support for Alt+u thru Alt+u10, with the U
> becoming a reserved keyword unambiguously signifying that what follows
> will be a Unicode code point between U+ and U+10.

There is no reason to make it different.  ASCII is a subset of Unicode,
with the same numbering.  So the "U" looks redundant to me.

> There's a huge competence gap between us,

Quite likely.  I'm so clueless that right now, i can't even seem to get
Compose to work even though i'm sure i had it working in the past.
This is on amd64-current, inside xterm(1) and ksh(1):

   $ locale
  LANG=
  LC_COLLATE="en_US.UTF-8"
  LC_CTYPE="en_US.UTF-8"
  LC_MONETARY="en_US.UTF-8"
  LC_NUMERIC="en_US.UTF-8"
  LC_TIME="en_US.UTF-8"
  LC_MESSAGES="en_US.UTF-8"
  LC_ALL=en_US.UTF-8
   $ setxkbmap -query -v -v -v 
  Setting verbose level to 8
  locale is en_US.UTF-8
  Trying to load rules file ./rules/base...
  Trying to load rules file /usr/X11R6/share/X11/xkb/rules/base...
  Success.
  Applied rules from base:
  rules:  base
  model:  pc105
  layout: de
  Trying to build keymap using the following components:
  keycodes:   xfree86+aliases(qwertz)
  types:  complete
  compat: complete
  symbols:pc+de+inet(pc105)+terminate(ctrl_alt_bksp)
  geometry:   pc(pc105)
  rules:  base
  model:  pc105
  layout: de

At this point, the caps key toggles caps lock, i.e. pressing

  caps a caps a

results in the input "Aa".

   $ setxkbmap -option compose:caps -v -v -v   
  Setting verbose level to 8
  locale is en_US.UTF-8
  Trying to load rules file ./rules/base...
  Trying to load rules file /usr/X11R6/share/X11/xkb/rules/base...
  Success.
  Applied rules from base:
  rules:  base
  model:  pc105
  layout: de
  options:compose:caps
  Trying to build keymap using the following components:
  keycodes:   xfree86+aliases(qwertz)
  types:  complete
  compat: complete
  symbols:pc+de+inet(pc105)+terminate(ctrl_alt_bksp)+compose(caps)
  geometry:   pc(pc105)

Now, the caps key no longer toggles caps lock and becomes a dead key,
i.e. pressing

  caps , c caps " a

results in the input "ca".  However, the resulting input is really
ASCII-c ASCII-a rather than the expected c-cedille a-umlaut.
It looks like Compose works well enough to discard the , and ",
but not well enough to actually generate non-ASCII characters.

Somewhat grumpy today,
  Ingo

Re: When will OpenBSD become a friendly place for bug reporters?

[ropers]

On 11/07/2019, Matthew Graybosch  wrote:
> On Thu, 11 Jul 2019 00:41:45 +0200
> ropers  wrote:
>
>> Any hints on how to even start with that hardest part, or what to read
>> or where to look would be MORE than welcome.
>
> Hi, ropers. I've been reading this thread about altnumd and using
> alt+000 to enter arbitrary Unicode characters like people did on DOS
> and Windows. I noticed that nobody mentioned that X11 already has a
> method for entering arbitrary Unicode characters using pre-defined key
> sequences. It's called the "Compose" key (aka the Multi_Key in X11
> header files).

I mentioned that, somewhat near the top of my earlier email:

>> I am very aware of the Compose key support in X11, and this isn't
>> about that. This isn't about inputting characters in a good or better
>> way. This is about inputting characters in exactly THAT way
>> (Alt+Numpad), "just like mammy used to".

However, I didn't go into details, and though Compose doesn't solve
what I want to do, you're right that these details may help somebody,
so thanks anyway.

> https://en.wikipedia.org/wiki/Compose_key
>
> PC and Mac keyboards don't provide a physical compose key, but you can
> remap a key you don't usually use (like Caps Lock or the right-hand ALT
> or CTRL keys) using setxkbmap(1).
>
> Here's the command I use in my ~/.xsession file:
>
> setxkbmap -option compose:caps &
>
> Hopefully this helps somebody.

(Personally, I like to use compose:rwin, though that right Windows key
doesn't exist on some (old or laptop) keyboards.)

I'm pretty damn sure Ingo knows about Compose too. His complaint was
subtly different also:

> > I'm not aware of a simple method of entering Unicode codepoints
> > numerically that works everywhere, and i find that lack annoying.

Emphasis on "that works everywhere".

I am aware of the "Ctrl+Shift U " Unicode input method too,
but this again does not work everywhere:


However (this also from my above email):

>> [4] Also, the console would only support a small subset anyway; I have
some questions and thoughts on that too, but that's really another
issue for another day, perhaps.

Plus, Ingo wrote:

> > What might be useful is understanding all the details of how
> > all the existing methods in ws* and X* (and possibly elsewhere?)
> > work, then simplify them.

Even if Ingo's interest in understanding and simplifying existing
methods, --and arguably Alt codes *are* an existing method, albeit one
that works "elsewhere" but not on Unix-likes thus far--, could be
partially aligned with what I want to do, and would hopefully be
simplifying things instead of being like the invention of another text
editor ,
Unicode support on console would necessarily be limited.  I alluded to
that in my [4] footnote, and maybe I should write up my extensive
thoughts on that, Real Soon Now.  The major limitation there is that
console font support is limited to 256 characters, or hackishly 512,
and the ways around that are not pretty.  So while Alt+, and
*perhaps* also Alt+u could possibly be made to work the same
throughout X11 and console, Alt+u would be limited on console.
I am aware that Alt+u would compete with Ctrl+Shift U
(which latter doesn't work everywhere).
Either way, actual Unicode character input methods like this, and
let's include Compose here, could perhaps only *input* characters
like, say, the ☃ U+2603 SNOWMAN, but the console would at best
*display* \xE2\x98\x83 in place of that, because the Unicode snowman
is not in any console font.
If we want to be precise about it, console "support" for *displaying*
a character that was typed is a different issue than inputting it
though, and being limited to a subset here and there shouldn't count
against the idea of installing universal plumbing.

And again, to me the hardest part of that is figuring out
>> how or where or what exactly to insert into the, or what, keyboard buffer.
>> (...)
>> Any hints on how to even start with that hardest part, or what to read
or where to look would be MORE than welcome.

Many thanks in advance,
Ian

Re: When will OpenBSD become a friendly place for bug reporters?

[Matthew Graybosch]

On Thu, 11 Jul 2019 00:41:45 +0200
ropers  wrote:

> Any hints on how to even start with that hardest part, or what to read
> or where to look would be MORE than welcome.

Hi, ropers. I've been reading this thread about altnumd and using
alt+000 to enter arbitrary Unicode characters like people did on DOS
and Windows. I noticed that nobody mentioned that X11 already has a
method for entering arbitrary Unicode characters using pre-defined key
sequences. It's called the "Compose" key (aka the Multi_Key in X11
header files).

https://en.wikipedia.org/wiki/Compose_key

PC and Mac keyboards don't provide a physical compose key, but you can
remap a key you don't usually use (like Caps Lock or the right-hand ALT
or CTRL keys) using setxkbmap(1).

Here's the command I use in my ~/.xsession file:

setxkbmap -option compose:caps &

Hopefully this helps somebody.

-- 
Matthew Graybosch
https://matthewgraybosch.com | gopher://asgartech.com:70
xmpp: starbrea...@matthewgraybosch.com
"'Out of order'? Even in the future nothing works!"

Re: When will OpenBSD become a friendly place for bug reporters?

[ropers]

On 10/07/2019, Ingo Schwarze  wrote:
> Hi Ian,
>
> ropers wrote on Wed, Jul 10, 2019 at 03:03:08AM +0200:
>
>> I intend to write feature "altnumd" and would like advice for how to
>> go about it.
>
> My opinion is that we don't have too few methods for entering
> characters.  One could argue that we already have too many,
> and the existing ones are too complicated.  So writing yet
> another system or daemon sounds like a bad idea to me.
>
> What might be useful is understanding all the details of how
> all the existing methods in ws* and X* (and possibly elsewhere?)
> work, then simplify them.
>
> I'm not aware of a simple method of entering Unicode codepoints
> numerically that works everywhere, and i find that lack annoying.
>
> Yours,
>   Ingo

While I'm personally only or mainly interested in Alt+numeric input,
if altnumd existed, it would probably be comparatively easy to then
extend it and add support for Alt+u thru Alt+u10, with the U
becoming a reserved keyword unambiguously signifying that what follows
will be a Unicode code point between U+ and U+10.
Maybe this is the hubris born of inexperience and ignorance speaking,
but I think that would be the easiest part. [1]

I think capturing keypresses would be harder, but the existence of
sxhkd, etc. suggests this is solvable, at least on X11 (though I'd
love to have something that works just the same on console).

The hardest part that I'm comprehensively clueless about is how or
where or what exactly to insert into the, or what, keyboard buffer.

So I think maybe our interests are aligned for the moment. There's a
huge competence gap between us, but I think on this part of the
possible journey, we'd have to go the same way.

Any hints on how to even start with that hardest part, or what to read
or where to look would be MORE than welcome.

Cheers,
Ian

[1] Okay, sorry, I lied: What'd be somewhat easy would be putting the
*plumbing* for Alt+u in place ONCE the plumbing for
Alt+ exists. I imagine the latter as being based on a simple
.altnumrc that's a configurable list of key-value pairs. Creating a
list of keys from 0 thru 255 with the corresponding values for e.g.
code page 437 would be relatively easy, HOWEVER creating or
maintaining anything close to a semi-comprehensive list for just about
every Unicode character you can think of would actually be very hard.
  My thinking is to more or less leave that work to users: altnumd
would read .altnumrc on (re)launch, and users could create and swap
out different versions of .altnumrc e.g. to switch from CP437 to
WinCP1252 or ISO-8859-1 or whatever. A thinly populated .altnumrc for
initially just the lowest Unicode blocks[2] in the Basic Multilingual
Plane[3] would still be easy to make, but with higher expectations of
how comprehensive an .altnumrc you want to support and maintain, the
difficulty goes up exponentially. And that's not even mentioning font
support, etc.[4] The trick is not to include every combination but
only those Alt codes (or code points) you'll actually use.

[2] https://en.wikipedia.org/wiki/Unicode_block
[3] https://en.wikipedia.org/wiki/Plane_(Unicode)#Basic_Multilingual_Plane
[4] Also, the console would only support a small subset anyway; I have
some questions and thoughts on that too, but that's really another
issue for another day, perhaps.

Re: When will OpenBSD become a friendly place for bug reporters?

[Consus]

On 17:14 Wed 10 Jul, Juan Francisco Cantero Hurtado wrote:
> On Wed, Jul 10, 2019 at 09:30:56AM +0300, Consus wrote:
> > On 18:56 Tue 09 Jul, Roderick wrote:
> > > 
> > > On Tue, 9 Jul 2019, cho...@jtan.com wrote:
> > > 
> > > > Perhaps rather than whining that OpenBSD lacks some specific feature,
> > > > those who want it could write it?
> > > 
> > > Or perhaps better not. All depends on what is a feature and for whom.
> > > 
> > > I, as normal user, am glad that packages are not inflated with debugging
> > > symbols.
> > 
> > That's why redhat and others offer *-debuginfo packages with DWARF
> > symbols. It's really helpful. It would be nice to have such in OpenBSD,
> > especially for base, because rebuilding something on my router is not
> > something I would like to do.
> 
> The symbols are included in the base libraries. We only strip the
> symbols in the packages. Try this on your router:
> objdump -x /usr/lib/libedit.so.* (just an example)

Yeah, but it's still stripped for tools (try gdb $(which smtpd) for
example).

Re: When will OpenBSD become a friendly place for bug reporters?

[Juan Francisco Cantero Hurtado]

On Wed, Jul 10, 2019 at 09:30:56AM +0300, Consus wrote:
> On 18:56 Tue 09 Jul, Roderick wrote:
> > 
> > On Tue, 9 Jul 2019, cho...@jtan.com wrote:
> > 
> > > Perhaps rather than whining that OpenBSD lacks some specific feature,
> > > those who want it could write it?
> > 
> > Or perhaps better not. All depends on what is a feature and for whom.
> > 
> > I, as normal user, am glad that packages are not inflated with debugging
> > symbols.
> 
> That's why redhat and others offer *-debuginfo packages with DWARF
> symbols. It's really helpful. It would be nice to have such in OpenBSD,
> especially for base, because rebuilding something on my router is not
> something I would like to do.

The symbols are included in the base libraries. We only strip the
symbols in the packages. Try this on your router:
objdump -x /usr/lib/libedit.so.* (just an example)


-- 
Juan Francisco Cantero Hurtado http://juanfra.info

Re: When will OpenBSD become a friendly place for bug reporters?

[Ingo Schwarze]

Hi Ian,

ropers wrote on Wed, Jul 10, 2019 at 03:03:08AM +0200:

> I intend to write feature "altnumd" and would like advice for how to
> go about it.

My opinion is that we don't have too few methods for entering
characters.  One could argue that we already have too many,
and the existing ones are too complicated.  So writing yet
another system or daemon sounds like a bad idea to me.

What might be useful is understanding all the details of how
all the existing methods in ws* and X* (and possibly elsewhere?)
work, then simplify them.

I'm not aware of a simple method of entering Unicode codepoints 
numerically that works everywhere, and i find that lack annoying.

Yours,
  Ingo

Re: When will OpenBSD become a friendly place for bug reporters?

[Consus]

On 18:56 Tue 09 Jul, Roderick wrote:
> 
> On Tue, 9 Jul 2019, cho...@jtan.com wrote:
> 
> > Perhaps rather than whining that OpenBSD lacks some specific feature,
> > those who want it could write it?
> 
> Or perhaps better not. All depends on what is a feature and for whom.
> 
> I, as normal user, am glad that packages are not inflated with debugging
> symbols.

That's why redhat and others offer *-debuginfo packages with DWARF
symbols. It's really helpful. It would be nice to have such in OpenBSD,
especially for base, because rebuilding something on my router is not
something I would like to do.

Re: When will OpenBSD become a friendly place for bug reporters?

[ropers]

On 09/07/2019, cho...@jtan.com  wrote:
> Perhaps rather than whining that OpenBSD lacks some specific feature, those
> who want it could write it? A novel idea, I know, but it IS specifically a
> development platform and there are precisely zero restrictions.
>
> Or if you don't wish to start with code, at least try a tack such as "I
> intend to write feature $foo and would like advice for how to go about it".

I intend to write feature "altnumd" and would like advice for how to
go about it.

To elaborate:
  Many older DOS/Windows users still have an old special character
input method in muscle memory that goes back all the way to the IBM
5150[0]: Alt+Numpad[1].
Meaning, hold Alt, and then on the number pad, type a decimal code for
a character in order to insert that character into the (originally
BIOS-)keyboard buffer, from which it is off to the races, to be picked
up by whoever is grabbing characters off the keyboard buffer.
I'm explaining this, because AFAIK that feature has never existed on
Unix-like OSes, and I think--corrections welcome--the BIOS isn't even
involved when OpenBSD talks to the keyboard or vice versa.
I am very aware of the Compose key support in X11, and this isn't
about that. This isn't about inputting characters in a good or better
way. This is about inputting characters in exactly THAT way
(Alt+Numpad), "just like mammy used to".
  Alt+Numpad was originally a BIOS feature for the Single-Byte
Character Set[2] DOS code pages[3] where--unlike in UTF-8 today--there
was no difference between code units and code points[4]. Every one of
the then at most 256 characters on the currently configured code page
was at users' fingertips with Alt+0 thru Alt+255.[5]
  At least part of reimplementing Alt+Numpad support on OpenBSD (as
3rd-party software; I don't expect this to become an OS feature)
should be possible in that these hotkey programs seem able to perform
fairly arbitrary actions in response to configurable key sequences:
, . I have
not yet explored this enough and not yet gotten them to respond to
Alt+Numpad, but I suspect that may be possible.
  However, I don't have a sufficient understanding of the basics of
OpenBSD/console/X11 keyboard support yet. I would like "altnumd" to
work the same or as similar as possible on both console and X11. My
very deficient understanding of OpenBSD non-BIOS keyboard support is
this: I. keypresses on the physical (USB/PS2) keyboard. I think the
actual keyboard has an internal microcontroller and small buffer, but
I don't think that's accessible to write back to. --> II. Low-level OS
keyboard device support. Presumably there's some sort of buffer there
that could be written to, given permission? --> III. At some point
things connect to console and/or X11, and then programs themselves
have their buffers too. Is wscons always involved? Is that the lowest
one can go and can that be written back to? I'm unsure. Essentially,
after registering the keypress sequence, which existing hotkey
programs seem up to, I would need to write something similar to a
software keyboard emulator. Apparently there used to be this TIOCSTI
thing that maybe could help, but this was removed for security
reasons?[6] Is there an alternative?
  Also, it's not even quite clear to me where along the line from
keyboard to program, the scan codes turn into code units/code points
and characters. I.e., assuming the use of UTF-8 at the end of the day,
once I've registered the right code for, say, the NOT character
(Alt+170 on DOS CP437 or Alt+172 on Win/CP1252 and OpenBSD's old
ISO-8859-1), once I've registered that, what goes into the buffer?
0xAA (170) or 0xAC (172), or UTF-8's ready-made 0xC2 0xAC (=U+00AC)?
(Speaking of multiple bytes, a "stretch goal" might be the option to
insert an entire string in response to an Alt code, but that's gravy
and not the old muscle memory.)

Yes, I seriously would like to do this if I can get good enough, from
zero to not-quite-hero. Yes, I would very much appreciate help and any
useful pointers. On-list, off-list, elsewhere, anything, from anyone.

Ian

PS: That said, nobody bet on me succeeding, unless you like losing money.

[0] 
[1] 
[2] 
[3] 
[4] :
"The compromise solution that was eventually found and developed into
Unicode was to break the assumption (dating back to telegraph codes)
that each character should always directly correspond to a particular
sequence of bits. Instead, characters would first be mapped to a
universal intermediate representation in the form of abstract numbers
called __code points__. Code points would then be represented in a
variety of ways and with various default numbers of bits per character
(__code units__) depending o

Re: When will OpenBSD become a friendly place for bug reporters?

[Chris Humphries]

Marc and Stuart (and whoever else was involved), thanks for the work
done on this stuff. It is simple and clean for me as a user and I have
trust in the integrity of what I'm receiving. I don't take it for
granted.

And there's probably a ton more involved that I'm not aware of yet,
but I trust in you guys to do the right thing (tm) all the time for
users. Trust is a big deal. 


Sadly, the OP could have saved a lot of effort for everyone if they
just would have remembered there's other humans on the other side of
the email. And the fundamentals of communication include if you want
to have a real conversation with someone where they listen to you,
don't come at them guns blazing and emotional.

Thanks for explaining things (for the rest of us that are curious).

Hope you didn't let the emotional outrage of a random person on the
Internet infect you. 




On Tue, Jul 09, 2019 at 08:40:29PM +0200, Marc Espie wrote:
> On Tue, Jul 09, 2019 at 08:04:23PM +0300, Leonid Bobrov wrote:
> > > An all-arches package snapshot currently runs at 200GB and adding
> > > symbols across the board would add a lot to this.
> > 
> > Stuart and Espie, have you ever heard of compression?
> 
> WTF is wrong with you ?
> 
> I haven't participated to that thread yet, but I wager I know more about
> compression than you do.
> 
> Just have a fucking look at the code in signify, the pkg tools, and more
> generally, EVERYWHERE I have touched in BSD before spouting insults like
> this.
> 
> Most specifically:
> - we do use gzip because other compression systems won't work with little
> memory/don't have the right licence.
> - gzip allows you to stick COMMENTS in the header, which is where the
> signature lives. I DID A WHOLE FUCKING PRESENTATION ABOUT THAT DESIGN
> CHOICE.
> - pkg_create uses an LRU cache to make updates faster.  And we use some
> specificities of gzip  to make packages amenable to using rsync actually.
> - Stuart has wasted hours getting mirrors to work as good as they can.
> - *we* have spent hours trying to share stuff while keeping things secure.
> 
> It *still* doesn't change the fact that a full snapshot takes up a lot of
> space.  And it's an important factor in having enough sites provide mirrors.
> 
> It's also an important factor in making sure snapshots are distributed 
> quickly.
> 
> Heck, there are design choices in package snapshots to avoid shearing.
> 
> So, to summarize. Get the hell away from this mailing-list.
> 
> I don't want to have anything to do with a condescending idiot who sends
> disparaging comments my way 100% out of the blue.
> 
> Fuck you very much,
> 
> -- 
>   Marc
> 

-- 
Chris Humphries 
5223 9548 E1DE DE87 F509  1888 8141 8451 6338 DD29

Re: When will OpenBSD become a friendly place for bug reporters?

[Nathan Hartman]

On Tue, Jul 9, 2019 at 2:43 PM Marc Espie  wrote:

> - we do use gzip because other compression systems won't work with little
> memory/don't have the right licence.


You might find this interesting:

https://github.com/silentbicycle/heatshrink

Re: When will OpenBSD become a friendly place for bug reporters?

[Roderick]

On Tue, 9 Jul 2019, cho...@jtan.com wrote:


Perhaps rather than whining that OpenBSD lacks some specific feature,
those who want it could write it?


Or perhaps better not. All depends on what is a feature and for whom.

I, as normal user, am glad that packages are not inflated with debugging
symbols.

Rod.

Re: When will OpenBSD become a friendly place for bug reporters?

[Marc Espie]

On Tue, Jul 09, 2019 at 08:04:23PM +0300, Leonid Bobrov wrote:
> > An all-arches package snapshot currently runs at 200GB and adding
> > symbols across the board would add a lot to this.
> 
> Stuart and Espie, have you ever heard of compression?

WTF is wrong with you ?

I haven't participated to that thread yet, but I wager I know more about
compression than you do.

Just have a fucking look at the code in signify, the pkg tools, and more
generally, EVERYWHERE I have touched in BSD before spouting insults like
this.

Most specifically:
- we do use gzip because other compression systems won't work with little
memory/don't have the right licence.
- gzip allows you to stick COMMENTS in the header, which is where the
signature lives. I DID A WHOLE FUCKING PRESENTATION ABOUT THAT DESIGN
CHOICE.
- pkg_create uses an LRU cache to make updates faster.  And we use some
specificities of gzip  to make packages amenable to using rsync actually.
- Stuart has wasted hours getting mirrors to work as good as they can.
- *we* have spent hours trying to share stuff while keeping things secure.

It *still* doesn't change the fact that a full snapshot takes up a lot of
space.  And it's an important factor in having enough sites provide mirrors.

It's also an important factor in making sure snapshots are distributed 
quickly.

Heck, there are design choices in package snapshots to avoid shearing.

So, to summarize. Get the hell away from this mailing-list.

I don't want to have anything to do with a condescending idiot who sends
disparaging comments my way 100% out of the blue.

Fuck you very much,

-- 
Marc

Re: When will OpenBSD become a friendly place for bug reporters?

[chohag]

Perhaps rather than whining that OpenBSD lacks some specific feature, those who 
want it could write it? A novel idea, I know, but it IS specifically a 
development platform and there are precisely zero restrictions.

Or if you don't wish to start with code, at least try a tack such as "I intend 
to write feature $foo and would like advice for how to go about it". Notice 
that the act of _writing actual code_ is still implied.

I imagine that if a patch came through which adapted the build/release process 
such that symbols weren't removed but extracted into their own set for post-hoc 
installation by interested individuals, for example, that it would at least 
receive discussion if not eventual inclusion.

The bitching and public masturbation in this and the recent X thread, among 
many other examples, helps no-one.

Matthew

Re: When will OpenBSD become a friendly place for bug reporters?

[Theo de Raadt]

Please leave the list.



Leonid Bobrov  wrote:

> > It is definately not a friendly place for people with a tone like yours.
> 
> Theo, your excuse that OpenBSD is not more popular than Linux because AT&T
> sued BSD in 90's is ridiculous, that's your own fault for being so
> terrible in technical field, also you are terrible person, just like
> me you can't communicate, so why do you think you can teach me
> communication? If my tone is that important you prefer ignoring important
> topic then you are even more terrible in terms of both personality and
> technical field.
> 
> > An all-arches package snapshot currently runs at 200GB and adding
> > symbols across the board would add a lot to this.
> 
> Stuart and Espie, have you ever heard of compression?
> 
> Again, what's wrong with my tone? I can't elaborate my thoughts in
> different ways, also my tone only bothers people in this community,
> most other communities don't see anything wrong about my way of speaking.
> 
> Anyway, Stuart suggests a really good solution: detaching symbols into
> subpackages, this practice is already used in Void Linux, I downloaded
> 1 GB of compressed archives and they expanded to 27 GB, so maybe you
> can learn how to use compression or at least switch packages to proper
> compression method.

Re: When will OpenBSD become a friendly place for bug reporters?

[Christopher Turkel]

Stop it now. You are being a troll.

On Tue, Jul 9, 2019 at 1:09 PM Leonid Bobrov  wrote:

> > It is definately not a friendly place for people with a tone like yours.
>
> Theo, your excuse that OpenBSD is not more popular than Linux because AT&T
> sued BSD in 90's is ridiculous, that's your own fault for being so
> terrible in technical field, also you are terrible person, just like
> me you can't communicate, so why do you think you can teach me
> communication? If my tone is that important you prefer ignoring important
> topic then you are even more terrible in terms of both personality and
> technical field.
>
> > An all-arches package snapshot currently runs at 200GB and adding
> > symbols across the board would add a lot to this.
>
> Stuart and Espie, have you ever heard of compression?
>
> Again, what's wrong with my tone? I can't elaborate my thoughts in
> different ways, also my tone only bothers people in this community,
> most other communities don't see anything wrong about my way of speaking.
>
> Anyway, Stuart suggests a really good solution: detaching symbols into
> subpackages, this practice is already used in Void Linux, I downloaded
> 1 GB of compressed archives and they expanded to 27 GB, so maybe you
> can learn how to use compression or at least switch packages to proper
> compression method.
>
>

Re: When will OpenBSD become a friendly place for bug reporters?

[Raul Miller]

On Tue, Jul 9, 2019 at 1:13 PM Leonid Bobrov  wrote:
> Theo, your excuse that OpenBSD is not more popular than Linux because AT&T
> sued BSD in 90's is ridiculous,

Nah, it's a relevant issue.

That said, it's not the only issue, which I imagine was the point you
were trying to get across.

-- 
Raul

Re: When will OpenBSD become a friendly place for bug reporters?

[Leonid Bobrov]

> It is definately not a friendly place for people with a tone like yours.

Theo, your excuse that OpenBSD is not more popular than Linux because AT&T
sued BSD in 90's is ridiculous, that's your own fault for being so
terrible in technical field, also you are terrible person, just like
me you can't communicate, so why do you think you can teach me
communication? If my tone is that important you prefer ignoring important
topic then you are even more terrible in terms of both personality and
technical field.

> An all-arches package snapshot currently runs at 200GB and adding
> symbols across the board would add a lot to this.

Stuart and Espie, have you ever heard of compression?

Again, what's wrong with my tone? I can't elaborate my thoughts in
different ways, also my tone only bothers people in this community,
most other communities don't see anything wrong about my way of speaking.

Anyway, Stuart suggests a really good solution: detaching symbols into
subpackages, this practice is already used in Void Linux, I downloaded
1 GB of compressed archives and they expanded to 27 GB, so maybe you
can learn how to use compression or at least switch packages to proper
compression method.

Re: When will OpenBSD become a friendly place for bug reporters?

[Theo de Raadt]

mazoc...@disroot.org wrote:

> Hi!
> 
> We all know that bugs don't get fixed without backtraces.
> 
> After few years of using OpenBSD I am annoyed to get mocked for not
> sending backtraces, but why I don't send them? The answer is: OpenBSD
> doesn't provide software packages with debugging symbols.
> 
> Do I look like a Gentoo user? It's not cool to leave no choice to bug
> reporters but to make them rebuild all ports they use with:
> $ env CFLAGS='-pipe -g' DEBUG=-g make -j $(sysctl -n hw.ncpu) reinstall
> 
> The current OpenBSD is definetely not friendly to bug reporters, so
> don't blame me when I refuse to send backtraces, I am simply not in mood
> to rebuild software when it shouldn't be necessary, I value my time.

It is definately not a friendly place for people with a tone like
yours.  Just read what is written above.  Like who do you think would
respond positively to what is written above?

Re: When will OpenBSD become a friendly place for bug reporters?

[STeve Andre']

On 7/8/19 10:57 PM, mazoc...@disroot.org wrote:

Hi!

We all know that bugs don't get fixed without backtraces.

After few years of using OpenBSD I am annoyed to get mocked for not
sending backtraces, but why I don't send them? The answer is: OpenBSD
doesn't provide software packages with debugging symbols.

Do I look like a Gentoo user? It's not cool to leave no choice to bug
reporters but to make them rebuild all ports they use with:
$ env CFLAGS='-pipe -g' DEBUG=-g make -j $(sysctl -n hw.ncpu) reinstall

The current OpenBSD is definetely not friendly to bug reporters, so
don't blame me when I refuse to send backtraces, I am simply not in mood
to rebuild software when it shouldn't be necessary, I value my time.


For heavens sake, why don't you compile the code with symbols?  If you
have the ability to go inside and look for problems, you can compile
stuff yourself.  If you're going to submit a patch you have to build
to test the fix!

--STeve Andre'

When will OpenBSD become a friendly place for bug reporters?

[mazocomp]

Hi!

We all know that bugs don't get fixed without backtraces.

After few years of using OpenBSD I am annoyed to get mocked for not
sending backtraces, but why I don't send them? The answer is: OpenBSD
doesn't provide software packages with debugging symbols.

Do I look like a Gentoo user? It's not cool to leave no choice to bug
reporters but to make them rebuild all ports they use with:
$ env CFLAGS='-pipe -g' DEBUG=-g make -j $(sysctl -n hw.ncpu) reinstall

The current OpenBSD is definetely not friendly to bug reporters, so
don't blame me when I refuse to send backtraces, I am simply not in mood
to rebuild software when it shouldn't be necessary, I value my time.