Re: [9fans] [PATCH] fossil: fix a deadlock in the caching logic

[Steffen Nurpmeso]

Dan Cross wrote in
 :
 |On Sat, Apr 8, 2023 at 10:37 AM Charles Forsyth
 | wrote:
 |> It was the different characteristics of hard drives, even decent \
 |> SATA, compared to SSD and nvme that I had in mind.
 ...
 |In short, when you change storage technologies, assumptions that were
 |made with, say, a filesystem was initially written may be invalidated.
 |Consider the BSD FFS for example: UFS was written in an era of VAXen
 |and slow, 3600 RPM spinning disks like RA81s attached to relatively
 |unintelligent controllers; it made a number of fundamental design
 |decisions based on that, trying to optimize placement of data and
 |metadata near each other (to minimize head travel--this is the whole
 |cylinder group thing), implementation that explicitly accounted for
 |platter rotation with respect to scheduling operations for the
 |underlying storage device, putting multiple copies of the superblock
 |in multiple locations in the disk to maximize the chances of recovery
 |in the event of the (all-too-common) head crashes of the era, etc.
 |They also did very careful ordering of operations for soft-updates in
 |UFS2 to ensure filesystem consistency when updating metadata in the
 |face of a system crash (or power failure, or whatever). It turns out
 |that many of those optimizations become pessimizations (or at least
 |irrelevant) when you're all of a sudden writing to a solid-state
 |device, nevermind battery-backed DRAM on a much more advanced
 |controller.

Funnily Kirk McKusick committed on March 29th
fe5e6e2cc5d6f2e4121eccdb3a8ceba646aef2c9, saying

Improvement in UFS/FFS directory placement when doing mkdir(2).

The algorithm for laying out new directories was devised in the 1980s
and markedly improved the performance of the filesystem. In those days
large disks had at most 100 cylinder groups and often as few as 10-20.
Modern multi-terrabyte disks have thousands of cylinder groups. The
original algorithm does not handle these large sizes well. This change
attempts to expand the scope of the original algorithm to work well
with these much larger disks while still retaining the properties
of the original algorithm for small disks.
...
This change updates the ffs_dirpref() routine which is responsible
for selecting the cylinder group into which a new directory should
be placed. If we are near the root of the filesystem we aim to
spread them out as much as possible. As we descend deeper from the
root we cluster them closer together around their parent as we
expect them to be more closely interactive. Higher-level directories
like usr/src/sys and usr/src/bin should be separated while the
directories in these areas are more likely to be accessed together
so should be closer. And directories within commands or kernel
subsystems should be closer still.

We pick a range of cylinder groups around the cylinder group of the
directory in which we are being created. The size of the range for
our search is based on our depth from the root of our filesystem.
We then probe that range based on how many directories are already
present. The first new directory is at 1/2 (middle) of the range;
the second is in the first 1/4 of the range, then at 3/4, 1/8, 3/8,

I only took a shallow look as i have no glue, but is sprung into
my eyes so i remembered it.

--steffen
|Der Kragenbaer,The moon bear,
|der holt sich munter   he cheerfully and one by one
|einen nach dem anderen runter  wa.ks himself off
|(By Robert Gernhardt)

--
9fans: 9fans
Permalink: 
https://9fans.topicbox.com/groups/9fans/T354fe702e1e9d5e9-Mc6cf9913394dadc41af96812
Delivery options: https://9fans.topicbox.com/groups/9fans/subscription

Re: [9fans] APL

[Steffen Nurpmeso]

Lyndon Nerenberg (VE7TFX/VE6BBM) wrote in
 <0903a00d50966...@orthanc.ca>:
 |Steffen Nurpmeso writes:
 |> It can even be as small as 
 |>
 |>   #?0|kent:unix-hist$ du -sh .
 |>   179M.
 |>
 |> when not including all the new FreeBSD things (for which i at
 |> least track the FreeBSD git repository directly):

Traffic size is a real issue for me.
(As is quality of the rtw88 driver of Linux 5.10.*, as is the fact
that git i think still cannot resume failed clones.  I anyway had
lots and lots of trouble and yes grief due to this, here.)

 |Okay, so what's the magic incantation to clone just that subset
 |of branches?  git-clone(1) is not helpful ...

Backward compatible for "the one real git" is

 $ cd DIR; git init
 $ git remote add origin -t BRANCH1 -t BRANCH2 -t 'release/*' URL
 $ git fetch -v

Or git init and then copy the snippet :)
(Mind you, just a few weeks ago on FreeBSD it turned out that
i should re-learn git from scratch.  I turned to it around
2010/11, wrote some scripts and aliases, and unless they break,
for example due to rev-list reverting output in about 2013, i have
a very basic way of doing, lots of update-ref and such, for
example.)

And sorry for the late reply, after weeks of -11° Celsius and
months of winter we had 31° more yesterday, including sunshine,
and i went for cycling.  Then someone reported a brain-damage of
mine in software i maintain, and i had to make a release, and then
it was about 3 o'clock in the morning.

--steffen
|Der Kragenbaer,The moon bear,
|der holt sich munter   he cheerfully and one by one
|einen nach dem anderen runter  wa.ks himself off
|(By Robert Gernhardt)

--
9fans: 9fans
Permalink: 
https://9fans.topicbox.com/groups/9fans/T476a1d7b83269775-Mded9eca8705205b15d2110ca
Delivery options: https://9fans.topicbox.com/groups/9fans/subscription

Re: [9fans] APL

[Steffen Nurpmeso]

Bakul Shah wrote in
 <5c11a3ac-a66c-4336-bf37-55db2c4eb...@iitbombay.org>:
 |Spinellis has put together a browsable repo based on various source \
 |distributions
 |which I find useful. I keep a local copy as it is under 2GB. All I \

It can even be as small as 

  #?0|kent:unix-hist$ du -sh .
  179M.

when not including all the new FreeBSD things (for which i at
least track the FreeBSD git repository directly):

  #?0|kent:unix-hist$ cat .git/config
  [core]
  repositoryformatversion = 0
  filemode = true
  bare = false
  logallrefupdates = true
  precomposeunicode = true
  [remote "origin"]
  url = https://github.com/dspinellis/unix-history-repo.git
fetch = 
+refs/heads/386BSD-0.0-Snapshot-Development:refs/remotes/origin/386BSD-0.0-Snapshot-Development
fetch = 
+refs/heads/386BSD-0.1-Snapshot-Development:refs/remotes/origin/386BSD-0.1-Snapshot-Development
fetch = 
+refs/heads/386BSD-0.1-patchkit:refs/remotes/origin/386BSD-0.1-patchkit
fetch = 
+refs/heads/386BSD-0.1-patchkit-Import:refs/remotes/origin/386BSD-0.1-patchkit-Import
fetch = +refs/heads/386BSD-Release:refs/remotes/origin/386BSD-Release
fetch = 
+refs/heads/BSD-1-Snapshot-Development:refs/remotes/origin/BSD-1-Snapshot-Development
fetch = 
+refs/heads/BSD-2-Snapshot-Development:refs/remotes/origin/BSD-2-Snapshot-Development
fetch = 
+refs/heads/BSD-3-Snapshot-Development:refs/remotes/origin/BSD-3-Snapshot-Development
fetch = 
+refs/heads/BSD-4-Snapshot-Development:refs/remotes/origin/BSD-4-Snapshot-Development
fetch = 
+refs/heads/BSD-4_1_snap-Snapshot-Development:refs/remotes/origin/BSD-4_1_snap-Snapshot-Development
fetch = 
+refs/heads/BSD-4_1c_2-Snapshot-Development:refs/remotes/origin/BSD-4_1c_2-Snapshot-Development
fetch = 
+refs/heads/BSD-4_2-Snapshot-Development:refs/remotes/origin/BSD-4_2-Snapshot-Development
fetch = 
+refs/heads/BSD-4_3-Snapshot-Development:refs/remotes/origin/BSD-4_3-Snapshot-Development
fetch = 
+refs/heads/BSD-4_3_Net_1-Snapshot-Development:refs/remotes/origin/BSD-4_3_Net_1-Snapshot-Development
fetch = 
+refs/heads/BSD-4_3_Net_2-Snapshot-Development:refs/remotes/origin/BSD-4_3_Net_2-Snapshot-Development
fetch = 
+refs/heads/BSD-4_3_Reno-Snapshot-Development:refs/remotes/origin/BSD-4_3_Reno-Snapshot-Development
fetch = 
+refs/heads/BSD-4_3_Tahoe-Snapshot-Development:refs/remotes/origin/BSD-4_3_Tahoe-Snapshot-Development
fetch = 
+refs/heads/BSD-4_4-Snapshot-Development:refs/remotes/origin/BSD-4_4-Snapshot-Development
fetch = 
+refs/heads/BSD-4_4_Lite1-Snapshot-Development:refs/remotes/origin/BSD-4_4_Lite1-Snapshot-Development
fetch = 
+refs/heads/BSD-4_4_Lite2-Snapshot-Development:refs/remotes/origin/BSD-4_4_Lite2-Snapshot-Development
fetch = +refs/heads/BSD-Release:refs/remotes/origin/BSD-Release
fetch = +refs/heads/BSD-SCCS:refs/remotes/origin/BSD-SCCS
fetch = 
+refs/heads/Bell-32V-Snapshot-Development:refs/remotes/origin/Bell-32V-Snapshot-Development
fetch = +refs/heads/Bell-Release:refs/remotes/origin/Bell-Release
fetch = +refs/heads/Epoch:refs/remotes/origin/Epoch
fetch = 
+refs/heads/Research-PDP7-Snapshot-Development:refs/remotes/origin/Research-PDP7-Snapshot-Development
fetch = +refs/heads/Research-Release:refs/remotes/origin/Research-Release
fetch = 
+refs/heads/Research-V1-Snapshot-Development:refs/remotes/origin/Research-V1-Snapshot-Development
fetch = 
+refs/heads/Research-V2-Snapshot-Development:refs/remotes/origin/Research-V2-Snapshot-Development
fetch = 
+refs/heads/Research-V3-Snapshot-Development:refs/remotes/origin/Research-V3-Snapshot-Development
fetch = 
+refs/heads/Research-V4-Snapshot-Development:refs/remotes/origin/Research-V4-Snapshot-Development
fetch = 
+refs/heads/Research-V5-Snapshot-Development:refs/remotes/origin/Research-V5-Snapshot-Development
fetch = 
+refs/heads/Research-V6-Snapshot-Development:refs/remotes/origin/Research-V6-Snapshot-Development
fetch = 
+refs/heads/Research-V7-Snapshot-Development:refs/remotes/origin/Research-V7-Snapshot-Development
fetch = 
+refs/heads/usr/src/BSD-SCCS-Import:refs/remotes/origin/usr/src/BSD-SCCS-Import

--steffen
|
|Der Kragenbaer,The moon bear,
|der holt sich munter   he cheerfully and one by one
|einen nach dem anderen runter  wa.ks himself off
|(By Robert Gernhardt)

Re: [9fans] Compiling ken-cc on Linux

[Steffen Nurpmeso]

erik quanstrom  wrote:
 |> memory bug in Jörg Schilling's Bourne shell (likely developed only
 |> on Solaris rooted) simply by compiling and starting it under
 |> FreeBSD.  And i have found stack read violations simply by running
 |
 |given cdrtools, this is not a surprise.  
 | https://en.wikipedia.org/wiki/Cdrtools#Device_naming
 |"all the world's a vax^h^h^hscsi"

..hm, i have read his dissertation some years ago, from the end of
the 80s?, in which he translates the SunOS operating system manual
to German for almost a hundred pages?

--steffen

Re: [9fans] Compiling ken-cc on Linux

[Steffen Nurpmeso]

arn...@skeeve.com wrote:
 |> Alternative compilers, like tcc, only build C on very few architectures /
 |> os with almost no optimization: they are much smaller, but still not
 |> standard compliant.

 |TCC compiles really fast, and it's (finally) good enough that I can use
 |it for my personal development / testing, but I would not use it to
 |build a production binary, the code quality is much poorer. On Linux
 |you can't use it for debugging either - it doesn't generate the
 |debug info you need. :-(  For that, GCC and clang are the way to go.

I just started to use tcc(1) for faster recompilations during my
chaotic way of walking, and it was an a-ha just as was the
detection of jikes(1) compared to the Java-shipped compiler in,
say, 1999.  (And in fact that was the reason to start learning
C++, later C, and not continue with only Perl and Java.)

  ?2[sdaoden@wales ]$ time make CC=clang devel >/dev/null 2>&1
  0m55.37s real 0m29.44s user 0m27.24s system
  ?0[sdaoden@wales ]$ time make CC=gcc devel >/dev/null 2>&1
  0m50.04s real 0m26.13s user 0m24.92s system
  ?2[sdaoden@wales ]$ time make CC=tcc devel >/dev/null 2>&1
  0m17.90s real 0m7.29s user 0m12.07s system

But without reconfiguration in normal development it feels more
drastic than these lines suggest.  Of course, ship-out user code.
Plan9 users don't have those problems in their native
self-contained environment, also the UnixWare and native Sun cc's
are ok, but on *BSD and Linux you really don't have any more
options.  E.g., i just tried pcc(1), which could be one, on
a Linux box:

  Using C compiler $CC="pcc"
  /usr/include/stdc-predef.h:59: warning: __STDC_ISO_10646__ redefined 
(previously defined at "./___tmp12443.c" line -9)
  ld: cannot find crtbegin.o: No such file or directory
well we could but..
  ld: cannot find -lpcc
there is none!
  error: ld terminated with status 1
  ERROR: i cannot compile a "Hello world" via
 pcc -I/usr/lib/gcc/x86_64-unknown-linux-gnu/5.2.0/include/ 
-I/home/sdaoden/usr/include -I/usr/local/include -I/usr/include  
-L/home/sdaoden/usr/lib -L/lib -L/usr/local/lib -L/usr/lib

That after some work already.  VoidLinux has a package, though,
i should try that.

 |I agree with the general sentiments - GLIBC and GCC are both bloated.
 |But for the day-to-day work that *I* do, they're livable.

Due to the large community and the commercial interest (remember
the billion dollar campaign of IBM over a decade ago) you have the
newest technologies in a usable state.  I.e., i've found a drastic
memory bug in Jörg Schilling's Bourne shell (likely developed only
on Solaris rooted) simply by compiling and starting it under
FreeBSD.  And i have found stack read violations simply by running
them under Linux?  I always had my own memory allocation stuff
with tracing and logging and canaries, but read violations are
hard to detect without real red zones.  And it's easy to simply
look into /proc/PID/ and see a lot of things at a glance via ls(1)
/ cat(1) that otherwise require special programs or things which
are real horror for me, like dtrace (no!).  Acid has it at least
in its name.  Never liked debug robots anyway.

But otherwise i always like going back to BSD, i guess just Plan9
people like to come back into the grown rather self-contained
environment.  You install the ISO and are ready to go, with
network and development.  I.e., i install a new FreeBSD and the
configuration files from 2002 still work.  That is valuable.
I think there is pcc for NetBSD, also.

--steffen

Re: [9fans] Compiling ken-cc on Linux

[Steffen Nurpmeso]

 |How they ever got it going on a system with 64Kbytes of address space\
 |, I'll never know.

Yeah!

--steffen

Re: [9fans] Undefined Behaviour in C

[Steffen Nurpmeso]

plann...@sigint.cs.purdue.edu wrote:
 |On Wed, Nov 25, 2015 at 09:25:55AM -0500, Brantley Coile wrote:
 |> unindented consequences
 |
 |Is that a class of Python bugs or an awesome name for a Nerdcore band?

all that new C stuff and sequencing is shit.  imho :)

--steffen

Re: [9fans] off topic - a good Git reference

[Steffen Nurpmeso]

erik quanstrom  wrote:
 |On Wed Oct  7 14:25:58 PDT 2015, elbingm...@gmail.com wrote:
 |in the case of system crash, fsync doesn't provide strong \
 |guarantees that the
 |write will not be lost, or the fs not corrupted even in li\
 |nux.  some versions of
 |the linux kernel do nothing on fsync, fsync doesn't actual\

Linux fsync(2) says that the file itself is fine after it, but for
the filesystem metadata you need an additional fsync(2) on the
directory the file resides in.
The final wording of the related POSIX issue[1] states that the
latter is only necessary if directory entries have been created or
removed (also regarding rename(2) which needs two fsync(2) calls;
luckily we're binary only), and noone has opposed to this wording
in more than two years.
(Mac OS X Snow Leopard had a F_FULLFSYNC fcntl(2) which provided
the fully-synced guarantee, as in
_os_fsync = os.fsync
def __fullfsync(fd):
if not SAFE_FSYNC:
return _os_fsync(fd)
try:
fcntl.fcntl(fd, fcntl.F_FULLFSYNC)
except IOError as e:
if e.errno == errno.ENOTTY:
return _os_fsync(fd)
raise
os.fsync = __fullfsync
which i thought of as a good idea, from a programmers point of
view who wants to get it done.)

  [1] http://austingroupbugs.net/view.php?id=672#c1618

--steffen

Re: [9fans] off topic - a good Git reference

[Steffen Nurpmeso]

Jeff Sickel  wrote:
 |> On Sep 30, 2015, at 2:59 AM, Charles Forsyth  rs...@gmail.com> wrote:
 |> 
 |> Here's a small but representative example.
 |
 |That’s just an example of a C file.  The joy of Git is the\
 | rest of the requirements to actually use it:
 |
 |- bash

Isn't a POSIX shell enough for the core?  I only have mksh at hand
as an alternative right now, but that looks good (on my old box
i think that was /bin/sh, too):

  ?0[sdaoden@wales git.git-no_reduce]$ for i in *.sh; do mksh -n $i;done
  ?0[sdaoden@wales git.git-no_reduce]$

 |- perl5

That seems to have been a temporary oops from a Mr. Eric Sunshine
who messed up git(1) circa at the end of June; i've ran into this,
but someone else just did one or two days before me (in august?),
so this really was the replacement of a working sh/sed/awk thing
with the same in perl, blowing the NO_PERL configuration option
for two months (and multiple releases).

The shown git reference is really nice and really, really nice to
look at; i've thrown away my progit chapters 6 and 9 (or so).

--steffen

Re: [9fans] Ports tree for Plan 9

[Steffen Nurpmeso]

cinap_len...@felloff.net wrote:
 |found it. the server sends Content-Encoding header which causes hget
 |to add a decompression filter, so you get as output a tarball.
 |
 |<- Content-Type: application/x-gzip
 |<- Content-Encoding: gzip

 |

 |this is clearly silly, as the file is already compressed, \
 |and decompressing it
 |will not yield the indicated content-type: application/x-gzip, \
 |but a tarball.
 |
 |maybe the w3c is wrong, or is ignored in practice or we need to handle gzip
 |specially. the problem is that some webservers compress the \

The problem is that IANA doesn't support a tar-gz MIME type, so
that mime.types(5) (tika [1] for Apache) will return "silly"
values, as in

  application/gzip  tgz gz emz
  application/x-bzip2   bz2 tbz2 boz
  # EXTENSION .tbz
  application/x-xz  xz tbz
  application/x-tar tar

  [1] 
http://svn.apache.org/viewvc/tika/trunk/tika-core/src/main/resources/org/apache/tika/mime/tika-mimetypes.xml

 |data, like you request
 |a html file and it gives you gzip back, thats why hget uncompresses.

mime.types(5) (re-)evaluating expanded content seems what IANA has
in mind with its decision (it would be all too simple if it would
just work (tm)).

--steffen

Re: [9fans] xz compression?

[Steffen Nurpmeso]

Have you considered MiniLZO [1]?
I think it *can* do the format, and it should be much easier to
port than xz(1) itself.

  [1] http://www.oberhumer.com/opensource/lzo/

--steffen
--- Begin Message ---
On Wednesday 14 January 2015 08:08:23 lu...@proxima.alt.za wrote:
> Even though at the back of my mind there is a nagging desire to
> implement gccgo in a Plan 9 fashion (for OpenLDAP, if anyone cares), I
> think the bccgo approach, useful as it is, should be limited to
> obsolete software.  But that's philosophy, rather than pragmatism.

There is a new-ish GCC port (4.8.3) to Plan9 made by Alvaro
http://marcus.biz.tm/jail/comment?threadid=62&title=GNU+port+for+Plan9+upgraded
http://marcus.biz.tm/jail/comment?threadid=64&title=GCC+4.8.3+and+Lighttpd+in+Plan9

I mirrored one older (gcc 4.5) version at
http://plan9.bell-labs.com/sources/contrib/staal1978/gcc/

Basically it is the APE libs built with GCC. It should apparently support many 
languages (it does not mention which ones). It might be possible to use to 
rebuild GCC that supports go (if it is not included by default). The new 
version uses another directory structure, but similar to regular APE it 
basically is a sepparate environment from the typical Plan9 (probably to treat 
more like linuxemu rather than as APE since GCC-compiled libraries can not be 
mixed with the native system libraries).

I have personally looked at kerberos/heimdall + ldap before... do not remember 
for what and then I tried building in "vanilla" APE (I think it was for 
quagga). I have also had thoughts about a Tor client (needs libevent) and/or 
one of the distributed DNS things. Right now no time to play however :(

Now that I saw this updated 4.8 port of GCC I am  eager to play around with it 
again some time...


--- End Message ---

Re: [9fans] plan 9 on github

[Steffen Nurpmeso]

David du Colombier <0in...@gmail.com> wrote:
 |I've recently converted my archives of /n/sources/plan9
 |to Git and uploaded the repository on GitHub.
 |
 |https://github.com/0intro/plan9

Thank you.

 |It contains the entire history of /n/sources/plan9,
 |one commit per day, from December 12, 2002 to today,
 |excluding the binaries and empty directories.

You could have used --aggressive garbage collection, it reduces
the package to exactly 100MB.

 |I tried to make it as accurate as possible with the
 |exact content, dates and permissions.

 |Please let me know if you notice any mistake.
 |
 |I think this repository could be used as a reference
 |for people interested in the history of Plan 9 or
 |wanting to use it as a basis for their own changes.
 |
 |It could also be useful for people wanting to merge
 |Plan 9 code in their projects like plan9port, vx32,
 |drawterm and others.
 |
 |Have fun.

I now have three incarnations of the codebase in my arena, that
gives a lot of precious code for rotting.

--steffen

Re: [9fans] Pjpeg in /sys/lib/mimetype

[Steffen Nurpmeso]

Hello,

arisawa  wrote:
 |Does this mean that file suffixes are case sensitive in determining \
 |the mime type?
 |If so, should files with suffix “.JPG” be renamed to “.jpg” ?

The RFC doesn't say a word on case, which i implicitly read as
case-insensitive (because how could it work otherwise).
(And note that i have forgotten which RFC this actually is, but
i can assure you i've read it before i wrote the code comment that
i've repeated in my first sentence.)
Note also that the pretty comprehensive TIKA database only uses
lowercase definitions [1].

  [1] http://svn.apache.org/viewvc/tika/trunk/tika-core/src/main/resources/\
  org/apache/tika/mime/tika-mimetypes.xml

--steffen

Re: [9fans] atexit() & atexitdont()

[Steffen Nurpmeso]

Charles Forsyth  wrote:
 |On 7 November 2014 10:57, Steffen Nurpmeso  wrote:
 |
 |> Safety against asynchronous un-/registration can't be it, anyway.
 |
 |No, there's a lock. I meant avoiding too many possible interactions between

I thought more about reentrancy because he referred to Linux.
But of course you're right.

 |low-level run-time
 |functions and the rest of the library. (I'd consider atexit and exit to be
 |lower-level functions than malloc.)

I do not really agree for the highly integrated and portable
solution that i worked with, it was just fine there and splitted
into normal and final handlers.  But..

 |Since atexit is already used by profile, and is called by _profmain, which
 |is called very early on,
 |putting a call to malloc in that chain means you have to think that much
 |harder about interactions that are already quite subtle.
 |Note that _profmain allocates its memory directly using sbrk, probably for
 |that reason.
 |Suppose I later want to add a malloc profiler, can I call atexit to write
 |the results, or not?

It seems you can add an atom and it works without spending any
further thought.
That is cool.

--steffen

Re: [9fans] atexit() & atexitdont()

[Steffen Nurpmeso]

Oleg  wrote:
 |On Fri, Nov 07, 2014 at 08:19:05AM +, Charles Forsyth wrote:
 |> On 6 November 2014 21:05, Oleg  wrote:
 |> 
 |>> I looked at atexit() and atexitdont() and i don't understand why these
 |>> functions are implemented with a static array instead of singly linked
 |>> list?

 |> It might have been to avoid malloc in a fairly low-level function (and of
 |> something that will never be freed),
 |
 |If malloc works like in linux (at first invocation allocate more bytes than
 |requested and then each malloc() use this already allocated by kernel area
 |of memory), i think this isn't a big performance impact.

Safety against asynchronous un-/registration can't be it, anyway.

--steffen

Re: [9fans] Setting up Mail in Acme on the Raspberry Pi.

[Steffen Nurpmeso]

erik quanstrom  wrote:
 |On Wed Nov  5 13:20:02 EST 2014, sdao...@yandex.com wrote:
 |> Anthony Sorace  wrote:
 |>|> I've been looking through the documentation and
 |>|> the 9fans archive but I can't get a clear answer on
 |>|> what to replace localhost.localdomain with.
 |>|
 |>|If the recipient's mail server is being strict (but within
 |>|the bounds of the RFCs), that name is expected to be
 |>|the real, externally-resolvable DNS name of the
 |>|system you're sending from. The RFCs used to be more
 |>|lax on that point, and some servers still are, but you
 |>|shouldn't assume you'll be able to send to arbitrary
 |>|endpoints unless you satisfy that.
 |> 
 |> gmail.com shouldn't care at all, so it must be his own SMTP server.
 |> (All i know in respect to this is Yandex.(ru|com), which requires
 |> that the hostname in the SMTP FROM:<> command _is_ a Yandex
 |> address, i.e., _no mismatch_ with _who_ you claim to be, which is
 |
 |that's not what anthony claimed.  he said that if you say
 | HELO example.com
 |that the following must be true
 |(a) dns return an a record for the query example.com, and

Yes -- i think (or say, i'm sure that) gmail.com doesn't take care
for that at all.  Neither does Yandex.  (Never tried any other
free mail provider, i think they all depend on user
authentication.)

 |(b) the ip returned must have a ptr record pointing to example.com
 |(this is less enforced these days due to the difficulty of \
 |maintaining pointer
 |records.)

..So reverse lookups don't even come into play here.
I'm no longer sure wether old-school really hates not to be able
to perform sender verification via DNS, today a lot of pretty
prominent people use those providers themselve.

 |i think this is compatible with what you're saying.  this doesn't make
 |sense to me.  i don't do this:
 |
 |> why i had to invent a *smtp-hostname* variable for the mailer
 |> i maintain in order to address the SMTP FROM:<> content directly:
 |
 |perhaps you're conflating the envelope with the message?

Puh Erik, maybe -- you know, i'm a boche :)
Flying over an official document is the maximum i can handle, just
enough to hammer the most important facts into some long-time
cells, so please excuse possible distortion of terms.
Indeed, looking into RFC 5321 (i have it even in my arena):

   o  The domain name given in the EHLO command MUST be either a primary
  host name (a domain name that resolves to an address RR) or, if
  the host has no name, an address literal, as described in
  Section 4.1.3 and discussed further in the EHLO discussion of
  Section 4.1.4.

   o  The reserved mailbox name "postmaster" may be used in a RCPT
  command without domain qualification (see Section 4.1.1.3) and
  MUST be accepted if so used.

So huch!  SMTP communication how it actually happens in between me
and the public mail providers is invalid, evil and yuck.
I think i just wanted to add some value to what Anthony said.

Regarding *smtp-hostname*: i think one cannot expect from what
i call a normal user to understand just about anything regarding
any protocol etc. internals -- for no other reasons but missing
context information and maybe add lack of interest.  In fact, like
i said above, the same is true for me.  Given that this BSD Mail
derivative already has a variable called *hostname*, and that BSD
/ Linux systems have a hostname(1) command (even though POSIX only
specifies uname(1) and documents "the name of this node within an
implementation-defined communications network"; but POSIX.. well)
i decided to name the capability to overwrite the hostname that is
used in the SMTP "MAIL FROM:<>" command *smtp-hostname* (but not
that the manual is really user friendly sofar).

So now i'm stuck with it.  But since Matt uses Google the address
used in "MAIL FROM:<>" cannot be the problem anyway, since Google
doesn't care wether the addresses in the messages' From: header
and the SMTP "MAIL FROM:<>" command match or not (the last time
i tried; i admit that the Google message i've posted doesn't
really make sense in this context; oops..).

--steffen

Re: [9fans] Setting up Mail in Acme on the Raspberry Pi.

[Steffen Nurpmeso]

Mats Olsson  wrote:
 |I mean: How can I get the DNS name of my machine using Plan 9 on a
 |Raspberry Pi? I've tried the names I can come up with but nada. Please
 |help me out!!!

Well i have no idea what your problem is, sorry :)
Iirc from back in October you already contacted gmail.com, but the
authentication failed, right?  So DNS can't be the problem.  Kurt
H Maier followed the error link, as i did, too.  Try to change the
password to all lowercase ASCII letters and then see if it still
fails:  Maybe encoding via web interface and what gets passed from
within Plan9 is mixed up.  I did manage to setup a machine with an
8-bit password from within the nice installer and then found
myself being unable to log in because of the american keyboard
mapping which didn't produce the necessary keycode.

--steffen

Re: [9fans] Setting up Mail in Acme on the Raspberry Pi.

[Steffen Nurpmeso]

Mats Olsson  wrote:
 |And that could be obtained how?

A german squirrel?
?
Hm... you could try it...
:-))  (for nature.)

But unfortunately, in the modern world...  Most promising seems to
me driving-over by car.  And indeed equally shocking stories Erik's
father (i think he is) has to tell in his dissertation.
One female was also shot by a man.  I have forgotten his name.
All i had to say this man: [1].  Don't you miss _that_!

  [1] 

Ciao!

--steffen

Re: [9fans] Setting up Mail in Acme on the Raspberry Pi.

[Steffen Nurpmeso]

Anthony Sorace  wrote:
 |> I've been looking through the documentation and
 |> the 9fans archive but I can't get a clear answer on
 |> what to replace localhost.localdomain with.
 |
 |If the recipient's mail server is being strict (but within
 |the bounds of the RFCs), that name is expected to be
 |the real, externally-resolvable DNS name of the
 |system you're sending from. The RFCs used to be more
 |lax on that point, and some servers still are, but you
 |shouldn't assume you'll be able to send to arbitrary
 |endpoints unless you satisfy that.

gmail.com shouldn't care at all, so it must be his own SMTP server.
(All i know in respect to this is Yandex.(ru|com), which requires
that the hostname in the SMTP FROM:<> command _is_ a Yandex
address, i.e., _no mismatch_ with _who_ you claim to be, which is
why i had to invent a *smtp-hostname* variable for the mailer
i maintain in order to address the SMTP FROM:<> content directly:

  |?0[steffen@sherwood nail.git]$ echo bla|s-nail -vvdAsn_sf -s du t...@bo.org
  |>>> EHLO yandex.com
  |>>> AUTH PLAIN
  |>>> ...
  |>>> MAIL FROM:
  |>>> RCPT TO:
  |>>> DATA
  |>>> Date: Wed, 05 Nov 2014 19:11:33 +0100
  |>>> From: sdao...@users.sf.net (Steffen Nurpmeso)
  |>>> To: t...@bo.org
  |>>> Subject: du
  |>>> Message-ID: <20141105181133.j3jtd1iu%sdao...@yandex.com>
  |>>> User-Agent: s-nail v14.7.8-70-g9310369
  |>>> MIME-Version: 1.0
  |>>> Content-Type: text/plain; charset=US-ASCII
  |>>> Content-Transfer-Encoding: 7bit
  |>>>
  |>>> bla
  |>>> .
  |>>> QUIT

  |?0[steffen@sherwood nail.git]$ echo bla|s-nail -vvdAsn_gm -s du t...@bo.org
  |>>> EHLO gmail.com
  |>>> STARTTLS
  |>>> EHLO gmail.com
  |>>> AUTH PLAIN
  |>>> ...
  |>>> MAIL FROM:
  |>>> RCPT TO:
  |>>> DATA
  |>>> Date: Wed, 05 Nov 2014 19:11:45 +0100
  |>>> From: Steffen Nurpmeso 
  |>>> To: t...@bo.org
  |>>> Subject: du
  |>>> Message-ID: <20141105181145.kun5nikj%sdao...@gmail.com>
  |>>> User-Agent: s-nail v14.7.8-70-g9310369
  |>>> MIME-Version: 1.0
  |>>> Content-Type: text/plain; charset=US-ASCII
  |>>> Content-Transfer-Encoding: 7bit
  |>>>
  |>>> bla
  |>>> .
  |>>> QUIT)

--steffen

Re: [9fans] Setting up Mail in Acme on the Raspberry Pi.

[Steffen Nurpmeso]

erik quanstrom  wrote:
 |

 |on the dissertation "some
 |aspects of the ethoecology of richardson's ground squirrel" that the two
 |are directly related.

This becomes really interesting.  Squirrels.., these girls and
boys are really a kind of an occupying sort.  Almost addictive.
And emphatic.  That i can understand.  I've downloaded it.

Just a few weeks ago a disaster happened why i had a walk, a young
squirrel girl tried to hide behind a trunk and jumped off to
another tree once i started talking to her (i have seen such
hiding in a much, much better way before), but unfortunately not
taking into account the bending of the trunk at all, falling down
from four or five meters!  Luckily immediately stepping forward
and jumping on her target..  but damn, what a shock we both had.

 |
 |my bad.
 |;-)

If it were like that!!!
Afaik the american squirrel slowly dispels the (smaller) german
Eichhörnchen from the remains of the german forest!
And isn't it hard enough as it is???

Sorry for that unacademic point of view, i have a little "Arche"
in between the Autobahn and railroad tracks where i place some
food for birds, which attracts squirrels and mice too, and it is
ever so surprising how reflective and emphatic all the mentioned
species are.  (In an environment which practically has no more
healthy trees, but only dead and ill ones.)

 |seriously, the error is important.

Plan9 definetely requires will and desire in order to be used by
a normal end-user; but especially if something doesn't work right
out of the box.  Network configuration is really horrible for
fools like me, for example.  Someone should sit down and spend
a year working on that...  imho.

 |p.s. did you  go 3rd person on me?  that's so meta.

Me?  No.  After the first year, only ear tags were used for
permanent identification.

--steffen

Re: [9fans] Setting up Mail in Acme on the Raspberry Pi.

[Steffen Nurpmeso]

erik quanstrom  wrote:

 |that thread's about p9p not plan 9, and i don't see the error \
 |at hand in the output.

Well i do see hints from Erik.  In that thread, that is.
I consider that is.. something..

--steffen

Re: [9fans] Setting up Mail in Acme on the Raspberry Pi.

[Steffen Nurpmeso]

Mats Olsson  wrote:

 |error message keeps coming up. Don't know what to do next. If anyone
 |has a solution to this problem I would be truly grateful to receive
 |it. Even if it's just a hint how to solve it.

Have you read this thread already?

  

--steffen

Re: [9fans] Setting up Mail in Acme on the Raspberry Pi.

[Steffen Nurpmeso]

trebol  wrote:
 |For a external imap server, like gmail, you can compile heirloom's mailx
 |with ape.  Works nice with the plumber, and setting the pager to cat
 |it integrates nice within acme, a rio window or 9term.  I use it also

So you like the bad girls honey,..
Is that true.

--steffen

Re: [9fans] Setting up Mail in Acme on the Raspberry Pi.

[Steffen Nurpmeso]

Bakul Shah  wrote:
 |On Mon, 13 Oct 2014 12:15:31 EDT erik quanstrom  \
 |wrote:
 |> On Sun Oct 12 14:37:47 EDT 2014, st...@quintile.net wrote:
 |>> I am fairly sure the problem is to do with RAM size rather than the ras=
 |> pberry pi per-se.
 |>> 4000 messages takes up a lot of space - and upas stores messages in RAM=
 |> .
 |> 
 |> it's a little worse than this, actually.

 |> the solutions to this are straightforward
 |> (1) store one message per file,
 |
 |This is what MH (an old mail client) does by default.
 |
 |> (2) cache important data in an index to avoid opening all files,
 |
 |This is what dovecot (an imap/pop3 server) does.

Jamie Zawinski implemented that for Netscape 2.0/3.0 [1] but still
went for MBOX format.  But of course that wasn't designed for
Plan9 and permanent backup storage.  The page says

 Right now I'm looking at a folder in 3.0. It has 15,466 messages
 in it. Selecting this folder takes less than a second (it's hard
 to eyeball it, but I'd say it takes about 1/2 to 3/4 second from
 when I click to when I see the message summary on the screen).
 The BSD mbox file is 57.2MB (1.2 million lines) and the summary
 file is 1.3MB (2% of the size of the folder.)

 This is on a P266 with a local IDE disk (Linux.)

  [1] 

--steffen
--- Begin Message ---
On Mon, 13 Oct 2014 12:15:31 EDT erik quanstrom  wrote:
> On Sun Oct 12 14:37:47 EDT 2014, st...@quintile.net wrote:
> > I am fairly sure the problem is to do with RAM size rather than the ras=
> pberry pi per-se.
> > 4000 messages takes up a lot of space - and upas stores messages in RAM=
> .
> 
> it's a little worse than this, actually.
> 
> since upas stores messages in mbox format, the whole file needs to be rea=
> d or written on
> update.  certainly one could optimize the read bit, but that would be dif=
> ficult  this means
> that the the whole mbox gets written to the dump every day, and you need =
> about 2x the
> mailbox size ram for each upas/fs that is run.  this does not work out we=
> ll for large mm
> messages, or small ram boxes like the pi.
> 
> the solutions to this are straightforward
> (1) store one message per file,

This is what MH (an old mail client) does by default.

> (2) cache important data in an index to avoid opening all files,

This is what dovecot (an imap/pop3 server) does.

One other thing such mailservers do is to usually only
*append* to an mbox file. Deleted messages are marked as such
but their space is not reclaimed until you force a rebuild of
the mbox file and its index.

Pure imap clients typically only read last N messages from a
given mailbox.  More may be loaded as you scroll back.

--- End Message ---

Re: [9fans] My 9pi box

[Steffen Nurpmeso]

Shane Morris  wrote:
 |On Sat, Aug 9, 2014 at 3:27 PM,  wrote:
 |> I forgot it to attach.

 |Not a bad effort. Gets the job done! And an inventive way to recycle!

Nice handycraft.
Selling pre-prepared tea in aluminium cans could be an effective
way to cut pension costs for the more traditional kind of
population (as in: extinction by heart-attack).
I presume, that is.

--steffen

Re: [9fans] kbdputc() in devcons.c in 9front?

[Steffen Nurpmeso]

kokam...@hera.eonet.ne.jp wrote:
 |have to save power as possible as we can (no
 |nuclear power energy here).

No!!  That surely can be helped, enough willingness provided.

--steffen

Re: [9fans] syscall 53

[Steffen Nurpmeso]

Skip Tavakkolian  wrote:
 |i like git.  as it is a kind of archival file system, one should be able to
 |build a plan9 file system interface for it.

Funky idea.
After reading through some Plan9 papers i had the impression that
the backing store of git(1) was designed with Venti in mind
(except that garbage collection is possible), so that would be
some kind of coming home.

--steffen

Re: [9fans] 9atom installation fails due to missing files [solved]

[Steffen Nurpmeso]

Well, maybe later, on the Athlon (i think trying notebook is hard).
But anyway i'm happy to have a running 9atom now.

(The ~28 percent CPU usage of VM i talked about is the idle load also
when installed, btw.  And awk doesn't support hex.  Ciao.)

On 8 July 2013 22:06, Matthew Veety  wrote:
> On Jul 8, 2013, at 15:55, Steffen "Daode" Nurpmeso  wrote:
>
>> Yapadapad!
>> While biking i became enlightened.
>> Just disabled the network and it boots (VirtualBox 4.2.16, OS X).
>> It seems the system hangs around trying to get some IP address via
>> DHCP??  I don't know why it does so long (minutes) and requires
>> such a lot of CPU time (though of the VM) doing so, ... but it's
>> working just fine!  Just some more years 'till i know what i'm
>> talking about!!
>>
>> --steffen
>>
>
> Yeah it does that. You have to give her a static ip. If you're on wireless, 
> promisc  sometimes helps. Best solution is install natively.
>
>