Re: shutdown busybox and start another PID1 process

[Harald Becker]

Hi James !

> This is very close to what I'm doing now.  The two changes are:


   1) I boot into Debian so I switch_root into their /sbin/init
  and I use "telinit u" to get back pid 1.


If that work, fine.  Not much difference, just use the init you like, 
the only question is how to shut down pid 1 in a resource saving way.



   2) I copy busybox et al. into the new root before switching
  so I don't need to unpack.


Nearly same if copying from your file system or from an archiv. Depends 
on your needs. I prefer to have a quick and simple unpack from a single 
file to get a working environment. Other info is then copied with a 
second cpio -p.



BTW: I wrote a space invaders game in busybox-shell, although I
cheated a little and wrote a tiny C program called getch that
gives me non-blocking input.  Maybe there is already a way to
do that built in.  If not, a getch applet might be very handy.


There are some neat stty/dd tricks and in addition the shell read has a 
timeout and a length option (never used the later). For the stty trick 
see the raw, min N and time N settings.



It is a small thing but it is all that is needed to write
truly interactive scripts with just busybox.  IMO it would
open the door to some really neat things.


Sure, some carefully designed interactive tools would open up whole new 
worlds of interactive programming with nothing more than a single binary 
and a few configuration files. That is, why I add in some own 
functionality on my self build BB versions :) ... depending on what I 
need, but those applets are not "upstream" compatible, so there 
inclusion into main stream BB tend to get blocked. Not my decision.


--
Harald

___
busybox mailing list
busybox@busybox.net
http://lists.busybox.net/mailman/listinfo/busybox

Re: How do I (unconditionally) enable unicode support in busybox?

[Harald Becker]

Hi James!

>> I assume the cases where you got correct results, was with

glibc linked BB.


Yes, but I only just got it working in the chroot today by
copying a 32 Meg locale-archive file into the chroot.


This is the problem with glibc, it's heavily overdone (means big). The 
reason why so many people link small programs (for e.g. initramfs / 
chroot statically with a smaller lib). One of those tested lib's is 
uClic, but this doesn't mean there are no bugs. Most functions work, but 
some flaws may be seen.



it when I was using glibc.  I believe I was told here that if I
was using glibc then I need to provide locale files in order to
get unicode support to work.  But no one would tell me *which*
files were needed.


Sorry for this: Have you ever tried to google for "glibc locale files"?
... but don't expect to get a simple answer. That's the big trouble with 
glibc. Either use it all (do a full install) or be on your own.
In additon things changed between glibc versions, so I don't know the 
exact files required to get things going. So you need to try.




I couldn't get it to work and I think it is
really silly anyway because it's horribly wasteful IMO to require
a bunch of files to provide the system with a single bit of
information: "treat strings as unicode".


Locale information is a bit more of information then a single bit, but 
tell it the glibc developers ... or look for an alternative. uClibc is 
one of those alternatives supported by Busybox (as Rob Landly create 
prebuild root systems with his tool chains based on uClibc), but this 
not the only alternative. Each libc, however, has it's own type of 
configuration and trouble. Sorry for this, but we are not living in a 
perfect world.



Since both uclibc and busybox have separate config options for
enabling unicode and using  locales, I thought uclibc was going
to be the answer but I was wrong.


UClibc is the well know answer to going into small intramfs/chroot 
environments ... but then you hit this sed bug ... sorry. I could detct 
there is a bug in regexp handling of uClibc. So someone of the uClibc 
maintainers need to look for this. So you may forward this to them.



files!  The curve-ball I wasn't expecting was (as you report)
unicode support in sed using uclibc is unconditionally broken.


No one did expect this, so we were confused and looked at the wrong 
topics first, especially as your explanation was confusing.



If the config options are right, there is the simple option of
setting LANG=UTF-8, and that works as you see with wc -m. The
sed problem is a different bug. Don't mix those.


No.  That is simply not true.  There is other breakage.

   1) it is claimed that unspecified and potentially large files
   need to be included in the initrd to get unicode support to
   work but they are not always needed and they don't always
   work (except for "wc -m").  IMO until the exact files that
   are needed are specified and until those files are
   reasonably small, this is still a problem for users of
   busybox.


The files required is a question of the libc you use. As we are here on 
BB list, don't expect to get every information outside the scope of BB 
cooked to the well done state. Many information can be found using a 
search engine. Beside this we try to give as much information as 
possible, but I couldn't tell you all the required file names as I do 
not know them all.



   2) sed is still broken but it sometimes works if I include
  locale files.  The strange thing isn't that it is broken;
  the strange thing is that it works sometimes even without
  locale files.  This was the only way I got busybox to count
  unicode characters at first.  I got it to work sometimes in
  the initrd without locale files so my focus was on getting
  it to work there reliably.  I admit this may have been
  misleading but it certainly seemed reasonable at the time.


I don't know what versions of libc/options you used for this. May be 
uClibc can be build with a working regexp. Currently I'm able to 
reproduce your problem with uClibc versions of BB (which don't need 
locale files, but all fail for this sed example. Based on the confusing 
descriptions you gave, I assume you mixed things during your tests. 
There were at least irregularities in your description.



   3) Then there was the bug (I think) of not calling setlocale
  if pid=1.  This was very unexpected and it made things
  more murker.


Missleading problem. This isn't the reason. Correct setting of the LANG 
variable shall work, even if setlocal is not done in pid 1, as all UTF-8 
handling is done in processes forked from your initial script.



   4) ${#x} is unconditionally broken unless the spec is for it to
  count bytes instead of characters.


This is a nasty question which behavior is right. How is the 
specification. Unconditionally returning number of characters may lead 
to buffers allocated with t

Re: How do I (unconditionally) enable unicode support in busybox?

[James Bowlin]

On Tue, Aug 12, 2014 at 05:42 AM, Harald Becker said:
> What do you mean with those different Busybox names? How do
> they differ, are all linked against your active glibc? So which
> options did you change for the build (Busybox .config not
> BuildfRoot). You missed the value of LANG and LC_* variables in
> environment.

I'm sorry.  Maybe I should call it a night.  I gave different
versions of busybox different names so I could keep track of them
and keep them all in one directory.  This test was to look at the
effect of different UNICODE config options.  There are two
different versions of busybox in the list that have the same
config options displayed but give different results. I think the
configurations for those must be different elsewhere.

I didn't start making different versions of busybox with this
test in mind.  I had saved some of the versions I had made trying
to fix the problem.  I just threw them together and write the
simple test script and ran it.  Even though the test is not pure
I think it was useful.

The different tests are separated by blank lines.  For example,
here is the first test:

/live/busybox-bin/busybox.always
 using sed 's/./x/g': 30
 using wc -m: 16
CONFIG_UNICODE_SUPPORT=y
CONFIG_FEATURE_CHECK_UNICODE_IN_ENV=y
CONFIG_UNICODE_WIDE_WCHARS=y
CONFIG_UNICODE_PRESERVE_BROKEN=y

The busybox executable is busybox.always which is for my own
internal reference.  The sed did not work but the wc -m did.
I list the UNICODE options that were enabled in the version
called busybox.always by using "busybox.always bbconfig" then
piped through some greps.

I'm sorry this was not clear.  I thought blank line separators
were so ubiquitous that the repeating pattern would make it
clear.  I had posted a similar table a week ago with a slightly
better explanation.  I should have repeated the explanation.

I'm getting worn out so I will need to take a break from this.
When I get tired I start to cut corners and I become less clear.
I'm sorry for all of the confusion I have caused.

I had a fever for most of last week and I have several things I
absolutely must get done this week so I think I will have to take
a break for a while otherwise I am going to get sick again. I'm
really sorry about this.  I tried to break away for a few days
now but you guys are so stimulating it is hard for me to stay
away even when it is for my own good.


Peace, James
ly
___
busybox mailing list
busybox@busybox.net
http://lists.busybox.net/mailman/listinfo/busybox

Re: shutdown busybox and start another PID1 process

[James Bowlin]

Thanks James for all the great info!  This candy store just gets
sweeter and sweeter.  Were you, by any chance, in physics grad
school in the 1970s?  I'm sorry for the delay in responding.
This was the first post I started to respond to this morning
but the way my interrupts work, it is the last one I'm finishing.

On Mon, Aug 11, 2014 at 11:34 PM, James B said: 
> This is a solved problem. Fatdog64, an 64-bit offshot of Puppy
> Linux (disclaimer: I'm one of the maintainers), does exactly
> this - [...]

I will definitely look at the ISO.  Thanks!  I didn't mean to
claim it was impossible (although maybe I inadvertently did).  I
meant to say that I tried and I could not find a solution.  Kudos
to you for succeeding!

> My latest solution does pivot_root too for cleanliness reason,
> but it is actually unnecessary. You can still be on your
> original aufs root and take off the layer one by one - if
> you're careful - and have a clean unmount.

I was thinking a pivot_root would make it easier to use a
dynamically linked BB but I think the best solution for that is
to use LD_PATH.

Slax does a pivot_root into a BB environment on shutdown, at
least on their live system.  They used pids for sorting processes
though.  This broke out of the box when I was testing it here
because the pids had already wrapped so I replaced pids with
start-time.

cut -d" " -f22 /proc/$pid/stat

I also use this for a 1/100th of a second timer:

cut -d" " -f22 /proc/self/stat

I've seen other Live systems use a pivot_root.  It seemed like
the canonical solution.

> If you really want to tell busybox init to quit - the simplest
> thing is to send SIGQUIT to it (kill -3 1). [...]

I never used the BB init so I didn't know much about it.  I
proposed something like this as a solution.  I didn't realize it
has already been done.  This might be of interest to the OP of
this thread.  If they are using BB init, it answers their
question.  We've gone horribly off-topic (probably mostly my
fault) and they may have stopped following the conversation.


Peace, James
___
busybox mailing list
busybox@busybox.net
http://lists.busybox.net/mailman/listinfo/busybox

Re: shutdown busybox and start another PID1 process

[James Bowlin]

On Mon, Aug 11, 2014 at 07:35 PM, Harald Becker said:
> Did I get it right?

This is very close to what I'm doing now.  The two changes are:

  1) I boot into Debian so I switch_root into their /sbin/init
 and I use "telinit u" to get back pid 1.

  2) I copy busybox et al. into the new root before switching
 so I don't need to unpack.  I suppose unpacking is slightly
 safer but I have things I need to communicate to the booted
 systems anyway.  This is partly a holdover from Knoppix
 design I started with that used symlinks instead of a
 switch_root.  Then I moved to bind mounts which were much
 better but eventually a few programs were unhappy with them
 so I ended up with the switch_root and I copied the BB stuff
 over so programs on the booted system that I communicated to
 didn't need to be changed.

 I also like the consistency of having my busybox stuff almost
 always under /live even though I have to copy things around
 to make it so.

BTW: I wrote a space invaders game in busybox-shell, although I
cheated a little and wrote a tiny C program called getch that
gives me non-blocking input.  Maybe there is already a way to
do that built in.  If not, a getch applet might be very handy.
It is a small thing but it is all that is needed to write
truly interactive scripts with just busybox.  IMO it would
open the door to some really neat things.


Peace, James
___
busybox mailing list
busybox@busybox.net
http://lists.busybox.net/mailman/listinfo/busybox

Re: How do I (unconditionally) enable unicode support in busybox?

[James Bowlin]

On Mon, Aug 11, 2014 at 10:44 PM, Harald Becker said:
> You mix things. On one side there is Busybox, which may contain
> it's own handling functions for UTF support, on the other side
> there is the libc which may contain standard UTF handling. All
> this has grown from a none UTF dominated time. At different
> times have functions bean implemented. When you build a BB with
> UTF support, you need to use a libc with correct UTF support.
> Both are different packages with different config options, but
> they need to fit to get correct behavior.

I did not mean to mislead you.  I had been using glibc
exclusively until yesterday (or the day before?).  I had said
this a few times.  I was under the impression that it was working
on your system and that you were using uclibc.  I know that
uclibc is recommended and there are claims that glibc is broken
in some ways so I embarked on building busybox with uclibc so it
would fix the problem.  It didn't.  But well over 95% of my
testing has been with glibc linked busybox.

> I assume the cases where you got correct results, was with
> glibc linked BB. 

Yes, but I only just got it working in the chroot today by
copying a 32 Meg locale-archive file into the chroot.

> The uClibc versions of your BB give wrong results from sed,
> correct values from wc -m when UTF is enabled. That is you
> mislead us to the wrong assumptions. The problem is the regexp
> scanner of uClibc not BB or the setting of the LANG variable.

I only started testing uclibc linked busybox yesterday or the day
before.  I started this thread a week ago.  I was hoping the
switch to uclibc would fix the problem.  I originally reported
it when I was using glibc.  I believe I was told here that if I
was using glibc then I need to provide locale files in order to
get unicode support to work.  But no one would tell me *which*
files were needed.  I couldn't get it to work and I think it is
really silly anyway because it's horribly wasteful IMO to require
a bunch of files to provide the system with a single bit of
information: "treat strings as unicode".

Since both uclibc and busybox have separate config options for
enabling unicode and using  locales, I thought uclibc was going
to be the answer but I was wrong.  

> ... and here you still mix uClibc and glibc. For BB linked with
> glibc you need locale files to function correct, but those
> locale files are just useless if you start a BB linked against
> uClibc. Sort that things out, or you produce confusion.

I understood this.  That is why I went to the effort to make a
busybox linked with uclibc.  I don't want no stinkin' locale
files!  The curve-ball I wasn't expecting was (as you report)
unicode support in sed using uclibc is unconditionally broken.
 
> If the config options are right, there is the simple option of
> setting LANG=UTF-8, and that works as you see with wc -m. The
> sed problem is a different bug. Don't mix those.

No.  That is simply not true.  There is other breakage.

  1) it is claimed that unspecified and potentially large files
  need to be included in the initrd to get unicode support to
  work but they are not always needed and they don't always
  work (except for "wc -m").  IMO until the exact files that
  are needed are specified and until those files are
  reasonably small, this is still a problem for users of
  busybox.

  2) sed is still broken but it sometimes works if I include
 locale files.  The strange thing isn't that it is broken;
 the strange thing is that it works sometimes even without
 locale files.  This was the only way I got busybox to count
 unicode characters at first.  I got it to work sometimes in
 the initrd without locale files so my focus was on getting
 it to work there reliably.  I admit this may have been
 misleading but it certainly seemed reasonable at the time.
 
  3) Then there was the bug (I think) of not calling setlocale
 if pid=1.  This was very unexpected and it made things
 more murker.

  4) ${#x} is unconditionally broken unless the spec is for it to
 count bytes instead of characters.  

  5) printf "%Ns" seems to be unconditionally broken. On my
 host system it counts characters not bytes. But with
 all glibc busybox configs I've tested, even when both
 sed and wc -m work, printf doesn't.

I agree that "wc -m" works, but that seems to be the exception
not the rule.  Thank you again for showing me "wc -m".

Things are starting to clear up but they are generally still
quite murky.  There is conflicting advice about which busybox
options to use.  There was (until the last day or two) incorrect
advice about which library to use.  It is still unclear to me
which locale files are needed and how small they can be.  Not
running setlocale when pid=1 seems bizarre.  Someone added that
so they must have had a good reason but it seems ripe for
frustrating users like the fellow who submitted a patch back in
January to wo

Re: How do I (unconditionally) enable unicode support in busybox?

[Harald Becker]

Hi Rich!

> I think this is the result of using uclibc with a broken regex

implementation -- either as a result of a build time option (omitting
locale? omitting full regex?) or just a deficiency in uclibc. Using
glibc or musl would solve it.


Until now I couldn't find a uClibc linked version of Busybox which works 
in that case, can you find a version (which would show it is an option 
problem, otherwise it looks more like a uClibc bug).


Prebuild statically linked Busybox versions with musl would be fine ... 
looking to see them any time.


--
Harald


___
busybox mailing list
busybox@busybox.net
http://lists.busybox.net/mailman/listinfo/busybox

Re: How do I (unconditionally) enable unicode support in busybox?

[Harald Becker]

> /live/busybox-bin/busybox.always> /live/busybox-bin/busybox.always-loc
> /live/busybox-bin/busybox.env
> /live/busybox-bin/busybox.loc
> /live/busybox-bin/busybox.no-loc
> /live/busybox-bin/busybox.norm

What do you mean with those different Busybox names? How do they differ, 
are all linked against your active glibc? So which options did you 
change for the build (Busybox .config not BuildfRoot). You missed the 
value of LANG and LC_* variables in environment.


All the rest of your mail is useless, without giving the information above.

___
busybox mailing list
busybox@busybox.net
http://lists.busybox.net/mailman/listinfo/busybox

Re: How do I (unconditionally) enable unicode support in busybox?

[Harald Becker]

Hi James!


I don't see how init itself shall benefit from UTF8 support? in
just forks and starts other processes with a copy of it's
environment.  Does init anything which needs UTF-8 support?


It would be useful to me.


NO, it wouldn't!


 My /init is translated into several different languages,


YOUR init is not the Busybox applet named init. I assume you call your 
initial script /init which is set executeable and with a first line of 
#!/bin/sh. This script will be sourced by Busybox ash not Busybox init. 
Changing anything to the Busybox init won't change anything to your 
/init script, and on the opposite the only input to Busybox /init is 
/etc/inittab which usually don't contain national characters. And even 
if /etc/inittab would contain UTF-8 characters they will get passed 
through unconditionally to the given command, which then may process 
national characters, but again won't benefit from changing anything for 
Busybox init applet (the setlocale is not passed by fork, only the 
environment).


All the rest of your mail describe what your /init script does, which 
does not use Busybox init applet. So changes or not of Denys won't hit 
you in any way.




The thing is that:
printf "%Ns"
is broken because it counts bytes, not characters.


Counting bytes is the standard behavior of C printf, which usually has 
been defined that way long before UTF support was added. Would be nice 
to have a printf which enables counting of characters, but I did not 
hear about this yet. So don't call it broken, when it behaves as defined 
(only due not working as you are expecting it).




___
busybox mailing list
busybox@busybox.net
http://lists.busybox.net/mailman/listinfo/busybox

Re: udhcpd on vlan interface

[Андрей Николаев]

Problem solved - misconfigured iptables that was. Thanks for your attention. 

Mon, 11 Aug 2014 12:44:31 +0400 от Андрей Николаев :
>Hello. 
>
>I'm trying to use udhcpd (version 1.20.2) on a vlan interface like eth0.100. 
>Here's config
>
>=== udhcpd.conf start ===
>start   10.100.31.40
>end 10.100.31.200
>interface   eth0.100
>max_leases  100 
>notify_file /etc/udhcp-notify.sh
>option  dns 8.8.8.8
>option  dns 8.8.4.4
>option  subnet  255.255.0.0
>option  router  10.100.31.1
>option  lease   864000
>option  broadcast   10.100.255.255
>=== udhcpd.conf end ===
>
>When I start udhcpd and try using dhcp client on another pc, it does not seem 
>to work. I can see DHCP REQs on eth0.100 with tcpdump, and that's it. No 
>reaction from udhcpd, nothing in logs, no executions of the notify_file.What 
>colud be wrong with this installation?
>
>--
>best regards, Andrei Nikolaev.
>___
>busybox mailing list
>busybox@busybox.net
>http://lists.busybox.net/mailman/listinfo/busybox


--
best regards, Andrei Nikolaev.
___
busybox mailing list
busybox@busybox.net
http://lists.busybox.net/mailman/listinfo/busybox

Re: How do I (unconditionally) enable unicode support in busybox?

[Rich Felker]

On Mon, Aug 11, 2014 at 05:15:21PM +0200, Harald Becker wrote:
> >IMO there is still something very strange with sed and unicode
> 
> YES! I did not stop looking for this. Looks like this is a problem
> in the regular expression parser.
> 
> s /./x/g
> 
> shall match every character and replace with a single x, but indeed
> it matches every byte of UTF-8 characters too (which is wrong). But
> this doesn't seam to depend on setting of LANG (which confused me).
> Is it possible, it only worked when BB is linked with glibc in a
> fully functional environment. Maybe than an UTF-8 aware regex
> scanner is used. We need to look further on this!

I think this is the result of using uclibc with a broken regex
implementation -- either as a result of a build time option (omitting
locale? omitting full regex?) or just a deficiency in uclibc. Using
glibc or musl would solve it.

Rich
___
busybox mailing list
busybox@busybox.net
http://lists.busybox.net/mailman/listinfo/busybox

Re: How do I (unconditionally) enable unicode support in busybox?

[James Bowlin]

On Mon, Aug 11, 2014 at 08:31 PM, Denys Vlasenko said:
> Or am I missing something? Do we also have a bug in
> CONFIG_UNICODE_USING_LOCALE=y case?

Here are my results running different busybox using glibc from
the command line with various UNICODE configurations.  If
sed is working the result will be 16, not 30:

/live/busybox-bin/busybox.always
 using sed 's/./x/g': 30
 using wc -m: 16
CONFIG_UNICODE_SUPPORT=y
CONFIG_FEATURE_CHECK_UNICODE_IN_ENV=y
CONFIG_UNICODE_WIDE_WCHARS=y
CONFIG_UNICODE_PRESERVE_BROKEN=y

/live/busybox-bin/busybox.always-loc
 using sed 's/./x/g': 16
 using wc -m: 16
CONFIG_UNICODE_SUPPORT=y
CONFIG_FEATURE_CHECK_UNICODE_IN_ENV=y
CONFIG_UNICODE_WIDE_WCHARS=y
CONFIG_UNICODE_PRESERVE_BROKEN=y

/live/busybox-bin/busybox.env
 using sed 's/./x/g': 30
 using wc -m: 16
CONFIG_UNICODE_SUPPORT=y
CONFIG_FEATURE_CHECK_UNICODE_IN_ENV=y
CONFIG_UNICODE_WIDE_WCHARS=y
CONFIG_UNICODE_PRESERVE_BROKEN=y

/live/busybox-bin/busybox.loc
 using sed 's/./x/g': 16
 using wc -m: 16
CONFIG_UNICODE_SUPPORT=y
CONFIG_UNICODE_USING_LOCALE=y
CONFIG_UNICODE_WIDE_WCHARS=y
CONFIG_UNICODE_PRESERVE_BROKEN=y

/live/busybox-bin/busybox.no-loc
 using sed 's/./x/g': 30
 using wc -m: 16
CONFIG_UNICODE_SUPPORT=y
CONFIG_UNICODE_WIDE_WCHARS=y
CONFIG_UNICODE_PRESERVE_BROKEN=y

/live/busybox-bin/busybox.norm
 using sed 's/./x/g': 16
 using wc -m: 16
CONFIG_UNICODE_SUPPORT=y
CONFIG_UNICODE_WIDE_WCHARS=y

The version with USING_LOCALE works but so do others.

Using ${#x} never works so I left it out of the results.

I have *finally* gotten it to work inside a chroot using
busybox.norm which does not set USING_LOCALE but I did have to
copy over 34 Meg of locale files from the host system to make
it work.  The files that finally turned the trick are under:

/usr/lib/locale/

I had already copied over files from /usr/share/i18n and
/usr/share/locale.  I don't know if those are needed or not.
I don't think /usr/share/locale is needed because it only
contains translations.

I'm sure not all of those files are needed.  Unfortunately when I
delete the 32 Meg locale-archive file then it stops working.

Note though that this works without the USING_LOCALE option.  In
the initrd, the USING_LOCALE options seems to prevent it from
working at all although I can try it again with the
/usr/lib/locale files assuming I can reduce the size of the
locale-archive file and still have it work.


Peace, James
___
busybox mailing list
busybox@busybox.net
http://lists.busybox.net/mailman/listinfo/busybox

Re: [PATCH] taskset: support more than 64 cores

[Isaac Dunham]

On Mon, Aug 11, 2014 at 10:35:46PM +0100, Laszlo Papp wrote:
> On Mon, Aug 11, 2014 at 10:33 PM, Aaro Koskinen 
> wrote:
> 
> > Hi,
> >
> > On Mon, Aug 11, 2014 at 10:27:12PM +0100, Laszlo Papp wrote:
> > > Show me one typical embedded system that is high-volume and has more than
> > > 64 cores. Even the full-fledged iphone tablets are not there and even if
> > > they were, they would use complete utils rather than chopped most
> > probably
> > > anyhow. To me, this feature does not seem to fit busybox's goal unless
> > > Apple, Samsung, etc were not notified about some recent boom in the
> > > semiconductor industry.
> >
> > Embedded != consumer electronics.
> >
> 
> Not quite sure what you are trying to achieve, but you wrote embedded
> systems _word-by-word_, and I asked for one typical example with more than
> 64 cores where busybox is so much needed.

I'm pretty sure that Matthieu (the patch author) just said that 6wind (his 
company)
uses or builds the sort of hardware where this is relevant.

HTH,
Isaac
___
busybox mailing list
busybox@busybox.net
http://lists.busybox.net/mailman/listinfo/busybox

Re: How do I (unconditionally) enable unicode support in busybox?

[James Bowlin]

On Mon, Aug 11, 2014 at 08:58 PM, Harald Becker said:
> I don't see how init itself shall benefit from UTF8 support? in
> just forks and starts other processes with a copy of it's
> environment.  Does init anything which needs UTF-8 support?

It would be useful to me.  My /init is translated into several
different languages, some of which contain utf characters.  The
translation scheme consists of a file for each language that
contains lines like:

_Welcome_to_X_="Welcome to %s"

The whole system is fully automated.  My init.src uses strings in
English.  I have a set of tools that create from it an init
script that replaces the English strings with things like:

"$_Welcome_to_X"

It also creates the en.xlat file with all of the English strings
as well as a .pot file for sending to translators via an
automated website.  I then automatically download the .po files
and use those to create the various $LANG.xlat files.

The thing is that:

   printf "%Ns"

is broken because it counts bytes, not characters.  I work around
this now by using the translation system.  If an English string
starts with leading whitespace then the translation system uses a
printf "%Ns" where N is the length of the English string.  This
way the printf happens on my host system were printf is utf
aware.

The .xlat files (except the English one) are created with the
standard gettext mechanism.  For each program there is a script
that has lines like:

printf "_Welcome_to_X_=\"%s\"\n" "$(gettext "Welcome to %s")"
printf "_Total_Memory_=\"%25s\"\n" "$(gettext "Total Memory")"

This script is run for each language we have a .po file
associated with the program. I just the command with:

LANG=$LANG

and it spits out a .xlat file for that language.

Notice the %25s" in the 2nd line.  This is because the original
English string had leading whitespace and 25 characters.

Anyway, I for one would would find unicode support in /init to be
useful it it includes printf.  I've got a workaround for now
so it is not urgent.  I really don't know if anyone else is
going in this direction.


Peace, James
___
busybox mailing list
busybox@busybox.net
http://lists.busybox.net/mailman/listinfo/busybox

Re: [PATCH] taskset: support more than 64 cores

[Doug Clapp]

Laszlo Papp wrote:


On Mon, Aug 11, 2014 at 10:15 PM, Aaro Koskinen > wrote:


Hi,

On Mon, Aug 11, 2014 at 02:25:04PM +0100, Laszlo Papp wrote:
> I wonder about the use case for this feature? I mean busybox is
meant for
> small systems in general, and >=64 cores are not that small embedded
> systems, at least not yet, yeah?

How you define small? Earlier you said using libc getconf is bloat
(as nproc replacement) for multicore systems, but now you suggest
that for cores > 64 we should't care. Where do you draw the line?

Embedded systems that have lots of RAM and CPUs/cores run-time may be
still limited to having a boot flash of just few megs where the whole
OS image needs to fit.


Show me one typical embedded system that is high-volume and has more 
than 64 cores. Even the full-fledged iphone tablets are not there and 
even if they were, they would use complete utils rather than chopped 
most probably anyhow. To me, this feature does not seem to fit 
busybox's goal unless Apple, Samsung, etc were not notified about some 
recent boom in the semiconductor industry.


I recently read a review of Nvidia's Jetson TK1 board and it occurs to 
me this may be such a system.  Check 
https://developer.nvidia.com/jetson-tk1 for more information.  While it 
has 2 GB RAM, I can believe there would be embedded systems where that 
would be reserved for the application data.  With the the Kepler GPU's 
"192 CUDA cores" it obviously exceeds 64.  I don't know of any projected 
products based on it, but I'd bet somebody is working on one.


Doug Clapp


___
busybox mailing list
busybox@busybox.net
http://lists.busybox.net/mailman/listinfo/busybox

udhcpd on vlan interface

[Андрей Николаев]

 Hello. 

I'm trying to use udhcpd (version 1.20.2) on a vlan interface like eth0.100. 
Here's config

=== udhcpd.conf start ===
start   10.100.31.40
end 10.100.31.200
interface   eth0.100
max_leases  100 
notify_file /etc/udhcp-notify.sh
option  dns 8.8.8.8
option  dns 8.8.4.4
option  subnet  255.255.0.0
option  router  10.100.31.1
option  lease   864000
option  broadcast   10.100.255.255
=== udhcpd.conf end ===

When I start udhcpd and try using dhcp client on another pc, it does not seem 
to work. I can see DHCP REQs on eth0.100 with tcpdump, and that's it. No 
reaction from udhcpd, nothing in logs, no executions of the notify_file.What 
colud be wrong with this installation?

--
best regards, Andrei Nikolaev.
___
busybox mailing list
busybox@busybox.net
http://lists.busybox.net/mailman/listinfo/busybox

Re: [PATCH] taskset: support more than 64 cores

[Aaro Koskinen]

Hi,

BTW, if you care about bloat, please consider stopping sending HTML
mails to the list.

A.
___
busybox mailing list
busybox@busybox.net
http://lists.busybox.net/mailman/listinfo/busybox

Re: [PATCH] taskset: support more than 64 cores

[Laszlo Papp]

On Mon, Aug 11, 2014 at 11:26 PM, Aaro Koskinen 
wrote:

> Hi,
>
> On Mon, Aug 11, 2014 at 11:19:34PM +0100, Laszlo Papp wrote:
> > Right, so you cannot bring up any valid and real use case for this, or do
> > not want despite the explicit clarification request, I take it. I do not
> > think theoretical changes should be added just because it is a good
> > technical challange. Valid use cases oughta be part of the commit
> message,
> > but at worst, comment explanation without even explicit request to be
> > fair.
>
> I think the use case is pretty clear: "support more than 64 cores".
>

No, that is a *theory*.

Use case would be: here is the hardware that is in desperate need of
busybox because although it has 96/128/core, memory and flash are s
limited. I honestly cannot find any hardware like that, and I have been
looking for one now for an hour with google. Not even the supercomputing
mobile platforms are like that and as indicated, even there, having
complete utils is a not issue. It is not a use case to add support for some
random technical challenge. When some submits a change, that person is
expected to provide real case scenarios. Busybox could also add support for
everything that the complete utils can handle in the desktop world, but
that is against its original goals, luckily, since I would not see the
point of NIH there.
___
busybox mailing list
busybox@busybox.net
http://lists.busybox.net/mailman/listinfo/busybox

Re: [PATCH] taskset: support more than 64 cores

[Aaro Koskinen]

Hi,

On Mon, Aug 11, 2014 at 11:19:34PM +0100, Laszlo Papp wrote:
> Right, so you cannot bring up any valid and real use case for this, or do
> not want despite the explicit clarification request, I take it. I do not
> think theoretical changes should be added just because it is a good
> technical challange. Valid use cases oughta be part of the commit message,
> but at worst, comment explanation without even explicit request to be
> fair.

I think the use case is pretty clear: "support more than 64 cores".

A.
___
busybox mailing list
busybox@busybox.net
http://lists.busybox.net/mailman/listinfo/busybox

Re: [PATCH] taskset: support more than 64 cores

[Laszlo Papp]

On Mon, Aug 11, 2014 at 11:13 PM, Aaro Koskinen 
wrote:

> Hi,
>
> On Mon, Aug 11, 2014 at 10:35:46PM +0100, Laszlo Papp wrote:
> > Not quite sure what you are trying to achieve, but you wrote embedded
> > systems _word-by-word_, and I asked for one typical example with more
> than
> > 64 cores where busybox is so much needed.
>
> I'm not trying the limit busybox usage, and I don't think there are
> any "typical" embedded system. And Apple or Samsung are definitely not
> any reference. But if you are happy with their HW, then good for you.
>

Right, so you cannot bring up any valid and real use case for this, or do
not want despite the explicit clarification request, I take it. I do not
think theoretical changes should be added just because it is a good
technical challange. Valid use cases oughta be part of the commit message,
but at worst, comment explanation without even explicit request to be fair.
___
busybox mailing list
busybox@busybox.net
http://lists.busybox.net/mailman/listinfo/busybox

Re: [PATCH] taskset: support more than 64 cores

[Aaro Koskinen]

Hi,

On Mon, Aug 11, 2014 at 10:35:46PM +0100, Laszlo Papp wrote:
> Not quite sure what you are trying to achieve, but you wrote embedded
> systems _word-by-word_, and I asked for one typical example with more than
> 64 cores where busybox is so much needed.

I'm not trying the limit busybox usage, and I don't think there are
any "typical" embedded system. And Apple or Samsung are definitely not
any reference. But if you are happy with their HW, then good for you.

A.
___
busybox mailing list
busybox@busybox.net
http://lists.busybox.net/mailman/listinfo/busybox

Re: [PATCH] taskset: support more than 64 cores

[Laszlo Papp]

On Mon, Aug 11, 2014 at 10:33 PM, Aaro Koskinen 
wrote:

> Hi,
>
> On Mon, Aug 11, 2014 at 10:27:12PM +0100, Laszlo Papp wrote:
> > Show me one typical embedded system that is high-volume and has more than
> > 64 cores. Even the full-fledged iphone tablets are not there and even if
> > they were, they would use complete utils rather than chopped most
> probably
> > anyhow. To me, this feature does not seem to fit busybox's goal unless
> > Apple, Samsung, etc were not notified about some recent boom in the
> > semiconductor industry.
>
> Embedded != consumer electronics.
>

Not quite sure what you are trying to achieve, but you wrote embedded
systems _word-by-word_, and I asked for one typical example with more than
64 cores where busybox is so much needed.
___
busybox mailing list
busybox@busybox.net
http://lists.busybox.net/mailman/listinfo/busybox

Re: [PATCH] taskset: support more than 64 cores

[Aaro Koskinen]

Hi,

On Mon, Aug 11, 2014 at 10:27:12PM +0100, Laszlo Papp wrote:
> Show me one typical embedded system that is high-volume and has more than
> 64 cores. Even the full-fledged iphone tablets are not there and even if
> they were, they would use complete utils rather than chopped most probably
> anyhow. To me, this feature does not seem to fit busybox's goal unless
> Apple, Samsung, etc were not notified about some recent boom in the
> semiconductor industry.

Embedded != consumer electronics.

A.
___
busybox mailing list
busybox@busybox.net
http://lists.busybox.net/mailman/listinfo/busybox

Re: [PATCH] taskset: support more than 64 cores

[Laszlo Papp]

On Mon, Aug 11, 2014 at 10:15 PM, Aaro Koskinen 
wrote:

> Hi,
>
> On Mon, Aug 11, 2014 at 02:25:04PM +0100, Laszlo Papp wrote:
> > I wonder about the use case for this feature? I mean busybox is meant for
> > small systems in general, and >=64 cores are not that small embedded
> > systems, at least not yet, yeah?
>
> How you define small? Earlier you said using libc getconf is bloat
> (as nproc replacement) for multicore systems, but now you suggest
> that for cores > 64 we should't care. Where do you draw the line?
>
> Embedded systems that have lots of RAM and CPUs/cores run-time may be
> still limited to having a boot flash of just few megs where the whole
> OS image needs to fit.
>

Show me one typical embedded system that is high-volume and has more than
64 cores. Even the full-fledged iphone tablets are not there and even if
they were, they would use complete utils rather than chopped most probably
anyhow. To me, this feature does not seem to fit busybox's goal unless
Apple, Samsung, etc were not notified about some recent boom in the
semiconductor industry.
___
busybox mailing list
busybox@busybox.net
http://lists.busybox.net/mailman/listinfo/busybox

Re: [PATCH] taskset: support more than 64 cores

[Aaro Koskinen]

Hi,

On Mon, Aug 11, 2014 at 02:25:04PM +0100, Laszlo Papp wrote:
> I wonder about the use case for this feature? I mean busybox is meant for
> small systems in general, and >=64 cores are not that small embedded
> systems, at least not yet, yeah?

How you define small? Earlier you said using libc getconf is bloat
(as nproc replacement) for multicore systems, but now you suggest
that for cores > 64 we should't care. Where do you draw the line?

Embedded systems that have lots of RAM and CPUs/cores run-time may be
still limited to having a boot flash of just few megs where the whole
OS image needs to fit.

A.
___
busybox mailing list
busybox@busybox.net
http://lists.busybox.net/mailman/listinfo/busybox

Re: How do I (unconditionally) enable unicode support in busybox?

[Harald Becker]

H James!

On 11.08.2014 21:40, James Bowlin wrote:

But that seems nuts.  I can understand maybe why a libc might tie
unicode support to using locales but uclibc does not.  Why is
there a separate config option to enable unicode support if it
always needs the option you give above in order to work?  IOW,
locale support depends on unicode support.  If unicode support
also depends on locale support then it is very confusing for
them to be separate options.


You mix things. On one side there is Busybox, which may contain it's own 
handling functions for UTF support, on the other side there is the libc 
which may contain standard UTF handling. All this has grown from a none 
UTF dominated time. At different times have functions bean implemented. 
When you build a BB with UTF support, you need to use a libc with 
correct UTF support. Both are different packages with different config 
options, but they need to fit to get correct behavior.


In addition you still see this LANG=utf to switch UTF support, but the 
sed problem is a different one. The regular expression scanner of uClibc 
seams to behave incorrect and match every byte not characters. I assume 
the cases where you got correct results, was with glibc linked BB. The 
uClibc versions of your BB give wrong results from sed,  correct values 
from wc -m when UTF is enabled. That is you mislead us to the wrong 
assumptions. The problem is the regexp scanner of uClibc not BB or the 
setting of the LANG variable.



If this is nuts and I do need locale files in order to enable
unicode support then which files do I need?  I copied every
locale file and directory I could find into my busybox chroot
and I never got unicode support to work fully.


Your thought is dominated from BuildRoot as a single Package building 
uClibc and Busybox, but all those are separate Packages. You need to put 
the right things together to get fully functioning results. This is not 
trivial and needs several try and error cases to get all things right.


... and here you still mix uClibc and glibc. For BB linked with glibc 
you need locale files to function correct, but those locale files are 
just useless if you start a BB linked against uClibc. Sort that things 
out, or you produce confusion.


You are right at one aspect: sed s/./x/g misbehaves for UTF locales 
*WHEN* linked against uClibc. I detected the regexp scanner matches 
bytes not characters, which is wrong (either a configuration problem or 
a bug of uClibc) ... but not a BB bug.



IMO there should be an easy and obvious way to enable unicode
support in busybox.  ATM some of it is very flaky.


If the config options are right, there is the simple option of setting 
LANG=UTF-8, and that works as you see with wc -m. The sed problem is a 
different bug. Don't mix those.



If worse comes to worse, a FORCE_UNICODE config option would work
(from my perspective).  Or a --unicode flag to sh and scripts
(although this may be horribly non-POSIX).  Even just obeying the
darned LANG variable would be great.  This fails to work even in
scripts that are not pid 1.


Are you talking about building BB or about usage? If you talk about 
build, think that this not homogeneous package from one group of 
developers. You are right things could be better, so provide patches for 
all those packages involved to fix things if you like. You will see that 
this is not so easy as it may be expected.



I'm sorry if I sound frustrated and complaining.  Busybox
is one of my all-time favorite programs.


You see BB as a single block of software. It is not. Options of 
different packages needs to match (that is glibc need locale files, 
uClibc its correct settings). When testing you definitely need to watch 
your steps. You mixed tests with glibc and uClibc. Giving us results of 
tests without telling which versions where involved mislead us wrong 
assumptions.


You hit a bug, correct, but you explained it wrong, and the bug is not 
in BB it is a problem with uClibc's regexp functions.


--
Harald

___
busybox mailing list
busybox@busybox.net
http://lists.busybox.net/mailman/listinfo/busybox

Re: How do I (unconditionally) enable unicode support in busybox?

[James Bowlin]

On Mon, Aug 11, 2014 at 08:28 PM, Denys Vlasenko said:

> Those routines are part of libc. Therefore, they will be
> Unicode-aware only if you are using
> CONFIG_UNICODE_USING_LOCALE=y.

But that seems nuts.  I can understand maybe why a libc might tie
unicode support to using locales but uclibc does not.  Why is
there a separate config option to enable unicode support if it
always needs the option you give above in order to work?  IOW,
locale support depends on unicode support.  If unicode support
also depends on locale support then it is very confusing for
them to be separate options.

If this is nuts and I do need locale files in order to enable
unicode support then which files do I need?  I copied every
locale file and directory I could find into my busybox chroot
and I never got unicode support to work fully.

As we agreed before, It work best when UNICODE_USING_LOCALE is
turned off.  I have gotten it to work in an initrd with no locale
files but it is fragile and depends in a weird way on the user
providing a lang= boot parameter.

IMO there should be an easy and obvious way to enable unicode
support in busybox.  ATM some of it is very flaky.

If worse comes to worse, a FORCE_UNICODE config option would work
(from my perspective).  Or a --unicode flag to sh and scripts
(although this may be horribly non-POSIX).  Even just obeying the
darned LANG variable would be great.  This fails to work even in
scripts that are not pid 1.

> Which problem? There are so many mails in this thread...

Sorry.  My problem was the one you addressed, counting characters
in a unicode string.

> This is clearly a bug (or at least "incompatibility").  Please
> report each such bug separately.

Are you suggesting I file a bunch of bug reports?

ISTM the major bug is there is no obvious way to reliably enable
unicode support in busybox.  If I do need to include files (which
seems nuts to me) then which files need to be included?  Do I need
to include locale support in uclibc in order to get unicode
support in busybox?

I see code in unicode.c that looks for "utf" and "UTF" in the
LANG variable but exporting the LANG variable and even exec'ing a
non pid-1 script does not fix the problem.

ISTM the path to a solution is for "you" to first provide clear
instructions on how unicode support is supposed to be enabled
with a minimum of extra files.  The second step (if needed) is to
make these clear and simple instructions work as advertised.

Without the first step, it is difficult for me to file bug
reports regarding the second step.  I need to guess at what
the instructions are and try an array of different configs
in a array of different environments with an array of small
variations in the scripts.  Without clear instructions, all
I can say for sure is that I have so far failed to find the
magic incantation needed to enable unicode.

I was really hoping that moving from glibc to uclibc would
magically fix this problem without the need for locale support.
It didn't work.  The title of my first post says it all:

  How do I (unconditionally) enable unicode support in busybox?

There seem to be a variety of paths to enabling unicode support
but they all seem to be broken.

I suggest we start with instructions on what is supposed to
be the simplest and easiest way to enable unicode support.
There could well be separate instructions for glibc and for
uclibc.  If there really is no way to enable unicode without
locale files then give me some idea of which locale files
are needed.


I'm sorry if I sound frustrated and complaining.  Busybox
is one of my all-time favorite programs.  It is almost always
a joy to work with.


Peace, James
___
busybox mailing list
busybox@busybox.net
http://lists.busybox.net/mailman/listinfo/busybox

Re: shutdown busybox and start another PID1 process

[Sam Liddicott]

I wonder if cgroups or namespaces have any role here.

The initrd mounts NTFS and then creates a new namespace for new init which
can't see or kill fuse.

Sam
On 11 Aug 2014 17:34, "James B"  wrote:

> Hi James (from another James).
>
> On Mon, 11 Aug 2014 03:17:52 -0600
> James Bowlin  wrote:
>
> >
> > There was a need to have static root persistence where the
> > changes are saved directly in the root persistence file so I
> > needed to bite the bullet and finally get clean umounts during
> > shutdown.
> >
> > Previously, filesystems were consistently reporting they were
> > uncleanly mounted.  I wanted to do it all cleanly.  I tried like
> > crazy to get remounting read-only to work but I just could not do
> > it.  Maybe the problem was that I couldn't remount the rw file
> > system under the aufs as ro (because the aufs was rw) and I
> > couldn't remount the aufs as ro because its mount options are
> > unconventional.
>
> This is a solved problem. Fatdog64, an 64-bit offshot of Puppy Linux
> (disclaimer: I'm one of the maintainers), does exactly this - and have
> clean unmounts on common filesystems - fat32, ext2/3/4, ntfs (through
> ntfs-3g). The key is to remount ro the aufs layers; once made ro you can
> "unbind" them one by one (including the r/w layer). This ability to
> dynamically bind/unbind layers is one of the thing that makes me loves aufs
> so much - it is so much more versatile than any other layered filesystem
> out there *combined*. Get the ISO and take a look at its
> /etc/rc.d/rc.cleanup if you're interested.
>
> >
> > Well busybox has all the tools I need.  The tmpfs gives me a place
> > to stand so I can move the world^H^H aufs file system. It gives me
> > a place where my tools and everything I need are off of the aufs.
>
> Agreed, busybox is all you need.
>
> >
> > I pivot onto a tmpfs that has the busybox stuff and very little
> > else.  Root could easily erase it all.  Since it is tmpfs there
> > is no need to umount it at all.  The point is to finish the
> > shutdown from a tmpfs so I can cleanly umount the aufs and the
> > filesystems under it including the file system on the host
> > machine or on the LiveUSB.
>
> My latest solution does pivot_root too for cleanliness reason, but it is
> actually unnecessary. You can still be on your original aufs root and take
> off the layer one by one - if you're careful - and have a clean unmount.
>
> >
> > >   Hehe. I like to say that initrd/initramfs is useless, [...]
> >
> > It is required for a live system.
>
> James, you won't convert Laurent :) I tried that before.
> I however is in the complete agreement with you. Initramfs is
> indispensable for this.
> Anyway, to each his own.
>
> ---
>
> If you really want to tell busybox init to quit - the simplest thing is to
> send SIGQUIT to it (kill -3 1). Once you do that, busybox init will execute
> all the ::shutdown scripts in /etc/inittab; and when done, it will exec to
> anything you put in ::restart. Whatever command you put in ::restart will
> run as pid 1 when all is said and done. This is however isn't useful for
> clean unmount because if you happen to use ntfs-3g, it will get killed too
> before it has the chance to unmount cleanly.
>
> cheers!
> James
>
> --
> James B 
> ___
> busybox mailing list
> busybox@busybox.net
> http://lists.busybox.net/mailman/listinfo/busybox
>
___
busybox mailing list
busybox@busybox.net
http://lists.busybox.net/mailman/listinfo/busybox

Re: How do I (unconditionally) enable unicode support in busybox?

[Harald Becker]

Denys!

> Changed this in git just now.

Note: you will likely need to set $LANG in init's environment
for setlocale(LC_ALL, "") to activate Unicode for your init.


I don't see how init itself shall benefit from UTF8 support? in just 
forks and starts other processes with a copy of it's environment. Does 
init anything which needs UTF-8 support?


The initial messages from James were missleading. Drop this 
init/initramfs/chrot stuff. The problem was/is the used regex parser 
(glibc, uClibc, etc.).


Can BB build create a warning if linked against uClibc without UTF 
support? Just as an idea.


--
Harald

___
busybox mailing list
busybox@busybox.net
http://lists.busybox.net/mailman/listinfo/busybox

Re: How do I (unconditionally) enable unicode support in busybox?

[Harald Becker]

Hi Denys!

On 11.08.2014 20:31, Denys Vlasenko wrote:

On Mon, Aug 11, 2014 at 8:22 PM, Harald Becker  wrote:

Hi Denys !

I looked a bit into James problem and were able to reproduce:

sed   s/./x/g

on an UTF-8 locale matches every byte not on character as expected. This at
least on current Ubuntu standard build of Buybox. Even if other UTF support
seam to work, the regex scanner hast some trouble. This needs some more
investigation.


Unfortunately, short of reimplementing regexp, we have to rely on libc here.
Which means, regexps will not be Unicode-aware if
CONFIG_UNICODE_USING_LOCALE is not set.


That is what I expected. Looks to me only the uClibc versions of BB got 
that sed problem (not tested grep etc.).


... but beside this:

shell substitution ${#var} gives different result
what says POSIX to this?
if the function returns number of characters not bytes buffer could be 
allocated to small etc., we shall be carefull


on BB awk: the function length() returns different value (bytes not 
characters), so what says POSIX?


--
Harald

___
busybox mailing list
busybox@busybox.net
http://lists.busybox.net/mailman/listinfo/busybox

Re: How do I (unconditionally) enable unicode support in busybox?

[Denys Vlasenko]

On Mon, Aug 11, 2014 at 8:28 PM, Denys Vlasenko
 wrote:
>> The most mysterious thing is that it sometimes works and what
>> I need to do to get it to work in /init in an initrd.
>
> You may have found a bug. bbox never runs setlocale()
> for init.

Changed this in git just now.
Note: you will likely need to set $LANG in init's environment
for setlocale(LC_ALL, "") to activate Unicode for your init.


commit 45b4ecc8689d1291b01793efab3ac25125e14e48
Author: Denys Vlasenko 
Date:   Mon Aug 11 20:33:18 2014 +0200

init: if libc-based Unicode support is on, run setlocale(LC_ALL,
"") at startup

It is not clear why we were excluding init from this.
The "getpid() != 1" clause was there from the beginning (2001)
but not explained.

Signed-off-by: Denys Vlasenko 

diff --git a/libbb/appletlib.c b/libbb/appletlib.c
index f7c416e..a015085 100644
--- a/libbb/appletlib.c
+++ b/libbb/appletlib.c
@@ -184,8 +184,7 @@ void lbb_prepare(const char *applet
 #endif
applet_name = applet;

-   /* Set locale for everybody except 'init' */
-   if (ENABLE_LOCALE_SUPPORT && getpid() != 1)
+   if (ENABLE_LOCALE_SUPPORT)
setlocale(LC_ALL, "");

 #if ENABLE_FEATURE_INDIVIDUAL
___
busybox mailing list
busybox@busybox.net
http://lists.busybox.net/mailman/listinfo/busybox

Re: How do I (unconditionally) enable unicode support in busybox?

[Denys Vlasenko]

On Mon, Aug 11, 2014 at 8:22 PM, Harald Becker  wrote:
> Hi Denys !
>
> I looked a bit into James problem and were able to reproduce:
>
> sed   s/./x/g
>
> on an UTF-8 locale matches every byte not on character as expected. This at
> least on current Ubuntu standard build of Buybox. Even if other UTF support
> seam to work, the regex scanner hast some trouble. This needs some more
> investigation.

Unfortunately, short of reimplementing regexp, we have to rely on libc here.
Which means, regexps will not be Unicode-aware if
CONFIG_UNICODE_USING_LOCALE is not set.


Or am I missing something? Do we also have a bug in
CONFIG_UNICODE_USING_LOCALE=y case?
___
busybox mailing list
busybox@busybox.net
http://lists.busybox.net/mailman/listinfo/busybox

Re: How do I (unconditionally) enable unicode support in busybox?

[Denys Vlasenko]

On Mon, Aug 11, 2014 at 8:12 PM, James Bowlin  wrote:
> On Mon, Aug 11, 2014 at 08:02 PM, Denys Vlasenko said:
>> Looks like you want to set CONFIG_UNICODE_SUPPORT=y and unset
>> both of these latter options.  This way, all busybox applets
>> should always work in Unicode mode.
>
> That set up does not work for sed, although it sometimes works.

I just tried your sed example.

busybox sed code uses regexp routines for s///.
Those routines are part of libc. Therefore, they will be Unicode-aware
only if you are using CONFIG_UNICODE_USING_LOCALE=y.

I just built busybox against glibc with these options:

CONFIG_LOCALE_SUPPORT=y
CONFIG_UNICODE_SUPPORT=y
CONFIG_UNICODE_USING_LOCALE=y
# CONFIG_FEATURE_CHECK_UNICODE_IN_ENV is not set
CONFIG_SUBST_WCHAR=63
CONFIG_LAST_SUPPORTED_WCHAR=4351
# CONFIG_UNICODE_COMBINING_WCHARS is not set
# CONFIG_UNICODE_WIDE_WCHARS is not set
# CONFIG_UNICODE_BIDI_SUPPORT is not set
# CONFIG_UNICODE_NEUTRAL_TABLE is not set
# CONFIG_UNICODE_PRESERVE_BROKEN is not set

And I'm getting this:

$ export LANG=en_US.UTF-8
$ echo ÀÀÀ | ./busybox sed 's/./x/g' | wc -c
4

> The most mysterious thing is that it sometimes works and what
> I need to do to get it to work in /init in an initrd.

You may have found a bug. bbox never runs setlocale()
for init. According to git log, this behavior was there from the very beginning:


commit e5dfced23a904d08afa5dcee190c3c3d845d9f50
Author: Eric Andersen 
Date:   Mon Apr 9 22:48:12 2001 +

Apply Vladimir's latest cleanup patch.

...
...
+#ifdef BB_LOCALE_SUPPORT
+   if(getpid()!=1) /* Do not set locale for `init' */
+   setlocale(LC_ALL, "");
+#endif


This probably should be changed so that init is not special.

As to your other cases, they are interesting too.
For example, you noticed that ${#VAR} handling is buggy.
Even on the above mentioned build, I get this

$ export LANG=en_US.UTF-8
$ ./busybox sh
/home/srcdevel/bbox/fix/busybox.4z $ a=ÀÀÀ; echo ${#a}
6

whereas "standard" shell gives 3.

This is clearly a bug (or at least "incompatibility").
Please report each such bug separately.

> The "wc -m" solution always works (for me) so my problem is solved.
> But there is still a strange problem with sed and unicode that
> Harald was able to reproduce.

Which problem? There are so many mails in this thread...
___
busybox mailing list
busybox@busybox.net
http://lists.busybox.net/mailman/listinfo/busybox

Re: How do I (unconditionally) enable unicode support in busybox?

[Harald Becker]

Hi Denys !

I looked a bit into James problem and were able to reproduce:

sed   s/./x/g

on an UTF-8 locale matches every byte not on character as expected. This 
at least on current Ubuntu standard build of Buybox. Even if other UTF 
support seam to work, the regex scanner hast some trouble. This needs 
some more investigation.


It does not seam to be a problem of setting LANG or chroot etc., may be 
it just depends on problems of uClibc.


--
Harald

___
busybox mailing list
busybox@busybox.net
http://lists.busybox.net/mailman/listinfo/busybox

Re: How do I (unconditionally) enable unicode support in busybox?

[James Bowlin]

On Mon, Aug 11, 2014 at 08:02 PM, Denys Vlasenko said:
> Looks like you want to set CONFIG_UNICODE_SUPPORT=y and unset
> both of these latter options.  This way, all busybox applets
> should always work in Unicode mode.

That set up does not work for sed, although it sometimes works.
The most mysterious thing is that it sometimes works and what
I need to do to get it to work in /init in an initrd.

In his last few posts in this thread, Harald Becker confirms this
is a problem.  He also suggested I use "wc -m" instead of:

sed 's/./x/g' | wc -c

Or my tr hack.

The "wc -m" solution always works (for me) so my problem is solved.
But there is still a strange problem with sed and unicode that
Harald was able to reproduce.


Peace, James
___
busybox mailing list
busybox@busybox.net
http://lists.busybox.net/mailman/listinfo/busybox

Re: How do I (unconditionally) enable unicode support in busybox?

[Denys Vlasenko]

On Mon, Aug 4, 2014 at 7:50 AM, James Bowlin  wrote:
> I run busybox in an initrd (initramfs) environment using both
> legacy Grub and isolinux as boot loaders.  I want to be able to
> get the correct length of unicode strings in characters, not
> bytes.  I always have these two options set:
>
> CONFIG_UNICODE_SUPPORT=y
> CONFIG_UNICODE_WIDE_WCHARS=y

Correct. Without CONFIG_UNICODE_SUPPORT=y,
busybox would simply pretend that every byte
is one character.

> and I've played with the 3 combinations of:
>
> CONFIG_UNICODE_USING_LOCALE
> CONFIG_FEATURE_CHECK_UNICODE_IN_ENV
>
> both off, and one or the other on.

I tried to supply help texts for these options.
Please let me know if they are not clear enough:

config UNICODE_USING_LOCALE
bool "Use libc routines for Unicode (else uses internal ones)"
default n
depends on UNICODE_SUPPORT && LOCALE_SUPPORT
help
  With this option on, Unicode support is implemented using libc
  routines. Otherwise, internal implementation is used.
  Internal implementation is smaller.

config FEATURE_CHECK_UNICODE_IN_ENV
bool "Check $LC_ALL, $LC_CTYPE and $LANG environment variables"
default n
depends on UNICODE_SUPPORT && !UNICODE_USING_LOCALE
help
  With this option on, Unicode support is activated
  only if locale-related variables have the value of the form
  ".utf8"

  Otherwise, Unicode support will be always enabled and active.


How it is intended to work:
If you are building busybox against libc with locale support,
you can select CONFIG_UNICODE_USING_LOCALE=y.
In this case, Unicode-ness will be determined by libc -
busybox just calls setlocale():

/* Set locale for everybody except 'init' */
if (ENABLE_LOCALE_SUPPORT && getpid() != 1)
setlocale(LC_ALL, "");

and thus, it's up to libc code in setlocale() to set up things.
(Note: in this case, busybox code has no control whether $LANG
has some effect or not: it depends on *libc*. For glibc,
$LANG will control Unicode; for some other libc, it may not).

If your libc has no locale support, you can't use
CONFIG_UNICODE_USING_LOCALE=y.

You also can decide to not use it even though libc supports it
(say, because libc locale code is big). If it is unset, an internal
implementation will be in use.

In this case, FEATURE_CHECK_UNICODE_IN_ENV controls
whether Unicode is "always on" or it depends on $LANG.

Looks like you want to set CONFIG_UNICODE_SUPPORT=y
and unset both of these latter options.
This way, all busybox applets should always work
in Unicode mode.

Please let me know if that setup does not work for you.
___
busybox mailing list
busybox@busybox.net
http://lists.busybox.net/mailman/listinfo/busybox

Re: shutdown busybox and start another PID1 process

[Harald Becker]

Hi James,

I looked a bit in what you are doing (if I understand it right) and 
suggest the following (doing something similar on some of my systems):



Startup your system from initramfs and setup required root file system

do switch_root starting /bin/busybox init on your root file system

with inittab containing (among others):

::shutdown:SHUTDOWN_ACTION
::restart:/bin/busybox sh YOUR_SHUTDOWN_SCRIPT

when done with your system, send PID #1 the SIGQUIT signal

this shall do all shutdown actions from your inittab and then exec in 
shell with your given shutdown script (shell running as pid #1).


reload a busybox system into your original root file system (has bean 
cleared by switch_root to save space) e.g. from a cpio archiv (better 
than tar).


pivot_root to your original root system

unmount your stacked file systems

do whatever backup you need (e.g. copy from tmpfs to persistent storage)

poweroff/reboot your system



Did I get it right?

--
Harald

___
busybox mailing list
busybox@busybox.net
http://lists.busybox.net/mailman/listinfo/busybox

Re: shutdown busybox and start another PID1 process

[James B]

Hi James (from another James).

On Mon, 11 Aug 2014 03:17:52 -0600
James Bowlin  wrote:

> 
> There was a need to have static root persistence where the
> changes are saved directly in the root persistence file so I
> needed to bite the bullet and finally get clean umounts during
> shutdown.
> 
> Previously, filesystems were consistently reporting they were
> uncleanly mounted.  I wanted to do it all cleanly.  I tried like
> crazy to get remounting read-only to work but I just could not do
> it.  Maybe the problem was that I couldn't remount the rw file
> system under the aufs as ro (because the aufs was rw) and I
> couldn't remount the aufs as ro because its mount options are
> unconventional.

This is a solved problem. Fatdog64, an 64-bit offshot of Puppy Linux 
(disclaimer: I'm one of the maintainers), does exactly this - and have clean 
unmounts on common filesystems - fat32, ext2/3/4, ntfs (through ntfs-3g). The 
key is to remount ro the aufs layers; once made ro you can "unbind" them one by 
one (including the r/w layer). This ability to dynamically bind/unbind layers 
is one of the thing that makes me loves aufs so much - it is so much more 
versatile than any other layered filesystem out there *combined*. Get the ISO 
and take a look at its /etc/rc.d/rc.cleanup if you're interested.

> 
> Well busybox has all the tools I need.  The tmpfs gives me a place
> to stand so I can move the world^H^H aufs file system. It gives me
> a place where my tools and everything I need are off of the aufs.

Agreed, busybox is all you need.

> 
> I pivot onto a tmpfs that has the busybox stuff and very little
> else.  Root could easily erase it all.  Since it is tmpfs there
> is no need to umount it at all.  The point is to finish the
> shutdown from a tmpfs so I can cleanly umount the aufs and the
> filesystems under it including the file system on the host
> machine or on the LiveUSB.

My latest solution does pivot_root too for cleanliness reason, but it is 
actually unnecessary. You can still be on your original aufs root and take off 
the layer one by one - if you're careful - and have a clean unmount.

> 
> >   Hehe. I like to say that initrd/initramfs is useless, [...]
> 
> It is required for a live system.  

James, you won't convert Laurent :) I tried that before. 
I however is in the complete agreement with you. Initramfs is indispensable for 
this.
Anyway, to each his own.

---

If you really want to tell busybox init to quit - the simplest thing is to send 
SIGQUIT to it (kill -3 1). Once you do that, busybox init will execute all the 
::shutdown scripts in /etc/inittab; and when done, it will exec to anything you 
put in ::restart. Whatever command you put in ::restart will run as pid 1 when 
all is said and done. This is however isn't useful for clean unmount because if 
you happen to use ntfs-3g, it will get killed too before it has the chance to 
unmount cleanly.

cheers!
James

-- 
James B 
___
busybox mailing list
busybox@busybox.net
http://lists.busybox.net/mailman/listinfo/busybox

Re: How do I (unconditionally) enable unicode support in busybox?

[Harald Becker]

Hi James!

> I am dubious about using a pre-built busybox because I am picky

about the config.  There are some less often used commands that I
need and there are many often used commands that I don't need and
don't want.


The question was not to force you to a prebuild version forever. The 
idea behind this was to see if your problem persists with one of the 
prebuild images. If yes we are able to reproduce the problem and 
obviously hit a bug (whichever kind it is). Taking prebuild images is 
the quick try and error check to narrow down the problem and eliminate 
simple usage errors during build step.



My system boots with the new BB I made with uclibc and it
contains the selection of commands I want.


... but will never be a prove of possible error you may have done during 
build.


Using a prebuild image is testing like a binary search: Split of the 
possible reasons anywhere in the middle and see in which part the error 
persists. So you narrowed down the big mass of errors that could be the 
reason for a problem.


--
Harald

___
busybox mailing list
busybox@busybox.net
http://lists.busybox.net/mailman/listinfo/busybox

Re: shutdown busybox and start another PID1 process

[Harald Becker]

Hi Laurent!

>> Before I can umount the ntfs (or other rw fs) I need to first

umount the squashfs file and the static persistence file (if it
is being used).  To do that I need to first umount the aufs which
is the root file system.  Without the pivot_root, this is not
possible.


  Isn't it possible to simply remount it read-only ? That's the
traditional way of fscking a rootfs, and it has always worked for
me - but maybe it's unapplicable to your situation, especially
with a HD install that users can modify.


The problem is the stacking of the file systems. You may remount the 
logical root file system read-only, but if the file system sits on top 
of a writeable file system this system is still blocked. You can't 
unmount this lower system until you finished unmounting of the higher 
system (not possible if root). Even lazy unmounting doesn't solve this 
(unmount scheduled forever but never executed).


--
Harald

___
busybox mailing list
busybox@busybox.net
http://lists.busybox.net/mailman/listinfo/busybox

Re: How do I (unconditionally) enable unicode support in busybox?

[Harald Becker]

Hi James!


export LANG=en_US.UTF-8
echo -n "$*" | wc -m


Yes! That works with both glibc and uclibc in the chroot and in
the initrd.  Thank you!


Fine, at least we could solve your major problem!

You may try: LANG=... wc -m (may be that works too).


I didn't know about the -m option to wc.  I had ASSumed -c was
for counting "c"haracters.  My bad.


Nothing wrong. Nobody can ever know everything.


IMO there is still something very strange with sed and unicode


YES! I did not stop looking for this. Looks like this is a problem in 
the regular expression parser.


s /./x/g

shall match every character and replace with a single x, but indeed it 
matches every byte of UTF-8 characters too (which is wrong). But this 
doesn't seam to depend on setting of LANG (which confused me). Is it 
possible, it only worked when BB is linked with glibc in a fully 
functional environment. Maybe than an UTF-8 aware regex scanner is used. 
We need to look further on this!


In addition on UTF-8 locales other awk display number of characters for 
the length() function. BB awk always displays number of bytes. Don't 
know which is right, just detected the difference. It belongs to same 
type of problem, reported by James. Only his initial detection of the 
reason seams to be not fully correct.



it is broken but that it sometimes works.  I can send you my
glibc version of busybox where the sed always works from my
command line and sometimes works in the initrd.


Not required, I finally managed to reproduce your problem :)


Thank you!  I really appreciate your help and your patience.


Sorry, when I initially stuck on the LANG question. Forget to focus on 
your major problem.



I feel like a kid in a candy store with this mailing list
although I am starting to get a tummy ache.  I've had a blast and
I've learned a lot but I hope I can tear myself away in order to
deal with some other pressing things that are on my plate.


Hey, you were right. There is a BB problem, but just to learn once more. 
Give the complete picture of what you are doing in which environment, 
else it may be difficult to detect the misbehaving problem and much to 
easy to stuck on a "user error" topic.


And again to note by others: James was right. There is at least on 
UTF-8 related problem in BB sed. Don't know what exactly fails, but 
regex matches wrong for uClibc linked version of BB. The dot "." shall 
match single characters not bytes. Anybody here who knows more on this 
topic?


--
Harald

___
busybox mailing list
busybox@busybox.net
http://lists.busybox.net/mailman/listinfo/busybox

Re: [PATCH] taskset: support more than 64 cores

[Matthieu Ternisien d'Ouville]

Yes, busybox is used on small system or small OS. In our case, all cores
are dedicated to network functions and the OS is minimal.


On Mon, Aug 11, 2014 at 3:25 PM, Laszlo Papp  wrote:

> I wonder about the use case for this feature? I mean busybox is meant for
> small systems in general, and >=64 cores are not that small embedded
> systems, at least not yet, yeah?
>
>
> On Mon, Aug 11, 2014 at 1:58 PM, Matthieu Ternisien d'Ouville <
> matthieu@6wind.com> wrote:
>
>> This patch adds support of more than 64 cores for taskset.
>>
>> Signed-off-by: Matthieu Ternisien d'Ouville 
>> ---
>>  miscutils/taskset.c |   39 +++
>>  1 file changed, 31 insertions(+), 8 deletions(-)
>>
>> diff --git a/miscutils/taskset.c b/miscutils/taskset.c
>> index 4a9e323..2695e20 100644
>> --- a/miscutils/taskset.c
>> +++ b/miscutils/taskset.c
>> @@ -129,16 +129,39 @@ int taskset_main(int argc UNUSED_PARAM, char **argv)
>> }
>>
>> { /* Affinity was specified, translate it into cpu_set_t */
>> -   unsigned i;
>> -   /* Do not allow zero mask: */
>> -   unsigned long long m = xstrtoull_range(aff, 0, 1,
>> ULLONG_MAX);
>> -   enum { CNT_BIT = CPU_SETSIZE < sizeof(m)*8 ? CPU_SETSIZE
>> : sizeof(m)*8 };
>> +   long cpu;
>> +   int len = strlen(aff);
>> +   const char *ptr = aff + len - 1;
>> +   const char *prefix = "";
>> +
>> +   if (len > 1 && !memcmp(aff, "0x", 2)) {
>> +   aff += 2;
>> +   prefix = "0x";
>> +   }
>>
>> CPU_ZERO(&mask);
>> -   for (i = 0; i < CNT_BIT; i++) {
>> -   unsigned long long bit = (1ULL << i);
>> -   if (bit & m)
>> -   CPU_SET(i, &mask);
>> +   cpu = 0;
>> +
>> +   for (; ptr >= aff; ptr--) {
>> +   int val;
>> +
>> +   if (*ptr <= '9' && *ptr >= '0')
>> +   val = *ptr - '0';
>> +   else if (*ptr <= 'f' && *ptr >= 'a')
>> +   val = *ptr + (10 - 'a');
>> +   else
>> +   bb_perror_msg_and_die("invalid mask
>> %s%s", prefix, aff);
>> +
>> +   if (val & 1)
>> +   CPU_SET(cpu, &mask);
>> +   if (val & 2)
>> +   CPU_SET(cpu+1, &mask);
>> +   if (val & 4)
>> +   CPU_SET(cpu+2, &mask);
>> +   if (val & 8)
>> +   CPU_SET(cpu+3, &mask);
>> +
>> +   cpu += 4;
>> }
>> }
>>
>> --
>> 1.7.10.4
>>
>> ___
>> busybox mailing list
>> busybox@busybox.net
>> http://lists.busybox.net/mailman/listinfo/busybox
>>
>
>
___
busybox mailing list
busybox@busybox.net
http://lists.busybox.net/mailman/listinfo/busybox

Re: [PATCH] taskset: support more than 64 cores

[Laszlo Papp]

I wonder about the use case for this feature? I mean busybox is meant for
small systems in general, and >=64 cores are not that small embedded
systems, at least not yet, yeah?


On Mon, Aug 11, 2014 at 1:58 PM, Matthieu Ternisien d'Ouville <
matthieu@6wind.com> wrote:

> This patch adds support of more than 64 cores for taskset.
>
> Signed-off-by: Matthieu Ternisien d'Ouville 
> ---
>  miscutils/taskset.c |   39 +++
>  1 file changed, 31 insertions(+), 8 deletions(-)
>
> diff --git a/miscutils/taskset.c b/miscutils/taskset.c
> index 4a9e323..2695e20 100644
> --- a/miscutils/taskset.c
> +++ b/miscutils/taskset.c
> @@ -129,16 +129,39 @@ int taskset_main(int argc UNUSED_PARAM, char **argv)
> }
>
> { /* Affinity was specified, translate it into cpu_set_t */
> -   unsigned i;
> -   /* Do not allow zero mask: */
> -   unsigned long long m = xstrtoull_range(aff, 0, 1,
> ULLONG_MAX);
> -   enum { CNT_BIT = CPU_SETSIZE < sizeof(m)*8 ? CPU_SETSIZE :
> sizeof(m)*8 };
> +   long cpu;
> +   int len = strlen(aff);
> +   const char *ptr = aff + len - 1;
> +   const char *prefix = "";
> +
> +   if (len > 1 && !memcmp(aff, "0x", 2)) {
> +   aff += 2;
> +   prefix = "0x";
> +   }
>
> CPU_ZERO(&mask);
> -   for (i = 0; i < CNT_BIT; i++) {
> -   unsigned long long bit = (1ULL << i);
> -   if (bit & m)
> -   CPU_SET(i, &mask);
> +   cpu = 0;
> +
> +   for (; ptr >= aff; ptr--) {
> +   int val;
> +
> +   if (*ptr <= '9' && *ptr >= '0')
> +   val = *ptr - '0';
> +   else if (*ptr <= 'f' && *ptr >= 'a')
> +   val = *ptr + (10 - 'a');
> +   else
> +   bb_perror_msg_and_die("invalid mask %s%s",
> prefix, aff);
> +
> +   if (val & 1)
> +   CPU_SET(cpu, &mask);
> +   if (val & 2)
> +   CPU_SET(cpu+1, &mask);
> +   if (val & 4)
> +   CPU_SET(cpu+2, &mask);
> +   if (val & 8)
> +   CPU_SET(cpu+3, &mask);
> +
> +   cpu += 4;
> }
> }
>
> --
> 1.7.10.4
>
> ___
> busybox mailing list
> busybox@busybox.net
> http://lists.busybox.net/mailman/listinfo/busybox
>
___
busybox mailing list
busybox@busybox.net
http://lists.busybox.net/mailman/listinfo/busybox

[PATCH] taskset: support more than 64 cores

[Matthieu Ternisien d'Ouville]

This patch adds support of more than 64 cores for taskset.

Signed-off-by: Matthieu Ternisien d'Ouville 
---
 miscutils/taskset.c |   39 +++
 1 file changed, 31 insertions(+), 8 deletions(-)

diff --git a/miscutils/taskset.c b/miscutils/taskset.c
index 4a9e323..2695e20 100644
--- a/miscutils/taskset.c
+++ b/miscutils/taskset.c
@@ -129,16 +129,39 @@ int taskset_main(int argc UNUSED_PARAM, char **argv)
}
 
{ /* Affinity was specified, translate it into cpu_set_t */
-   unsigned i;
-   /* Do not allow zero mask: */
-   unsigned long long m = xstrtoull_range(aff, 0, 1, ULLONG_MAX);
-   enum { CNT_BIT = CPU_SETSIZE < sizeof(m)*8 ? CPU_SETSIZE : 
sizeof(m)*8 };
+   long cpu;
+   int len = strlen(aff);
+   const char *ptr = aff + len - 1;
+   const char *prefix = "";
+
+   if (len > 1 && !memcmp(aff, "0x", 2)) {
+   aff += 2;
+   prefix = "0x";
+   }
 
CPU_ZERO(&mask);
-   for (i = 0; i < CNT_BIT; i++) {
-   unsigned long long bit = (1ULL << i);
-   if (bit & m)
-   CPU_SET(i, &mask);
+   cpu = 0;
+
+   for (; ptr >= aff; ptr--) {
+   int val;
+
+   if (*ptr <= '9' && *ptr >= '0')
+   val = *ptr - '0';
+   else if (*ptr <= 'f' && *ptr >= 'a')
+   val = *ptr + (10 - 'a');
+   else
+   bb_perror_msg_and_die("invalid mask %s%s", 
prefix, aff);
+
+   if (val & 1)
+   CPU_SET(cpu, &mask);
+   if (val & 2)
+   CPU_SET(cpu+1, &mask);
+   if (val & 4)
+   CPU_SET(cpu+2, &mask);
+   if (val & 8)
+   CPU_SET(cpu+3, &mask);
+
+   cpu += 4;
}
}
 
-- 
1.7.10.4

___
busybox mailing list
busybox@busybox.net
http://lists.busybox.net/mailman/listinfo/busybox

Re: shutdown busybox and start another PID1 process

[James Bowlin]

On Mon, Aug 11, 2014 at 08:06 AM, Laurent Bercot said:

> Isn't it possible to simply remount it read-only ? That's the
> traditional way of fscking a rootfs, and it has always worked
> for me - but maybe it's unapplicable to your situation,
> especially with a HD install that users can modify.

I used to do this but I think the aufs (which combines several file
systems into one) made it not work as well as I wanted.  One of
my workarounds was to use dynamic root persistence where changes
to the fs (the rw part) was still in tmpfs but we copied in to
the tmpfs from our root persistence file on startup and copied
from tmpfs to the root persistence file before shutdown.  This
ensured that all the writing I needed to do happened before the
real shutdown process.

There was a need to have static root persistence where the
changes are saved directly in the root persistence file so I
needed to bite the bullet and finally get clean umounts during
shutdown.

Previously, filesystems were consistently reporting they were
uncleanly mounted.  I wanted to do it all cleanly.  I tried like
crazy to get remounting read-only to work but I just could not do
it.  Maybe the problem was that I couldn't remount the rw file
system under the aufs as ro (because the aufs was rw) and I
couldn't remount the aufs as ro because its mount options are
unconventional.

> I don't think functionality is your argument here: you'll never
> have *more* functionality than a full-fledged system that has
> been live and made it all the way to your shutdown procedure.

Well busybox has all the tools I need.  The tmpfs gives me a place
to stand so I can move the world^H^H aufs file system. It gives me
a place where my tools and everything I need are off of the aufs.

> AIUI, your argument is reliability: by pivot_rooting on a known,
> static, unmodifiable-by-the-user image, you have the guarantee
> that you can perform clean operations even if the user has hosed
> the root filesystem.

I pivot onto a tmpfs that has the busybox stuff and very little
else.  Root could easily erase it all.  Since it is tmpfs there
is no need to umount it at all.  The point is to finish the
shutdown from a tmpfs so I can cleanly umount the aufs and the
filesystems under it including the file system on the host
machine or on the LiveUSB.

>   Hehe. I like to say that initrd/initramfs is useless, [...]

It is required for a live system.  It is needed at the very least
to find the squashfs file and mount the aufs.  Ours does a bunch
more than this.  We offer home persistence and two kinds of root
persistence, and live remastering and the "toram" feature and an
md5 check of almost all of the major files (just not the
bootloader), and probably a bunch of other things.   As I said
before we check the rw filesystems we use to see if they need to
be fscked.  We perform the fsck if it is needed.  In the last
round of development I added the easy frugal install and easy
persistence.  If you select persistence and the persistence
file(s) are not found then we offer to create them for you.
These need to be in place *before* /sbin/init starts because the
root filesystem is based on the aufs which is based on the data
in the root persistence file.

Basically, any futzing around you need to do to create the root
filesystem needs to be done in the initrd.  In addition, by doing
the frugal install and persistence file creation in the initrd,
it makes things much easier and faster for the user.  Previously,
users would have to boot, get passed some error messages saying
files were missing to do what they selected and then they had to
run some program(s) to create the files (or do the frugal
install) and then reboot.  Now instead of throwing an error or a
warning when the files don't exist to do what the user asked to
do, I create them on the spot.  There were a number of standard
complaints we used to get on the forums regarding the convoluted
reboot process.  We don't get these complaints any more.

> it adds more complexity than it helps you.

We've certainly added a lot of complexity.  OTOH it makes it
really easy and convenient for the users.  I have a theory about
this which explains why Flash beat Java in the browser even
though Java had almost every conceivable advantage: first to
market, it came pre-installed, it has a clean code base, it
was a full fledged programming language.

Flash won because they absorbed the complexity into their
code base, making things easier for both developers and users.
Java took the opposite approach and made their code base as
simple as possible (for easier porting) but this meant you needed
to be a programmer plus a UX designer to work in Java and the
user experience usually sucked unless the developer absorbed
complexity in order to shield the user from it.  In the end Flash
won even though its code base was/is a nightmare because you
didn't need to be a programmer to write in it.  It was designed
to be programmed by graphics designer.  This, in