Re: [Toybox] [PATCH] vi: rename `-s` flag to `-c`
hello
On Wed, Jun 5, 2024 at 2:02 AM Eric Roshan-Eisner
wrote:
>
> POSIX, busybox vi, vim, and nvim all use -c.
-c is not same as -s {script}. While -c is implemented in few vi
clones its not same as -s since -c reads EX commands. -s {script}
implemeted here is vim spesific and not implemented on regular vi.
taken from vim man page:
-s {scriptin}
The script file {scriptin} is read. The characters in
the file are interpreted as if you had typed them.
The same can be done with the command ":source!
{scriptin}". If the end of the file is reached before
the editor exits, further characters are read from the
keyboard.
-c {command}
{command} will be executed after the first file has
been read. {command} is interpreted as an Ex command.
If the {command} contains spaces it must be enclosed
in double quotes (this depends on the shell that is
used). Example: vim "+set si" main.c
Note: You can use up to 10 "+" or "-c" commands.
You cannot test against other vi clones with -c after this patch. But
you could have test against vim with -s {script} implementation. I
used vim as reference for testing with original test case files.
Ex command only switch -c could be added as addition to -s if you
wanna achieve something with ex commands, but maybe dont delete -s
implementation, unless you have better way to test vi mode motions.
> Toybox mailing list
> Toybox@lists.landley.net
> http://lists.landley.net/listinfo.cgi/toybox-landley.net
br.
Jarno
___
Toybox mailing list
Toybox@lists.landley.net
http://lists.landley.net/listinfo.cgi/toybox-landley.net
Re: [Toybox] Microsoft github took down the xz repo.
On Sun, Apr 14, 2024 at 9:14 AM Oliver Webb via Toybox wrote: > > To revive a old thread with new technical info I stumbled upon: > > On Saturday, March 30th, 2024 at 15:58, Rob Landley wrote: > > > I set up gitea for Jeff on a j-core internal server, and it was fine except > > it > > used a BUNCH of memory and cpu for very vew users. Running cgi on > > dreamhost's > > servers is a bother at the best of times (I don't want to worry about > > exploits), > > and the available memory/CPU there is wind-up toy levels. > > > > My website is a bunch of static pages rsynced into place, some of which use > > xbithack to enable a crude #include syntax, and that's about what the > > server can > > handle. > > Going through the list of "minimal tools" on https://suckless.org/rocks/, I > stumbled > upon a git frontend called stagit > (https://git.codemadness.org/stagit/file/README.html) > which the suckless project uses as it's git frontend. And from looking at it, > works purely > via static pages and is fairly minimal (2000 LOC in pure C with only non-libc > dependency being > libgit2. One of it's "cons" is that it's "slow on repo's with 2000+ commits > due to diff > generation", And then says that it takes 3s to run on a repo with 1500 > commits). Have you considered it? > > The main downside of it is that it's MIT Licensed, which means if you wanted > to bundle it > in with the main repo there could be be licensing kerfuffles to deal with. > > But to have a solution, you must have a problem. The 2 main issues I have > with the current git management > are the fact there doesn't seem to be a way to clone the current repo > directly from landley.net (Making Microsoft > GitHub the middleman). And the fact I can't browse the source code without > github or android code search acting as > the middleman (This is a problem I've actually ran into on networks where > github is blocked for some utterly insane > reason, meaning I have to go to android code search to read through the code > occasionally). have you tried cloning the repo? seems to work fine to me. $> git clone https://landley.net/toybox/git/ toybox > > The first of these seems near impossible to solve with _only_ static > webpages, since a git server is not > a static thing. I don't know yet tho. Git repo with read-only access over http(s) is a static thing. you dont need dedicated git spesific server software in order to give simple pull access. Just host your git folder with any generic http/https server that can host folder of files and knows atleast simple GET request. https://dev.to/chr15m/git-hacks-self-host-a-minimal-git-repo-over-ssh-388h so you can just serve your git folder with any http server (example with bloated python3. httpd should work fine but needs more setup) $>python -m http.server --directory whereever/mybare_repo.git then clone $>git clone http://localhost:8000/ test > The second one seems easier though, copying or maybe symlinking stuff from > the source directory with "find" > in a pipeline with bash to make a simple, browse-able tree would take > probably take a few dozen lines of > at most, It could also probably just be an rsync command if you don't want to > worry about listing out directory > contents. > > Both of these problems are remediable now, but in a year they might not be > (ProtonMail just said "If you > don't sign in for a long enough time we will delete all your data" like > Google drive is doing, it's not > hard to imagine Microsoft GitHub doing a similar thing with accounts they > locked out by their 2FA crusade) > Rob, Are you interested in future-proofing the codebase from whatever GitHub > and AOSP decide to do? This is a situation > where I'd normally create a patch and let Rob decide whether to apply it or > not. But since I probably shouldn't > put the source tree inside the commit/ folder, and this operation can be done > with a fancy rsync invocation > I'm not sure if writing any code is the correct solution. > > Thanks, > > - Oliver Webb > > > ___ > Toybox mailing list > Toybox@lists.landley.net > http://lists.landley.net/listinfo.cgi/toybox-landley.net Jarno ___ Toybox mailing list Toybox@lists.landley.net http://lists.landley.net/listinfo.cgi/toybox-landley.net
Re: [Toybox] [PATCH] nuke wcrtomb()
there is slight difference between wctoutf8 and wcrtomb, wcrtomb
returns -1 if its presented with non valid char, of its char is not
presentable on current locale. I think wctoutf8 only returns positive
integers.
so your patch to strlower for example might introduce unreachable code path.
diff --git a/lib/lib.c b/lib/lib.c
index 6a4a77dd..79ae2a1d 100644
--- a/lib/lib.c
+++ b/lib/lib.c
@@ -426,7 +426,7 @@ char *strlower(char *s)
// if we had a valid utf8 sequence, convert it to lower case, and can't
// encode back to utf8, something is wrong with your libc. But just
// in case somebody finds an exploit...
-len = wcrtomb(new, c, 0);
+len = wctoutf8(new, c);
if (len < 1) error_exit("bad utf8 %x", (int)c);
new += len;
On Wed, Apr 10, 2024 at 12:54 AM Oliver Webb via Toybox
wrote:
>
> Not pulling in 2 localization functions (One from libc, one from lib.c)
> reduces executable
> size, also more portable on glibc systems because locale installation
> nonsense. No
> typecasting to int's either.
>
> 15 bytes saved in bloatcheck. tests pass for everything with changes applied
> (Except the shell).
>
> - Oliver Webb
> ___
> Toybox mailing list
> Toybox@lists.landley.net
> http://lists.landley.net/listinfo.cgi/toybox-landley.net
-Jarno
___
Toybox mailing list
Toybox@lists.landley.net
http://lists.landley.net/listinfo.cgi/toybox-landley.net
Re: [Toybox] [PATCH] more.c: More stuff, down cursor key scrolls down. Also stuff about less
On Thu, Mar 21, 2024 at 1:08 AM Rob Landley wrote:
>
> On 3/20/24 11:56, Oliver Webb wrote:
> > On Wednesday, March 20th, 2024 at 11:39, Rob Landley
> > wrote:
> >> More never had the ability to go backwards, less did. Different command.
> >
> >>From the more help text you get when you press "h":
> >
> > b or ctrl-B Skip backwards k screenfuls of text [1]
>
> $ ls -l /bin/more
> -rwxr-xr-x 1 root root 47816 Nov 27 2019 /bin/more
> $ dpkg-query -S /bin/more
> util-linux: /bin/more
> $ cat README | more
>
> I hit space twice to advance, then hit b and ctrl-b a lot, no effect. At a
> guess, that particular gnu/dammit extension which isn't in posix or busybox
> more
> only works on a seekable file.
>
> I personally don't want to complicate more because less exists.
>
> >> > Looking at the other keybindings GNU more provides which I can
> >> > implement, There's "=" (prints current
> >> > line number) ":f" (print filename and line), as well as being able to
> >> > use the down arrow to go down
> >> > (with the added side effect of any escape key doing so too, not the end
> >> > of the world, especially
> >> > since we can't scroll up) That are Implemented them in the attached
> >> > patch.
> >>
> >> Again, more and less are not the same command.
> >
> > No, all of that is more behavior that you can use in more. Try it.
>
> I just did, above.
>
> One easy way to distinguish between less and more is that ctrl-c exits more,
> but
> ctrl-c only kills the command producing less's output while leaving less
> displaying the scrollback buffer. You need to hit 'q' to get out of less
> (unless
> you've hit something like forward slash where q just appends to the string and
> esc doesn't exit that either, but ctrl-c will exit... back to the less
> prompt),
> meaning newbies can get STUCK in less not knowing how to exit it, the way you
> can't in "more".
>
> Oh, here's another difference:
>
> $ { echo -e '\e[42mcolor\e[0m text'; while true; do echo hello; done; } | more
> shows the color change
> $ { echo -e '\e[42mcolor\e[0m text'; while true; do echo hello; done; } | less
> shows the escape sequences
>
> That's why I'm still pondering if they can/should usefully share code.
>
> >> > There is also a testing problem. vi.c doesn't do TEST_HOST because it
> >> > needs a -s option
> >> > to pass in scripts to test with.
> >>
> >> Which is an issue I need to figure out how to address. What does a test
> >> that
> >> only toybox passes actually prove? (That it hasn't changed since we last
> >> looked at it?)
> >
> > There is vi -c which preforms a ex command which we could implement
I took -s from vim, so toybox vi could be tested comparing to vim,
since vi itself does not have -s. And I was not interested in -c since
ex was out of the scope of implementation at that time.
>
> I leave vi to the people who are maintaining that vi. I got out of way for
> that
> command.
>
Well im not sure who is "maintaining" vi.c at this point, I wrote base
implementation years ago, Elliott extended it with few commands,
because he had some use case for it. But mostly development has been
dormant for few years with few segfault bugfix here and there. Its not
very pleasant experience to maintain it, since everything lead to huge
bikeshedding, since there is no particular standard to follow,
everyone want different things. Also from what I understand reading
your postings, you have never been very satisfied on it. And that is
understandable.
> >> I have been planning one all along, yes. The crunch_str() stuff I did was a
> >> first pass at general line handling stuff that could be used by less and by
> >> shell line editing and by vi and so on, but people wrote a vi that does
> >> not and
> >> never will share code with the rest of those so that's off the table
> >> permanently.
vi.c uses crunch_str from lib for utf8 handling, there was just few
corner cases it needs to use vi only crunch_nstr, since it cant spit
up text until nul all the time. vi.c tried to use some other
functionality from lib also, but some of it got removed from lib and
some functionality have probably been added way after vi.c was written
in 2018-2020.
I think less and vi could share some infrastructure, either by taking
stuff from vi.c to lib, or rewriting it into lib and taking it out
from vi. more could be way simpler, since it should not have backward
movement.
> >
> > My experience is in vi.c which is why I mentioned using code from it. I
> > haven't read
> > through top or hexedit
>
> I haven't read through the vi.c in pending.
>
> >> > But I have to ask the question "If it's so easy, why isn't it in toybox
> >> > yet?" Is it just because
> >> > other TODO items taking up time, or is it because it's harder to
> >> > implement than it seems.
> >>
> >> Because I care about edge cases like ansi escapes and utf8 fontmetrics and
> >> resizing the screen partway through displaying, because I haven't got test
> >> suite
> >>
Re: [Toybox] [PATCH] vi.c: Backspace to merge lines when at beginning, get_endline is no more, cleanup
On Fri, Mar 8, 2024 at 3:07 AM Oliver Webb wrote:
>
> On Thursday, March 7th, 2024 at 14:23, Jarno Mäkipää
> wrote:
> [...]
> > Feel free to fix it up, some other features are simpler since you can
> > just read the man page and make it behave accordingly, but backspace
> > is maybe not very trivial one since its either you follow heirloom vi,
> > or make some educated assumption how average person would want to use
> > it. To me this patch is not necessarily fixing it, but making it
> > behave the way you want, maybe more vim way?
>
> The way most people use vi-like text editors today is with
> vim (or one of it's many forks). The same way that people usually interact
> with unix-like operating systems with linux. Making toybox vi act more
> vim-like would it easier to use for 90% of vi users.
I use vim myself, but the thing is vim behavior is not constant
target, every vim user tend to have different vimrc, either modified
by them or by distro vendor.
On my distro there default is vimrc.tiny that does not allow you
backspacing into previous line. So for these features that are not
really standard vi stuff, its just someones personal preference. I
dont personally mind either way.
if you want backspace to jump into previous line, you should not call
ex commands dispatcher in middle of normal mode. but instead just
teach cur_left to ignore /n so it jumps over newline...
static int cur_left_force(int count0, int count1, char *unused)
{
int count = count0*count1;
TT.vi_mov_flag |= 0x8000;
for (;count && TT.cursor; count--) {
TT.cursor--;
check_cursor_bounds();
}
return 1;
}
and call that instead of normal cursor left? would this do the trick
with simpler code path...
size_t from = 0;
size_t to = TT.cursor;
cur_left_force(1, 1, 0);
from = TT.cursor;
if (from != to)
vi_delete(reg, to, 0);
check_cursor_bounds();
return 1;
every delete, yank action should work the same, you find two cursor
locations and delete or yank between them...
>
> > About join lines function I dont see any issues with original version,
> > its doing what it supposed to do, searching newline and taking it out,
> > (with some buffer management magic called piece table method, here is
> > some info if you interested on how it works
> > https://www.cs.unm.edu/~crowley/papers/sds.pdf)
>
> Vim does insert a space in the merged line, which is nice for formatting
> and would fix the "deletion of last character" problem. Dunno how to implement
> it yet without doing ugly stuff with run_vi_cmd() yet
if you want to insert something just use the same function as any
other place in code uses. for example space at cursor position, which
should be correct if you just deleted "/n" at TT.cursor...
insert_str(xstrdup(" "), TT.cursor, 1, 1, HEAP);
but thing is join should not always add space, it was implemented the
simplest way "just join, let user handle whitespace". If you wanna
teach it to handle whitespace better, you need to implement following
from man page:
1. Discard leading characters from the line to be
joined.
2. If the line to be joined is now empty, delete it, and skip
steps 3 through 5.
3. If the current line ends in a , or the first character
of the line to be joined is a ')' character, join the lines
without further modification.
4. If the last character of the current line is a '.', join the
lines with two characters between them.
5. Otherwise, join the lines with a single between them.
>
> > My original idea was to make vi as simple as possible: move with
> > simple motions, delete, yank and push and insert few words with insert
> > mode. Kinda rescue editor, in 1000 lines of code and leave it to Rob
> > to clean up, (or yank out and rewrite.) Then people started using it,
> > at least as rescue editor, so some quality of life patches came in,
> > and now its near 2000 lines.
>
> A minimal development environment needs at least a basic text editor.
> I noticed while trying to do stuff with mkroot (Getting runtest.sh to run on
> it
> so we can test commands like passwd (or nommu support assuming qemu has
> options for it)) That
> vi didn't have any ex commands other then write and quit. Since batch
> processing
> with ex commands is convenient (I know sed exists, but having it built in is
> more useful)
> I tried (am trying) to implement some of them. And once you have ex commands,
> implementing ed
> (and ex) are really easy (a for loop with run_ex_cmd()).
I was not that interested on ex, and I think Rob said ex is obsolete
and not on his roadmap. But yes some of ex commands are just vi normal
mode commands, with special addressing. If someone n
Re: [Toybox] [PATCH] vi.c: Backspace to merge lines when at beginning, get_endline is no more, cleanup
On Thu, Mar 7, 2024 at 6:32 AM Oliver Webb via Toybox wrote: > > Looking at vi.c again, (I _know_ I can write code better then I did in > October, and > since the cleanup pass hasn't happened yet I thought I'd start improving > things now) > noticed there was no support for merging lines when doing a backspace at the > start > of a line (e.g. deleting blank lines by pressing backspace) So I added that. > There > is a bug where it will delete the last character of the line you are merging > into, > Still. That behavior is considerably better than nothing (And dunno how to > debug it, > vi_join isn't doing it's job for some reason) so I added it into the patch. Well I am unsure what backspace should do on vi, I dont remember why I even added it. If i look vi man page there is no mention of backspace existing. I dont personally backspace except while typing on insert mode or ex mode and only the data I am about to insert not the data already inside buffer. Perhaps I added backspace delete for buffer stuff since someone request it on mailinglist, but cant find who, when, and why. If looking other vi clones by just using them, and comparing backspace behavior nvi does the thing how i originally intended, backspace can only be used to cut your typo out of your ongoing insert, not to delete stuff from buffer. and it does not move your cursor backward inside buffer vim on :set compatible; behaves almost same as nvi. it does not delete stuff from buffer but it allows you to move cursor left while in normal mode, but only up until newline vim on :set nocompatible deletes stuff from buffer if in insert mode, and goes backward on normal mode, but jumps into previous line busybox vi, has bit similar behavior as vim in compatible mode, but it has its own weirdness too (I backspaced around, press undo 2 times and it inserted some garbage into my open buffer. ) Feel free to fix it up, some other features are simpler since you can just read the man page and make it behave accordingly, but backspace is maybe not very trivial one since its either you follow heirloom vi, or make some educated assumption how average person would want to use it. To me this patch is not necessarily fixing it, but making it behave the way you want, maybe more vim way? About join lines function I dont see any issues with original version, its doing what it supposed to do, searching newline and taking it out, (with some buffer management magic called piece table method, here is some info if you interested on how it works https://www.cs.unm.edu/~crowley/papers/sds.pdf) My original idea was to make vi as simple as possible: move with simple motions, delete, yank and push and insert few words with insert mode. Kinda rescue editor, in 1000 lines of code and leave it to Rob to clean up, (or yank out and rewrite.) Then people started using it, at least as rescue editor, so some quality of life patches came in, and now its near 2000 lines. br Jarno > > As far as ex commands go, this patch removes the function get_endline, and > replaces it with directly > probing the buffer. Which is much faster then redrawing the page 2 times. > > Other then that, This replaces a couple of variable names and removes a > unnecessary goto > > - Oliver Webb > ___ > Toybox mailing list > Toybox@lists.landley.net > http://lists.landley.net/listinfo.cgi/toybox-landley.net ___ Toybox mailing list Toybox@lists.landley.net http://lists.landley.net/listinfo.cgi/toybox-landley.net
Re: [Toybox] vi 'b' command broken
On Thu, Oct 5, 2023 at 11:15 PM enh via Toybox wrote:
>
> On Wed, Oct 4, 2023 at 11:51 PM Rob Landley wrote:
> >
> > On 10/4/23 21:51, Oliver Webb wrote:
> > > --- Original Message ---
> > > On Wednesday, October 4th, 2023 at 18:59, Rob Landley
> > > wrote:
> > >> On 10/4/23 13:51, enh via Toybox wrote:
> > >>
> > >> > (since it looks like there are folks actively working on vi atm...)
> > >> >
> > >> > looks like 'b' goes to the end of the previous word, rather than
> > >> > the beginning of the current word?
> > >>
> > >>
> > >> There's no 'b' but there is a "b" (which is weird because all the
> > >> vi_mov_param
> > >> are chars so why is that a string) which dispatches to vi_movb() which is
> > >> multiplying count0 by count1. What ARE count0 and count1?
> > >
> > >>From my looking over of the vi.c, count0 and count1 specify arguments to
> > >>commands
> > > ("15G" gives 15 as count0 in vi_go()). As for the multiplying, all my
> > > debug printf's
> > > show count1 being set to 1 so I have no idea why it's multiplying.
> >
> > Ah. command0 is the number before the command, command1 is the number after
> > the
> > command. If you ":5d" it deletes the 5th line. If you ":d5" it deletes 5
> > lines
> > starting from the current one. Presumably if you ":5d5"... yup, it deletes 5
> > lines starting from line 5.
>
> ah, that's why i was confused by two counts --- i didn't know there
> was a postfix count too. (i'd wondered whether they were just bad
> names for the two numbers in something like :1,5s/// instead!)
>
> > Before I wrote my own "sed" I didn't really know how to use sed. Before I
> > wrote
> > my own "find" I know like 1/3 of the things find could do.
>
> judging by some of the sed in the toybox build, you know more sed than
> macOS' sed does :-)
>
> > And the reasong
> > writing my own bash takes so long is I'm learning ALL SORTS of weird corner
> > cases of bash behavior.
> >
> > Here's a fun one I learned today:
> >
> > $ bash -c 'set -e;x() { echo one;false;echo two; }; x || echo three; x'
> > one
> > two
> > one
> >
> > Being on the left side of the || operator disables set -e (exit on error)
> > in a
> > way that permeates INTO shell functions. I need to add a test for that. (And
> > then implement it.)
> >
> > The main reasons I haven't cracked open bc and awk and vi and so on yet is I
> > need to learn to USE them at a level I've never previously needed to. I
> > taught
> > an "intro to unix" course in a previous life which spent two classes on vi
> > and
> > taught the students something like 15 commands which would be on the test.
> > That
> > was "goddess I'm old" many years ago, I remember that "set mark" exists but
> > not
> > what it did. But even then I only taught the students a smattering of what
> > vi
> > can do, and all these years later I remember maybe half of what I taught.
> > (I use
> > a tiny subset of the available commands. Mostly I just type fast.)
> >
> > Luckily my model here isn't vim, it's probably busybox vi. In fact ubuntu's
> > intentionally sabotaged vi (because mark shuttleworth prefers emacs) that
> > doesn't let you cursor around in insert mode until you "sudo ln -f vimrc
> > /etc/vim/vimrc.tiny" (which is part of my install checklist when I upgrade
> > debian versions; I never apt-get dist-update, I back up my home dir and do a
> > fresh install so unreproducible debris doesn't accumulate where my magic
> > system
> > works and nobody else does because I have a 12 year config tweak under
> > /var/lib/pam disabling some stupid default behavior I've long forgotten
> > about,
> > and also so I know the last time I didn't just backup but RESTORED from
> > backup.)
> >
> > And of course the main downside of wanting to implement the subset of vi
> > that
> > busybox implements is their LOVELY documentation:
> >
> > $ ./busybox --help vi
> > BusyBox v1.37.0.git (2023-10-02 07:51:31 CDT) multi-call binary.
> >
> > Usage: vi [-c CMD] [-R] [-H] [FILE]...
> >
> > Edit FILE
> >
> > -c CMD Initial command to run ($EXINIT and ~/.exrc also available)
> > -R Read-only
> > -H List available features
> >
> > And if you run it and :help it says not implemented. So yay. It does, for
> > some
> > reason, have :features which says:
> >
> > These features are available:
> > Pattern searches with / and ?
> > Last command repeat with .
> > Line marking with 'x
> > Named buffers with "x
> > Some colon mode commands with :
> > Settable options with ":set"
> > Signal catching- ^C
> > Job suspend and resume with ^Z
> > Adapt to window re-sizes
> >
> > For whatever help that is.
>
> "some colon mode commands" is especially good. i mean, why even bother
> saying anything if you're going to be that vague?
>
> > Anyway, opening the vi can of worms? If Elliott says it's suddenly on his
> > critical path, I can pivot.
>
> no, i don't care, and i don't
[Toybox] [PATCH] vi: list commands supported in help desc
implemented commands are listed and grouped command functionality is not described --- toys/pending/vi.c | 16 +++- 1 file changed, 15 insertions(+), 1 deletion(-) From eeaa9b5e3181656f847339c8d1bf0566dcae637b Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Jarno=20M=C3=A4kip=C3=A4=C3=A4?= Date: Mon, 2 Oct 2023 11:34:43 +0300 Subject: [PATCH] vi: list commands supported in help desc implelemented commands are listed and grouped command functionality is not described --- toys/pending/vi.c | 16 +++- 1 file changed, 15 insertions(+), 1 deletion(-) diff --git a/toys/pending/vi.c b/toys/pending/vi.c index 385ad7a8..0b51fb28 100644 --- a/toys/pending/vi.c +++ b/toys/pending/vi.c @@ -17,8 +17,22 @@ config VI so the controls are weird and historical. -s script: run script file -*/ +vi mode commands: +[count][cmd][motion] +cmd: c d y +motion: 0 b e G H h j k L l M w $ f F + +[count][cmd] +cmd: D I J O n o p x dd yy + +[cmd] +cmd: / ? : A a i CTRL_D CTRL_B CTRL_E CTRL_F CTRL_Y \e \b + +ex mode commands: +[cmd] +\b \e \n w wq q! 'set list' 'set nolist' d $ % +*/ #define FOR_vi #include "toys.h" #define CTL(a) a-'@' -- 2.39.2 ___ Toybox mailing list Toybox@lists.landley.net http://lists.landley.net/listinfo.cgi/toybox-landley.net
Re: [Toybox] [PATCH] vi.c: line ranges, fixed line gotos, CTRL-D, etc
Hello Oliver ex mode command parsing is not something that has been designed carefully. Its more of hack to accept write file and exit commands in order to make editor somewhat usable. Even the vi mode command execution is not very good. At least one issue is that it has function dispatch table that handles most of the commands and also switch case that handles some... It should probably have only one or the other and not both... I see that you added some direct calls to run_vi_cmd() in ex mode and other places. I am not sure is that going to work out since run_vi_cmd was supposed to be only called when in vi mode. I don’t really use ex commands other than r, w, q so I don’t really have idea how they work. But if you wish to implement some commonly used ones, you could try to implement better parsing logic for ex commands so that they are parsed similar logic as vi mode: do movement, execute action based on movement, repeat if counter has been given. If you plan to add features you could add some tests into /tests/vi.test While tests don’t cover how buffer looks visually in terminal emulator, it can cover how buffer is edited and stored on output file. This can detect at least some regressions. -Jarno On Sun, Oct 1, 2023 at 7:55 AM Oliver Webb via Toybox wrote: > > Heya, I have been working a few days on vi.c and have managed to > add a few things. I have a barely-working version of the "g" command > which I have ultimately decided to omit from this patch. > > First thing I added was line ranges, the ability to specify > stuff like "10,45d" and have it delete lines 10-45, and also "%" > so you can do stuff like "%d" (and "%s" if I can get that working) > It only supports plain numbers and doesn't do anything fancy like "+NUMBER" > yet. > To get line ranges to work properly, The page has to be re-drawn or else the > frame > breaks (At least while deleting lines, the only thing I can do with these > features as of now) > > Also, after some testing I realized that the line gotos that I > implemented didn't work in some circumstances, such as at the > top or bottom of a file and/or after moving around in a file using > the cursor keys. I don't know why it does this... my code called > vi_go() and after some testing, calls it in the exact same > way that the "G" command does. I fixed this by making all line goto's run the > "G" command and let that do whatever it does to prevent that. This is where unit tests would be useful. To test all the corner cases. There might be also some underlying bug relating to arrow keys since they are handled bit different than some other commands. > > I have added in the support for "Go down half a page" CTRL-D, > and added in the CTL macro after > mentioned it. and replaced "27" with "\e" > > - Oliver Webb > ___ > Toybox mailing list > Toybox@lists.landley.net > http://lists.landley.net/listinfo.cgi/toybox-landley.net -Jarno ___ Toybox mailing list Toybox@lists.landley.net http://lists.landley.net/listinfo.cgi/toybox-landley.net
[Toybox] [PATCH] i2cget: add support for reading without passing command
i2cget can be used without passing command byte
i2cget 3 0x50 0x5F <-- should shift out register 0x5F
i2cget 3 0x50 <-- should shift out register 0x60
---
toys/other/i2ctools.c | 5 -
1 file changed, 4 insertions(+), 1 deletion(-)
Related to https://github.com/landley/toybox/issues/414
From a914d1c316bc2c1a7c97994e5d4d7bd2d3cc8380 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Jarno=20M=C3=A4kip=C3=A4=C3=A4?=
Date: Fri, 17 Mar 2023 23:03:10 +0200
Subject: [PATCH] i2cget: add support for reading without passing command
i2cget can be used without passing command byte
i2cget 3 0x50 0x5F <-- should shift out register 0x5F
i2cget 3 0x50 <-- should shift out register 0x60
---
toys/other/i2ctools.c | 5 -
1 file changed, 4 insertions(+), 1 deletion(-)
diff --git a/toys/other/i2ctools.c b/toys/other/i2ctools.c
index 8c9adca9..d20eb1ab 100644
--- a/toys/other/i2ctools.c
+++ b/toys/other/i2ctools.c
@@ -271,7 +271,10 @@ void i2cget_main(void)
confirm("Read register 0x%02x from chip 0x%02x on bus %d?", addr, chip, bus);
fd = i2c_open(bus, FLAG(f) ? I2C_SLAVE_FORCE : I2C_SLAVE, chip);
- if (i2c_read_byte(fd, addr, )==-1) perror_exit("i2c_read_byte");
+ if (toys.optc == 3) {
+if (i2c_read_byte(fd, addr, )==-1) perror_exit("i2c_read_byte");
+ } else if (read(fd, , 1) != 1) perror_exit("i2c_read");
+
printf("0x%02x\n", byte);
close(fd);
}
--
2.34.1
___
Toybox mailing list
Toybox@lists.landley.net
http://lists.landley.net/listinfo.cgi/toybox-landley.net
Re: [Toybox] [PATCH] ps, vi: fix flicker.
On Thu, Feb 23, 2023 at 10:42 PM enh via Toybox wrote: > > My earlier trick to ensure that we buffer whole screens full works fine > on glibc, but both bionic and musl have a tiny 1024-byte BUFSIZ that > makes it unsuitable for this kind of use, even on laptop screens. > > Explicitly say 8192, since 4096 is slightly too small for my larger > laptop's screen (and I don't use a particularly small font). Should setvbuf be used instead of setbuf if passing buffer size other than BUFSIZ? > > At some point we should probably move this into tty.c, dynamically > allocate based on screen size (plus space for escape sequences/non-ASCII > characters), and track SIGWINCH in case the window grows. But this stops > top and vi flickering today, which is good enough for now. (Amusingly, I > hit the vi problem -- which is actually much worse, for dense strace > output -- while debugging the top problem, while debugging the ps > problem, while debugging the thing I was actually supposed to be doing > _yesterday_. So definitely time to back out of a few rat holes!) > --- > toys/pending/vi.c | 2 +- > toys/posix/ps.c | 2 +- > 2 files changed, 2 insertions(+), 2 deletions(-) > ___ > Toybox mailing list > Toybox@lists.landley.net > http://lists.landley.net/listinfo.cgi/toybox-landley.net -Jarno ___ Toybox mailing list Toybox@lists.landley.net http://lists.landley.net/listinfo.cgi/toybox-landley.net
Re: [Toybox] [PATCH] vi: added backspace
On Sat, Feb 4, 2023 at 8:23 AM Rob Landley wrote: > > On 2/3/23 13:29, Jarno Mäkipää wrote: > > On Fri, Feb 3, 2023 at 6:09 PM Rob Landley wrote: > >> > >> On 2/1/23 15:18, enh via Toybox wrote: > >> > heh, to be clear: i wasn't "dissing" drive-by patching. it accounts > >> > for at least 80% of my entire career :-) > >> > >> https://landley.net/toybox/downloads/binaries/mkroot/0.8.9/linux-patches/ > >> > >> > i use the term just to acknowledge that for some things -- like this > >> > -- there isn't anyone else who's actually working on the thing full > >> > time, which is my personal rationale for wanting "the simplest thing > >> > that could possibly work", and why my definition of "simple" is > >> > something like "the most easily understood by an average programmer > >> > who hasn't seen this particular code before". > >> > >> I mean to take custody of this thing this year. I'm trying to get to a 1.0 > >> release, which means (among other things) emptying the pending directory > >> entirely. > > > > Well it might be complete rewrite when you get to it, but hopefully > > its better than starting from nothing :) > > Indeed. > > >> I do admit vi is something I haven't worked out how to regression test > >> yet, but > >> dogfooding it would presumably cover a multitude of sins. (I do edit both > >> my > >> code and my blog in vi...) > > > > I was trying to code with toybox vi, but I seem to be too blind and I > > need syntax hilighting. And I really dont wanna implement it here. Its > > not in scope of this. > > Syntax highlighting that's built _in_, no. But a syntax highlight thingy based > on regexes sounds like it might make sense? > > Although telling it that bash has if/then/fi stages starts to sound sed-ish > and > I'd want to pace and stare at trees quite a bit to come up with an actual idea > there. The point would be vi.c remaining simple and the /etc/vi/c.hi and > /etc/vi/bash.hi files it imported would have all the brains... Sounds smart way to do it. Perhaps something like file containing lists of regexp followed by escape codes user wants for each regex if I understood you correctly. If highlighting would be kept line based, it could be simple enough to just write modified version of crunch_str that would accept array of regex + escapes instead of char * + escape. Then vi.c would not need to know much/anything about highlighting. Ofc this would not support code block highlight such as showing if whole function is commented with c comment blocks /* */ This would reduce need of processing full file/buffer to only selected lines that have been drawn into screen. > > But THAT can of worms belongs AFTER toysh. And after other cleanup/rewrite of > the basic vi functionality. And I still need an awk for a bunch of build > scripts... > > >> > (and, yes, in addition to the open() error -- which at least led to a > >> > small simplification of the code -- i've shot myself in the foot by > >> > forgetting that there even are vi tests, not running them, and > >> > breaking them with my recent commit, which i'll have to do something > >> > about before i can sync to AOSP. > >> > >> Huh, I forgot that too. :) > >> > >> Ok, "vi -s" is a good start... > > > > "vi -s" is ok for text edit operation testing, and tests with weird > > files and such. But visual testing I think using manually with > > different terminal emulators is only way. > > I have a vague idea for automated pty master/slave tests, but it's another can > of worms I dowanna open just now, and it's the kind of thing you have to > design > as you go. Start with a vague idea and let the implementation guide you or > you're just gonna wind up ripping it all out and starting over repeatedly. (I > mean, that's likely to happen twice _anyway_, but the "I dunno what I'm doing > until I've done it" smells strong with this one...) > > > At some point I forked ST to print out escape codes for me when I was > > trying to debug some drawing issues. But fortunately currently drawing > > is just one function (monsterous that needs to be rewritten for sure). > > > > I think that most work is actually going through all movement commands > > corner cases, things like cw and dw has different movement. > > > > And parsing required ex and vi commands better way. But I was hoping > > you can use something from bash implementation perhaps. > > I haven't o
Re: [Toybox] [PATCH] vi: added backspace
On Fri, Feb 3, 2023 at 6:09 PM Rob Landley wrote: > > On 2/1/23 15:18, enh via Toybox wrote: > > heh, to be clear: i wasn't "dissing" drive-by patching. it accounts > > for at least 80% of my entire career :-) > > https://landley.net/toybox/downloads/binaries/mkroot/0.8.9/linux-patches/ > > > i use the term just to acknowledge that for some things -- like this > > -- there isn't anyone else who's actually working on the thing full > > time, which is my personal rationale for wanting "the simplest thing > > that could possibly work", and why my definition of "simple" is > > something like "the most easily understood by an average programmer > > who hasn't seen this particular code before". > > I mean to take custody of this thing this year. I'm trying to get to a 1.0 > release, which means (among other things) emptying the pending directory > entirely. Well it might be complete rewrite when you get to it, but hopefully its better than starting from nothing :) > > I do admit vi is something I haven't worked out how to regression test yet, > but > dogfooding it would presumably cover a multitude of sins. (I do edit both my > code and my blog in vi...) I was trying to code with toybox vi, but I seem to be too blind and I need syntax hilighting. And I really dont wanna implement it here. Its not in scope of this. > > > (and, yes, in addition to the open() error -- which at least led to a > > small simplification of the code -- i've shot myself in the foot by > > forgetting that there even are vi tests, not running them, and > > breaking them with my recent commit, which i'll have to do something > > about before i can sync to AOSP. > > Huh, I forgot that too. :) > > Ok, "vi -s" is a good start... > "vi -s" is ok for text edit operation testing, and tests with weird files and such. But visual testing I think using manually with different terminal emulators is only way. At some point I forked ST to print out escape codes for me when I was trying to debug some drawing issues. But fortunately currently drawing is just one function (monsterous that needs to be rewritten for sure). I think that most work is actually going through all movement commands corner cases, things like cw and dw has different movement. And parsing required ex and vi commands better way. But I was hoping you can use something from bash implementation perhaps. > > i'll admit i'm very tempted to just > > locally ignore those tests for now so i can get people kicking the > > `tar --sort=name` tires asap, and coming back to worry about vi > > later!) > > You have applied quantum indeterminacy to "pending" status. It is > simultaneously > pending and not pending. > > I'm kind of impressed. > > Rob -Jarno > ___ > Toybox mailing list > Toybox@lists.landley.net > http://lists.landley.net/listinfo.cgi/toybox-landley.net ___ Toybox mailing list Toybox@lists.landley.net http://lists.landley.net/listinfo.cgi/toybox-landley.net
[Toybox] 0001-vi-fix-regression-on-wq-add-modified-subroutine.patch
Replaced TT.modified with function that checks if buffer has changed.
This is easier solution than relying that every piece of code that
touch buffers knows how to update some global variable.
Fix problems with empty file, by adding LF on load.
(when checking output of other editors this seemst to be common practice)
---
toys/pending/vi.c | 25 +
From 51474ff51943fe9806bc2f4c91a69f72a4f88ef3 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Jarno=20M=C3=A4kip=C3=A4=C3=A4?=
Date: Fri, 3 Feb 2023 20:08:42 +0200
Subject: [PATCH] vi: fix regression on wq, add modified() subroutine
Replaced TT.modified with function that checks if buffer has changed.
Fix problems with empty file, by adding LF on load.
(when checking output this is behavior most other editors do)
---
toys/pending/vi.c | 25 +
1 file changed, 21 insertions(+), 4 deletions(-)
diff --git a/toys/pending/vi.c b/toys/pending/vi.c
index 77a031ba..30a6e03b 100644
--- a/toys/pending/vi.c
+++ b/toys/pending/vi.c
@@ -42,8 +42,6 @@ GLOBALS(
char* data;
} yank;
- int modified; // TODO: no editing operations actually set this!
-
size_t filesize;
// mem_block contains RO data that is either original file as mmap
// or heap allocated inserted data
@@ -292,6 +290,17 @@ static int cut_str(size_t offset, size_t len)
return 0;
}
+static int modified()
+{
+ if (TT.text->next != TT.text->prev) return 1;
+ if (TT.slices->next != TT.slices->prev) return 1;
+ if (!TT.text || !TT.slices) return 0;
+ if (!TT.text->node || !TT.slices->node) return 0;
+ if (TT.text->node->alloc != MMAP) return 1;
+ if (TT.text->node->len != TT.slices->node->len) return 1;
+ if (!TT.text->node->len) return 1;
+ return 0;
+}
//find offset position in slices
static struct slice_list *slice_offset(size_t *start, size_t offset)
@@ -538,6 +547,7 @@ static void linelist_load(char *filename, int ignore_missing)
if (!filename) filename = TT.filename;
if (!filename) {
// `vi` with no arguments creates a new unnamed file.
+insert_str(xstrdup("\n"), 0, 1, 1, HEAP);
return;
}
@@ -547,6 +557,7 @@ static void linelist_load(char *filename, int ignore_missing)
show_error("Couldn't open \"%s\" for reading: %s", filename,
strerror(errno));
}
+insert_str(xstrdup("\n"), 0, 1, 1, HEAP);
return;
}
@@ -554,6 +565,8 @@ static void linelist_load(char *filename, int ignore_missing)
if (size > 0) {
insert_str(xmmap(0,size,PROT_READ,MAP_SHARED,fd,0), 0, size, size, MMAP);
TT.filesize = text_filesize();
+ } else if (!size) {
+insert_str(xstrdup("\n"), 0, 1, 1, HEAP);
}
xclose(fd);
}
@@ -563,6 +576,9 @@ static int write_file(char *filename)
struct slice_list *s = TT.slices;
struct stat st;
int fd = 0;
+ if (!modified()) {
+show_error("Not modified");
+ }
if (!filename) filename = TT.filename;
if (!filename) {
@@ -1309,13 +1325,14 @@ static int run_ex_cmd(char *cmd)
// TODO: backwards search.
} else if (cmd[0] == ':') {
if (!strcmp([1], "q") || !strcmp([1], "q!")) {
- if (cmd[2] != '!' && TT.modified) {
+ if (cmd[2] != '!' && modified()) {
show_error("Unsaved changes (\"q!\" to ignore)");
} else return 1;
} else if (strstr([1], "w ")) {
write_file([3]);
} else if (strstr([1], "wq")) {
- return write_file(0);
+ if (!write_file(0)) return 1;
+ show_error("Unsaved changes (\"q!\" to ignore)");
} else if (strstr([1], "w")) {
write_file(0);
} else if (strstr([1], "set list")) {
--
2.34.1
___
Toybox mailing list
Toybox@lists.landley.net
http://lists.landley.net/listinfo.cgi/toybox-landley.net
Re: [Toybox] [PATCH] vi: added backspace
On Wed, Feb 1, 2023 at 11:19 PM enh wrote: > > heh, to be clear: i wasn't "dissing" drive-by patching. it accounts > for at least 80% of my entire career :-) > Yeah I understood that. Just made joke since this will probably be just one off patch unless I actually gather some motivation somewhere to actually work on this again :) I just happened to have file open and remembered that insert mode was very wonky for typical "arrow keys user" and already knew how to fix it. I was intending this as quality of life update with minimum effort. > i use the term just to acknowledge that for some things -- like this > -- there isn't anyone else who's actually working on the thing full > time, which is my personal rationale for wanting "the simplest thing > that could possibly work", and why my definition of "simple" is > something like "the most easily understood by an average programmer > who hasn't seen this particular code before". Implementing gap buffer wold simplify some aspects of program. Mainly by allowing some direct usage of familliar standard c library functions for string manipulation and searching. But I think best way to keep it simple is just to limit features. > > (and, yes, in addition to the open() error -- which at least led to a > small simplification of the code -- i've shot myself in the foot by > forgetting that there even are vi tests, not running them, and > breaking them with my recent commit, which i'll have to do something > about before i can sync to AOSP. i'll admit i'm very tempted to just > locally ignore those tests for now so i can get people kicking the > `tar --sort=name` tires asap, and coming back to worry about vi > later!) > > On Wed, Feb 1, 2023 at 12:52 PM Jarno Mäkipää wrote: > > > > Drive by patching :) > > > > Added better backspace support, now cuts in both insert and normal > > mode and also original text and not just what is in temporary buffer. > > Fix arrow key behavior on insert mode slightly by inserting what ever > > has been typed before moving. > > > > -Jarno > > ___ > > Toybox mailing list > > Toybox@lists.landley.net > > http://lists.landley.net/listinfo.cgi/toybox-landley.net ___ Toybox mailing list Toybox@lists.landley.net http://lists.landley.net/listinfo.cgi/toybox-landley.net
[Toybox] [PATCH] vi: added backspace
Drive by patching :)
Added better backspace support, now cuts in both insert and normal
mode and also original text and not just what is in temporary buffer.
Fix arrow key behavior on insert mode slightly by inserting what ever
has been typed before moving.
-Jarno
From bb7f9e74678fdbb7232224141334e821c66ad054 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Jarno=20M=C3=A4kip=C3=A4=C3=A4?=
Date: Wed, 1 Feb 2023 22:37:16 +0200
Subject: [PATCH] vi: added backspace
---
toys/pending/vi.c | 24
1 file changed, 24 insertions(+)
diff --git a/toys/pending/vi.c b/toys/pending/vi.c
index 22a08690..77a031ba 100644
--- a/toys/pending/vi.c
+++ b/toys/pending/vi.c
@@ -869,6 +869,18 @@ static int vi_x(char reg, int count0, int count1)
return 1;
}
+static int backspace(char reg, int count0, int count1)
+{
+ size_t from = 0;
+ size_t to = TT.cursor;
+ cur_left(1, 1, 0);
+ from = TT.cursor;
+ if (from != to)
+vi_delete(reg, to, 0);
+ check_cursor_bounds();
+ return 1;
+}
+
static int vi_movw(int count0, int count1, char *unused)
{
int count = count0*count1;
@@ -1576,6 +1588,12 @@ void vi_main(void)
// TODO: support cursor keys in ex mode too.
if (TT.vi_mode && key>=256) {
key -= 256;
+ //if handling arrow keys insert what ever is in input buffer before moving
+ if (TT.il->len) {
+ i_insert(TT.il->data, TT.il->len);
+ TT.il->len = 0;
+ memset(TT.il->data, 0, TT.il->alloc);
+ }
if (key==KEY_UP) cur_up(1, 1, 0);
else if (key==KEY_DOWN) cur_down(1, 1, 0);
else if (key==KEY_LEFT) cur_left(1, 1, 0);
@@ -1622,6 +1640,10 @@ void vi_main(void)
vi_buf[0] = 0;
vi_buf_pos = 0;
break;
+case 0x7F: //FALLTHROUGH
+case 0x08:
+ backspace(TT.vi_reg, 1, 1);
+ break;
default:
if (key > 0x20 && key < 0x7B) {
vi_buf[vi_buf_pos] = key;//TODO handle input better
@@ -1688,6 +1710,8 @@ void vi_main(void)
int shrink = strlen(last);
memset(last, 0, shrink);
TT.il->len -= shrink;
+ } else {
+backspace(TT.vi_reg, 1, 1);
}
break;
case 0x0A:
--
2.34.1
___
Toybox mailing list
Toybox@lists.landley.net
http://lists.landley.net/listinfo.cgi/toybox-landley.net
Re: [Toybox] [PATCH] vi: add "w ".
On Sun, Jan 29, 2023 at 9:04 PM enh via Toybox wrote: > > (Issues reported by an Android partner; basically "vi" with no arguments > is broken, and "vi foo" where foo doesn't yet exist is broken, and even > if they weren't, ":w filename" isn't implemented. This patch fixes the > last of those more convincingly than the other two.) To me it looks like there has been some regression since my last commit as on my local working copy creating empty file works. commit I had on my old working directory: 6f0f61ad4430f3df72b118371a8b1643ddb69429 In my system current vi from master does not even compile. So I cant briefly check what is current state of this toys/pending/vi.c In function 'open', inlined from 'write_file' at toys/pending/vi.c:575:13: /usr/include/x86_64-linux-gnu/bits/fcntl2.h:50:11: error: call to '__open_missing_mode' declared with attribute error: open with O_CREAT or O_TMPFILE in second argument needs 3 arguments > > Don't crash (or write '(null).swp') when started with no filename; > refuse to write without a filename (with ":w filename" being the > workaround). > > Also fix insert to work when started with a filename where no such file > yet exists. (This appears to still be buggy in some cases, depending > on the exact sequence of editing operations performed. An 'i' followed > by an 'A' for example suggests that there's an off-by-one in the whole > block_list/slice_list thing. I still think we should just replace that > with an easy to get right https://en.wikipedia.org/wiki/Gap_buffer > instead, or even just the trivial case where the gap is always at the > end! Especially when we come to dealing with stuff like undo/redo...) I tried to implement buffer handling into few interface functions (ones starting with "text_" such as text_sol, text_eol) Therefore changing to different buffer mechanism at this point should be trivial. If someone with spare time decides to implement Gap buffer, linked list, line span, rope etc.. they should look into those functions. But same bugs will mostly still exist since I have feeling that text_ functions work as intended but vi command implementations are using them wrong. These errors not relating to buffer mechanism itself but "user interfacing". Way the "i" and "A" etc are just probably implemented is wrong and do not handle all the corner cases. In case for commands like 'A', 'dd', 'yy' that deal with line endings, when implementing you need to think do you need to go before, at or after newline character. Version that still compiles to me (6f0f61ad4430f3df72b118371a8b1643ddb69429) works in my perspective but if there is something miss behaving its best to write unit test into tests/vi.test by using -s argument to run sequence of operations for input file. I think better would be first fix user command handling to actually handle all inputs correctly, but that takes dedication since vi and ex command syntax is very complex. And perhaps after that implementing rest of useful commands. Also drawing actual text buffer to output is currently just mostly debug implementation :) But I am not totally against of Gap buffer, its good for small files. But its not silver bullet for implementing vi since its not text editor where user is constantly in insert mode and moving with arrow keys. > > Also add basic error reporting similar to hexedit. This (and the notion > of a status line that would help make both even better, in particular > removing the bogus "should be long enough" sleeps, should probably end > up in tty.c, but this is better than nothing.) > > Also simplify the three (0/1/-1) return values from run_ex_cmd(), > which appears to have been a bug anyway. > > Also document that TT.modified isn't actually updated (a bug that would > be a lot easier to fix with a simpler text representation). This means > that although I've added the missing ":q" warning for modified files, it > doesn't actually ever fire yet. > > Bug: https://issuetracker.google.com/242636400 > --- > toys/pending/vi.c | 114 +- > 1 file changed, 71 insertions(+), 43 deletions(-) > ___ > Toybox mailing list > Toybox@lists.landley.net > http://lists.landley.net/listinfo.cgi/toybox-landley.net Anyway its nice that someone has interest on using this very early prototype version of text editor. I dont mind if someone decides to do even some drastic code changes to make it more usable and robust. br. Jarno ___ Toybox mailing list Toybox@lists.landley.net http://lists.landley.net/listinfo.cgi/toybox-landley.net
Re: [Toybox] vi
Hello On Wed, Jun 16, 2021 at 9:32 PM Frank Liu wrote: > > Hi, > > I am not sure how far we are on the vi in toybox. Just enabled it in the > build and tried: vi is still unfinished, it can be used to modify some text files sometimes successfully. > > "vi" alone gives: > Floating point exception (core dumped) > > touch newfile; vi newfile > vi: mmap: Invalid argument This seems to be regression. I did git bisect and 0b6757da5d84a7cdd903099aefd2ff7fe668705c started giving me these errors. Earlier versions such as cae14933a6b32bc7260964439f9def316c7520ba seems to work for me. > > vi existing_non_empty_file > seems to work > I normally use Space key to navigate to the right but that doesn't seem to > work, I have to use l key or right arrow key. > :1 doesn't bring me to line 1 (:number doesn't seem to work). Most keycommands are still unimplemented, and some implemented ones may work incorrectly. Basic browsing of text file and some simple edit commands work. > > Thanks! > Frank br Jarno > ___ > Toybox mailing list > Toybox@lists.landley.net > http://lists.landley.net/listinfo.cgi/toybox-landley.net ___ Toybox mailing list Toybox@lists.landley.net http://lists.landley.net/listinfo.cgi/toybox-landley.net
Re: [Toybox] [PATCH] lib/lib human_readable_long fix utf-8 LC_NUMERIC
On Thu, Sep 10, 2020 at 3:20 AM enh wrote: > > if you've ever wondered why the same person (me) worked so hard to ensure > that OEMs couldn't remove locale data from icu4c but also personally removed > all the localization from the core Java libraries and libc... > > i'd always been a strong proponent of localization, but one of the first > things i did on Android was to remove this sort of "low-level localization" > where i found it. i was finding that bugs were getting less attention than > they should because developers didn't know what to do with (say) a Turkish > error message. automated bug report clustering was failing to realize that > (say) `Datei oder Verzeichnis nicht gefunden` and `그런 파일이나 디렉터리가 없습니다` and > `No such file or directory` are the same. or scripts failing to parse output > because they've been trained on en_US. Yes, googling problems based on error messages is a lot easier when errors are on english. > > for *apps* -- anything that real people interact with directly -- > localization is massively important. but, at least after working on Android, > i came to believe that it's a _mistake_ and actively harmful for development > tools. the fact that i've had to (say) help a native Russian speaker fix a > bug where `x = 70,2` was valid but very much not what they meant only > _strengthens_ this belief for me --- if you're going to work on this stuff, > you're going to have to learn the C/POSIX locale sooner or later. I'm ok with the C/Posix locale. It does not have thousands separators so there is no confusion. But I think forcing en_US on the other hand is not ok. > > see also: why ISO-8601 is the one true date format. > > don't apps need libc localization? not really. the POSIX localization > functionality is so anaemic that it's really not useful even for "major > minority" languages. if you're serious about localization, you're going to > need icu4c anyway, which isn't scared to embrace all the diversity that's > actually out there (rather than the tiny subset that the POSIX folks could > imagine, which doesn't even stretch to the need for the genitive case in > dates, to pick one random fairly mainstream example). > > Luckily, i've also been able to neuter Android's libc so none of this will > affect Android whichever way toybox goes[1]. but i still think it's a bad > idea. no "real people" should ever need to look at this, but machines and > developers will, and every bit of localization hurts the real audience. > > at least 15'936.2 would be a valid C++14 identifier (and i'm assuming will > make it into C2x) :-) And rust has underlines 15_936.2 to add confusion. > > ___ > 1. strictly, the fact that you're doing your own insertion of ',' separators > might hurt me (in the `top -b` case), but i'll worry about that if i notice > it actually break any parsing. i know that's included in Android's standard > bugreports, but i _don't_ know that anyone's parsing it. > > On Wed, Sep 9, 2020 at 10:37 AM Jarno Mäkipää wrote: >> >> Apparently LC_NUMERIC thousands_sep can be NARROW NO-BREAK SPACE >> >> There might be cleaner fix than this, but copying just char out of >> thousands_sep spit out >> >> >> Mem: 15�36M total,4�92M used, 11�44M free, 674M buffers >> Swap:2�47M total,0M used,2�47M free,1�97M cached >> >> >> after patch >> Mem: 15 936M total, 4 658M used, 11 277M free, 677M buffers >> Swap: 2 047M total,0M used, 2 047M free, 1 675M cached >> >> >> -Jarno >> ___ >> Toybox mailing list >> Toybox@lists.landley.net >> http://lists.landley.net/listinfo.cgi/toybox-landley.net ___ Toybox mailing list Toybox@lists.landley.net http://lists.landley.net/listinfo.cgi/toybox-landley.net
[Toybox] [PATCH] lib/lib human_readable_long fix utf-8 LC_NUMERIC
Apparently LC_NUMERIC thousands_sep can be NARROW NO-BREAK SPACE
There might be cleaner fix than this, but copying just char out of
thousands_sep spit out
Mem: 15�36M total,4�92M used, 11�44M free, 674M buffers
Swap:2�47M total,0M used,2�47M free,1�97M cached
after patch
Mem: 15 936M total, 4 658M used, 11 277M free, 677M buffers
Swap: 2 047M total,0M used, 2 047M free, 1 675M cached
-Jarno
From 04ae21f85a606038710ccfe1118a3dc6a7f33632 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Jarno=20M=C3=A4kip=C3=A4=C3=A4?=
Date: Wed, 9 Sep 2020 20:20:40 +0300
Subject: [PATCH] lib/lib human_readable_long fix utf-8 LC_NUMERIC
Apparently LC_NUMERIC thousands_sep can be NARROW NO-BREAK SPACE
---
lib/lib.c | 17 +
1 file changed, 9 insertions(+), 8 deletions(-)
diff --git a/lib/lib.c b/lib/lib.c
index 319b6af4..a7ca8077 100644
--- a/lib/lib.c
+++ b/lib/lib.c
@@ -1151,7 +1151,7 @@ match:
int human_readable_long(char *buf, unsigned long long num, int dgt, int unit,
int style)
{
- static char cc, dot;
+ static char *cc, *dot;
unsigned long long snap = 0;
int len, commas = 0, off, ii, divisor = (style_1000) ? 1000 : 1024;
@@ -1173,26 +1173,27 @@ int human_readable_long(char *buf, unsigned long long num, int dgt, int unit,
setlocale(LC_NUMERIC, "");
ll = localeconv();
- dot = *ll->decimal_point ? : '.';
- cc = *ll->thousands_sep ? : ',';
-} else cc = ',', dot = '.';
+ dot = ll->decimal_point ? : ".";
+ cc = ll->thousands_sep ? : ",";
+} else cc = ",", dot = ".";
}
len = sprintf(buf, "%llu", num);
if (style_COMMAS) {
+int clen = strlen(cc);
for (ii = 0; ii___
Toybox mailing list
Toybox@lists.landley.net
http://lists.landley.net/listinfo.cgi/toybox-landley.net
Re: [Toybox] [PATCH] top: don't report GiB sizes in KiB.
Im sorry didint want to cause argument. But atleast 1/3 of world teach to use comma as decimal separator (most old french colonies etc...) so someone else might have got confused and report this sooner or later. On Mon, Sep 7, 2020 at 11:29 PM Rob Landley wrote: > > On 9/6/20 6:45 AM, Jarno Mäkipää wrote: > > On Sun, Sep 6, 2020 at 12:34 PM Rob Landley wrote: > >> Elliott says there's a maximum limit on the number of digits users are > >> willing > >> to parse, and you're saying it's better to just have large blank gaps > >> between > >> the numbers than to use that space for anything, AND that the cap on the > >> maximum > >> number of digits is insurmountable rather than using separators like > >> people have > >> been doing for hundreds of years to cope with long numbers in "human > >> readable" > >> output? > >> > >> It's certainly a point of view. > > > > Groups of 3 are indeed easier for eye. I would suggest using something > > more sensible like spaces. > > Which is not what any country uses by default and thus makes everyone equally > do > a double take? Egalitarian badness? > > Hardwiring it to the esperanto of formats is certainly a suggestion. https://en.wikipedia.org/wiki/Decimal_separator#Examples_of_use SI style formats 123 456.789 or 123 456,789 are used in some countries, including mine. > > > SI system uses spaces as thousands separator, comma and period both > > being valid decimal separator. > > 123 456.789 or 123 456,789 > > Ok, I'll bite: which countries teach SI to their kids in primary school? Eh? not sure are you joking. > > >> Bravo. And bionic's libc/bionic/locale.gratuitouslycppbutactuallyc says: > >> > >> // We only support two locales, the "C" locale (also known as "POSIX"), > >> // and the "C.UTF-8" locale (also known as "en_US.UTF-8"). > >> > >> So they don't support it either. > > > > C.UTF-8 and en_US.UTF-8 are not same. > > I cut and pasted that out of the bionic source. > > >> However, if the commas go, why doesn't the period in human_readable() go? I > >> don't see how they're conceptually different? > > I'm waiting for an opinion from Elliott, which might be a "meh?" because it's > not exactly his area either. > > A proper fix would be a localeconv() in libc that DOESN'T return constant stub > info, which is out of scope for toybox. (And is as much an ADB thing as a > bionic > thing since android seems to be using adb instead of ssh, so that would have > to > marshall the locale environment variables from the host into the target. But I > often "wait for somebody to complain", you complained, and therefore I want to > fix it PROPERLY.) Yeah I agree fixing it properly would move the problem out of toybox context. Im sure we dont want to be arguing what is comma used for. > > In the meantime, I can add a call to localeconv() that would use "," if that > returns "" which means right now it would be a NOP but then it's not my fault > it's getting it wrong. And I can test against glibc which does have an > overengineered version of this in it. Way back when uClibc had a much > compressed > format for the localeconv data, but didn't have a database of countries and > thus > copied its data from glibc, which it couldn't distribute for licensing > reasons: > > https://lists.uclibc.org/pipermail/uclibc/2015-June/049000.html > > Rob > > P.S. I ranted about this sort of aesthetic issue being something the open > source > development model can't deal with 10 years ago, almost to the day: > > https://landley.net/notes-2010.html#13-08-2010 > > And included it in my 2013 talk: > > https://www.youtube.com/watch?v=SGmtP5Lg_t0#t=11m30s I think its not only Open Source problem, if you look how horrible some user interfaces grow into, after company making them grows. -Jarno ___ Toybox mailing list Toybox@lists.landley.net http://lists.landley.net/listinfo.cgi/toybox-landley.net
Re: [Toybox] [PATCH] top: don't report GiB sizes in KiB.
On Sun, Sep 6, 2020 at 12:34 PM Rob Landley wrote: > > > > On 9/5/20 1:42 AM, Jarno Mäkipää wrote: > > On Fri, Sep 4, 2020 at 9:22 AM Rob Landley wrote: > >> > >> On 9/3/20 9:56 AM, enh wrote: > >>> I think hard-coded MiB would be fine for me right now, > >> > >> I'm worried that on embedded systems that _don't_ have gigs of memory, > >> kilobytes > >> may be the desired units, so auto-adjusting makes sense. (If you have 16 > >> megs > >> ram you care about a 300k fluctuation which you can't even SEE if the > >> units are > >> megabytes.) > >> > >>> but -- re-reading the > >>> full discussion from last time rather than relying on my memory -- it > >>> seems like > >>> your main complaint the first time I tried to fix this was that you > >>> didn't like > >>> the possibility of not using the same unit for every value. > >> > >> I don't, but it's a lesser of two evils kind of thing. > >> > >>> So hard-coding MiB > >>> might be better in that it would be more likely to stick rather than > >>> getting > >>> reverted. > >> > >> Given that I'd be the one doing the reverting and I'm suggesting this > >> solution... :) > >> > >> Ok, human_readable_long() is more or less only used directly by ps.c. (The > >> other > >> 2 files are demo_number.c and lib/lib.c wrapping it in human_readable() > >> providing defaults for the micromanagement fields.) So instead of detecting > >> "units" from 0 I can feed it a minimum units (k=1, M=2, and so on). So I > >> can > >> have the ps.c check the total size field and if it's 10 gig or larger force > >> megabytes, and otherwise do kilobytes. (It's gotta be at least kilobytes > >> because > >> the granularity of the input is kilobytes anyway...) > >> > >> I pushed a change. On my laptop it now looks like: > >> > >> Mem: 15,955M total, 15,426M used, 528M free, 151M buffers > >> Swap: 16,286M total, 523M used, 15,763M free, 1594M cached > > > > Comma is a decimal separator in Finland and also in some other parts > > of Europe. So quick glance I thought you have 16megs of ram. > > Yet this wasn't a problem before on all other human_readable() output using > "." > as the decimal separator? (I.E. you consider it a _new_ problem?) Problem for me is not using "." as decimal separator, both "." and "," are easily understandable. But having thousands separator makes things different when command spits out 16,256 or 16.256 > > I can call some sort of lc_eldrich_nonsense() function, although reading > https://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap07.html is > not > immediately providing usable example code... > > > https://docs.oracle.com/cd/E19455-01/806-0169/overview-9/index.html > > Oratroll's a patent troll at this point, on the SCO level. (Dying business > models explode into a cloud of IP litigation.) I'm reluctant to follow a link > to > anything on their website because it feels like entrapment. Sorry perhaps oracle website is not best source of information. But it happened to be first link on google search to show table of some numbering systems. > > > I would drop thousands separators since it's probably more confusing > > than helpful. > > Elliott says there's a maximum limit on the number of digits users are willing > to parse, and you're saying it's better to just have large blank gaps between > the numbers than to use that space for anything, AND that the cap on the > maximum > number of digits is insurmountable rather than using separators like people > have > been doing for hundreds of years to cope with long numbers in "human readable" > output? > > It's certainly a point of view. Groups of 3 are indeed easier for eye. I would suggest using something more sensible like spaces. SI system uses spaces as thousands separator, comma and period both being valid decimal separator. 123 456.789 or 123 456,789 > > > Ubuntu with my locale settings actually gives me this, and i'm not > > even sure what is the meaning of the number after comma :) > > MiB Mem : 15936,4 total, 13507,9 free, 1196,7 used, 1231,9 buff/cache > > MiB Swap: 2048,0 total, 2048,0 free, 0,0 used. 14407,5 avail Mem > > Tenths. It's your decimal separator. > > Your inability to read your OWN locale, in the existing ubuntu output, kind of > undermines your objection that the n
Re: [Toybox] [PATCH] top: don't report GiB sizes in KiB.
On Fri, Sep 4, 2020 at 9:22 AM Rob Landley wrote: > > On 9/3/20 9:56 AM, enh wrote: > > I think hard-coded MiB would be fine for me right now, > > I'm worried that on embedded systems that _don't_ have gigs of memory, > kilobytes > may be the desired units, so auto-adjusting makes sense. (If you have 16 megs > ram you care about a 300k fluctuation which you can't even SEE if the units > are > megabytes.) > > > but -- re-reading the > > full discussion from last time rather than relying on my memory -- it seems > > like > > your main complaint the first time I tried to fix this was that you didn't > > like > > the possibility of not using the same unit for every value. > > I don't, but it's a lesser of two evils kind of thing. > > > So hard-coding MiB > > might be better in that it would be more likely to stick rather than getting > > reverted. > > Given that I'd be the one doing the reverting and I'm suggesting this > solution... :) > > Ok, human_readable_long() is more or less only used directly by ps.c. (The > other > 2 files are demo_number.c and lib/lib.c wrapping it in human_readable() > providing defaults for the micromanagement fields.) So instead of detecting > "units" from 0 I can feed it a minimum units (k=1, M=2, and so on). So I can > have the ps.c check the total size field and if it's 10 gig or larger force > megabytes, and otherwise do kilobytes. (It's gotta be at least kilobytes > because > the granularity of the input is kilobytes anyway...) > > I pushed a change. On my laptop it now looks like: > > Mem: 15,955M total, 15,426M used, 528M free, 151M buffers > Swap: 16,286M total, 523M used, 15,763M free, 1594M cached Comma is a decimal separator in Finland and also in some other parts of Europe. So quick glance I thought you have 16megs of ram. https://docs.oracle.com/cd/E19455-01/806-0169/overview-9/index.html I would drop thousands separators since it's probably more confusing than helpful. Ubuntu with my locale settings actually gives me this, and i'm not even sure what is the meaning of the number after comma :) MiB Mem : 15936,4 total, 13507,9 free, 1196,7 used, 1231,9 buff/cache MiB Swap: 2048,0 total, 2048,0 free, 0,0 used. 14407,5 avail Mem > > Even though the top in debian is still doing: > > KiB Mem : 16337976 total, 524876 free, 13746900 used, 2066200 buff/cache > KiB Swap: 16677884 total, 16141660 free, 536224 used. 1502312 avail Mem > > Which is why my output ooked like it did before. Sigh. I have NO idea why they > renamed "cached" to "avail mem", it isn't unless you wanna fight the balance > logic or echo 3 > /proc/sys/vm/drop_caches > > > Want me to send that patch? > > I have a tropism for "what we're doing makes logical sense" vs just "tweak to > fix the problem I hit". I tend to want a _systemic_ fix, which is inconvenient > at times. :) > > Rob -Jarno > ___ > Toybox mailing list > Toybox@lists.landley.net > http://lists.landley.net/listinfo.cgi/toybox-landley.net ___ Toybox mailing list Toybox@lists.landley.net http://lists.landley.net/listinfo.cgi/toybox-landley.net
Re: [Toybox] Phone docking stations for general purpose computing.
On Mon, May 18, 2020 at 3:41 PM David Seikel wrote: > > On 2020-05-17 20:42:05, Rob Landley wrote: > > Hey Elliott, USB-C to HDMI adapters are going for $5 now. If I grabbed > > something > > like https://www.ebay.com/i/312982429778 or > > https://www.ebay.com/i/302957153131 > > or any of the dozen others, and plugged that and my pixel 3a into the same > > usb-c > > hub, could I get the display on my TV and if so what would I need to do? I have pixel 3a and usb-c docking station for laptop. Docking station has 1 hdmi and 2 display ports, and few usb ports, lan etc... With quick 'plug and pray' test I could not get displays to work, but USB keyboard works nicely. Googling problem points to few msg boards that say Nexus 5 was last google phone supporting video output on usb-c but cant find any reliable source for information. Chromecast could be used to share screen. Having keyboard and mice working is at least something. -Jarno > > > > Next question: ARE there any usb-c hubs, and if so what keyword should I be > > looking for to find them? All the hubs seem to be "USB-C to 4 USB-A" > > splitters. > > so far. (It's like trying to find a gigabit ethernet switch in 2005: it's > > all > > one uplink port and the rest is 100baseT.) > > The big problem vith that is that not all USB-C is created equal. > Specifically, not all of them support video out. > > For example - My Motolora Z has a USB-C, and is compatible with the > Motorola Moto Mod system (add on hardware system that clips hardware > extensions onto the back magnetically). I can plug HDMI and USB keyboard > / mouse into that, but not using the USB-C that is on the phone itself. > I have to use the USB-C that is on the Moto Mod developers kit, which is > a Moto Mod with three USB ports, two USB-C and one micro B. The HDMI > comes out of one of those extra USB-C, but only using a hacked up Moto Mod > firmware. No video is available on the phones own USB-C, so I use that > for the keyboard / mouse. I plug the lot into my KVM. > > A powered hub that can supply power to the phone while you spend all day > developing on it would also be useful. > > Another option for getting video out of a phone is Chromecast via WiFi. > That's what I use to give Google Daydream VR demos, so I can watch what is > on the screen of the phone strapped to someone elses head. > > > Anyway, it occurs to me that with the shift from 32 to 64 bit hardware > > (which is > > why so many devices got stuck on Android-M: they were 32 bit) plus the > > shift to > > USB-C, means I may need to draw a line in the sand. > > > > What I probably should be targeting is "5 years from now there's gonna be a > > lot > > of old 64 bit phone hardware with USB-C in the backs of drawers", and people > > will want to use that as hobbyist development systems the way Linux took > > over > > all the old 386 PCs in the 1990's but ignored the 286 systems. > > > > Everything _older_ than 64 bits with USB-C is basically 16-bit ISA PCs at > > this > > point, which would be _nice_ to support but getting that to rebuild itself > > under > > itself is less useful, because that pool of hardware is shrinking from here > > on > > out and the other is doing all the growing. > > > > What's useful is teaching new systems to have a "general purpose computing > > mode" (in a container or whatever) that can plug the phone into keyboard, > > mouse, > > and big display, and having THAT instead of having a PC means you are not a > > second class citizen but a full-fledged developer. The "docking station" to > > give > > a phone a real screen, keyboard, and mouse is a usb-c hub, usb mouse and > > keyboard, and a $5 hdmi adapter, which is all cheap generic and (eventually) > > ubiquitous. > > There might be ubiquitous USB-C gadgets, but it'll be the return of Plug > and Pray to see if you are lucky enough to have the correct set of > gadgets that will talk to each other in the way you want. > > On the plus side, once the hardware is sorted, things just work. I > didn't have to teach my phone about external mice or keyboards. > > > At some point I'd like to be able to draw a line in the sand and say "from > > this > > system and on newer, you don't need a PC anymore, not even to do Android OS > > development". Then the PC can go the way of the mainframe and minicomputer > > before it, no longer the machine anyone sits down at to do their work, it's > > just > > big iron at the other end of some network cable that only its priesthood > > ever > > needs to touch. > > > > But what would be really nice is the ability to prototype this now. Can I > > put > > together such a docking station that works with a modern phone? Getting a > > terminal on the screen with a posix container that can build AOSP is just > > software at that point. (And stripping down AOSP so it doesn't take an 16x > > cloud > > server hours to build it is also just a question of putting in the work. > > For one > > thing, you don't
Re: [Toybox] [PATCH] cp: fix -D (--parents)
Oh nevermind half of what i just said
getbasename() does not change data. idk what I was looking at on my
ipad when I wrote half of the msg. Well I had few versions of old
toybox getdirname() and some random c library basename implementations
on browser tabs open so mix up is just human error...
strdup on else case is not needed, and therefore fix can be just
simplified by replacing dirname only, so the minimal fix is just
- char *s = FLAG(D) ? dirname(src) : getbasename(src);
+ char *s = FLAG(D) ? src : getbasename(src);
this is probably something you liked to see... :)
-Jarno
ps. I still dont understand how fileunderdir() if is triggered
On Sun, Mar 8, 2020 at 5:14 PM Jarno Mäkipää wrote:
>
> Alright you are right I should do better job cleaning up. I attached
> reworked patch that deals with leaking issues, should apply to current
> master.
>
> So original problem with current cp.c code was: data under *src should
> be kept intact since its used later in rename and friends, dirname()
> is not actually needed to be called since we dont need modify src with
> flag(D). getbasename() is toybox/lib/ version of basename() but it
> shares same trait as basename that input data may be modified so to be
> save side strdup before and free strdup after getbasename result has
> been used.
>
> things that allocate here
> s = fileunderdir()
> s = strdup()
> TT.destname = xmprintf()
>
> TT.destname have free() near end of for loop. (And to be completely
> clean extra free() could be put into fileunderdir() error_msg case
> too. But I left it out since I dont know when this case even happens?)
>
> I did not free() my strdup before but if we move free(s) from
> fileunderdir check code block few lines out of else block so it covers
> both if and else branches we get both cases freed.
>
> Now changes look like this and valgrind shows that we are leaking 2
> blocks instead of 3, but I have feeling they are not in this section
> of cp.c since normal file to file case leaks 2 blocks also.
>
> if (destdir) {
> - char *s = FLAG(D) ? dirname(src) : getbasename(src);
> -
> - TT.destname = xmprintf("%s/%s", destname, s);
> + char *s;
>if (FLAG(D)) {
> +TT.destname = xmprintf("%s/%s", destname, src);
> if (!(s = fileunderdir(TT.destname, destname))) {
>error_msg("%s not under %s", TT.destname, destname);
>continue;
> }
> // TODO: .. follows abspath, not links...
> -free(s);
> mkpath(TT.destname);
> + } else {
> +s = strdup(src);
> +TT.destname = xmprintf("%s/%s", destname, getbasename(s));
>}
> + free(s);
> } else TT.destname = destname;
>
> On Sat, Mar 7, 2020 at 2:14 AM Rob Landley wrote:
> >
> > On 3/4/20 12:51 PM, Jarno Mäkipää wrote:
> > > I know you are busy and backlog grows, but friendly reminder that this
> > > patch is still here.
> >
> > Sorry.
> >
> > You add two allocations (strdup and xmprintf()) but no corresponding free.
> > It's
> > not in cp_node(), so it's not leaking per file copied, but it _is_ leaking
> > per
> > file listed on the command line, which is probably ok but I'm less
> > comfortable
> > with it these days than I used to be.
> >
> > Mostly, I want to get it clear in my head why it needs to copy it _twice_,
> > so I
> > have a window open for it, but my head is full of shell corner cases right
> > now
> > and I'm making "melt through this glacier to the other side" style progress.
> >
> > Rob
> >
> > P.S. There was a Doctor Who episode about making that kind of progress a
> > couple
> > years back, and if you google for "doctor who hugo" and then click on the
> > 2016
> > entry, Google pulls up a page with the exact same summary information at
> > the top
> > that does an endless loop as you click on the same entry over and over.
> > Which I
> > suppose is apropos for the episode.
From 516d8b518635a4d506705895ff805105c1979021 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Jarno=20M=C3=A4kip=C3=A4=C3=A4?=
Date: Sat, 15 Feb 2020 13:31:48 +0200
Subject: [PATCH] cp: fix -D (--parents) (REWORK MINIMAL FIX)
add: test for -D
fix: b/c/d/FILE not copying into a/ with -D option
dirname() is not needed when handling FLAG(D) since filename under
src or dest should not be changed.
github.com/landley/toybox/issues/165
---
tests/cp.test | 7 +++
toys/posix/cp.c | 2 +-
2 files changed, 8 insertions(+), 1 deletion(-)
diff --git a/tests/cp.test b/tests/cp.test
index dfb80ea1..5c4a7474 100755
--- a/tests/cp.test
+++ b/tests
Re: [Toybox] [PATCH] cp: fix -D (--parents)
Alright you are right I should do better job cleaning up. I attached
reworked patch that deals with leaking issues, should apply to current
master.
So original problem with current cp.c code was: data under *src should
be kept intact since its used later in rename and friends, dirname()
is not actually needed to be called since we dont need modify src with
flag(D). getbasename() is toybox/lib/ version of basename() but it
shares same trait as basename that input data may be modified so to be
save side strdup before and free strdup after getbasename result has
been used.
things that allocate here
s = fileunderdir()
s = strdup()
TT.destname = xmprintf()
TT.destname have free() near end of for loop. (And to be completely
clean extra free() could be put into fileunderdir() error_msg case
too. But I left it out since I dont know when this case even happens?)
I did not free() my strdup before but if we move free(s) from
fileunderdir check code block few lines out of else block so it covers
both if and else branches we get both cases freed.
Now changes look like this and valgrind shows that we are leaking 2
blocks instead of 3, but I have feeling they are not in this section
of cp.c since normal file to file case leaks 2 blocks also.
if (destdir) {
- char *s = FLAG(D) ? dirname(src) : getbasename(src);
-
- TT.destname = xmprintf("%s/%s", destname, s);
+ char *s;
if (FLAG(D)) {
+TT.destname = xmprintf("%s/%s", destname, src);
if (!(s = fileunderdir(TT.destname, destname))) {
error_msg("%s not under %s", TT.destname, destname);
continue;
}
// TODO: .. follows abspath, not links...
-free(s);
mkpath(TT.destname);
+ } else {
+s = strdup(src);
+TT.destname = xmprintf("%s/%s", destname, getbasename(s));
}
+ free(s);
} else TT.destname = destname;
On Sat, Mar 7, 2020 at 2:14 AM Rob Landley wrote:
>
> On 3/4/20 12:51 PM, Jarno Mäkipää wrote:
> > I know you are busy and backlog grows, but friendly reminder that this
> > patch is still here.
>
> Sorry.
>
> You add two allocations (strdup and xmprintf()) but no corresponding free.
> It's
> not in cp_node(), so it's not leaking per file copied, but it _is_ leaking per
> file listed on the command line, which is probably ok but I'm less comfortable
> with it these days than I used to be.
>
> Mostly, I want to get it clear in my head why it needs to copy it _twice_, so
> I
> have a window open for it, but my head is full of shell corner cases right now
> and I'm making "melt through this glacier to the other side" style progress.
>
> Rob
>
> P.S. There was a Doctor Who episode about making that kind of progress a
> couple
> years back, and if you google for "doctor who hugo" and then click on the 2016
> entry, Google pulls up a page with the exact same summary information at the
> top
> that does an endless loop as you click on the same entry over and over. Which
> I
> suppose is apropos for the episode.
From a63a62823c8e47df54538d09b049bb8bc7c00c4d Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Jarno=20M=C3=A4kip=C3=A4=C3=A4?=
Date: Sat, 15 Feb 2020 13:31:48 +0200
Subject: [PATCH] cp: fix -D (--parents) (REWORK)
add: test for -D
fix: b/c/d/FILE not copying into a/ with -D option
dirname() is not needed when handling FLAG(D) since filename under
src or dest should not be changed. strdup should be used with
getbasename() since path under src should not change here
either but path under dest should contain basename.
github.com/landley/toybox/issues/165
---
tests/cp.test | 7 +++
toys/posix/cp.c | 10 ++
2 files changed, 13 insertions(+), 4 deletions(-)
diff --git a/tests/cp.test b/tests/cp.test
index dfb80ea1..5c4a7474 100755
--- a/tests/cp.test
+++ b/tests/cp.test
@@ -120,6 +120,13 @@ testing "-T file" "cp -T b file && cat file" "b\n" "" ""
testing "-T dir" "cp -T b dir 2>/dev/null || echo expected" "expected\n" "" ""
rm b file
+mkdir -p b/c/d/
+mkdir a/
+echo a > b/c/d/file
+testing "-D b/c/d/file a/" "cp -D b/c/d/file a/ && cat a/b/c/d/file" "a\n" "" ""
+rm -rf a/
+rm -rf b/
+
# cp -r ../source destdir
# cp -r one/two/three missing
# cp -r one/two/three two
diff --git a/toys/posix/cp.c b/toys/posix/cp.c
index f70c0501..8c47b31c 100644
--- a/toys/posix/cp.c
+++ b/toys/posix/cp.c
@@ -407,18 +407,20 @@ void cp_main(void)
if (*--trail == '/') *trail = 0;
if (destdir) {
- char *s = FLAG(D) ? dirname(src) : getbasename(src);
-
- TT.destname = xmprintf("%s/%s", destname, s);
+ char *s;
if (FLAG(D)) {
+TT.destname = xmprintf("%s/%s", d
[Toybox] [PATCH] vi: Rearrange functions, add o, O, I
Place function calls in order so that there is no unneeded
declarations, clear some whitespace stuff. Add few commands
that are commonly used.
cleanup: reorganize functions
cleanup: some whitespace stuff
add: vi_o vi_O vi_I
fix: stop at edges when h and l
fix: fix dd not updating screen
fix: render after all delete moves
---
toys/pending/vi.c | 1158 ++---
1 file changed, 561 insertions(+), 597 deletions(-)
From 0947603c8687903df206ea0d5094d2ae2217c538 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Jarno=20M=C3=A4kip=C3=A4=C3=A4?=
Date: Sat, 15 Feb 2020 23:55:06 +0200
Subject: [PATCH] vi: Rearrange functions, add o, O, I
Place function calls in order so that there is no unneeded
declarations, clear some whitespace stuff. Add few commands
that are commonly used.
cleanup: reorganize functions
cleanup: some whitespace stuff
add: vi_o vi_O vi_I
fix: stop at edges when h and l
fix: fix dd not updating screen
fix: render after all delete moves
---
toys/pending/vi.c | 1158 ++---
1 file changed, 561 insertions(+), 597 deletions(-)
diff --git a/toys/pending/vi.c b/toys/pending/vi.c
index c6f44ef6..b5f34e21 100644
--- a/toys/pending/vi.c
+++ b/toys/pending/vi.c
@@ -21,37 +21,28 @@ config VI
#include "toys.h"
GLOBALS(
-char *s;
-int cur_col;
-int cur_row;
-int scr_row;
-int drawn_row;
-int drawn_col;
-unsigned screen_height;
-unsigned screen_width;
-int vi_mode;
-int count0;
-int count1;
-int vi_mov_flag;
-int modified;
-char vi_reg;
-char *last_search;
-int tabstop;
-int list;
-struct str_line {
- int alloc;
- int len;
- char *data;
-} *il;
-size_t screen; //offset in slices must be higher than cursor
-size_t cursor; //offset in slices
-//yank buffer
-struct yank_buf {
- char reg;
- int alloc;
- char* data;
-} yank;
-
+ char *s;
+ int vi_mode, tabstop, list;
+ int cur_col, cur_row, scr_row;
+ int drawn_row, drawn_col;
+ int count0, count1, vi_mov_flag;
+ unsigned screen_height, screen_width;
+ char vi_reg, *last_search;
+ struct str_line {
+int alloc;
+int len;
+char *data;
+ } *il;
+ size_t screen, cursor; //offsets
+ //yank buffer
+ struct yank_buf {
+char reg;
+int alloc;
+char* data;
+ } yank;
+
+ int modified, fd;
+ size_t filesize;
// mem_block contains RO data that is either original file as mmap
// or heap allocated inserted data
//
@@ -85,47 +76,58 @@ GLOBALS(
const char *data;
} *node;
} *slices;
-
- size_t filesize;
- int fd; //file_handle
-
)
static const char *blank = " \n\r\t";
static const char *specials = ",.:;=-+*/(){}<>[]!@#$%^&|\\?\"\'";
-// TT.vi_mov_flag is used for special cases when certain move
-// acts differently depending is there DELETE/YANK or NOP
-// Also commands such as G does not default to count0=1
-// 0x1 = Command needs argument (f,F,r...)
-// 0x2 = Move 1 right on yank/delete/insert (e, $...)
-// 0x4 = yank/delete last line fully
-// 0x1000 = redraw after cursor needed
-// 0x2000 = full redraw needed
-// 0x4000 = count0 not given
-// 0x8000 = move was reverse
+//get utf8 length and width at same time
+static int utf8_lnw(int *width, char *s, int bytes)
+{
+ wchar_t wc;
+ int length = 1;
+ if (*s == '\t') *width = TT.tabstop;
+ else {
+length = utf8towc(, s, bytes);
+if (length < 1) length = 0, *width = 0;
+else *width = wcwidth(wc);
+ }
+ return length;
+}
-static void draw_page();
+static int utf8_dec(char key, char *utf8_scratch, int *sta_p)
+{
+ int len = 0;
+ char *c = utf8_scratch;
+ c[*sta_p] = key;
+ if (!(*sta_p)) *c = key;
+ if (*c < 0x7F) { *sta_p = 1; return 1; }
+ if ((*c & 0xE0) == 0xc0) len = 2;
+ else if ((*c & 0xF0) == 0xE0 ) len = 3;
+ else if ((*c & 0xF8) == 0xF0 ) len = 4;
+ else {*sta_p = 0; return 0; }
-//utf8 support
-static int utf8_lnw(int* width, char* str, int bytes);
-static int utf8_dec(char key, char *utf8_scratch, int *sta_p);
-static char* utf8_last(char* str, int size);
+ (*sta_p)++;
+ if (*sta_p == 1) return 0;
+ if ((c[*sta_p-1] & 0xc0) != 0x80) {*sta_p = 0; return 0; }
-static int cur_left(int count0, int count1, char* unused);
-static int cur_right(int count0, int count1, char* unused);
-static int cur_up(int count0, int count1, char* unused);
-static int cur_down(int count0, int count1, char* unused);
-static void check_cursor_bounds();
-static void adjust_screen_buffer();
-static int search_str(char *s);
+ if (*sta_p == len) { c[(*sta_p)] = 0; return 1; }
-//from TT.cursor to
-static int vi_yank(char reg, size_t from, int flags);
-static int vi_delete(char reg, size_t from, int flags);
+ return 0;
+}
+static char* utf8_last(char* str, int size)
+{
+ char* end = str+size;
+ int pos = size, len, width = 0;
+ for (;pos >= 0; end--, pos--) {
+len = utf8_lnw(, end, size-pos);
+if (len && width) return
Re: [Toybox] fixed fuzz in patch
suckless project has some features that can be applied as patches cloning st and then trying to apply old patch for scrollback gives me some failed hunks and some applied with fuzz https://st.suckless.org/patches/scrollback/ git clone git://git.suckless.org/st cd st wget https://st.suckless.org/patches/scrollback/st-scrollback-0.8.1.diff patch -p1 < st-scrollback-0.8.1.diff > Can anybody think of good fuzz tests, or do I need to try to come up with > something by hand? (Is there a large pile of known stale patches I can run > through it easily?) > Rob ___ Toybox mailing list Toybox@lists.landley.net http://lists.landley.net/listinfo.cgi/toybox-landley.net
Re: [Toybox] [PATCH] cp: fix -D (--parents)
No worries, thanks for the update. Some notes on this. I did git bisect and feature was partly working when introduced 8fdd58a except it did not go deep enough I think? Then yanking out getdirname() at b301240 made it not work much at all since dirname() modifies original string. So my fix is to copy original string before calling dirname On Sun, Feb 23, 2020 at 1:40 PM Rob Landley wrote: > > Sorry, flew back from Japan to Austin on tuesday am I'm still deep in the grip > of jetlag, and buried about 5 todo items deep. > > Lemme try to get the patch fuzz fix committed, and then track down the use > after > free in config2help, and then I can do this before getting back to the mkroot > and toolchain stuff from https://landley.net/notes.html#18-02-2020 and then > I'd > think maybe I could get back to finishing the toysh variable expansion > plumbing > (where did I leave off... I need to unify the quote traversal logic so I don't > have to teach two different places that $((echo hello) | wc) parses as $( ( ) > ) > and not $(( )), but I got distracted from that by implementing the "!" command > which is like ":" but doesn't have a base command name I can feed to NEWTOY() > and I can't do an OLDTOY() without a NEWTOY(), so there's design work...) > > But it'll be monday by then and I should do $DAYJOB stuff again. > > (While I'm at it, I need to figure out why ifconfig in mkroot called from > toysh > is saying "ifconfig: bad argument '.0.2.15'" (is that a toysh bug or an > ifconfig > bug), oh and I want to add xattr support to tar before the next release, which > is a month overdue. And I finished setting up the local debug environment for > the README->README.md conversion but didn't _do_ the conversion because I > haven't learned the new markup yet. Oh and md5sum is throwing a broken warning > with gcc 9 I can't typecast away so I need to turn a memset() into a for loop > so > gcc can't understand what I'm doing enough to complain about it (I'm blanking > the second half of a struct after calculating the hash, I sorted the members > so > this does what I want and yes it's intentional)), I wanted to make mount use > block2mtd with jffs2.img, swapon -a, test should learn the ~= regex match, the > can of worms that is faq.html...) > > Rob > > (P.S. I'm sorry I haven't reproduced > https://github.com/landley/toybox/issues/170 , yes I should get back to > https://github.com/landley/toybox/issues/166 , is > https://github.com/landley/toybox/issues/164 still active?) > > On 2/21/20 11:00 AM, Jarno Mäkipää wrote: > > ping > > > > On Sat, Feb 15, 2020 at 2:52 PM Jarno Mäkipää wrote: > >> > >> add test for -D > >> fix b/c/d/FILE not copying into a/ with -D option > >> > >> github.com/landley/toybox/issues/165 > >> --- > >> tests/cp.test | 7 +++ > >> toys/posix/cp.c | 7 +-- > >> 2 files changed, 12 insertions(+), 2 deletions(-) > > ___ > > Toybox mailing list > > Toybox@lists.landley.net > > http://lists.landley.net/listinfo.cgi/toybox-landley.net > > ___ Toybox mailing list Toybox@lists.landley.net http://lists.landley.net/listinfo.cgi/toybox-landley.net
Re: [Toybox] [PATCH] cp: fix -D (--parents)
ping On Sat, Feb 15, 2020 at 2:52 PM Jarno Mäkipää wrote: > > add test for -D > fix b/c/d/FILE not copying into a/ with -D option > > github.com/landley/toybox/issues/165 > --- > tests/cp.test | 7 +++ > toys/posix/cp.c | 7 +-- > 2 files changed, 12 insertions(+), 2 deletions(-) ___ Toybox mailing list Toybox@lists.landley.net http://lists.landley.net/listinfo.cgi/toybox-landley.net
[Toybox] [PATCH] cp: fix -D (--parents)
add test for -D
fix b/c/d/FILE not copying into a/ with -D option
github.com/landley/toybox/issues/165
---
tests/cp.test | 7 +++
toys/posix/cp.c | 7 +--
2 files changed, 12 insertions(+), 2 deletions(-)
From d906c5388c9f645f6ecfb4b0248d7efd3d6953b8 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Jarno=20M=C3=A4kip=C3=A4=C3=A4?=
Date: Sat, 15 Feb 2020 13:31:48 +0200
Subject: [PATCH] cp: fix -D (--parents)
add test for -D
fix b/c/d/FILE not copying into a/ with -D option
github.com/landley/toybox/issues/165
---
tests/cp.test | 7 +++
toys/posix/cp.c | 7 +--
2 files changed, 12 insertions(+), 2 deletions(-)
diff --git a/tests/cp.test b/tests/cp.test
index dfb80ea1..5c4a7474 100755
--- a/tests/cp.test
+++ b/tests/cp.test
@@ -120,6 +120,13 @@ testing "-T file" "cp -T b file && cat file" "b\n" "" ""
testing "-T dir" "cp -T b dir 2>/dev/null || echo expected" "expected\n" "" ""
rm b file
+mkdir -p b/c/d/
+mkdir a/
+echo a > b/c/d/file
+testing "-D b/c/d/file a/" "cp -D b/c/d/file a/ && cat a/b/c/d/file" "a\n" "" ""
+rm -rf a/
+rm -rf b/
+
# cp -r ../source destdir
# cp -r one/two/three missing
# cp -r one/two/three two
diff --git a/toys/posix/cp.c b/toys/posix/cp.c
index 13bfd7ef..d6b3e973 100644
--- a/toys/posix/cp.c
+++ b/toys/posix/cp.c
@@ -415,10 +415,10 @@ void cp_main(void)
if (*--trail == '/') *trail = 0;
if (destdir) {
- char *s = FLAG(D) ? dirname(src) : getbasename(src);
+ char *s = strdup(src);
- TT.destname = xmprintf("%s/%s", destname, s);
if (FLAG(D)) {
+TT.destname = xmprintf("%s/%s", destname, s);
if (!(s = fileunderdir(TT.destname, destname))) {
error_msg("%s not under %s", TT.destname, destname);
continue;
@@ -426,6 +426,9 @@ void cp_main(void)
// TODO: .. follows abspath, not links...
free(s);
mkpath(TT.destname);
+ } else {
+s = getbasename(s);
+TT.destname = xmprintf("%s/%s", destname, s);
}
} else TT.destname = destname;
--
2.11.0
___
Toybox mailing list
Toybox@lists.landley.net
http://lists.landley.net/listinfo.cgi/toybox-landley.net
[Toybox] [PATCH] vi: fix pointer pos when at end of line
Going to $ made draw_page render cursor to wrong line
---
toys/pending/vi.c | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
From a2cce5c7cec24adfe403cc33a4738bd0e360d9ae Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Jarno=20M=C3=A4kip=C3=A4=C3=A4?=
Date: Thu, 6 Feb 2020 21:06:43 +0200
Subject: [PATCH] vi: fix pointer pos when at end of line
Going to $ made draw_page render cursor to wrong line
---
toys/pending/vi.c | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/toys/pending/vi.c b/toys/pending/vi.c
index 5086a0da..c6f44ef6 100644
--- a/toys/pending/vi.c
+++ b/toys/pending/vi.c
@@ -418,9 +418,9 @@ static int text_codepoint(char *dest, size_t offset)
static size_t text_sol(size_t offset)
{
size_t pos;
- if (!TT.filesize) return 0;
+ if (!TT.filesize || !offset) return 0;
else if (TT.filesize <= offset) return TT.filesize-1;
- else if ((pos = text_strrchr(offset, '\n')) == SIZE_MAX) return 0;
+ else if ((pos = text_strrchr(offset-1, '\n')) == SIZE_MAX) return 0;
else if (pos < offset) return pos+1;
return offset;
}
--
2.19.1
___
Toybox mailing list
Toybox@lists.landley.net
http://lists.landley.net/listinfo.cgi/toybox-landley.net
Re: [Toybox] [PATCH] vi: Replace linelist with Piece table based design
Oh yeah, sorry. Well techically they are not broken, but drawing of cursor position at end of line symbol does not work. its issue in draw_page() Those movement commands work fine, and tests return correct results. And text is edited right i think. Hard to cover this with testing :( but anyway this needs to be fixed. -Jarno On Thu, Feb 6, 2020 at 1:54 AM enh wrote: > > seems like $, A, and D are all broken now? > > On Sun, Feb 2, 2020 at 5:52 AM Jarno Mäkipää wrote: > > > > I think this can be applied now. Unless someone is against it. Im > > personally starting to be happy with it. All tests pass, more tests > > can be written. > > > > (applies directly to master, on top or without tests patch send few minutes > > ago) > > > > Replaced dlist linelist with continuous memory blocks. This will allow > > editing huge files without billion mallocs. File is first opened with > > mmap() and mapped region is fully described in block_list as one block. > > > > Currently "valid" data is described as slices, when first loading file > > there is only one slice that points to memory existing in block_list. > > When cutting text, block_list is not freed or modified, but instead > > slice_list is modified to have "hole" between 2 slices. when inserting > > new mem_block is added, previos slices are cut in cursor position and > > new slice is added... > > > > Added functions to handling data inside block_list+slice_list > > > > insert_str(), cut_str() are used for all delete and add operations > > text_strrchr(), text_strchr() are used for searching lineendings > > text_byte(), text_codepoint(), text_getline() are for simple data access > > > > Implemented: more or less all previous functionality > > > > Implemented more proper file write: > > file is saved to .swp, blocks are unloaded, file permissions are copied, > > and atomic rename is called, block is reloaded > > chmod some defaults(rw-rw-r--) if original file could not be fstat (does > > not exist) > > > > FIX make all tests pass > > > > Removed: Some unused functions > > --- > > toys/pending/vi.c | 1347 > > +++-- > > 1 file changed, 791 insertions(+), 556 deletions(-) > > ___ > > Toybox mailing list > > Toybox@lists.landley.net > > http://lists.landley.net/listinfo.cgi/toybox-landley.net ___ Toybox mailing list Toybox@lists.landley.net http://lists.landley.net/listinfo.cgi/toybox-landley.net
[Toybox] [PATCH] vi: Replace linelist with Piece table based design
I think this can be applied now. Unless someone is against it. Im
personally starting to be happy with it. All tests pass, more tests
can be written.
(applies directly to master, on top or without tests patch send few minutes ago)
Replaced dlist linelist with continuous memory blocks. This will allow
editing huge files without billion mallocs. File is first opened with
mmap() and mapped region is fully described in block_list as one block.
Currently "valid" data is described as slices, when first loading file
there is only one slice that points to memory existing in block_list.
When cutting text, block_list is not freed or modified, but instead
slice_list is modified to have "hole" between 2 slices. when inserting
new mem_block is added, previos slices are cut in cursor position and
new slice is added...
Added functions to handling data inside block_list+slice_list
insert_str(), cut_str() are used for all delete and add operations
text_strrchr(), text_strchr() are used for searching lineendings
text_byte(), text_codepoint(), text_getline() are for simple data access
Implemented: more or less all previous functionality
Implemented more proper file write:
file is saved to .swp, blocks are unloaded, file permissions are copied,
and atomic rename is called, block is reloaded
chmod some defaults(rw-rw-r--) if original file could not be fstat (does
not exist)
FIX make all tests pass
Removed: Some unused functions
---
toys/pending/vi.c | 1347 +++--
1 file changed, 791 insertions(+), 556 deletions(-)
From 522871db55a3aeac6574b88bef19f3eb1db8d57e Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Jarno=20M=C3=A4kip=C3=A4=C3=A4?=
Date: Sat, 11 Jan 2020 00:36:02 +0800
Subject: [PATCH] vi: Replace linelist with Piece table based design
Replaced dlist linelist with continuous memory blocks. This will allow
editing huge files without billion mallocs. File is first opened with
mmap() and mapped region is fully described in block_list as one block.
Currently "valid" data is described as slices, when first loading file
there is only one slice that points to memory existing in block_list.
When cutting text, block_list is not freed or modified, but instead
slice_list is modified to have "hole" between 2 slices. when inserting
new mem_block is added, previos slices are cut in cursor position and
new slice is added...
Added functions to handling data inside block_list+slice_list
insert_str(), cut_str() are used for all delete and add operations
text_strrchr(), text_strchr() are used for searching lineendings
text_byte(), text_codepoint(), text_getline() are for simple data access
Implemented: more or less all previous functionality
Implemented more proper file write:
file is saved to .swp, blocks are unloaded, file permissions are copied,
and atomic rename is called, block is reloaded
chmod some defaults(rw-rw-r--) if original file could not be fstat (does
not exist)
FIX make all tests pass
Removed: Some unused functions
---
toys/pending/vi.c | 1347 +++--
1 file changed, 791 insertions(+), 556 deletions(-)
diff --git a/toys/pending/vi.c b/toys/pending/vi.c
index a21867b2..551184c0 100644
--- a/toys/pending/vi.c
+++ b/toys/pending/vi.c
@@ -43,20 +43,67 @@ GLOBALS(
int len;
char *data;
} *il;
-struct linelist {
- struct linelist *up;//next
- struct linelist *down;//prev
- struct str_line *line;
-} *text, *screen, *c_r;
+size_t screen; //offset in slices must be higher than cursor
+size_t cursor; //offset in slices
//yank buffer
struct yank_buf {
char reg;
int alloc;
char* data;
} yank;
+
+// mem_block contains RO data that is either original file as mmap
+// or heap allocated inserted data
+//
+//
+//
+ struct block_list {
+struct block_list *next, *prev;
+struct mem_block {
+ size_t size;
+ size_t len;
+ enum alloc_flag {
+MMAP, //can be munmap() before exit()
+HEAP, //can be free() before exit()
+STACK, //global or stack perhaps toybuf
+ } alloc;
+ const char *data;
+} *node;
+ } *text;
+
+// slices do not contain actual allocated data but slices of data in mem_block
+// when file is first opened it has only one slice.
+// after inserting data into middle new mem_block is allocated for insert data
+// and 3 slices are created, where first and last slice are pointing to original
+// mem_block with offsets, and middle slice is pointing to newly allocated block
+// When deleting, data is not freed but mem_blocks are sliced more such way that
+// deleted data left between 2 slices
+ struct slice_list {
+struct slice_list *next, *prev;
+struct slice {
+ size_t len;
+ const char *data;
+} *node;
+ } *slices;
+
+ size_t filesize;
+ int fd; //file_handle
+
)
+static const char *blank = " \n\r\t";
+static const char *specials =
[Toybox] [PATCH] vi: fix dw_last test, add more tests
(One test will fail, but next patch will fix it. git am will warn, since some tests outputs contain escape char and lines ending with spaces, tried to make patch with --binary flag but didint help. anyway patch applied and worked for me on clean master) Fixed delete last word test. (script should not have \n since it is cursor down in vim) Added tests to check inserts Added tests to check yank and push --- tests/files/vi/ascii_dw_last.out | 2 +- tests/files/vi/ascii_insert_after_w.out | 4 + tests/files/vi/ascii_insert_eof.out | 4 + tests/files/vi/ascii_insert_multi.out | 4 + tests/files/vi/ascii_insert_multi_yy_push.out | 5 ++ tests/files/vi/ascii_insert_sof.out | 4 + tests/files/vi/ascii_yw_push.out | 4 + tests/files/vi/dw_last.in | 5 +- tests/files/vi/insert_after_w.in | 2 + tests/files/vi/insert_eof.in | 2 + tests/files/vi/insert_multi.in| 2 + tests/files/vi/insert_multi_yy_push.in| 2 + tests/files/vi/insert_sof.in | 2 + tests/files/vi/yw_push.in | 3 + tests/vi.test | 108 -- 15 files changed, 126 insertions(+), 27 deletions(-) create mode 100644 tests/files/vi/ascii_insert_after_w.out create mode 100644 tests/files/vi/ascii_insert_eof.out create mode 100644 tests/files/vi/ascii_insert_multi.out create mode 100644 tests/files/vi/ascii_insert_multi_yy_push.out create mode 100644 tests/files/vi/ascii_insert_sof.out create mode 100644 tests/files/vi/ascii_yw_push.out create mode 100644 tests/files/vi/insert_after_w.in create mode 100644 tests/files/vi/insert_eof.in create mode 100644 tests/files/vi/insert_multi.in create mode 100644 tests/files/vi/insert_multi_yy_push.in create mode 100644 tests/files/vi/insert_sof.in create mode 100644 tests/files/vi/yw_push.in From 097efa0945c619fc3c18cd3e03a5332820410134 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Jarno=20M=C3=A4kip=C3=A4=C3=A4?= Date: Sat, 1 Feb 2020 11:03:38 +0200 Subject: [PATCH] vi: fix dw_last test, add more tests Fixed delete last word test. (script should not have \n since it is cursor down in vim) Added tests to check inserts Added tests to check yank and push --- tests/files/vi/ascii_dw_last.out | 2 +- tests/files/vi/ascii_insert_after_w.out | 4 + tests/files/vi/ascii_insert_eof.out | 4 + tests/files/vi/ascii_insert_multi.out | 4 + tests/files/vi/ascii_insert_multi_yy_push.out | 5 ++ tests/files/vi/ascii_insert_sof.out | 4 + tests/files/vi/ascii_yw_push.out | 4 + tests/files/vi/dw_last.in | 5 +- tests/files/vi/insert_after_w.in | 2 + tests/files/vi/insert_eof.in | 2 + tests/files/vi/insert_multi.in| 2 + tests/files/vi/insert_multi_yy_push.in| 2 + tests/files/vi/insert_sof.in | 2 + tests/files/vi/yw_push.in | 3 + tests/vi.test | 108 -- 15 files changed, 126 insertions(+), 27 deletions(-) create mode 100644 tests/files/vi/ascii_insert_after_w.out create mode 100644 tests/files/vi/ascii_insert_eof.out create mode 100644 tests/files/vi/ascii_insert_multi.out create mode 100644 tests/files/vi/ascii_insert_multi_yy_push.out create mode 100644 tests/files/vi/ascii_insert_sof.out create mode 100644 tests/files/vi/ascii_yw_push.out create mode 100644 tests/files/vi/insert_after_w.in create mode 100644 tests/files/vi/insert_eof.in create mode 100644 tests/files/vi/insert_multi.in create mode 100644 tests/files/vi/insert_multi_yy_push.in create mode 100644 tests/files/vi/insert_sof.in create mode 100644 tests/files/vi/yw_push.in diff --git a/tests/files/vi/ascii_dw_last.out b/tests/files/vi/ascii_dw_last.out index ee9f73ba..7a965051 100644 --- a/tests/files/vi/ascii_dw_last.out +++ b/tests/files/vi/ascii_dw_last.out @@ -1,4 +1,4 @@ abc def hij klm nop qrs -tuv wxy z +tuv wxy diff --git a/tests/files/vi/ascii_insert_after_w.out b/tests/files/vi/ascii_insert_after_w.out new file mode 100644 index ..6fc7 --- /dev/null +++ b/tests/files/vi/ascii_insert_after_w.out @@ -0,0 +1,4 @@ +abc hellodef hij +klm nop qrs +tuv wxy z + diff --git a/tests/files/vi/ascii_insert_eof.out b/tests/files/vi/ascii_insert_eof.out new file mode 100644 index ..deb55c1d --- /dev/null +++ b/tests/files/vi/ascii_insert_eof.out @@ -0,0 +1,4 @@ +abc def hij +klm nop qrs +tuv wxy z +hello diff --git a/tests/files/vi/ascii_insert_multi.out b/tests/files/vi/ascii_insert_multi.out new file mode 100644 index ..85591656 --- /dev/null +++ b/tests/files/vi/ascii_insert_multi.out @@ -0,0 +1,4 @@ +abcdef abc def hij +klm nop qrs +tuv wxy z + diff --git a/tests/files/vi/ascii_insert_multi_yy_push.out
[Toybox] [PATCH] vi: Add tests
Hi This patch goes directly on top of current master. (I came into conclusion that it is better to have some tests, before actually doing such a big change as switching buffer management to piece table or gap buffer. So I will rebase, fix, rework, etc etc... and hopefully resubmit those when Im entirely sure its the way to go.) Test file integrity after load, move, delete and save+exit. Drawing of buffer is not tested yet. Added -s script option, accept file that is run as startup script of commands. File is parsed byte at time and handled as you had typed it. If EOF has been reached without editor close command, editing is continued normally using keyboard. This functionality is in vim and neovim, but not in POSIX vi standard. nvi (vi used in some macs) has -s with different meaning... Some simple tests added, dw last line test fails, so test is disabled. -Jarno From bed6a72ff0ccb08499398402fd16b52baa1d91b9 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Jarno=20M=C3=A4kip=C3=A4=C3=A4?= Date: Sat, 25 Jan 2020 20:16:49 +0200 Subject: [PATCH] vi: Add tests Test file integrity after load, move, delete and save+exit. Drawing of buffer is not tested yet. Added -s script option, accept file that is run as startup script of commands. File is parsed byte at time and handled as you had typed it. If EOF has been reached without editor close command, editing is continued normally using keyboard. This functionality is in vim and neovim, but not in POSIX vi standard. nvi (vi used in some macs) has -s with different meaning... Some simple tests added, dw last line test fails, so test is disabled. --- tests/files/vi/D_first.in | 3 ++ tests/files/vi/D_last.in | 4 ++ tests/files/vi/ascii.txt | 4 ++ tests/files/vi/ascii_D_first.out | 4 ++ tests/files/vi/ascii_D_last.out | 4 ++ tests/files/vi/ascii_dd_first.out | 3 ++ tests/files/vi/ascii_dd_last.out | 3 ++ tests/files/vi/ascii_dw_first.out | 4 ++ tests/files/vi/ascii_dw_last.out | 4 ++ tests/files/vi/dd_first.in| 3 ++ tests/files/vi/dd_last.in | 4 ++ tests/files/vi/dw_first.in| 3 ++ tests/files/vi/dw_last.in | 6 +++ tests/vi.test | 72 +++ toys/pending/vi.c | 20 +++-- 15 files changed, 138 insertions(+), 3 deletions(-) create mode 100644 tests/files/vi/D_first.in create mode 100644 tests/files/vi/D_last.in create mode 100644 tests/files/vi/ascii.txt create mode 100644 tests/files/vi/ascii_D_first.out create mode 100644 tests/files/vi/ascii_D_last.out create mode 100644 tests/files/vi/ascii_dd_first.out create mode 100644 tests/files/vi/ascii_dd_last.out create mode 100644 tests/files/vi/ascii_dw_first.out create mode 100644 tests/files/vi/ascii_dw_last.out create mode 100644 tests/files/vi/dd_first.in create mode 100644 tests/files/vi/dd_last.in create mode 100644 tests/files/vi/dw_first.in create mode 100644 tests/files/vi/dw_last.in create mode 100644 tests/vi.test diff --git a/tests/files/vi/D_first.in b/tests/files/vi/D_first.in new file mode 100644 index ..0e3ce347 --- /dev/null +++ b/tests/files/vi/D_first.in @@ -0,0 +1,3 @@ +D +:wq + diff --git a/tests/files/vi/D_last.in b/tests/files/vi/D_last.in new file mode 100644 index ..355c8c4f --- /dev/null +++ b/tests/files/vi/D_last.in @@ -0,0 +1,4 @@ +G +D +:wq + diff --git a/tests/files/vi/ascii.txt b/tests/files/vi/ascii.txt new file mode 100644 index ..ee9f73ba --- /dev/null +++ b/tests/files/vi/ascii.txt @@ -0,0 +1,4 @@ +abc def hij +klm nop qrs +tuv wxy z + diff --git a/tests/files/vi/ascii_D_first.out b/tests/files/vi/ascii_D_first.out new file mode 100644 index ..83a48a25 --- /dev/null +++ b/tests/files/vi/ascii_D_first.out @@ -0,0 +1,4 @@ + +klm nop qrs +tuv wxy z + diff --git a/tests/files/vi/ascii_D_last.out b/tests/files/vi/ascii_D_last.out new file mode 100644 index ..ee9f73ba --- /dev/null +++ b/tests/files/vi/ascii_D_last.out @@ -0,0 +1,4 @@ +abc def hij +klm nop qrs +tuv wxy z + diff --git a/tests/files/vi/ascii_dd_first.out b/tests/files/vi/ascii_dd_first.out new file mode 100644 index ..3361750a --- /dev/null +++ b/tests/files/vi/ascii_dd_first.out @@ -0,0 +1,3 @@ +klm nop qrs +tuv wxy z + diff --git a/tests/files/vi/ascii_dd_last.out b/tests/files/vi/ascii_dd_last.out new file mode 100644 index ..dd43ed6d --- /dev/null +++ b/tests/files/vi/ascii_dd_last.out @@ -0,0 +1,3 @@ +abc def hij +klm nop qrs +tuv wxy z diff --git a/tests/files/vi/ascii_dw_first.out b/tests/files/vi/ascii_dw_first.out new file mode 100644 index ..400e1cb1 --- /dev/null +++ b/tests/files/vi/ascii_dw_first.out @@ -0,0 +1,4 @@ +def hij +klm nop qrs +tuv wxy z + diff --git a/tests/files/vi/ascii_dw_last.out b/tests/files/vi/ascii_dw_last.out new file mode 100644 index ..ee9f73ba --- /dev/null +++ b/tests/files/vi/ascii_dw_last.out @@ -0,0 +1,4 @@ +abc def hij +klm nop qrs +tuv wxy z + diff
Re: [Toybox] [PATCH] vi: mem_block/piece_table part 2
On Mon, Jan 20, 2020 at 2:31 AM Rob Landley wrote: > > > > On 1/19/20 3:21 AM, Jarno Mäkipää wrote: > > If this proof to be too complicated, I can redesign with buffer_gap on > > top of this. > > I haven't opened the vi review can of worms yet. (I haven't gotten back to the > bc review in months. I'm trying to finish toysh.) > > If this is the direction you want to take, I'll apply the patches. (You seemed > unsure so I asked for confirmation, and you didn't confirm.) My only concern > is > that Elliott has been patching your stuff, implying he's using your stuff out > of > pending. (I'm not yet.) Yes I was and am unsure, so I wanted some pointers from you and Elliott. Thank you for your time. Every advice has been helpful on one way or another. Elliott is building this out of pending, so I dont want this to be applied if he is not ok with it. his idea of using buffer gap instead is better for common cases, and for minimal code size and complexity if our features are kept small. Anyway I had fun hacking, lets wait for Elliott to reply if he thinks it ok to apply or not. > > > But goals should be set up first, if aiming for mostly > > full POSIX vi compliance I have feeling that with buffer gap some > > features like marks are hard to implement, since lines before mark > > changing should not effect mark position. > > Rule of thumb here: busybox has had a vi for coming up on 20 years, if it's > still missing a feature it is at least possible to do with out it. (That > doesn't > mean you shouldn't do it if you want to, just that a critical mass userbase > hasn't bothered to submit a patch implementing it yet.) > > > Also ability to mmap big > > junk of data will be lost, and buffer_gap requires malloc continuous > > region of filesize+few bytes for gap, this system I wrote could > > fallback to loading file in page size inmemory chunks, without > > modifying any logic. > > I remind you that 32 bit mmap maxes out somewhere around 2 gigabytes, > depending > on the memory layout of the architecture in question. (Virtual address space > within a process becomes a limiting factor.) Yeah cant support everything on few hundred lines of code. Jarno > > Define "scalability"... > > Rob ___ Toybox mailing list Toybox@lists.landley.net http://lists.landley.net/listinfo.cgi/toybox-landley.net
Re: [Toybox] [PATCH] vi: mem_block/piece_table part 2
If this proof to be too complicated, I can redesign with buffer_gap on top of this. But goals should be set up first, if aiming for mostly full POSIX vi compliance I have feeling that with buffer gap some features like marks are hard to implement, since lines before mark changing should not effect mark position. Also ability to mmap big junk of data will be lost, and buffer_gap requires malloc continuous region of filesize+few bytes for gap, this system I wrote could fallback to loading file in page size inmemory chunks, without modifying any logic. Then again if aiming for just very minimal editor in (1000-2000loc) buffer gap might be the way to go. But I think next focus should be to write ability to do automated tests. I think using ex commands dont help since they are mostly line based, so difficult utf8 cases cant be tested. I think simple solution is just to supply moves as parameter line/file and exit. vi inputfile -test 5G\nw\nd2w\n:wq\n -Jarno On Sun, Jan 19, 2020 at 10:39 AM Jarno Mäkipää wrote: > > After this patch functionality should be around the same as with > linelist > > Implement more proper file write: > file is saved to .swp, blocks are unloaded, file permissions are copied, > and atomic rename is called, block is reloaded > chmod some defaults(rw-r--r--) if original file could not be fstat (does > not exist) > > Reimplement simple search /string > > fix w, b, e moves > fix 1G > fix moving cur_right on zalgo text > > Count line numbers for drawing purposes (placeholder needs rewrite) > (If file is too big, gives up) this needs rework. > > cleanup Make function for searching offset slice > cleanup Make local functions static > --- > toys/pending/vi.c | 361 -- > 1 file changed, 218 insertions(+), 143 deletions(-) ___ Toybox mailing list Toybox@lists.landley.net http://lists.landley.net/listinfo.cgi/toybox-landley.net
[Toybox] [PATCH] vi: mem_block/piece_table part 2
After this patch functionality should be around the same as with
linelist
Implement more proper file write:
file is saved to .swp, blocks are unloaded, file permissions are copied,
and atomic rename is called, block is reloaded
chmod some defaults(rw-r--r--) if original file could not be fstat (does
not exist)
Reimplement simple search /string
fix w, b, e moves
fix 1G
fix moving cur_right on zalgo text
Count line numbers for drawing purposes (placeholder needs rewrite)
(If file is too big, gives up) this needs rework.
cleanup Make function for searching offset slice
cleanup Make local functions static
---
toys/pending/vi.c | 361 --
1 file changed, 218 insertions(+), 143 deletions(-)
From 05b8323412626d8bc83ae6cefd233a1e6ec59acd Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Jarno=20M=C3=A4kip=C3=A4=C3=A4?=
Date: Fri, 17 Jan 2020 08:48:08 +0200
Subject: [PATCH] vi: mem_block/piece_table part 2
After this patch functionality should be around the same as with
linelist
Implement more proper file write:
file is saved to .swp, blocks are unloaded, file permissions are copied,
and atomic rename is called, block is reloaded
chmod some defaults(rw-rw-r--) if original file could not be fstat (does
not exist)
Reimplement simple search /string
fix w, b, e moves
fix 1G
fix moving cur_right on zalgo text
Count line numbers for drawing purposes (placeholder needs rewrite)
(If file is too big, gives up) this needs rework.
cleanup Make function for searching offset slice
cleanup Make local functions static
---
toys/pending/vi.c | 361 --
1 file changed, 218 insertions(+), 143 deletions(-)
diff --git a/toys/pending/vi.c b/toys/pending/vi.c
index 07413c9b..2f5611a9 100644
--- a/toys/pending/vi.c
+++ b/toys/pending/vi.c
@@ -85,9 +85,12 @@ GLOBALS(
} *slices;
size_t filesize;
+ int fd; //file_handle
)
+static const char *empties = " \t\n\r";
+static const char *specials = ",.:;=-+*/(){}<>[]!@#$%^&|\\?\"\'";
// TT.vi_mov_flag is used for special cases when certain move
// acts differently depending is there DELETE/YANK or NOP
@@ -155,7 +158,7 @@ struct double_list *dlist_add_after(struct double_list **head,
// size, data allocation size of given data
// len, length of the string
// type, define allocation type for cleanup purposes at app exit
-int insert_str(const char *data, size_t offset, size_t size, size_t len,
+static int insert_str(const char *data, size_t offset, size_t size, size_t len,
enum alloc_flag type)
{
struct mem_block *b = xmalloc(sizeof(struct mem_block));
@@ -219,7 +222,7 @@ int insert_str(const char *data, size_t offset, size_t size, size_t len,
// this will not free any memory
// will only create more slices depending on position
-int cut_str(size_t offset, size_t len)
+static int cut_str(size_t offset, size_t len)
{
struct slice_list *e, *s = TT.slices;
size_t end = offset+len;
@@ -290,23 +293,33 @@ int cut_str(size_t offset, size_t len)
return 0;
}
-size_t text_strchr(size_t offset, char c)
+//find offset position in slices
+static struct slice_list *slice_offset(size_t *start, size_t offset)
{
struct slice_list *s = TT.slices;
- size_t epos, spos = 0;
- int i = 0;
-
- if (!s) return SIZE_MAX;
+ size_t spos = 0;
//find start
- for (;;) {
+ for ( ;s ; ) {
if (spos<=offset && spos+s->node->len>offset) break;
spos += s->node->len;
s = s->next;
-if (s == TT.slices) return SIZE_MAX; //error out of bounds
+if (s == TT.slices) s = 0; //error out of bounds
}
+ if (s) *start = spos;
+ return s;
+}
+
+static size_t text_strchr(size_t offset, char c)
+{
+ struct slice_list *s = TT.slices;
+ size_t epos, spos = 0;
+ int i = 0;
+
+ //find start
+ if (!(s = slice_offset(, offset))) return SIZE_MAX;
i = offset-spos;
epos = spos+i;
@@ -321,23 +334,14 @@ size_t text_strchr(size_t offset, char c)
}
-size_t text_strrchr(size_t offset, char c)
+static size_t text_strrchr(size_t offset, char c)
{
struct slice_list *s = TT.slices;
size_t epos, spos = 0;
int i = 0;
- if (!s) return SIZE_MAX;
-
//find start
- for (;;) {
-if (spos<=offset && spos+s->node->len>offset) break;
-
-spos += s->node->len;
-s = s->next;
-
-if (s == TT.slices) return SIZE_MAX; //error out of bounds
- }
+ if (!(s = slice_offset(, offset))) return SIZE_MAX;
i = offset-spos;
epos = spos+i;
@@ -352,7 +356,7 @@ size_t text_strrchr(size_t offset, char c)
}
-size_t text_filesize()
+static size_t text_filesize()
{
struct slice_list *s = TT.slices;
size_t pos = 0;
@@ -366,25 +370,38 @@ size_t text_filesize()
return pos;
}
-char text_byte(size_t offset)
+static int text_count(size_t start, size_t end, char c)
{
struct slice_list *s = TT.slices;
- size_t spos = 0;
- //find start
- for (;;) {
-if (spos<=offset && spos+s->node->len>offset) break;
+ size_t i, count = 0, spos = 0;
+ if (!(s =
Re: [Toybox] [PATCH] vi: Replace linelist with mem_block based design
On Fri, Jan 17, 2020 at 7:48 PM enh wrote: > > On Fri, Jan 17, 2020 at 1:10 AM Jarno Mäkipää wrote: > > > > I think both has ups and downs, piece table is better for large files, > > and also for doing repeated inserts in random parts of file, such as > > regexp find and replace. > > > > Gap buffer is better for doing lots of inserts around same area. > > also known as "typing", the common case :-) Vi is not like basic editors where you just do small movements with cursor keys and type alot, instead you use movement commands, jump into line numbers, yank data into registers etc... > > > Also > > searching is easier since it has 2 continuous regions. > > historically i've actually just cheated and moved the gap to the end > of the file. that lets you just use existing regex implementations > without having to try to get them to work across gaps. afaik, a piece > table will require a regular expression API that takes something like > a Java CharSequence where you can hide that the bytes aren't actually > consecutive. which would be fine if you weren't trying to use C. This is the best benefit on gap buffer I see. With my version of piece table regexp search needs to copy some data into temporary buffer unless file has no inserts. > > > Also data does > > not need to be copied as much for drawing, only around gap. Good for > > small files. Cant really open large files since it needs heap of > > filesize+gap or perhaps as read-only gap can be zero and memory be > > mmap() into file. > > gap is usually small. humans can't type fast, and computers can copy > large blocks of memory *very* fast, so even 128 bytes or whatever > amortizes fine. (you can special case "paste" so it doesn't get > pathological on large pastes.) > > > both suck at jumping into line number > > but piece table sucks a lot harder, because you have to effectively > apply all the edits to work out where you are. Yeah finding text under cursor position on gap buffer is just multiply, but ordered piece table worst case is iterate number of inserts + multiply But I think that yank registers and marks could benefit from piece table. I dont have any idea how to implement marks in gap buffer. With piece table I could just slice the current line when user saves mark, and check if given slice still exist when user try to jump into mark. Well if we decide to only have one yank register and marks reset after every edit, then gap buffer is fine. > > > Ideally commands such as less should share vi code just editing > > commands disabled, so buffer opening, drawing, searching, cursoring > > and jumping should be same. > > > > Right now I only have 2 functions for insert and cut, and i think its > > all needed. > > like i said (you should read those papers), you actually only need > "replace". doesn't make much difference yet (and i've certainly had > one-line helpers to translate insert and cut in replace historically), > but only having one primitive edit makes undo/redo easier when you get > to them. > > > int insert_str(const char *data, size_t offset, size_t size, size_t len, > > enum alloc_flag type) > > int cut_str(size_t offset, size_t len) > > > > and few commands for searching line breaks, copying current line, > > checking that current codepoint is valid for cursoring... > > size_t text_strchr(size_t offset, char c) > > size_t text_strrchr(size_t offset, char c) > > size_t text_filesize() > > char text_byte(size_t offset) > > int text_codepoint(char *dest, size_t offset) > > size_t text_getline(char *dest, size_t offset, size_t max_len) > > > > So I think now that I added extra layer we can still later switch into > > gap_buffer, if we dont touch directly into buffer handing in actual vi > > commands? (and some of this abstraction can be removed when cleaning > > up codebase after we are satisfied what type of buffer we should use) > > personally i'd go the other way. gap buffer is a lot easier, and you > can always switch to a piece table if/when it proves necessary. if we > were aiming for syntax coloring, say, that would be a mark in favor of > the piece table. but aiui we're not [and i've seen a separate table of > style runs work fine/better for that in the past anyway]. like i said, > it would be good to explicitly agree the goals/non-goals with rob. > I think I fill finish up what I have to be in same level of functionality as linelist one was. Since im very close to that. I will send patch number 2 hopefully this weekend. If we decide to switch to gap buffer later its much easier on top of this, than on top of current master, since linelist based one iterated data directly on linkedlists all over the vi.c. -Jarno ___ Toybox mailing list Toybox@lists.landley.net http://lists.landley.net/listinfo.cgi/toybox-landley.net
Re: [Toybox] [PATCH] vi: don't exit on ^C or ^D.
I looked into ex commands little bit more, All the edits seem to be
line based or correct me if im wrong? So ex commands dont naturally
provide way to test the difficult cases such as: delete until start of
word at current cursor position in last line of document when we are
sitting in this zalgo text separated by few invalid utf8 bytes so
atleast we need to extend it, so that every visual command has toy_ex
substitute...
On Sat, Jan 18, 2020 at 8:15 AM Rob Landley wrote:
>
> On 1/17/20 11:36 AM, enh via Toybox wrote:
> > alternatively we could say "we don't actually care about ex, but do
> > want to have a 'scriptable vi'" and have a mode where toybox vi takes
> > commands that are just vi commands, not ex commands. that's better in
> > terms of smaller code, even if it's then a toybox "exclusive".
>
> If ex is basically free and there's no reason _not_ to have it, fine. I just
> don't know any demand for it as its own thing.
>
> > either sounds like a good idea to me (with me personally leaning
> > towards "just take vi commands until/unless there's a proven need to
> > actually be ex compatible, since our only known use case is testing
> > vi"), but sounds like a decision for rob.
>
> Accidentally typing the key so that vi claims it isn't in "visual" mode
> anymore
> annoys me, and then I need to undo it. (I think hit escape and i a lot until
> it
> stops?)
>
> That's pretty much my entire interaction with ex. I vaguely recall Al Viro
> uses
> it sometimes instead of sed.
>
> Rob
>
> > /me wonders how vim testing works. anyone know?
I dont know what Bram uses, but both neovim and vis seem to use lua on
their testing
https://github.com/neovim/neovim/tree/master/test
https://github.com/martanne/vis-test/tree/46a2b9991ff3faf47b02dfe9b37ffb0efe354b3d
>
> I made a "txpect" command for shell testing, might want to look at that. Note
> that zero length input is "read a line and discard it"... oh here:
>
> > # Prompt changes for root/normal user
> > [ $(id -u) -eq 0 ] && P='# ' || P='$ '
> > # run sufficiently isolated shell child process to get predictable results
> > SH="env -i PATH=${PATH@Q} PS1='\\$ ' $SH --noediting --noprofile --norc -is"
>
> The @Q syntax is a horrible obscure bash thing that gives you a quoted string
> version of the shell variable contents. So the above sets two environment
> variables:
>
> $P = whatever your default shell prompt should be, given whether you're root
> or not.
>
> $SH = a shell command line to run to tell the shell NOT to read /etc/profile
> and
> friends, marshall over the host $PATH, use a predictable prompt, and behave in
> an interactive manner even though stdin isn't a tty.
>
> Then the tests:
>
> > txpect "prompt and exit" "$SH" "E$P" "Iexit\n" X0
>
> Run "$SH", then read the $P prompt from stderr, then write "exit\n" to stdin,
> then expect it to exit with rc 0.
>
> Yes, the prompt is on stderr, not stdout. Because posix says so. I was
> surprised
> too, and had to change it.
>
> And yes, if this has a 2 byte string it will read 2 bytes from the source in
> question and match it, and NOT read any other pending input yet.
>
> > txpect "prompt and echo" "$SH" "E$P" "Iecho hello\n" "Ohello"$'\n' "E$P" X0
>
> Run $SH, read the prompt from stderr, send "echo hello\n" to stdin, read
> "hello\n" from stdout (note: $'\n' is another bash syntax, says echo -e the
> contents of the '' so that turns the \n into ascii 10, which none of these
> strings are doing for you), read the prompt, exit 0.
>
> > txpect "redirect err" "$SH" "E$P" "Iecho > /dev/full\n" "E" "E$P" X0
>
> Run $SH, read prompt, write "echo > /dev/fnull\n" to stdin, read a line from
> stderr and discard it, read the prompt from stderr again, exit with 0 (note: X
> closes the program's stdin so it should exit at that point).
>
> Note "E" by itself reads a full line (until newline) and then there's more
> data
> after that, which we match specifically. O by itself should also do that.
>
> Rob
Need to look into this.
___
Toybox mailing list
Toybox@lists.landley.net
http://lists.landley.net/listinfo.cgi/toybox-landley.net
Re: [Toybox] [PATCH] vi: Replace linelist with mem_block based design
Hi On Fri, Jan 17, 2020 at 8:13 AM enh wrote: > > On Thu, Jan 16, 2020 at 9:59 AM Jarno Mäkipää wrote: > > > > hey > > > > On Thu, Jan 16, 2020 at 6:28 PM enh wrote: > > > > > > my 2c: i wouldn't do anything this complicated. every editor i've > > > written or worked on in the past used the much simpler scheme of one > > > buffer with a gap at the most recent insertion point > > > [https://en.wikipedia.org/wiki/Gap_buffer]. the most extra complexity > > > i've seen this cause is a separate array of line breaks, but since we > > > have the busybox vi style of not doing line wrapping, that seems > > > unnecessary? > > > > I actually found this approach much simpler to implement than atleast > > the dlist based one I previously had. > > oh, yeah, i agree that the "list of lines" is the worst option. > especially if you want to scale to big files. Yeah I think we are both in same page that list of lines needs to go. Its easy to draw and scroll but its very difficult to do inserts and cuts that are not full lines and searching is slow. > > > Difficulties compared to linked > > list of lines, is scrolling down or counting line numbers. But > > inserting and cutting text is actually much easier. I think Gap_buffer > > shares same difficulties without benefit of loading file fast, or > > handling big files. I can load 8gb file with this patch in blink of > > eye. > > that's true for gap buffer too. traditionally you just put the gap at > the end. (but if there's one thing computers are good at, it's copying > data around. and the gap amortizes that anyway, because you only move > the "after" part every "gap_length" bytes.) > > > Reason why I started doing this was that I wanted to do undo, with > > infinite history. And could not figure out easy way with linelist. Now > > undo history should be really easy to make, by just having separate > > slice_list of cut/insert sections in order. > > this is easy with the gap buffer too. the best trick i learned was > from Rob Pike's various editors (you can read the sam > [http://doc.cat-v.org/plan_9/4th_edition/papers/sam/] and acme > [http://doc.cat-v.org/plan_9/4th_edition/papers/acme/] papers on the > web): there's only one edit, and that's replace-range(start_offset, > end_offset, bytes). insert just has a start == end, delete has no > bytes, and replace is replace. and there's a trivial transformation > from each operation to its "undo". and you can put the undone edits on > the redo stack too if you want. I looked into Gap buffer and Piece table more now. I think what I did was Piece table without knowing it. My piece table is slice_list and i just combined source and start to be just one pointer, to origin/add, and I could have multiple origin files mmap() ed. (ex command :r could just mmap new region extending add list) https://darrenburns.net/posts/piece-table/ I think both has ups and downs, piece table is better for large files, and also for doing repeated inserts in random parts of file, such as regexp find and replace. Gap buffer is better for doing lots of inserts around same area. Also searching is easier since it has 2 continuous regions. Also data does not need to be copied as much for drawing, only around gap. Good for small files. Cant really open large files since it needs heap of filesize+gap or perhaps as read-only gap can be zero and memory be mmap() into file. both suck at jumping into line number Ideally commands such as less should share vi code just editing commands disabled, so buffer opening, drawing, searching, cursoring and jumping should be same. Right now I only have 2 functions for insert and cut, and i think its all needed. int insert_str(const char *data, size_t offset, size_t size, size_t len, enum alloc_flag type) int cut_str(size_t offset, size_t len) and few commands for searching line breaks, copying current line, checking that current codepoint is valid for cursoring... size_t text_strchr(size_t offset, char c) size_t text_strrchr(size_t offset, char c) size_t text_filesize() char text_byte(size_t offset) int text_codepoint(char *dest, size_t offset) size_t text_getline(char *dest, size_t offset, size_t max_len) So I think now that I added extra layer we can still later switch into gap_buffer, if we dont touch directly into buffer handing in actual vi commands? (and some of this abstraction can be removed when cleaning up codebase after we are satisfied what type of buffer we should use) > > > > i do know that Visual Studio Code started with an array of lines and > > > switched to a piece table (common in word processors) > > > [https://en.wikipedia.org/wiki/Piece_table], but then you end up > > > n
Re: [Toybox] [PATCH] vi: don't exit on ^C or ^D.
Yeah even tho Rob didint have ex in roadmap, it has started to be more
clear that it is needed for testing purposes. Shame that I dont know
mostly any of the commands, I only use r, w, s//, absolute jumps with
numbers, and few set variables...
ex mode command launcher should be rewritten to parse actual command
syntax, as well as most of the vi_main() should be clean up. (this is
code that I dont need to touch for now in my text buffer handling
rework)
-Jarno
On Fri, Jan 17, 2020 at 7:47 AM enh wrote:
>
> On Tue, Jan 14, 2020 at 11:05 PM Jarno Mäkipää wrote:
> >
> > Hi
> >
> > On Wed, Jan 15, 2020 at 12:23 AM enh via Toybox
> > wrote:
> > >
> > > ^D is the opposite of ^U in vi (the ^D/^U pair is the half-screen
> > > version of ^F/^B). ^C is unbound in vi. It's pretty surprising for these
> > > to cause toybox vi to exit, and it's annoying as long as toybox vi
> > > unconditionally exits rather than checks whether there are unsaved
> > > modifications!
> > >
> > > (I'm tempted to implement ^D/^U and ^F/^B, but I don't want to make
> > > Jarno's rebase of his in-progress changes any harder.)
> >
> > Dont worry, if you feel like you need some feature implement it. I can
> > always rebase. I send small patch of changes I did few weeks ago,
> > rebased on top of this.
> >
> > I had idea of changing dlist of heap allocated lines into memory
> > blocks that can be mmap:ed file and ordered list of offsets that
> > describe data in its current state.
> >
> > // mem_block contains RO data that is either original file as mmap
> > // or heap allocated inserted data. mem_blocks are not deleted or modified
> > // but instead slice_list is modified on cut operations, and insert
> > // allocates new memblock into list and adds slices referencing it.
> > //
> > struct block_list {
> > struct block_list *next, *prev;
> > struct mem_block {
> > size_t size;
> > size_t len;
> > enum alloc_flag {
> > MMAP, //can be munmap() before exit()
> > HEAP, //can be free() before exit()
> > STACK, //global or stack perhaps toybuf
> > } alloc;
> > const char *data;
> > } *node;
> > } *text;
> >
> > // slices do not contain actual allocated data but slices of data in
> > mem_block
> > // when file is first opened it has only one slice.
> > // after inserting data into middle new mem_block is allocated for insert
> > data
> > // and 3 slices are created, where first and last slice are pointing
> > to original
> > // mem_block with offsets, and middle slice is pointing to newly
> > allocated block
> > // When deleting, data is not freed but mem_blocks are sliced more
> > such way that
> > // deleted data left between 2 slices
> > struct slice_list {
> > struct slice_list *next, *prev;
> > struct slice {
> > size_t len;
> > const char *data;
> > } *node;
> > } *slices;
> >
> > So when you open file you have one mem_block and one slice, and when
> > you cut line at middle
> > you still have one block but 2 slices with hole between them. and
> > insert adds new memblock, and adds/modifies 1-3 slices depending on
> > position.
> >
> > This should make opening big files possible, implementing undo history
> > will be simpler, few operations will be much simpler, few harder...
> > but unfortunately big portion of movements needs to be rewritten.
> >
> > I had 11hour flight back to europe on Monday with chromebook sitting
> > on my lap, and I got about half of the implementation ready. But I
> > dont want to send patch before it is usable working state, since this
> > thing might already have users :)
>
> i suspect you and i are the most frequent users :-)
>
> i am now _building_ vi as part of the Android toybox binary, but i
> haven't given it a symlink. so you need to know it's there (simon says
> `toybox vi`). i did this because vi is one of the oldest requests we
> have, and the computers keep hassling me to do something with the bug.
> i was nagged again last week, and finally gave in. i try to use toybox
> as much as possible on my own machines at home, but i've mostly been
> using Visual Studio Code there lately!
>
> one thing that did occur to me is that although ex(1) is obsolete,
> having an ex mode for toybox vi would make it a lot easier to write
> tests. then we wouldn't have to worry so much about regressions...
>
> > Do you think this is better design than previous
Re: [Toybox] [PATCH] vi: Replace linelist with mem_block based design
hey On Thu, Jan 16, 2020 at 6:28 PM enh wrote: > > my 2c: i wouldn't do anything this complicated. every editor i've > written or worked on in the past used the much simpler scheme of one > buffer with a gap at the most recent insertion point > [https://en.wikipedia.org/wiki/Gap_buffer]. the most extra complexity > i've seen this cause is a separate array of line breaks, but since we > have the busybox vi style of not doing line wrapping, that seems > unnecessary? I actually found this approach much simpler to implement than atleast the dlist based one I previously had. Difficulties compared to linked list of lines, is scrolling down or counting line numbers. But inserting and cutting text is actually much easier. I think Gap_buffer shares same difficulties without benefit of loading file fast, or handling big files. I can load 8gb file with this patch in blink of eye. Reason why I started doing this was that I wanted to do undo, with infinite history. And could not figure out easy way with linelist. Now undo history should be really easy to make, by just having separate slice_list of cut/insert sections in order. > > i do know that Visual Studio Code started with an array of lines and > switched to a piece table (common in word processors) > [https://en.wikipedia.org/wiki/Piece_table], but then you end up > needing a cache to make that fast, and... > https://code.visualstudio.com/blogs/2018/03/23/text-buffer-reimplementation Actually I think this method i did is Piece table? I didint know what to call it. > > On Thu, Jan 16, 2020 at 8:17 AM Rob Landley wrote: > > > > On 1/16/20 7:58 AM, Jarno Mäkipää wrote: > > > Replaced dlist linelist with continuous memory blocks. This will allow > > > editing huge files without billion mallocs. > > > > More or less what I was trying to implement back when vi was a thing I was > > going > > to work on. > > > > > File is first opened with > > > mmap() and mapped region is fully described in block_list as one block. > > > > When I tried this it turned into a complex mess of different objects with > > different allocation/lifetime rules that had to interact. I intended to cap > > the > > block size because splitting becomes painful otherwise (and then wound up > > having > > strings individually allocated with the array of pointers to the start of > > strings being a seperate allocation), and of course mmap() requiring the > > offset > > to be a multiple of page size combined badly with line starts being all > > over the > > place... > > > > (I never did figure out what to do about a file that changed out from under > > you > > while you were editing it. The MAP_SHARED version has the changes invalidate > > your indexes and segfault, especially if it's truncated out from under you. > > The > > MAP_PRIVATE version doesn't work on nommu, I believe still has the truncate > > problem, and doesn't help you with parts it hasn't faulted in until after > > the > > file changed.) I had idea that I could just heap allocate and load file with read() as fallback or if file is small. Implementation does not care what type of memory original block is. Only unload on closing needs to call either free() or munmap() > > > > > Currently "valid" data is described as slices, when first loading file > > > there is only one slice that points to memory existing in block_list. > > > When cutting text, block_list is not freed or modified, but instead > > > slice_list is modified to have "hole" between 2 slices. when inserting > > > new mem_block is added, previos slices are cut in cursor position and > > > new slice is added... > > > > *shrug* I leave this to you. > > > > > Implementation is not as finished as old linelist, but most of the heavy > > > lifting is done so wanted to share this at this state. > > > > > > Added functions to handling data inside block_list+slice_list > > > > > > insert_str(), cut_str() are used for all delete and add operations > > > text_strrchr(), text_strchr() are used for searching lineendings > > > text_byte(), text_codepoint(), text_getline() are for simple data access > > > > > > Implemented: yank,delete,insert,push and most of the previos moves > > > > > > Implemented partly: saving of file, since current file is still mmaped > > > cant save on top of it, > > > > You never want to do that anyway because a crash halfway through (battery > > dies, > > etc) leaves you a corrupted file. > > > > > so now saves into "filename"
[Toybox] [PATCH] vi: Replace linelist with mem_block based design
Replaced dlist linelist with continuous memory blocks. This will allow
editing huge files without billion mallocs. File is first opened with
mmap() and mapped region is fully described in block_list as one block.
Currently "valid" data is described as slices, when first loading file
there is only one slice that points to memory existing in block_list.
When cutting text, block_list is not freed or modified, but instead
slice_list is modified to have "hole" between 2 slices. when inserting
new mem_block is added, previos slices are cut in cursor position and
new slice is added...
Implementation is not as finished as old linelist, but most of the heavy
lifting is done so wanted to share this at this state.
Added functions to handling data inside block_list+slice_list
insert_str(), cut_str() are used for all delete and add operations
text_strrchr(), text_strchr() are used for searching lineendings
text_byte(), text_codepoint(), text_getline() are for simple data access
Implemented: yank,delete,insert,push and most of the previos moves
Implemented partly: saving of file, since current file is still mmaped
cant save on top of it, so now saves into "filename".swp, later need
to move .swp into original file after munmap
Not Implemented: search, linenumber counting, etc..
Removed: Some unused functions
br
-Jarno
---
toys/pending/vi.c | 1237 +
1 file changed, 692 insertions(+), 545 deletions(-)
From ecc4485700c4db93820513d94e632910543b24d2 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Jarno=20M=C3=A4kip=C3=A4=C3=A4?=
Date: Sat, 11 Jan 2020 00:36:02 +0800
Subject: [PATCH] vi: Replace linelist with mem_block based design
Replaced dlist linelist with continuous memory blocks. This will allow
editing huge files without billion mallocs. File is first opened with
mmap() and mapped region is fully described in block_list as one block.
Currently "valid" data is described as slices, when first loading file
there is only one slice that points to memory existing in block_list.
When cutting text, block_list is not freed or modified, but instead
slice_list is modified to have "hole" between 2 slices. when inserting
new mem_block is added, previos slices are cut in cursor position and
new slice is added...
Implementation is not as finished as old linelist, but most of the heavy
lifting is done so wanted to share this at this state.
Added functions to handling data inside block_list+slice_list
insert_str(), cut_str() are used for all delete and add operations
text_strrchr(), text_strchr() are used for searching lineendings
text_byte(), text_codepoint(), text_getline() are for simple data access
Implemented: yank,delete,insert,push and most of the previos moves
Implemented partly: saving of file, since current file is still mmaped
cant save on top of it, so now saves into "filename".swp, later need
to move .swp into original file after munmap
Not Implemented: search, linenumber counting, etc..
Removed: Some unused functions
---
toys/pending/vi.c | 1237 +
1 file changed, 692 insertions(+), 545 deletions(-)
diff --git a/toys/pending/vi.c b/toys/pending/vi.c
index 9c47a4c5..07413c9b 100644
--- a/toys/pending/vi.c
+++ b/toys/pending/vi.c
@@ -41,20 +41,64 @@ GLOBALS(
int len;
char *data;
} *il;
-struct linelist {
- struct linelist *up;//next
- struct linelist *down;//prev
- struct str_line *line;
-} *text, *screen, *c_r;
+size_t screen; //offset in slices must be higher than cursor
+size_t cursor; //offset in slices
//yank buffer
struct yank_buf {
char reg;
int alloc;
char* data;
} yank;
+
+// mem_block contains RO data that is either original file as mmap
+// or heap allocated inserted data
+//
+//
+//
+ struct block_list {
+struct block_list *next, *prev;
+struct mem_block {
+ size_t size;
+ size_t len;
+ enum alloc_flag {
+MMAP, //can be munmap() before exit()
+HEAP, //can be free() before exit()
+STACK, //global or stack perhaps toybuf
+ } alloc;
+ const char *data;
+} *node;
+ } *text;
+
+// slices do not contain actual allocated data but slices of data in mem_block
+// when file is first opened it has only one slice.
+// after inserting data into middle new mem_block is allocated for insert data
+// and 3 slices are created, where first and last slice are pointing to original
+// mem_block with offsets, and middle slice is pointing to newly allocated block
+// When deleting, data is not freed but mem_blocks are sliced more such way that
+// deleted data left between 2 slices
+ struct slice_list {
+struct slice_list *next, *prev;
+struct slice {
+ size_t len;
+ const char *data;
+} *node;
+ } *slices;
+
+ size_t filesize;
+
)
+// TT.vi_mov_flag is used for special cases when certain move
+// acts differently depending is there DELETE/YANK or NOP
Re: [Toybox] [PATCH] vi: don't exit on ^C or ^D.
Hi
On Wed, Jan 15, 2020 at 12:23 AM enh via Toybox
wrote:
>
> ^D is the opposite of ^U in vi (the ^D/^U pair is the half-screen
> version of ^F/^B). ^C is unbound in vi. It's pretty surprising for these
> to cause toybox vi to exit, and it's annoying as long as toybox vi
> unconditionally exits rather than checks whether there are unsaved
> modifications!
>
> (I'm tempted to implement ^D/^U and ^F/^B, but I don't want to make
> Jarno's rebase of his in-progress changes any harder.)
Dont worry, if you feel like you need some feature implement it. I can
always rebase. I send small patch of changes I did few weeks ago,
rebased on top of this.
I had idea of changing dlist of heap allocated lines into memory
blocks that can be mmap:ed file and ordered list of offsets that
describe data in its current state.
// mem_block contains RO data that is either original file as mmap
// or heap allocated inserted data. mem_blocks are not deleted or modified
// but instead slice_list is modified on cut operations, and insert
// allocates new memblock into list and adds slices referencing it.
//
struct block_list {
struct block_list *next, *prev;
struct mem_block {
size_t size;
size_t len;
enum alloc_flag {
MMAP, //can be munmap() before exit()
HEAP, //can be free() before exit()
STACK, //global or stack perhaps toybuf
} alloc;
const char *data;
} *node;
} *text;
// slices do not contain actual allocated data but slices of data in mem_block
// when file is first opened it has only one slice.
// after inserting data into middle new mem_block is allocated for insert data
// and 3 slices are created, where first and last slice are pointing
to original
// mem_block with offsets, and middle slice is pointing to newly
allocated block
// When deleting, data is not freed but mem_blocks are sliced more
such way that
// deleted data left between 2 slices
struct slice_list {
struct slice_list *next, *prev;
struct slice {
size_t len;
const char *data;
} *node;
} *slices;
So when you open file you have one mem_block and one slice, and when
you cut line at middle
you still have one block but 2 slices with hole between them. and
insert adds new memblock, and adds/modifies 1-3 slices depending on
position.
This should make opening big files possible, implementing undo history
will be simpler, few operations will be much simpler, few harder...
but unfortunately big portion of movements needs to be rewritten.
I had 11hour flight back to europe on Monday with chromebook sitting
on my lap, and I got about half of the implementation ready. But I
dont want to send patch before it is usable working state, since this
thing might already have users :)
Do you think this is better design than previous one? if you have time
or interest to take a look I attached my vi.c from my working tree
br
Jarno
> ---
> toys/pending/vi.c | 10 +++---
> 1 file changed, 3 insertions(+), 7 deletions(-)
> ___
> Toybox mailing list
> Toybox@lists.landley.net
> http://lists.landley.net/listinfo.cgi/toybox-landley.net
/* vi.c - You can't spell "evil" without "vi".
*
* Copyright 2015 Rob Landley
* Copyright 2019 Jarno Mäkipää
*
* See http://pubs.opengroup.org/onlinepubs/9699919799/utilities/vi.html
USE_VI(NEWTOY(vi, "<1>1", TOYFLAG_USR|TOYFLAG_BIN))
config VI
bool "vi"
default n
help
usage: vi FILE
Visual text editor. Predates the existence of standardized cursor keys,
so the controls are weird and historical.
*/
#define FOR_vi
#include "toys.h"
GLOBALS(
int cur_col;
int cur_row;
int scr_row;
int drawn_row;
int drawn_col;
unsigned screen_height;
unsigned screen_width;
int vi_mode;
int count0;
int count1;
int vi_mov_flag;
int modified;
char vi_reg;
char *last_search;
int tabstop;
int list;
struct str_line {
int alloc;
int len;
char *data;
} *il;
size_t screen; //offset in slices must be higher than cursor
size_t cursor; //offset in slices
//yank buffer
struct yank_buf {
char reg;
int alloc;
char* data;
} yank;
// mem_block contains RO data that is either original file as mmap
// or heap allocated inserted data
//
//
//
struct block_list {
struct block_list *next, *prev;
struct mem_block {
size_t size;
size_t len;
enum alloc_flag {
MMAP, //can be munmap() before exit()
HEAP, //can be free() before exit()
STACK, //global or stack perhaps toybuf
} alloc;
const char *data;
} *node;
} *text;
// slices do not contain actual allocated data but slices of data in mem_block
// when file is first opened it has only one slice.
// after inserting data into mid
[Toybox] [PATCH] vi: fixes and small cleanup
Sorry somehow didint copy [PATCH] keyword into start of the subject...
this is duplicate of previous msg but just in case someone filters out
[PATCH] messages
Hello
Fix annoying first line delete bug, and some tidying up.
fix: first line delete
fix: delete with e move
fix: statusline 1 row lower, remove eol
cleanup: use dlist_pop on delete
cleanup: move globals into GLOBALS
---
toys/pending/vi.c | 517 ++
1 file changed, 251 insertions(+), 266 deletions(-)
From 9c2223c66bf303e1f2b8b6d2f84a4aa5a1fbb959 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Jarno=20M=C3=A4kip=C3=A4=C3=A4?=
Date: Fri, 3 Jan 2020 11:22:38 +0800
Subject: [PATCH] vi: fixes and small cleanup
fix: first line delete
fix: delete with e move
fix: statusline 1 row lower, remove eol
cleanup: use dlist_pop on delete
cleanup: move globals into GLOBALS
---
toys/pending/vi.c | 517 ++
1 file changed, 251 insertions(+), 266 deletions(-)
diff --git a/toys/pending/vi.c b/toys/pending/vi.c
index 2fb6d442..9c47a4c5 100644
--- a/toys/pending/vi.c
+++ b/toys/pending/vi.c
@@ -36,28 +36,26 @@ GLOBALS(
char *last_search;
int tabstop;
int list;
+struct str_line {
+ int alloc;
+ int len;
+ char *data;
+} *il;
+struct linelist {
+ struct linelist *up;//next
+ struct linelist *down;//prev
+ struct str_line *line;
+} *text, *screen, *c_r;
+//yank buffer
+struct yank_buf {
+ char reg;
+ int alloc;
+ char* data;
+} yank;
)
-struct str_line {
- int alloc;
- int len;
- char *data;
-};
-//yank buffer
-struct yank_buf {
- char reg;
- int alloc;
- char* data;
-};
-//lib dllist uses next and prev kinda opposite what im used to so I just
-//renamed both ends to up and down
-struct linelist {
- struct linelist *up;//next
- struct linelist *down;//prev
- struct str_line *line;
-};
static void draw_page();
@@ -79,13 +77,7 @@ static int search_str(char *s);
static int vi_yank(char reg, struct linelist *row, int col, int flags);
static int vi_delete(char reg, struct linelist *row, int col, int flags);
-//inserted line not yet pushed to buffer
-struct str_line *il;
-struct linelist *text; //file loaded into buffer
-struct linelist *screen;//current screen coord 0 row
-struct linelist *c_r;//cursor position row
-struct yank_buf yank; //single yank
// TT.vi_mov_flag is used for special cases when certain move
// acts differently depending is there DELETE/YANK or NOP
@@ -128,15 +120,15 @@ void linelist_free(void *node)
void linelist_unload()
{
void* list = 0;
- for (;text->down; text = text->down);
- list = (void*)text;
- text = screen = c_r = 0;
+ for (;TT.text->down; TT.text = TT.text->down);
+ list = (void*)TT.text;
+ TT.text = TT.screen = TT.c_r = 0;
llist_traverse(list, linelist_free);
}
void write_file(char *filename)
{
- struct linelist *lst = text;
+ struct linelist *lst = TT.text;
FILE *fp = 0;
if (!filename) filename = (char*)*toys.optargs;
if (!(fp = fopen(filename, "w")) ) return;
@@ -149,7 +141,7 @@ void write_file(char *filename)
int linelist_load(char *filename)
{
- struct linelist *lst = c_r;//cursor position or 0
+ struct linelist *lst = TT.c_r;//cursor position or 0
FILE *fp = 0;
if (!filename)
filename = (char*)*toys.optargs;
@@ -163,8 +155,8 @@ int linelist_load(char *filename)
lst->line->alloc = alc;
lst->line->len = 0;
lst->line->data = line;
-text = lst;
-dlist_terminate(text->up);
+TT.text = lst;
+dlist_terminate(TT.text->up);
return 1;
}
@@ -189,10 +181,10 @@ int linelist_load(char *filename)
lst->line->data[len-1] = 0;
lst->line->len--;
}
-if (text == 0) text = lst;
+if (TT.text == 0) TT.text = lst;
}
- if (text) dlist_terminate(text->up);
+ if (TT.text) dlist_terminate(TT.text->up);
fclose(fp);
return 1;
@@ -201,7 +193,7 @@ int linelist_load(char *filename)
int vi_yy(char reg, int count0, int count1)
{
- struct linelist *pos = c_r;
+ struct linelist *pos = TT.c_r;
int col = TT.cur_col;
TT.cur_col = 0;
TT.vi_mov_flag |= 0x4;
@@ -210,13 +202,13 @@ int vi_yy(char reg, int count0, int count1)
vi_yank(reg, pos, 0, 0);
- TT.cur_col = col, c_r = pos;
+ TT.cur_col = col, TT.c_r = pos;
return 1;
}
int vi_dd(char reg, int count0, int count1)
{
- struct linelist *pos = c_r;
+ struct linelist *pos = TT.c_r;
TT.cur_col = 0;
TT.vi_mov_flag |= 0x4;
if (count0>1) cur_down(count0-1, 1, 0);
@@ -230,12 +222,12 @@ static int vi_x(char reg, int count0, int count1)
{
char *last = 0, *cpos = 0, *start = 0;
int len = 0;
- struct linelist *pos = c_r;
+ struct linelist *pos = TT.c_r;
int col = TT.cur_col;
- if (!c_r) return 0;
+ if (!TT.c_r) return 0;
- start = c_r->line->data;
- len = c_r->line->len;
+ start = TT.c_r->line->data;
+ len = TT.c_r->line->len;
last =
[Toybox] vi: fixes and small cleanup
Hello
Fix annoying first line delete bug, and some tidying up.
fix: first line delete
fix: delete with e move
fix: statusline 1 row lower, remove eol
cleanup: use dlist_pop on delete
cleanup: move globals into GLOBALS
---
toys/pending/vi.c | 517 ++
1 file changed, 251 insertions(+), 266 deletions(-)
From 9c2223c66bf303e1f2b8b6d2f84a4aa5a1fbb959 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Jarno=20M=C3=A4kip=C3=A4=C3=A4?=
Date: Fri, 3 Jan 2020 11:22:38 +0800
Subject: [PATCH] vi: fixes and small cleanup
fix: first line delete
fix: delete with e move
fix: statusline 1 row lower, remove eol
cleanup: use dlist_pop on delete
cleanup: move globals into GLOBALS
---
toys/pending/vi.c | 517 ++
1 file changed, 251 insertions(+), 266 deletions(-)
diff --git a/toys/pending/vi.c b/toys/pending/vi.c
index 2fb6d442..9c47a4c5 100644
--- a/toys/pending/vi.c
+++ b/toys/pending/vi.c
@@ -36,28 +36,26 @@ GLOBALS(
char *last_search;
int tabstop;
int list;
+struct str_line {
+ int alloc;
+ int len;
+ char *data;
+} *il;
+struct linelist {
+ struct linelist *up;//next
+ struct linelist *down;//prev
+ struct str_line *line;
+} *text, *screen, *c_r;
+//yank buffer
+struct yank_buf {
+ char reg;
+ int alloc;
+ char* data;
+} yank;
)
-struct str_line {
- int alloc;
- int len;
- char *data;
-};
-//yank buffer
-struct yank_buf {
- char reg;
- int alloc;
- char* data;
-};
-//lib dllist uses next and prev kinda opposite what im used to so I just
-//renamed both ends to up and down
-struct linelist {
- struct linelist *up;//next
- struct linelist *down;//prev
- struct str_line *line;
-};
static void draw_page();
@@ -79,13 +77,7 @@ static int search_str(char *s);
static int vi_yank(char reg, struct linelist *row, int col, int flags);
static int vi_delete(char reg, struct linelist *row, int col, int flags);
-//inserted line not yet pushed to buffer
-struct str_line *il;
-struct linelist *text; //file loaded into buffer
-struct linelist *screen;//current screen coord 0 row
-struct linelist *c_r;//cursor position row
-struct yank_buf yank; //single yank
// TT.vi_mov_flag is used for special cases when certain move
// acts differently depending is there DELETE/YANK or NOP
@@ -128,15 +120,15 @@ void linelist_free(void *node)
void linelist_unload()
{
void* list = 0;
- for (;text->down; text = text->down);
- list = (void*)text;
- text = screen = c_r = 0;
+ for (;TT.text->down; TT.text = TT.text->down);
+ list = (void*)TT.text;
+ TT.text = TT.screen = TT.c_r = 0;
llist_traverse(list, linelist_free);
}
void write_file(char *filename)
{
- struct linelist *lst = text;
+ struct linelist *lst = TT.text;
FILE *fp = 0;
if (!filename) filename = (char*)*toys.optargs;
if (!(fp = fopen(filename, "w")) ) return;
@@ -149,7 +141,7 @@ void write_file(char *filename)
int linelist_load(char *filename)
{
- struct linelist *lst = c_r;//cursor position or 0
+ struct linelist *lst = TT.c_r;//cursor position or 0
FILE *fp = 0;
if (!filename)
filename = (char*)*toys.optargs;
@@ -163,8 +155,8 @@ int linelist_load(char *filename)
lst->line->alloc = alc;
lst->line->len = 0;
lst->line->data = line;
-text = lst;
-dlist_terminate(text->up);
+TT.text = lst;
+dlist_terminate(TT.text->up);
return 1;
}
@@ -189,10 +181,10 @@ int linelist_load(char *filename)
lst->line->data[len-1] = 0;
lst->line->len--;
}
-if (text == 0) text = lst;
+if (TT.text == 0) TT.text = lst;
}
- if (text) dlist_terminate(text->up);
+ if (TT.text) dlist_terminate(TT.text->up);
fclose(fp);
return 1;
@@ -201,7 +193,7 @@ int linelist_load(char *filename)
int vi_yy(char reg, int count0, int count1)
{
- struct linelist *pos = c_r;
+ struct linelist *pos = TT.c_r;
int col = TT.cur_col;
TT.cur_col = 0;
TT.vi_mov_flag |= 0x4;
@@ -210,13 +202,13 @@ int vi_yy(char reg, int count0, int count1)
vi_yank(reg, pos, 0, 0);
- TT.cur_col = col, c_r = pos;
+ TT.cur_col = col, TT.c_r = pos;
return 1;
}
int vi_dd(char reg, int count0, int count1)
{
- struct linelist *pos = c_r;
+ struct linelist *pos = TT.c_r;
TT.cur_col = 0;
TT.vi_mov_flag |= 0x4;
if (count0>1) cur_down(count0-1, 1, 0);
@@ -230,12 +222,12 @@ static int vi_x(char reg, int count0, int count1)
{
char *last = 0, *cpos = 0, *start = 0;
int len = 0;
- struct linelist *pos = c_r;
+ struct linelist *pos = TT.c_r;
int col = TT.cur_col;
- if (!c_r) return 0;
+ if (!TT.c_r) return 0;
- start = c_r->line->data;
- len = c_r->line->len;
+ start = TT.c_r->line->data;
+ len = TT.c_r->line->len;
last = utf8_last(start, len);
cpos = start+TT.cur_col;
@@ -262,35 +254,35 @@ int vi_movw(int count0, int count1, char* unused)
const char *empties = " \t\n\r";
const char
Re: [Toybox] [PATCH] vi: fix warnings, improve status display.
Hi Yeah I have been about to remove that statusline thingy, it was mostly to debug some stuff, such as mode switch and cursoring around. This more classical look is better. In fact I already had similar patch ready, along with few other fixes, but I have been busy travelling past few weeks in Taiwan. So didint have time to send them. I will rebase myself on top of this and send my own fixes after Im done with travelling next week or so. -Jarno ___ Toybox mailing list Toybox@lists.landley.net http://lists.landley.net/listinfo.cgi/toybox-landley.net
[Toybox] Did I mention why I didn't implement out of tree build?
It seems that gmail stopped receiving emails from list again. I got kicked out of list for excessive bounces yesterday... Somehow gmail worked for month or two after I complained about it last time. -Jarno > Hi Rob, > I saw this email and thought this patch might help make > `relative_path()` better. > > Commit Message: > While trying to add features to `readlink` and `realpath` I saw that > adding a "/" to the end of the `from` variable, made it possible for > me to get `realpath --relative-to=DIR` to work. > The change seems like a hack, but I saw Rob's email > https://www.mail-archive.com/toybox@lists.landley.net/msg06136.html > and though this might be able to help. This works for `realpath` > because `--relative-to` is only supposed to take a directory. > ~Andrew > PS: I hope this email looks correct. I made it by hand because of > Gmail deciding not to give me the original message. > https://www.mail-archive.com/toybox@lists.landley.net/msg06136.html > > On 08 Nov 2019 18:27:44 -0800, Rob via Toybox wrote: > > In theory if you're using toybox cp you can use a relative path, which was > > part > > of why I did relative_path but darn it it's STILL broken: ___ Toybox mailing list Toybox@lists.landley.net http://lists.landley.net/listinfo.cgi/toybox-landley.net
Re: [Toybox] Testing: how to check return code
Hi
Tested ls with invalid options on freebsd and it returns only 0 or 1
and never 2. And quick look at source confirms my test... So returning
2 is gnuism.
I dont see any actual use case for writing script with invalid
argument and checking specific return code?
On Mon, Nov 4, 2019 at 3:07 AM Rob Landley wrote:
>
> On 10/30/19 1:56 PM, enh via Toybox wrote:
> >> I am with you enh. In this case, I wanted the error code because
> >> GNU `ls` says the command can exit with { 0, 1, 2 }.
> >> https://linux.die.net/man/1/ls
> >> POSIX says exit status is { 0, >0 }
> >> https://pubs.opengroup.org/onlinepubs/9699919799/
> >> In this case which specification wins?
> >
> > they're both true, but the GNU ls is more specific :-)
>
> Hmmm, I need to add http://man7.org/linux/man-pages/dir_section_1.html to the
> roadmap, don't I? Except a quick cut and paste pipe through wc says there's
> just
> under 1500 entries in there...
>
> >> enh, the other week you mentioned Android will not use the build if
> >> there are failing tests.
> >
> > correct. that's why i'm a bit behind --- xargs is still failing.
>
> I thought I checked in your 4k workaround last week? (Commit f95d580892e2 ?)
>
> Hmmm, just did a "git push" and it found stuff... (Sorry, Japan's been as busy
> as you'd expect.)
>
> >> Does the above apply to `TEST_HOST=1
> >> make tests`? If so, then I will rewrite the test.
> >
> > not at the moment, no, but you probably want `SKIP_HOST=1` before the
> > affected `testing` (but on the same line so it doesn't apply to later
> > instances)?
>
> I'm trying to migrate from SKIP_HOST to toyonly, because you could in theory
> have toybox on the host and the decision we're really making is "this is known
> to work in toybox, but not anywhere else".
>
> Note: toyonly is a shell function that does an eval on its arguments instead
> of
> an environment variable that signals the plumbing to make a decision, so it's
> called slightly differently. But there should be a few examples in the tree...
>
> > or fix ls to behave like the host. see grep for a similar example.
> > (or, generally, grep for `exitval`.)
>
> I still wince at being too specific in the tests, but if there are actual
> examples of scripts caring about this level of detail and being broken by not
> getting it right... then the help text output should probably mention the
> error
> codes.
>
> And _possibly_ we should work out some standard behavior to enforce across the
> commands? If "0" and "1" determine the result of a test (like cmp), neither of
> which is actually an error, then "2" being the default _error_ value
> everywhere
> might make sense? (But would we then break scripts expecting 1?)
>
> I'm uncomfortable whack-a-moling this. I kinda want a rule.
>
> Rob
> ___
> Toybox mailing list
> Toybox@lists.landley.net
> http://lists.landley.net/listinfo.cgi/toybox-landley.net
___
Toybox mailing list
Toybox@lists.landley.net
http://lists.landley.net/listinfo.cgi/toybox-landley.net
Re: [Toybox] Testing: how to check return code
Hi I think some other tests use this kind of execution to test commands that should fail cmd1 && cmd2 && echo success || echo epic fail which is same as if cmd1 && cmd2; then echo success else echo epic fail fi So maybe your test could be written something like this, where stderr is piped to null and when failing command we should get "yes\n" testing "with -w fail negative" "ls -w -5 2> /dev/null && echo no || echo yes" "yes\n" "" "" try check other tests that has echo yes in them for reference. -Jarno On Wed, Oct 30, 2019 at 3:44 PM Andrew Ilijic wrote: > > While implementing `-w` for the `ls` command, I coded it so that the > command would fail if the user tried to pass a negative value. I > wanted to test the return code with a negative argument but could not > make it work. I am not great at Bash. If anyone can tell me what I was > doing wrong, I would appreciate it. > > ``` > testing "with -w - Fail on negative" \ > "$IN && ls -w -5 > /dev/null; echo $? && $OUT" \ > "1" "" "" > ``` > The command would fail and return 1 but, in testing it would always > write zero to the file. > > ~Andrew > ___ > Toybox mailing list > Toybox@lists.landley.net > http://lists.landley.net/listinfo.cgi/toybox-landley.net ___ Toybox mailing list Toybox@lists.landley.net http://lists.landley.net/listinfo.cgi/toybox-landley.net
Re: [Toybox] [PATCH] ls: Ensure file names are separated by 2 spaces
On Wed, Oct 30, 2019 at 1:08 AM Denys Nykula wrote: > > > > be able to check "where is the cursor now" after > > > operations, and maybe get a dump of the simulated screen output. > > Jarno, you could add vi an option that applies headlessly a key sequence > from a file? For example, a regression test with it would catch this > first line deletion error that I have now. Pretending the option exists: > > printf '\nseitan\n' >seitan; printf 'dd\n:wq!\n' >key > vi -s key seitan; cat seitan > # Want: seitan\n > # Have: Segmentation fault Hi Denys Yeah this would be easiest way to do it. vi posix specification already has -c option but that parsers ex commands. This visual mode input keyfile could be handy, but Im not sure do I like to add extensions before even completing posix level of functionality. But things like this first line deletion seqfault that i introduced probably 1 or 2 patches ago when rewriting delete operation really motivates me having some sort of testing framework, and your solution would be easiest to implement for now. I would just need to find flagname that no common vi implementation uses yet. > > > That could be very handy for testing things like less, man and vi that > > should perhaps eventually share drawing, movement and reg exp search > > codebase... > > I don't install either more or less, and will be glad to not need either > vim or a lynx wrapper, if you teach toy vi stdin. Other tools should > just pipe to it. It's a case where reuse of entire command makes more > sense than sharing a library function. Yeah we just need -R option for viewonly and handling of stdin to handle things like cat file | vi -R - -Jarno ___ Toybox mailing list Toybox@lists.landley.net http://lists.landley.net/listinfo.cgi/toybox-landley.net
Re: [Toybox] [PATCH] ls: Ensure file names are separated by 2 spaces
On Tue, Oct 29, 2019 at 6:08 AM Rob Landley wrote: > > On 10/28/19 10:07 AM, Jarno Mäkipää wrote: > > I can try to help writing tests for lib/ utf-8 stuff, and I am sure > > there is lot of things to do there to do for Andrew also. Anyway there > > is ton of corner cases. > > > > I would also like to know if there would be anyway to do automated > > testing for VI, ofc I could do it by implementing ex command > > substitute for every visual mode command and then implement -c option > > with multiple commands in row parsing. But I feel that implementing ex > > commands just for testing purposes is bit tedious. And I am not sure > > if there is visual mode commands that does not have ex mode > > substitute. So if there would be easy way to push "interactive" > > command slowly like "5G3dd:wq\n" and then check diff if 3 lines are > > missing after line 5 or not... > > For years I've wanted to do a pty master/slave command that would run another > command in a simulated pty, feed input into it ala expect with simulated X > and Y > size of the "screen", and be able to check "where is the cursor now" after > operations, and maybe get a dump of the simulated screen output. That could be very handy for testing things like less, man and vi that should perhaps eventually share drawing, movement and reg exp search codebase... I was little bit thinking could I just hack suckless terminal run my own vi tests, since its simple enough to understand, already used it for debugging escape codes > > Alas, it was on my todo list along with implementing screen, and people on > here > insist that only tmux with random features I have no interest in is ever worth > doing, so... nope, no plans. I dont know reason why tmux should be in scope of toybox, it already has bsd style licence and it is semi active, why not just build the real thing on top of toybox on need basis... ok they need find curses implementation and few other libs, but still. Screen on other hand could be useful with only ctrl+A+D and -r feature, would probably cover 90% of users. -Jarno > > Rob ___ Toybox mailing list Toybox@lists.landley.net http://lists.landley.net/listinfo.cgi/toybox-landley.net
Re: [Toybox] [PATCH] ls: Ensure file names are separated by 2 spaces
On Mon, Oct 28, 2019 at 8:14 AM Rob Landley wrote: > > > > On 10/25/19 4:12 PM, Jarno Mäkipää wrote: > > hi > > > > Feel free to ask me if you run into any questions. And also share your > > experiments and results. I have gained some knowledge on utf-8 > > encoding past years. I did internationalization support for > > microcontroller based system around ~2011 on my $DAYJOB. And end up > > writing utf-8 handling and bitmap font rendering as well as bitmap > > font generator that limits glyphs "installed" to system to only the > > ones that exist in translation package to save limited flash memory. > > > > Last spring I started refreshing my memory bit while writing vi > > implementation. But under posix system things are tiny bit easier > > since there is proper c library that provides mbtowc and wcwidth > > etc... > > > > Most languages dont actually use combining characters, but one that I > > find difficult to handle is hindi, even vim/neovim has problems > > stepping around some hindi text I found > > If you could add more tests to the test suite for utf8, that would be lovely. > I > do not have domain expertise here. > > I note the toys/example directory has demo_*.c commands which can have their > own tests/demo_blah.test files (see tests/demo_number.test for example), and > that's the way I test lib/ infrastructure when there isn't really a command > for > it. The tests/files/utf8 directory is just "some stuff I collected". It's not > remotely systematic... I can try to help writing tests for lib/ utf-8 stuff, and I am sure there is lot of things to do there to do for Andrew also. Anyway there is ton of corner cases. I would also like to know if there would be anyway to do automated testing for VI, ofc I could do it by implementing ex command substitute for every visual mode command and then implement -c option with multiple commands in row parsing. But I feel that implementing ex commands just for testing purposes is bit tedious. And I am not sure if there is visual mode commands that does not have ex mode substitute. So if there would be easy way to push "interactive" command slowly like "5G3dd:wq\n" and then check diff if 3 lines are missing after line 5 or not... -Jarno > > Rob ___ Toybox mailing list Toybox@lists.landley.net http://lists.landley.net/listinfo.cgi/toybox-landley.net
Re: [Toybox] [PATCH] ls: Ensure file names are separated by 2 spaces
hi Feel free to ask me if you run into any questions. And also share your experiments and results. I have gained some knowledge on utf-8 encoding past years. I did internationalization support for microcontroller based system around ~2011 on my $DAYJOB. And end up writing utf-8 handling and bitmap font rendering as well as bitmap font generator that limits glyphs "installed" to system to only the ones that exist in translation package to save limited flash memory. Last spring I started refreshing my memory bit while writing vi implementation. But under posix system things are tiny bit easier since there is proper c library that provides mbtowc and wcwidth etc... Most languages dont actually use combining characters, but one that I find difficult to handle is hindi, even vim/neovim has problems stepping around some hindi text I found -Jarno On Fri, Oct 25, 2019 at 11:26 PM Andrew Ilijic wrote: > > Hi Jarno, > > Thank you for your email. It caused me to learn a lot about Unicode > today. I still need to collect my thoughts but I wanted to respond to > your email. I tried throwing all kinds of crazy Unicode at XTerm, > Gnome Terminal, and Suckless Terminal. I don't think any of them do > the "right" thing in all cases but, XTerm seems to be the best. `ls` > seems to be handling combining characters well. I am going to compare > how `cut` and `ls` handle clipping the string. I have started to > implement the column (`-w`) option in `ls` so I may shoot you an email > if I have questions. > > ~Andrew ___ Toybox mailing list Toybox@lists.landley.net http://lists.landley.net/listinfo.cgi/toybox-landley.net
Re: [Toybox] [PATCH] cut: re-enable crunch_str on cut -C
ping
While its not necessary to apply this patch I was hoping to rise some
conversation. My point is that crunch_str() works correctly. It pushes
0 width combining chars to stdout even after colums==width. And reason
it renders wrong is terminal clients fault. While there is no
standard, the xterm terminal client that is closest thing to reference
implementation works correctly. And if user just pipe outputs to file
or program that can actually handle them, it would be right thing to
not clip them in wrong place.
cut -C is toybox extension so it does not matter how it works, but if
it would work correctly it could be used to test other commands maybe?
br
Jarno
On Fri, Oct 18, 2019 at 9:23 PM Jarno Mäkipää wrote:
>
> Since there has been lots of topic about rendering combining
> characters I leave this commit here for discussion. crunch_str
> currently prints combining characters right. you are free to prove me
> wrong
>
> Combining characters follow the character which they modify.
> https://www.cl.cam.ac.uk/~mgk25/unicode.html#comb
>
> xterm renders cut test1.txt -C -1 now correctly, other terminal
> emulators have different results, but anyway leaving glyph incomplete
> would sacrifice that you wont be later able view it inside browser for
> example either.
> My previous message to board about vi combining chars had some utf-8
> with combining characters in it. and my ipad:s browser rendered it
> correctly, but this desktop browser did not...
>
> I dont think its good idea to leave glyph imcomplete just because some
> terminal emulators misbehave and render combining chars sometimes
> after the main glyph. Leaving glyph incomplete without combining
> characters could leave word unreadable for user that is actually
> reading his own language with his own terminal or browser that
> actually works.
>
> I will also attach 2 screenshots into mailinglist where without_crunch
> is the old implementation
> cut_with_crunch.png
> cut_without_crunch.png
>
> regards
> -Jarno
From 94bfdef4132b0b345dbce2caa13cd5c0bd560a9d Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Jarno=20M=C3=A4kip=C3=A4=C3=A4?=
Date: Fri, 18 Oct 2019 20:21:57 +0300
Subject: [PATCH] cut: re-enable crunch_str on cut -C
Reason: unicolumns() does not print combining characters correctly
Combining characters follow the character which they modify.
https://www.cl.cam.ac.uk/~mgk25/unicode.html#comb
xterm renders cut test1.txt -C -1 now correctly
---
tests/cut.test | 3 +++
toys/posix/cut.c | 38 +++---
2 files changed, 10 insertions(+), 31 deletions(-)
diff --git a/tests/cut.test b/tests/cut.test
index e475288a..8d8c4ba1 100755
--- a/tests/cut.test
+++ b/tests/cut.test
@@ -34,6 +34,9 @@ testing "-c a,b-c,d" "cut -c 3,5-7,10 abc.txt" "etwoh\npa:ba\nequi \n" "" ""
toyonly testing "-c japan.txt" 'cut -c 3-6,9-12 "$FILES/utf8/japan.txt"' \
"ガラスをられます\n" "" ""
+toyonly testing "-C test1.txt" 'cut -C -1 "$FILES/utf8/test1.txt"' \
+ "l̴̗̞̠\n" "" ""
+
# substitute for awk
toyonly testcmd "-DF" "-DF 2,7,5" \
"said and your\nare\nis demand. supply\nforecast :\nyou you better,\n\nEm: Took hate\n" "" \
diff --git a/toys/posix/cut.c b/toys/posix/cut.c
index 9f7f7458..61b2b409 100644
--- a/toys/posix/cut.c
+++ b/toys/posix/cut.c
@@ -46,28 +46,6 @@ GLOBALS(
regex_t reg;
)
-// Return number of bytes to start of first column fitting in columns
-// invalid sequences are skipped/ignored
-int unicolumns(char *start, unsigned columns)
-{
- int i, j = 0;
- wchar_t wc;
- char *s = start, *ss = start;
-
- // Skip start, rounding down if we hit a multicolumn char
- while (jcolumns) break;
-ss = s;
- }
-}
- }
-
- return ss-start;
-}
// Apply selections to an input line, producing output
static void cut_line(char **pline, long len)
@@ -99,15 +77,13 @@ static void cut_line(char **pline, long len)
// crunch_str() currently assumes that combining characters get
// escaped, to provide an unambiguous visual representation.
// This assumes the input string is null terminated.
- //if (start) crunch_str(, start, 0, 0, 0);
- //if (!*s) continue;
- //start = s-line;
- //ss = s;
- //crunch_str(, count, 0, 0, 0);
- //count = ss-s;
-
- s += unicolumns(s, start);
- count = unicolumns(s, end-start);
+ if (start) crunch_str(, start, 0, 0, 0);
+ if (!*s) continue;
+ start = s-line;
+ ss = s;
+ crunch_str(, count, 0, 0, 0);
+ count = ss-s;
+
} else if (toys.optflags_c) {
wchar_t wc;
char *sss;
--
2.19.1
___
Toybox mailing list
Toybox@lists.landley.net
http://lists.landley.net/listinfo.cgi/toybox-landley.net
Re: [Toybox] [PATCH] ls: Ensure file names are separated by 2 spaces
Hey Andrew Relating to combining char "issue" did you see patch I send to list. http://lists.landley.net/pipermail/toybox-landley.net/2019-October/011076.html I would say that crunch_str() on lib cuts strings with combining chars already works correctly, and properly implemented terminal client like xterm renders them ok. And even if user has broken terminal combining chars should be pushed after main glyph, since there might be use case that user wants to clip his program output at 80 columns and push it to file and later render file with program that works such as web browser Then again there is probably lots of toys in toybox that does not use crunch_str and have there own logic where to clip string and need lots of testing. br Jarno On Thu, Oct 24, 2019 at 5:34 PM Andrew Ilijic wrote: > > Hi Rob, > > > In theory there's an ls -w option that > > lets you set the width, but I never needed/implemented it. (Andrew: you're > > welcome to if you like. > I am going to take your advice and try to implement the `-w` argument. > It should make testing easier. Right now, I have to adjust the size of > my terminal to test certain things. After that, I am going to look at > the UTF-8 combining characters issue. > > The closer I look at the `ls` command on my system, Debian, the > weirder it gets. For example, in a directory with many files, > redirecting `ls -C` or `ls -x` to a file, the Debian implementation > puts tabs in some places and spaces in others. I don't think that this > is a behavior that we want to emulate, or do we? > > I found a link for POSIX `ls` and GNU `ls,` can you let me know if you > have any other resources? I am going to look at them to get an idea of > what is and isn't implementation-defined. > https://pubs.opengroup.org/onlinepubs/9699919799/utilities/ls.html#tag_20_73 > https://www.gnu.org/software/coreutils/manual/html_node/ls-invocation.html#ls-invocation > ___ > Toybox mailing list > Toybox@lists.landley.net > http://lists.landley.net/listinfo.cgi/toybox-landley.net ___ Toybox mailing list Toybox@lists.landley.net http://lists.landley.net/listinfo.cgi/toybox-landley.net
[Toybox] [PATCH] lib: getdirname fix seqfault on null ptr
In order to be used as drop in replacement for dirname()
If path is a null pointer or points to an empty string,
dirname() shall return a pointer to the string "." .
---
lib/lib.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
From 884abd7b484c9ecf357004eb8e123e32b6b5cd01 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Jarno=20M=C3=A4kip=C3=A4=C3=A4?=
Date: Sat, 19 Oct 2019 09:47:10 +0300
Subject: [PATCH] lib: getdirname fix seqfault on null ptr
In order to be used as drop in replacement for dirname()
If path is a null pointer or points to an empty string,
dirname() shall return a pointer to the string "." .
---
lib/lib.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/lib/lib.c b/lib/lib.c
index f98f00a5..be490eb3 100644
--- a/lib/lib.c
+++ b/lib/lib.c
@@ -1012,7 +1012,7 @@ char *getdirname(char *name)
char *s, *ss, *keep;
for (s = name, ss = keep = 0; ; s++) {
-if (!*s) return keep ? xstrndup(name, keep-name) : xstrdup(".");
+if (!s || !*s) return keep ? xstrndup(name, keep-name) : xstrdup(".");
if (*s != '/') keep = ss;
else if (s == name) keep = ss = s+1;
else if (s[-1] != '/') ss = s;
--
2.19.1
___
Toybox mailing list
Toybox@lists.landley.net
http://lists.landley.net/listinfo.cgi/toybox-landley.net
Re: [Toybox] [PATCH] wget: Added support for HTTP 301 and 302 redirects
Sure thing, here is the modified patch
-Jarno
On Sat, Oct 19, 2019 at 2:54 AM Rob Landley wrote:
>
> This got eaten by gmail's insane spam filter, but I noticed it in the web
> archive. I tried to apply it, but it doesn't apply anymore. Could you send me
> an
> updated one?
>
> http://lists.landley.net/pipermail/toybox-landley.net/2019-September/010931.html
>
> Thanks,
>
> Rob
From 6a44958825f76dc2fafbecc1a76b3b57b575039c Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Jarno=20M=C3=A4kip=C3=A4=C3=A4?=
Date: Sat, 19 Oct 2019 08:37:09 +0300
Subject: [PATCH] wget: Added support for HTTP 301 and 302 redirects
Added: parsing of redirectation responses, will now follow redirectation
path until maximum of 10 jumps. This will probably lead to server
advertising user the https url.
---
toys/pending/wget.c | 67 +++--
1 file changed, 41 insertions(+), 26 deletions(-)
diff --git a/toys/pending/wget.c b/toys/pending/wget.c
index eb496da2..405ed942 100644
--- a/toys/pending/wget.c
+++ b/toys/pending/wget.c
@@ -100,7 +100,7 @@ static int conn_svr(const char *hostname, const char *port) {
continue;
}
if (connect(sock, rp->ai_addr, rp->ai_addrlen) != -1)
- break; // succeed in connecting to any server IP
+ break; // succeed in connecting to any server IP
else perror_msg("connect error");
close(sock);
}
@@ -131,10 +131,10 @@ static char *get_body(ssize_t len, ssize_t *body_len) {
void wget_main(void)
{
- int sock;
+ int sock, redirects = 10;
FILE *fp;
ssize_t len, body_len;
- char *body, *result, *rc, *r_str;
+ char *body, *result, *rc, *r_str, *redir_loc = 0;
char ua[18] = "toybox wget", ver[6], hostname[1024], port[6], path[1024];
// TODO extract filename to be saved from URL
@@ -144,34 +144,49 @@ void wget_main(void)
if(!toys.optargs[0]) help_exit("no URL");
get_info(toys.optargs[0], hostname, port, path);
- sock = conn_svr(hostname, port);
- // compose HTTP request
- sprintf(toybuf, "GET %s HTTP/1.1\r\n", path);
- mk_fld("Host", hostname);
#ifdef TOYBOX_VERSION
strcat(ua, "/"), strncpy(ver, TOYBOX_VERSION, 5), strcat(ua, ver);
#endif
- mk_fld("User-Agent", ua);
- mk_fld("Connection", "close");
- strcat(toybuf, "\r\n");
+ for (;; redirects--) {
+sock = conn_svr(hostname, port);
+// compose HTTP request
+sprintf(toybuf, "GET %s HTTP/1.1\r\n", path);
+mk_fld("Host", hostname);
+mk_fld("User-Agent", ua);
+mk_fld("Connection", "close");
+strcat(toybuf, "\r\n");
+
+// send the HTTP request
+len = strlen(toybuf);
+if (write(sock, toybuf, len) != len) perror_exit("write error");
+
+// read HTTP response
+if ((len = read(sock, toybuf, 4096)) == -1) perror_exit("read error");
+if (!strstr(toybuf, "\r\n\r\n")) error_exit("too long HTTP response");
+body = get_body(len, _len);
+redir_loc = strstr(toybuf, "Location: ");
+result = strtok(toybuf, "\r");
+strtok(result, " ");
+rc = strtok(NULL, " ");
+r_str = strtok(NULL, " ");
+
+// HTTP res code check
+if (!strcmp(rc, "301") || !strcmp(rc, "302")) {
+ char* eol = 0;
+ if ((eol = strchr(redir_loc, '\r')) > 0) *eol = 0;
+ else if (redir_loc) error_exit("Could not parse redirect URL");
+ if (redirects < 0) error_exit("Too many redirects");
+
+ printf("Redirection: %s %s \n", rc, r_str);
+ printf("%s \n", redir_loc);
+ redir_loc = redir_loc+strlen("Location: ");
+ close(sock);
+ get_info(redir_loc, hostname, port, path);
+} else if (!strcmp(rc, "200")) break;
+else error_exit("res: %s(%s)", rc, r_str);
+ }
- // send the HTTP request
- len = strlen(toybuf);
- if (write(sock, toybuf, len) != len) perror_exit("write error");
-
- // read HTTP response
- if ((len = read(sock, toybuf, 4096)) == -1) perror_exit("read error");
- if (!strstr(toybuf, "\r\n\r\n")) error_exit("too long HTTP response");
- body = get_body(len, _len);
- result = strtok(toybuf, "\r");
- strtok(result, " ");
- rc = strtok(NULL, " ");
- r_str = strtok(NULL, " ");
-
- // HTTP res code check
- // TODO handle HTTP 302 Found(Redirection)
- if (strcmp(rc, "200")) error_exit("res: %s(%s)", rc, r_str);
if (!(fp = fopen(TT.filename, "w"))) perror_exit("fopen error");
if (fwrite(body, 1, body_len, fp) != body_len)
--
2.19.1
___
Toybox mailing list
Toybox@lists.landley.net
http://lists.landley.net/listinfo.cgi/toybox-landley.net
[Toybox] [PATCH] cut: re-enable crunch_str on cut -C
Since there has been lots of topic about rendering combining
characters I leave this commit here for discussion. crunch_str
currently prints combining characters right. you are free to prove me
wrong
Combining characters follow the character which they modify.
https://www.cl.cam.ac.uk/~mgk25/unicode.html#comb
xterm renders cut test1.txt -C -1 now correctly, other terminal
emulators have different results, but anyway leaving glyph incomplete
would sacrifice that you wont be later able view it inside browser for
example either.
My previous message to board about vi combining chars had some utf-8
with combining characters in it. and my ipad:s browser rendered it
correctly, but this desktop browser did not...
I dont think its good idea to leave glyph imcomplete just because some
terminal emulators misbehave and render combining chars sometimes
after the main glyph. Leaving glyph incomplete without combining
characters could leave word unreadable for user that is actually
reading his own language with his own terminal or browser that
actually works.
I will also attach 2 screenshots into mailinglist where without_crunch
is the old implementation
cut_with_crunch.png
cut_without_crunch.png
regards
-Jarno
From 94bfdef4132b0b345dbce2caa13cd5c0bd560a9d Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Jarno=20M=C3=A4kip=C3=A4=C3=A4?=
Date: Fri, 18 Oct 2019 20:21:57 +0300
Subject: [PATCH] cut: re-enable crunch_str on cut -C
Reason: unicolumns() does not print combining characters correctly
Combining characters follow the character which they modify.
https://www.cl.cam.ac.uk/~mgk25/unicode.html#comb
xterm renders cut test1.txt -C -1 now correctly
---
tests/cut.test | 3 +++
toys/posix/cut.c | 38 +++---
2 files changed, 10 insertions(+), 31 deletions(-)
diff --git a/tests/cut.test b/tests/cut.test
index e475288a..8d8c4ba1 100755
--- a/tests/cut.test
+++ b/tests/cut.test
@@ -34,6 +34,9 @@ testing "-c a,b-c,d" "cut -c 3,5-7,10 abc.txt" "etwoh\npa:ba\nequi \n" "" ""
toyonly testing "-c japan.txt" 'cut -c 3-6,9-12 "$FILES/utf8/japan.txt"' \
"ガラスをられます\n" "" ""
+toyonly testing "-C test1.txt" 'cut -C -1 "$FILES/utf8/test1.txt"' \
+ "l̴̗̞̠\n" "" ""
+
# substitute for awk
toyonly testcmd "-DF" "-DF 2,7,5" \
"said and your\nare\nis demand. supply\nforecast :\nyou you better,\n\nEm: Took hate\n" "" \
diff --git a/toys/posix/cut.c b/toys/posix/cut.c
index 9f7f7458..61b2b409 100644
--- a/toys/posix/cut.c
+++ b/toys/posix/cut.c
@@ -46,28 +46,6 @@ GLOBALS(
regex_t reg;
)
-// Return number of bytes to start of first column fitting in columns
-// invalid sequences are skipped/ignored
-int unicolumns(char *start, unsigned columns)
-{
- int i, j = 0;
- wchar_t wc;
- char *s = start, *ss = start;
-
- // Skip start, rounding down if we hit a multicolumn char
- while (jcolumns) break;
-ss = s;
- }
-}
- }
-
- return ss-start;
-}
// Apply selections to an input line, producing output
static void cut_line(char **pline, long len)
@@ -99,15 +77,13 @@ static void cut_line(char **pline, long len)
// crunch_str() currently assumes that combining characters get
// escaped, to provide an unambiguous visual representation.
// This assumes the input string is null terminated.
- //if (start) crunch_str(, start, 0, 0, 0);
- //if (!*s) continue;
- //start = s-line;
- //ss = s;
- //crunch_str(, count, 0, 0, 0);
- //count = ss-s;
-
- s += unicolumns(s, start);
- count = unicolumns(s, end-start);
+ if (start) crunch_str(, start, 0, 0, 0);
+ if (!*s) continue;
+ start = s-line;
+ ss = s;
+ crunch_str(, count, 0, 0, 0);
+ count = ss-s;
+
} else if (toys.optflags_c) {
wchar_t wc;
char *sss;
--
2.19.1
___
Toybox mailing list
Toybox@lists.landley.net
http://lists.landley.net/listinfo.cgi/toybox-landley.net
Re: [Toybox] How can I contribute
On Thu, Oct 17, 2019 at 1:27 AM Rob Landley wrote:
>
> On 10/14/19 3:07 PM, Denys Nykula wrote:
> >> saw one of the Toybox talks and wanted to see how I could contribute
> >
> > Completing landley.net/toybox/cleanup.html on toys/pending/{dhcp,route}.c
> > to help them out of pending would be most demanded I think.
>
> That's kind of a high bar, though. :)
>
> Right now toys/posix/ls.c only puts one space between filenames in -C or -x
> mode
> (which is the default output), and I have a todo item to increase that to 2
> spaces (which is what other implementations do). The reason is I misunderstood
> how unicode worked and it turns out combining characters don't come _after_
> the
> character they combine with but before, which means a filename that ends with
> a
> combining character will attach to the space after the filename, and thus make
> two filenames visually run together unless you have a two space gap. The loop
> that needs adjusting is probably the one starting around line 401, I don't
> _think_ tests/ls.test cares (since it has to pass with TEST_HOST and the host
> version generally uses different spacing anyway), but make sure that still
> passes.
If terminal emulator behaves correctly, such as UXTerm/Xterm does.
Combining chars dont render in space after, but on same area as
previous char. So 'ls' implementation even with on space only between
filenames are readable. That been said most other terminals dont
render them correctly. I tested this on xterm, uxterm, st, alacritty,
termite, terminator (I have lots of terminals installed because i
were testing CSI escapes for vi)
with terminator even the two spaces are not enough between zalgo text
filenames... :)
but long story short this problem is mostly terminal emulators
problem, but having 2 spaces between filenames brings consistency with
other implementations anyway.
-Jarno
>
> Or, I just taught xargs to ignore -P but it would be nice if somebody actually
> implemented that instead of ignoring it.
>
> Rob
> ___
> Toybox mailing list
> Toybox@lists.landley.net
> http://lists.landley.net/listinfo.cgi/toybox-landley.net
___
Toybox mailing list
Toybox@lists.landley.net
http://lists.landley.net/listinfo.cgi/toybox-landley.net
Re: [Toybox] [PATCH] vi: changes to buffer drawing
I dare to say that crunch_str calculates width right already. if i do
void hello_main(void)
{
char* str = "lll\xcc\xb4\xcc\x97\n";
char* end = str;
int width;
width = crunch_str(, 3, stdout, 0,0);
xprintf("\nwidth: %d\n",width);
end = str;
width = crunch_str(, 3, 0, 0,0);
xprintf("width: %d\n",width);
xprintf("len: %d\n",end-str);
}
I get output
jarno@Snow:~/work/src/toybox/toybox$ ./hello
lll̴̗
width: 3
width: 3
len: 7
And if i use xterm combining chars are rendered on top of last ascii
char as they are supposed to.
crunch_str loop finished only if
if (width-columns wrote:
>
> n 9/19/19 2:52 PM, Jarno Mäkipää wrote:
> > Actually I think that current crunch_str prints trailing zero width
> > combining chars just fine?
> >
> > since when width==columns its still >= 0
> >
> > .
> > for (end = start = *str; *end; columns += col, end += bytes) {
> > wchar_t wc;
> >
> > if ((bytes = utf8towc(, end, 4))>0 && (col = wcwidth(wc))>=0) {
> > if (!escmore || wc>255 || !strchr(escmore, wc)) {
> > if (width-columns > <--col is 0 when U-0x300-0x36f
> > if (out) fwrite(end, bytes, 1, out);
> >
> > continue;
> > }
> > }
> > ..
> >
>
> The problem is when you ask it how many bytes of input will fit in a given
> number of columns (or to print up to this many columns), it doesn't give the
> trailing combining characters. It stops right after the last printing
> character
> that fits.
>
> And the callers need to be adjusted to ask "how many bytes will fit into 0
> chars" so they can add combining characters to an existing column when
> characters are coming in incrementally (from some outside source you're
> reading
> in chunks, or which is delivering individual characters like serial ports),
> and
> they've filled up the space _but_ there may be more combining characters
> coming
> in in future that attach to the existing space.
>
> > And yeah UTF-8 is good because it was originally written on napkin at
> > dinner table
> > by Ken Thompson and Rob Pike. Unicode on the other hand... not written
> > in napkin.
>
> Unicode is insane. When characters come in incrementally, you have to redraw
> the
> same glyph repeatedly because you _can't_ know when you're done until you
> overshoot.
>
> Rob
___
Toybox mailing list
Toybox@lists.landley.net
http://lists.landley.net/listinfo.cgi/toybox-landley.net
[Toybox] [PATCH] vi: unalloc used memory, cleanups, fixes
add: linelist unload
fix: proper utf8 handling on insert mode backspace
fix: free allocated data at exit
cleanup: rename some variables to be describive but short
cleanup: reorganize some variables in main
---
toys/pending/vi.c | 406 --
1 file changed, 208 insertions(+), 198 deletions(-)
From 3c4ce960a3f4dbb796f6dd88418c1e4f48fe1fe7 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Jarno=20M=C3=A4kip=C3=A4=C3=A4?=
Date: Tue, 15 Oct 2019 18:25:59 +0300
Subject: [PATCH] vi: unalloc used memory, cleanups, fixes
add: linelist unload
fix: proper utf8 handling on insert mode backspace
fix: free allocated data at exit
cleanup: rename some variables to be describive but short
cleanup: reorganize some variables in main
---
toys/pending/vi.c | 406 --
1 file changed, 208 insertions(+), 198 deletions(-)
diff --git a/toys/pending/vi.c b/toys/pending/vi.c
index d39e66ee..1cf8bc7b 100644
--- a/toys/pending/vi.c
+++ b/toys/pending/vi.c
@@ -39,9 +39,9 @@ GLOBALS(
)
struct str_line {
- int alloc_len;
- int str_len;
- char *str_data;
+ int alloc;
+ int len;
+ char *data;
};
//yank buffer
struct yank_buf {
@@ -83,7 +83,7 @@ static int vi_delete(char reg, struct linelist *row, int col, int flags);
//inserted line not yet pushed to buffer
struct str_line *il;
struct linelist *text; //file loaded into buffer
-struct linelist *scr_r;//current screen coord 0 row
+struct linelist *screen;//current screen coord 0 row
struct linelist *c_r;//cursor position row
struct yank_buf yank; //single yank
@@ -119,24 +119,32 @@ struct double_list *dlist_insert(struct double_list **list, char *data)
return new;
}
-//TODO implement
-void linelist_unload()
+
+void linelist_free(void *node)
{
+ struct linelist *lst = (struct linelist *)node;
+ free(lst->line->data), free(lst->line), free(lst);
+}
+void linelist_unload()
+{
+ void* list = 0;
+ for (;text->down; text = text->down);
+ list = (void*)text;
+ text = screen = c_r = 0;
+ llist_traverse(list, linelist_free);
}
void write_file(char *filename)
{
struct linelist *lst = text;
FILE *fp = 0;
- if (!filename)
-filename = (char*)*toys.optargs;
- fp = fopen(filename, "w");
- if (!fp) return;
- while (lst) {
-fprintf(fp, "%s\n", lst->line->str_data);
-lst = lst->down;
- }
+ if (!filename) filename = (char*)*toys.optargs;
+ if (!(fp = fopen(filename, "w")) ) return;
+
+ for (;lst; lst = lst->down)
+fprintf(fp, "%s\n", lst->line->data);
+
fclose(fp);
}
@@ -153,9 +161,9 @@ int linelist_load(char *filename)
ssize_t alc = 80;
lst = (struct linelist*)dlist_add((struct double_list**),
xzalloc(sizeof(struct str_line)));
-lst->line->alloc_len = alc;
-lst->line->str_len = 0;
-lst->line->str_data = line;
+lst->line->alloc = alc;
+lst->line->len = 0;
+lst->line->data = line;
text = lst;
dlist_terminate(text->up);
return 1;
@@ -174,13 +182,13 @@ int linelist_load(char *filename)
}
lst = (struct linelist*)dlist_add((struct double_list**),
xzalloc(sizeof(struct str_line)));
-lst->line->alloc_len = alc;
-lst->line->str_len = len;
-lst->line->str_data = line;
+lst->line->alloc = alc;
+lst->line->len = len;
+lst->line->data = line;
-if (lst->line->str_data[len-1] == '\n') {
- lst->line->str_data[len-1] = 0;
- lst->line->str_len--;
+if (lst->line->data[len-1] == '\n') {
+ lst->line->data[len-1] = 0;
+ lst->line->len--;
}
if (text == 0) text = lst;
}
@@ -227,8 +235,8 @@ static int vi_x(char reg, int count0, int count1)
int col = TT.cur_col;
if (!c_r) return 0;
- start = c_r->line->str_data;
- len = c_r->line->str_len;
+ start = c_r->line->data;
+ len = c_r->line->len;
last = utf8_last(start, len);
cpos = start+TT.cur_col;
@@ -257,33 +265,33 @@ int vi_movw(int count0, int count1, char* unused)
// char *current = 0;
if (!c_r)
return 0;
- if (TT.cur_col == c_r->line->str_len-1 || !c_r->line->str_len)
+ if (TT.cur_col == c_r->line->len-1 || !c_r->line->len)
goto next_line;
- if (strchr(empties, c_r->line->str_data[TT.cur_col]))
+ if (strchr(empties, c_r->line->data[TT.cur_col]))
goto find_non_empty;
- if (strchr(specials, c_r->line->str_data[TT.cur_col])) {
-for (;strchr(specials, c_r->line->str_data[TT.cur_col]); ) {
+ if (strchr(specials, c_r->line->data[TT.cur_col])) {
+for (;strchr(specials, c_r->line->data[TT.cur_col]); ) {
TT.cur_col++;
- if (TT.cur_col == c_r->line->str_len-1)
+ if (TT.cur_col == c_r->line->len-1)
goto next_line;
}
- } else for (;!strchr(specials, c_r->line->str_data[TT.cur_col]) &&
- !strchr(empties, c_r->line->str_data[TT.cur_col]);) {
+ } else for (;!strchr(specials, c_r->line->data[TT.cur_col]) &&
+ !strchr(empties, c_r->line->data[TT.cur_col]);) {
TT.cur_col++;
- if (TT.cur_col ==
[Toybox] [PATCH] lib getdirname()
This patch relates pull request created by E5ten
https://github.com/landley/toybox/pull/145
I would write getdirname() following way so it should output similar
output as libc dirname
but return always pointer that can be freed. You might have more
elegant solution but here is my take.
return similar dirnames as libc dirname
path getdirname
/usr/lib/ /usr
/usr/lib /usr
/usr/ /
usr.
/ /
. .
.. .
---
lib/lib.c | 8 +++-
1 file changed, 7 insertions(+), 1 deletion(-)
From 2197840947813992d3b7ef5e8230150fbb2872e3 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Jarno=20M=C3=A4kip=C3=A4=C3=A4?=
Date: Mon, 14 Oct 2019 17:50:07 +0300
Subject: [PATCH] lib getdirname()
return similar dirnames as libc dirname
path getdirname
/usr/lib/ /usr
/usr/lib /usr
/usr/ /
usr.
/ /
. .
.. .
---
lib/lib.c | 8 +++-
1 file changed, 7 insertions(+), 1 deletion(-)
diff --git a/lib/lib.c b/lib/lib.c
index fe402af8..25bd3b65 100644
--- a/lib/lib.c
+++ b/lib/lib.c
@@ -1010,7 +1010,13 @@ char *getdirname(char *name)
{
char *s = xstrdup(name), *ss = strrchr(s, '/');
- while (ss && *ss && *ss == '/' && s != ss) *ss-- = 0;
+ if (!ss) free(s), xstrdup(".");
+ else {
+ss = s+strlen(s)-1;
+if (ss != s) do {
+ *ss-- = 0;
+} while(*ss != '/');
+ }
return s;
}
--
2.19.1
___
Toybox mailing list
Toybox@lists.landley.net
http://lists.landley.net/listinfo.cgi/toybox-landley.net
[Toybox] [PATCH] ls: fix seqfault on broken locale
relates to issue reported in
https://github.com/landley/toybox/issues/146
When user builds toybox CFG_TOYBOX_I18N disabled and tries to list
folder contents with multibyte characters other than UTF-8 ls might
seqfault since wcrtomb returns -1
while locale set to fi_FI.UTF-8 disable CFG_TOYBOX_I18N
touch aaőő
ls
-Jarno
---
toys/posix/ls.c | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
From 4bacf2de7264c293cf506efa310f1f34f7df4707 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Jarno=20M=C3=A4kip=C3=A4=C3=A4?=
Date: Thu, 10 Oct 2019 00:06:03 +0300
Subject: [PATCH] ls: fix seqfault on broken locale
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
When user builds toybox CFG_TOYBOX_I18N disabled and tries to list
folder contents with multibyte characters other than UTF-8 ls might
seqfault since wcrtomb returns -1
while locale set to fi_FI-UTF-8 disable CFG_TOYBOX_I18N
touch aaőő
ls
---
toys/posix/ls.c | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/toys/posix/ls.c b/toys/posix/ls.c
index 1640ea70..0956902f 100644
--- a/toys/posix/ls.c
+++ b/toys/posix/ls.c
@@ -66,14 +66,14 @@ GLOBALS(
// Callback from crunch_str to represent unprintable chars
static int crunch_qb(FILE *out, int cols, int wc)
{
- unsigned len = 1;
+ int len = 1;
char buf[32];
if (FLAG(q)) *buf = '?';
else {
if (wc<256) *buf = wc;
// scrute the inscrutable, eff the ineffable, print the unprintable
-else len = wcrtomb(buf, wc, 0);
+else if ((len = wcrtomb(buf, wc, 0) ) == -1) len = 1;
if (FLAG(b)) {
char *to = buf, *from = buf+24;
int i, j;
--
2.19.1
___
Toybox mailing list
Toybox@lists.landley.net
http://lists.landley.net/listinfo.cgi/toybox-landley.net
[Toybox] [PATCH] vi: fixes
fix: force redraw after :set (no)list
fix: force redraw after insert
fix: split on zero cursor position
fix: yank and push with utf-8 content
---
toys/pending/vi.c | 40 +++-
1 file changed, 27 insertions(+), 13 deletions(-)
From e248fabfed87155b8ed2c7122044d43f078ab51b Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Jarno=20M=C3=A4kip=C3=A4=C3=A4?=
Date: Mon, 7 Oct 2019 21:22:07 +0300
Subject: [PATCH] vi: fixes
fix: force redraw after :set (no)list
fix: force redraw after insert
fix: split on zero cursor position
fix: yank and push with utf-8 content
---
toys/pending/vi.c | 40 +++-
1 file changed, 27 insertions(+), 13 deletions(-)
diff --git a/toys/pending/vi.c b/toys/pending/vi.c
index 9334e033..d39e66ee 100644
--- a/toys/pending/vi.c
+++ b/toys/pending/vi.c
@@ -365,30 +365,41 @@ static void i_insert(char* str, int len)
strcpy([TT.cur_col+len], t);
TT.cur_col += len;
if (TT.cur_col) TT.cur_col--;
+
c_r->line->str_len += len;
free(t);
+ TT.vi_mov_flag |= 0x3000;
}
//new line at split pos;
void i_split()
{
+ int alloc = 0, len = 0, idx = 0;
struct str_line *l = xmalloc(sizeof(struct str_line));
- int l_a = c_r->line->alloc_len;
- int l_len = c_r->line->str_len-TT.cur_col-1;
- l_len = (l_len >= 0) ? l_len : 0;
- l->str_data = xzalloc(l_a);
- l->alloc_len = l_a;
- l->str_len = l_len;
- strncpy(l->str_data, _r->line->str_data[TT.cur_col+1], l_len);
- l->str_data[l_len] = 0;
- c_r->line->str_len -= l_len;
+ alloc = c_r->line->alloc_len;
+
+ if (TT.cur_col) len = c_r->line->str_len-TT.cur_col-1;
+ else len = c_r->line->str_len;
+ if (len < 0) len = 0;
+
+ l->str_data = xzalloc(alloc);
+ l->alloc_len = alloc;
+ l->str_len = len;
+ idx = c_r->line->str_len - len;
+
+ strncpy(l->str_data, _r->line->str_data[idx], len);
+ memset(>str_data[len], 0, alloc-len);
+
+ c_r->line->str_len -= len;
if (c_r->line->str_len <= 0) c_r->line->str_len = 0;
- c_r->line->str_data[c_r->line->str_len] = 0;
+
+ len = c_r->line->str_len;
+
+ memset(_r->line->str_data[len], 0, alloc-len);
c_r = (struct linelist*)dlist_insert((struct double_list**)_r, (char*)l);
c_r->line = l;
TT.cur_col = 0;
- check_cursor_bounds();
}
@@ -422,7 +433,7 @@ static int vi_push(char reg, int count0, int count1)
if (*(end-1) == '\n') for (;start != end;) {
TT.vi_mov_flag |= 0x1000;
char *next = strchr(start, '\n');
-vi_eol(1, 1, 0);
+TT.cur_col = (c_r->line->str_len) ? c_r->line->str_len-1: 0;
i_split();
if (next) {
i_insert(start, next-start);
@@ -846,10 +857,12 @@ int run_ex_cmd(char *cmd)
}
else if (strstr([1], "set list")) {
TT.list = 1;
+ TT.vi_mov_flag |= 0x3000;
return 1;
}
else if (strstr([1], "set nolist")) {
TT.list = 0;
+ TT.vi_mov_flag |= 0x3000;
return 1;
}
}
@@ -1271,7 +1284,8 @@ static void check_cursor_bounds()
return;
} else if (c_r->line->str_len-1 < TT.cur_col) TT.cur_col = c_r->line->str_len-1;
- if (utf8_width(_r->line->str_data[TT.cur_col], c_r->line->str_len-TT.cur_col) <= 0)
+ if (TT.cur_col && utf8_width(_r->line->str_data[TT.cur_col],
+c_r->line->str_len-TT.cur_col) <= 0)
TT.cur_col--, check_cursor_bounds();
}
--
2.19.1
___
Toybox mailing list
Toybox@lists.landley.net
http://lists.landley.net/listinfo.cgi/toybox-landley.net
[Toybox] [PATCH] vi: Scroll unmodified lines using escape codes
Scroll visual buffer up and down using standard escapes when content
has not changed, instead of redrawing every key press.
Side scroll now moves all the lines instead of only cursor one.
Side scrolling long lines unfortunately causes redraw to whole screen.
-Jarno
From d55db4770505e3d044681350b2f829e9e8b54fae Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Jarno=20M=C3=A4kip=C3=A4=C3=A4?=
Date: Sun, 29 Sep 2019 11:31:05 +0300
Subject: [PATCH] vi: Scroll unmodified lines using escape codes
Scroll visual buffer up and down using standard escapes when content
has not changed, instead of redrawing every keypress.
Sidescroll now moves all the lines instead of only cursor one.
Side scrolling long lines unfortunately causes redraw to whole screen.
-Jarno
---
toys/pending/vi.c | 141 --
1 file changed, 86 insertions(+), 55 deletions(-)
diff --git a/toys/pending/vi.c b/toys/pending/vi.c
index 50b8f0ab..9334e033 100644
--- a/toys/pending/vi.c
+++ b/toys/pending/vi.c
@@ -22,6 +22,9 @@ config VI
GLOBALS(
int cur_col;
int cur_row;
+int scr_row;
+int drawn_row;
+int drawn_col;
unsigned screen_height;
unsigned screen_width;
int vi_mode;
@@ -91,6 +94,8 @@ struct yank_buf yank; //single yank
// 0x1 = Command needs argument (f,F,r...)
// 0x2 = Move 1 right on yank/delete/insert (e, $...)
// 0x4 = yank/delete last line fully
+// 0x1000 = redraw after cursor needed
+// 0x2000 = full redraw needed
// 0x4000 = count0 not given
// 0x8000 = move was reverse
@@ -211,7 +216,6 @@ int vi_dd(char reg, int count0, int count1)
vi_delete(reg, pos, 0, 0);
check_cursor_bounds();
- adjust_screen_buffer();
return 1;
}
@@ -287,7 +291,6 @@ next_line:
return vi_movw(count, 1, 0);
check_cursor_bounds();
- adjust_screen_buffer();
return 1;
}
@@ -319,7 +322,6 @@ exit_function:
return vi_movb(count, 1, 0);
TT.vi_mov_flag |= 0x8000;
check_cursor_bounds();
- adjust_screen_buffer();
return 1;
}
@@ -338,7 +340,6 @@ static int vi_move(int count0, int count1, char *unused)
TT.vi_mov_flag |= 2;
check_cursor_bounds();
- adjust_screen_buffer();
return 1;
}
@@ -388,7 +389,6 @@ void i_split()
c_r->line = l;
TT.cur_col = 0;
check_cursor_bounds();
- adjust_screen_buffer();
}
@@ -415,10 +415,12 @@ static int vi_eol(int count0, int count1, char *unused)
//TODO check register where to push from
static int vi_push(char reg, int count0, int count1)
{
- char *start = yank.data;
- char *end = yank.data+strlen(yank.data);
+ char *start = yank.data, *end = yank.data+strlen(yank.data);
+ struct linelist *cursor = c_r;
+ int col = TT.cur_col;
//insert into new lines
if (*(end-1) == '\n') for (;start != end;) {
+TT.vi_mov_flag |= 0x1000;
char *next = strchr(start, '\n');
vi_eol(1, 1, 0);
i_split();
@@ -432,6 +434,7 @@ static int vi_push(char reg, int count0, int count1)
else for (;start != end;) {
char *next = strchr(start, '\n');
if (next) {
+ TT.vi_mov_flag |= 0x1000;
i_insert(start, next-start);
i_split();
start = next+1;
@@ -440,6 +443,10 @@ static int vi_push(char reg, int count0, int count1)
start = end;
}
}
+ //if row changes during push original cursor position is kept
+ //vi inconsistancy
+ if (c_r != cursor) c_r = cursor, TT.cur_col = col;
+
return 1;
}
@@ -475,7 +482,6 @@ static int vi_go(int count0, int count1, char *symbol)
TT.cur_col = 0;
check_cursor_bounds(); //adjusts cursor column
- adjust_screen_buffer(); //adjusts screen buffer
if (prev_row>TT.cur_row) TT.vi_mov_flag |= 0x8000;
return 1;
@@ -505,6 +511,7 @@ static int vi_delete(char reg, struct linelist *row, int col, int flags)
start = start->down;
full_line_delete:
+ TT.vi_mov_flag |= 0x1000;
for (;start != end;) {
struct linelist* lst = start;
//struct linelist *lst = dlist_pop();
@@ -518,6 +525,7 @@ full_line_delete:
free(lst);
}
last_line_delete:
+ TT.vi_mov_flag |= 0x1000;
if (TT.vi_mov_flag&2) col_e = start->line->str_len;
if (TT.vi_mov_flag&4) {
if (!end->down && !end->up)
@@ -813,7 +821,6 @@ static int search_str(char *s)
c_r = lst;
TT.cur_col = c-c_r->line->str_data;
check_cursor_bounds();
- adjust_screen_buffer();
return 0;
}
@@ -882,6 +889,7 @@ void vi_main(void)
tty_esc("?1049h");
tty_esc("H");
xflush(1);
+ TT.vi_mov_flag = 0x2000;
draw_page();
while(1) {
int key = scan_key(keybuf, -1);
@@ -1080,7 +1088,7 @@ int crunch_nstr(char **str, int width, int n, FILE *out, char *escmore,
static void draw_page()
{
- struct linelist *scr_buf = scr_r;
+ struct linelist *scr_buf = 0;
unsigned y = 0;
int x = 0;
@@ -1093,26 +1101,26 @@ static void draw_page()
//variables used only for cursor handling
int aw = 0, iw = 0, clip = 0, margin = 8;
- //clear screen
- tty_esc("2J");
-
Re: [Toybox] [PATCH] ln: add -T, use FLAG()
Its ok. Gmail seems to work all wonky, I receive about half of the emails sent to mailing list. And get bounced out of the list every few days. Tried to check all the spam filter settings and everything. Seriously thinking of changing email address for this list. There is still one more patch for this same issue that i did on morning in your mailbox somewhere but that can be ignored cos you fixed that already... -Jarno On Thu, Sep 26, 2019 at 7:21 PM Rob Landley wrote: > > On 9/25/19 4:31 PM, Jarno Mäkipää wrote: > > This second patch should fix the -T allow only 2 args > > > > All the previous test cases seems to pass. Should write test cases for this. > > And now I see your patch for this. :) > > (Gmail splits up emails I'm cc:'d on to my inbox or to the toybox folder, > depending on which is randomly delivered first by the mail servers. It's > refused > to send both copies (so I don't have broken threading in my archives) for over > 10 years, hundreds of people have complained to them about it, and they don't > care.) > > Rob ___ Toybox mailing list Toybox@lists.landley.net http://lists.landley.net/listinfo.cgi/toybox-landley.net
Re: [Toybox] [PATCH] ln: add -T, use FLAG()
yeah its ok. Your patch is better anyway with tests, so ignore mine. But note that there is still issue or interface difference with coreutils and busybox what Ryan Prichard mentioned. Other implementations check if there is more than 2 args with -T. -Jarno On Thu, Sep 26, 2019 at 4:46 PM Rob Landley wrote: > > On 9/25/19 3:20 PM, Jarno Mäkipää wrote: > > Added: -T > > Replaced: & flag_# with FLAG(#) > > > > There was feature request on github. Apparently there is some android > > scripts with ln -sfT > > I did that about 15 minutes before you sent this. (There was a github feature > request thingy.) > > Rob ___ Toybox mailing list Toybox@lists.landley.net http://lists.landley.net/listinfo.cgi/toybox-landley.net
[Toybox] [PATCH] hexedit: fix scrolling on legacy terminals
Thanks for James McMechan for pointing this out.
Using esc[1L and esc[1M escapes with cursor jump to
1, 1 to make scrolling effect instead of S and T
fixes scrolling inside Linux terminal and tmux
-Jarno
From 74c1f5e12f1c73a2176a818370e9b9b7a0a2a6dc Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Jarno=20M=C3=A4kip=C3=A4=C3=A4?=
Date: Thu, 26 Sep 2019 09:22:39 +0300
Subject: [PATCH] hexedit: fix scrolling on legacy terminals
Thanks for James McMechan for pointing this out.
Using esc[1L and esc[1M escapes with cursor jump to
1, 1 to make scrolling effect instead of S and T
fixes scrolling inside Linux terminal and tmux
-Jarno
---
toys/other/hexedit.c | 5 +++--
1 file changed, 3 insertions(+), 2 deletions(-)
diff --git a/toys/other/hexedit.c b/toys/other/hexedit.c
index 809340a1..4b628463 100644
--- a/toys/other/hexedit.c
+++ b/toys/other/hexedit.c
@@ -158,8 +158,8 @@ void hexedit_main(void)
} else {
TT.base--;
i++;
-tty_esc("1T");
tty_jump(0, 0);
+tty_esc("1L");
draw_line(0);
}
}
@@ -170,7 +170,8 @@ void hexedit_main(void)
} else {
TT.base++;
i++;
-tty_esc("1S");
+tty_jump(0, 0);
+tty_esc("1M");
tty_jump(0, TT.height-1);
draw_line(TT.height-1);
}
--
2.19.1
___
Toybox mailing list
Toybox@lists.landley.net
http://lists.landley.net/listinfo.cgi/toybox-landley.net
Re: [Toybox] Fwd: hexedit uses VT-420 scroll ctrl sequences which dont work on tty1
L and M escapes seems to work nicely. I will send patch to Rob for hexedit and continue working on vi thank you -Jarno On Thu, Sep 26, 2019 at 7:42 AM James McMechan wrote: > > The console_codes page does not seem to include SD/SU \e[1S/\e[1T. > > I however have used IL/DL \e[1L/\e[1M which insert/remove lines at the cursor. > e.g. \e[1;1H\e[5L puts 5 blank lines at the top of screen scrolling > everything down and \e[1;1H\e[2M deletes the top two lines scrolling > everything up. > In a quick test with "echo -e" seems to work on framebuffer console, screen > and xterm > > Personally, I use screen instead of either tmux or minicom > I use it for the multiple windows and less insane serial port handling. > With minicom I spent a bit of each setup fixing it so it stopped trying so > hard to help me and would just connect me to the serial port... > If I needed some weird setting with screen I would just use stty > > One reason for screen etc. is that even now we have embedded boards that > can't seem to have their serial ports work correctly above 9600 and on those > occasions I need to run a serial port remotely via modem it has been getting > slower. Even high speed modems tend to max out around 24Kbps which seems to > be dropping recently, I suspect the switching to packet switched digital > backhaul with modern codecs breaks the assumptions modems were built with so > they run slower. > > Jim McMechan > ___ > Toybox mailing list > Toybox@lists.landley.net > http://lists.landley.net/listinfo.cgi/toybox-landley.net ___ Toybox mailing list Toybox@lists.landley.net http://lists.landley.net/listinfo.cgi/toybox-landley.net
[Toybox] [PATCH] ln: -T should allow only 2 arguments
To follow similar behavior than coreutils and busybox with this
GNU extension -T should only allow maximum of 2 arguments even
when accompanied with -f
touch a b
ln -sfT a b L
I had this fixed in my other patch on list already but Rob added his
own commit with
-T so this is on top of current master.
-Jarno
From 3c60f334159296089a1332aa5cf49f52594c1695 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Jarno=20M=C3=A4kip=C3=A4=C3=A4?=
Date: Thu, 26 Sep 2019 08:47:40 +0300
Subject: [PATCH] ln: -T should allow only 2 arguments
To follow similar behavior than coreutils and busybox with this
GNU extension -T should only allow maximum of 2 arguments even
when accompanied with -f
touch a b
ln -sfT a b L
-Jarno
---
toys/posix/ln.c | 5 -
1 file changed, 4 insertions(+), 1 deletion(-)
diff --git a/toys/posix/ln.c b/toys/posix/ln.c
index fb47e911..74b26f68 100644
--- a/toys/posix/ln.c
+++ b/toys/posix/ln.c
@@ -11,6 +11,7 @@ config LN
default y
help
usage: ln [-sfnv] [FROM...] TO
+ or ln [-sfnv] -T FROM TO
Create a link between FROM and TO.
One/two/many arguments work like "mv" or "cp".
@@ -41,7 +42,9 @@ void ln_main(void)
if (!((FLAG(n)||FLAG(T)) ? lstat : stat)(dest, )) {
i = S_ISDIR(buf.st_mode);
-if ((FLAG(T) && i) || (!i && toys.optc>1))
+if (FLAG(T) && toys.optc>1)
+ error_exit("-T max 2 args");
+else if ((FLAG(T) && i) || (!i && toys.optc>1))
error_exit("'%s' %s a directory", dest, i ? "is" : "not");
} else buf.st_mode = 0;
--
2.19.1
___
Toybox mailing list
Toybox@lists.landley.net
http://lists.landley.net/listinfo.cgi/toybox-landley.net
Re: [Toybox] [PATCH] ln: add -T, use FLAG()
This second patch should fix the -T allow only 2 args
All the previous test cases seems to pass. Should write test cases for this.
-Jarno
On Wed, Sep 25, 2019 at 11:40 PM Ryan Prichard wrote:
>
> It looks like coreutils and busybox allow only 2 args with -T. Do we need to
> diagnose this?
>
> touch A B
> ln -sfT A B L
> ln: extra operand 'L'
>
> busybox ln -sfT A B L
> ln: -T accepts 2 args max
>
> toybox ln -sfT A B L
> ls -l L
> lrwxrwxrwx 1 rprichard primarygroup 1 Sep 25 13:38 L -> B
>
> With this patch, ln is overwriting the symlink twice, leaving it pointing at
> B.
>
> -Ryan
>
>
> On Wed, Sep 25, 2019 at 1:21 PM Jarno Mäkipää wrote:
>>
>> Added: -T
>> Replaced: & flag_# with FLAG(#)
>>
>> There was feature request on github. Apparently there is some android
>> scripts with ln -sfT
>>
>>
>> -Jarno
>> ___
>> Toybox mailing list
>> Toybox@lists.landley.net
>> http://lists.landley.net/listinfo.cgi/toybox-landley.net
From 049f57fc0b1d10327e773e95e3402e2b2e2d3a00 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Jarno=20M=C3=A4kip=C3=A4=C3=A4?=
Date: Thu, 26 Sep 2019 00:10:36 +0300
Subject: [PATCH 2/2] ln: fix -T to accept only 2 args
---
toys/posix/ln.c | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/toys/posix/ln.c b/toys/posix/ln.c
index 7d5ade99..d636d296 100644
--- a/toys/posix/ln.c
+++ b/toys/posix/ln.c
@@ -39,7 +39,8 @@ void ln_main(void)
// Is destination a directory?
if (FLAG(T)) {
lstat(dest, );
-if (S_ISDIR(buf.st_mode)) error_exit("'%s' is a directory", dest);
+if (toys.optc >= 2) error_exit("-T allows only 2 args");
+else if (S_ISDIR(buf.st_mode)) error_exit("'%s' is a directory", dest);
} else if (((FLAG(n)) ? lstat : stat)(dest, )
|| !S_ISDIR(buf.st_mode))
--
2.19.1
___
Toybox mailing list
Toybox@lists.landley.net
http://lists.landley.net/listinfo.cgi/toybox-landley.net
[Toybox] [PATCH] ln: add -T, use FLAG()
Added: -T
Replaced: & flag_# with FLAG(#)
There was feature request on github. Apparently there is some android
scripts with ln -sfT
-Jarno
From ed03f8cdeb5560b9ac7dd40d1ce1ef061b47ab77 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Jarno=20M=C3=A4kip=C3=A4=C3=A4?=
Date: Wed, 25 Sep 2019 23:08:43 +0300
Subject: [PATCH] ln: add -T, use FLAG()
Added: -T
Replaced: & flag_# with FLAG(#)
There was feature request on github. Apparently there is some android
scripts with ln -sfT
---
toys/posix/ln.c | 22 +-
1 file changed, 13 insertions(+), 9 deletions(-)
diff --git a/toys/posix/ln.c b/toys/posix/ln.c
index 06700dd0..7d5ade99 100644
--- a/toys/posix/ln.c
+++ b/toys/posix/ln.c
@@ -4,13 +4,13 @@
*
* See http://opengroup.org/onlinepubs/9699919799/utilities/ln.html
-USE_LN(NEWTOY(ln, "<1vnfs", TOYFLAG_BIN))
+USE_LN(NEWTOY(ln, "<1vnfsT", TOYFLAG_BIN))
config LN
bool "ln"
default y
help
-usage: ln [-sfnv] [FROM...] TO
+usage: ln [-sfnvT] [FROM...] TO
Create a link between FROM and TO.
With only one argument, create link in current directory.
@@ -19,6 +19,7 @@ config LN
-f Force the creation of the link, even if TO already exists
-n Symlink at destination treated as file
-v Verbose
+-T No target directory
*/
#define FOR_ln
@@ -35,9 +36,12 @@ void ln_main(void)
toys.optc++;
dest=".";
}
-
// Is destination a directory?
- if (((toys.optflags_n) ? lstat : stat)(dest, )
+ if (FLAG(T)) {
+lstat(dest, );
+if (S_ISDIR(buf.st_mode)) error_exit("'%s' is a directory", dest);
+
+ } else if (((FLAG(n)) ? lstat : stat)(dest, )
|| !S_ISDIR(buf.st_mode))
{
if (toys.optc>1) error_exit("'%s' not a directory", dest);
@@ -55,7 +59,7 @@ void ln_main(void)
// a temp version and renaming it over the old one, so we can retain the
// old file in cases we can't replace it (such as hardlink between mounts).
oldnew = new;
-if (toys.optflags & FLAG_f) {
+if (FLAG(f)) {
new = xmprintf("%s_XX", new);
rc = mkstemp(new);
if (rc >= 0) {
@@ -64,8 +68,8 @@ void ln_main(void)
}
}
-rc = (toys.optflags & FLAG_s) ? symlink(try, new) : link(try, new);
-if (toys.optflags & FLAG_f) {
+rc = (FLAG(s)) ? symlink(try, new) : link(try, new);
+if (FLAG(f)) {
if (!rc) {
int temp;
@@ -79,9 +83,9 @@ void ln_main(void)
}
if (rc)
perror_msg("cannot create %s link from '%s' to '%s'",
-(toys.optflags & FLAG_s) ? "symbolic" : "hard", try, new);
+(FLAG(s)) ? "symbolic" : "hard", try, new);
else
- if (toys.optflags & FLAG_v) fprintf(stderr, "'%s' -> '%s'\n", new, try);
+ if (FLAG(v)) fprintf(stderr, "'%s' -> '%s'\n", new, try);
if (new != dest) free(new);
}
--
2.19.1
___
Toybox mailing list
Toybox@lists.landley.net
http://lists.landley.net/listinfo.cgi/toybox-landley.net
Re: [Toybox] Fwd: hexedit uses VT-420 scroll ctrl sequences which dont work on tty1
On Wed, Sep 25, 2019 at 8:23 PM Rob Landley wrote: > > On 9/24/19 1:18 AM, Jarno Mäkipää wrote: > > Hi > > > > I now tested to run hexedit in tmux: downscroll works but upscroll does > > not... > > Well we might say its tmux fault, but lots of people use tmux nowadays. > > And this behavior seemed to be same in framebuffer console and xterms... > > I have a todo item to write a screen for toybox. I'll make sure this works > there. If you want to submit a bug report to the old one, have at. > > No, I am not redrawing the screen every time you scroll up. If hexedit isn't > usable in some broken environments, I'm ok with that. > > Rob Well my original intention was to ask is it ok to use S and T escapes in vi Running hexedit inside gnu screen seems to fix scrolling everywhere I tested, and ironically tmux sets up $TERM=screen and breaks functionality even inside xterm. Perhaps I try to figure is there any other way to scroll up without drawing whole page every line moved... -Jarno ___ Toybox mailing list Toybox@lists.landley.net http://lists.landley.net/listinfo.cgi/toybox-landley.net
[Toybox] Fwd: hexedit uses VT-420 scroll ctrl sequences which dont work on tty1
Hi I now tested to run hexedit in tmux: downscroll works but upscroll does not... Well we might say its tmux fault, but lots of people use tmux nowadays. And this behavior seemed to be same in framebuffer console and xterms... On Tue, Sep 24, 2019 at 4:42 AM Rob Landley wrote: > > > > On 9/23/19 4:02 PM, Jarno Mäkipää wrote: > > Hey > > > > I was looking at how should I limit unchanged lines being rewritten > > every keypress in vi editor. > > So I look reference in hexedit and noticed that it does not work > > properly on virtual terminals > > (one that open with ctrl+alt+f1...) > > > > hexedit seems to use esc[1S and esc[1T for scrolling and found them in here > > > > https://invisible-island.net/xterm/ctlseqs/ctlseqs.html > > http://man7.org/linux/man-pages/man4/console_codes.4.html > > > CSI Ps S Scroll up Ps lines (default = 1) (SU), VT420, ECMA-48. > > CSI Ps T Scroll down Ps lines (default = 1) (SD), VT420. > > > > While most of the time its not really issue, since people run X or > > have ssh/telnet or serial connection from other device with proper > > terminal emulator. But in those rare case when you have keyboard and > > framebuffer console and just messed up that one byte in your file it > > might come handy to be able fix it. > > > > Is there any easy way checking if terminal currently running supports > > these? > > In theory there's the "TERM" environment variable, in practice it's > essentially > useless. I was looking that most of the terminals set up TERM to be some string that contains xterm* in it. And linux framebuffer console seems to be TERM=linux. But then there is exceptions such as stterm that reports stterm-256color so just searching if TERM contains xterm will not always work. > > > Or should I just ignore and use them anyway, perhaps have :set > > something to disable them? > > I'll see if I can send a patch to Linux to fix the text console. It worked on > the actual VGA system hardware I tested it on, but my new laptop has a frame > buffer for the text console and it doesn't work there. > > Rob If you get your patch accepted now, it might take few years until some hardware vendors have updated their default kernel on their development board and some unlucky people might run into issues with this when running latest toybox without updating kernel. And sometimes they cant update kernel because that hardware vendor provides crummy binary driver for some interface that they cant rebuild. But anyway its simple feature and would be nice that Linux has it. -Jarno ___ Toybox mailing list Toybox@lists.landley.net http://lists.landley.net/listinfo.cgi/toybox-landley.net
[Toybox] hexedit uses VT-420 scroll ctrl sequences which dont work on tty1
Hey I was looking at how should I limit unchanged lines being rewritten every keypress in vi editor. So I look reference in hexedit and noticed that it does not work properly on virtual terminals (one that open with ctrl+alt+f1...) hexedit seems to use esc[1S and esc[1T for scrolling and found them in here https://invisible-island.net/xterm/ctlseqs/ctlseqs.html CSI Ps S Scroll up Ps lines (default = 1) (SU), VT420, ECMA-48. CSI Ps T Scroll down Ps lines (default = 1) (SD), VT420. While most of the time its not really issue, since people run X or have ssh/telnet or serial connection from other device with proper terminal emulator. But in those rare case when you have keyboard and framebuffer console and just messed up that one byte in your file it might come handy to be able fix it. Is there any easy way checking if terminal currently running supports these? Or should I just ignore and use them anyway, perhaps have :set something to disable them? br -Jarno ___ Toybox mailing list Toybox@lists.landley.net http://lists.landley.net/listinfo.cgi/toybox-landley.net
[Toybox] [PATCH] vi: Added yank
Added: yank and push
Rewrote: delete operations
Minor cleanups:
Rewrote delete operations to use one delete function instead of having
separate behavior here and there.
Now delete and yank both always move cursor and then clip the whole
cursor area into yank register. For example x is just ld or jd depeding
are we right edge or not, and dd is jd with some special flags etc.
Now only default yank register is implemented, but implemeting
yank register list should be trivial since cmd execution already passes
register char.
-Jarno
---
toys/pending/vi.c | 472 ++
1 file changed, 271 insertions(+), 201 deletions(-)
From a81acdda2f13f78ea13d9d65155585c6611cfa0e Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Jarno=20M=C3=A4kip=C3=A4=C3=A4?=
Date: Sat, 21 Sep 2019 22:20:36 +0300
Subject: [PATCH] vi: Added yank
Added: yank and push
Rewrote: delete operations
Minor cleanups:
Rewrote delete operations to use one delete function instead of having
separate behavior here and there.
Now delete and yank both always move cursor and then clip the whole
cursor area into yank register. For example x is just ld or jd depeding
are we right edge or not, and dd is jd with some special flags etc.
Now only default yank register is implemented, but implemeting
yank register list should be trivial since cmd execution already passes
register char.
---
toys/pending/vi.c | 472 ++
1 file changed, 271 insertions(+), 201 deletions(-)
diff --git a/toys/pending/vi.c b/toys/pending/vi.c
index 80be7e80..50b8f0ab 100644
--- a/toys/pending/vi.c
+++ b/toys/pending/vi.c
@@ -35,31 +35,29 @@ GLOBALS(
int list;
)
-/*
- *
- * TODO:
- * BUGS: screen pos adjust does not cover "widelines"
- *
- *
- * REFACTOR: use dllist functions where possible.
- *draw_page dont draw full page at time if nothing changed...
- *ex callbacks
- *
- * FEATURE: ex: / ? % //atleast easy cases
- *ex: r
- *ex: !external programs
- *ex: w filename //only writes to same file now
- *big file support?
- */
-
-
-struct linestack_show {
- struct linestack_show *next;
- long top, left;
- int x, width, y, height;
+struct str_line {
+ int alloc_len;
+ int str_len;
+ char *str_data;
+};
+//yank buffer
+struct yank_buf {
+ char reg;
+ int alloc;
+ char* data;
+};
+
+
+//lib dllist uses next and prev kinda opposite what im used to so I just
+//renamed both ends to up and down
+struct linelist {
+ struct linelist *up;//next
+ struct linelist *down;//prev
+ struct str_line *line;
};
static void draw_page();
+
//utf8 support
static int utf8_lnw(int* width, char* str, int bytes);
static int utf8_dec(char key, char *utf8_scratch, int *sta_p);
@@ -76,25 +74,25 @@ static void check_cursor_bounds();
static void adjust_screen_buffer();
static int search_str(char *s);
-struct str_line {
- int alloc_len;
- int str_len;
- char *str_data;
-};
+static int vi_yank(char reg, struct linelist *row, int col, int flags);
+static int vi_delete(char reg, struct linelist *row, int col, int flags);
-//lib dllist uses next and prev kinda opposite what im used to so I just
-//renamed both ends to up and down
-struct linelist {
- struct linelist *up;//next
- struct linelist *down;//prev
- struct str_line *line;
-};
//inserted line not yet pushed to buffer
struct str_line *il;
struct linelist *text; //file loaded into buffer
struct linelist *scr_r;//current screen coord 0 row
struct linelist *c_r;//cursor position row
+struct yank_buf yank; //single yank
+
+// TT.vi_mov_flag is used for special cases when certain move
+// acts differently depending is there DELETE/YANK or NOP
+// Also commands such as G does not default to count0=1
+// 0x1 = Command needs argument (f,F,r...)
+// 0x2 = Move 1 right on yank/delete/insert (e, $...)
+// 0x4 = yank/delete last line fully
+// 0x4000 = count0 not given
+// 0x8000 = move was reverse
void dlist_insert_nomalloc(struct double_list **list, struct double_list *new)
{
@@ -179,14 +177,11 @@ int linelist_load(char *filename)
lst->line->str_data[len-1] = 0;
lst->line->str_len--;
}
-if (text == 0) {
- text = lst;
-}
-
- }
- if (text) {
-dlist_terminate(text->up);
+if (text == 0) text = lst;
}
+
+ if (text) dlist_terminate(text->up);
+
fclose(fp);
return 1;
@@ -194,116 +189,58 @@ int linelist_load(char *filename)
int vi_yy(char reg, int count0, int count1)
{
+ struct linelist *pos = c_r;
+ int col = TT.cur_col;
+ TT.cur_col = 0;
+ TT.vi_mov_flag |= 0x4;
+
+ if (count0>1) cur_down(count0-1, 1, 0);
+
+ vi_yank(reg, pos, 0, 0);
+
+ TT.cur_col = col, c_r = pos;
return 1;
}
-//TODO this is overly complicated refactor with lib dllist
int vi_dd(char reg, int count0, int count1)
{
- int count = count0*count1;
- struct linelist *lst = c_r;
- if (c_r == text && text == scr_r) {
-if
Re: [Toybox] [PATCH] vi: changes to buffer drawing
Actually I think that current crunch_str prints trailing zero width
combining chars just fine?
since when width==columns its still >= 0
.
for (end = start = *str; *end; columns += col, end += bytes) {
wchar_t wc;
if ((bytes = utf8towc(, end, 4))>0 && (col = wcwidth(wc))>=0) {
if (!escmore || wc>255 || !strchr(escmore, wc)) {
if (width-columns wrote:
>
> Yeah combining chars follow up the main glyph. My draw_str_until had
> extra for loop to just to check if there is 0 width chars after we are
> at correct width and I only pushed data to stdout when I was sure
> about length.
>
> But interface in crunch_str is better, since it has support for
> rendering special chars with custom function.
>
> Now there is bit incorrect rendering when stepping around
> tests/files/test1.txt so I need to patch this up. Perhaps I try to
> make crunch_nstr() work correctly...
>
> -Jarno
>
> On Thu, Sep 19, 2019 at 5:34 PM Rob Landley wrote:
> >
> > On 9/15/19 8:05 AM, Jarno Mäkipää wrote:
> > > Replaced: draw_str_until with lib/crunch_str() where possible
> > >
> > > Removed: Unused char draw functions.
> > >
> > > Implemented: crunch_nstr() which is crunch_str with additional check
> > > for byte length, this can be used to draw substrings or non null
> > > terminated strings. (This can be moved to lib/ if its useful for others)
> >
> > Applied, but I note when I wrote crunch_str() I assumed that unicode was
> > sane,
> > which was wrong.
> >
> > UTF-8 is very well done. Unicode combining characters are as stupid as it's
> > possible to be: they TRAIL the printing character, meaning that you have a
> > base
> > character that gets displayed, and then you redraw over it repeatedly as
> > you get
> > each new modifier attaching to the _previous_ character you already drew,
> > and
> > then you can't tell you've gone past your length allocation until you parse
> > the
> > first character you _can't_ display in that space, which you then need to
> > unget.
> >
> > I thought combining characters were stored up and then applied to the _next_
> > character (which would have been the sane thing to do), and the measuring
> > logic
> > works based on that assumption. So it probably won't display combining
> > characters on the last UTF8 character because the unicode committe is too
> > dumb
> > to live.
> >
> > Rob
___
Toybox mailing list
Toybox@lists.landley.net
http://lists.landley.net/listinfo.cgi/toybox-landley.net
Re: [Toybox] [PATCH] vi: changes to buffer drawing
Yeah combining chars follow up the main glyph. My draw_str_until had extra for loop to just to check if there is 0 width chars after we are at correct width and I only pushed data to stdout when I was sure about length. But interface in crunch_str is better, since it has support for rendering special chars with custom function. Now there is bit incorrect rendering when stepping around tests/files/test1.txt so I need to patch this up. Perhaps I try to make crunch_nstr() work correctly... -Jarno On Thu, Sep 19, 2019 at 5:34 PM Rob Landley wrote: > > On 9/15/19 8:05 AM, Jarno Mäkipää wrote: > > Replaced: draw_str_until with lib/crunch_str() where possible > > > > Removed: Unused char draw functions. > > > > Implemented: crunch_nstr() which is crunch_str with additional check > > for byte length, this can be used to draw substrings or non null > > terminated strings. (This can be moved to lib/ if its useful for others) > > Applied, but I note when I wrote crunch_str() I assumed that unicode was sane, > which was wrong. > > UTF-8 is very well done. Unicode combining characters are as stupid as it's > possible to be: they TRAIL the printing character, meaning that you have a > base > character that gets displayed, and then you redraw over it repeatedly as you > get > each new modifier attaching to the _previous_ character you already drew, and > then you can't tell you've gone past your length allocation until you parse > the > first character you _can't_ display in that space, which you then need to > unget. > > I thought combining characters were stored up and then applied to the _next_ > character (which would have been the sane thing to do), and the measuring > logic > works based on that assumption. So it probably won't display combining > characters on the last UTF8 character because the unicode committe is too dumb > to live. > > Rob ___ Toybox mailing list Toybox@lists.landley.net http://lists.landley.net/listinfo.cgi/toybox-landley.net
[Toybox] [PATCH] vi: changes to buffer drawing
Replaced: draw_str_until with lib/crunch_str() where possible
Removed: Unused char draw functions.
Implemented: crunch_nstr() which is crunch_str with additional check
for byte length, this can be used to draw substrings or non null
terminated strings. (This can be moved to lib/ if its useful for others)
Reimplemented: Buffer drawing without line wrapping. Now too long lines are
drawn with @ in end. And cursor line scrolls left and right when hitting
right margin point. This will simplify buffer handling alot.
Linewrapping can be reimplemented later if needed but will add
complexity
Implemented: set list and set nolist ex commands, set list will show
escape codes such as tabs
Fix: Bug on splitting 2 lines, split was 1 byte off.
-Jarno
---
toys/pending/vi.c | 314 ++
1 file changed, 153 insertions(+), 161 deletions(-)
From c794074ada7cb057ebba54f54466d96099fa452f Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Jarno=20M=C3=A4kip=C3=A4=C3=A4?=
Date: Fri, 13 Sep 2019 20:14:32 +0300
Subject: [PATCH] vi: changes to buffer drawing
Replaced: draw_str_until with lib/crunch_str() where possible
Removed: Unused char draw functions.
Implemented: crunch_nstr() which is crunch_str with additional check
for byte length, this can be used to draw substrings or non null
terminated strings. (This can be moved to lib/ if its useful for others)
Reimplemented: Buffer drawing without line wrapping. Now too long lines are
drawn with @ in end. And cursor line scrolls left and right when hitting
right margin point. This will simplify buffer handling alot.
Linewrapping can be reimplemented later if needed but will add
complexity
Implemented: set list and set nolist ex commands, set list will show
escape codes such as tabs
Fix: Bug on splitting 2 lines, split was 1 byte off.
---
toys/pending/vi.c | 314 ++
1 file changed, 153 insertions(+), 161 deletions(-)
diff --git a/toys/pending/vi.c b/toys/pending/vi.c
index b861a667..80be7e80 100644
--- a/toys/pending/vi.c
+++ b/toys/pending/vi.c
@@ -32,6 +32,7 @@ GLOBALS(
char vi_reg;
char *last_search;
int tabstop;
+int list;
)
/*
@@ -59,14 +60,11 @@ struct linestack_show {
};
static void draw_page();
-static int draw_str_until(int *drawn, char *str, int width, int bytes);
-static void draw_char(char c, int x, int y, int highlight);
//utf8 support
static int utf8_lnw(int* width, char* str, int bytes);
static int utf8_dec(char key, char *utf8_scratch, int *sta_p);
static int utf8_len(char *str);
static int utf8_width(char *str, int bytes);
-static int draw_rune(char *c, int x, int y, int highlight);
static char* utf8_last(char* str, int size);
@@ -437,13 +435,15 @@ void i_split()
{
struct str_line *l = xmalloc(sizeof(struct str_line));
int l_a = c_r->line->alloc_len;
- int l_len = c_r->line->str_len-TT.cur_col;
+ int l_len = c_r->line->str_len-TT.cur_col-1;
+ l_len = (l_len >= 0) ? l_len : 0;
l->str_data = xzalloc(l_a);
l->alloc_len = l_a;
l->str_len = l_len;
- strncpy(l->str_data, _r->line->str_data[TT.cur_col], l_len);
+ strncpy(l->str_data, _r->line->str_data[TT.cur_col+1], l_len);
l->str_data[l_len] = 0;
c_r->line->str_len -= l_len;
+ if (c_r->line->str_len <= 0) c_r->line->str_len = 0;
c_r->line->str_data[c_r->line->str_len] = 0;
c_r = (struct linelist*)dlist_insert((struct double_list**)_r, (char*)l);
c_r->line = l;
@@ -517,7 +517,7 @@ static int vi_delete(char reg, struct linelist *row, int col, int flags)
int distance = col - TT.cur_col;
if (distance > 0) vi_x(reg, distance, 1);
}
-if (TT.vi_mov_flag&2)
+if (TT.vi_mov_flag&2)
vi_x(reg, 1, 1);
}
return 1;
@@ -765,6 +765,14 @@ int run_ex_cmd(char *cmd)
write_file(0);
return 1;
}
+else if (strstr([1], "set list")) {
+ TT.list = 1;
+ return 1;
+}
+else if (strstr([1], "set nolist")) {
+ TT.list = 0;
+ return 1;
+}
}
return 0;
@@ -804,6 +812,8 @@ void vi_main(void)
while(1) {
int key = scan_key(keybuf, -1);
+ terminal_size(_width, _height);
+ TT.screen_height -= 2; //TODO this is hack fix visual alignment
// TODO: support cursor keys in ex mode too.
if (TT.vi_mode && key>=256) {
key -= 256;
@@ -942,6 +952,58 @@ cleanup_vi:
tty_esc("?1049l");
}
+int vi_crunch(FILE* out, int cols, int wc)
+{
+ int ret = 0;
+ if (wc < 32 && TT.list) {
+tty_esc("1m");
+ret = crunch_escape(out,cols,wc);
+tty_esc("m");
+ } else if (wc == 0x09) {
+if (out) {
+ int i = TT.tabstop;
+ for (;i--;) fputs(" ", out);
+}
+ret = TT.tabstop;
+ }
+ return ret;
+}
+
+//crunch_str with n bytes restriction for printing substrings or
+//non null terminated strings
+int crunch_nstr(char **str, int width, int n, FILE *out, char *escmore,
+ int (*escout)(FILE *out, int cols, int wc))
+{
+ int columns = 0, col, bytes;
+ char *start,
[Toybox] [PATCH] wget: Added support for HTTP 301 and 302 redirects
Replaced: -f file with -O file to behave more like GNU wget
Added: parsing of redirect responses, will now follow redirectation path
until maximum of 10 jumps. This will not help much because most
likely this will lead to server advertising new unsupported HTTPS url.
-Jarno
---
toys/pending/wget.c | 81 +++--
1 file changed, 48 insertions(+), 33 deletions(-)
From 94f01ac9df401ba25be399d8bd4fcebd2a6826e9 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Jarno=20M=C3=A4kip=C3=A4=C3=A4?=
Date: Sat, 14 Sep 2019 13:15:04 +0300
Subject: [PATCH] wget: Added support for HTTP 301 and 302 redirects
Replaced: -f file with -O file to behave more like GNU wget
Added: parsing of redirect responses, will now follow redirectation path
until maximum of 10 jumps. This will not help much because most
likely this will lead to server advertising new unsupported HTTPS url.
---
toys/pending/wget.c | 81 +++--
1 file changed, 48 insertions(+), 33 deletions(-)
diff --git a/toys/pending/wget.c b/toys/pending/wget.c
index ced51d9c..7feda8e8 100644
--- a/toys/pending/wget.c
+++ b/toys/pending/wget.c
@@ -3,19 +3,19 @@
* Copyright 2016 Lipi C.H. Lee
*
-USE_WGET(NEWTOY(wget, "f:", TOYFLAG_USR|TOYFLAG_BIN))
+USE_WGET(NEWTOY(wget, "O:", TOYFLAG_USR|TOYFLAG_BIN))
config WGET
bool "wget"
default n
help
-usage: wget -f filename URL
--f filename: specify the filename to be saved
+usage: wget -O filename URL
+-O filename: specify the filename to be saved
URL: HTTP uniform resource location and only HTTP, not HTTPS
examples:
- wget -f index.html http://www.example.com
- wget -f sample.jpg http://www.example.com:8080/sample.jpg
+ wget -O index.html http://www.example.com
+ wget -O sample.jpg http://www.example.com:8080/sample.jpg
*/
#define FOR_wget
@@ -96,7 +96,7 @@ static int conn_svr(const char *hostname, const char *port) {
continue;
}
if (connect(sock, rp->ai_addr, rp->ai_addrlen) != -1)
- break; // succeed in connecting to any server IP
+ break; // succeed in connecting to any server IP
else perror_msg("connect error");
close(sock);
}
@@ -132,41 +132,56 @@ void wget_main(void)
ssize_t len, body_len;
char *body, *result, *rc, *r_str;
char ua[18] = "toybox wget/", ver[6], hostname[1024], port[6], path[1024];
-
+ int redirects = 10;
+ char* redir_loc = 0;
// TODO extract filename to be saved from URL
- if (!(toys.optflags & FLAG_f)) help_exit("no filename");
+ if (!(toys.optflags & FLAG_O)) help_exit("no filename");
if (fopen(TT.filename, "r")) error_exit("'%s' already exists", TT.filename);
if(!toys.optargs[0]) help_exit("no URL");
get_info(toys.optargs[0], hostname, port, path);
- sock = conn_svr(hostname, port);
-
- // compose HTTP request
- sprintf(toybuf, "GET %s HTTP/1.1\r\n", path);
- mk_fld("Host", hostname);
strncpy(ver, TOYBOX_VERSION, 5);
strcat(ua, ver);
- mk_fld("User-Agent", ua);
- mk_fld("Connection", "close");
- strcat(toybuf, "\r\n");
-
- // send the HTTP request
- len = strlen(toybuf);
- if (write(sock, toybuf, len) != len) perror_exit("write error");
-
- // read HTTP response
- if ((len = read(sock, toybuf, 4096)) == -1) perror_exit("read error");
- if (!strstr(toybuf, "\r\n\r\n")) error_exit("too long HTTP response");
- body = get_body(len, _len);
- result = strtok(toybuf, "\r");
- strtok(result, " ");
- rc = strtok(NULL, " ");
- r_str = strtok(NULL, " ");
-
- // HTTP res code check
- // TODO handle HTTP 302 Found(Redirection)
- if (strcmp(rc, "200")) error_exit("res: %s(%s)", rc, r_str);
+ for(;;redirects--) {
+sock = conn_svr(hostname, port);
+
+// compose HTTP request
+sprintf(toybuf, "GET %s HTTP/1.1\r\n", path);
+mk_fld("Host", hostname);
+mk_fld("User-Agent", ua);
+mk_fld("Connection", "close");
+strcat(toybuf, "\r\n");
+
+// send the HTTP request
+len = strlen(toybuf);
+if (write(sock, toybuf, len) != len) perror_exit("write error");
+
+// read HTTP response
+if ((len = read(sock, toybuf, 4096)) == -1) perror_exit("read error");
+if (!strstr(toybuf, "\r\n\r\n")) error_exit("too long HTTP response");
+body = get_body(len, _len);
+redir_loc = strstr(toybuf, "Location: ");
+result = strtok(toybuf, "\r");
+strtok(result, " ");
+rc = strtok(NULL, " ");
+r_str = strtok(NULL, " ");
+
+// HTTP res code check
+if (!strcmp(rc, "301") || !strcmp(rc, "302")) {
+ char* eol = 0;
+ if ((eol = strchr(redir_loc, '\r')) > 0) *eol = 0;
+ else if (redir_loc) error_exit("Could not parse redirect URL");
+ if (redirects < 0) error_exit("Too many redirects");
+
+ printf("Redirection: %s %s \n", rc, r_str);
+ printf("%s \n", redir_loc);
+ redir_loc = redir_loc+strlen("Location: ");
+ close(sock);
+ get_info(redir_loc, hostname, port, path);
+} else if (!strcmp(rc,
[Toybox] [PATCH] vi: added support for tabs
Add: tabs, follows tabstop variable currently hardcoded to 8. Should
be adjustable with :set tabstop N according to man page
Fix: search, issue with searching substring that is more left than
cursor position on following lines.
---
toys/pending/vi.c | 36 +++-
1 file changed, 27 insertions(+), 9 deletions(-)
From b92ffbd25cd8827f8a91074b8d9f6af7be4e7f93 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Jarno=20M=C3=A4kip=C3=A4=C3=A4?=
Date: Mon, 9 Sep 2019 18:34:27 +0300
Subject: [PATCH] vi: added support for tabs
Add: tabs, follows tabstop variable currently hardcoded to 8. Should be adjustable with :set tabstop N according to man page
fix: search, issue with searching substring that is more left than
cursor position on following lines.
---
toys/pending/vi.c | 36 +++-
1 file changed, 27 insertions(+), 9 deletions(-)
diff --git a/toys/pending/vi.c b/toys/pending/vi.c
index 712bf160..b861a667 100644
--- a/toys/pending/vi.c
+++ b/toys/pending/vi.c
@@ -31,6 +31,7 @@ GLOBALS(
int modified;
char vi_reg;
char *last_search;
+int tabstop;
)
/*
@@ -731,12 +732,11 @@ static int search_str(char *s)
}
if (c) {
-TT.cur_col = c_r->line->str_data-c; //TODO ??
TT.cur_col = c-c_r->line->str_data;
} else for (; !c;) {
lst = lst->down;
if (!lst) return 1;
-c = strstr(>line->str_data[TT.cur_col], s);
+c = strstr(lst->line->str_data, s);
}
c_r = lst;
TT.cur_col = c-c_r->line->str_data;
@@ -791,6 +791,7 @@ void vi_main(void)
TT.screen_width = 80;
TT.screen_height = 24;
TT.vi_mode = 1;
+ TT.tabstop = 8;
terminal_size(_width, _height);
TT.screen_height -= 2; //TODO this is hack fix visual alignment
set_terminal(0, 1, 0, 0);
@@ -905,9 +906,7 @@ void vi_main(void)
il->str_data[il->str_len--] = 0;
break;
case 0x09:
- //TODO implement real tabs
- il->str_data[il->str_len++] = ' ';
- il->str_data[il->str_len++] = ' ';
+ il->str_data[il->str_len++] = '\t';
break;
case 0x0D:
@@ -1198,7 +1197,10 @@ static int utf8_lnw(int* width, char* str, int bytes)
wchar_t wc;
int length = 1;
*width = 1;
-// if (str < 0x7F) return length;
+ if (*str == 0x09) {
+*width = TT.tabstop;
+return 1;
+ }
length = mbtowc(, str, bytes);
switch (length) {
case -1:
@@ -1218,6 +1220,7 @@ static int utf8_lnw(int* width, char* str, int bytes)
static int utf8_width(char *str, int bytes)
{
wchar_t wc;
+ if (*str == 0x09) return TT.tabstop;
switch (mbtowc(, str, bytes)) {
case -1:
mbtowc(0,0,4);
@@ -1267,13 +1270,18 @@ static char* utf8_last(char* str, int size)
static int draw_str_until(int *drawn, char *str, int width, int bytes)
{
+ int len = 0;
int rune_width = 0;
int rune_bytes = 0;
int max_bytes = bytes;
int max_width = width;
char* end = str;
for (;width && bytes;) {
-rune_bytes = utf8_lnw(_width, end, 4);
+if (*end == 0x09) {
+ rune_bytes = 1;
+ rune_width = TT.tabstop;
+} else rune_bytes = utf8_lnw(_width, end, 4);
+
if (!rune_bytes) break;
if (width - rune_width < 0) goto write_bytes;
width -= rune_width;
@@ -1281,14 +1289,24 @@ static int draw_str_until(int *drawn, char *str, int width, int bytes)
end += rune_bytes;
}
for (;bytes;) {
-rune_bytes = utf8_lnw(_width, end, 4);
+if (*end == 0x09) {
+ rune_bytes = 1;
+ rune_width = TT.tabstop;
+} else rune_bytes = utf8_lnw(_width, end, 4);
+
if (!rune_bytes) break;
if (rune_width) break;
bytes -= rune_bytes;
end += rune_bytes;
}
write_bytes:
- fwrite(str, max_bytes-bytes, 1, stdout);
+ len = max_bytes-bytes;
+ for (;len--; str++) {
+if (*str == 0x09) {
+ int i = 8;
+ for (;i--;) fwrite(" ", 1, 1, stdout);
+} else fwrite(str, 1, 1, stdout);
+ }
*drawn = max_bytes-bytes;
return max_width-width;
}
--
2.19.1
___
Toybox mailing list
Toybox@lists.landley.net
http://lists.landley.net/listinfo.cgi/toybox-landley.net
Re: [Toybox] [PATCH] vi: small fixes.
Mailinglist didint send me Rob:s message and I feel like I am missing sometimes other messages too. But I saw robs message on archive so wanted to reply anyway so i copy pasted it here... Rob Landley wrote: >So never edit a makefile with this? >What _other_ changes does it make the data it's modifying? (My plan for >non-unix >line endings was to treat them like any other low ascii character and show the >^M version. I made escape character display logic and everything...) >Rob Right now tabs are just converted to 2 spaces, for making displaying easier but ofc this should not happen. But instead tabs should be kept intact in editing buffer, but perhaps being converted to spaces when displaying. Or just display them as tabs and have headache with long lines. So you should not edit makefiles yet Line ending rule should be parsed from file when loading and preserved, unless user wants to convert them. Perhaps the next focus should be to preserve file as intact as possible... -Jarno On Sat, Sep 7, 2019 at 10:23 PM Jarno Mäkipää wrote: > > I apologize for potentially being responsible of destroying config > files on your device :-) > > Happy to hear someone already tried to use it. There will be some > problems with files that has non UNIX line-endings also tabs will be > replaced by spaces etc... > But for task like changing variable or commenting out few lines it > might come handy when there is nothing else available. > > I'm trying to find time and motivation to continue working on this, > now as the summer has past. > > -Jarno > > On Sat, Sep 7, 2019 at 10:20 AM enh via Toybox > wrote: > > > > I really needed to be able to edit a file on the device, and this was > > the nearest thing handy, and it turns out to be more or less usable for > > basic editing, so... > > > > Support cursor keys. > > > > Support :q (since there's currently no record of whether the file's > > modified or not, :q is the same as :q!). > > > > Add 'A' to insert at end of line. > > > > Add 'n' to find next after '/'. > > > > Fix backspace all the way to get out of ex command mode. > > > > Fix escape sequences to not hard-code assumptions about the terminal's > > default background and foreground colors. > > > > Fix 'spesial' typo for 'special', and remove explicit array sizes. > > --- > > toys/pending/vi.c | 109 -- > > 1 file changed, 66 insertions(+), 43 deletions(-) > > ___ > > Toybox mailing list > > Toybox@lists.landley.net > > http://lists.landley.net/listinfo.cgi/toybox-landley.net ___ Toybox mailing list Toybox@lists.landley.net http://lists.landley.net/listinfo.cgi/toybox-landley.net
Re: [Toybox] [PATCH] vi: small fixes.
I apologize for potentially being responsible of destroying config files on your device :-) Happy to hear someone already tried to use it. There will be some problems with files that has non UNIX line-endings also tabs will be replaced by spaces etc... But for task like changing variable or commenting out few lines it might come handy when there is nothing else available. I'm trying to find time and motivation to continue working on this, now as the summer has past. -Jarno On Sat, Sep 7, 2019 at 10:20 AM enh via Toybox wrote: > > I really needed to be able to edit a file on the device, and this was > the nearest thing handy, and it turns out to be more or less usable for > basic editing, so... > > Support cursor keys. > > Support :q (since there's currently no record of whether the file's > modified or not, :q is the same as :q!). > > Add 'A' to insert at end of line. > > Add 'n' to find next after '/'. > > Fix backspace all the way to get out of ex command mode. > > Fix escape sequences to not hard-code assumptions about the terminal's > default background and foreground colors. > > Fix 'spesial' typo for 'special', and remove explicit array sizes. > --- > toys/pending/vi.c | 109 -- > 1 file changed, 66 insertions(+), 43 deletions(-) > ___ > Toybox mailing list > Toybox@lists.landley.net > http://lists.landley.net/listinfo.cgi/toybox-landley.net ___ Toybox mailing list Toybox@lists.landley.net http://lists.landley.net/listinfo.cgi/toybox-landley.net
[Toybox] [PATCH] vi.c changes to vi cmd execution
Hello
Sorry that this patch got bit too big and its untidy. Next step should be
to clean these basic movement commands and make them all utf-8 aware. And
also handle some special cases with setting up extra flags..
-Jarno
Reimplemented to command mode execution to follow vi cmd pattern.
(REG)[COUNT0]{CMD}[COUNT1](SYM)
Most of the moves can be executed intependently or before command,
some require character after. (possibly with utf8)
Some of the commands do not require move, such as D, J, dd, yy, x...
There is also tons of special cases where move behaves differently
depending on command. For example 1cw and 1ce appear to be the same
but 1dw and 1de are not...
Most of the operations still need reimplementing and lots of cleanup
in order them to behave correctly
refactored word move to work with utf-8
---
toys/pending/vi.c | 390 +-
1 file changed, 320 insertions(+), 70 deletions(-)
From 30340f4c5d1659599fa880e29ca5d20fcd70b948 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Jarno=20M=C3=A4kip=C3=A4=C3=A4?=
Date: Sat, 13 Apr 2019 15:25:59 +0300
Subject: [PATCH] vi.c changes to vi cmd execution
Reimplemented to command mode execution to follow vi cmd pattern.
(REG)[COUNT0]{CMD}[COUNT1](SYM)
Most of the moves can be executed intependently or before command,
some require character after. (possibly with utf8)
Some of the commands do not require move, such as D, J, dd, yy, x...
There is also tons of special cases where move behaves differently
depending on command. For example 1cw and 1ce appear to be the same
but 1dw and 1de are not...
Most of the operations still need reimplementing and lots of cleanup
in order them to behave correctly
refactored word move to work with utf-8
---
toys/pending/vi.c | 390 +-
1 file changed, 320 insertions(+), 70 deletions(-)
diff --git a/toys/pending/vi.c b/toys/pending/vi.c
index 2beedf49..f7edc074 100644
--- a/toys/pending/vi.c
+++ b/toys/pending/vi.c
@@ -25,6 +25,11 @@ GLOBALS(
unsigned screen_height;
unsigned screen_width;
int vi_mode;
+int count0;
+int count1;
+int vi_mov_flag;
+int modified;
+char vi_reg;
)
/*
@@ -38,9 +43,6 @@ GLOBALS(
*ex callbacks
*
* FEATURE: ex: / ? % //atleast easy cases
- *vi: x dw d$ d0
- *vi: yw yy (y0 y$)
- *vi+ex: gg G //line movements
*ex: r
*ex: !external programs
*ex: w filename //only writes to same file now
@@ -59,17 +61,17 @@ static int draw_str_until(int *drawn, char *str, int width, int bytes);
static void draw_char(char c, int x, int y, int highlight);
//utf8 support
static int utf8_lnw(int* width, char* str, int bytes);
-static int utf8_dec(char key, char *utf8_scratch, int *sta_p) ;
+static int utf8_dec(char key, char *utf8_scratch, int *sta_p);
static int utf8_len(char *str);
static int utf8_width(char *str, int bytes);
static int draw_rune(char *c, int x, int y, int highlight);
static char* utf8_last(char* str, int size);
-static int cur_left(int count);
-static int cur_right(int count);
-static int cur_up(int count);
-static int cur_down(int count);
+static int cur_left(int count0, int count1, char* unused);
+static int cur_right(int count0, int count1, char* unused);
+static int cur_up(int count0, int count1, char* unused);
+static int cur_down(int count0, int count1, char* unused);
static void check_cursor_bounds();
static void adjust_screen_buffer();
@@ -92,7 +94,7 @@ struct str_line *il;
struct linelist *text; //file loaded into buffer
struct linelist *scr_r;//current screen coord 0 row
struct linelist *c_r;//cursor position row
-int modified;
+
void dlist_insert_nomalloc(struct double_list **list, struct double_list *new)
{
@@ -114,6 +116,7 @@ struct double_list *dlist_insert(struct double_list **list, char *data)
return new;
}
+//TODO implement
void linelist_unload()
{
@@ -126,7 +129,7 @@ void write_file(char *filename)
if (!filename)
filename = (char*)*toys.optargs;
fp = fopen(filename, "w");
- if (!fp) return ;
+ if (!fp) return;
while (lst) {
fprintf(fp, "%s\n", lst->line->str_data);
lst = lst->down;
@@ -188,9 +191,16 @@ int linelist_load(char *filename)
return 1;
}
+
+int vi_yy(char reg, int count0, int count1)
+{
+ return 1;
+}
+
//TODO this is overly complicated refactor with lib dllist
-int ex_dd(int count)
+int vi_dd(char reg, int count0, int count1)
{
+ int count = count0*count1;
struct linelist *lst = c_r;
if (c_r == text && text == scr_r) {
if (!text->down && !text->up && text->line) {
@@ -235,26 +245,17 @@ int ex_dd(int count)
recursion_exit:
count--;
//make this recursive
- if (count)
-return ex_dd(count);
+ if (count>0)
+return vi_dd(reg, count, 1);
success_exit:
check_cursor_bounds();
adjust_screen_buffer();
return 1;
}
-
-int ex_dw(int count)
-{
- return 1;
-}
-
-int
Re: [Toybox] [PATCH] scan_key: support more terminals.
vi does not necessarily need so many function keys in order to be usable. Mostly escape, and 30 or so different control key commands in order to implement everything on man page. But I think after vi has all the basic functionality done, things like utf-8 and handling of text buffer, could be reused to make clones of other editors. -Jarno On Sat, Apr 13, 2019 at 6:15 AM Rob Landley wrote: > On 4/12/19 8:58 PM, David Seikel wrote: > > I presume you meant to send this to the list. > > Yup. > > > On Fri, 12 Apr 2019 15:52:10 -0500 Rob Landley wrote: > > > >> On 4/11/19 9:46 PM, David Seikel wrote: > > luckily i suspect in 2019 we mainly want function keys just so we > > can throw them away and get on to the next meaningful keypress! > > but i can easily put this back to one constant per key if you > > prefer. > > Unfortunately a command line utility doesn't get to _see_ the > function keys, so yeah probably. > >>> > >>> Users of mc and mcedit need functioning function keys, though they > >>> do have a backup of Esc $DIGIT, which helps when a lot of X > >>> terminals use F10 for their menus. I know I long ago tried to get > >>> mcedit into toybox. > >> > >> I googled, the first page is entirey about a world editor for > >> minecraft? > > > > mcedit is the text editor that comes with midnight commander, it can > > be used stand alone. Dunno about minecraft. > > I'd planned to make an editor I could program to act like (at least) vi, > microemacs, joe, and nano. But somebody else is writing a vi so I guess > not... > > Rob > ___ > Toybox mailing list > Toybox@lists.landley.net > http://lists.landley.net/listinfo.cgi/toybox-landley.net > ___ Toybox mailing list Toybox@lists.landley.net http://lists.landley.net/listinfo.cgi/toybox-landley.net
[Toybox] [PATCH] VI rewrote char delete and hjkl movements
Reimplemented basic cursor movements and char delete.
In order to work more correctly with utf-8 data.
x,h,j,k,l seems to work now with test data such as
tests/files/utf8/test2.txt
hjkl now accept count parameter so 1000j will scroll file 1000 lines
relative move to bottom
word movements w,e,b... still need to be still reimplemented in order to
step correctly on utf-8 data
---
toys/pending/vi.c | 154 ++
1 file changed, 100 insertions(+), 54 deletions(-)
-Jarno
From 24d6d7c5ecb1fa58d96b35f8905e6ea0fd691c10 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Jarno=20M=C3=A4kip=C3=A4=C3=A4?=
Date: Wed, 3 Apr 2019 21:44:59 +0300
Subject: [PATCH] VI rewrote char delete and hjkl movements
Reimplemented basic cursor movements and char delete.
In order to work more correctly with utf-8 data.
x,h,j,k,l seems to work now with test data such as
tests/files/utf8/test2.txt
hjkl now accept count parameter so 1000j will scroll file 1000 lines
relative move to bottom
word movements w,e,b... still need to be still reimplemented in order to
step correctly on utf-8 data
---
toys/pending/vi.c | 154 ++
1 file changed, 100 insertions(+), 54 deletions(-)
diff --git a/toys/pending/vi.c b/toys/pending/vi.c
index c5c87507..31516b55 100644
--- a/toys/pending/vi.c
+++ b/toys/pending/vi.c
@@ -63,12 +63,13 @@ static int utf8_dec(char key, char *utf8_scratch, int *sta_p) ;
static int utf8_len(char *str);
static int utf8_width(char *str, int bytes);
static int draw_rune(char *c, int x, int y, int highlight);
+static char* utf8_last(char* str, int size);
-static void cur_left();
-static void cur_right();
-static void cur_up();
-static void cur_down();
+static int cur_left(int count);
+static int cur_right(int count);
+static int cur_up(int count);
+static int cur_down(int count);
static void check_cursor_bounds();
static void adjust_screen_buffer();
@@ -252,29 +253,57 @@ int ex_deol(int count)
return 1;
}
-//does not work with utf8 yet
int vi_x(int count)
{
char *s;
+ char *last;
int *l;
- int *p;
+ int length = 0;
+ int width = 0;
+ int remaining = 0;
+ char *end;
+ char *start;
if (!c_r)
return 0;
s = c_r->line->str_data;
l = _r->line->str_len;
- p = _col;
- if (!(*l)) return 0;
- if ((*p) == (*l)-1) {
-s[*p] = 0;
-if (*p) (*p)--;
-(*l)--;
+
+ last = utf8_last(s,*l);
+ if (last == s+TT.cur_col) {
+memset(last, 0, (*l)-TT.cur_col);
+*l = TT.cur_col;
+if (!TT.cur_col) return 1;
+last = utf8_last(s, TT.cur_col);
+TT.cur_col = last-s;
+return 1;
+ }
+
+ start = s+TT.cur_col;
+ end = start;
+ remaining = (*l)-TT.cur_col;
+ for (;remaining;) {
+int next = utf8_lnw(, end, remaining);
+if (next && width) {
+ if (!count) break;
+ count--;
+} if (!next) break;
+length += next;
+end += next;
+remaining -= next;
+ }
+ if (remaining) {
+memmove(start, end, remaining);
+memset(end+remaining,0,end-start);
} else {
-memmove(s+(*p), s+(*p)+1, (*l)-(*p));
-s[*l] = 0;
-(*l)--;
+memset(start,0,(*l)-TT.cur_col);
}
- count--;
- return (count) ? vi_x(count) : 1;
+ *l -= end-start;
+ if (!TT.cur_col) return 1;
+ if (TT.cur_col == (*l)) {
+last = utf8_last(s, TT.cur_col);
+TT.cur_col = last-s;
+ }
+ return 1;
}
//move commands does not behave correct way yet.
@@ -402,7 +431,7 @@ struct vi_cmd_param {
int (*vi_cmd_ptr)(int);
};
-struct vi_cmd_param vi_cmds[7] =
+struct vi_cmd_param vi_cmds[11] =
{
{"dd", _dd},
{"dw", _dw},
@@ -411,6 +440,10 @@ struct vi_cmd_param vi_cmds[7] =
{"b", _movb},
{"e", _move},
{"x", _x},
+ {"h", _left},
+ {"j", _down},
+ {"k", _up},
+ {"l", _right},
};
int run_vi_cmd(char *cmd)
@@ -426,7 +459,7 @@ int run_vi_cmd(char *cmd)
else {
cmd = cmd_e;
}
- for (; i<7; i++) {
+ for (; i < 11; i++) {
if (strstr(cmd, vi_cmds[i].cmd)) {
return vi_cmds[i].vi_cmd_ptr(val);
}
@@ -440,7 +473,7 @@ int search_str(char *s)
struct linelist *lst = c_r;
char *c = strstr(_r->line->str_data[TT.cur_col], s);
if (c) {
-TT.cur_col = c_r->line->str_data-c;
+TT.cur_col = c_r->line->str_data-c; //TODO ??
TT.cur_col = c-c_r->line->str_data;
}
else for (; !c;) {
@@ -523,18 +556,6 @@ void vi_main(void)
}
if (TT.vi_mode == 1) { //NORMAL
switch (key) {
-case 'h':
- cur_left();
- break;
-case 'j':
- cur_down();
- break;
-case 'k':
- cur_up();
- break;
-case 'l':
- cur_right();
- break;
case '/':
case '?':
case ':':
@@ -554,7 +575,7 @@ void vi_main(void)
break;
default:
if (key > 0x20 && key < 0x7B) {
-vi_buf[vi_buf_pos] = key;
+vi_buf[vi_buf_pos] = key;//TODO handle input better
vi_buf_pos++;
if
[Toybox] [PATCH] vi: bug fixes
Style cleanups:
Removing whitespaces at end of lines, hopefully reduces git am
warnings
Bug fixes:
fix segfault if file did not exist, now creates one empty line
fix insert mode text not showing on start of line
fix append on empty line
fix cursor move right on empty line
---
toys/pending/vi.c | 203 --
1 file changed, 107 insertions(+), 96 deletions(-)
From adab1704cdfb99f06863a7a1db16b4f4e1990d6b Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Jarno=20M=C3=A4kip=C3=A4=C3=A4?=
Date: Fri, 29 Mar 2019 18:44:23 +0200
Subject: [PATCH] vi: bug fixes
Style cleanups:
Removing whitespaces at end of lines, hopefully reduces git am
warnings
Bug fixes:
fix segfault if file did not exist, now creates one empty line
fix insert mode text not showing on start of line
fix append on empty line
fix cursor move right on empty line
---
toys/pending/vi.c | 203 --
1 file changed, 107 insertions(+), 96 deletions(-)
diff --git a/toys/pending/vi.c b/toys/pending/vi.c
index ab15bfe0..c5c87507 100644
--- a/toys/pending/vi.c
+++ b/toys/pending/vi.c
@@ -31,20 +31,20 @@ GLOBALS(
*
* TODO:
* BUGS: screen pos adjust does not cover "widelines"
- *
+ *
*
* REFACTOR: use dllist functions where possible.
*draw_page dont draw full page at time if nothing changed...
*ex callbacks
- *
+ *
* FEATURE: ex: / ? % //atleast easy cases
- *vi: x dw d$ d0
+ *vi: x dw d$ d0
*vi: yw yy (y0 y$)
*vi+ex: gg G //line movements
- *ex: r
+ *ex: r
*ex: !external programs
*ex: w filename //only writes to same file now
- *big file support?
+ *big file support?
*/
@@ -75,7 +75,7 @@ static void adjust_screen_buffer();
struct str_line {
int alloc_len;
- int str_len;
+ int str_len;
char *str_data;
};
@@ -90,7 +90,7 @@ struct linelist {
struct str_line *il;
struct linelist *text; //file loaded into buffer
struct linelist *scr_r;//current screen coord 0 row
-struct linelist *c_r;//cursor position row
+struct linelist *c_r;//cursor position row
int modified;
void dlist_insert_nomalloc(struct double_list **list, struct double_list *new)
@@ -113,9 +113,9 @@ struct double_list *dlist_insert(struct double_list **list, char *data)
return new;
}
-void linelist_unload()
+void linelist_unload()
{
-
+
}
void write_file(char *filename)
@@ -133,7 +133,7 @@ void write_file(char *filename)
fclose(fp);
}
-int linelist_load(char *filename)
+int linelist_load(char *filename)
{
struct linelist *lst = c_r;//cursor position or 0
FILE *fp = 0;
@@ -141,7 +141,18 @@ int linelist_load(char *filename)
filename = (char*)*toys.optargs;
fp = fopen(filename, "r");
- if (!fp) return 0;
+ if (!fp) {
+char *line = xzalloc(80);
+ssize_t alc = 80;
+lst = (struct linelist*)dlist_add((struct double_list**),
+xzalloc(sizeof(struct str_line)));
+lst->line->alloc_len = alc;
+lst->line->str_len = 0;
+lst->line->str_data = line;
+text = lst;
+dlist_terminate(text->up);
+return 1;
+ }
for (;;) {
char *line = xzalloc(80);
@@ -177,7 +188,7 @@ int linelist_load(char *filename)
}
//TODO this is overly complicated refactor with lib dllist
-int ex_dd(int count)
+int ex_dd(int count)
{
struct linelist *lst = c_r;
if (c_r == text && text == scr_r) {
@@ -185,7 +196,7 @@ int ex_dd(int count)
text->line->str_len = 1;
sprintf(text->line->str_data, " ");
goto success_exit;
-}
+}
if (text->down) {
text = text->down;
text->up = 0;
@@ -236,7 +247,7 @@ int ex_dw(int count)
return 1;
}
-int ex_deol(int count)
+int ex_deol(int count)
{
return 1;
}
@@ -253,27 +264,27 @@ int vi_x(int count)
l = _r->line->str_len;
p = _col;
if (!(*l)) return 0;
- if ((*p) == (*l)-1) {
+ if ((*p) == (*l)-1) {
s[*p] = 0;
-if (*p) (*p)--;
+if (*p) (*p)--;
+(*l)--;
+ } else {
+memmove(s+(*p), s+(*p)+1, (*l)-(*p));
+s[*l] = 0;
(*l)--;
- } else {
-memmove(s+(*p), s+(*p)+1, (*l)-(*p));
-s[*l] = 0;
-(*l)--;
}
count--;
- return (count) ? vi_x(count) : 1;
+ return (count) ? vi_x(count) : 1;
}
//move commands does not behave correct way yet.
//only jump to next space for now.
-int vi_movw(int count)
+int vi_movw(int count)
{
if (!c_r)
return 0;
//could we call moveend first
- while (c_r->line->str_data[TT.cur_col] > ' ')
+ while (c_r->line->str_data[TT.cur_col] > ' ')
TT.cur_col++;
while (c_r->line->str_data[TT.cur_col] <= ' ') {
TT.cur_col++;
@@ -283,7 +294,7 @@ int vi_movw(int count)
c_r = c_r->down;
TT.cur_col = 0;
}
- }
+ }
count--;
if (count>1)
return vi_movw(count);
@@ -293,7 +304,7 @@ int vi_movw(int count)
return 1;
}
-int vi_movb(int
[Toybox] [PATCH] vi.c improved utf-8 support
Now calculates utf-8 rune width properly before trying to print on screen.
works with test1.txt and test2.txt on tests/files/utf8 folder with
0x0300-0x036F combining chars
Uses mbtowc and wcwidth to calculate width of rune. These both should be
implemented on c runtime that conforms POSIX-1.2001. Different c
runtimes might have different level of support to combining char
ranges etc...
I think there is no standard way to calculate utf-8 rune width without
converting it first to widechar. And i think conversion to widechar just
to calculate width is silly, since all write calls can be done with utf8
directly (on utf8 locales ofc), but in order to calculate them yourself
without pointless conversion, one would need to write variable byte lookup
array for binary searching weird ranges and make sure it works with
big-endian
systems too...
By the way running ./watch ./cat tests/files/utf8/japan.txt does not print
the
text for some reason, but other test data does... I was checking how
well original crunch_str works and noticed it.
-Jarno
From fb744874ced9f883888f5cfbe9752eb1fecd01cb Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Jarno=20M=C3=A4kip=C3=A4=C3=A4?=
Date: Thu, 28 Mar 2019 08:32:36 +0200
Subject: [PATCH] vi.c improved utf-8 support
Now calculates utf-8 rune width properly before trying to print on screen.
works with test1.txt and test2.txt on tests/files/utf8 folder with
0x0300-0x036F combining chars
Uses mbtowc and wcwidth to calculate width of rune. These both should be
implemented on c runtime that conforms POSIX-1.2001. Different c
runtimes might have different level of support to combining char
ranges etc...
I think there is no standard way to calculate utf-8 rune width without
converting it first to widechar. And i think conversion to widechar just
to calculate width is silly, since all write calls can be done with utf8
directly (on utf8 locales ofc), but in order to calculate them yourself
without pointless conversion, one would need to write variable byte lookup
array for binary searching weird ranges and make sure it works with big-endian
systems too...
By the way running ./watch ./cat tests/files/utf8/japan.txt does not print the
text for some reason, but other test data does... I was checking how
well original crunch_str works and noticed it.
-Jarno
---
toys/pending/vi.c | 231 +-
1 file changed, 165 insertions(+), 66 deletions(-)
diff --git a/toys/pending/vi.c b/toys/pending/vi.c
index 6a1c2349..ab15bfe0 100644
--- a/toys/pending/vi.c
+++ b/toys/pending/vi.c
@@ -31,8 +31,6 @@ GLOBALS(
*
* TODO:
* BUGS: screen pos adjust does not cover "widelines"
- *utf8 problems with some files. perhaps use lib utf8 functions instead
- *append to EOL does not show input but works when ESC out
*
*
* REFACTOR: use dllist functions where possible.
@@ -57,10 +55,13 @@ struct linestack_show {
};
static void draw_page();
+static int draw_str_until(int *drawn, char *str, int width, int bytes);
static void draw_char(char c, int x, int y, int highlight);
//utf8 support
+static int utf8_lnw(int* width, char* str, int bytes);
static int utf8_dec(char key, char *utf8_scratch, int *sta_p) ;
static int utf8_len(char *str);
+static int utf8_width(char *str, int bytes);
static int draw_rune(char *c, int x, int y, int highlight);
@@ -644,79 +645,110 @@ static void draw_page()
int cy_scr = 0;
int cx_scr = 0;
int utf_l = 0;
+
+ char* line = 0;
+ int bytes = 0;
+ int drawn = 0;
struct linelist *scr_buf= scr_r;
//clear screen
tty_esc("2J");
tty_esc("H");
-
tty_jump(0, 0);
+
+ //draw lines until cursor row
for (; y < TT.screen_height; ) {
-if (scr_buf && scr_buf->line->str_data && scr_buf->line->str_len) {
- int p = 0;
- for (; p < scr_buf->line->str_len; y++) {
-unsigned x = 0;
-for (; x < TT.screen_width; x++) {
- if (p < scr_buf->line->str_len) {
-int hi = 0;
-if (scr_buf == c_r && p == TT.cur_col) {
- if (TT.vi_mode == 2) {
-tty_jump(x, y);
-
-tty_esc("1m"); //bold
-printf("%s", il->str_data);
-x += il->str_len;
-tty_esc("0m");
- }
- cy_scr = y;
- cx_scr = x;
-}
-utf_l = draw_rune(_buf->line->str_data[p], x, y, hi);
-if (!utf_l)
- break;
-p += utf_l;
-if (utf_l > 2) x++;//traditional chinese is somehow 2 width in tty???
- }
- else {
-if (scr_buf == c_r && p == TT.cur_col) {
- if (TT.vi_mode == 2) {
-tty_jump(x, y);
-
-tty_esc("1m"); //bold
-printf("%s", il->str_data);
-x += il->str_len;
-tty_esc("0m");
- }
- cy_scr = y;
- cx_scr =
[Toybox] [PATCH] vi: Code style cleanup
Variable initialization to start of blocks
Space after if,for,while: if() -> if ()
Space after comma on function calls:
write(fd,buf,count); -> write(fd, buf, count);
Spaces surrounding variable initialization
Pointer * binding to variable instead of type: int* i -> int *i
Spaces surrounding compare operators
No spaces surrounding arimetic operators
Some aligntment whitespace fixes
Still messy and needs more cleanup, but there is bigger issues to solve
first.
---
toys/pending/vi.c | 369 +++---
1 file changed, 186 insertions(+), 183 deletions(-)
From 1f5f8db56e4bd7660a83c59d48eb9adbcde54a55 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Jarno=20M=C3=A4kip=C3=A4=C3=A4?=
Date: Sun, 24 Mar 2019 18:45:27 +0200
Subject: [PATCH] vi: Code style cleanup
Variable initialization to start of blocks
Space after if,for,while: if() -> if ()
Space after comma on function calls:
write(fd,buf,count); -> write(fd, buf, count);
Spaces surrounding variable initialization
Pointer * binding to variable instead of type: int* i -> int *i
Spaces surrounding compare operators
No spaces surrounding arimetic operators
Some aligntment whitespace fixes
Still messy and needs more cleanup, but there is bigger issues to solve
first.
---
toys/pending/vi.c | 369 +++---
1 file changed, 186 insertions(+), 183 deletions(-)
diff --git a/toys/pending/vi.c b/toys/pending/vi.c
index 20368563..6a1c2349 100644
--- a/toys/pending/vi.c
+++ b/toys/pending/vi.c
@@ -59,9 +59,9 @@ struct linestack_show {
static void draw_page();
static void draw_char(char c, int x, int y, int highlight);
//utf8 support
-static int utf8_dec(char key, char* utf8_scratch,int* sta_p) ;
-static int utf8_len(char* str);
-static int draw_rune(char* c,int x,int y, int highlight);
+static int utf8_dec(char key, char *utf8_scratch, int *sta_p) ;
+static int utf8_len(char *str);
+static int draw_rune(char *c, int x, int y, int highlight);
static void cur_left();
@@ -75,7 +75,7 @@ static void adjust_screen_buffer();
struct str_line {
int alloc_len;
int str_len;
- char* str_data;
+ char *str_data;
};
//lib dllist uses next and prev kinda opposite what im used to so I just
@@ -97,7 +97,7 @@ void dlist_insert_nomalloc(struct double_list **list, struct double_list *new)
if (*list) {
new->next = *list;
new->prev = (*list)->prev;
-if((*list)->prev) (*list)->prev->next = new;
+if ((*list)->prev) (*list)->prev->next = new;
(*list)->prev = new;
} else *list = new->next = new->prev = new;
}
@@ -117,39 +117,38 @@ void linelist_unload()
}
-void write_file(char* filename)
+void write_file(char *filename)
{
struct linelist *lst = text;
FILE *fp = 0;
if (!filename)
filename = (char*)*toys.optargs;
- fp = fopen(filename,"w");
+ fp = fopen(filename, "w");
if (!fp) return ;
- while(lst) {
-fprintf(fp,"%s\n",lst->line->str_data);
+ while (lst) {
+fprintf(fp, "%s\n", lst->line->str_data);
lst = lst->down;
}
fclose(fp);
}
-int linelist_load(char* filename)
+int linelist_load(char *filename)
{
struct linelist *lst = c_r;//cursor position or 0
- FILE* fp = 0;
+ FILE *fp = 0;
if (!filename)
filename = (char*)*toys.optargs;
fp = fopen(filename, "r");
if (!fp) return 0;
-
for (;;) {
-char* line = xzalloc(80);
-ssize_t alc =80;
+char *line = xzalloc(80);
+ssize_t alc = 80;
ssize_t len;
-if ((len = getline(, (void *), fp))== -1) {
+if ((len = getline(, (void *), fp)) == -1) {
if (errno == EINVAL || errno == ENOMEM) {
-printf("error %d\n",errno);
+printf("error %d\n", errno);
}
free(line);
break;
@@ -160,8 +159,8 @@ int linelist_load(char* filename)
lst->line->str_len = len;
lst->line->str_data = line;
-if (lst->line->str_data[len-1]=='\n') {
- lst->line->str_data[len-1]=0;
+if (lst->line->str_data[len-1] == '\n') {
+ lst->line->str_data[len-1] = 0;
lst->line->str_len--;
}
if (text == 0) {
@@ -179,23 +178,23 @@ int linelist_load(char* filename)
//TODO this is overly complicated refactor with lib dllist
int ex_dd(int count)
{
- struct linelist* lst = c_r;
- if (c_r==text && text == scr_r) {
+ struct linelist *lst = c_r;
+ if (c_r == text && text == scr_r) {
if (!text->down && !text->up && text->line) {
- text->line->str_len=1;
- sprintf(text->line->str_data," ");
+ text->line->str_len = 1;
+ sprintf(text->line->str_data, " ");
goto success_exit;
}
if (text->down) {
- text =text->down;
- text->up=0;
- c_r=text;
- scr_r=text;
+ text = text->down;
+ text->up = 0;
+ c_r = text;
+ scr_r = text;
free(lst->line->str_data);
free(lst->line);
free(lst);
}
- goto recursion_exit;
+goto recursion_exit;
}
//TODO use lib dllist stuff
if (lst)
@@
[Toybox] [PATCH] vi: Removed C99/GNU99 style for loop initializers
Removed for(int i = 0;) style loop initializers to be consistant
with project style.
Removed few unused global variables
Added 2 empty white space lines back to CONFIG comment section.
---
toys/pending/vi.c | 16 +---
1 file changed, 9 insertions(+), 7 deletions(-)
diff --git a/toys/pending/vi.c b/toys/pending/vi.c
index db2f42c9..20368563 100644
--- a/toys/pending/vi.c
+++ b/toys/pending/vi.c
@@ -4,7 +4,9 @@
* Copyright 2019 Jarno Mäkipää
*
* See http://pubs.opengroup.org/onlinepubs/9699919799/utilities/vi.html
+
USE_VI(NEWTOY(vi, "<1>1", TOYFLAG_USR|TOYFLAG_BIN))
+
config VI
bool "vi"
default n
@@ -18,9 +20,6 @@ config VI
#include "toys.h"
GLOBALS(
-struct termios default_opts;
-struct linestack *ls;
-char *statline;
int cur_col;
int cur_row;
unsigned screen_height;
@@ -406,7 +405,8 @@ int run_vi_cmd(char* cmd)
else {
cmd = cmd_e;
}
- for(int i=0;i<7;i++) {
+ int i = 0;
+ for(;i<7;i++) {
if (strstr(cmd,vi_cmds[i].cmd)) {
return vi_cmds[i].vi_cmd_ptr(val);
}
@@ -654,7 +654,8 @@ static void draw_page()
tty_jump(0,0);
for(; y < TT.screen_height; ) {
if (scr_buf && scr_buf->line->str_data && scr_buf->line->str_len) {
- for(int p = 0; p < scr_buf->line->str_len;y++) {
+ int p = 0;
+ for(; p < scr_buf->line->str_len;y++) {
unsigned x = 0;
for(;xline->str_len) {
@@ -842,7 +843,8 @@ static void adjust_screen_buffer()
//40xxx10xx10xx10xx
static int utf8_len(char* str)
{
- int len=0;
+ int len = 0;
+ int i = 0;
uint8_t *c = (uint8_t*)str;
if (!c || !(*c)) return 0;
if (*c < 0x7F) return 1;
@@ -851,7 +853,7 @@ static int utf8_len(char* str)
else if ((*c & 0xF8) == 0xF0 ) len = 4;
else return 0;
c++;
- for(int i = len-1;i>0;i--) {
+ for(i = len-1;i>0;i--) {
if ((*c++ & 0xc0)!=0x80) return 0;
}
return len;
--
2.19.1
___
Toybox mailing list
Toybox@lists.landley.net
http://lists.landley.net/listinfo.cgi/toybox-landley.net
Re: [Toybox] [PATCH] vi.c double_list based implementation.
Alright I have
ubuntu 18.10 with gcc 8.2.0
and
Archlinux with gcc 8.2.1
both builds fine with freshly cloned repo and commands
make distclean
make defconfig
make vi
I can set up virtual image with ubuntu 14.04 perhaps tomorrow. what
compiler version you might be using?
-Jarno
On Sat, Mar 23, 2019 at 9:59 PM Rob Landley wrote:
> I just did:
>
> make distclean
> make defconfig
> make vi
>
> On ubuntu 14.04.
>
> Rob
>
> On 3/23/19 2:27 PM, jarno Mäkipää wrote:
> > Hello
> >
> > Sorry I just build with default build flags made by:
> > make menuconfig
> > make
> >
> > Do you have instructions on some page what build flags should normally
> be set
> > on? So I can fix those few errors.
> >
> > -Jarno
> >
> >
> > On Sat, Mar 23, 2019 at 8:59 PM Rob Landley > <mailto:r...@landley.net>> wrote:
> >
> > On 3/23/19 6:37 AM, Jarno Mäkipää wrote:
> > > Has beginnings of reading file, saving file, hjkl movement,
> insert, ex
> > > (only w, wq, q!), search with /, some other normal mode actions
> (dd, w, b,
> > > e), some utf8 support
> >
> > Compile vitoys/pending/vi.c: In function 'run_vi_cmd':
> > toys/pending/vi.c:409:3: error: 'for' loop initial declarations are
> only allowed
> > in C99 mode
> >for(int i=0;i<7;i++) {
> >^
> > toys/pending/vi.c:409:3: note: use option -std=c99 or -std=gnu99 to
> compile your
> > code
> > toys/pending/vi.c: In function 'draw_page':
> > toys/pending/vi.c:657:7: error: 'for' loop initial declarations are
> only allowed
> > in C99 mode
> >for(int p = 0; p < scr_buf->line->str_len;y++) {
> >^
> > toys/pending/vi.c: In function 'utf8_len':
> > toys/pending/vi.c:854:3: error: 'for' loop initial declarations are
> only allowed
> > in C99 mode
> >for(int i = len-1;i>0;i--) {
> >^
> > lib/lib.c: In function 'parse_formats':
> > lib/lib.c:1563:3: error: implicit declaration of function 'check_tm'
> > [-Werror=implicit-function-declaration]
> >check_tm();
> >^
> > lib/lib.c: At top level:
> > lib/lib.c:1482:12: warning: 'parse_formats' defined but not used
> > [-Wunused-function]
> > static int parse_formats(char *str, time_t *t, int *nano)
> > ^
> > cc1: some warnings being treated as errors
> > make: *** [vi] Error 1
> >
> > Rob
> >
>
___
Toybox mailing list
Toybox@lists.landley.net
http://lists.landley.net/listinfo.cgi/toybox-landley.net
Re: [Toybox] [PATCH] vi.c double_list based implementation.
Hello
Sorry I just build with default build flags made by:
make menuconfig
make
Do you have instructions on some page what build flags should normally be
set on? So I can fix those few errors.
-Jarno
On Sat, Mar 23, 2019 at 8:59 PM Rob Landley wrote:
> On 3/23/19 6:37 AM, Jarno Mäkipää wrote:
> > Has beginnings of reading file, saving file, hjkl movement, insert, ex
> > (only w, wq, q!), search with /, some other normal mode actions (dd, w,
> b,
> > e), some utf8 support
>
> Compile vitoys/pending/vi.c: In function 'run_vi_cmd':
> toys/pending/vi.c:409:3: error: 'for' loop initial declarations are only
> allowed
> in C99 mode
>for(int i=0;i<7;i++) {
>^
> toys/pending/vi.c:409:3: note: use option -std=c99 or -std=gnu99 to
> compile your
> code
> toys/pending/vi.c: In function 'draw_page':
> toys/pending/vi.c:657:7: error: 'for' loop initial declarations are only
> allowed
> in C99 mode
>for(int p = 0; p < scr_buf->line->str_len;y++) {
>^
> toys/pending/vi.c: In function 'utf8_len':
> toys/pending/vi.c:854:3: error: 'for' loop initial declarations are only
> allowed
> in C99 mode
>for(int i = len-1;i>0;i--) {
>^
> lib/lib.c: In function 'parse_formats':
> lib/lib.c:1563:3: error: implicit declaration of function 'check_tm'
> [-Werror=implicit-function-declaration]
>check_tm();
>^
> lib/lib.c: At top level:
> lib/lib.c:1482:12: warning: 'parse_formats' defined but not used
> [-Wunused-function]
> static int parse_formats(char *str, time_t *t, int *nano)
> ^
> cc1: some warnings being treated as errors
> make: *** [vi] Error 1
>
> Rob
>
___
Toybox mailing list
Toybox@lists.landley.net
http://lists.landley.net/listinfo.cgi/toybox-landley.net
[Toybox] [PATCH] vi.c double_list based implementation.
Has beginnings of reading file, saving file, hjkl movement, insert, ex
(only w, wq, q!), search with /, some other normal mode actions (dd, w, b,
e), some utf8 support
Everything is still very unfinished and partly behaves wrongly comparing
to original vi. But simple tasks like modifying short config files
should be possible.
Some things like draw_page needs serious refactor since it now writes
whole screen after every keypress. Didint bother to refactor yet if
linked list needs to be replaced with something else...
---
toys/pending/vi.c | 893 +-
1 file changed, 879 insertions(+), 14 deletions(-)
diff --git a/toys/pending/vi.c b/toys/pending/vi.c
index bf8db841..db2f42c9 100644
--- a/toys/pending/vi.c
+++ b/toys/pending/vi.c
@@ -1,17 +1,15 @@
/* vi.c - You can't spell "evil" without "vi".
*
* Copyright 2015 Rob Landley
+ * Copyright 2019 Jarno Mäkipää
*
* See http://pubs.opengroup.org/onlinepubs/9699919799/utilities/vi.html
-
USE_VI(NEWTOY(vi, "<1>1", TOYFLAG_USR|TOYFLAG_BIN))
-
config VI
bool "vi"
default n
help
usage: vi FILE
-
Visual text editor. Predates the existence of standardized cursor keys,
so the controls are weird and historical.
*/
@@ -20,29 +18,896 @@ config VI
#include "toys.h"
GLOBALS(
- struct linestack *ls;
- char *statline;
+struct termios default_opts;
+struct linestack *ls;
+char *statline;
+int cur_col;
+int cur_row;
+unsigned screen_height;
+unsigned screen_width;
+int vi_mode;
)
+/*
+ *
+ * TODO:
+ * BUGS: screen pos adjust does not cover "widelines"
+ *utf8 problems with some files. perhaps use lib utf8 functions instead
+ *append to EOL does not show input but works when ESC out
+ *
+ *
+ * REFACTOR: use dllist functions where possible.
+ *draw_page dont draw full page at time if nothing changed...
+ *ex callbacks
+ *
+ * FEATURE: ex: / ? % //atleast easy cases
+ *vi: x dw d$ d0
+ *vi: yw yy (y0 y$)
+ *vi+ex: gg G //line movements
+ *ex: r
+ *ex: !external programs
+ *ex: w filename //only writes to same file now
+ *big file support?
+ */
+
+
struct linestack_show {
struct linestack_show *next;
long top, left;
int x, width, y, height;
};
-// linestack, what to show, where to show it
-void linestack_show(struct linestack *ls, struct linestack_show *lss)
+static void draw_page();
+static void draw_char(char c, int x, int y, int highlight);
+//utf8 support
+static int utf8_dec(char key, char* utf8_scratch,int* sta_p) ;
+static int utf8_len(char* str);
+static int draw_rune(char* c,int x,int y, int highlight);
+
+
+static void cur_left();
+static void cur_right();
+static void cur_up();
+static void cur_down();
+static void check_cursor_bounds();
+static void adjust_screen_buffer();
+
+
+struct str_line {
+ int alloc_len;
+ int str_len;
+ char* str_data;
+};
+
+//lib dllist uses next and prev kinda opposite what im used to so I just
+//renamed both ends to up and down
+struct linelist {
+ struct linelist *up;//next
+ struct linelist *down;//prev
+ struct str_line *line;
+};
+//inserted line not yet pushed to buffer
+struct str_line *il;
+struct linelist *text; //file loaded into buffer
+struct linelist *scr_r;//current screen coord 0 row
+struct linelist *c_r;//cursor position row
+int modified;
+
+void dlist_insert_nomalloc(struct double_list **list, struct double_list *new)
{
- return;
+ if (*list) {
+new->next = *list;
+new->prev = (*list)->prev;
+if((*list)->prev) (*list)->prev->next = new;
+(*list)->prev = new;
+ } else *list = new->next = new->prev = new;
}
-void vi_main(void)
+
+// Add an entry to the end of a doubly linked list
+struct double_list *dlist_insert(struct double_list **list, char *data)
{
- int i;
+ struct double_list *new = xmalloc(sizeof(struct double_list));
+ new->data = data;
+ dlist_insert_nomalloc(list, new);
- if (!(TT.ls = linestack_load(*toys.optargs)))
-TT.ls = xzalloc(sizeof(struct linestack));
+ return new;
+}
+void linelist_unload()
+{
- for (i=0; ilen; i++)
-printf("%.*s\n", (int)TT.ls->idx[i].len, (char *)TT.ls->idx[i].ptr);
+}
+
+void write_file(char* filename)
+{
+ struct linelist *lst = text;
+ FILE *fp = 0;
+ if (!filename)
+filename = (char*)*toys.optargs;
+ fp = fopen(filename,"w");
+ if (!fp) return ;
+ while(lst) {
+fprintf(fp,"%s\n",lst->line->str_data);
+lst = lst->down;
+ }
+ fclose(fp);
+}
+
+int linelist_load(char* filename)
+{
+ struct linelist *lst = c_r;//cursor position or 0
+ FILE* fp = 0;
+ if (!filename)
+filename = (char*)*toys.optargs;
+
+ fp = fopen(filename, "r");
+ if (!fp) return 0;
+
+
+ for (;
