Re: [hackers] [dmenu][RFC][PATCH 0/4] Using sort and simple C program to get dmenu history functionality

[Xarchus]

Hi Silvan,

On Tue, Dec 01, 2015 at 08:04:04PM +0100, Silvan Jegen wrote:
> Do you know if your
> script works with the BSD awk implementation?

Yes, it should: I am trying to use only *nawk* features; nawk shipped first
in 1987; by now every unix under the sun should have it (and most simply
call it 'awk'; plus, if gawk is available, awk will be a symlink to gawk
anyway)

> > awk -F$'\t' '!x[$1]++' "$HISTORY" "$cache"
> 
> For some reason this does not work for me using gawk. I get both cache
> and history results (deduplicated) but the history results are not split
> on tab for some reason.

Probably your shell is not bash: I think the $'\t' construct is a
"bashism", sorry for that. You can insert instead as an argument to -F a
literal tab character (^I), but that is not easily visible in text (or
worse, might be expanded by your editor into spaces). A better and
guaranteed portable way is to move the definition of the field separator
directly into the awk script. With that the command becomes:

awk 'BEGIN { FS="\t" } !x[$1]++' "$HISTORY" "$cache"

However, that still uses the initial format for the history file:
"$cmdname\t$count" and we are going to swap that ...

> I like your solution using awk better and would vote for putting a
> version of your patch on suckless.org after adjusting it to deal with
> the history file in "$count\t$cmdname" format. Do you agree?

I agree.

Now swapping places for the count would complicate a bit the awk one-liner
I was talking about above and it would need to be like this:

awk 'NR==FNR { sub("^[0-9]+\t","") } !x[$0]++' "$HISTORY" "$cache"

The 'NR==FNR' test is true only for the first file (the history) and the
sub will cut the count and the tab in the front of the history lines,
leaving just the name; then it's the same de-duplication for all of them.
And because I don't care about fields anymore, I don't even set the FS.


On Wed, Dec 02, 2015 at 09:05:57AM +0100, Silvan Jegen wrote:
> On Tue, Dec 1, 2015 at 8:04 PM, Silvan Jegen  wrote:
> >
> > Personally I don't use multiselect so I will let others deal with that
> > use case...
> 
> Maybe I care a bit after all...
> 
> Using something like
> 
> echo -e "echo 1\nsleep 5; echo awesome\necho 200" | while read cmd; do
> ${SHELL:-"/bin/sh"} -c "$cmd" & done
> 
> would start each program after after a newline has been read.

Yep, that's exactly what I was thinking. Another way is to append an '&'
after all names except the last one before sending them into the $SHELL.

> I don't
> know how multiselect is implemented but it should be easily possible
> to send the program name with newline after Ctrl-Return has been
> pressed.

Actually that's how multisel works right now: a Ctrl-Return just outputs
that entry name (as if you'd pressed Return), but instead of exiting, dmenu
highlights the entry and keeps going. It's even possible to hit Ctrl-Return
repeatedly on the same entry and it will be outputted every time, so you
can get it multiple times ... 

For dmenu's bread and butter function -- selecting programs to run -- that
multisel behaviour is of questionable usefulness. However, it can be very
valuable when used in some special menu applications.


On Tue, Dec 01, 2015 at 08:04:04PM +0100, Silvan Jegen wrote:
> The patch did not apply for me, saying that updhist.awk does not exist.
> I think something went wrong with the patch creation because the diff
> mentions an existing (empty?) updhist.awk file. See below.
> 
> [...]
>
> This seems odd because it implies there already being an updhist.awk. I
> assume that's why the patch did not apply for me.
>
Yeah, sorry for that.  I was still trying to figure out how to coerce git
to generate a diff _without_ staging the new file. (I assume the 'git
apply' was the one that complained for you ?  Just use the 'patch' program
to apply the diff directly without git's involvement, that works for me)



Anyway, based on the change in the history format, I reworked all the
modifications and a new diff is attached -- this time there is no new file:
there is no separate awk file, I've just inserted it as a literal awk
program in the dmenu_run shell script.

Other changes:

- making a history file optional; the trigger to use history is a shell
  variable being present and non-null: DMENU_RUN_USE_HISTORY:

* if its value is "DEFAULT", it will use a default location/name for
  the history, that is in ~/.cache or $HOME; in both cases the default
  name for the history file will mimic the one for the dmenu_run cache

* otherwise, the variable's value will be used as the history file
  path; so make sure it exists, it's writable, etc.

* if that variable is not set or empty, no history is used: dmenu_run
  will behave as it does now

- added your change to deal with multisel (the while loop)

The new patch attached. Give it a try.

diff --git a/dmenu_path b/dmenu_path
old mode 100644
new mode 100755
index 338bac4..3f168aa
--- 

Re: [hackers] [dmenu][RFC][PATCH 0/4] Using sort and simple C program to get dmenu history functionality

[Silvan Jegen]

On Tue, Dec 1, 2015 at 8:04 PM, Silvan Jegen  wrote:
> Heyho!
>
> On Tue, Dec 01, 2015 at 05:51:59AM -0800, Xarchus wrote:
>>
>> This updhist awk script replacement will work with multiselect (multiple
>> inputs will simply increment their count or added as new). This includes
>> the case when dmenu outputs duplicate strings (with multiselect, a same
>> entry can be generated multiple times).
>>
>> But speaking of multiselect, the dmenu_run as it is now does not handle
>> very gracefully multiple selections ... Multiple programs selected in dmenu
>> will all be started, but sequentially, with the next waiting for the
>> previous to exit (an artifact of them all being fed to one shell instance).
>> I am not sure if that is the intention: my preference would be to start
>> each program as soon as it's selected with Ctrl-Return.
>
> Personally I don't use multiselect so I will let others deal with that
> use case...

Maybe I care a bit after all...

Using something like

echo -e "echo 1\nsleep 5; echo awesome\necho 200" | while read cmd; do
${SHELL:-"/bin/sh"} -c "$cmd" & done

would start each program after after a newline has been read. I don't
know how multiselect is implemented but it should be easily possible
to send the program name with newline after Ctrl-Return has been
pressed.

Re: [hackers] [dmenu][RFC][PATCH 0/4] Using sort and simple C program to get dmenu history functionality

[Silvan Jegen]

Heyho!

On Tue, Dec 01, 2015 at 05:51:59AM -0800, Xarchus wrote:
> On Mon, Nov 30, 2015 at 03:28:42PM +0100, Silvan Jegen wrote:
> > Heyho!
> > 
> > On Sat, Nov 28, 2015 at 11:25 PM, Hiltjo Posthuma
> >  wrote:
> > >
> > > This can be implemented in a few lines of shell (wc, sort) and maybe awk.
> > 
> > I *have* implemented the history part with sort. If you think the
> > history updating functionality that I ended up writing in C can be
> > (easily?) implemented in some shell script and/or awk then I would
> > like to see it on the list :)
> > 
> > I thought it should be possible to implement in awk but because you
> > have to both read input from stdin (the command) and from a file (the
> > history file) I couldn't figure it out in the admittedly short time I
> > kept trying to do it.
> > 
> > I tried abusing the -v option of gawk to set the command name as a
> > variable value but that only seems to work in a BEGIN block which is
> > executed before the input file is read. It may be possible to use the
> > shell to read the command and then initialize a variable in the awk
> > code given on the command line but I think the argument-escaping hell
> > would be very annoying to deal with.
> > 
> 
> As both Silvan and Hiltjo mentioned, here is an awk script that does the
> job of updhist. (more than that, it fixes a problem: the updhist.c as sent
> does not work with multisel, it will corrupt the history file if multiple
> selections are generated from a single dmenu invocation; but more about
> multiselect later)

I was aware that update.c does not deal with the case of multiple-line
commands. I did not know that there exists such a functionality for
dmenu though so I ignored it in the first version...


> In order to make the script more portable, I tried to keep the awk features
> limited to the nawk set and not using any of the gawk stuff. With gawk,
> things can get even simpler: for example gawk has sort, so the external sort
> would not be needed in the END block.
> (BTW, which is the target as the *official* awk?)

I prefer using the external sort.

I don't know if there is an "official" suckless awk. Do you know if your
script works with the BSD awk implementation?


> The script assumes the format of the history file to be 'filename tab
> count', the same as the C program. The problem with this is what happens
> if a file name contains tabs ... Crazy, but not impossible. A more reliable
> approach would be to put the count first then the remaining line is all a
> file name.

While I would be willing to take that risk, doing it your way should
simplify the sort command slightly as well and is a good idea.


> Anyhow, here is the script (tested) :
> 
> -%<- updhist.awk
> #!/usr/bin/awk -f
> 
> BEGIN {
> if(histfile=="") {
> print "updhist.awk: no history file specified" > "/dev/stderr"
> print "usage: awk -v histfile= -f updhist.awk" 
> > "/dev/stderr"
> exit(1)
> }
> FS=OFS="\t" # explicit tabs for input to allow file names with spaces 
> while ( (getline < histfile) > 0 )
> history[$1]=$2 # assumption: file name does not contain tabs
> close(histfile)
> }
> 
> {
> history[$0]++
> print
> }
> 
> END {
> for (f in history)
> print f,history[f] | "sort -t '\t' -k2rn >" histfile
> }
> -%<-
> 
> To use the above, replace in dmenu_run the call to updhist with 'awk -v
> histfile=$historyfile -f updhist.awk' (assuming updhist.awk is placed
> somewhere in AWKPATH, or /usr/share/awk, otherwise use the full path to the
> awk script).
> 
> Because the script will keep the history file already sorted, in
> 'dmenu_path' there is no need to sort the history when fed to dmenu (so
> leave out the 'sort -r -n -t ' ' -k 2').
> 
> Silvan talked about the fact that the commands in the history will show up
> twice: once from the history, once in the normal list. This too can be
> taken care of with a tiny awk one-liner to filter duplicates; that will
> replace the 'cat' at the end of the two lines in dmenu_path with this:
> 
> awk '!x[$0]++' - "$cache"
> 
> Going even further, the cut operation can be factored into awk (I assume it
> does cut on the tab that separates the name from the counter), so the whole
> line now becomes (this relies on a history format with the name first, tab,
> followed by the count):
> 
> awk -F$'\t' '!x[$1]++' "$HISTORY" "$cache"

For some reason this does not work for me using gawk. I get both cache
and history results (deduplicated) but the history results are not split
on tab for some reason.

Using external cut works fine for me though.


> P.S. Now back to multiselect:
> 
> This updhist awk script replacement will work with multiselect (multiple
> inputs will simply increment their count or added as new). This includes
> the case when dmenu outputs duplicate strings (with multiselect, a same
> entry can be 

Re: [hackers] [dmenu][RFC][PATCH 0/4] Using sort and simple C program to get dmenu history functionality

[Hiltjo Posthuma]

On Mon, Nov 30, 2015 at 3:28 PM, Silvan Jegen  wrote:
> Heyho!
>
> On Sat, Nov 28, 2015 at 11:25 PM, Hiltjo Posthuma
>  wrote:
>> On Fri, Nov 27, 2015 at 7:38 PM, Silvan Jegen  wrote:
>> This can be implemented in a few lines of shell (wc, sort) and maybe awk.
>
> I *have* implemented the history part with sort. If you think the
> history updating functionality that I ended up writing in C can be
> (easily?) implemented in some shell script and/or awk then I would
> like to see it on the list :)
>

Something like (quick hack):

cat historyfile | awk '//{x[$0]++; } END { for (k in x) { print x[k] "
   " k; }}' | sort -k 1rn,2 | cut -f 2- | dmenu >> historyfile

> I thought it should be possible to implement in awk but because you
> have to both read input from stdin (the command) and from a file (the
> history file) I couldn't figure it out in the admittedly short time I
> kept trying to do it.
>

The input is read from X, not stdin, but if you mean additional input
apart from the history file you can append it and probably use some
kind of "weights" (like used count), then after it use sort like
above.

Kind regards,
Hiltjo

Re: [hackers] [dmenu][RFC][PATCH 0/4] Using sort and simple C program to get dmenu history functionality

[Roberto E. Vargas Caballero]

On Mon, Nov 30, 2015 at 06:51:02PM +0100, Hiltjo Posthuma wrote:
> Something like (quick hack):
> 
> cat historyfile | awk '//{x[$0]++; } END { for (k in x) { print x[k] "
>" k; }}' | sort -k 1rn,2 | cut -f 2- | dmenu >> historyfile
> 

Avoid the death cat!!!. Use something like:

awk '{x[$0]++} END {for (k in x) { print x[k],k; }}' historyfile |
sort -k 1rn,2 | cut -f 2- | dmenu >> historyfile


Regards,

Re: [hackers] [dmenu][RFC][PATCH 0/4] Using sort and simple C program to get dmenu history functionality

[Silvan Jegen]

Heyho!

On Sat, Nov 28, 2015 at 11:25 PM, Hiltjo Posthuma
 wrote:
> On Fri, Nov 27, 2015 at 7:38 PM, Silvan Jegen  wrote:
>> Heyhey
>>
>> I kept thinking about a more general way to implement history
>> functionality for dmenu and this is what I came up with.
>>
>> We use the sort command to generate an input list for dmenu sorted by
>> count (first patch). dmenu itself is not modified and shows the available
>> commands with the most often used ones first (due to the sorting done
>> by sort). Before sending the dmenu output to a shell we use a simple
>> C program that reads a command from stdin for which it increments the
>> usage count in the history file before sending the command to the shell
>> to execute (patch 4; rest of the patches are just glue code).
>>
>
> This can be implemented in a few lines of shell (wc, sort) and maybe awk.

I *have* implemented the history part with sort. If you think the
history updating functionality that I ended up writing in C can be
(easily?) implemented in some shell script and/or awk then I would
like to see it on the list :)

I thought it should be possible to implement in awk but because you
have to both read input from stdin (the command) and from a file (the
history file) I couldn't figure it out in the admittedly short time I
kept trying to do it.

I tried abusing the -v option of gawk to set the command name as a
variable value but that only seems to work in a BEGIN block which is
executed before the input file is read. It may be possible to use the
shell to read the command and then initialize a variable in the awk
code given on the command line but I think the argument-escaping hell
would be very annoying to deal with.


>> * It uses the -t and -k options of sort which are not available for
>>   all sort implementations (not in sbase for example). If there is an
>>   easy way to replicate this functionality without using these sort
>>   options I would like to hear about it.
>
> Sbase sort supports -t and -k, if some part of it is broken: send a patch.

You are right. I must have checked an old sbase version. I will test
the patch with sbase sort as soon as I find the time.


Cheers,

Silvan

Re: [hackers] [dmenu][RFC][PATCH 0/4] Using sort and simple C program to get dmenu history functionality

[Hiltjo Posthuma]

On Fri, Nov 27, 2015 at 7:38 PM, Silvan Jegen  wrote:
> Heyhey
>
> I kept thinking about a more general way to implement history
> functionality for dmenu and this is what I came up with.
>
> We use the sort command to generate an input list for dmenu sorted by
> count (first patch). dmenu itself is not modified and shows the available
> commands with the most often used ones first (due to the sorting done
> by sort). Before sending the dmenu output to a shell we use a simple
> C program that reads a command from stdin for which it increments the
> usage count in the history file before sending the command to the shell
> to execute (patch 4; rest of the patches are just glue code).
>

This can be implemented in a few lines of shell (wc, sort) and maybe awk.

>
> * It uses the -t and -k options of sort which are not available for
>   all sort implementations (not in sbase for example). If there is an
>   easy way to replicate this functionality without using these sort
>   options I would like to hear about it.

Sbase sort supports -t and -k, if some part of it is broken: send a patch.