iterrr: extensible iterator library

[hamidrb80]

**new release note** :

  * operators `><` and `>!<` replaced with `|>` and `!>`
  * non-operator version added. now you can use template `iterrr`.



read more in `readme.md` of project

iterrr: extensible iterator library

[hamidrb80]

> What's the status here? Are there any plans to provide some DSL or other way 
> for defining custom adapters?

I think `imap` and `ifilter` is enough. unless you give me an example(s) that 
can change my mind

Read output only while input is not asked

[hamidrb80]

I guess you should use Stream thingy in `a.nim`. take a look at: 

iterrr: extensible iterator library

[hamidrb80]

> reusing [] for declaring idents looks poorly readable. Overloading the 
> meaning of a basic language construct is adding to the mental overhead. I 
> don't mind the slightly longer pseudo-closure syntax, as it's unambiguous and 
> non-intrusive.

I'm kinda agree with you, let me know if you think this is intuitive,


points |> imap((x,y) => x+y) ...


Run

iterrr: extensible iterator library

[Zoom]

> I think imap and ifilter is enough. unless you give me example(s) that can 
> change my mind

I agree that most cases are covered with those two, and some adapters can be 
boiled down to just a filter and a map, but not always. Some actions which 
require changes to the number of iterations can't be expressed with them, some 
require internal state, some act on multiple iterables. For example:

  * `flatten` is necessary to change the number of iterations inside your 
pipeline, such as unpacking containers on-the-go
  * `intersperse` inserts some items between your iterations, such as `','` 
between your chars
  * `chunks` collects and produces intermediate items `[1,2,3,4,5] |> chunks(2) 
# (1,2), (3,4), (5)` (basically, a filter with a state)
  * `zip`, `split`, `chain`, `cycle`...
  * `scan` is a map with a state (achievable by using a pre-declared variable)

iterrr: extensible iterator library

[hamidrb80]

> Looks good to me!

**see new version `0.2.1`** with removed `i` prefix and infix style custom 
idents `=>`, you can write:


(1..10) |> map(n => _ )
(1..10) |> map((a1, a2, ...) => _ )


Run

iterrr: extensible iterator library

[hamidrb80]

> I agree that most cases are covered with those two, and some adapters can be 
> boiled down to just a filter and a map, but not always. Some actions which 
> require changes to the number of iterations can't be expressed with them, 
> some require internal state, some act on multiple iterables. For example:

why not use `itertools`? 

iterrr: extensible iterator library

[Zoom]

I could use itertools for a simple case, but the main benefit of a shorter 
syntax for iterators is composition, i.e. building long, concise custom 
pipelines. If one doesn't care much about composability, templates from std or 
`itertools` are indeed good enough.

To date, the most able solution I know of is ZF, but it could benefit from more 
adapters and the current behaviour is a bit quirky from time to time.

Read output only while input is not asked

[ynfle]

Your issue is that b is waiting to receive input and you are waiting to read a 
line until the stream closes. Try 

iterrr: extensible iterator library

[Hlaaftana]

If your aim is extensibility I have a library which abstracts templates as 
static variables and using some weird combinator stuff I don't really 
understand you can [build your own custom 
iterators](https://github.com/metagn/applicates/blob/master/tests/test_iterator_typed.nim).
 Obviously I have no idea how to polish this idea but maybe someone can figure 
it out. The iterators you listed such as `flatten`, `intersperse`, 
`chunks`/`windows` etc should be possible as far as I can tell.

The macro being as part of recursive function body multiply

[mardiyah]

How correctly metaprogram, a macro (template failed on try) as part of 
recursive function body, which will be used as many times as multiple number of 
header function as superset the recursive function ?

Why doesn't echo have tag WriteIOEffect?

[federico3]

Dedicated effects for debugging+logging+metrics could be useful. Most of the 
time these are seen as "out-of-band" activities.

Read output only while input is not asked

[demotomohiro]

Both `readXYZ` and `peekXYZ` procs in stream module block until it get enough 
data to return. `stdin.readLine` in your b.nim blocks your code until it get a 
line.

You can read all lines from b.nim and write stdin for b.nim if you know many 
lines b.nim writes:

a1.nim:


import std/[osproc, streams]

let myprocess = startProcess("./b")
var i = 0
block mainLoop:
  while true:
var s: string
for j in 0..https://www.man7.org/linux/man-pages/man2/select.2.html>

In windows, it returns immediately in a single-threaded application but blocks 
in a multi-threaded application: 


Following code can works without stucking, but it implements 
`hasDataNoBlocking` that works only on posix:

a2.nim:


import std/[osproc, streams, posix]

proc hasDataNoBlocking(p: Process): bool =
  var rd: TFdSet
  
  FD_ZERO(rd)
  let m = max(0, int(p.outputHandle))
  FD_SET(cint(p.outputHandle), rd)
  
  var tv: Timeval
  
  result = int(select(cint(m+1), addr(rd), nil, nil, addr tv)) == 1

let myprocess = startProcess("./b")
var i = 0
block mainLoop:
  while true:
while true:
  var s: string
  if myprocess.outputstream.readLine(s):
echo s
  else:
break mainLoop
  if not myprocess.hasDataNoBlocking():
break
echo i
i += 1
myprocess.inputstream.writeLine($i)
myprocess.inputstream.flush()
  myprocess.close()

Run

b.nim:


import std/strutils

echo "START B"

for i in 0..10:
  var s = stdin.readLine
  for i in 1..parseInt(s):
echo "Hello " & $i

Run

asynctools might better in this case: