Passing iterators as arguments

But if was merged. would it then be possible to implement it as a concept ?

Passing iterators as arguments

is `Iterable[T]` implemented through concepts the correct way ?

Passing iterators as arguments

`iterable[T]` is a compiler supporting type that says "i want an iterator". It is not a concept.

Passing iterators as arguments

I would not call it a hack as much as ill thought out, it cannot be used with a typeclass and does not automatically use `pairs` or `items` so sequences/arrays require manual `.items`. It cannot be used as an inline iterator parameter, as you may know inline iterators are after all just fancy t

Passing iterators as arguments

Can you elaborate ? why is it a hack ? if it is a hack, why was it introduced ? what's the alternative ?

Passing iterators as arguments

its a hack

Passing iterators as arguments

Hi, now that `iterable[T]` was introduced in 1.6, Why wasn't the standard library updated to work with it yet ? is it just a matter of time ?

Passing iterators as arguments

> I wanted to avoid passing around containers that would then get copied, and > pass around iterators instead This is exactly what @timothee's proposal is good for. It allows you to pass iterators directly. The compiler expands the iterator structurally via inlining, or, at least, it has the po

Passing iterators as arguments

But this misses the point a little. I wanted to avoid passing around containers that would then get copied, and pass around iterators instead. In your examples ivec, fvec and dict would get copied, wouldn't they? With the help of the Discord chat, I now ended up defining `toFirstClassIter` like

Passing iterators as arguments

> And how would I do that? Take this: import tables iterator vecitems[S](x:openarray[S]) : S = for y in x: echo "k ",y yield y iterator tabitems[A,B]( vtb : Table[A,B]) : B = for a,y in vtb : yield y #proc v

Passing iterators as arguments

> However, structural expansion of the template "in place" might still be an > interesting alternative. And how would I do that?

Passing iterators as arguments

Well, I copy/pasted your example, made some minor changes - it wörks :) import tables type Iterable[T] = iterator: T proc iterSomething(xs: Iterable[int])= for x in xs(): echo x template toSecondClassIter(x) : Iterable = let myit = iter

Passing iterators as arguments

I should've posted a minimal example. So in the following example I hope everything becomes clear. Auto not working would be no problem, I could use generics, but I cant get it to work either way, so here is the code: import tables, sugar type Iterable[T] = iterator: T

Passing iterators as arguments

> which yields the error: Error: redefinition of 'iter'; previous declaration > here: ... where did the name 'iter' come from ? The other solution: if a template gives back an 'it' , the 'it' is in scope then as a value (it's a closure) and indeed, a redefinition is not possible.

Passing iterators as arguments

Originally I used araqs code, that produced the `Error: redefinition of 'it'; previous declaration here: ...` error, when I called it 2 times in the same function. Then I tried to fix that by making the iterator anonymous, so there would be no symbol that could collide with itself. This then pro

Passing iterators as arguments

@Araq's code has an extra `it` as a return of template. Your code above is missing it. template toFirstClassIter(x): untyped = iterator it(): auto {.closure.} = for y in x: yield y it Run

Passing iterators as arguments

It might be strange to ask this now, after the thread derailed a little, BUT: Araqs solution to my question worked until now. I tried to call that template a 2nd time2 lines later, which yields to the error: ` Error: redefinition of 'iter'; previous declaration here: ...`

Passing iterators as arguments

It might be strange to ask this now, after the thread derailed a little, BUT: Araqs solution to my question worked until now. I tried to call that template a 2nd time2 lines later, which yields to the error: ` Error: redefinition of 'iter'; previous declaration here: ...` I tried to return it as

Passing iterators as arguments

@Araq > Impressive, unfortunately I can only give you a single upvote. Thank you for your attention. It's all about structural substitution and reduction, something that Nim is very good at (by design). So, let's write a substitution S as S(lside ; rside) where the lside left of the semicolon

Passing iterators as arguments

Read my iterator as a library PR, it supports break and continue

Passing iterators as arguments

I have also been experimenting with templates and for loop macros. I agree they can be a nice workaround for iterator limitations, but it seems different from regular iterators and for loops, for loop macros don't support flow control with `break` and `continue`. Possibly a macro could inspect

Passing iterators as arguments

Impressive, unfortunately I can only give you a single upvote.

Passing iterators as arguments

Well, as long as generic proc's can't be precompiled, in fact rendering an (untyped) AST only, a complete type-check can't be performed. That aside, we define at first our template, the fn1, basically an identity function > template fn1(a: typed) = a and now, a function without generics proc

Passing iterators as arguments

I'm intrigued, please tell me more about how this "witness" works.

Passing iterators as arguments

I agree with that: > But I long for type-checked generic proc bodies. This is exactly the reason why I want to see the witness here. The witness has to compile (aka type-check) together with the proc body. If it doesn't typecheck, then something, proc or witness (template,macro etc.) is ill-typ

Passing iterators as arguments

Thanks, I now understand the feature better. However, I'm even less convinced that it's a good idea for Nim. Even assuming that it is bug-free and works well with the existing features, it would cement our current generics system (generic proc bodies are not type-checked) futher. But I long for

Passing iterators as arguments

@timothee's alias resp. aliassym should provide a witness. > I still don't understand the problem domain and solution space well enough. > Your tests only prove that the feature works as you claim it works. We have > no idea what an "alias"-heavy codecase looks like, how the feature interacts >

Passing iterators as arguments

@cblake: The [original implementation](https://forum.nim-lang.org/t/6649#41330) of `toItr` does not allow for multiple for-loop-variables. The code like `for x,y,z in toItr(points3D("ball")): ...` does not work. Below I reimplemented `toItr` macro to support multiple for-loop-variables:

Passing iterators as arguments

Your example works with copy and paste "out of nothing". I modified the `let par2` accordingly, following @solo989's proposal. I am baffled.

Passing iterators as arguments

It sounds like his approach might be more general, but mine "just works with stock Nim going back 2 years". Anyway, I think that is a comparison better left to @Timothee. The old chestnut of tail recursive "spelling" of "iteration" obviously makes these things all closely related "up to" syntact

Passing iterators as arguments

> @cblake: You're welcome, but really do look into algorithm.nextPermutation. > It is probably >10x faster (although that may only translate to "n+1" with a > use that also grows combinatorially.) I chose permutation generator as a useful and short illustration of recursive iterators, nothing m

Passing iterators as arguments

You're welcome, but really do look into `algorithm.nextPermutation`. It is probably >10x faster (although that may only translate to "n+1" with a use that also grows combinatorially.) Doesn't seem Timothee's pull request or RFC got as far as the symbol name bikeshedding processs. I should maybe

Passing iterators as arguments

@cblake: Fantastic! This is what I have been looking for. Works like a charm. And seems to be a simpler solution that of @timothee. BTW, any pros and cons of `toItr` vs `iterate` from ?

Passing iterators as arguments

@slonik_az, Using only `toItr` from my aforementioned [very close](https://forum.nim-lang.org/t/6649#41330) and regular old "official" Nim 1.4 (earlier works with formerly experimental for loop macros activated), the translation is also almost line for line: proc lex_perm[T](x: se

Passing iterators as arguments

And why does it have to be in fusion? Please create a Nimble package.

Passing iterators as arguments

@solo989 > You could make iterates work with calling cont as a named parameter if you > used this works, thanks!! Will update PR (in fusion) with this and other improvements

Passing iterators as arguments

> I could add it to fusion; it does solve problems as mentioned. Fusion seems to be the right place, thanks.

Passing iterators as arguments

> Your code does not compile in the standard Nim-1.4.0 it does: `nim r --lib:lib main.nim` (or `nim r main.nim` if nim from this PR is in path); the error you have just means it's using nim-1.4.0's stdlib, not the one from this PR > Can your PR be converted to a standalone library? So that the

Passing iterators as arguments

@timothee: Your code does not compile in the standard Nim-1.4.0 /usercode/in.nim(2, 13) Error: cannot open file: std/iterates Run Can parts of your PR be converted to a library? So that the community can benefit from your solutions

Passing iterators as arguments

If I could figure out a clean way to support break and continue with procs simulating iterators I would never use the default style of iterator again. Tuple unpacking of var returns could also be an issue though that might be solvable once views become more stable.

Passing iterators as arguments

You could make iterates work with calling cont as a named parameter if you used let par2 = nnkExprEqExpr.newTree(ident"cont",par) Run instead of let par2 = quote do: # cont = `par` # bug: doesn't work: Error: undeclared identifier: 'cont' (and

Passing iterators as arguments

why don't you just try ? ;-) it works, here's a straightforward port: when true: import std/iterates proc lex_perm(s: auto, cont: proc(_: seq[int])) = if s.len == 0: cont @[] for i in 0..

Passing iterators as arguments

The following python code, a recursive iterator, generates all permutations of a sequence and yields them in lexicographic order. I am curios, whether it can be done in Nim in a similarly recursive way. ## Python code follows... def lex_perm(seq): if not seq: yield seq[

Passing iterators as arguments

> Closure iterators are also great and have their place. Just one tiny thing > missing -- recursion. :-) recursion works out of the box using my iterator-implemented-in-library PR when true: import std/iterates import timn/db

Passing iterators as arguments

@slonik_az \- unless your closure iterator has no parameters or is controlled via global variables, both of which seem uncommon cases, you likely want a closure iterator factory proc anyway. So, this thing you want is, I think, not as commonly useful as what already works.

Passing iterators as arguments

> @slonik_az: "iterator lite" \- zero cost abstraction, compile time > code-substituting and for-loop inverting like the current inline-iterator > > @Araq: it seems to me that they really are "satisfactory", even for you, they > fit your description. I find the concept of zero-cost _inline iter

Passing iterators as arguments

I am fine that the inline iterators are not recursive. This is a fair price to pay for zero-cost abstraction. But I the manual says that the closure iterators cannot be recursive. This, I hope, can be addressed.

Passing iterators as arguments

@slonik_az \- closure iterators are really [very close](https://forum.nim-lang.org/t/6649#41330) to being recursive. @Araq \- we could consider adding that little `toItr` for loop macro to the manual. I think it is useful for less effortful calling of even non-recursive closure iterator factori

Passing iterators as arguments

> The fact that iterators are widely and successfully used in the wild does not > automatically make their implementation satisfactory. For me it does mean this, esp if you then wander off into insolvable problem spaces. So yeah, inline iterators cannot be recursive. Their name gives it away --

Passing iterators as arguments

> @Araq: Both are. Both are used extensively in real world projects. The fact that iterators are widely and successfully used in the wild does not automatically make their implementation satisfactory. Neither inline nor closure iterators support recursion. Inline iterators cannot be passed arou

Passing iterators as arguments

> Neither is being satisfactory supported in current Nim. Both are. Both are used extensively in real world projects.

Passing iterators as arguments

@timothee: A question regarding your Does this RFC allow for recursive iterators? From skimming over the RFC i doubt that the aliasing approach allows for truly 1-st class iterators that support recursion or can be passed as proc arguments at runtim

Passing iterators as arguments

Then how come I understand how it works in Python but not really in your implementation? The very existence of `tyAliasSym` in your implementation implies there is something else going on. In theory aliases can forward to what they alias to during symbol lookup, no type system extensions are req

Passing iterators as arguments

> We have no idea what an "alias"-heavy codecase looks like, how the feature > interacts with the rest of Nim, what the gotchas might be By definition you'll never know until you make this feature available. Same goes for anything: `--gc:arc`, `--experimental:dotOperators` etc. The PR already h

Passing iterators as arguments

I still don't understand the problem domain and solution space well enough. Your tests only prove that the feature works as you claim it works. We have no idea what an "alias"-heavy codecase looks like, how the feature interacts with the rest of Nim, what the gotchas might be, what the theory be

Passing iterators as arguments

> Is there a linked RFC?

Passing iterators as arguments

Is there a linked RFC?

Passing iterators as arguments

I'm resisting the urge to make a fake account just to give this more than one heart.

Passing iterators as arguments

Wow, that looks awesome 👍

Passing iterators as arguments

> By wrapping them in a .closure iterator. closure iterators have limitations (no js, no vm, no nimscript, and have performance overhead) Once again, (every symbol becomes 1st class; defines 0-cost lambda and aliases; inline iterators/templates/etc

Passing iterators as arguments

I guess that makes sense. Thx

Passing iterators as arguments

Inline iterators are basically just an AST transformation as far as I can tell, there's no way you can translate that to a runtime behavior.

Passing iterators as arguments

I see, thank you. I don't want to criticize nim, this is pure curiosity: coming from python it seems very strange to me to not be able to pass raw iterators around. Why was it designed like this?

Passing iterators as arguments

By wrapping them in a .closure iterator. import tables, sugar type Iterable[T] = iterator: T proc iterSomething(xs: Iterable[int])= for x in xs(): echo x template toFirstClassIter(x): untyped = iterator it(): auto {.closure.} = fo

Passing iterators as arguments

Hey, I want to use an iterator as argument for a function. I know, that iterators cant be passed around, and they need to be wrapped in functions. However, I cant get it to work. I something like this: import tables, sugar type Iterable[T] = proc: iterator: T proc