`import foo {.private.}` to allows access to private fields (eg: package-level visibility)
[Does Nim need package-level visibility? - Nim
forum]([https://forum.nim-lang.org/t/4293](https://forum.nim-lang.org/t/4293))
suggested introducing package-level visibility and gave proper rationale for
why something like that; however, as noted, it would add complexity to the
language (eg: no syntax for that was suggested and it's not clear how to extend
the simple * for package-level visibility; likely a macro {.pkg.} would be
needed.
Instead, I'd like to suggest something simpler:import foo {.private.} (syntax
can be discussed) which imports symbols in foo, as if they were declared
public, eg:
foo.nim:
type Bar* = object
field1*: int
field2: float
proc bar(a:int): int = ...
Run
tfoo.nim:
import foo {.private.}
doAssert bar(10) == 10 # we can access `bar` thanks to `{.private.}` pragma
Run
The idea is to provide an escape hatch to visibility, and the user of
{.private.} knows what he's doing. In particular, changes to private symbols
are still allowed and are **not considered breaking changes**.
Since typically {.private.} would be used within boundaries of a library, this
wouldn't result in actual breakages.
Additional use cases are enabled by this feature, notably for debugging
purposes.
Re: Does Nim need package-level visibility?
I'd like to suggest an alternative proposal: see [https://forum.nim-lang.org/t/4296#26721](https://forum.nim-lang.org/t/4296#26721)
Re: runnableExample considered harmful / good feature to deprecate before 1.0?
> no, they indeed belong elsewhere - at the end of the file or in different
> files altogether, potentially to be pulled in by documentation tool - not as
> distractions to the actual implementation.
that pretty much guarantees lower test coverage because people are lazy. Having
unittests (via runnableExamples) right where a symbol is defined lowers the
barrier for adding tests, and makes it easier to write/read/maintain source
code. Also true when procs are moved around. Case in point: D benefits from
excellent test coverage thanks to its unttitest {} feature, which is similar to
runnableExamples. That being said, there's a number of things we can improve in
runnableExamples but replacing it is not the answer.
Re: Possible problem with my bcrypt module?
OK thanks. I updated the github repo with `installExt` for c and h files and that seems to have fixed it.
Re: runnableExample considered harmful / good feature to deprecate before 1.0?
Python DocTests get deleted in the long run (>15 years coding Python).
Re: Quick Start Documentation
Here are some existing docs aimed at Nim beginners beyond the tutorials: * [Nim basics](https://narimiran.github.io/nim-basics/) (active) * [Nim snippets](https://scripter.co/notes/nim/) (active) * [Nim cookbook](http://nim-cookbook.btbytes.com/) (inactive)
Re: Possible problem with my bcrypt module?
Check what Nimble is installing using the \--verbose flag. It likely isn't installing your .c files (the rules were changed recently for this)
Re: How do I make macros that build up and use data structures?
Nim does support your original example, e.g. macro mkParser*(name: untyped, args: untyped): untyped = # process name and args here # generate the setup code(type & parser) with possibly # compile time proc helpers # generate the flag code directly here # or with flag(..) which will be invoked later discard # if you demand autocompletion, this is a hack, but it's a very simple solution: and it describes correctly the flag function in your dsl proc flag*(names: varargs[string], help: string = "") = discard Run If you demand flag to be explicitly defined, use a macro, not a compile time function: this way you can block: assign tmpParser flag(..) Run and generate code working on tmpParser in flag . This way you'll have your macro mkParser and your macro flag with the right signature and autocompletion
Re: Drop RST and join the Markdown train?
> planned to map to proc arguments nobody contends that rST is more powerful, syntax- and feature-wise. This thread is about making a pragmatic choice such that Nim development can reuse a well-known and established format that brings 95% of the functionality and instead allowing compiler, tooling and downstream developers focus their attention on things more impactful.
Re: runnableExample considered harmful / good feature to deprecate before 1.0?
> Put tests in comments so that they are easier to ignore no, they indeed belong elsewhere - at the end of the file or in different files altogether, potentially to be pulled in by documentation tool - not as distractions to the actual implementation. there exists another way to separate the two: by using forward declarations and placing all documentation and other cruft there while implementation lives separately. not sure this is an improvement though, lots of repetition of proc headers.
Re: Does Nim need package-level visibility?
> removal of the "forwarding requirement" oh, yes, please! forward declarations are so 90's! they can be a nice touch to optionally have, for example when exposing a documented public API, but mandatory??
Re: How do I make macros that build up and use data structures?
I think I've found a solution that will work for me (still working through all
the details):
I define `mkParser` and `flag` as `{.compileTime.}` procs. Then when I want to
make a command line parser, in my application code, I define and use a macro
like this:
macro makeParser(): untyped =
return mkParser("some name"):
flag("-a")
flag("-b")
var p = makeParser()
Run
I think it would be cool if Nim supported doing stuff like this without the
wrapping macro syntax, but it's way cool that Nim supports this at all!
Possible problem with my bcrypt module?
My bcrypt module did not compile correctly when I tested it recently as a
dependency. I created a simple test project and made bcrypt a require in the
nimble file. Then I ran `nimble install` and it said it was ok. But with my
test code for bcrypt in my test.nim file, `nim c test.nim` gave an error that
it could not find the file I used in the `.compile` pragma.
However, when I used `nimble develop` and tried again, the test program
compiled fine.
The project is on github at
[https://github.com/runvnc/bcryptnim](https://github.com/runvnc/bcryptnim)
This is how I am using .compile:
{.compile: "arc4random.c".}
{.compile: "blowfish.c".}
{.compile: "crypt-blowfish.c".}
{.pragma: mydll.}
proc bcrypt_gensalt(rounds: int8): cstring {.cdecl, mydll, importc:
"bcrypt_gensalt".}
Run
So originally I was getting this error:
Hint: used config file
'/home/runvnc/.choosenim/toolchains/nim-0.19.0/config/nim.cfg' [Conf]
Hint: system [Processing]
Hint: test [Processing]
Hint: bcrypt [Processing]
Error: unhandled exception: cannot open: arc4random.c [IOError]
Run
But when I cloned my repo, ran nimble install and nimble develop, then went
back and tried to compile the test, it worked with no problem for some reason.
Do I need to change the pragmas for newer version of nim, and if so, can I get
a hint as to what is different for this type of thing? Thanks.
Re: Does Nim need package-level visibility?
> You can work-around this by making all fields private and by introducing > accessor templates, but at this point everything is so tedious that it's > clear the language needs fixing. Sounds like the perfect job for a macro ;)
Re: Drop RST and join the Markdown train?
Didn't know about HastyScribe. Installed and tested. I'll look forward to using this in the future for documentation and potentially course material.
Re: Drop RST and join the Markdown train?
> For example, the amount of single backticks denoting code/verbatim (like it > is in markdown) instead of double backticks (the RST way) is astounding. Single backticks are also an RST feature, it's some custom markup that I planned to map to proc arguments but never did.
Re: runnableExamples should generally be preferred to `.. code-block:: nim`
> True, but I expect editor and IDE devs to put in this additional effort > anyway: recognizing fenced markup code blocks inside comments will spread > because Rust uses it. Again, runnableExamples are not just documentation. They are also tests. So far nobody argued for putting tests into comments.
SHGetSpecialFolderLocation
Why is **SHGetSpecialFolderLocation** commented out in the oldwinapi module? What nim declaration would allow access to that function?
Re: Does Nim need package-level visibility?
I think some `submodule` idea combined with a removal of the "forwarding requirement" is the best way to do it but this needs an RFC. I'm not convinced that more visibility rules would change the language for the better.
Re: Does Nim need package-level visibility?
Isn't it the case though, that this can be solved by not introducing a types module, and instead have smaller modules that implement the individual types in a more separate way, with a clean, orthogonal and complete API? Incidentally, this approach encourages the creation of good unit tests as well, that at the same time: * serve as good examples of API usage * help make the API complete without being redundant * serve as a check against overly complex modules
Re: Loop backward through array -- howto do it brachless
> you have to be careful with assuming that x mod n will be optimized to an and > operation where n is a power of 2 That is a very good hint indeed. In C code often unsigned ints are used, so its a plain "and" op, but in Nim we most often use signed ints. I have not benchmarked these statements myself, and speed was not really a concern. It was more about simple code in this case, I typed i = (i + 1) mod 4 Run for the forward loop, as it looks simpler for me than j -= 1 if j < 0: j = 3 Run and I assumed that the mod operation at least would not be slower, which seems to be wrong indeed. Indeed simple if ... else statements can be very fast due to cmov instruction, I tested it recently: [https://forum.nim-lang.org/t/4142#25901](https://forum.nim-lang.org/t/4142#25901)
Re: runnableExamples should generally be preferred to `.. code-block:: nim`
This isn't specific to Nim. Nim's runnableExamples is a close cousin to D's unittest blocks; as in D, unittest blocks are not embedded in documentation even though they end up in documentation and can be run from html. A difference is that unittest block appears right after proc body, rather than right after proc header (which has pros and cons but the self contained aspect of it is nice). > I expect nim doc to throw an error when it tries to compile the > runnableExample block, unless it is ignored. But it doesn't. Am I missing > something here? thanks for reporting this; but please report as github issue next time instead of forum to increase change it gets fixed; just sent out [https://github.com/nim-lang/Nim/pull/9262](https://github.com/nim-lang/Nim/pull/9262) to fix this particular case; the more general case would be to fix [https://github.com/nim-lang/Nim/issues/9216](https://github.com/nim-lang/Nim/issues/9216) but that's a separate discussion
Re: How do I make macros that build up and use data structures?
Btw, you can be even more ambigious with your dsl:
mkParser("Some Program"):
-a
-b --optionb "This is an option"
Run
You can also have this.
Now, writing a macro for this might be a little bit complicated, but it's a
good example that macros can be more expressive
I think the first approach is fine: flag is part of your dsl, so I am not sure
what would a proc/macro flag would do outside of it (e.g. what will help do?)
You should document your dsl in the mkParser/module docs anyway
Re: How do I make macros that build up and use data structures?
I guess you can always make a dummy flag proc with the same signature for doc/completion, but for the implementation, the first approach still seems simpler to me
Re: Does Nim need package-level visibility?
I don't want package level visibility, because once implemented it will be an excuse to not implement proper cyclic imports support :).
Re: Does Nim need package-level visibility?
@nepeckman, I've considered this work-around myself, but the problem is that it doesn't work for object fields. You can select which procs and types to export, but you cannot alter the fields that are marked as public.
Re: Does Nim need package-level visibility?
Ah yeah, that makes sense. I've been primarily using procs to access object fields, so I haven't run up against that problem. I definitely see the utility of package level visibility, but I wouldn't want to introduce unnecessary complexity to the language. I really like the cleanliness of `*` exporting; I was skeptical at first but Nim won me over.
Re: Does Nim need package-level visibility?
Would it make sense to have an internal type, and a distinct public alias? The implementation of the public functions use a private converter from the alias to the internal type. Never tried it though.
Re: Does Nim need package-level visibility?
I don't know if this is sufficient for your purposes, but I've emulated package-level visibility in a side project I've been developing. Each folder has a `.nim` file that imports the other modules in the folder, and exports the public interface of that folder. So package-level visibility is emulated through public symbols that are not forwarded in `.nim`. I know this method comes with some drawbacks (requires certain project structure, not enforceable by default, etc), but I figured I'd mention it in case it helps.
Re: Loop backward through array -- howto do it brachless
Out of curiosity, have you actually checked whether the branchless version is faster? For me, using the following code is actually faster by a factor of almost three than any of the variants using bit masking or modulo: j -= 1 if j < 0: j = 3 Run This is most likely because branch predictors tend to be very good at such regular branching patterns. Results may vary if there's a lot of interleaved code, but as always: measure first, then optimize. I'll also note that you have to be careful with assuming that `x mod n` will be optimized to an `and` operation where `n` is a power of 2. It generally will _not_ happen, as the compiler can only rarely prove that `x` is non-negative. The reason is that in C99, integer modulo is defined to yield negative results for negative `xx`. You may get something like this for `x mod 8`, where `x` is a 32-bit integer. movl%edi, %eax sarl$31, %eax shrl$29, %eax addl%edi, %eax andl$-8, %eax subl%eax, %edi movl%edi, %eax Run A safer approach would be a template that special-cases the case where the modulo is a power of 2, e.g. via: when (m and (m-1)) == 0: # m is a power of 2 if non-negative. x and (m-1) else: # modulo code for arbitrary x and m Run Alternatively, cast to `uint` before the `mod` if you know that the value is non-negative, and back to an `int` afterwards.
Does Nim need package-level visibility?
I'm getting more and more convinced that Nim needs package-level visibility (or some other similar mechanism). In our own codebases, we are repeatedly running into instances of the following problematic pattern: 1) You try to break your code into multiple small modules. 2) Your modules need to work with the same data types, so you introduce a `types` module. 3) The `types` module is useless on its own. It provides only type definitions, so all types and fields must be public. 4) Public fields are appropriate for your own code, but they are not appropriate for the users of your package. At this point, you consider either adopting some naming convention for "private" exported fields or you merge all of these small modules into a single huge module representing the entire package. The situation is made worse by the lack of support for cyclic imports, because this is an additional major reason to introduce such a `types` module. There are other problems affected by the lack of package level visibility, but they might have other better solutions. For example, it's currently not possible to write unit test files that test the private routines defined in another module, but this may be better solved with one of the following options: T1) Creating a new unit test framework that is able to collect test cases defined across your entire program (each module defines its own tests) T2) Testing private routines is probably not a great practice anyway. Even the "private" helpers can be considered an API that lives in its own private module that has public exports.
How do I make macros that build up and use data structures?
I'm trying to make an argument parser as an excuse to learn how to use macros
and templates. But I'm getting stuck.
As a starting point, my goal is to use macros and templates to convert this:
var p = mkParser("Some Program"):
flag("-a")
flag("-b", "--optionb", help = "This is an option")
Run
into this:
var p =
type
Parser[T] = object
helptext*: string
Opts = object
a*: bool
optionb*: bool
var tmp = Parser[Opts]()
tmp.helptext = """Some Program
-a
--optionb,-b This is an option
"""
tmp
Run
(I'll eventually want to add code to actually parse arguments, but being able
to do this first step is currently a hurdle for me).
The puzzle is how to implement these two things:
* `mkParser`
* `flag`
Also, I want code-completion with the `flag` procedure.
# What I've tried
I wish I had saved each of my non-working attempts, but I only have my memory,
unfortunately.
**single macro**
I first attempted to define `macro mkParser` which reads through it's child
nodes, looks for calls named `flag` and interprets the arguments -- all within
the body of the macro. While this worked (at least as far as I pursued it), it
suffers from not having a `flag` proc/macro/template defined anywhere. That is,
if I generate the documentation, there's no `flag` proc/macro/template.
This approach seems unsustainable.
**template + macro**
I also attempted with `template mkParser` and `macro flag`. This helped
solidify for me that templates are really just substitutions and the code
within them represent pretty much what the code will look like at runtime.
The main sticking point with my problem is that I want `mkParser` to iterate
through each of it's children and evaluate them to add information to a list of
flags that I'm building up.
In pseudo code:
macro mkParser(body) =
var listofFlags = @[]
for child in body:
let data = child()
listofFlags.add(data.name)
result = listofFlag.makeSomeTypeDefinitions()
Run
**macro + macro**
I also attempted with `macro mkParser` and `macro flag` but it had essentially
the same problem as the template + macro attempt.
I've read and re-read how the `unittest` module works, though it uses some
deprecated features so I'm not sure it's the best model.
**If I could figure out how to run through the children of a macro /template to
build up some data structures** then I feel like I could build the AST pretty
easily (either by hand or with `quote` using something like this):
type
Flag = object
name*: string
kind*: string
macro mkOptType(flags:seq[Flag]): untyped =
var flagdefs:string
for flag in flags:
flagdefs.add(&"{flag.name}*: {flag.kind}\n")
result = quote do:
type
Parser[T] = object
helptext*: string
type
Opts = object
`flagdefs`
var p = Parser[Opts]()
p
)
Run
Re: runnableExample considered harmful / good feature to deprecate before 1.0?
> I think that something like Python's doctest is cleaner than runnableExamples: And I think it's much more messy and I lose the syntax highlighting in my editor. ;-)
Re: runnableExample considered harmful / good feature to deprecate before 1.0?
I think that something like Python's doctest is cleaner than `runnableExamples`:
def factorial(n):
"""Return the factorial of n, an exact integer >= 0.
>>> [factorial(n) for n in range(6)]
[1, 1, 2, 6, 24, 120]
>>> factorial(30)
26525285981219105863630848000
>>> factorial(-1)
Traceback (most recent call last):
...
ValueError: n must be >= 0
Factorials of floats are OK, but the float must be an exact integer:
>>> factorial(30.1)
Traceback (most recent call last):
...
ValueError: n must be exact integer
>>> factorial(30.0)
26525285981219105863630848000
It must also not be ridiculously large:
>>> factorial(1e100)
Traceback (most recent call last):
...
OverflowError: n too large
"""
import math
if not n >= 0:
raise ValueError("n must be >= 0")
if math.floor(n) != n:
raise ValueError("n must be exact integer")
if n+1 == n: # catch a value like 1e300
raise OverflowError("n too large")
result = 1
factor = 2
while factor <= n:
result *= factor
factor += 1
return result
if __name__ == "__main__":
import doctest
doctest.testmod()
Run
[https://docs.python.org/3.7/library/doctest.html](https://docs.python.org/3.7/library/doctest.html)
Re: Systems Languages: An Experience Report
No and IMHO it shouldn't ever become a tag/concept/type-system-feature, it's just some archaic mechanism before multi-threading was discovered, ymmv.
Re: runnableExample considered harmful / good feature to deprecate before 1.0?
Doc comments themselves are annoying in the source code but outsourcing them to different files is not a good way of doing things either. What's currently been missed in this discussion here IMO is that `runnableExamples` are not just documentation, they are also tests and I've yet to see somebody arguing for "Put tests in comments so that they are easier to ignore".
Re: how to increase velocity for merging PRs?
Well I'm back from vacation and agree with all that's been said here. :-) Will write a code-owners document and also consider cdome's proposal. Stay tuned.
CMS
Hello, I would like to know if there is a CMS for this type of language ? Thank you :)
Quick Start Documentation
Hello all, I've been thinking of the ideal introduction to Nim, for people who just heard about it. The current documentation (the manual) can be a little "intimidating". Although it is very thorough and explains rationale for things, it is a far cry from being a "Nim in a nutshell" type of reference. I envision the following: a user visits the website. There is a getting started page, that lists: Getting started with Windows, Mac, Linux, Android, iPhone, (wasm?) in each platform, you can see how to do things, preferably the portable way, but also the complementing native api's, like bindings to cocoa api for mac, or the lean windows api, etc. This is from a platform point of view. There should also be an aspect point of view: So, you want to make a command line tool. So, you want to make a game, So, you want to make a web application, So, you want to make a GUI Desktop application. So you want to make an android app ...etc With step by step instructions, following the recommended workflow: start with git/hg source control, then how to do unit testing, how to best use the vscode ide (shortcuts and productivity), and how to debug your code. But most important is to show the general steps (regardless of editor) of how to compile, how to quickly get a skeleton application up and running, then where to fill the missing code. (your actual app). So to summarize, having a getting started documentation that shows two different perspectives: platform and purpose. I would love to hear your ideas and how to make this happen easily, and cooperatively. (one idea is to have a wiki, at first, so we can share our knowhow easily) Another related issue is that many programmers already know a mainstream language, and its sdk. We can simply show the corresponding syntax in nim (when available), and the packages (or stdlib types) that provide the equivalent functionality - this can look like a reference table, or a "cheatsheet", and the programmer will then dive deeper to understand how to use them. Nim to programmers with background in Java, or C#, or python, or ruby, or nodejs, or c++, etc. what are your thoughts? Would this be a good way to structure a beginner's understanding and get quick results for him/her? Would you like to cooperate and make this happen?
Re: runnableExamples should generally be preferred to `.. code-block:: nim`
> Everything is always a "tooling" problem. [...] If by that you mean that people sometimes tend to conveniently delegate things they don't want to deal with to tooling in a thoughtless way, I agree. I just don't think it's like that in this case. > Ah we play the noun game now. Runnable examples are code and should be > written as code, don't separate identical concerns. My definition of the separate concerns at hand is just different from yours. Mine are "define what a module/proc etc. does" (regular source code) vs. "help people understand what it does" (docs including example code). That's not a noun game, that's a different view. I also like that the Nim project keeps testing code separate, because testing is another separate concern. > > Searching for runnableExamples in Nim's github project even finds some code > > where it is used outside of any doc comment context, which means it is > > silently ignored. > > That's just not true. OK now I'm confused: found this [https://github.com/nim-lang/Nim/blob/3d60f1b0af6fc3d8d9446a87bd26ff054562fa20/tests/generics/t8694.nim](https://github.com/nim-lang/Nim/blob/3d60f1b0af6fc3d8d9446a87bd26ff054562fa20/tests/generics/t8694.nim) which looks like this (excerpt): when true: # Error: undeclared identifier: '|' proc bar[T](t:T): bool = runnableExamples: type Foo = int | float true echo bar(0) Run When `int` in the type declaration is replaced with `nosuchtype`, I expect `nim doc` to throw an error when it tries to compile the `runnableExample` block, unless it is ignored. But it doesn't. Am I missing something here? Tested with the latest devel, installed with choosenim.
Systems Languages: An Experience Report
Has there been any effort in the Signal handler / safety? See: [https://github.com/nim-lang/Nim/issues/7042](https://github.com/nim-lang/Nim/issues/7042)
Re: AddOns to Nim? (a bit similar to rubygems.org + projects)
> I remember dom96 has had something like that but it was a bit cumbersome to > use What is your concrete problem with nimble? (Nim package manager) I think such unfounded critics makes absolutely no sense. I have used rubygems.org in the past and have the feeling that using nimble is very similar. > My major problem was and is lack of time For this topic we can not really help you. Learning Nim is some effort, but at first you can ignore macros, then it is really easy. I think much easier than learning other powerful languages as C++, Rust or Haskell. Note that serious Ruby devs generally have to learn how to include C code in their Ruby programs which is not fully trivial -- I did that for using CGAL or Boost libs from my Ruby code. When coming from languages like Ruby, then there are two major problems: 1: Transferring Ruby OOP code to Nim is some work, more as I first expected. Problem is OOP and missing data types in Ruby. 2. Not every needed Lib is already available in Nim, for example RTree was missing (I wrote that myself) and Constrained Delaunay Triangulation is still missing (I plan to write it, but maybe interfacing to CGAL is sufficient. (For Rust you would have already both (spade package) and many more packages.)
Re: What does asyncCheck do?
Thanks guys, it all makes sense now.
