trying to parallelize zippy file extraction on Windows

I've moved to my old "faster" Windows PC. Despite being an older model: Intel(R) Core(TM) i5-7300U CPU @ 2.60GHz 2.70 GHz with just 8 GB RAM (older and I assume slower SSD, maybe not), it's not plagued by the same CPU-hungry Anti-Virus Anti-Malware software running in background. The situation i

trying to parallelize zippy file extraction on Windows

Ok... I will try to my other older but "free-of-contraints" Windows laptop at home (there I had 1.6 amd I've just upgraded to Nim 2.0). Regarding the false AV positives you're right... but I'm particularly cautious because I'm pleased to use Nim for doing occassionaly some useful stuff at work..

trying to parallelize zippy file extraction on Windows

Hi to all, I was trying to speed-up zip archives decompression by running more threads in parallel. I have a Intel(R) Core(TM) i5-10310U CPU @ 1.70GHz 2.21 GHz laptop with 16 GB RAM and SSD, so not exactly a power horse, but it still have 4 Cores, 8 Threads. I've a folder containing 4x .zip file

Pure Nim FFT library

I'm not sure, but I would expect that division by N should included in the outcome, without your intervention. I would look at the "competitor" implementation. Y = fft(X) X = iftt(Y)

A few (perhaps naive) questions

Leaving a community just because two out of several answers are not pleasing you (BTW irony != offense) seems to me a bit of over-reaction. Nim is a wonderful programming instrument... if you like it... enjoy it. It's like stopping to play the piano, only because you had the impression a couple

Allowed key types in std/tables

@awr1 thanks for the explanation @xigoi thanks for further clarifications. Yes, you're right, in Python _values_ of certain data types are not mutable (like tuples' content), all variables can always be re-assigned, there is no equivalent for `let =`, I did express my point wrongly.

Allowed key types in std/tables

Hi to all, in the std/tables documentation is stated: `If you are using simple standard types like int or string for the keys of the table you won't have any problems, but as soon as you try to use a more complex object as a key you will be greeted by a strange compiler error (...)` Since I nee

fetching data from many small .txt files

@cblake, sorry for the late follow up. Each object is described by a .TXT file, containing only grid pixels (one line is a grid pixel) having a numeric value (float). No, the grid is not the same across the .TXT files, and a pixel may contain more than one object values. The subset I used to run

fetching data from many small .txt files

Amazing! Yeah, new release it solved. I removed cligen 1.6.1 and installed 1.6.2 on my Work laptop (the one with Nim 1.4.4), and compilation was just fine. I also ran a few tests on subset of data and (I needed to throw out the first run, as always) the cligen-powered version is around 20% faste

fetching data from many small .txt files

@cblake Well, I tried a bit more with your cligen powered memory map code,... **Work Laptop with Windows 10 (Nim 1.4.4)** : initially compiler complained about a missing c header... but I was running an old version of cligen so I uninstalled it and installed again 1.6.1 with nimble. The missing

fetching data from many small .txt files

Hi, thank you all for your suggestions. @cblake, your comment `{ EDIT1: but I am a bit skeptical that you timed things right as with the code you showed you are unlikely to parse & print as quickly as even a SATA SSD never mind an NVMe SSD.` was most likely right. The Windows PC I'm using is a D

fetching data from many small .txt files

Hi, I was wondering if there is a way to optimize this kind of data processing. I'm using Windows 10, and I need to parse many relatively small .txt files and rearrange their content in a table, where table index is a pair of 2D grid coordinates and table value is a sequence of strings. The orde

EU Cyber Resilience Act impact on Open Source community

Good rules are better than no rules... but bad rules are worse than no rules. I think that rules advocating for transparency or for freedom of choice or breaking monopolies are good rules. For instance forcing Microsoft to not pre-install Internet Explorer in every windows PC, and let user decid

EU Cyber Resilience Act impact on Open Source community

I was asking if someone had read about those concerns, the "papers" I've posted are much more approachable than the EU regulation text, likely not for the laymen (or boring to death...) Rules are important... think about how software licences like GPL (and later more liberal ones like BSD and M

EU Cyber Resilience Act impact on Open Source community

The Eclipse (second link) open letter to EU parlament is co-signed by many Open source players. European citizens could try to reach out for some european parlament member by email, inviting him/her to seriously care of these arguments. That is exactly what I"ve done. (...) The undersigned orga

EU Cyber Resilience Act impact on Open Source community

@nasl Yes, as it has been highlighted by someone, this is precisely the model of company like e.g. Red Hat. Commercial definition covers a wide range.

EU Cyber Resilience Act impact on Open Source community

Look also at Eclipse's open letter

EU Cyber Resilience Act impact on Open Source community

Has anyone read about this? It seems that a controversial EU proposal law has received great attention from some of the Open Source major players, most notably PSF, and Eclipse.

What GPT-4 knows and thinks about Nim

Hi to all, I seldomly write on this forum, but I read it and I still like Nim language and its community very much. The argument of this thread is intriguing and broad. I liked @Zoom thoughts/comments. I've played myself a bit with ChatGPT-3 and my feelings are mixed about it. I think is a signi

csv column type setting

Thank you for your answer. I knew your package and I pay great respect for the incredible work you've done (and for the one _mratsim_ has done with `arraymancer`). Since `datamancer` codebase is quite large and use macros and some other more advanced / performance optimized stuff (buf reading),

csv column type setting

Hi, I was looking for something simple (no macros), relying just on Nim's standard libraries, to dynamically infer column type and assign it to the proper seq type. I know that there are pretty advanced packages like `datamancer` out there, but I wish to build my own small toy, just to better u

Hello Javascript!

I don't think @araq reaction was because of a boring question. He was reacting to a, frankly speaking, a bit ridiculous claim about his (or let's say Nim team) alleged responsibility to "energy wasting", because "every KB counts". The argument is poor because in telecommunications (I know the fi

how to properly init a Table of seq

@cblake I didn't know the `pairs iterator` trick... that's really nice. If I understand you point correctly, you suggest to not use `newSeqUninitialized[int](n)` because it's a bit exotic (maybe it will disappear in future Nim versions?), not so relevant peformance-wise, and `newSeq[int](n)` is

how to properly init a Table of seq

@ElegantBeef Thank you. Yes, the hasKeyOrPut solution is more compact and elegant.

how to properly init a Table of seq

Hello, I wish to use a Table of seq[int] to store values associated to some keys across different .txt files. I learn how many files I need to scan at runtime, so I was thinking to declare the size of seq of each table entry with: newSeqUninitialized[int](n) Run Howev

collecting keys and randomly picking values from tables

actually I like let keys = toSeq(myTable.keys) Run even more, I was writing my reply before seeing your solution

collecting keys and randomly picking values from tables

what about collect (you need to import sugar) and iteration? import std/tables, sugar var a = { 'y': 1, 'm': 2, 'c': 3, 'a': 4, }.toOrderedTable let keyseq = collect(newSeq): for key in a.keys: key assert keyseq == @['y'

Anti-virus at work prevents the use of Nim binaries

Anyway is really changing over time. I decided to retry Microsoft Defender reaction to Nim > 1.4.4 and I found no issue. I've recompiled a bunch of small programs I developed at work on my home laptop with Nim 1.6.6 ad none was detected as malicious by MS. So I made them to be scanned also on my

Anti-virus at work prevents the use of Nim binaries

Did we miss the egg of Columbus? Indeed for MS Security I found: and probably we have to go for Report a false positive/negative to Microsoft, consid

Anti-virus at work prevents the use of Nim binaries

Ah, OK, I thought it was intentional. I tested choosenim latest version, 0.8.4 both on my home windows laptop (Microsoft Defender), and VirusTotal online checker, so it seems choosenim causes no alarms.

Anti-virus at work prevents the use of Nim binaries

I ran in the same problem. In the office we have Windows Defender, no options to disable it. I decided to install the latest Nim version not triggering the malware false alarm, i.e. Nim 1.4.4. After downloading chosenim.zip (to me it worked with chosenim 0.7.4, this is the version dom96 is link

Mastering Nim: A complete guide to the programming language

@pietroppeter Typos? Araq has compiled the book with -d:releaseNoTypos... ;)

Mastering Nim: A complete guide to the programming language

That's pretty outreageous! I checked, and Masterig Nim is available on amazon co.uk .it, .fr... and not .de??? I hope it's temporary. However for european potential buyers, you may consider alternatives to .com. I'll probably buy it too... I'm just a bit scared about print quality. I really hope

Please help me understand the performance difference between Nim/Rust in this (contrived) example

I was thinking something like @shirleyquirk... and indeed it runs much faster. I don't post absolute values because I'have slower PC, older Nim version, but in relative the time reduction is evident. At the end I guess it's like providing a lookup table to compiler. Modifying a little @sky_khan

passing tab separator as option value

Thank you @xigoi, @cblake, unfortunately on Windows (my environment) the smart trick to pass the literal Tab to program cmdtest --sep:$'\t' Run on CMD shell produces $'\t'a$'\t'test Run and I didn't yet found alternatives. However on Windows Power

passing tab separator as option value

Is there a way to pass \t as "tab separator" as on option value (e.g. to set it as separator for a csv output), or the only way is to check against raw string, like this: > > import std/parseopt, std/strutils > var > sep: string > p = initOptParser() > while true: >

Beware: phishing attempts on Nim users (PSA)

Wow, it's an homographic attack, isn't it? They've copied your avatar too. For certain stuff (I think about e-mail and http clients, e.g. browsers) it seems to me we still have naive implementations in place. If you look to URL on the address bar on Chrome, it's almost impossible to notice the d

Seeking advices for a C programming book

Hi to all! First of all I want to thank all the passionate Nimmers spending their time to spread their knowledge about Nim and programming in general... Araq (waiting for your book on Nim), Dominik (I bought your book, wonderful!), Dr. Salewski, Narimiran, Mratism, Vindaar, just to name a few. S

There is no finish.exe in the NIM installation zip file

A viable alternative (at least for me) was to use choosenim. You have to go to: and download latest choosnim (I didn't have any problem with choosenim, but I used 0.7.4 when I performed this procedure) choosenim is a package manager, so it's useful

linux moving to C11

Linus is famous to blame C++ and supporters of C++ for Linux Kernel. Here you can find his arguments about (2007 and 2004). Back in 2004 he didn't seem to trust on C++ compilers too. Nowadays he really seems to still hate C++ and more open to consid

linux moving to C11

This is just for curiosity. It appears Linux will move to C11. What C version does Nim currently compile to? Will this migration of Linux to C11 have any foreseeable impacts on Nim?

To have nre regex usage

You can discover the data type yourself using something like echo(type(mat)) Run eventually taking advantage of wonderful Nim's playground

Nim FAQ

Awesome work. Really appreciated.

Benchmarking

I've some problem with site rendering on Safari on my MacBook Pro (apparently the flags to filter results with / without parallelization do not work), but not with Android smartphone. It seems that a lots of Rust results are with parallelization, while Nim has just one for spectral-norm (much sl

zlib.h: No such file or directory error on Windows

Thank you sls1005!

zlib.h: No such file or directory error on Windows

Sorry, I send it again in code block: PS C:\Users\Andrea> nim c -r "c:\Users\Andrea\Documents\Nim\working_code\zip_import.nim" Hint: used config file 'C:\Users\Andrea\.choosenim\toolchains\nim-1.6.0\config\nim.cfg' [Conf] Hint: used config file 'C:\Users\Andrea\.choosenim\t

zlib.h: No such file or directory error on Windows

Hello everyone, I've installed successfully zip 0.3.1 with Nimble on Windows. I'm using Nim release is 1.6.0 (but I have same problem on another PC with Nim 1.4.4). When trying to build a program importing zip module, it fails. Minimal code to trigger the problem is: import zip/zip

Trojan:Win32/Wacatac.B!ml

Well apparently in the paste MS hit also Chrome. (...) A well-known example of a false positive was when Microsoft Security Essentials tagged Google Chrome as malicious. The software removed Chrome from around 3,000

Trojan:Win32/Wacatac.B!ml

I share most of concerns of Coffeepot. It's not Nim's fault, but it can hinder its adoption for sure... and it's a pity. Nim is a fantastic programming language and I think it could be used effectively also on Windows platform, even in a not "serious-developer" corporate environment, today using

Why learn NIM

You are right. Maybe is not totally off-topic because DontForceMe said of being interested to data Science... and Julia came out as a good language for that. But the argument is mostly off-topic I agree. I apologize, my mistake. Anyway to me it was a useful discussion. It changed my initial poin

Why learn NIM

I used Matlab in the past (many years ago), and I recall it was using index start at 0, but I remember wrong (probably I made confusion with 2D plotting, where I was normally defining 0 : N-1 array as discrete X axis). So, OK, this seems the standard way for scientific-targeted language (excludi

Why learn NIM

Yes, Julia is an interesting and nice language... though I really do not understand why a programming language made "by scientists, for scientists" has decided that index of strings, arrays etc. starts with 1 and not with 0. That's probably one of its major "design quirks".

Why learn NIM

I agree with HJarausch. Among new system languages NIM has extraordinary potential because is smart and flexible, without be intimidating as other ones. Also it has very fast compile time and produce small binary files that is a huge benefit. You may also think to write your own superfast Python

XML parsing performance

You're a gentle community with rookies... I like it. :)

XML parsing performance

Thank you for your suggestions ynfle. Yeah, I probably should move to 1.4.6 and learn more about compile options.

XML parsing performance

WOW! Fantastic !!! With -d:danger option the execution time of the posted code dropped to slightly more than 3 secs. If applied to the version using shallowCopy, even less, around 2 secs and something! Thank you SolitudeSF. P.S. I just hope that applying -d:danger did not modify the space-time c

XML parsing performance

Hi to all, I'm new to NIM, but for what I've seen so far, I like it a lot. I really hope NIM's community and NIM adoption will grow up. I was wondering if I could use NIM for some specific tasks I have to deal with, and one of these is (huge) XML files parsing. I've seen a bit the online documen