I know there’s been some talk <https://github.com/joyent/libuv/issues/931> about adding streaming readdir() to libuv recently, but that’s of little use to you now. There are also some modules (like Thorsten Lorenz’s readdirp) that have streaming interfaces, but that only works if you want a recursive readdir(); it’s no help if you’ve got one enormous directory (like you do). Unless you want to write your own native module that follows the approach described in that libuv thread, I think you’re out of luck. Sorry! :/
F On Tue, Oct 7, 2014 at 6:04 PM, Aseem Kishore <aseem.kish...@gmail.com> wrote: > Hi there, > > I have a directory with a very large number of files in them (over 1M). I > need to process them, so I'm using fs.readdir() on the directory. > > The problem is, fs.readdir() returns everything at once, causing my script > to suddenly consume >1 GB of RAM. > > AFAICT, there's no way to stream this list instead of returning it all at > once. Is there anything equivalent that I can do? > > Thanks! > > Aseem > > -- > Job board: http://jobs.nodejs.org/ > New group rules: > https://gist.github.com/othiym23/9886289#file-moderation-policy-md > Old group rules: > https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines > --- > You received this message because you are subscribed to the Google Groups > "nodejs" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to nodejs+unsubscr...@googlegroups.com. > To post to this group, send email to nodejs@googlegroups.com. > To view this discussion on the web visit > https://groups.google.com/d/msgid/nodejs/CAETHeihHkaQAbj9Ug5MMeNR6MHPR%3DfV21F%3DaMYMHQ-cNsQakPw%40mail.gmail.com > <https://groups.google.com/d/msgid/nodejs/CAETHeihHkaQAbj9Ug5MMeNR6MHPR%3DfV21F%3DaMYMHQ-cNsQakPw%40mail.gmail.com?utm_medium=email&utm_source=footer> > . > For more options, visit https://groups.google.com/d/optout. > -- Job board: http://jobs.nodejs.org/ New group rules: https://gist.github.com/othiym23/9886289#file-moderation-policy-md Old group rules: https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines --- You received this message because you are subscribed to the Google Groups "nodejs" group. To unsubscribe from this group and stop receiving emails from it, send an email to nodejs+unsubscr...@googlegroups.com. To post to this group, send email to nodejs@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/nodejs/CAHG3qKoE484fVCmP5Tzqwv-G1k82Z%2B0OMypt7vc%3D_UA%2B2gBtig%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
