[nodejs] Re: Database with streaming blob support

2014-05-20 Thread Luca Morandini
On 21/05/14 00:00, Matt wrote: What options are out there currently for streaming blob support in Node.js databases? I know that MongoDB has GridFS which seems to support streams. Are there any others? I suppose you mean "DBMS supported by Node,js", rather than "Node.js databases". Anyway,

Re: [nodejs] Poor WriteStream Performance

2014-05-20 Thread Denys Khanzhyiev
I will agree that WriteStream shows some strange performance loss. It may be explained by WriteStream's overhead for buffering and eventing. But that overhead should be almost same for equal amount of write calls. But it is not so. I just have tested with much bigger DUMMY_MESSAGE - it is still 2 t

Re: [nodejs] Poor WriteStream Performance

2014-05-20 Thread Denys Khanzhyiev
this variant is more efficient. function benchmarkWriteStream() { const stream = fs.createWriteStream( './dummyWriteStream.log', { encoding: "utf8", mode: parseInt('0644', 8) } ); const startTime = new Date(); var i = NUM_WRITES; (function doWrite(){ if(i--) {

Re: [nodejs] Poor WriteStream Performance

2014-05-20 Thread Denys Khanzhyiev
Or even simplier (function doWrite(){ if(i--) stream.write(DUMMY_MESSAGE,doWrite); else stream.end(''); })(); 2014-05-20 22:45 GMT+03:00 Denys Khanzhyiev : > this variant is more efficient. > > function benchmarkWriteStream() { > const stream = fs.createWriteStream( > './dummyW

Re: [nodejs] Poor WriteStream Performance

2014-05-20 Thread Jeff Jolma
Hi Aria. I see similar results even after using stream.on('finish'). (I removed the async benchmark altogether.) $ node --harmony benchmarks.js Running benchmarks with 10 writes fs.writeSync took 0.32 seconds writeStream.write took 4.073 seconds $ node --harmony benchmarks.js Running ben

Re: [nodejs] Poor WriteStream Performance

2014-05-20 Thread Forrest Norvell
I am surprised at the performance hit this buffering causes. Should it really take 4.5 seconds to write 100,000 lines and 290 seconds to write 200,000 lines? Node 0.12 will have better support for writev / corking and uncorking, which will allow you to coalesce writes and regain some of the effici

Re: [nodejs] Poor WriteStream Performance

2014-05-20 Thread Jeff Jolma
Hi Denys. Thanks for taking a look. 1. Yes, that is a bug in the async benchmark. Fixing that doesn't change the timings I see, though. 2. Yup, but I am surprised at the performance hit this buffering causes. Should it really take 4.5 seconds to write 100,000 lines and 290 seconds to write 2

Re: [nodejs] Poor WriteStream Performance

2014-05-20 Thread Aria Stewart
On May 20, 02014, at 13:29, Jeff Jolma wrote: > Hello. > > I am working on a node app and found that its performance bottleneck is > logging. And digging deeper, it looks like the root bottleneck is on the > WriteStream. > > I ran some quick benchmarks of the WriteStream performance, and it

Re: [nodejs] Poor WriteStream Performance

2014-05-20 Thread Denys Khanzhyiev
1. Your async code is incorect as you close file before all write operation complete. 2. WriteStream buffers written data until other end is ready to consume data. So you actually first fill memory with data and then write it to file. 2014-05-20 20:29 GMT+03:00 Jeff Jolma : > Hello. > > I am wor

[nodejs] Poor WriteStream Performance

2014-05-20 Thread Jeff Jolma
Hello. I am working on a node app and found that its performance bottleneck is logging. And digging deeper, it looks like the root bottleneck is on the WriteStream. I ran some quick benchmarks of the WriteStream performance, and it seems really slow. Can someone help me understand why it is t

Re: [nodejs] Database with streaming blob support

2014-05-20 Thread Aria Stewart
On May 20, 02014, at 11:27, Matt wrote: > > On Tue, May 20, 2014 at 11:00 AM, Aria Stewart wrote: > LevelDB, of course > > I don't see the support in LevelDB for treating a single value (i.e. > db.get(key)) as a true stream efficiently. It appears that one 'data' event > contains the entire

Re: [nodejs] Database with streaming blob support

2014-05-20 Thread Matt
On Tue, May 20, 2014 at 11:00 AM, Aria Stewart wrote: > LevelDB, of course I don't see the support in LevelDB for treating a single value (i.e. db.get(key)) as a true stream efficiently. It appears that one 'data' event contains the entire value. Am I reading the docs wrong? I'm looking for lar

Re: [nodejs] Best way to search through a big plain object

2014-05-20 Thread George Stagas
2014-05-19 5:25 GMT+03:00 Joshua Lunsford : > Just remember that you have 16.66ms to do any calculations to produce a > fluid 60 frames per second. > Not true. If you separate rendering from value updates, you can get away with 10 updates/sec or even less for certain objects. You don't really n

Re: [nodejs] Database with streaming blob support

2014-05-20 Thread Aria Stewart
On May 20, 02014, at 10:00, Matt wrote: > What options are out there currently for streaming blob support in Node.js > databases? > > I know that MongoDB has GridFS which seems to support streams. Are there any > others? LevelDB, of course. signature.asc Description: Message signed with O

[nodejs] Database with streaming blob support

2014-05-20 Thread Matt
What options are out there currently for streaming blob support in Node.js databases? I know that MongoDB has GridFS which seems to support streams. Are there any others? -- Job board: http://jobs.nodejs.org/ New group rules: https://gist.github.com/othiym23/9886289#file-moderation-policy-md Ol

[nodejs] Re: Node.js in ERP-land

2014-05-20 Thread Yujie Zhou
Thanks, Bruno. I wish your new product a big success! 在 2014年5月20日星期二UTC+8上午5时11分25秒,Bruno Jouhier写道: > > @Michael, thanks for the details. Looks like the 100% zoom is by design; > the message is pretty explicit about it. > > @Oliver The application is single page. We don't use any framework lik

[nodejs] Re: Best way to search through a big plain object

2014-05-20 Thread Stefano Baghino
*their, sry Il giorno martedì 20 maggio 2014 08:07:36 UTC+2, Stefano Baghino ha scritto: > > I don't know about quadtrees but you should check out R-Trees. It's a data > structure used in geographical information systems to index coordinates and > efficiently perform spatial queries on them. I k

[nodejs] Best way to search through a big plain object

2014-05-20 Thread Stefano Baghino
I don't know about quadtrees but you should check out R-Trees. It's a data structure used in geographical information systems to index coordinates and efficiently perform spatial queries on them. I know there's at least one JS implementation in NPM that you could use in a Web app with Browserify