Hi Woo,
It may be reasonable, if you see, nodes resources utilization is too low
and rising per-node-buffer size have no effect (that means you prepare data
for nodes too slow).
Of course, you should check first if network isn't a bottleneck.
On Tue, Apr 25, 2017 at 10:08 AM, woo charles
wrote:
Same data set mean that I separate original data into 2 parts & input them
from 2 separate programs.
E.g. a data set with id 1 - 100. Program A input id 1-50. Program B input
51 - 100.
2017-04-21 17:24 GMT+08:00 Andrey Mashenkov :
> Hi Woo,
>
> DataStreamer is designed to fill cache with maximu
Hi Woo,
DataStreamer is designed to fill cache with maximum throughput. By default,
streamer will not rewrite cache data, until allowOverwite option is set.
Why you need to input same set of data? Why do you expected data input time
will change significantly with 2 programs compared to 1 if data
If the data is buffered in client side, the bottleneck should be also in
client side.
If I use 2 programs to input same set data, it should be a significant
change in data input time.
Is it right?
2017-04-21 6:46 GMT+08:00 Dmitriy Setrakyan :
>
> On Wed, Apr 19, 2017 at 10:16 PM, woo charles
> w
On Wed, Apr 19, 2017 at 10:16 PM, woo charles
wrote:
> When I call addData() in streamer. this data will send & buffer in server
> node. is that correct?
> If I correct, this data will buffer in random server node or only the one
> it directly connected?
>
addData() will buffer the data on the c
Hi Woo,
AddData() add entry to one of local buffer according to key affinity, then
buffer is sent to server node that is primary for all buffer keys.
On Thu, Apr 20, 2017 at 8:16 AM, woo charles
wrote:
> When I call addData() in streamer. this data will send & buffer in server
> node. is that c
When I call addData() in streamer. this data will send & buffer in server
node. is that correct?
If I correct, this data will buffer in random server node or only the one
it directly connected?
2017-04-19 18:33 GMT+08:00 Andrey Mashenkov :
> It may have effect if you prepare data for streamer (ca
It may have effect if you prepare data for streamer (call addData) slowly
and it is possible to utilize more resources for it. Of course remote nodes
should be able to bear pressure of data.
Performance can increased, but usually slightly as network will be a
bottleneck.
On Wed, Apr 19, 2017 at 1
Is that mean the performance of input data will not affect if I use 2
IgniteDataStreamer(2
client program) to input data as they use the same queue in remote nodes?
2017-04-19 10:02 GMT+08:00 Andrey Mashenkov :
> Hi Woo,
>
> IgniteDataStreamer uses per node buffer to make bulk cache updates that
Hi Woo,
IgniteDataStreamer uses per node buffer to make bulk cache updates that
shows much better throughput than single updates.
Also, IgniteDataStreamer send jobs to remote nodes, to utilize multiple
threads on remote nodes.
In multi-node grid IgniteDataStreamer usually shows better results tha
When I try to input data(80 table, each 1 records) to a cluster with 3
server node(each 2 gb), it only has a small change in time if multi thread
is performed
(ie. max decrease from 8s to 6.5s if using IgniteCache)
Is it normal?
Also, I found that multi thread do not affect the data input spe
11 matches
Mail list logo