Re: Data Streamer not flushing data to cache

2018-03-31 Thread Andrey Kuznetsov
Indeed, the only reliable way is flush/close. Nonzero automatic flush
frequency doesn't provide the same guarantee.

2018-03-31 21:11 GMT+03:00 begineer :

> One more query.. Would it never flush the data if nothing more is added to
> streamer and current size is less than buffer size ?
> What is the default time. I can see only flush frequency
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>



-- 
Best regards,
  Andrey Kuznetsov.


Re: Data Streamer not flushing data to cache

2018-03-31 Thread begineer
One more query.. Would it never flush the data if nothing more is added to
streamer and current size is less than buffer size ?
What is the default time. I can see only flush frequency 



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/


Re: Data Streamer not flushing data to cache

2018-03-31 Thread begineer
Thanks for reply... Its works after invoking flush(). 



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/


Re: Data Streamer not flushing data to cache

2018-03-31 Thread David Harvey
By default, DataStreamer will only send a full buffer, unless you
explicitly flush or close it, or as suggested, implicitly close it.   You
can set a flush timer also.

The last time I looked (2.3), flush timer is implemented to flush
periodically, and this is unaffected by when data was last added to the
stream, i.e., if you want to ensure that it has stored all of the records
older than 5 seconds, setting the flush timer to 5 seconds will cause it to
flush every 5 seconds, even when all of the outstanding data was added in
the last 10ms.  So the flush timer creates an (unnecessary) performance
anomaly under heavy load.

Note that while the DataStreamer is thread safe, if you have a
multi-threaded producer, works much better to have a DataStreamer per
producer thread, especially if there are explicit or timer driven flushes.


-DH


On Sat, Mar 31, 2018 at 5:38 AM, begineer  wrote:

> Hi, This must be something very simple. I am adding 100 items to data
> streamer. But it is not flushing items to cache. Is there a settings much
> enables it. Cache size is zero. Am I doing something wrong ?
>
> public class DataStreamerExample {
> public static void main(String[] args) throws InterruptedException {
> Ignite ignite = Ignition.start("examples/
> config/example-ignite.xml");
> CacheConfiguration config = new
> CacheConfiguration<>("mycache");
> IgniteCache cache = ignite.getOrCreateCache(config);
> IgniteDataStreamer streamer =
> ignite.dataStreamer("mycache");
> LongStream.range(1, 100).forEach( l->{
> System.out.println("Adding to streamer "+ l);
> streamer.addData(l, l);
> });
> System.out.println(streamer.perNodeBufferSize());
> System.out.println("Cache size : "+ cache.size(CachePeekMode.ALL))
> ;
> cache.query(new ScanQuery<>()).getAll().stream().forEach(entry->{
> System.out.println("cache Entry: " + entry.getKey()+" "+
> entry.getValue());
> });
> }
> }
>
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>

Disclaimer

The information contained in this communication from the sender is 
confidential. It is intended solely for use by the recipient and others 
authorized to receive it. If you are not the recipient, you are hereby notified 
that any disclosure, copying, distribution or taking action in relation of the 
contents of this information is strictly prohibited and may be unlawful.

This email has been scanned for viruses and malware, and may have been 
automatically archived by Mimecast Ltd, an innovator in Software as a Service 
(SaaS) for business. Providing a safer and more useful place for your human 
generated data. Specializing in; Security, archiving and compliance. To find 
out more visit the Mimecast website.


Re: Data Streamer not flushing data to cache

2018-03-31 Thread Andrey Kuznetsov
Hello!

The simplest way to ensure your data have got to the cache is to use
IgniteDataStreamer in try-with-resources block. I some rare scenarios it
can make sense to call {{flush()}} or {{close()}} on streamer instance
directly.

2018-03-31 12:38 GMT+03:00 begineer :

> Hi, This must be something very simple. I am adding 100 items to data
> streamer. But it is not flushing items to cache. Is there a settings much
> enables it. Cache size is zero. Am I doing something wrong ?
>
> public class DataStreamerExample {
> public static void main(String[] args) throws InterruptedException {
> Ignite ignite = Ignition.start("examples/
> config/example-ignite.xml");
> CacheConfiguration config = new
> CacheConfiguration<>("mycache");
> IgniteCache cache = ignite.getOrCreateCache(config);
> IgniteDataStreamer streamer =
> ignite.dataStreamer("mycache");
> LongStream.range(1, 100).forEach( l->{
> System.out.println("Adding to streamer "+ l);
> streamer.addData(l, l);
> });
> System.out.println(streamer.perNodeBufferSize());
> System.out.println("Cache size : "+ cache.size(CachePeekMode.ALL))
> ;
> cache.query(new ScanQuery<>()).getAll().stream().forEach(entry->{
> System.out.println("cache Entry: " + entry.getKey()+" "+
> entry.getValue());
> });
> }
> }
>
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>



-- 
Best regards,
  Andrey Kuznetsov.


Data Streamer not flushing data to cache

2018-03-31 Thread begineer
Hi, This must be something very simple. I am adding 100 items to data
streamer. But it is not flushing items to cache. Is there a settings much
enables it. Cache size is zero. Am I doing something wrong ?

public class DataStreamerExample {
public static void main(String[] args) throws InterruptedException {
Ignite ignite = Ignition.start("examples/config/example-ignite.xml");
CacheConfiguration config = new
CacheConfiguration<>("mycache");
IgniteCache cache = ignite.getOrCreateCache(config);
IgniteDataStreamer streamer = 
ignite.dataStreamer("mycache");
LongStream.range(1, 100).forEach( l->{
System.out.println("Adding to streamer "+ l);
streamer.addData(l, l);
});
System.out.println(streamer.perNodeBufferSize());
System.out.println("Cache size : "+ cache.size(CachePeekMode.ALL));
cache.query(new ScanQuery<>()).getAll().stream().forEach(entry->{
System.out.println("cache Entry: " + entry.getKey()+" "+
entry.getValue());
});
}
}




--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/