Thanks Saikat for the feedback. I have a simple design. I have external data sources which send data to the ignite cluster via http post requests. I wanted to see if I can utilize ignite's rest API for this purpose and at the same time take advantage of the streamer high performant data load ( the same way that it is possible with TCP socket via socket streamer). It seems that is not possible. I will need to write a http listener myself that ingest the data and persist the data into cache via the streamer.
On Sun, Jan 12, 2020 at 11:28 AM Saikat Maitra <saikat.mai...@gmail.com> wrote: > Hi, > > I think rest api interface and streamer interface are separate interfaces > where streamer interface is built for high performant data loading > capabilities and rest interface is built for easily access the cache, run > task or run sql queries. > > I could not clearly understand the design of the application like how are > you planning to use streamer using REST api. If you have a design docs then > we can review and share feedback. > > Regards, > Saikat > > On Sun, Jan 12, 2020 at 12:24 AM narges saleh <snarges...@gmail.com> > wrote: > >> Hi All, >> The issue with socket streamer example is resolved (the try clause was >> closing the cache). >> I'd appreciate the answer to the question regarding rest API and ignite >> streamer. >> >> thanks. >> >> On Sat, Jan 11, 2020 at 4:03 PM narges saleh <snarges...@gmail.com> >> wrote: >> >>> Hi All, >>> Can someone please provide a concrete answer whether it is possible to >>> load data via the streamer using the REST API? If yes, can you please >>> provide an example? I have searched and watched webinars and so far have >>> not been to find an answer/example. >>> >>> Can also provide a working example of using ignite's socket streamer for >>> loading data? The one that comes with ignite source does not work for me. >>> >>> Do I need to write my own tcp socket or http listener to accept the data >>> from the client and load the cache with ignite data streamer interface? >>> >>> thanks. >>> >>> On Fri, Jan 10, 2020 at 4:36 AM narges saleh <snarges...@gmail.com> >>> wrote: >>> >>>> Thanks Saikat. Please let me know if you find anything. I'll try to >>>> debug on my side too. >>>> >>>> Does anyone have a working example for using socket streamer? >>>> >>>> On Thu, Jan 9, 2020 at 9:07 PM Saikat Maitra <saikat.mai...@gmail.com> >>>> wrote: >>>> >>>>> Hi, >>>>> >>>>> I tried to reproduce the issue in local in debug mode and I can see >>>>> that after sockStmr.start(); the WordsSocketStreamerServer node is >>>>> getting stopped. >>>>> >>>>> I do not see any exception in console and I think it stopped >>>>> gracefully. >>>>> >>>>> We will need to debug further why it is getting stopped and not >>>>> listening to port 5555 for client messages. >>>>> >>>>> Regards, >>>>> Saikat >>>>> >>>>> On Thu, Jan 9, 2020 at 5:48 AM narges saleh <snarges...@gmail.com> >>>>> wrote: >>>>> >>>>>> Any clue what could be wrong? >>>>>> >>>>>> On Tue, Jan 7, 2020 at 8:52 AM narges saleh <snarges...@gmail.com> >>>>>> wrote: >>>>>> >>>>>>> Thanks Saikat for the reply. >>>>>>> >>>>>>> 1) I will check out the webinar. thanks. >>>>>>> 2) The socket streamer server code goes to the finally clause right >>>>>>> after the socket streamer is starts. The screen snapshot with 2 servers >>>>>>> and >>>>>>> one client is here. >>>>>>> >>>>>>> [image: image.png] >>>>>>> >>>>>>> On Mon, Jan 6, 2020 at 7:59 PM Saikat Maitra < >>>>>>> saikat.mai...@gmail.com> wrote: >>>>>>> >>>>>>>> Hi, >>>>>>>> >>>>>>>> 1. If you define schema for your dataset then you should be able to >>>>>>>> execute qry to get the desired resultset. Please take a look into >>>>>>>> Valentin >>>>>>>> Kulichenko's webinar related to using jdbc thin client and getting >>>>>>>> resultset. You can execute similar qry using rest api. Alternatively, >>>>>>>> you >>>>>>>> can also use rest api to Execute command to perform any task in the >>>>>>>> Ignite >>>>>>>> grid. >>>>>>>> >>>>>>>> https://www.youtube.com/watch?v=eMs_2vEsbBk&t=3103s >>>>>>>> >>>>>>>> 2. If I understand correctly the topology for the grid, we have 4 >>>>>>>> ignite servers and 1 Data streamer client which is pushing the data to >>>>>>>> Ignite cluster. If the Ignite Data Streamer client is stopped in >>>>>>>> finally >>>>>>>> clause then where do you receive the "Ignite node stopped message". >>>>>>>> >>>>>>>> Also, can you please share if this is a INFO message or ERROR >>>>>>>> message? >>>>>>>> >>>>>>>> Regards, >>>>>>>> Saikat >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> On Sun, Jan 5, 2020 at 8:23 PM narges saleh <snarges...@gmail.com> >>>>>>>> wrote: >>>>>>>> >>>>>>>>> Hello Saikat, >>>>>>>>> 1) Any service method; say I have an affinity based deployed >>>>>>>>> service with a method that aggregates the balances for each customer >>>>>>>>> based >>>>>>>>> on their location. >>>>>>>>> Also, I take it that it is not possible to have rest >>>>>>>>> API/streamer producer/consumer arrangement as you would have in tcp >>>>>>>>> socket >>>>>>>>> connection between the client data producer and streamer data >>>>>>>>> consumer, >>>>>>>>> right? >>>>>>>>> 2)Yes the nodes report number of servers=4 and clients=1 and but >>>>>>>>> then the number of clients goes to 0 when the streamer node stops >>>>>>>>> (but the >>>>>>>>> number of servers stays at 4). >>>>>>>>> thanks. >>>>>>>>> >>>>>>>>> On Sun, Jan 5, 2020 at 6:56 PM Saikat Maitra < >>>>>>>>> saikat.mai...@gmail.com> wrote: >>>>>>>>> >>>>>>>>>> Hi, >>>>>>>>>> >>>>>>>>>> 1. The rest api is separate interface to load data in cache. I do >>>>>>>>>> not think we have rest endpoint to call ignite service method, can >>>>>>>>>> you >>>>>>>>>> share an example of which service method you want to call using rest >>>>>>>>>> api. >>>>>>>>>> You can check the rest api docs to learn more about rest apis >>>>>>>>>> supported >>>>>>>>>> functionality. >>>>>>>>>> >>>>>>>>>> https://apacheignite.readme.io/docs/rest-api >>>>>>>>>> >>>>>>>>>> 2. Can you see if any active node running, all the nodes will log >>>>>>>>>> cluster health information and number of servers running etc >>>>>>>>>> >>>>>>>>>> Regards, >>>>>>>>>> Saikat >>>>>>>>>> >>>>>>>>>> On Sun, Jan 5, 2020 at 3:21 PM narges saleh <snarges...@gmail.com> >>>>>>>>>> wrote: >>>>>>>>>> >>>>>>>>>>> Thanks Saikat for the reply. >>>>>>>>>>> >>>>>>>>>>> 1) Rest API: how does the streamer listen to the REST requests? >>>>>>>>>>> Is there an example? Or do I just enable streaming an option (e.g., >>>>>>>>>>> the way >>>>>>>>>>> you do it with JDBC/SQL)? Sorry if this is a trivial question. >>>>>>>>>>> Another >>>>>>>>>>> question(unrelated to the topic of data load via streamers): is it >>>>>>>>>>> possible >>>>>>>>>>> to use Ignite's rest API to call an ignite service method (via a >>>>>>>>>>> service >>>>>>>>>>> proxy? If yes, is there an example? >>>>>>>>>>> 2) I started anywhere from 2 to 4 ExampleNodeStartup nodes >>>>>>>>>>> (different tries), but still got the "ignite node stopped" message. >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> On Sun, Jan 5, 2020 at 12:04 PM Saikat Maitra < >>>>>>>>>>> saikat.mai...@gmail.com> wrote: >>>>>>>>>>> >>>>>>>>>>>> Hi, >>>>>>>>>>>> >>>>>>>>>>>> Thank you for your email and using DataStreamer. >>>>>>>>>>>> >>>>>>>>>>>> 1. Yes, there are rest endpoints available to send data in >>>>>>>>>>>> Ignite cache. Here is the docs for the same >>>>>>>>>>>> https://apacheignite.readme.io/docs/rest-api >>>>>>>>>>>> 2. As per the docs as mentioned below, can you please confirm >>>>>>>>>>>> if you have few nodes running using ExampleNodeStartup? >>>>>>>>>>>> >>>>>>>>>>>> * To start the example, you should: >>>>>>>>>>>> * <ul> >>>>>>>>>>>> * <li>Start a few nodes using {@link ExampleNodeStartup}.</li> >>>>>>>>>>>> * <li>Start socket server using {@link >>>>>>>>>>>> WordsSocketStreamerServer}.</li> >>>>>>>>>>>> * <li>Start a few socket clients using {@link >>>>>>>>>>>> WordsSocketStreamerClient}.</li> >>>>>>>>>>>> * <li>Start querying popular words using {@link >>>>>>>>>>>> QueryWords}.</li> >>>>>>>>>>>> * </ul> >>>>>>>>>>>> Regards, >>>>>>>>>>>> Saikat >>>>>>>>>>>> >>>>>>>>>>>> On Sat, Jan 4, 2020 at 2:33 PM narges saleh < >>>>>>>>>>>> snarges...@gmail.com> wrote: >>>>>>>>>>>> >>>>>>>>>>>>> Hi All, >>>>>>>>>>>>> >>>>>>>>>>>>> I have two questions regarding loading data with the streamer, >>>>>>>>>>>>> with non ignite client. >>>>>>>>>>>>> >>>>>>>>>>>>> 1) Is there a REST/HTTP counterpart for the TCP socket >>>>>>>>>>>>> streamer, meaning the client sending the data to the streamer via >>>>>>>>>>>>> HTTP POST? >>>>>>>>>>>>> 2) I am trying to play with ignite's TCP socket streamer >>>>>>>>>>>>> example, but the client node stops with the message "Ignite node >>>>>>>>>>>>> stopped" >>>>>>>>>>>>> -- It hits the finally clause after the socket streamer is >>>>>>>>>>>>> started. Any >>>>>>>>>>>>> idea what could be wrong? >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> https://github.com/apache/ignite/blob/master/examples/src/main/java/org/apache/ignite/examples/streaming/wordcount/socket/WordsSocketStreamerServer.java >>>>>>>>>>>>> >>>>>>>>>>>>> thanks. >>>>>>>>>>>>> >>>>>>>>>>>>>
