Re: Info about time series support
hello all Stephen Darlington: I read https://www.gridgain.com/docs/latest/installation-guide/deployment-modes#choosing-a-client and since I'm developing in Java and I need more than SQL and key-values (I need for sure ML) I opted for thick client Jeremy McMillan: I'm using linear regression actually. Since I'm pretty sure my scenario will evolve in making forecasts about some events distribution in the future (next N month or year) according to the saved data (the training models) I was studying how to use time series algorithms. DJL seems interesting to me and I wanted to deep dive into it but I need to understand how to fit all pieces together in my scenario where I'm using a thick client. In any case I really thank you all for these very useful tips and discussions Angelo Il giorno ven 5 gen 2024 alle ore 15:51 Jeremy McMillan < jeremy.mcmil...@gridgain.com> ha scritto: > To answer the OP question, maybe linear regression is sufficient for > making predictions in your data. > > Ignite isn't really designed for exploratory data analysis, so it really > helps to understand the character of your data. Linear models are usually a > good place to start. Does a regression line make sense if you plot one on > one of your time value columns in a spreadsheet? If so, you may not need to > deploy any additional libraries. > > > https://ignite.apache.org/docs/latest/machine-learning/regression/linear-regression > > On Fri, Jan 5, 2024, 08:25 Stephen Darlington > wrote: > >> Normally we recommend using thin-clients if you can. Though, in this >> case, using a thick-client makes your life easier. Thick clients can deploy >> Java code for you. >> >> There are a few different ways to do it. The "easy" option is to just >> deploy the JAR files to the server nodes "manually." You could also >> consider peer class loading ( >> https://ignite.apache.org/docs/2.11.1/code-deployment/peer-class-loading), >> which is where the client automatically sends classes to the remote nodes. >> Or UriDeployment ( >> https://ignite.apache.org/docs/2.11.1/code-deployment/deploying-user-code), >> where Ignite copies the Jar files from a central location. GridGain's >> Control Center (not open source) is also able to deploy code. >> >> On Fri, 5 Jan 2024 at 14:04, Angelo Immediata >> wrote: >> >>> hello Gianluca and all >>> >>> Regarding to thin client, in my architecture I avoided to use thin >>> clients; I'm using thick clients; so if python is supported only in "thin >>> client" mode, I'd prefer to avoid it >>> >>> Regarding distributed computing, I didn't see it but it seems to be >>> interesting but something is missing me. Let's suppose I want to use djl >>> https://djl.ai/ and its timeseries support ( >>> https://djl.ai/extensions/timeseries/) I can use the distributed >>> computing; as far as I understood the distributed computing allows to me to >>> distribute computations across all my cluster nodes. Now I'm using thick >>> clients, this means my java application is remotely connected to the apache >>> ignite "master nodes"; in distributed computing I should execute the >>> computation on master nodes but if I use a custom dependency (e.g. djl) how >>> can these master remote nodes execute the computation if they don't have >>> the libraries? >>> Am I missing anything? >>> >>> Thank you >>> Angelo >>> >>> Il giorno ven 5 gen 2024 alle ore 14:24 Gianluca Bonetti < >>> gianluca.bone...@gmail.com> ha scritto: >>> Hello Jagat There are Ignite thin clients for a number of languages, including Python. For a full list of functionalities and comparison, please always refer to the official documentation. https://ignite.apache.org/docs/latest/thin-clients/getting-started-with-thin-clients All thin clients should perform around the same in tasks such as storing and retrieving data as they use the Apache Ignite binary protocol. As you know performance also varies case by case, because of different setups, configurations, and versions of software/frameworks/libraries being used, and of course the performance of the code that you will write yourself. For my specific use cases, Apache Ignite always performed extremely well. As I don't know anything about your project, there are far too many possible variables to be able to reduce to a yes/no answer. The advice is to run your own benchmarks on your infrastructure to get some meaningful figures for your specific project and infrastructure. Cheers Gianluca Bonetti On Fri, 5 Jan 2024 at 12:40, Jagat Singh wrote: > Hi Gianluna, > > Does the Python client miss any functionality or performance compared > to Java? > > Thanks > > On Fri, 5 Jan 2024 at 15:55, Gianluca Bonetti < > gianluca.bone...@gmail.com> wrote: > >> Hello Angelo >> >> It seems to be an interesting use case for Ignite. >> >>
Re: Info about time series support
To answer the OP question, maybe linear regression is sufficient for making predictions in your data. Ignite isn't really designed for exploratory data analysis, so it really helps to understand the character of your data. Linear models are usually a good place to start. Does a regression line make sense if you plot one on one of your time value columns in a spreadsheet? If so, you may not need to deploy any additional libraries. https://ignite.apache.org/docs/latest/machine-learning/regression/linear-regression On Fri, Jan 5, 2024, 08:25 Stephen Darlington wrote: > Normally we recommend using thin-clients if you can. Though, in this case, > using a thick-client makes your life easier. Thick clients can deploy Java > code for you. > > There are a few different ways to do it. The "easy" option is to just > deploy the JAR files to the server nodes "manually." You could also > consider peer class loading ( > https://ignite.apache.org/docs/2.11.1/code-deployment/peer-class-loading), > which is where the client automatically sends classes to the remote nodes. > Or UriDeployment ( > https://ignite.apache.org/docs/2.11.1/code-deployment/deploying-user-code), > where Ignite copies the Jar files from a central location. GridGain's > Control Center (not open source) is also able to deploy code. > > On Fri, 5 Jan 2024 at 14:04, Angelo Immediata wrote: > >> hello Gianluca and all >> >> Regarding to thin client, in my architecture I avoided to use thin >> clients; I'm using thick clients; so if python is supported only in "thin >> client" mode, I'd prefer to avoid it >> >> Regarding distributed computing, I didn't see it but it seems to be >> interesting but something is missing me. Let's suppose I want to use djl >> https://djl.ai/ and its timeseries support ( >> https://djl.ai/extensions/timeseries/) I can use the distributed >> computing; as far as I understood the distributed computing allows to me to >> distribute computations across all my cluster nodes. Now I'm using thick >> clients, this means my java application is remotely connected to the apache >> ignite "master nodes"; in distributed computing I should execute the >> computation on master nodes but if I use a custom dependency (e.g. djl) how >> can these master remote nodes execute the computation if they don't have >> the libraries? >> Am I missing anything? >> >> Thank you >> Angelo >> >> Il giorno ven 5 gen 2024 alle ore 14:24 Gianluca Bonetti < >> gianluca.bone...@gmail.com> ha scritto: >> >>> Hello Jagat >>> >>> There are Ignite thin clients for a number of languages, including >>> Python. >>> For a full list of functionalities and comparison, please always refer >>> to the official documentation. >>> >>> https://ignite.apache.org/docs/latest/thin-clients/getting-started-with-thin-clients >>> >>> All thin clients should perform around the same in tasks such as storing >>> and retrieving data as they use the Apache Ignite binary protocol. >>> As you know performance also varies case by case, because of different >>> setups, configurations, and versions of software/frameworks/libraries >>> being used, and of course the performance of the code that you will write >>> yourself. >>> >>> For my specific use cases, Apache Ignite always performed extremely well. >>> As I don't know anything about your project, there are far too many >>> possible variables to be able to reduce to a yes/no answer. >>> The advice is to run your own benchmarks on your infrastructure to get >>> some meaningful figures for your specific project and infrastructure. >>> >>> Cheers >>> Gianluca Bonetti >>> >>> On Fri, 5 Jan 2024 at 12:40, Jagat Singh wrote: >>> Hi Gianluna, Does the Python client miss any functionality or performance compared to Java? Thanks On Fri, 5 Jan 2024 at 15:55, Gianluca Bonetti < gianluca.bone...@gmail.com> wrote: > Hello Angelo > > It seems to be an interesting use case for Ignite. > > However, you should consider what Ignite is, and what is not. > Essentially, Ignite is a distributed in-memory > database/cache/grid/etc... > It also has some distributed computing API capabilities. > > You can store data easily in Ignite, and consume data by your code > written in Java. > You can also use Python since there is a Python Ignite Client > available if it makes your time series analysis easier. > You can also use the Ignite Computing API to execute code on your > cluster > https://ignite.apache.org/docs/latest/distributed-computing/distributed-computing > but in this case I think Python is not supported. > > Cheers > Gianluca Bonetti > > On Fri, 5 Jan 2024 at 08:52, Angelo Immediata > wrote: > >> I'm pretty new to Apache Ignite >> >> >> I asked this also on stackoverflow ( >> https://stackoverflow.com/questions/77667648/apache-ignite-time-series-forecasting) >> but I received no answer
Re: Info about time series support
Normally we recommend using thin-clients if you can. Though, in this case, using a thick-client makes your life easier. Thick clients can deploy Java code for you. There are a few different ways to do it. The "easy" option is to just deploy the JAR files to the server nodes "manually." You could also consider peer class loading ( https://ignite.apache.org/docs/2.11.1/code-deployment/peer-class-loading), which is where the client automatically sends classes to the remote nodes. Or UriDeployment ( https://ignite.apache.org/docs/2.11.1/code-deployment/deploying-user-code), where Ignite copies the Jar files from a central location. GridGain's Control Center (not open source) is also able to deploy code. On Fri, 5 Jan 2024 at 14:04, Angelo Immediata wrote: > hello Gianluca and all > > Regarding to thin client, in my architecture I avoided to use thin > clients; I'm using thick clients; so if python is supported only in "thin > client" mode, I'd prefer to avoid it > > Regarding distributed computing, I didn't see it but it seems to be > interesting but something is missing me. Let's suppose I want to use djl > https://djl.ai/ and its timeseries support ( > https://djl.ai/extensions/timeseries/) I can use the distributed > computing; as far as I understood the distributed computing allows to me to > distribute computations across all my cluster nodes. Now I'm using thick > clients, this means my java application is remotely connected to the apache > ignite "master nodes"; in distributed computing I should execute the > computation on master nodes but if I use a custom dependency (e.g. djl) how > can these master remote nodes execute the computation if they don't have > the libraries? > Am I missing anything? > > Thank you > Angelo > > Il giorno ven 5 gen 2024 alle ore 14:24 Gianluca Bonetti < > gianluca.bone...@gmail.com> ha scritto: > >> Hello Jagat >> >> There are Ignite thin clients for a number of languages, including Python. >> For a full list of functionalities and comparison, please always refer to >> the official documentation. >> >> https://ignite.apache.org/docs/latest/thin-clients/getting-started-with-thin-clients >> >> All thin clients should perform around the same in tasks such as storing >> and retrieving data as they use the Apache Ignite binary protocol. >> As you know performance also varies case by case, because of different >> setups, configurations, and versions of software/frameworks/libraries >> being used, and of course the performance of the code that you will write >> yourself. >> >> For my specific use cases, Apache Ignite always performed extremely well. >> As I don't know anything about your project, there are far too many >> possible variables to be able to reduce to a yes/no answer. >> The advice is to run your own benchmarks on your infrastructure to get >> some meaningful figures for your specific project and infrastructure. >> >> Cheers >> Gianluca Bonetti >> >> On Fri, 5 Jan 2024 at 12:40, Jagat Singh wrote: >> >>> Hi Gianluna, >>> >>> Does the Python client miss any functionality or performance compared to >>> Java? >>> >>> Thanks >>> >>> On Fri, 5 Jan 2024 at 15:55, Gianluca Bonetti < >>> gianluca.bone...@gmail.com> wrote: >>> Hello Angelo It seems to be an interesting use case for Ignite. However, you should consider what Ignite is, and what is not. Essentially, Ignite is a distributed in-memory database/cache/grid/etc... It also has some distributed computing API capabilities. You can store data easily in Ignite, and consume data by your code written in Java. You can also use Python since there is a Python Ignite Client available if it makes your time series analysis easier. You can also use the Ignite Computing API to execute code on your cluster https://ignite.apache.org/docs/latest/distributed-computing/distributed-computing but in this case I think Python is not supported. Cheers Gianluca Bonetti On Fri, 5 Jan 2024 at 08:52, Angelo Immediata wrote: > I'm pretty new to Apache Ignite > > > I asked this also on stackoverflow ( > https://stackoverflow.com/questions/77667648/apache-ignite-time-series-forecasting) > but I received no answer > > I need to make some forecasting analysis > > Basically I can collect data in Ignite in real time. Ignite will store > data in its own caches > > Now I need to make some forecasting showing me the distribution of > data in the next X months/years by starting from observed and collected > data. > > As far as I know, this kind of forecasting can be realized by time > series forecasting. In Ignite I see no time series based algorithm. Am I > right? > > If I'm correct I may use or tensor flow or Deep Java Library. But in > this case what I don't understand is: where should I use these libraries? > Inside my thick client
Re: Info about time series support
hello Gianluca and all Regarding to thin client, in my architecture I avoided to use thin clients; I'm using thick clients; so if python is supported only in "thin client" mode, I'd prefer to avoid it Regarding distributed computing, I didn't see it but it seems to be interesting but something is missing me. Let's suppose I want to use djl https://djl.ai/ and its timeseries support ( https://djl.ai/extensions/timeseries/) I can use the distributed computing; as far as I understood the distributed computing allows to me to distribute computations across all my cluster nodes. Now I'm using thick clients, this means my java application is remotely connected to the apache ignite "master nodes"; in distributed computing I should execute the computation on master nodes but if I use a custom dependency (e.g. djl) how can these master remote nodes execute the computation if they don't have the libraries? Am I missing anything? Thank you Angelo Il giorno ven 5 gen 2024 alle ore 14:24 Gianluca Bonetti < gianluca.bone...@gmail.com> ha scritto: > Hello Jagat > > There are Ignite thin clients for a number of languages, including Python. > For a full list of functionalities and comparison, please always refer to > the official documentation. > > https://ignite.apache.org/docs/latest/thin-clients/getting-started-with-thin-clients > > All thin clients should perform around the same in tasks such as storing > and retrieving data as they use the Apache Ignite binary protocol. > As you know performance also varies case by case, because of different > setups, configurations, and versions of software/frameworks/libraries > being used, and of course the performance of the code that you will write > yourself. > > For my specific use cases, Apache Ignite always performed extremely well. > As I don't know anything about your project, there are far too many > possible variables to be able to reduce to a yes/no answer. > The advice is to run your own benchmarks on your infrastructure to get > some meaningful figures for your specific project and infrastructure. > > Cheers > Gianluca Bonetti > > On Fri, 5 Jan 2024 at 12:40, Jagat Singh wrote: > >> Hi Gianluna, >> >> Does the Python client miss any functionality or performance compared to >> Java? >> >> Thanks >> >> On Fri, 5 Jan 2024 at 15:55, Gianluca Bonetti >> wrote: >> >>> Hello Angelo >>> >>> It seems to be an interesting use case for Ignite. >>> >>> However, you should consider what Ignite is, and what is not. >>> Essentially, Ignite is a distributed in-memory >>> database/cache/grid/etc... >>> It also has some distributed computing API capabilities. >>> >>> You can store data easily in Ignite, and consume data by your code >>> written in Java. >>> You can also use Python since there is a Python Ignite Client available >>> if it makes your time series analysis easier. >>> You can also use the Ignite Computing API to execute code on your >>> cluster >>> https://ignite.apache.org/docs/latest/distributed-computing/distributed-computing >>> but in this case I think Python is not supported. >>> >>> Cheers >>> Gianluca Bonetti >>> >>> On Fri, 5 Jan 2024 at 08:52, Angelo Immediata >>> wrote: >>> I'm pretty new to Apache Ignite I asked this also on stackoverflow ( https://stackoverflow.com/questions/77667648/apache-ignite-time-series-forecasting) but I received no answer I need to make some forecasting analysis Basically I can collect data in Ignite in real time. Ignite will store data in its own caches Now I need to make some forecasting showing me the distribution of data in the next X months/years by starting from observed and collected data. As far as I know, this kind of forecasting can be realized by time series forecasting. In Ignite I see no time series based algorithm. Am I right? If I'm correct I may use or tensor flow or Deep Java Library. But in this case what I don't understand is: where should I use these libraries? Inside my thick client microservice or should I write an Ignite plugin in order to use the scalability feature provided by Ignite? Thank you Angelo >>>
Re: Info about time series support
Hello Jagat There are Ignite thin clients for a number of languages, including Python. For a full list of functionalities and comparison, please always refer to the official documentation. https://ignite.apache.org/docs/latest/thin-clients/getting-started-with-thin-clients All thin clients should perform around the same in tasks such as storing and retrieving data as they use the Apache Ignite binary protocol. As you know performance also varies case by case, because of different setups, configurations, and versions of software/frameworks/libraries being used, and of course the performance of the code that you will write yourself. For my specific use cases, Apache Ignite always performed extremely well. As I don't know anything about your project, there are far too many possible variables to be able to reduce to a yes/no answer. The advice is to run your own benchmarks on your infrastructure to get some meaningful figures for your specific project and infrastructure. Cheers Gianluca Bonetti On Fri, 5 Jan 2024 at 12:40, Jagat Singh wrote: > Hi Gianluna, > > Does the Python client miss any functionality or performance compared to > Java? > > Thanks > > On Fri, 5 Jan 2024 at 15:55, Gianluca Bonetti > wrote: > >> Hello Angelo >> >> It seems to be an interesting use case for Ignite. >> >> However, you should consider what Ignite is, and what is not. >> Essentially, Ignite is a distributed in-memory database/cache/grid/etc... >> It also has some distributed computing API capabilities. >> >> You can store data easily in Ignite, and consume data by your code >> written in Java. >> You can also use Python since there is a Python Ignite Client available >> if it makes your time series analysis easier. >> You can also use the Ignite Computing API to execute code on your cluster >> https://ignite.apache.org/docs/latest/distributed-computing/distributed-computing >> but in this case I think Python is not supported. >> >> Cheers >> Gianluca Bonetti >> >> On Fri, 5 Jan 2024 at 08:52, Angelo Immediata >> wrote: >> >>> I'm pretty new to Apache Ignite >>> >>> >>> I asked this also on stackoverflow ( >>> https://stackoverflow.com/questions/77667648/apache-ignite-time-series-forecasting) >>> but I received no answer >>> >>> I need to make some forecasting analysis >>> >>> Basically I can collect data in Ignite in real time. Ignite will store >>> data in its own caches >>> >>> Now I need to make some forecasting showing me the distribution of data >>> in the next X months/years by starting from observed and collected data. >>> >>> As far as I know, this kind of forecasting can be realized by time >>> series forecasting. In Ignite I see no time series based algorithm. Am I >>> right? >>> >>> If I'm correct I may use or tensor flow or Deep Java Library. But in >>> this case what I don't understand is: where should I use these libraries? >>> Inside my thick client microservice or should I write an Ignite plugin in >>> order to use the scalability feature provided by Ignite? >>> >>> Thank you >>> >>> Angelo >>> >>
Re: Info about time series support
Hi Gianluna, Does the Python client miss any functionality or performance compared to Java? Thanks On Fri, 5 Jan 2024 at 15:55, Gianluca Bonetti wrote: > Hello Angelo > > It seems to be an interesting use case for Ignite. > > However, you should consider what Ignite is, and what is not. > Essentially, Ignite is a distributed in-memory database/cache/grid/etc... > It also has some distributed computing API capabilities. > > You can store data easily in Ignite, and consume data by your code written > in Java. > You can also use Python since there is a Python Ignite Client available if > it makes your time series analysis easier. > You can also use the Ignite Computing API to execute code on your cluster > https://ignite.apache.org/docs/latest/distributed-computing/distributed-computing > but in this case I think Python is not supported. > > Cheers > Gianluca Bonetti > > On Fri, 5 Jan 2024 at 08:52, Angelo Immediata wrote: > >> I'm pretty new to Apache Ignite >> >> >> I asked this also on stackoverflow ( >> https://stackoverflow.com/questions/77667648/apache-ignite-time-series-forecasting) >> but I received no answer >> >> I need to make some forecasting analysis >> >> Basically I can collect data in Ignite in real time. Ignite will store >> data in its own caches >> >> Now I need to make some forecasting showing me the distribution of data >> in the next X months/years by starting from observed and collected data. >> >> As far as I know, this kind of forecasting can be realized by time series >> forecasting. In Ignite I see no time series based algorithm. Am I right? >> >> If I'm correct I may use or tensor flow or Deep Java Library. But in this >> case what I don't understand is: where should I use these libraries? Inside >> my thick client microservice or should I write an Ignite plugin in order to >> use the scalability feature provided by Ignite? >> >> Thank you >> >> Angelo >> >
Re: Info about time series support
Hello Angelo It seems to be an interesting use case for Ignite. However, you should consider what Ignite is, and what is not. Essentially, Ignite is a distributed in-memory database/cache/grid/etc... It also has some distributed computing API capabilities. You can store data easily in Ignite, and consume data by your code written in Java. You can also use Python since there is a Python Ignite Client available if it makes your time series analysis easier. You can also use the Ignite Computing API to execute code on your cluster https://ignite.apache.org/docs/latest/distributed-computing/distributed-computing but in this case I think Python is not supported. Cheers Gianluca Bonetti On Fri, 5 Jan 2024 at 08:52, Angelo Immediata wrote: > I'm pretty new to Apache Ignite > > > I asked this also on stackoverflow ( > https://stackoverflow.com/questions/77667648/apache-ignite-time-series-forecasting) > but I received no answer > > I need to make some forecasting analysis > > Basically I can collect data in Ignite in real time. Ignite will store > data in its own caches > > Now I need to make some forecasting showing me the distribution of data in > the next X months/years by starting from observed and collected data. > > As far as I know, this kind of forecasting can be realized by time series > forecasting. In Ignite I see no time series based algorithm. Am I right? > > If I'm correct I may use or tensor flow or Deep Java Library. But in this > case what I don't understand is: where should I use these libraries? Inside > my thick client microservice or should I write an Ignite plugin in order to > use the scalability feature provided by Ignite? > > Thank you > > Angelo >
Info about time series support
I'm pretty new to Apache Ignite I asked this also on stackoverflow ( https://stackoverflow.com/questions/77667648/apache-ignite-time-series-forecasting) but I received no answer I need to make some forecasting analysis Basically I can collect data in Ignite in real time. Ignite will store data in its own caches Now I need to make some forecasting showing me the distribution of data in the next X months/years by starting from observed and collected data. As far as I know, this kind of forecasting can be realized by time series forecasting. In Ignite I see no time series based algorithm. Am I right? If I'm correct I may use or tensor flow or Deep Java Library. But in this case what I don't understand is: where should I use these libraries? Inside my thick client microservice or should I write an Ignite plugin in order to use the scalability feature provided by Ignite? Thank you Angelo
