Re: Apache Ignite Cloud Formation Template
Hello!
If that is really Apache Ignite 1.5, please don't use it. Please only use
the latest 2.7.6 release.
Regards,
--
Ilya Kasnacheev
пт, 27 сент. 2019 г. в 18:56, sri hari kali charan Tummala <
kali.tumm...@gmail.com>:
> I found one example of cloud formation after launching the template and
> started to execute ignite.sh it is failing with class not found exception
> AWSCredentialsProviderWrapper.
>
> I sued:-
> Ignitefabric.1.5.0
> aws java sdk 1.3.21.1
>
>
>
> https://github.com/aws-samples/aws-big-data-blog/blob/master/aws-blog-real-time-in-memory-oltp-and-analytics-with-apache-ignite/cloudformation/configignite.json
>
> ConfireIgnite.sh:-
>
> #!/bin/bash
>
> #
> # This is a modified version of the file stored at
> s3://publicbucketbabupe/ignitelibrary/configureIgnite.sh
> # which changes the config to use the instance provided credentials rather
> than requiring access/secret to be passed in
> #
> # Parameters are
> # 1 - Cache Name
> # 2 - Number of replicas
> # 3 - S3 Bucket Name
> #
>
> echo "
>
>
>
> http://www.springframework.org/schema/beans\;
>xmlns:util=\"http://www.springframework.org/schema/util\;
>xmlns:xsi=\"http://www.w3.org/2001/XMLSchema-instance\;
>xsi:schemaLocation=\"
>http://www.springframework.org/schema/beans
>http://www.springframework.org/schema/beans/spring-beans.xsd
>http://www.springframework.org/schema/util
>http://www.springframework.org/schema/util/spring-util.xsd\;>
>
>class=\"org.apache.ignite.configuration.IgniteConfiguration\">
>
>
>
> " > /tmp/igniteconfig.xml
>
> echo "
>
>value=\"PRIMARY_SYNC\"/>" >> /tmp/igniteconfig.xml
>
> availfreeMemory=$(cat /proc/meminfo|grep MemTotal|awk '{print $2}')
> memoryOverhead=$((availfreeMemory/1024/1024/10))
> availfreeMemoryinGB=$((availfreeMemory/1024/1024 - memoryOverhead))
> if [[ $availfreeMemoryinGB -gt 8 ]]; then
> offheapmemoryinGB=$((availfreeMemoryinGB-8))
> echo "
> 1024L * 1024L * 1024L}\" />" >> /tmp/igniteconfig.xml
> echo "8g" > /tmp/heapsize.log
> else
> echo "${availfreeMemoryinGB}g" > /tmp/heapsize.log
> fi
> echo "
>class=\"org.apache.ignite.cache.eviction.lru.LruEvictionPolicy\">
>
>
> " >> /tmp/igniteconfig.xml
> echo " " >>
> /tmp/igniteconfig.xml
> echo "" >>
> /tmp/igniteconfig.xml
> echo "" >> /tmp/igniteconfig.xml
> echo "" >> /tmp/igniteconfig.xml
> echo " " >> /tmp/igniteconfig.xml
> echo "" >> /tmp/igniteconfig.xml
> echo "
>
>
> class=\"org.apache.ignite.spi.discovery.tcp.ipfinder.s3.TcpDiscoveryS3IpFinder\">
>
>
>
>
>
>
>
> class=\"org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi\">
>
>
>
>
>
>class=\"com.amazonaws.auth.DefaultAWSCredentialsProviderChain\"/>
>class=\"com.amazon.connections.data.auth.AWSCredentialsProviderWrapper\">
>
>
>
> " >> /tmp/igniteconfig.xml
>
>
> On Thu, Sep 26, 2019 at 5:34 PM sri hari kali charan Tummala <
> kali.tumm...@gmail.com> wrote:
>
>> looks complex and hard to get it right, it would be nice if Ignite
>> community builds one working cloud formation template to launch ignite
>> cluster and in the cloud formation template output a JDBC connection for
>> connecting ignite cluster which would be awesome.
>>
>> Many would leverage this stack to build Ignite cluster on AWS or how
>> about ignite on EMR , while launching emr add ingite bootstrap script which
>> launches ignite in emr which would be lot easier also JDBC connection for
>> connecting ignite, this would change things for Ignite world.
>>
>> Thanks
>> Sri
>>
>>
>>
>> On Thu, Sep 26, 2019 at 3:51 PM Denis Magda wrote:
>>
>>> Hello Sri,
>>>
>>> You’re right, you need to start several aws instances to create a
>>> distributed cluster if you follow the guide below. Ignite configuration for
>>> all the nodes/instances has to have a properly configured IPFinder. You can
>>> use a static IP finder if IPs are know in advance or AWS IPfinder if IPs
>>> are assigned dynamically.
>>> https://apacheignite.readme.io/docs/tcpip-discovery
>>>
>>> Let us know if it helps and we’ll update the docs.
>>>
>>> Denis
>>>
>>> On Thursday, September 26, 2019, sri hari kali charan Tummala <
>>> kali.tumm...@gmail.com> wrote:
>>>
Hi Community,
this doesn't launch a multinode ignite cluster am I right? it just
launches a single node ignite cluster on one ec2 instance.
https://apacheignite.readme.io/docs/aws-deployment
Thanks
Sri
On Thu, Sep 26, 2019 at 8:49 AM sri hari kali charan Tummala <
kali.tumm...@gmail.com> wrote:
> Hi All,
>
> did anyone have an cloud formation example for deploying Apache Ignite
> on AWS or steps to deploy
Re: Apache Ignite Cloud Formation Template
I found one example of cloud formation after launching the template and
started to execute ignite.sh it is failing with class not found exception
AWSCredentialsProviderWrapper.
I sued:-
Ignitefabric.1.5.0
aws java sdk 1.3.21.1
https://github.com/aws-samples/aws-big-data-blog/blob/master/aws-blog-real-time-in-memory-oltp-and-analytics-with-apache-ignite/cloudformation/configignite.json
ConfireIgnite.sh:-
#!/bin/bash
#
# This is a modified version of the file stored at
s3://publicbucketbabupe/ignitelibrary/configureIgnite.sh
# which changes the config to use the instance provided credentials
rather than requiring access/secret to be passed in
#
# Parameters are
# 1 - Cache Name
# 2 - Number of replicas
# 3 - S3 Bucket Name
#
echo "
http://www.springframework.org/schema/beans\;
xmlns:util=\"http://www.springframework.org/schema/util\;
xmlns:xsi=\"http://www.w3.org/2001/XMLSchema-instance\;
xsi:schemaLocation=\"
http://www.springframework.org/schema/beans
http://www.springframework.org/schema/beans/spring-beans.xsd
http://www.springframework.org/schema/util
http://www.springframework.org/schema/util/spring-util.xsd\;>
" > /tmp/igniteconfig.xml
echo "
" >> /tmp/igniteconfig.xml
availfreeMemory=$(cat /proc/meminfo|grep MemTotal|awk '{print $2}')
memoryOverhead=$((availfreeMemory/1024/1024/10))
availfreeMemoryinGB=$((availfreeMemory/1024/1024 - memoryOverhead))
if [[ $availfreeMemoryinGB -gt 8 ]]; then
offheapmemoryinGB=$((availfreeMemoryinGB-8))
echo "
" >> /tmp/igniteconfig.xml
echo "8g" > /tmp/heapsize.log
else
echo "${availfreeMemoryinGB}g" > /tmp/heapsize.log
fi
echo "
" >> /tmp/igniteconfig.xml
echo "" >>
/tmp/igniteconfig.xml
echo "" >>
/tmp/igniteconfig.xml
echo "" >> /tmp/igniteconfig.xml
echo " " >> /tmp/igniteconfig.xml
echo " " >> /tmp/igniteconfig.xml
echo "" >> /tmp/igniteconfig.xml
echo "
" >> /tmp/igniteconfig.xml
On Thu, Sep 26, 2019 at 5:34 PM sri hari kali charan Tummala <
kali.tumm...@gmail.com> wrote:
> looks complex and hard to get it right, it would be nice if Ignite
> community builds one working cloud formation template to launch ignite
> cluster and in the cloud formation template output a JDBC connection for
> connecting ignite cluster which would be awesome.
>
> Many would leverage this stack to build Ignite cluster on AWS or how about
> ignite on EMR , while launching emr add ingite bootstrap script which
> launches ignite in emr which would be lot easier also JDBC connection for
> connecting ignite, this would change things for Ignite world.
>
> Thanks
> Sri
>
>
>
> On Thu, Sep 26, 2019 at 3:51 PM Denis Magda wrote:
>
>> Hello Sri,
>>
>> You’re right, you need to start several aws instances to create a
>> distributed cluster if you follow the guide below. Ignite configuration for
>> all the nodes/instances has to have a properly configured IPFinder. You can
>> use a static IP finder if IPs are know in advance or AWS IPfinder if IPs
>> are assigned dynamically.
>> https://apacheignite.readme.io/docs/tcpip-discovery
>>
>> Let us know if it helps and we’ll update the docs.
>>
>> Denis
>>
>> On Thursday, September 26, 2019, sri hari kali charan Tummala <
>> kali.tumm...@gmail.com> wrote:
>>
>>> Hi Community,
>>>
>>> this doesn't launch a multinode ignite cluster am I right? it just
>>> launches a single node ignite cluster on one ec2 instance.
>>>
>>> https://apacheignite.readme.io/docs/aws-deployment
>>>
>>> Thanks
>>> Sri
>>>
>>> On Thu, Sep 26, 2019 at 8:49 AM sri hari kali charan Tummala <
>>> kali.tumm...@gmail.com> wrote:
>>>
Hi All,
did anyone have an cloud formation example for deploying Apache Ignite
on AWS or steps to deploy or some successful deployment of Ignite onto AWS.
this below blog has missing artifacts so not working.
https://aws.amazon.com/blogs/big-data/real-time-in-memory-oltp-and-analytics-with-apache-ignite-on-aws/
--
Thanks & Regards
Sri Tummala
>>>
>>> --
>>> Thanks & Regards
>>> Sri Tummala
>>>
>>>
>>
>> --
>> -
>> Denis
>>
>>
>
> --
> Thanks & Regards
> Sri Tummala
>
>
--
Thanks & Regards
Sri Tummala
Re: Apache Ignite Cloud Formation Template
looks complex and hard to get it right, it would be nice if Ignite community builds one working cloud formation template to launch ignite cluster and in the cloud formation template output a JDBC connection for connecting ignite cluster which would be awesome. Many would leverage this stack to build Ignite cluster on AWS or how about ignite on EMR , while launching emr add ingite bootstrap script which launches ignite in emr which would be lot easier also JDBC connection for connecting ignite, this would change things for Ignite world. Thanks Sri On Thu, Sep 26, 2019 at 3:51 PM Denis Magda wrote: > Hello Sri, > > You’re right, you need to start several aws instances to create a > distributed cluster if you follow the guide below. Ignite configuration for > all the nodes/instances has to have a properly configured IPFinder. You can > use a static IP finder if IPs are know in advance or AWS IPfinder if IPs > are assigned dynamically. > https://apacheignite.readme.io/docs/tcpip-discovery > > Let us know if it helps and we’ll update the docs. > > Denis > > On Thursday, September 26, 2019, sri hari kali charan Tummala < > kali.tumm...@gmail.com> wrote: > >> Hi Community, >> >> this doesn't launch a multinode ignite cluster am I right? it just >> launches a single node ignite cluster on one ec2 instance. >> >> https://apacheignite.readme.io/docs/aws-deployment >> >> Thanks >> Sri >> >> On Thu, Sep 26, 2019 at 8:49 AM sri hari kali charan Tummala < >> kali.tumm...@gmail.com> wrote: >> >>> Hi All, >>> >>> did anyone have an cloud formation example for deploying Apache Ignite >>> on AWS or steps to deploy or some successful deployment of Ignite onto AWS. >>> >>> this below blog has missing artifacts so not working. >>> >>> >>> https://aws.amazon.com/blogs/big-data/real-time-in-memory-oltp-and-analytics-with-apache-ignite-on-aws/ >>> >>> -- >>> Thanks & Regards >>> Sri Tummala >>> >>> >> >> -- >> Thanks & Regards >> Sri Tummala >> >> > > -- > - > Denis > > -- Thanks & Regards Sri Tummala
Re: Apache Ignite Cloud Formation Template
Hello Sri, You’re right, you need to start several aws instances to create a distributed cluster if you follow the guide below. Ignite configuration for all the nodes/instances has to have a properly configured IPFinder. You can use a static IP finder if IPs are know in advance or AWS IPfinder if IPs are assigned dynamically. https://apacheignite.readme.io/docs/tcpip-discovery Let us know if it helps and we’ll update the docs. Denis On Thursday, September 26, 2019, sri hari kali charan Tummala < kali.tumm...@gmail.com> wrote: > Hi Community, > > this doesn't launch a multinode ignite cluster am I right? it just > launches a single node ignite cluster on one ec2 instance. > > https://apacheignite.readme.io/docs/aws-deployment > > Thanks > Sri > > On Thu, Sep 26, 2019 at 8:49 AM sri hari kali charan Tummala < > kali.tumm...@gmail.com> wrote: > >> Hi All, >> >> did anyone have an cloud formation example for deploying Apache Ignite on >> AWS or steps to deploy or some successful deployment of Ignite onto AWS. >> >> this below blog has missing artifacts so not working. >> >> https://aws.amazon.com/blogs/big-data/real-time-in-memory- >> oltp-and-analytics-with-apache-ignite-on-aws/ >> >> -- >> Thanks & Regards >> Sri Tummala >> >> > > -- > Thanks & Regards > Sri Tummala > > -- - Denis
Re: Apache Ignite Cloud Formation Template
Hi Community, this doesn't launch a multinode ignite cluster am I right? it just launches a single node ignite cluster on one ec2 instance. https://apacheignite.readme.io/docs/aws-deployment Thanks Sri On Thu, Sep 26, 2019 at 8:49 AM sri hari kali charan Tummala < kali.tumm...@gmail.com> wrote: > Hi All, > > did anyone have an cloud formation example for deploying Apache Ignite on > AWS or steps to deploy or some successful deployment of Ignite onto AWS. > > this below blog has missing artifacts so not working. > > > https://aws.amazon.com/blogs/big-data/real-time-in-memory-oltp-and-analytics-with-apache-ignite-on-aws/ > > -- > Thanks & Regards > Sri Tummala > > -- Thanks & Regards Sri Tummala
Apache Ignite Cloud Formation Template
Hi All, did anyone have an cloud formation example for deploying Apache Ignite on AWS or steps to deploy or some successful deployment of Ignite onto AWS. this below blog has missing artifacts so not working. https://aws.amazon.com/blogs/big-data/real-time-in-memory-oltp-and-analytics-with-apache-ignite-on-aws/ -- Thanks & Regards Sri Tummala
