Re:Ignite support for IoC beyond spring framework
Hello! I've started the review of your PR, should be ready tomorrow, but looks good overall! Kind regards, Semyon.
Re: Ignite support for IoC beyond spring framework
Hi!
This sounds like a good enhancement. Feel free to create a pull request!
As I see it, for now we should continue using SpringResource annotation (even
if it injects non-spring resources), but at the same time create a new
annotation (maybe something like @IgniteInject, I am bad with naming) and
deprecate SpringResource annotation. Both annotations should work exactly the
same.
Then we will have a spring version of injector interface and some generic
version that any user can extend and use without spring.
WDYT?
P.S. You can create a JIRA ticket and mention me there, I'll be happy to review
and merge your PR
Kind regards, Semyon.
> Hello,
> I would like to bring a proposal for Ignite to include support for
> @SpringResource without spring at all.
> We currently use ignite without Spring Framework and, to make things
> harder, under OSGi. As a part of exercise couple weeks ago I implemented
> support for IoC [1] in environment we run. I did it by simulating
> spring's ApplicationContext instance expected by IgnitionEx [2] and
> GridSpringResourceContext.
>
> Turns out that Ignite IoC is fine with below API:
> public interface BeanRegistry {
> T lookup(Class type);
> Object lookup(String name);
> }
>
> Given that we look forward to keep use of Ignite I was asked to
> contribute my work back to your project.
> Would you be up to discuss, and if agreed, guide me on a shape of ioc
> updates which would best fit Ignite itself? If you do not expect changes
> in Spring integration then, maybe we could get a working solution within
> ignite-osgi itself?
>
> Best,
> Łukasz
>
> [1]
> https://github.com/OpenNMS/horizon-stream/tree/develop/minion/minion-taskset/taskset-worker/src/main/java/org/opennms/horizon/minion/taskset/worker/ignite/resource
>
> [2]
> https://github.com/OpenNMS/horizon-stream/blob/develop/minion/minion-taskset/taskset-worker/src/main/java/org/opennms/horizon/minion/taskset/worker/ignite/IgnitionFactory.java#L15
Re: Control.sh command that schedules index rebuild in the maintenance mode
Hello, Nikolay! Online rebuild causes unpredictable behaviour of the indexed SQL queries. For example, if a query plan requires an index scan, we will instead resort to the full scan which is unacceptable for some users. So sometimes it’s better to shutdown the node than mess up the application. Kind regards, Semyon. > Hello, Семен. > > Why index rebuild should be done in maintenance mode? > >> 31 мая 2022 г., в 18:24, Данилов Семён написал(а): >> >> Hello, igniters! >> >> I want to propose a new control.sh cache sub-command that schedules a >> rebuild of cache indexes in the maintenance mode. >> We already have a force rebuild command but sometimes (for example, under >> load) user may want to first stop the node and then rebuild indexes. >> For this scenario we can add a command that creates a maintenance task and >> then after restart node will enter the maintenance mode and rebuild >> specified indexes. >> >> I propose the following API: >> --cache schedule_indexes_rebuild --node-id nodeId --target >> cacheName1=index1,...indexN >> Schedules rebuild of the indexes for specified caches via the Maintenance >> Mode. >> >> Parameters: >> --node-id - (Optional) Specify node for indexes rebuild. >> --target - Cache name with optionally specified indexes. If indexes are not >> specified then all indexes of the cache will be scheduled for the rebuild >> operation. >> >> As you can see, the user can provide a node id to schedule the rebuild on a >> specific node (otherwise the rebuild will be scheduled for all of the nodes) >> and targets (specific indexes of a cache or all of the indexes of a cache). >> There is a ticket[1] and a pull request[2] for my proposal. >> >> WDYT? >> >> Kind regards, >> Semyon. >> >> [1] https://issues.apache.org/jira/browse/IGNITE-17002 >> [2] https://github.com/apache/ignite/pull/10042
Control.sh command that schedules index rebuild in the maintenance mode
Hello, igniters! I want to propose a new control.sh cache sub-command that schedules a rebuild of cache indexes in the maintenance mode. We already have a force rebuild command but sometimes (for example, under load) user may want to first stop the node and then rebuild indexes. For this scenario we can add a command that creates a maintenance task and then after restart node will enter the maintenance mode and rebuild specified indexes. I propose the following API: --cache schedule_indexes_rebuild --node-id nodeId --target cacheName1=index1,...indexN Schedules rebuild of the indexes for specified caches via the Maintenance Mode. Parameters: --node-id - (Optional) Specify node for indexes rebuild. --target- Cache name with optionally specified indexes. If indexes are not specified then all indexes of the cache will be scheduled for the rebuild operation. As you can see, the user can provide a node id to schedule the rebuild on a specific node (otherwise the rebuild will be scheduled for all of the nodes) and targets (specific indexes of a cache or all of the indexes of a cache). There is a ticket[1] and a pull request[2] for my proposal. WDYT? Kind regards, Semyon. [1] https://issues.apache.org/jira/browse/IGNITE-17002 [2] https://github.com/apache/ignite/pull/10042
Request for a review (Java 17 support)
Hello Igniters! I'd like to request a review for an improvement that adds support for Java 17. Description of the improvement: - Added new maven profile that allows running tests with Java 17 (--add-opens and all that). This maven profile is also used for source folder with IgniteLinkTaglet (the same way as with java 11 profile) - Fixed fsync for mmap'ped byte buffers, because the original one uses internal fields and APIs that were changed in JDK 15. Now there are two fsync mechanisms, one for older java and one for java >= 15. - Rewritten shell and batch scripts to eliminate copy-pasted JVM_OPTS, now they all are in one place, jvmdefaults.(sh|bat) - Fixed an issue with TLS v1.3: there was a buffer underflow in the client's handshake process. Ticket: https://issues.apache.org/jira/browse/IGNITE-16622 Pull request: https://github.com/apache/ignite/pull/9849 Semyon.
Re: [MTCGA]: new failures in builds [5994077, 5994075] needs to be handled
Hi! The fix was merged yesterday and the JDBC driver suite https://ci.ignite.apache.org/buildConfiguration/IgniteTests24Java8_JdbcDriver?branch=%3Cdefault%3E=overview=builds#all-projects is stable again. Semyon. 19.05.2021, 14:07, "Ilya Kasnacheev" : > Hello! > > I suggest that you may revert this change right away, and reopen the ticket. > > Then patch author (Semyon) can take a look. > > Regards, > -- > Ilya Kasnacheev > > пн, 17 мая 2021 г. в 14:16, Maxim Muzafarov : > >> Folks, >> >> All tests passed successfully in the branch with reverted commit [1]: >> >> >> https://ci.ignite.apache.org/viewType.html?buildTypeId=IgniteTests24Java8_JdbcDriver=buildTypeStatusDiv_IgniteTests24Java8=pull%2F9100%2Fhead >> >> Will anyone take a look? >> I propose to revert the [1] commit and rework the patch. >> >> [1] >> >> https://github.com/apache/ignite/commit/e32691b9d9ef103aa884cf0d4f48abe9d98c8e50 >> >> On Fri, 14 May 2021 at 17:47, Maxim Muzafarov wrote: >> > >> > Folks, >> > >> > The JDBC Driver fails with the same error in the PR branch also [2]. >> > The most suspicious commit is related to the IGNITE-14575 issue. I've >> > left comment [2]. >> > Can you re-check the patch? >> > >> > [1] >> >> https://ci.ignite.apache.org/buildConfiguration/IgniteTests24Java8_JdbcDriver?branch=pull%2F9012%2Fhead=builds >> > [2] >> >> https://issues.apache.org/jira/browse/IGNITE-14575?focusedCommentId=17344632=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-17344632 >> > >> > On Wed, 5 May 2021 at 13:13, Ilya Kasnacheev >> wrote: >> > > >> > > Hello! >> > > >> > > I have re-run these suites and they did not fail. >> > > >> > > I think it is some transient issue with TC. >> > > >> > > Regards, >> > > -- >> > > Ilya Kasnacheev >> > > >> > > >> > > ср, 5 мая 2021 г. в 04:55, : >> > > >> > > > Hi Igniters, >> > > > >> > > > I've detected some new issue on TeamCity to be handled. You are >> more than >> > > > welcomed to help. >> > > > >> > > > If your changes can lead to this failure(s): We're grateful that >> you were >> > > > a volunteer to make the contribution to this project, but things >> change and >> > > > you may no longer be able to finalize your contribution. >> > > > Could you respond to this email and indicate if you wish to >> continue and >> > > > fix test failures or step down and some committer may revert you >> commit. >> > > > >> > > > * New Critical Failure in master RDD >> > > > >> >> https://ci.ignite.apache.org/buildConfiguration/IgniteTests24Java8_Rdd?branch=%3Cdefault%3E >> > > > Changes may lead to failure were done by >> > > > - ilya kasnacheev >> > > > https://ci.ignite.apache.org/viewModification.html?modId=924767 >> > > > - semyon danilov >> > > > https://ci.ignite.apache.org/viewModification.html?modId=924731 >> > > > >> > > > * New Critical Failure in master JDBC Driver >> > > > >> >> https://ci.ignite.apache.org/buildConfiguration/IgniteTests24Java8_JdbcDriver?branch=%3Cdefault%3E >> > > > Changes may lead to failure were done by >> > > > - ilya kasnacheev >> > > > https://ci.ignite.apache.org/viewModification.html?modId=924767 >> > > > - semyon danilov >> > > > https://ci.ignite.apache.org/viewModification.html?modId=924731 >> > > > >> > > > - Here's a reminder of what contributors were agreed to do >> > > > https://cwiki.apache.org/confluence/display/IGNITE/How+to+Contribute >> > > > - Should you have any questions please contact >> > > > dev@ignite.apache.org >> > > > >> > > > Best Regards, >> > > > Apache Ignite TeamCity Bot >> > > > https://github.com/apache/ignite-teamcity-bot >> > > > Notification generated at 04:55:24 05-05-2021 >> > > >
Obsolete classnames.properties
Hello, Igniters! I've recently stumbled across an issue where some tests can't be run locally due to the absence of some entries in "classnames.properties" file. For example, TcpRestUnmarshalVulnerabilityTest can't be run locally because "classnames.properties" lacks org.apache.ignite.configuration.EncryptionConfiguration and org.apache.ignite.configuration.PageReplacementMode. On teamcity, however, such tests pass without any problem, because "classnames.properties" is also generated by org.apache.ignite.tools.classgen.ClassesGenerator during process-classes phase in maven build. I propose removing this file, as it is generated during the build which means it's a pointless job keeping it up-to-date in the repository. Also it's impossible to tell right now if it is or it is not up-to-date (without executing ClassesGenerator). Kind regards, Semyon.
Obsolete classnames.properties
Hello, Igniters! I've recently stumbled across an issue where some tests can't be run locally due to the absence of some entries in "classnames.properties" file. For example, TcpRestUnmarshalVulnerabilityTest can't be run locally because "classnames.properties" lacks org.apache.ignite.configuration.EncryptionConfiguration and org.apache.ignite.configuration.PageReplacementMode. On teamcity, however, such tests pass without any problem, because "classnames.properties" is also generated by org.apache.ignite.tools.classgen.ClassesGenerator during process-classes phase in maven build. I propose removing this file, as it is generated during the build which means it's a pointless job keeping it up-to-date in the repository. Also it's impossible to tell right now if it is or it is not up-to-date (without executing ClassesGenerator). Kind regards, Semyon.
Re: Significant Items to Tackle
Hello Atri! Would you like to contribute into Ignite Spring integrations? You can take a look at this particular jira filter: https://issues.apache.org/jira/browse/IGNITE-9524?jql=project%20%3D%20Ignite%20and%20summary%20~%20%22spring%22%20and%20status%20not%20in%20(Resolved%2C%20Closed) . Personally, I think one of the most valuable issues there is https://issues.apache.org/jira/browse/IGNITE-9524. I have a lot of experience with Spring, so you can reach out to me if you have any questions. Cheers, Sam. 25.03.2021, 23:33, "Atri Sharma" : > Hi Community, > > First off, I want to thank the community for being so welcoming and > helpful. This is an awesome place to be in. > > Now that I have worked on some issues, I would like to get my hands deeper > and with larger issues in the core. Also, some more intermediate tickets to > tackle, my queue has gotten empty :) > > Please help and advice. > > Regards, > > Atri
Re: IGNITE-2399: Implement acquireAndExecute Utility Method
Hello Atri, I've read through your changes, have a look at the review. Kind regards, Sam. 23.02.2021, 08:07, "Atri Sharma" : > Hi All, > > Please help in reviewing the following PR: > > https://github.com/apache/ignite/pull/8820 > > Regards, > > Atri > > -- > Regards, > > Atri > Apache Concerted
Re: Move WAL archive cleanup from checkpoint to rollover
Hello! I also have an issue in progress regarding WAL archive: https://issues.apache.org/jira/browse/IGNITE-12892. In this ticket I tried clarifying WAL archive size configuration, removing usage of the deprecated walHistSize property. Also, IGNITE_PDS_MAX_CHECKPOINT_MEMORY_HISTORY_SIZE is now only used when archive is managed externally and you can set maxWalArchiveSize to -1 so WAL truncation is disabled. PR is already submitted and approved https://github.com/apache/ignite/pull/8550 Regards, Semyon. 09.12.2020, 19:17, "ткаленко кирилл" : > Hi, Andrey! > > Users expect DataStorageConfiguration#maxWalArchiveSize to mean that WAL > archive will not exceed this value, but it is not. > It seems that to reduce the chance of getting into a situation when we exceed > WAL archive, it will be lowed when we clean it when switching to a new > segment than at the end of the checkpoint. > After that, we can think about and make a hard limit on WAL archive, but for > this will need to solve a few more problems. > > 09.12.2020, 17:24, "Andrey Gura" : >> Kiriill, >> >> Issue description contains the following: >> >>> At the moment, WAL archive is cleared at the end of the checkpoint, which >>> does not seem correct and needs to be moved >> >> Could you please explain why existing behavior is not correct. It >> seems that it is not enough motivation for change. >> >> On Wed, Dec 9, 2020 at 5:05 PM vbm wrote: >>> Hi Kirill Tkalenko, >>> >>> Is there any relation to rate of ingestion of data to ignite ? >>> >>> We had seen the issue of WAL growing infinitely recently in our K8s >>> cluster. >>> We were ingesting data at around 2Mbps. >>> In other clusters where we did not have such a fast ingestion of data, >>> this >>> issue was not observed. >>> >>> Regards, >>> Vishwas >>> >>> -- >>> Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/
Re: [DISCUSSION] Java 11 for Ignite 3.0 development
+1 for sure. AFAIK, the only thing holding us back from using Java 11 is the dominance of Java 8, but I'm sure that by the time Ignite 3 is GA, there will be much fewer Java 8 users if any significant number at all. By the by, Ignite's sources need minimal effort to be able to be compiled with Java 11 as a target. 08.12.2020, 15:00, "Nikolay Izhikov" : > +1 for using java 11. > >> 8 дек. 2020 г., в 13:18, ткаленко кирилл написал(а): >> >> +1 >> >> 08.12.2020, 12:48, "Philipp Masharov" : >>> Hello! >>> >>> Andrey's arguments are solid. >>> >>> On Tue, Dec 8, 2020 at 12:23 PM Pavel Tupitsyn >>> wrote: >>> +1, Java 11 seems to be the only right choice at the moment. On Tue, Dec 8, 2020 at 12:08 PM Alexey Zinoviev wrote: > I totally support Java 11 for development. It's time to go forward > > вт, 8 дек. 2020 г. в 11:40, Andrey Gura : > > > Igniters, > > > > We already had some discussion about using modern Java versions for > > Ignite 3.0 development [1] but we still don't have consensus. > > As I see from this discussion the strongest argument for Java 11 is > > the fact that Java 11 is the latest LTS release which will have > > premier support until September 2023. So I don't see any reason for > > preferring any other version of Java at this moment. > > > > The purpose of this thread is to gather opinions about using Java 11 > > in the Ignite 3.0 project and, eventually, reach a consensus on this. > > > > I want to share my several arguments in favor of abandoning Java 8 and > > preferring Java 11: > > > > * Java 8 has gone through the End of Public Updates process for legacy > > releases. So it doesn't make sense to start new development on Java 8. > > > > * Java 9+ brings Jigsaw modularization which allows us to have more > > fine-grained structure of Ignite modules and APIs in the future. > > > > * Ignite actively uses Unsafe functionality which, firstly, isn't > > public, and secondly, leads to problems with running Ignite under Java > > 9+ (modularization which requires dozens of command-line options in > > order to forcibly export corresponding packages) and GraalVM. Such a > > situation could be described as bad user experience and we should fix > > it. Var handles [2] could be used for solving described problems. > > > > * Java 9+ introduces Flight Recorder API [3] which could be used in > > the Ignite project for lightweight profiling of internal processes. > > > > Please, share your opinions, objections and ideas about this topic. I > > hope we will not have serious disagreements and the consensus will be > > reached quickly. > > > > > > 1. > > > http://apache-ignite-developers.2346864.n4.nabble.com/DISCUSS-Ignite-3-0-development-approach-tp49922p50295.html > > 2. > > > https://docs.oracle.com/javase/9/docs/api/java/lang/invoke/VarHandle.html > > 3. > > > https://docs.oracle.com/en/java/javase/11/docs/api/jdk.jfr/jdk/jfr/FlightRecorder.html > > >
Re: Migrating NodeJS client to TypeScript
Yes, TS compiler produces JS files and TS-typings. Users that are already using JS version will have a seamless migration. Note: we'll have to publish compiled JS files along with typings. - Semyon. 01.12.2020, 09:59, "Ivan Daschinsky" : >> Is there any value in keeping both versions - the plain JavaScript one > > and the TypeScript specific > Hi! No, there is no value. TS sources should be transpiled to JS before > execution. > > вт, 1 дек. 2020 г. в 01:31, Denis Magda : > >> Hi Semyon, >> >> Is there any value in keeping both versions - the plain JavaScript one and >> the TypeScript specific? >> >> - >> Denis >> >> On Mon, Nov 30, 2020 at 12:16 PM Данилов Семён wrote: >> >> > Hello Igniters! >> > >> > I'd like to propose a big change for the nodejs thin client: a full >> > rewrite from JavaScript to TypeScript. >> > >> > Strong typing will not only help us in future while developing new >> > features, but will also indicate existing type inconsistencies (I know >> > there is a couple). >> > Also, we will have out of the box types for API documentation, which is >> > very handy for users. >> > >> > So what do you think? >> > >> > Kind regards, >> > Semyon. >> > > > -- > Sincerely yours, Ivan Daschinskiy
Migrating NodeJS client to TypeScript
Hello Igniters! I'd like to propose a big change for the nodejs thin client: a full rewrite from JavaScript to TypeScript. Strong typing will not only help us in future while developing new features, but will also indicate existing type inconsistencies (I know there is a couple). Also, we will have out of the box types for API documentation, which is very handy for users. So what do you think? Kind regards, Semyon.
Spring Session integration module
Hello, Igniters! I recently submitted a pull request to the Spring Framework's spring-session repository adding Apache Ignite integration module so that users will be able to store HTTP sessions in Ignite backed repository. If you know any Spring committers that can review and merge these changes, please let me know. You can also help if you leave a thumbs up at the pull request: https://github.com/spring-projects/spring-session/pull/1730. Kind regards, Semyon.
Re: Migration of spring-data modules to ignite-extensions
+1 And, as a matter of fact, I think that spring data integration should be in spring-projects organization, just like spring-data-neo4j, spring-data-mongodb and the others. This way we will always have spring-data integration working correctly with the latest version of spring-data. 09.10.2020, 12:42, "Nikolay Izhikov" : > Ilya, we should move modules that is not related to Ignite core to extension. > We shouldn’t support all possible integrations in the Ignite itself. > >> I would argue that it should only moved to extensions last, >> when this process is debugged thoroughly and when we have at least one >> "essentials + extensions" release shipped with feedback. > > Each extension should be released separately. > What is «essentials + extensions» release do you have in mind? > > Moreover, ignite-spring-data release is not related to any Ignite releases. > Why do we have to wait Ignite release to get spring-data fixes or updates? > >> 9 окт. 2020 г., в 12:34, Ilya Kasnacheev >> написал(а): >> >> Hello! >> >> -1 from me for now. >> >> "Since Spring Data integration is actively used (compared to most others), >> I would argue that it should only moved to extensions last, when this >> process is debugged thoroughly and when we have at least one "essentials + >> extensions" release shipped with feedback. >> >> I think such release would be 2.9, so I think we should only move Spring >> Data to extensions after 2.9 is released and feedback from this release is >> processed." >> >> Regards, >> -- >> Ilya Kasnacheev >> >> пт, 9 окт. 2020 г. в 12:10, Nikolay Izhikov : >> >>> +1. >>> >>> Saikat, can you, please, take a look? >>> 9 окт. 2020 г., в 11:54, Nikita Amelchev >>> написал(а): Mikhail, +1 for migrate чт, 8 окт. 2020 г. в 18:34, Mikhail Petrov : > Igniters, > > I propose to migrate spring-data modules from ignite main repository to > ignite-extensions. > > Are there any objections? > > I've created ticket [1] and PR to both ignite [2] and ignite-extensions > [3] repositories. > > [1] - https://issues.apache.org/jira/browse/IGNITE-13559 > > [2] - https://github.com/apache/ignite/pull/8334 > > [3] - https://github.com/apache/ignite-extensions/pull/25 -- Best wishes, Amelchev Nikita
Security issue with control.sh and ignite.sh
Hello, Igniters! I recently got my eye on the fact that we have JMX enabled by default and it's configured in a very insecure way. Our default JMX parameters are authenticate=false and ssl=false. I propose removing default configuration of JMX altogether, as user must *consciously* and carefully configure such dangerous things. I created an issue (https://issues.apache.org/jira/browse/IGNITE-13478) and pull request for those changes (https://github.com/apache/ignite/pull/8304). Cheers, Sam.
Re: JVM_OPTS in control.sh and ignite.sh
Hello!
I added message
"JVM_OPTS environment variable is set, but will not be used. To pass JVM
options use CONTROL_JVM_OPTS"
"JVM_OPTS=${JVM_OPTS}"
in both control.sh and control.bat files.
Cheers, Sam.
28.09.2020, 16:56, "Ilya Kasnacheev" :
> Hello!
>
> Can we at least print a warning when control.sh is run and JVM_OPTS is set?
> Then you can use other env var. Or get rid of it and rely on -J.
>
> Regards,
> --
> Ilya Kasnacheev
>
> пн, 28 сент. 2020 г. в 12:39, Данилов Семён :
>
>> Hello, everyone!
>>
>> Let's wrap this up somehow.
>>
>> I think that creating a different control.sh script for docker will create
>> a precedent and we might end up with a plethora of scripts for different
>> situations.
>>
>> Control.sh should be independent from ignite.sh thus using different set
>> of environment variables seems like a viable option to me.
>> Breaking compatibility isn't the best thing, yes, but in the same time,
>> control.sh shouldn't really have used JVM_OPTS in the first place (and if
>> you really must provide some JVM options, you'll still be able to do it via
>> new variable).
>>
>> Cheers,
>> Sam.
>>
>> 24.09.2020, 17:58, "Valentin Kulichenko" :
>> > Why does the control.sh use JVM_OPTS in the first place? Is there a case
>> > when a user might need to modify them? I can't think of one.
>> >
>> > -Val
>> >
>> > On Thu, Sep 24, 2020 at 6:42 AM Evgenii Zhuravlev <
>> e.zhuravlev...@gmail.com>
>> > wrote:
>> >
>> >> Ilya,
>> >>
>> >> You can get absolutely the same behaviour when you set JVM_OPTS even
>> >> without Docker.
>> >>
>> >> Evgenii
>> >>
>> >> чт, 24 сент. 2020 г. в 05:44, Ilya Kasnacheev <
>> ilya.kasnach...@gmail.com>:
>> >>
>> >> > Hello!
>> >> >
>> >> > If the issue is with docker only, then maybe we should get rid of
>> >> JVM_OPTS
>> >> > with docker entirely? E.g. pass them as parameters.
>> >> >
>> >> > I'm not sold on this change yet, it breaks backward compatibility for
>> >> > marginal benefit.
>> >> >
>> >> > Regards,
>> >> > --
>> >> > Ilya Kasnacheev
>> >> >
>> >> >
>> >> > чт, 24 сент. 2020 г. в 15:35, Данилов Семён :
>> >> >
>> >> > > Hello, Igniters!
>> >> > >
>> >> > > I recently discovered, that control.sh and ignite.sh both use
>> JVM_OPTS
>> >> > > environment variable. This can lead to various issues (especially
>> in
>> >> > > docker), such as:
>> >> > > * Control utility will have the same xms/xmx parameters.
>> >> > > * Control utility won't launch due to JMX port being in use (as it
>> is
>> >> set
>> >> > > in JVM_OPTS and already occupied by ignite).
>> >> > > And so on.
>> >> > >
>> >> > > I suggest using different environment variable in control.sh
>> >> > > (CONTROL_JVM_OPTS for example).
>> >> > >
>> >> > > Here is the JIRA issue —
>> >> > > https://issues.apache.org/jira/browse/IGNITE-13479
>> >> > > And a pull request — https://github.com/apache/ignite/pull/8275/
>> >> > >
>> >> > > Regards, Semyon.
>> >> > >
>> >> >
Re: JVM_OPTS in control.sh and ignite.sh
Hello, everyone! Let's wrap this up somehow. I think that creating a different control.sh script for docker will create a precedent and we might end up with a plethora of scripts for different situations. Control.sh should be independent from ignite.sh thus using different set of environment variables seems like a viable option to me. Breaking compatibility isn't the best thing, yes, but in the same time, control.sh shouldn't really have used JVM_OPTS in the first place (and if you really must provide some JVM options, you'll still be able to do it via new variable). Cheers, Sam. 24.09.2020, 17:58, "Valentin Kulichenko" : > Why does the control.sh use JVM_OPTS in the first place? Is there a case > when a user might need to modify them? I can't think of one. > > -Val > > On Thu, Sep 24, 2020 at 6:42 AM Evgenii Zhuravlev > wrote: > >> Ilya, >> >> You can get absolutely the same behaviour when you set JVM_OPTS even >> without Docker. >> >> Evgenii >> >> чт, 24 сент. 2020 г. в 05:44, Ilya Kasnacheev : >> >> > Hello! >> > >> > If the issue is with docker only, then maybe we should get rid of >> JVM_OPTS >> > with docker entirely? E.g. pass them as parameters. >> > >> > I'm not sold on this change yet, it breaks backward compatibility for >> > marginal benefit. >> > >> > Regards, >> > -- >> > Ilya Kasnacheev >> > >> > >> > чт, 24 сент. 2020 г. в 15:35, Данилов Семён : >> > >> > > Hello, Igniters! >> > > >> > > I recently discovered, that control.sh and ignite.sh both use JVM_OPTS >> > > environment variable. This can lead to various issues (especially in >> > > docker), such as: >> > > * Control utility will have the same xms/xmx parameters. >> > > * Control utility won't launch due to JMX port being in use (as it is >> set >> > > in JVM_OPTS and already occupied by ignite). >> > > And so on. >> > > >> > > I suggest using different environment variable in control.sh >> > > (CONTROL_JVM_OPTS for example). >> > > >> > > Here is the JIRA issue — >> > > https://issues.apache.org/jira/browse/IGNITE-13479 >> > > And a pull request — https://github.com/apache/ignite/pull/8275/ >> > > >> > > Regards, Semyon. >> > > >> >
JVM_OPTS in control.sh and ignite.sh
Hello, Igniters! I recently discovered, that control.sh and ignite.sh both use JVM_OPTS environment variable. This can lead to various issues (especially in docker), such as: * Control utility will have the same xms/xmx parameters. * Control utility won't launch due to JMX port being in use (as it is set in JVM_OPTS and already occupied by ignite). And so on. I suggest using different environment variable in control.sh (CONTROL_JVM_OPTS for example). Here is the JIRA issue — https://issues.apache.org/jira/browse/IGNITE-13479 And a pull request — https://github.com/apache/ignite/pull/8275/ Regards, Semyon.
Re: [DISCUSSION] Add autocompletion for commands in control.sh
Hello! I've looked through the picocli manual and it looks really great. We will be able easily add i18n and stuff like this. Nevertheless, we will get rid of the manual formatting of examples and help texts. Kind regards, Semyon. 14.09.2020, 13:00, "Alexey Goncharuk" : > Hi folks, > > Despite the autocompletion support only for bash, I see the following > benefits from this change: > * It may unify all the CLI tooliing in Ignite, providing a better user > experience > * The library has an ability to generate man pages, which may be nice > * I see there is an open issue for adding support to powershell, so > Windows platform will be also somewhat covered > > Overall, I support this idea. > > вт, 8 сент. 2020 г. в 10:43, ткаленко кирилл : > >> Hello, Ilya! >> >> I agree that it would be better if we found a common approach, but I >> haven't found one yet. >> >> 07.09.2020, 18:50, "Ilya Kasnacheev" : >> > Hello! >> > >> > Not everyone is using bash, which leads me to question whether there's >> any >> > common approach where we can hint a shell what our executable can do so >> > that it can discover and auto-complete our control.sh >> > >> > Regards, >> > -- >> > Ilya Kasnacheev >> > >> > пн, 7 сент. 2020 г. в 17:47, ткаленко кирилл : >> > >> >> Hello, folks! >> >> >> >> I spent time to analyze the possibility of adding auto completion for >> the >> >> "control.sh" with the [1]. >> >> >> >> To do this, at the beginning, we need to adapt the "control.sh" code to >> >> [1], then we can automatically create a "bash completion script" via >> [2], >> >> and then install it, for example, with the "source" command and the >> >> "control.sh" script itself via "install". >> >> >> >> This is only possible for nix systems. >> >> >> >> It is theoretically possible to add the "control.sh" extension via >> plugins >> >> and auto-generate "bash completion script". >> >> >> >> Thus, I propose a plan: >> >> 1)Adapt "control.sh" to [1]; >> >> 2)Automatic creation of "bash completion script" for the release build; >> >> 3)Adding extensibility "control.sh" and automatic re-creation of "bash >> >> completion script". (optional) >> >> >> >> What do you think, comments? >> >> >> >> [1] - https://picocli.info/ >> >> [2] - >> >> >> https://picocli.info/autocomplete.html#_completion_script_generation_details
Changes in Spring Data Integration
Hello igniters! I've been working on a tutorial regarding Spring Cloud + Ignite and came across this issue: IgniteRepositoryFactoryBean (Spring Data module) requires Ignite instance bean with specific name! And in the same time, Ignite autoconfiguration module provides Ignite instance bean with another name. I propose a simple fix: remove constraints on a bean name and do injecting by class, as it is more Spring-way. I created an issue (https://issues.apache.org/jira/browse/IGNITE-13169) and a pull request (https://github.com/apache/ignite/pull/7949). These changes shouldn't break anything, as the injection still works, we just ask by class and not by name (bringing type safety in!). Kind regards, Semyon.
Re: Spring Committers Wanted
Hello Denis, We need people whose opinion is valued in the Spring community, yes, because so far Spring Project members rejected the idea of adding Ignite dependency. - Semyon 15.06.2020, 21:40, "Denis Magda" : > Hello Semyon, > > That's a very reasonable suggestion. Whom do you mean under the Spring > Project committers - those who already contribute to the Spring > ecosystem and have some merit there, or those among us who are willing to > contribute the integration? > > - > Denis > > On Mon, Jun 15, 2020 at 6:27 AM Данилов Семён wrote: > >> Hello Igniters! >> >> I recently stumbled upon a fact that Apache Ignite doesn't have any Spring >> integrations in spring-project repositories. >> To increase Ignite's popularity, I think it would be best to fix that >> inconvenience. The best way to start would be to create >> a web-session repository based on Ignite cache. For this task we need >> Spring Project committers who can support the idea >> of creating separate module for Ignite in Spring Session repository. The >> main objection of Spring Community members is >> that Spring Session project already provides MapSession which can be >> backed by Apache Ignite map. But I think that this >> solution is very limited, because we have zero control on flushing policy. >> Also, we must keep track if changes have been >> made since last save and we should persist session to cache again. >> >> Kind regards, >> Semyon.
Spring Committers Wanted
Hello Igniters! I recently stumbled upon a fact that Apache Ignite doesn't have any Spring integrations in spring-project repositories. To increase Ignite's popularity, I think it would be best to fix that inconvenience. The best way to start would be to create a web-session repository based on Ignite cache. For this task we need Spring Project committers who can support the idea of creating separate module for Ignite in Spring Session repository. The main objection of Spring Community members is that Spring Session project already provides MapSession which can be backed by Apache Ignite map. But I think that this solution is very limited, because we have zero control on flushing policy. Also, we must keep track if changes have been made since last save and we should persist session to cache again. Kind regards, Semyon.
Re: Moving binary metadata to PDS storage folder
Hello everyone, can we move forward with the merge of IGNITE-12994? Semyon 18.05.2020, 19:08, "Denis Mekhanikov" : > Users shouldn't care about binary_meta and marshaller directories. Those > are internal details. Having to go through the documentation on those > things to figure out whether you need to run each of the provided scripts > or not will lead to terrible user experience. > If we add the binary_marshaller_move.sh script for this specific case, it > will stay with us forever just because migration from Ignite 2.8 to 2.9 > required running this thing. > > Denis
Re: Can I get write permissions for Apache Wiki?
Hello! It's sdanilov Regards, Semyon Danilov 09.06.2020, 13:04, "Ilya Kasnacheev" : > Hello! > > What is your login for apache wiki? > > Regards, > -- > Ilya Kasnacheev > > вт, 9 июн. 2020 г. в 10:11, Данилов Семён : > >> ello Igniters, >> >> I’m planning to change >> >> https://cwiki.apache.org/confluence/display/IGNITE/Ignite+Persistent+Store+-+under+the+hood >> (because I have pending pull request for changing binary meta location), >> but don’t have rights for this. >> Can someone please give me the access? >> >> Sincerely yours, >> Semyon Danilov.
Can I get write permissions for Apache Wiki?
ello Igniters, I’m planning to change https://cwiki.apache.org/confluence/display/IGNITE/Ignite+Persistent+Store+-+under+the+hood (because I have pending pull request for changing binary meta location), but don’t have rights for this. Can someone please give me the access? Sincerely yours, Semyon Danilov.
Re: Moving binary metadata to PDS storage folder
Hello Denis, I found out that without persistence being enabled, no metadata is written on disk (where is an if clause at the start of every method of BinaryMetadataFileStore) and everything is stored in metadataLocalCache. Semyon. 13.05.2020, 21:10, "Denis Mekhanikov" : > Sounds great! > > It happens pretty frequently that users migrate to a new version of > Ignite and preserve persistence files only without caring too much about > the work folder. But it turns out, that the work folder actually has > some important stuff. > This improvement should help with this issue. > > What's about in-memory mode? As far as I know, we write binary metadata > to disk even when no persistence is configured. Do you plan to address > it in any way? > > Denis > > On 12.05.2020 15:56, Sergey Antonov wrote: >> Hello Semyon, >> >> This is a good idea! >> >> вт, 12 мая 2020 г. в 15:53, Вячеслав Коптилин : >> >>> Hello Semyon, >>> >>> This is a good and long-awaited improvement! Thank you for your efforts! >>> >>> Thanks, >>> S. >>> >>> вт, 12 мая 2020 г. в 15:11, Данилов Семён : >>> >>>> Hello! >>>> >>>> I would like to propose moving /binary_meta and /marshaller folders to >>> the >>>> PDS folder. >>>> >>>> Motivation: data, directly related to the persistence, is stored outside >>>> the persistence dir, which can lead to various issues and also is not >>> very >>>> convenient to use. In particular, with k8s, deployment disk that is >>>> attached to a container can not be accessed from other containers or >>>> outside of k8s. In case if support will need to drop persistence except >>>> data, there will be no way to recover due to binary metadata is required >>> to >>>> process PDS files. >>>> >>>> I created an issue (https://issues.apache.org/jira/browse/IGNITE-12994) >>> and a >>>> pull request(https://github.com/apache/ignite/pull/7792) that fixes the >>>> issue. >>>> >>>> In that PR I made the following: >>>> >>>> * store binary meta and marshaller data inside db/ folder >>>> * if binary meta of marshaller are found in "legacy" locations -- >>>> safely move them to new locations during the node startup >>>> >>>> Kind regards, >>>> >>>> Semyon Danilov.
Moving binary metadata to PDS storage folder
Hello! I would like to propose moving /binary_meta and /marshaller folders to the PDS folder. Motivation: data, directly related to the persistence, is stored outside the persistence dir, which can lead to various issues and also is not very convenient to use. In particular, with k8s, deployment disk that is attached to a container can not be accessed from other containers or outside of k8s. In case if support will need to drop persistence except data, there will be no way to recover due to binary metadata is required to process PDS files. I created an issue (https://issues.apache.org/jira/browse/IGNITE-12994) and a pull request(https://github.com/apache/ignite/pull/7792) that fixes the issue. In that PR I made the following: * store binary meta and marshaller data inside db/ folder * if binary meta of marshaller are found in "legacy" locations -- safely move them to new locations during the node startup Kind regards, Semyon Danilov.
Joining Ignite ASF
Hello! I would like to start contributing, could you please add me to contributors list? My login for https://issues.apache.org/ is sdanilov. Kind regards, Semyon Danilov.
