Re: Custom string encoding
Andrey, Can you elaborate more on this? What is your concern? -Val On Fri, Jun 30, 2017 at 6:17 PM Andrey Mashenkov wrote: > Val, > > Looks like make sense. > > This will not affect FullText index, as Lucene has own format for storing > data. > > But.. would it be compatible with H2 indexing ? I doubt. > > 1 июля 2017 г. 2:27 пользователь "Valentin Kulichenko" < > valentin.kuliche...@gmail.com> написал: > > > Folks, > > > > Currently binary marshaller always encodes strings in UTF-8. However, > > sometimes it can be useful to customize this. For example, if data > contains > > a lot of Cyrillic, Chinese or other symbols, but not so many Latin > symbols, > > memory is used very inefficiently. In this case it would be great to > encode > > most frequently used symbols in one byte instead of two or three. > > > > I propose to introduce BinaryStringEncoder interface that will convert > > strings to byte arrays and back, and make it pluggable via > > BinaryConfiguration. This will allow users to plug in any encoding > > algorithms based on their requirements. > > > > Thoughts? > > > > https://issues.apache.org/jira/browse/IGNITE-5655 > > > > -Val > > >
Re: Custom string encoding
Val, Looks like make sense. This will not affect FullText index, as Lucene has own format for storing data. But.. would it be compatible with H2 indexing ? I doubt. 1 июля 2017 г. 2:27 пользователь "Valentin Kulichenko" < valentin.kuliche...@gmail.com> написал: > Folks, > > Currently binary marshaller always encodes strings in UTF-8. However, > sometimes it can be useful to customize this. For example, if data contains > a lot of Cyrillic, Chinese or other symbols, but not so many Latin symbols, > memory is used very inefficiently. In this case it would be great to encode > most frequently used symbols in one byte instead of two or three. > > I propose to introduce BinaryStringEncoder interface that will convert > strings to byte arrays and back, and make it pluggable via > BinaryConfiguration. This will allow users to plug in any encoding > algorithms based on their requirements. > > Thoughts? > > https://issues.apache.org/jira/browse/IGNITE-5655 > > -Val >
Custom string encoding
Folks, Currently binary marshaller always encodes strings in UTF-8. However, sometimes it can be useful to customize this. For example, if data contains a lot of Cyrillic, Chinese or other symbols, but not so many Latin symbols, memory is used very inefficiently. In this case it would be great to encode most frequently used symbols in one byte instead of two or three. I propose to introduce BinaryStringEncoder interface that will convert strings to byte arrays and back, and make it pluggable via BinaryConfiguration. This will allow users to plug in any encoding algorithms based on their requirements. Thoughts? https://issues.apache.org/jira/browse/IGNITE-5655 -Val
[jira] [Created] (IGNITE-5655) Introduce pluggable string encoder/decoder
Valentin Kulichenko created IGNITE-5655:
---
Summary: Introduce pluggable string encoder/decoder
Key: IGNITE-5655
URL: https://issues.apache.org/jira/browse/IGNITE-5655
Project: Ignite
Issue Type: New Feature
Components: binary
Affects Versions: 2.0
Reporter: Valentin Kulichenko
Currently binary marshaller encodes strings in UTF-8. However, sometimes it
makes sense to customize this.
Need to:
# Create {{BinaryStringEncoder}} interface with {{encode}} and {{decode}}
methods that will convert string to byte array and back.
# Create default implementation based on UTF-8.
# Add {{stringEncoder}} configuration property to {{BinaryConfiguration}}.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
Re: Apache Ignite 2.1 scope
Igniters, It’s time to refresh this abandoned thread and finally rollout out all the changes appeared in 2.1. In addition, recently donated Persistent Store got the green light [1] to become a part of the master branch (no one asked for extra time to dive into its details) and, personally, it’s absolutely fine to make it available in the nearest release. My proposal is to do the release by mid of July (closer to July 15th). Is there anyone who is ready to take over as a release manager creating the page like this [2] and handling all release related activities? [1] http://apache-ignite-developers.2346864.n4.nabble.com/Ignite-Persistent-Store-Ready-for-merge-td19160.html [2] https://cwiki.apache.org/confluence/display/IGNITE/Apache+Ignite+2.0 — Denis > On Jun 1, 2017, at 9:24 AM, Alexander Paschenko > wrote: > > IGNITE-5327 Create predefined cache templates for CREATE TABLE command > - minor comments left, ETA is Friday. > > IGNITE-5380 Validate cache QueryEntities in discovery thread - in > progress, the meat of code is written, but need to add lots of tests. > ETA is Friday. > > IGNITE-5188 Support AFFINITY KEY keyword for CREATE TABLE command - in > progress, made few first small steps, ETA is Friday. > > Rest is closed/patch available, ignite-4994 has been moved to 2.2. > > - Alex > > 2017-06-01 19:03 GMT+03:00 Sergey Chugunov : >> 1. IGNITE-5386 Inactive mode must be forced on starting up grid with >> persistence is enabled >> It is important improvement to fix critical bug IGNITE-5363. >> Working on it, ETA - tomorrow. >> 2. IGNITE-5375 New PersistentStoreMetrics, MemoryMetrics interface >> improvements >> A lot of discussions were on this topic, ticket created only today and >> requires several days to implement. >> >> >> >> On Thu, Jun 1, 2017 at 6:56 PM, Taras Ledkov wrote: >> >>> Folks, >>> >>> IGNITE-4922 JDBC Driver: renew thin client based solution: >>> >>> On 2.1 the functionality of the new thin client JDBC driver will be >>> between deprecated Ignite thin JDBC and Ignite JDBCv2. >>> 1. The most functions of SQL query (include DML) are implemented and ready >>> for review; >>> 2. The most functions of JDBC metadata are implemented and ready for >>> review; >>> 3. Transactions, batching, streaming, blobs, scrollable / writable cursors >>> will not be supported in 2.1. >>> >>> >>> >>> On 01.06.2017 18:43, Vladimir Ozerov wrote: >>> Folks, We are almost reached proposed feature-complete date (June 2), Could you please share current status of your major features? On Tue, May 16, 2017 at 3:51 AM, Dmitriy Setrakyan wrote: Looks a little tight. Let's hope we can make it. > > On Mon, May 15, 2017 at 1:29 PM, Denis Magda wrote: > > Well, let me propose the following milestones for 2.1 release then. >> >> Code freeze: June 2nd. >> Final QA and benchmarking: June 5 - June 8 >> Voting: ~ June 9 >> Release: ~ June 13 >> >> Also I heard H2 has to be released once again to support Ignite’s CREATE >> table command. Think that we should talk to H2 folks to make it happen >> in >> June 22nd - June 2nd time frame. >> >> — >> Denis >> >> On May 11, 2017, at 2:26 AM, Pavel Tupitsyn >>> >> wrote: >> >>> As for .NET, I would propose to concentrate on peer deployment >>> >> (IGNITE-2492) >> >>> and related stuff, like IGNITE-1894 .NET: Delegate support in the API >>> >> via > >> extension methods. >>> >>> SQL Dependency does not look important to me, we can reschedule it for >>> later versions. >>> >>> On Thu, May 11, 2017 at 12:01 PM, Dmitriy Setrakyan < >>> >> dsetrak...@apache.org> >> >>> wrote: >>> >>> Vyacheslav, I think it is worth the research, but you should always >>> keep > >> data querying and indexing in mind. For example, I don't see how >>> by-page > >> compression will solve it. On Thu, May 11, 2017 at 1:52 AM, Vyacheslav Daradur < >>> daradu...@gmail.com> >> >>> wrote: Dmitriy, > > I'm researching a best way for this future. > > At the moment I found only one way (querying and indexing > compatible), > >> this > is per-objects-field compression. > > But there is a good proffit only for long strings or fields with > large > >> objects. > > Maybe it makes sense just to introduce compression for string fileds. > > I'm researching the new page-memory architecture as applied to > by-page > >> compression. > > 2017-05-11 11:30 GMT+03:00 Dmitriy Setrakyan : >> >>> On Thu, May 11, 2017 at 12
Re: Ignite Persistent Store: Ready for merge?
Well, since there are no objections I’ll refresh our discussion about 2.1 releasing considering the persistent store to be released as a part of it. — Denis > On Jun 28, 2017, at 5:54 PM, Dmitriy Setrakyan wrote: > > My vote would be to prepare the donation for the 2.1 release. It is been > out for about 1.5 months and the community was given enough time to > familiarize with the code. Denis has also conducted a well-attended > community webinar explaining the donated code in detail. > > If there are no objections, let's fix any outstanding bugs and merge it to > master. > > D. > > On Tue, Jun 27, 2017 at 3:35 PM, Denis Magda wrote: > >> Guys, >> >> According to the discussions I see on the @dev list the donated Ignite >> Persistent Store [1] is stable and ready to be rolled out to our users. >> >> Considering that we’ve been already getting to know the donation for a >> while and had the webinar dedicated to the functionality trying to answer >> on all the technical questions and resolve uncertainties, do you think it’s >> a good time to merge it to the master branch and start thinking of the >> feature release? Is there anyone who needs more time to look into the >> donation? >> >> [1] http://incubator.apache.org/ip-clearance/persistent- >> distributed-store-ignite.html >> >> — >> Denis
[jira] [Created] (IGNITE-5654) Distributed queries throw exception if joins with replicated caches are not the last on the list
Alexander Paschenko created IGNITE-5654: --- Summary: Distributed queries throw exception if joins with replicated caches are not the last on the list Key: IGNITE-5654 URL: https://issues.apache.org/jira/browse/IGNITE-5654 Project: Ignite Issue Type: Bug Components: sql Affects Versions: 2.0 Environment: Subj., this needs to be investigated. Reporter: Alexander Paschenko Fix For: 2.2 -- This message was sent by Atlassian JIRA (v6.4.14#64029)
Re: Distributed scheduling
I think this functionality should provide durable way of scheduled task or
closure execution on the cluster. Job descriptors should be persisted on
server side and executed there.
As for API, I believe this should be part of Compute Grid. I suggest to
introduce IgniteCompute#withSchedulingPolicy(SchedulingPolicy policy)
method, where SchedulingPolicy is smth like this:
public interface SchedulingPolicy {
/**
* @return Timestamp of next execution.
*/
public Date nextTime();
}
This will enable scheduling for all compute features (tasks, callables,
closures, etc.) and also very flexible. Policy implementation can provide
simple periodic scheduling, scheduling based on Cron or anything else.
Thoughts?
-Val
On Fri, Jun 30, 2017 at 7:55 AM, Dmitriy Setrakyan
wrote:
> On Fri, Jun 30, 2017 at 12:29 AM, Alexey Kuznetsov
> wrote:
>
> > Dmitriy,
> >
> > >> Can you provide a simple example of API calls that will make this
> > possible?
> > API could be like this:
> > 1) via scheduler:
> > Ignite ignite = Ignition.start();
> >
> > ignite.scheduler().schedulel(job, "0 0 * * *"); // This will execute job
> > every day at 00:00
> >
> > 2) via compute
> >
> > Ignite ignite = Ignition.start();
> >
> > ignite.compute().schedulel(task, "0 0 * * *"); // This will execute
> > compute
> > task every day at 00:00
> >
> > Make sense?
> >
> >
> Yes, it does, but I am failing to see how is this a *distributed*
> scheduling. Are we persisting the scheduler somewhere in the cluster or is
> it only triggered on the client side?
>
[jira] [Created] (IGNITE-5653) Add to query execution plan debug data for joins
Alexander Paschenko created IGNITE-5653: --- Summary: Add to query execution plan debug data for joins Key: IGNITE-5653 URL: https://issues.apache.org/jira/browse/IGNITE-5653 Project: Ignite Issue Type: Improvement Components: sql Affects Versions: 2.0 Environment: Plan should output table type (replicated/partitioned) and colocation information if possible. If we have this than we can warn (or throw exception) if users try to join non colocated tables with local joins. Reporter: Alexander Paschenko Fix For: 2.2 -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (IGNITE-5652) Print slow query warnings on client node
Alexander Paschenko created IGNITE-5652: --- Summary: Print slow query warnings on client node Key: IGNITE-5652 URL: https://issues.apache.org/jira/browse/IGNITE-5652 Project: Ignite Issue Type: Improvement Components: sql Affects Versions: 2.0 Environment: Currently, only worker (MAP) nodes of the query print long query execution time warning to their console, for usability it would be nice to propagate this to the client node as well. Reporter: Alexander Paschenko Fix For: 2.2 -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (IGNITE-5651) Add long query timeout property
Alexander Paschenko created IGNITE-5651: --- Summary: Add long query timeout property Key: IGNITE-5651 URL: https://issues.apache.org/jira/browse/IGNITE-5651 Project: Ignite Issue Type: Task Affects Versions: 2.0 Environment: It would be nice to allow users setting long execution timeout when sending a query - by doing so, user won't start getting dull long execution warnings which may be well expected. Instead, they will set their own acceptable timeout and will start seeing warnings only when they want to. Reporter: Alexander Paschenko Fix For: 2.2 -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (IGNITE-5650) Add convenient API for getting column values
Alexander Paschenko created IGNITE-5650: --- Summary: Add convenient API for getting column values Key: IGNITE-5650 URL: https://issues.apache.org/jira/browse/IGNITE-5650 Project: Ignite Issue Type: Task Affects Versions: 2.0 Reporter: Alexander Paschenko Fix For: 2.2 It's desirable to have some API for getting column values from query results as current API operates only rows (raw Lists). -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (IGNITE-5649) REST API, metadata commands always returns list of cache
Aleksandr created IGNITE-5649: - Summary: REST API, metadata commands always returns list of cache Key: IGNITE-5649 URL: https://issues.apache.org/jira/browse/IGNITE-5649 Project: Ignite Issue Type: Bug Affects Versions: 2.0 Reporter: Aleksandr How to reproduce: 1) start Ignite server with example config with Person and Organization. 2) run: curl http://localhost:8080/ignite?cmd=metadata&cacheName=Person curl http://localhost:8080/ignite?cmd=metadata&cacheName=Organization curl http://localhost:8080/ignite?cmd=metadata&cacheName=blablabla Ignite always returns list of cache instead of information about selected cache or error -- This message was sent by Atlassian JIRA (v6.4.14#64029)
Re: [jira] [Created] (IGNITE-5647) Suggestion for Apache Ignite Generic Transactional Receiver Implementation for Concurrency
Hi Fatih, You can find this information here: https://ignite.apache.org/community/contribute.html#ignite-dev -Val On Fri, Jun 30, 2017 at 11:43 AM, fatih wrote: > Hi > > Could please send a guide for a new committer explaining how to create a > branch to the dedicated Jira Ticket and other things that might be > necessary? > > Regards > > > > -- > View this message in context: http://apache-ignite- > developers.2346864.n4.nabble.com/jira-Created-IGNITE-5647- > Suggestion-for-Apache-Ignite-Generic-Transactional- > Receiver-Implementation-y-tp19313p19314.html > Sent from the Apache Ignite Developers mailing list archive at Nabble.com. >
Re: DataStreamer Transactional and Timestamp Implementation
Hi Fatih, This makes sense to me, but frankly I don't see anything that can be included in the product here. It's very specific to your case and doesn't add much value in general. Do you have a blog by any chance? :) It looks like a very good topic for an article (describing the use case, proposing solution, etc.). -Val On Fri, Jun 30, 2017 at 11:17 AM, fatih wrote: > Hi > > May I kindly ask if you had a chance to look into the implementation and > the > use case description in our previous post > > > > -- > View this message in context: http://apache-ignite- > developers.2346864.n4.nabble.com/DataStreamer-Transactional-and-Timestamp- > Implementation-tp19129p19312.html > Sent from the Apache Ignite Developers mailing list archive at Nabble.com. >
[jira] [Created] (IGNITE-5648) NOT NULL constraint support for CREATE TABLE operator
Denis Magda created IGNITE-5648:
---
Summary: NOT NULL constraint support for CREATE TABLE operator
Key: IGNITE-5648
URL: https://issues.apache.org/jira/browse/IGNITE-5648
Project: Ignite
Issue Type: Bug
Affects Versions: 2.0
Reporter: Denis Magda
Fix For: 2.3
This is an umbrella ticket intended to aggregate all the activities related to
{{NOT NULL}} constraint support for {{CREATE TABLE}} commands.
{code}
CREATE TABLE legs(legid INT NOT NULL);
{code}
Ignite must prevent setting {{legid}} to {{null}} value.
The feature has to be supported for:
* ODBC and JDBC drivers.
* Native APIs (Java, .NET, C++)
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
Re: [jira] [Created] (IGNITE-5647) Suggestion for Apache Ignite Generic Transactional Receiver Implementation for Concurrency
Hi Could please send a guide for a new committer explaining how to create a branch to the dedicated Jira Ticket and other things that might be necessary? Regards -- View this message in context: http://apache-ignite-developers.2346864.n4.nabble.com/jira-Created-IGNITE-5647-Suggestion-for-Apache-Ignite-Generic-Transactional-Receiver-Implementation-y-tp19313p19314.html Sent from the Apache Ignite Developers mailing list archive at Nabble.com.
[jira] [Created] (IGNITE-5647) Suggestion for Apache Ignite Generic Transactional Receiver Implementation for Concurrency
fatih created IGNITE-5647: - Summary: Suggestion for Apache Ignite Generic Transactional Receiver Implementation for Concurrency Key: IGNITE-5647 URL: https://issues.apache.org/jira/browse/IGNITE-5647 Project: Ignite Issue Type: Improvement Components: cache Reporter: fatih Fix For: 2.1 Hi Related to that ticket, I would like to commit the attached implementations as described in ticket http://apache-ignite-developers.2346864.n4.nabble.com/DataStreamer-Transactional-and-Timestamp-Implementation-td19129.html#a19312. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
Re: DataStreamer Transactional and Timestamp Implementation
Hi May I kindly ask if you had a chance to look into the implementation and the use case description in our previous post -- View this message in context: http://apache-ignite-developers.2346864.n4.nabble.com/DataStreamer-Transactional-and-Timestamp-Implementation-tp19129p19312.html Sent from the Apache Ignite Developers mailing list archive at Nabble.com.
[jira] [Created] (IGNITE-5646) Use affinity keys for distributed matrice blocks
Yury Babak created IGNITE-5646: -- Summary: Use affinity keys for distributed matrice blocks Key: IGNITE-5646 URL: https://issues.apache.org/jira/browse/IGNITE-5646 Project: Ignite Issue Type: New Feature Components: ml Reporter: Yury Babak Fix For: 2.2 We want to implement affinity collocation for distributed matrices. We must guarantee that the new block for computation result will be stored in the same node like the initial blocks -- This message was sent by Atlassian JIRA (v6.4.14#64029)
Re: By bytes access to binary format
On Fri, Jun 30, 2017 at 10:35 AM, Valentin Kulichenko <
valentin.kuliche...@gmail.com> wrote:
> Vova,
>
> Generally this can be useful. If you have a read-only binary object with a
> large blob as a field, you don't want to copy this array when reading it.
> Instead, we can return a ByteBuffer or a stream wrapping the corresponding
> portion.
>
> However, I currently don't see how this can be smoothly added to existing
> API. Vlad, do you have any concrete proposal on how it should look like?
>
Huge +1 from me. We should not require a copy for read-only data. We should
give users an ability to get the original byte stream, especially if it is
immutable.
>
> -Val
>
> On Thu, Jun 29, 2017 at 2:11 PM, Vladimir Ozerov
> wrote:
>
> > Hi Vlad,
> >
> > I am not quite sure I understand the problem. Can you show how the API
> you
> > propose would look like? Remember that "field" method can return anything
> > from primitive, String or byte array, to another BinaryObject. And
> returned
> > BinaryObject can have references outside of itself, so it cannot be
> > serialized easily without full rebuild. .
> >
> > On Thu, Jun 29, 2017 at 10:16 AM, Vladislav Pyatkov <
> vpyat...@gridgain.com
> > >
> > wrote:
> >
> > > Val,
> > >
> > > I proposal, access as a stream to binary object, because we have
> doubled
> > > copy on touch a field (first at copy from cache and second - on
> getting a
> > > field).
> > >
> > > For the stream in/out to cache I will be used IGFS.
> > > Main idea to avoid GC pressure when make a massive read from key-value
> > > storage.
> > >
> > > On Wed, Jun 28, 2017 at 9:36 PM, Valentin Kulichenko <
> > > valentin.kuliche...@gmail.com> wrote:
> > >
> > > > Vladislav,
> > > >
> > > > Are you suggesting to stream directly from cache. or from a binary
> > object
> > > > that is already copied from cache?
> > > >
> > > > -Val
> > > >
> > > > On Wed, Jun 28, 2017 at 2:52 AM, Vladislav Pyatkov <
> > > vpyat...@gridgain.com>
> > > > wrote:
> > > >
> > > > > Hi,
> > > > >
> > > > > Recently, from one of Ignite user, I listened interest idea.
> > > > > What if I want to pass some date to java stream from cache.
> > > > >
> > > > > With binary I do it like this:
> > > > >
> > > > > BinaryObject get = (BinaryObject) cache.get(key);
> > > > > byte[] dataFromCache = get.field("data");
> > > > > System.out.write(dataFromCache, 0, dataFromCache.length);
> > > > >
> > > > > But in this case we got garbage a lot, due to each time new bytes
> > array
> > > > is
> > > > > creating.
> > > > >
> > > > > This will lead to many GC events in case we load a some of million
> > > > entries.
> > > > > Could we offer additional API for working with java stream:
> > > > >
> > > > > BinaryObject.writeBytesToBuf("data", ByteBuffer.allocate(1024));
> > > > >
> > > > > or with buffer
> > > > >
> > > > > BinaryObject.writeBytesToBuf("data", new byte[1000], 100);
> > > > >
> > > > > I already created a Jira ticket.
> > > > > https://issues.apache.org/jira/browse/IGNITE-5602
> > > > >
> > > > > --
> > > > > Vladislav Pyatkov
> > > > > Architect-Consultant "GridGain Rus" Llc.
> > > > > +7 963 716 68 99
> > > > >
> > > >
> > >
> > >
> > >
> > > --
> > > Vladislav Pyatkov
> > > Architect-Consultant "GridGain Rus" Llc.
> > > +7 963 716 68 99
> > >
> >
>
Re: By bytes access to binary format
Vova,
Generally this can be useful. If you have a read-only binary object with a
large blob as a field, you don't want to copy this array when reading it.
Instead, we can return a ByteBuffer or a stream wrapping the corresponding
portion.
However, I currently don't see how this can be smoothly added to existing
API. Vlad, do you have any concrete proposal on how it should look like?
-Val
On Thu, Jun 29, 2017 at 2:11 PM, Vladimir Ozerov
wrote:
> Hi Vlad,
>
> I am not quite sure I understand the problem. Can you show how the API you
> propose would look like? Remember that "field" method can return anything
> from primitive, String or byte array, to another BinaryObject. And returned
> BinaryObject can have references outside of itself, so it cannot be
> serialized easily without full rebuild. .
>
> On Thu, Jun 29, 2017 at 10:16 AM, Vladislav Pyatkov >
> wrote:
>
> > Val,
> >
> > I proposal, access as a stream to binary object, because we have doubled
> > copy on touch a field (first at copy from cache and second - on getting a
> > field).
> >
> > For the stream in/out to cache I will be used IGFS.
> > Main idea to avoid GC pressure when make a massive read from key-value
> > storage.
> >
> > On Wed, Jun 28, 2017 at 9:36 PM, Valentin Kulichenko <
> > valentin.kuliche...@gmail.com> wrote:
> >
> > > Vladislav,
> > >
> > > Are you suggesting to stream directly from cache. or from a binary
> object
> > > that is already copied from cache?
> > >
> > > -Val
> > >
> > > On Wed, Jun 28, 2017 at 2:52 AM, Vladislav Pyatkov <
> > vpyat...@gridgain.com>
> > > wrote:
> > >
> > > > Hi,
> > > >
> > > > Recently, from one of Ignite user, I listened interest idea.
> > > > What if I want to pass some date to java stream from cache.
> > > >
> > > > With binary I do it like this:
> > > >
> > > > BinaryObject get = (BinaryObject) cache.get(key);
> > > > byte[] dataFromCache = get.field("data");
> > > > System.out.write(dataFromCache, 0, dataFromCache.length);
> > > >
> > > > But in this case we got garbage a lot, due to each time new bytes
> array
> > > is
> > > > creating.
> > > >
> > > > This will lead to many GC events in case we load a some of million
> > > entries.
> > > > Could we offer additional API for working with java stream:
> > > >
> > > > BinaryObject.writeBytesToBuf("data", ByteBuffer.allocate(1024));
> > > >
> > > > or with buffer
> > > >
> > > > BinaryObject.writeBytesToBuf("data", new byte[1000], 100);
> > > >
> > > > I already created a Jira ticket.
> > > > https://issues.apache.org/jira/browse/IGNITE-5602
> > > >
> > > > --
> > > > Vladislav Pyatkov
> > > > Architect-Consultant "GridGain Rus" Llc.
> > > > +7 963 716 68 99
> > > >
> > >
> >
> >
> >
> > --
> > Vladislav Pyatkov
> > Architect-Consultant "GridGain Rus" Llc.
> > +7 963 716 68 99
> >
>
[jira] [Created] (IGNITE-5645) Locking mechanism for distributed matrices.
Yury Babak created IGNITE-5645: -- Summary: Locking mechanism for distributed matrices. Key: IGNITE-5645 URL: https://issues.apache.org/jira/browse/IGNITE-5645 Project: Ignite Issue Type: New Feature Components: ml Reporter: Yury Babak Fix For: 2.2 We must to have mechanism for protect distributed matrix from changes during calculations. Current locking mechanism is bad choice for locking a huge cache keyset, so we need a new one. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[GitHub] ignite pull request #2221: IGNITE-3935
GitHub user Grigory2000 opened a pull request: https://github.com/apache/ignite/pull/2221 IGNITE-3935 Added test for Ignite-3935 issue. You can merge this pull request into a Git repository by running: $ git pull https://github.com/Grigory2000/ignite Ignite-3935 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/ignite/pull/2221.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2221 commit 66ede2de4fafbc9ae63abaac8623d4fb89f267f9 Author: Grigory Date: 2017-06-30T14:32:13Z StreamingExample for reproducing the issue added commit 7e06da526adb96bf09a094107118041654ca6ed7 Author: Grigory Date: 2017-06-30T16:26:11Z Added self test/testsuite according to other tests --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Created] (IGNITE-5644) Metrics collection must be removed from discovery thread.
Mikhail Cherkasov created IGNITE-5644: - Summary: Metrics collection must be removed from discovery thread. Key: IGNITE-5644 URL: https://issues.apache.org/jira/browse/IGNITE-5644 Project: Ignite Issue Type: Bug Components: cache Reporter: Mikhail Cherkasov Assignee: Mikhail Cherkasov Fix For: 2.1 Cache metrics are copied in discovery worker threads. This looks a bit risky because in case of metrics collection may stall the whole cluster. We need to make sure that when the heartbeat message is processed, we already have a metrics snapshot enabled -- This message was sent by Atlassian JIRA (v6.4.14#64029)
Re: distributed transaction of non-single coordinator
Thanks! Do you think all test scenarios results, presented in table(in ticket comments) , are acceptable ? пт, 30 июн. 2017 г., 18:28 Yakov Zhdanov : > Alex, I have commented in the ticket. Please take a look. > > Thanks! > -- > Yakov Zhdanov, Director R&D > *GridGain Systems* > www.gridgain.com > > 2017-06-29 17:27 GMT+03:00 ALEKSEY KUZNETSOV : > > > I've attached HangTest. I suppose it should not hang, am i right ? > > > > чт, 29 июн. 2017 г. в 14:54, ALEKSEY KUZNETSOV >: > > > > > Igntrs. > > > Im rewieving all usages of threadId of > > > transaction.(IgniteTxAdapter#threadID). What is the point of usage > > threadId > > > in mvcc entry ? > > > > > > пн, 3 апр. 2017 г. в 9:47, ALEKSEY KUZNETSOV >: > > > > > >> so what do u think on my idea? > > >> > > >> пт, 31 Мар 2017 г., 11:05 ALEKSEY KUZNETSOV >: > > >> > > >>> sorry for misleading you. We planned to support multi-node > > transactions, > > >>> but failed. > > >>> > > >>> пт, 31 мар. 2017 г. в 10:51, Alexey Goncharuk < > > >>> alexey.goncha...@gmail.com>: > > >>> > > >>> Well, now the scenario is more clear, but it has nothing to do with > > >>> multiple coordinators :) Let me think a little bit about it. > > >>> > > >>> 2017-03-31 9:53 GMT+03:00 ALEKSEY KUZNETSOV < > alkuznetsov...@gmail.com > > >: > > >>> > > >>> > so what do u think on the issue ? > > >>> > > > >>> > чт, 30 Мар 2017 г., 17:49 ALEKSEY KUZNETSOV < > > alkuznetsov...@gmail.com > > >>> >: > > >>> > > > >>> > > Hi ! Thanks for help. I've created ticket : > > >>> > > https://issues.apache.org/jira/browse/IGNITE-4887 > > >>> > > and a commit : > > >>> > > > > >>> > https://github.com/voipp/ignite/commit/aa3487bd9c203394f534c605f84e06 > > >>> > 436b638e5c > > >>> > > We really need this feature > > >>> > > > > >>> > > чт, 30 мар. 2017 г. в 11:31, Alexey Goncharuk < > > >>> > alexey.goncha...@gmail.com > > >>> > > >: > > >>> > > > > >>> > > Aleksey, > > >>> > > > > >>> > > I doubt your approach works as expected. Current transaction > > recovery > > >>> > > protocol heavily relies on the originating node ID in its > internal > > >>> logic. > > >>> > > For example, currently a transaction will be rolled back if you > > want > > >>> to > > >>> > > transfer a transaction ownership to another node and original tx > > >>> owner > > >>> > > fails. An attempt to commit such a transaction on another node > may > > >>> fail > > >>> > > with all sorts of assertions. After transaction ownership > changed, > > >>> you > > >>> > need > > >>> > > to notify all current transaction participants about this change, > > >>> and it > > >>> > > should also be done failover-safe, let alone that you did not add > > any > > >>> > tests > > >>> > > for these cases. > > >>> > > > > >>> > > I back Denis here. Please create a ticket first and come up with > > >>> clear > > >>> > > use-cases, API and protocol changes design. It is hard to reason > > >>> about > > >>> > the > > >>> > > changes you've made when we do not even understand why you are > > making > > >>> > these > > >>> > > changes and how they are supposed to work. > > >>> > > > > >>> > > --AG > > >>> > > > > >>> > > 2017-03-30 10:43 GMT+03:00 ALEKSEY KUZNETSOV < > > >>> alkuznetsov...@gmail.com>: > > >>> > > > > >>> > > > So, what do u think on my idea ? > > >>> > > > > > >>> > > > ср, 29 мар. 2017 г. в 10:35, ALEKSEY KUZNETSOV < > > >>> > alkuznetsov...@gmail.com > > >>> > > >: > > >>> > > > > > >>> > > > > Hi! No, i dont have ticket for this. > > >>> > > > > In the ticket i have implemented methods that change > > transaction > > >>> > status > > >>> > > > to > > >>> > > > > STOP, thus letting it to commit transaction in another > thread. > > In > > >>> > > another > > >>> > > > > thread you r going to restart transaction in order to commit > > it. > > >>> > > > > The mechanism behind it is obvious : we change thread id to > > >>> newer one > > >>> > > in > > >>> > > > > ThreadMap, and make use of serialization of txState, > > transactions > > >>> > > itself > > >>> > > > to > > >>> > > > > transfer them into another thread. > > >>> > > > > > > >>> > > > > > > >>> > > > > вт, 28 мар. 2017 г. в 20:15, Denis Magda >: > > >>> > > > > > > >>> > > > > Aleksey, > > >>> > > > > > > >>> > > > > Do you have a ticket for this? Could you briefly list what > > >>> exactly > > >>> > was > > >>> > > > > done and how the things work. > > >>> > > > > > > >>> > > > > — > > >>> > > > > Denis > > >>> > > > > > > >>> > > > > > On Mar 28, 2017, at 8:32 AM, ALEKSEY KUZNETSOV < > > >>> > > > alkuznetsov...@gmail.com> > > >>> > > > > wrote: > > >>> > > > > > > > >>> > > > > > Hi, Igniters! I 've made implementation of transactions of > > >>> > non-single > > >>> > > > > > coordinator. Here you can start transaction in one thread > and > > >>> > commit > > >>> > > it > > >>> > > > > in > > >>> > > > > > another thread. > > >>> > > > > > Take a look on it. Give your thoughts on it. > > >>> > > > > > > > >>> > > > > > > > >>> > > > > https://github.com/voipp/ignite/pull/10
[jira] [Created] (IGNITE-5643) REST API, call cache command without cacheName param - got java.lang.NullPointerException
Aleksandr created IGNITE-5643:
-
Summary: REST API, call cache command without cacheName param -
got java.lang.NullPointerException
Key: IGNITE-5643
URL: https://issues.apache.org/jira/browse/IGNITE-5643
Project: Ignite
Issue Type: Bug
Affects Versions: 2.0
Environment: any
Reporter: Aleksandr
Priority: Minor
Steps to reproduce:
1) start Ignite with REST API enabled
2) run:
curl http://localhost:8080/ignite?cmd=cache
3) got on server:
[18:50:42,296][SEVERE][qtp1985938863-60][GridJettyRestProtocol] Failed to
process HTTP request [action=/ignite, req=(GET /ignite?cmd=cache)@359127791
org.eclipse.jetty.server.Request@1567daef]
class org.apache.ignite.IgniteCheckedException: null
at
org.apache.ignite.internal.util.IgniteUtils.cast(IgniteUtils.java:7242)
at
org.apache.ignite.internal.processors.rest.GridRestProcessor$2.body(GridRestProcessor.java:172)
at
org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:110)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.NullPointerException
at
java.util.concurrent.ConcurrentHashMap.get(ConcurrentHashMap.java:936)
at
org.apache.ignite.internal.processors.cache.GridCacheProcessor.cache(GridCacheProcessor.java:3439)
at
org.apache.ignite.internal.processors.rest.handlers.cache.GridCacheCommandHandler.localCache(GridCacheCommandHandler.java:752)
at
org.apache.ignite.internal.processors.rest.handlers.cache.GridCacheCommandHandler.executeCommand(GridCacheCommandHandler.java:716)
at
org.apache.ignite.internal.processors.rest.handlers.cache.GridCacheCommandHandler.handleAsync(GridCacheCommandHandler.java:582)
at
org.apache.ignite.internal.processors.rest.GridRestProcessor.handleRequest(GridRestProcessor.java:266)
at
org.apache.ignite.internal.processors.rest.GridRestProcessor.access$100(GridRestProcessor.java:89)
at
org.apache.ignite.internal.processors.rest.GridRestProcessor$2.body(GridRestProcessor.java:155)
4) HTTP response body:
{
"successStatus": 1,
"error": null,
"response": null,
"sessionToken": null
}
5) Documentation tells cacheName is optional:
https://apacheignite.readme.io/docs/rest-api#section-cache-metrics
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[GitHub] ignite pull request #2216: For testing
Github user mcherkasov closed the pull request at: https://github.com/apache/ignite/pull/2216 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
Re: distributed transaction of non-single coordinator
Alex, I have commented in the ticket. Please take a look. Thanks! -- Yakov Zhdanov, Director R&D *GridGain Systems* www.gridgain.com 2017-06-29 17:27 GMT+03:00 ALEKSEY KUZNETSOV : > I've attached HangTest. I suppose it should not hang, am i right ? > > чт, 29 июн. 2017 г. в 14:54, ALEKSEY KUZNETSOV : > > > Igntrs. > > Im rewieving all usages of threadId of > > transaction.(IgniteTxAdapter#threadID). What is the point of usage > threadId > > in mvcc entry ? > > > > пн, 3 апр. 2017 г. в 9:47, ALEKSEY KUZNETSOV : > > > >> so what do u think on my idea? > >> > >> пт, 31 Мар 2017 г., 11:05 ALEKSEY KUZNETSOV : > >> > >>> sorry for misleading you. We planned to support multi-node > transactions, > >>> but failed. > >>> > >>> пт, 31 мар. 2017 г. в 10:51, Alexey Goncharuk < > >>> alexey.goncha...@gmail.com>: > >>> > >>> Well, now the scenario is more clear, but it has nothing to do with > >>> multiple coordinators :) Let me think a little bit about it. > >>> > >>> 2017-03-31 9:53 GMT+03:00 ALEKSEY KUZNETSOV >: > >>> > >>> > so what do u think on the issue ? > >>> > > >>> > чт, 30 Мар 2017 г., 17:49 ALEKSEY KUZNETSOV < > alkuznetsov...@gmail.com > >>> >: > >>> > > >>> > > Hi ! Thanks for help. I've created ticket : > >>> > > https://issues.apache.org/jira/browse/IGNITE-4887 > >>> > > and a commit : > >>> > > > >>> https://github.com/voipp/ignite/commit/aa3487bd9c203394f534c605f84e06 > >>> > 436b638e5c > >>> > > We really need this feature > >>> > > > >>> > > чт, 30 мар. 2017 г. в 11:31, Alexey Goncharuk < > >>> > alexey.goncha...@gmail.com > >>> > > >: > >>> > > > >>> > > Aleksey, > >>> > > > >>> > > I doubt your approach works as expected. Current transaction > recovery > >>> > > protocol heavily relies on the originating node ID in its internal > >>> logic. > >>> > > For example, currently a transaction will be rolled back if you > want > >>> to > >>> > > transfer a transaction ownership to another node and original tx > >>> owner > >>> > > fails. An attempt to commit such a transaction on another node may > >>> fail > >>> > > with all sorts of assertions. After transaction ownership changed, > >>> you > >>> > need > >>> > > to notify all current transaction participants about this change, > >>> and it > >>> > > should also be done failover-safe, let alone that you did not add > any > >>> > tests > >>> > > for these cases. > >>> > > > >>> > > I back Denis here. Please create a ticket first and come up with > >>> clear > >>> > > use-cases, API and protocol changes design. It is hard to reason > >>> about > >>> > the > >>> > > changes you've made when we do not even understand why you are > making > >>> > these > >>> > > changes and how they are supposed to work. > >>> > > > >>> > > --AG > >>> > > > >>> > > 2017-03-30 10:43 GMT+03:00 ALEKSEY KUZNETSOV < > >>> alkuznetsov...@gmail.com>: > >>> > > > >>> > > > So, what do u think on my idea ? > >>> > > > > >>> > > > ср, 29 мар. 2017 г. в 10:35, ALEKSEY KUZNETSOV < > >>> > alkuznetsov...@gmail.com > >>> > > >: > >>> > > > > >>> > > > > Hi! No, i dont have ticket for this. > >>> > > > > In the ticket i have implemented methods that change > transaction > >>> > status > >>> > > > to > >>> > > > > STOP, thus letting it to commit transaction in another thread. > In > >>> > > another > >>> > > > > thread you r going to restart transaction in order to commit > it. > >>> > > > > The mechanism behind it is obvious : we change thread id to > >>> newer one > >>> > > in > >>> > > > > ThreadMap, and make use of serialization of txState, > transactions > >>> > > itself > >>> > > > to > >>> > > > > transfer them into another thread. > >>> > > > > > >>> > > > > > >>> > > > > вт, 28 мар. 2017 г. в 20:15, Denis Magda : > >>> > > > > > >>> > > > > Aleksey, > >>> > > > > > >>> > > > > Do you have a ticket for this? Could you briefly list what > >>> exactly > >>> > was > >>> > > > > done and how the things work. > >>> > > > > > >>> > > > > — > >>> > > > > Denis > >>> > > > > > >>> > > > > > On Mar 28, 2017, at 8:32 AM, ALEKSEY KUZNETSOV < > >>> > > > alkuznetsov...@gmail.com> > >>> > > > > wrote: > >>> > > > > > > >>> > > > > > Hi, Igniters! I 've made implementation of transactions of > >>> > non-single > >>> > > > > > coordinator. Here you can start transaction in one thread and > >>> > commit > >>> > > it > >>> > > > > in > >>> > > > > > another thread. > >>> > > > > > Take a look on it. Give your thoughts on it. > >>> > > > > > > >>> > > > > > > >>> > > > > https://github.com/voipp/ignite/pull/10/commits/ > >>> > > > 3a3d90aa6ac84f125e4c3ce4ced4f269a695ef45 > >>> > > > > > > >>> > > > > > пт, 17 мар. 2017 г. в 19:26, Sergi Vladykin < > >>> > > sergi.vlady...@gmail.com > >>> > > > >: > >>> > > > > > > >>> > > > > >> You know better, go ahead! :) > >>> > > > > >> > >>> > > > > >> Sergi > >>> > > > > >> > >>> > > > > >> 2017-03-17 16:16 GMT+03:00 ALEKSEY KUZNETSOV < > >>> > > > alkuznetsov...@gmail.com > >>> > > > > >: > >>> > > > > >> > >>> > > > > >>> we've discovered several proble
Re: Distributed scheduling
On Fri, Jun 30, 2017 at 12:29 AM, Alexey Kuznetsov wrote: > Dmitriy, > > >> Can you provide a simple example of API calls that will make this > possible? > API could be like this: > 1) via scheduler: > Ignite ignite = Ignition.start(); > > ignite.scheduler().schedulel(job, "0 0 * * *"); // This will execute job > every day at 00:00 > > 2) via compute > > Ignite ignite = Ignition.start(); > > ignite.compute().schedulel(task, "0 0 * * *"); // This will execute > compute > task every day at 00:00 > > Make sense? > > Yes, it does, but I am failing to see how is this a *distributed* scheduling. Are we persisting the scheduler somewhere in the cluster or is it only triggered on the client side?
[GitHub] ignite pull request #1975: IGNITE-5113: Implemented basic distributed/local ...
Github user artemmalykh closed the pull request at: https://github.com/apache/ignite/pull/1975 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] ignite pull request #2096: ignite-5383
Github user asfgit closed the pull request at: https://github.com/apache/ignite/pull/2096 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
Re: Request for contributor permission
Hi, Added you to the contributors list. Please go ahead and assign the tickets on yourself. — Denis > On Jun 30, 2017, at 12:10 AM, Александр Метерко > wrote: > > Dear Ignite team, > > I would like to start contributing to your project starting with ticket > https://issues.apache.org/jira/browse/IGNITE-5592 . Could you grant me > permissions in Jira to assign this ticket to me? My login is ameterko. > > Thanks in advance.
[jira] [Created] (IGNITE-5642) Web Console: improve usability of caches and metadata on Queries screen
Alexey Kuznetsov created IGNITE-5642: Summary: Web Console: improve usability of caches and metadata on Queries screen Key: IGNITE-5642 URL: https://issues.apache.org/jira/browse/IGNITE-5642 Project: Ignite Issue Type: Task Components: wizards Reporter: Alexey Kuznetsov Assignee: Alexey Kuznetsov -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (IGNITE-5641) Web Console: In addition to export also implement copy result set to clipboard.
Alexey Kuznetsov created IGNITE-5641: Summary: Web Console: In addition to export also implement copy result set to clipboard. Key: IGNITE-5641 URL: https://issues.apache.org/jira/browse/IGNITE-5641 Project: Ignite Issue Type: Task Components: wizards Reporter: Alexey Kuznetsov Assignee: Alexey Kuznetsov -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (IGNITE-5640) Web Console: when possible do not block UI with modal windows
Alexey Kuznetsov created IGNITE-5640: Summary: Web Console: when possible do not block UI with modal windows Key: IGNITE-5640 URL: https://issues.apache.org/jira/browse/IGNITE-5640 Project: Ignite Issue Type: Task Components: wizards Reporter: Alexey Kuznetsov Assignee: Alexey Kuznetsov -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (IGNITE-5639) Web Console: Need to printout time taken evenif result set is empty.
Alexey Kuznetsov created IGNITE-5639: Summary: Web Console: Need to printout time taken evenif result set is empty. Key: IGNITE-5639 URL: https://issues.apache.org/jira/browse/IGNITE-5639 Project: Ignite Issue Type: Task Components: wizards Reporter: Alexey Kuznetsov Assignee: Alexey Kuznetsov -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (IGNITE-5638) Web Console: Add ability to cancel query
Alexey Kuznetsov created IGNITE-5638: Summary: Web Console: Add ability to cancel query Key: IGNITE-5638 URL: https://issues.apache.org/jira/browse/IGNITE-5638 Project: Ignite Issue Type: Task Components: wizards Reporter: Alexey Kuznetsov Assignee: Alexey Kuznetsov -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (IGNITE-5637) Web Console: Reset previous results when executing new query
Alexey Kuznetsov created IGNITE-5637: Summary: Web Console: Reset previous results when executing new query Key: IGNITE-5637 URL: https://issues.apache.org/jira/browse/IGNITE-5637 Project: Ignite Issue Type: Task Components: wizards Reporter: Alexey Kuznetsov Assignee: Alexey Kuznetsov -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (IGNITE-5636) Web Console: result "set too large" and no info about node on which error occurred and stack trace was unavailable.
Alexey Kuznetsov created IGNITE-5636: Summary: Web Console: result "set too large" and no info about node on which error occurred and stack trace was unavailable. Key: IGNITE-5636 URL: https://issues.apache.org/jira/browse/IGNITE-5636 Project: Ignite Issue Type: Task Components: wizards Reporter: Alexey Kuznetsov Assignee: Alexey Kuznetsov Hack Ignite and make it throw this exception each time for queries from WC- org.apache.ignite.internal.processors.query.h2.twostep.GridMergeIndex#checkBounds I had impression that stack trace has not been sent to WC. Please check it for any message/operation. WC should always show details for exceptions. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (IGNITE-5635) Web Console: Show query execution progress
Alexey Kuznetsov created IGNITE-5635: Summary: Web Console: Show query execution progress Key: IGNITE-5635 URL: https://issues.apache.org/jira/browse/IGNITE-5635 Project: Ignite Issue Type: Task Reporter: Alexey Kuznetsov Assignee: Alexey Kuznetsov Show query execution progress, some progress or spinning wheel. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (IGNITE-5634) Web Console: add ability to share query notebooks (for token / user)
Alexey Kuznetsov created IGNITE-5634: Summary: Web Console: add ability to share query notebooks (for token / user) Key: IGNITE-5634 URL: https://issues.apache.org/jira/browse/IGNITE-5634 Project: Ignite Issue Type: Task Reporter: Alexey Kuznetsov Add ability to share query notebooks (for token / user). Can be useful to share complex SQL queries -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (IGNITE-5633) Web Console: Agent hangs on flaky connection to cluster
Alexey Kuznetsov created IGNITE-5633: Summary: Web Console: Agent hangs on flaky connection to cluster Key: IGNITE-5633 URL: https://issues.apache.org/jira/browse/IGNITE-5633 Project: Ignite Issue Type: Task Components: wizards Reporter: Alexey Kuznetsov Assignee: Alexey Kuznetsov Agent hangs on flaky connection to cluster Launch agent on a linux machine and periodically drop packets to/from cluster with ip tables but leave console.gridgain.com available. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (IGNITE-5632) Web Console: add some kind of agent management and administration.
Alexey Kuznetsov created IGNITE-5632: Summary: Web Console: add some kind of agent management and administration. Key: IGNITE-5632 URL: https://issues.apache.org/jira/browse/IGNITE-5632 Project: Ignite Issue Type: Task Components: wizards Reporter: Alexey Kuznetsov Assignee: Alexey Kuznetsov In admin mode we should see all connected agents and mange their tokens lists (add / remove). May by stop/restart/agent logs view will be also useful. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (IGNITE-5631) Can't write value greater then wal segment
Alexander Belyak created IGNITE-5631: Summary: Can't write value greater then wal segment Key: IGNITE-5631 URL: https://issues.apache.org/jira/browse/IGNITE-5631 Project: Ignite Issue Type: Bug Components: persistence Affects Versions: 2.1 Reporter: Alexander Belyak Priority: Minor Fix For: 2.1 Step to reproduce: insert value greater then wal segment size. Expected behavior: get few wal segments and insert value Current behavior: infinite writing of wal archive For test I use 256Kb of WAL segment size and value from 10M length String. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (IGNITE-5630) Exception for node disconnect
Sergey Kozlov created IGNITE-5630:
-
Summary: Exception for node disconnect
Key: IGNITE-5630
URL: https://issues.apache.org/jira/browse/IGNITE-5630
Project: Ignite
Issue Type: Bug
Affects Versions: 1.8
Reporter: Sergey Kozlov
Fix For: 2.1
{noformat}
[09:27:05,772][SEVERE][tcp-comm-worker-#1%null%][TcpCommunicationSpi] Failed to
establish connection to a remote node [node=TcpDiscoveryNode
[id=f0d08a47-cccf-48b2-aeb2-9591baae6726, addrs=[127.0.0.1, 172.17.42.1,
172.25.2.24], sockAddrs=[/127.0.0.1:47503, agent-24/172.25.2.24:47503,
/172.17.42.1:47503], discPort=47503, order=88, intOrder=46,
lastExchangeTime=1498804024671, loc=false, ver=1.8.8#20170629-sha1:82e7d269,
isClient=false], addr=/127.0.0.1:47103, connectAttempts=2,
failureDetThrReachedfalse]
class org.apache.ignite.internal.cluster.ClusterTopologyCheckedException:
Failed to send message (node left topology): TcpDiscoveryNode
[id=f0d08a47-cccf-48b2-aeb2-9591baae6726, addrs=[127.0.0.1, 172.17.42.1,
172.25.2.24], sockAddrs=[/127.0.0.1:47503, agent-24/172.25.2.24:47503,
/172.17.42.1:47503], discPort=47503, order=88, intOrder=46,
lastExchangeTime=1498804024671, loc=false, ver=1.8.8#20170629-sha1:82e7d269,
isClient=false]
at
org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.createTcpClient(TcpCommunicationSpi.java:2841)
at
org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.createNioClient(TcpCommunicationSpi.java:2597)
at
org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.reserveClient(TcpCommunicationSpi.java:2484)
at
org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.access$5800(TcpCommunicationSpi.java:245)
at
org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi$CommunicationWorker.processDisconnect(TcpCommunicationSpi.java:3859)
at
org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi$CommunicationWorker.body(TcpCommunicationSpi.java:3685)
at org.apache.ignite.spi.IgniteSpiThread.run(IgniteSpiThread.java:62)
{noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Created] (IGNITE-5629) .NET: CacheConfiguration copy constructor
Pavel Tupitsyn created IGNITE-5629: -- Summary: .NET: CacheConfiguration copy constructor Key: IGNITE-5629 URL: https://issues.apache.org/jira/browse/IGNITE-5629 Project: Ignite Issue Type: Improvement Components: platforms Reporter: Pavel Tupitsyn Assignee: Pavel Tupitsyn Fix For: 2.1 Can be useful to start more caches with existing config -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (IGNITE-5628) .NET: incorrect jvm.dll lookup paths for JRE
Pavel Tupitsyn created IGNITE-5628:
--
Summary: .NET: incorrect jvm.dll lookup paths for JRE
Key: IGNITE-5628
URL: https://issues.apache.org/jira/browse/IGNITE-5628
Project: Ignite
Issue Type: Bug
Components: platforms
Reporter: Pavel Tupitsyn
Assignee: Pavel Tupitsyn
Fix For: 2.1
{{jvm.dll}} is located in different subfolders in JRE and JDK. Current code
only accounts for JDK. So if we set {{JAVA_HOME}} to a custom xcopy-deployed
JRE folder, jvm.dll can not be found.
With normal Windows installations it works because we use registry for that.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
RE: golang client for Ignite
Dear Semyon, Thanks. Understood. ((( Thanks, Aleksandr From: Semyon Boikov Sent: 30 июня 2017 г. 10:51 To: dev@ignite.apache.org Cc: Roman Shtykh Subject: Re: golang client for Ignite Hi Aleksandr, Regarding transactions: now implementation of transactions assumes that transaction always belongs to some Ignite node, and it seems it is not simple task to support transactions for 'thin' clients. Thanks On Fri, Jun 30, 2017 at 9:55 AM, Aleksandr Sokolovskii wrote: > Dear Vladimir, > > Thanks for your answer. > > Now I understand the difference between ‘client’ and ‘thin client’. > > Just to be clear I develop ‘thin client’ for DataGrid now: > https://github.com/amsokol/go-ignite-client > and all my questions are in context of ‘thin client’. > > So the questions are: > 1) Does EdgeNode provide binary endpoint that supports cache management, > put-get, sql exec queries, fetch, transactions and I can use it by ‘thin’ > golang client? > 2) If the answer to the question (1) is ‘NO’ – is it possible to add > transactions to ODBC endpoint? > 3) If the answer to the question (2) is ‘NO’ – is it possible to add > transactions to REST API? > > Thanks, > Aleksandr > > From: Vladimir Ozerov > Sent: 30 июня 2017 г. 8:18 > To: dev@ignite.apache.org > Cc: Roman Shtykh > Subject: Re: golang client for Ignite > > Aleksandr, > > Currently Ignite.NET and Ignite C++ work as follows: > >>> Client[platform->*jni->java*]->network->Server[java] > > Or as follows in some cases (e.g. calling remote .NET job from local .NET > node): > >>> Client[platform->*jni->java*]->network->Server[java->*jni->platform*] > > Possible thin client will work as follows (ODBC driver already works this > way): > >>> Client[platform]->*network*->EdgeNode[java]->network->Server[java] > > Note additional network hop from client to edge node because thin client > lacks routing logic. > > > On Fri, Jun 30, 2017 at 12:38 AM, Aleksandr Sokolovskii > > wrote: > > > Dear Vladimir, > > > > Thanks for your answer. > > > > I’m a little bit confused. > > > > You write: > > We have a plan to implement platform-independent thin client protocol > which > > could be re-used from many languages. But you should understand, that in > > most cases it will be *slower* than current implementation, because Java > > core contains essential logic for efficient request routing. > > > > Current C++ realization in Ignite now: > > Client[cpp]->network->Server[cpp->java] > > > > If you recommend use “Java core contains essential logic for efficient > > request routing” why you don’t implement the following for C++?: > > Client[cpp->java]->network->Server[java] > > > > > > Why platform-independent protocol: > > Client[cpp->]->network->Server[java] > > > > Will be slow than current?: > > Client[cpp]->network->Server[cpp->java] > > > > Thanks, > > Aleksandr > > > > From: Vladimir Ozerov > > Sent: 30 июня 2017 г. 0:25 > > To: dev@ignite.apache.org > > Cc: Roman Shtykh > > Subject: Re: golang client for Ignite > > > > Aleksandr, > > > > Ignite is distributed system. Typical call to get/put/sql is of micro- > and > > millisecond magnitude. Java->CPP call takes several dozen nanoseconds. > > CPP->Java (so-called "reverse JNI") takes several hundred nanoseconds. > > Typical marshalling overhead is of nano- and micro-second scale as well. > > That is, total overhead of platform->Java->platform interaction is > several > > orders of magnitude *smaller* than any call to remote node. So if Golang > > users are afraid of JNI, than any distributed system would scare them to > > death. > > > > We have a plan to implement platform-independent thin client protocol > which > > could be re-used from many languages. But you should understand, that in > > most cases it will be *slower* than current implementation, because Java > > core contains essential logic for efficient request routing. Native thin > > client could be faster only if you port many and many thousands of > relevant > > non-trivial Java code to native platform, which can be estimated not to > > man-months, but to man-years to complete. > > > > Having said that, I would recommend you to look closer to current > JNI-based > > implementation :-) > > > > > > On Thu, Jun 29, 2017 at 9:11 AM, Aleksandr Sokolovskii < > amso...@gmail.com> > > wrote: > > > > > Guys, > > > > > > Thanks for your answers. > > > > > > So If users want to use Ignite DataGrid from “unsupported” language > they > > > have two choice for now: > > > > > > 1) REST API > > > It supports: cache management, put-get, sql exec queries, fetch > > > It does not support: sql transactions > > > REST API is easy to use. > > > REST API has a lot of overhead (HTTP, json marshalling/unmarshalling,.. > > > you know) > > > > > > 2) develop pure SQL driver for Ignite ODBC endpoint > > > It supports: sql exec queries, fetch > > > It does not support: cache management, put-get, sql transactions > > > Roman is right – using cgo is not very good idea. > > > Honestly It’s not trivial
Re: golang client for Ignite
Hi Aleksandr, Regarding transactions: now implementation of transactions assumes that transaction always belongs to some Ignite node, and it seems it is not simple task to support transactions for 'thin' clients. Thanks On Fri, Jun 30, 2017 at 9:55 AM, Aleksandr Sokolovskii wrote: > Dear Vladimir, > > Thanks for your answer. > > Now I understand the difference between ‘client’ and ‘thin client’. > > Just to be clear I develop ‘thin client’ for DataGrid now: > https://github.com/amsokol/go-ignite-client > and all my questions are in context of ‘thin client’. > > So the questions are: > 1) Does EdgeNode provide binary endpoint that supports cache management, > put-get, sql exec queries, fetch, transactions and I can use it by ‘thin’ > golang client? > 2) If the answer to the question (1) is ‘NO’ – is it possible to add > transactions to ODBC endpoint? > 3) If the answer to the question (2) is ‘NO’ – is it possible to add > transactions to REST API? > > Thanks, > Aleksandr > > From: Vladimir Ozerov > Sent: 30 июня 2017 г. 8:18 > To: dev@ignite.apache.org > Cc: Roman Shtykh > Subject: Re: golang client for Ignite > > Aleksandr, > > Currently Ignite.NET and Ignite C++ work as follows: > >>> Client[platform->*jni->java*]->network->Server[java] > > Or as follows in some cases (e.g. calling remote .NET job from local .NET > node): > >>> Client[platform->*jni->java*]->network->Server[java->*jni->platform*] > > Possible thin client will work as follows (ODBC driver already works this > way): > >>> Client[platform]->*network*->EdgeNode[java]->network->Server[java] > > Note additional network hop from client to edge node because thin client > lacks routing logic. > > > On Fri, Jun 30, 2017 at 12:38 AM, Aleksandr Sokolovskii > > wrote: > > > Dear Vladimir, > > > > Thanks for your answer. > > > > I’m a little bit confused. > > > > You write: > > We have a plan to implement platform-independent thin client protocol > which > > could be re-used from many languages. But you should understand, that in > > most cases it will be *slower* than current implementation, because Java > > core contains essential logic for efficient request routing. > > > > Current C++ realization in Ignite now: > > Client[cpp]->network->Server[cpp->java] > > > > If you recommend use “Java core contains essential logic for efficient > > request routing” why you don’t implement the following for C++?: > > Client[cpp->java]->network->Server[java] > > > > > > Why platform-independent protocol: > > Client[cpp->]->network->Server[java] > > > > Will be slow than current?: > > Client[cpp]->network->Server[cpp->java] > > > > Thanks, > > Aleksandr > > > > From: Vladimir Ozerov > > Sent: 30 июня 2017 г. 0:25 > > To: dev@ignite.apache.org > > Cc: Roman Shtykh > > Subject: Re: golang client for Ignite > > > > Aleksandr, > > > > Ignite is distributed system. Typical call to get/put/sql is of micro- > and > > millisecond magnitude. Java->CPP call takes several dozen nanoseconds. > > CPP->Java (so-called "reverse JNI") takes several hundred nanoseconds. > > Typical marshalling overhead is of nano- and micro-second scale as well. > > That is, total overhead of platform->Java->platform interaction is > several > > orders of magnitude *smaller* than any call to remote node. So if Golang > > users are afraid of JNI, than any distributed system would scare them to > > death. > > > > We have a plan to implement platform-independent thin client protocol > which > > could be re-used from many languages. But you should understand, that in > > most cases it will be *slower* than current implementation, because Java > > core contains essential logic for efficient request routing. Native thin > > client could be faster only if you port many and many thousands of > relevant > > non-trivial Java code to native platform, which can be estimated not to > > man-months, but to man-years to complete. > > > > Having said that, I would recommend you to look closer to current > JNI-based > > implementation :-) > > > > > > On Thu, Jun 29, 2017 at 9:11 AM, Aleksandr Sokolovskii < > amso...@gmail.com> > > wrote: > > > > > Guys, > > > > > > Thanks for your answers. > > > > > > So If users want to use Ignite DataGrid from “unsupported” language > they > > > have two choice for now: > > > > > > 1) REST API > > > It supports: cache management, put-get, sql exec queries, fetch > > > It does not support: sql transactions > > > REST API is easy to use. > > > REST API has a lot of overhead (HTTP, json marshalling/unmarshalling,.. > > > you know) > > > > > > 2) develop pure SQL driver for Ignite ODBC endpoint > > > It supports: sql exec queries, fetch > > > It does not support: cache management, put-get, sql transactions > > > Roman is right – using cgo is not very good idea. > > > Honestly It’s not trivial task to develop pure SQL driver for Ignite > ODBC > > > endpoint. > > > I spent some hours to remember how STL serializes std::string to > > > unmarshall it in golang. ))) > > > My last C+
Re: Distributed scheduling
Dmitriy, >> Can you provide a simple example of API calls that will make this possible? API could be like this: 1) via scheduler: Ignite ignite = Ignition.start(); ignite.scheduler().schedulel(job, "0 0 * * *"); // This will execute job every day at 00:00 2) via compute Ignite ignite = Ignition.start(); ignite.compute().schedulel(task, "0 0 * * *"); // This will execute compute task every day at 00:00 Make sense? On Fri, Jun 30, 2017 at 12:56 PM, Dmitriy Setrakyan wrote: > I am still not clear how it can be used or useful. Can you provide a simple > example of API calls that will make this possible? > > On Thu, Jun 29, 2017 at 7:57 PM, Alexey Kuznetsov > wrote: > > > Hi, > > > > >> Alexey, why do you think it will be useful? > > > > I need to execute some tasks periodically on cluster. I think it is a > > common task. > > I could aggregate data once a day, I could generate reports and so on... > > > > Nodes can fail, cluster could be restarted. And with new persistence > > feature distributed scheduling > > that survives cluster restart could be implemented. > > > > >>A similar topic was raised and discussed some time ago: > > >>http://apache-ignite-developers.2346864.n4.nabble. > > com/Tasks-Scheduling-and-Chaining-td14293.html > > > > I read that topic it is a bit different from my point of view. > > I'm talking only about periodical or one-time planned jobs on cluster > that > > will be executed with some guaranties. > > > > But we also can take into account that use-case. > > > > > > On Fri, Jun 30, 2017 at 5:53 AM, Dmitriy Setrakyan < > dsetrak...@apache.org> > > wrote: > > > > > Alexey, why do you think it will be useful? > > > > > > On Thu, Jun 29, 2017 at 12:22 PM, Alexey Kuznetsov < > > akuznet...@apache.org> > > > wrote: > > > > > > > Hi, All! > > > > > > > > I would like to start discussion about distributed scheduling. > > > > > > > > So, Ignite already has a module "ignite-schedule" that provide API > for > > > > LOCAL scheduling on node. > > > > And if node failed - schedule will be lost. > > > > > > > > So, it will be very useful feature to have distributed scheduling. > > > > > > > > Lets discuss how it could be implemented. > > > > > > > > I see two options: > > > > 1) Extend "ignite-schedule" module to have API for distributed > > > > scheduling. > > > > 2) Extend compute API with methods that will allow scheduling of > > tasks > > > on > > > > cluster. > > > > 3) Implement both of 1) and 2) ? > > > > > > > > Any ideas and thought are welcomed! > > > > > > > > -- > > > > Alexey Kuznetsov > > > > > > > > > > > > > > > -- > > Alexey Kuznetsov > > > -- Alexey Kuznetsov
[GitHub] ignite pull request #1930: logging of memory configuration improvements on n...
Github user asfgit closed the pull request at: https://github.com/apache/ignite/pull/1930 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
Request for contributor permission
Dear Ignite team, I would like to start contributing to your project starting with ticket https://issues.apache.org/jira/browse/IGNITE-5592 . Could you grant me permissions in Jira to assign this ticket to me? My login is ameterko. Thanks in advance.
