答复: 答复: 答复: 答复: ignite Connection refused exception
Hi Val, After setting join timeout to 30s and socket timeout to 30s too. I can connect to the server. If each connection takes so long time, it may cause performance issue. Do you have any suggestions on client to server connect? Best regards, Kevin -邮件原件- 发件人: vkulichenko [mailto:valentin.kuliche...@gmail.com] 发送时间: 2016年7月26日 6:45 收件人: user@ignite.apache.org 主题: Re: 答复: 答复: 答复: ignite Connection refused exception Hi Kevin, The exception itself is actually printed out only in DEBUG mode. It will always happen if you provide a range of addresses, but only some of them are available. So this is not an issue. I noticed that it takes around a second to get 'Connection refused' from each address, so going through all of them can take some time. Can you try increasing the joinTimeout value to, for example, 30 seconds? -Val -- View this message in context: http://apache-ignite-users.70518.x6.nabble.com/ignite-Connection-refused-exception-tp6412p6523.html Sent from the Apache Ignite Users mailing list archive at Nabble.com.
Re: How Asynchronous support works underneath? 60 * N put calls every second
Hi, Val, but if it is not blocking, how it could be done without using an additional thread? If you return the Future back, the operation must be happening in the backround (i.e. a new thread), is not it? Thank you. -- View this message in context: http://apache-ignite-users.70518.x6.nabble.com/How-Asynchronous-support-works-underneath-60-N-put-calls-every-second-tp6482p6539.html Sent from the Apache Ignite Users mailing list archive at Nabble.com.
Re: "ArrayIndexOutOfBoundsException" happened when doing qurey in version 1.6.0
Looks like the same issue as reported by Agneeswaran: http://apache-ignite-users.70518.x6.nabble.com/Issue-with-concurrent-users-on-Ignite-1-6-0-ODBC-td6217.html#a6385 Best Regards, Igor On Tue, Jul 26, 2016 at 8:43 AM, Semyon Boikov wrote: > We already have similar issue reproduced in our benchmarks - > https://issues.apache.org/jira/browse/IGNITE-3300. I think this is > related to optimziation done in 1.6 to store key partition in the cache > key. I believe fix will be included in 1.7 release. > > On Tue, Jul 26, 2016 at 2:13 AM, Valentin Kulichenko < > valentin.kuliche...@gmail.com> wrote: > >> Crossposting to dev@ >> >> Folks, >> >> I noticed the exception below in couple of user threads already. This >> actually means that partition number for the key is -1, which doesn't make >> much sense to me. Does anyone has any ideas? >> >> Caused by: java.lang.ArrayIndexOutOfBoundsException: -1 >> at java.util.ArrayList.elementData(ArrayList.java:400) >> at java.util.ArrayList.get(ArrayList.java:413) >> at >> org.apache.ignite.internal.processors.affinity.GridAffinityAssignment.get(GridAffinityAssignment.java:152) >> at >> org.apache.ignite.internal.processors.affinity.GridAffinityAssignmentCache.nodes(GridAffinityAssignmentCache.java:387) >> at >> org.apache.ignite.internal.processors.cache.GridCacheAffinityManager.nodes(GridCacheAffinityManager.java:251) >> at >> org.apache.ignite.internal.processors.cache.GridCacheAffinityManager.primary(GridCacheAffinityManager.java:287) >> at >> org.apache.ignite.internal.processors.cache.GridCacheAffinityManager.primary(GridCacheAffinityManager.java:278) >> at >> org.apache.ignite.internal.processors.cache.GridCacheAffinityManager.primary(GridCacheAffinityManager.java:302) >> at >> org.apache.ignite.internal.processors.query.h2.IgniteH2Indexing$7$3.apply(IgniteH2Indexing.java:1730) >> at >> org.apache.ignite.internal.processors.query.h2.opt.GridH2IndexBase$FilteringIterator.accept(GridH2IndexBase.java:251) >> at >> org.apache.ignite.internal.processors.query.h2.opt.GridH2IndexBase$FilteringIterator.accept(GridH2IndexBase.java:200) >> at >> org.apache.ignite.internal.util.lang.GridFilteredIterator.hasNext(GridFilteredIterator.java:59) >> at >> org.apache.ignite.internal.processors.query.h2.opt.GridH2Cursor.next(GridH2Cursor.java:59) >> at org.h2.index.IndexCursor.next(IndexCursor.java:274) >> at org.h2.table.TableFilter.next(TableFilter.java:359) >> at org.h2.command.dml.Select.queryFlat(Select.java:527) >> at org.h2.command.dml.Select.queryWithoutCache(Select.java:632) >> at org.h2.command.dml.Query.query(Query.java:297) >> at org.h2.command.dml.Query.query(Query.java:284) >> at org.h2.command.dml.Query.query(Query.java:36) >> at org.h2.command.CommandContainer.query(CommandContainer.java:91) >> at org.h2.command.Command.executeQuery(Command.java:196) >> ... 26 more >> >> >> -Val >> >> >> On Mon, Jul 25, 2016 at 5:07 AM, ght230 wrote: >> >>> ignite-related.xml >>> < >>> http://apache-ignite-users.70518.x6.nabble.com/file/n6516/ignite-related.xml >>> > >>> Please refer to the attachment. >>> >>> >>> >>> >>> -- >>> View this message in context: >>> http://apache-ignite-users.70518.x6.nabble.com/ArrayIndexOutOfBoundsException-happened-when-doing-qurey-in-version-1-6-0-tp6245p6516.html >>> Sent from the Apache Ignite Users mailing list archive at Nabble.com. >>> >> >> >
Is there any such thing as SQlBuilder in apache Ignite for SQLQuery
I am using sql queries to implement pagination in ignite cache. Below is my
code to build sql to be passed to SQLQuery. It looks very messy as there are
lot of checks and if else. Is there something in ignite like builder which
will do the task?
String sql = " select * from Employee"
private String buildSql(String sql, String sortByField, String sortOrder,
Integer startIndex, Integer endIndex) {
boolean isOrdered = false;
if (DataConstants.START_DATE.equals(sortByField)) {
sql = sql + "order by " + DataConstants.START_DATE;
isOrdered = true;
} else if (DataConstants.ISACTIVE.equals(sortByField)) {
sql=sql +"order by "+DataConstants.ISACTIVE;
isOrdered = true;
}...
.
if (isOrdered &&
DataConstants.DESCENDING_ORDER.equals(sortOrder)) {
sql = sql + " " + DataConstants.DESCENDING_ORDER;
}
if (startIndex != null) {
sql = sql + " limit " + startIndex;
if (endIndex != null) {
sql = sql + " , " + endIndex;
}
}
return sql;
}
--
View this message in context:
http://apache-ignite-users.70518.x6.nabble.com/Is-there-any-such-thing-as-SQlBuilder-in-apache-Ignite-for-SQLQuery-tp6541.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.
exception when calling ignite.getOrCreateCache(CacheConfiguration cfg)
Hi there, When using ignite.getOrCreateCache(CacheConfiguration cfg) method to create a cache from client side using java, I got the following error message: [20:47:11,991][ERROR][exchange-worker-#41%127.0.0.1%][GridDhtPartitionsExchangeFuture] Failed to reinitialize local partitions (preloading will be stopped): GridDhtPartitionExchangeId [topVer=AffinityTopologyVersion [topVer=9, minorTopVer=11], nodeId=71f82b28, evt=DISCOVERY_CUSTOM_EVT] java.lang.AssertionError at org.apache.ignite.internal.processors.query.GridQueryProcessor.processIndexes(GridQueryProcessor.java:1565) at org.apache.ignite.internal.processors.query.GridQueryProcessor.processBinaryMeta(GridQueryProcessor.java:1491) at org.apache.ignite.internal.processors.query.GridQueryProcessor.initializeCache(GridQueryProcessor.java:271) at org.apache.ignite.internal.processors.query.GridQueryProcessor.onCacheStart(GridQueryProcessor.java:457) at org.apache.ignite.internal.processors.cache.GridCacheProcessor.startCache(GridCacheProcessor.java:1038) at org.apache.ignite.internal.processors.cache.GridCacheProcessor.prepareCacheStart(GridCacheProcessor.java:1648) at org.apache.ignite.internal.processors.cache.GridCacheProcessor.prepareCachesStart(GridCacheProcessor.java:1563) at org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.startCaches(GridDhtPartitionsExchangeFuture.java:956) at org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.init(GridDhtPartitionsExchangeFuture.java:523) at org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager$ExchangeWorker.body(GridCachePartitionExchangeManager.java:1297) at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:110) at java.lang.Thread.run(Thread.java:745) [20:47:11,994][ERROR][exchange-worker-#41%127.0.0.1%][GridCachePartitionExchangeManager] Runtime error caught during grid runnable execution: GridWorker [name=partition-exchanger, gridName=127.0.0.1, finished=false, isCancelled=false, hashCode=1603452138, interrupted=false, runner=exchange-worker-#41%127.0.0.1%] java.lang.NullPointerException at org.apache.ignite.internal.processors.cache.GridCacheProcessor.onExchangeDone(GridCacheProcessor.java:1723) at org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.onDone(GridDhtPartitionsExchangeFuture.java:1109) at org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.onDone(GridDhtPartitionsExchangeFuture.java:87) at org.apache.ignite.internal.util.future.GridFutureAdapter.onDone(GridFutureAdapter.java:334) at org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.init(GridDhtPartitionsExchangeFuture.java:873) at org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager$ExchangeWorker.body(GridCachePartitionExchangeManager.java:1297) at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:110) at java.lang.Thread.run(Thread.java:745) BTW, the cache configuration was generated from schema-import utility from ignite. any idea what was wrong? Best regards, Kevin
Re: QuerySqlFunction methods with variable arguments do not seem to work
Thanks Val, - yes, this class is listed in sql-function-classes. Equivalent method with fixed argument list works as expected. I'm configuring these UDFs to support interactive SQL queries from jdbc tools. Seems like Ignite fails to find the variable argument method to invoke ... -- View this message in context: http://apache-ignite-users.70518.x6.nabble.com/QuerySqlFunction-methods-with-variable-arguments-do-not-seem-to-work-tp6524p6543.html Sent from the Apache Ignite Users mailing list archive at Nabble.com.
Load data from Hadoop
Hello. I'm trying load data from Hadoop(20G) to Ignite. Can anybode tell how should I do this? I have 5 servers in cluster. There are 8 ignite nodes and one hadoop node on each server. Now I have one client node which take data by jdbc(using Hive) from Hadoop and put it by IgniteStream.addData() into Ignite. But it is too slow. I assume because all data go through Hadoop nameNode and then load by one client node. If I will use loadCache instead of IgniteDataStream, I apprehend that Hive database can fall because of 40 queries for 20G files. In ideal I want to get data straight from Hadoop dataNodes and process a small piece on each Ignite node. Maybe I can use IGFS integrated with Hadoop, IgniteHadoopSecondryFileSystem to achieve this result? Sincerely, Dmitry -- View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Load-data-from-Hadoop-tp6544.html Sent from the Apache Ignite Users mailing list archive at Nabble.com.
Re: How to use the logger in the BackupFilter?
Jason, As far as I understood your use case, machine-to group assignment (the digit in your example) is constant and cannot change over time. In this case user attributes should work perfectly like Val suggested. If, for some reason, this does not meet your requirements, my suggestion would be to implement an IgnitePlugin and create a component which maintains this mapping. Ignite has an internal mechanism to exchange arbitrary node information for it's components at the time of node discovery. Take a look at org.apache.ignite.internal.GridComponent#collectDiscoveryData and org.apache.ignite.internal.GridComponent#onDiscoveryDataReceived. This should be enough for you to implement virtually any strategy for affinity assignment. Let us know if you need help with plugin development. Thanks, AG 2016-07-25 6:09 GMT+03:00 Jason : > hi Val, > > Another question about point 3: > Event with user attributes, but when do the deployment of adding new > machines to the node, it will roll out one machine group by one, so this > means during the period of deployment, all the nodes cannot get the same > user attributes (machines group info) and if the Rendezvous hashing return > a > new machine list for one partition with a new machine in the top N (N is > the > backup # + 1), the backup filter in different nodes may made different > decision. > > E.g. for partition p of cache c, its node list after Rendezvous hashing is > A1, X1, B2, C3, D2, ..., where letter is machine and digit is group; and X > is a new added machine and for rolled machine, its group is 1 but for > others, it's unknown; and suppose the backup = 2. Say, we've rolled out > group 1 and are rolling out group 2 now, so in machine A, it selects (A1, > B2, C3), but in machine C, it may select (A1, X, B2), because it doesn't > know which group X belongs to now. This will cause inconsistency. > > And we must ensure no data loss in the memory only. > > BTW, do you have any successful case for this usage scenario? how do you > resolve this problem? > > Thanks, > -Jason > > > > > -- > View this message in context: > http://apache-ignite-users.70518.x6.nabble.com/How-to-use-the-logger-in-the-BackupFilter-tp6442p6499.html > Sent from the Apache Ignite Users mailing list archive at Nabble.com. >
Re: Is there any such thing as SQlBuilder in apache Ignite for SQLQuery
Hi, Unfortunately, nothing like this is available now, so you will have to build the query manually. BTW, note that such approach for pagination will not work properly if there are updates happening concurrently with the queries. I.e., if you query twice with the same offset, you can get different results in case an update happened between two executions of the query. -Val -- View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Is-there-any-such-thing-as-SQlBuilder-in-apache-Ignite-for-SQLQuery-tp6541p6546.html Sent from the Apache Ignite Users mailing list archive at Nabble.com.
Re: exception when calling ignite.getOrCreateCache(CacheConfiguration cfg)
Hi Kevin, Can you attach the generated configuration? -Val -- View this message in context: http://apache-ignite-users.70518.x6.nabble.com/exception-when-calling-ignite-getOrCreateCache-CacheConfiguration-cfg-tp6542p6547.html Sent from the Apache Ignite Users mailing list archive at Nabble.com.
Stored Procedure
Hi, I understand the stored procedures are NOT supported in apache ignite. Please let me know if I missed out something. Also, is there any SQL CLI/shell available for Apache ignite? Regards Abhishek -- View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Stored-Procedure-tp6548.html Sent from the Apache Ignite Users mailing list archive at Nabble.com.
Re: How Asynchronous support works underneath? 60 * N put calls every second
Ignite uses async IO for communication. So it just asynchronously send a request message and creates a future which will be completed when response for this request is received. So there is nothing actually happening while we're waiting for response and another thread is not needed. -Val -- View this message in context: http://apache-ignite-users.70518.x6.nabble.com/How-Asynchronous-support-works-underneath-60-N-put-calls-every-second-tp6482p6549.html Sent from the Apache Ignite Users mailing list archive at Nabble.com.
Re: 答复: 答复: 答复: 答复: ignite Connection refused exception
Kevin, Actually, connections are not established frequently. If there is a lot of communication between two nodes, they will create it once and then reuse. So I'm not sure if it will cause degradation. In any case, this sounds like a network issue. I would recommend to investigate this separately from Ignite, using the network diagnostic tools like iperf and others. -Val -- View this message in context: http://apache-ignite-users.70518.x6.nabble.com/ignite-Connection-refused-exception-tp6412p6550.html Sent from the Apache Ignite Users mailing list archive at Nabble.com.
Re: Can not start another client when doing ContinuousAsyncQuery
I can't reproduce it. How stable is this issue for you? What is ExampleNodeClient? -Val -- View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Can-not-start-another-client-when-doing-ContinuousAsyncQuery-tp6375p6551.html Sent from the Apache Ignite Users mailing list archive at Nabble.com.
Re: QuerySqlFunction methods with variable arguments do not seem to work
Hi, Sorry, I missed the main point of the question :) But it actually works for me with varargs in custom functions. Can you prepare a small example that will reproduce the issue? I will be able to take a look. -Val -- View this message in context: http://apache-ignite-users.70518.x6.nabble.com/QuerySqlFunction-methods-with-variable-arguments-do-not-seem-to-work-tp6524p6552.html Sent from the Apache Ignite Users mailing list archive at Nabble.com.
Re: Stored Procedure
Hi, Abhishek, Stored procedures are not supported, but you can use Compute Grid [1] to execute any custom code on server nodes. As for the shell, Ignite provides JDBC driver, so you can use any JDBC compliant tool to query the data. [1] https://github.com/julianhyde/sqlline [2] https://apacheignite.readme.io/docs/jdbc-driver -Val -- View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Stored-Procedure-tp6548p6553.html Sent from the Apache Ignite Users mailing list archive at Nabble.com.
Re: Stored Procedure
Thankyou Val for your very quick response. Regards Abhishek -- View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Stored-Procedure-tp6548p6554.html Sent from the Apache Ignite Users mailing list archive at Nabble.com.
Re: Re: SQL join query return different result under the same data when having different ignite instance.
Hi Val, My query is complex, I use another example to descripte my scenario. There are three objects: School, Class, Student. School: id name address 1 xxx aaa Class: id name number 1 yyy bbb Student: id name sex schoolID classID 1 zzz male 1 1 Now, the question is coming: I want to query as the following sql: select * from school, class, student where student.id = school.id and student.id = class.id; It means, the object student produced two affinity column: schoolID and classID, but as you said: "Generally that's right, you can collocate by a single field only, otherwise the data will end up on different nodes" How to do for this scenario? Bob From: vkulichenko Date: 2016-07-23 03:08 To: user@ignite.apache.org Subject: Re: Re: SQL join query return different result under the same data when having different ignite instance. Hi Bob, It's hard to tell how to collocate your data because it's not clear what this query is doing. Can you describe? Generally that's right, you can collocate by a single field only, otherwise the data will end up on different nodes. If the data in two caches can't be collocated, you can also switch one of the caches to REPLICATED mode. Replicated caches don't need collocation, but they replicate all the data to all nodes. -Val -- View this message in context: http://apache-ignite-users.70518.x6.nabble.com/SQL-join-query-return-different-result-under-the-same-data-when-having-different-ignite-instance-tp6433p6478.html Sent from the Apache Ignite Users mailing list archive at Nabble.com. --- Confidentiality Notice: The information contained in this e-mail and any accompanying attachment(s) is intended only for the use of the intended recipient and may be confidential and/or privileged of Neusoft Corporation, its subsidiaries and/or its affiliates. If any reader of this communication is not the intended recipient, unauthorized use, forwarding, printing, storing, disclosure or copying is strictly prohibited, and may be unlawful.If you have received this communication in error,please immediately notify the sender by return e-mail, and delete the original message and all copies from your system. Thank you. ---
Ignite - Using - complex object with nested attributes and search
Hi:
Can we insert a complex object into ignite cache and search it ?
Eg ( psuedocode ):public class Person { int age,
String name,
List address,
Tuple2 parents,
}
If it is possible, how can we annotated them ? Also, can these nested
attributes be used in SQL joins ?
Thanks
Re: transaction not timing out
Hi, Trying to find an answer to the above. I do not understand why commit() is unable to finish. In an optimistic/serializable transaction, it should not even reach the commit phase using the 2 phase protocol. If the txn reached he commit() phase and then it was successful in acquiring all the locks it needs and it should not stall. Another related question, is it not recommended to start an ignite transaction from the client? what happens if the client crashes in the middle of the transaction? With pessimistic txn on client side, and if I kill the client in the middle of the txn, those keys that it had locks on are never released. now that same entry n cache cannot be updated by any process. how do I get out of such a situation. However, in optimistic locking this does not happen until commit, so it behaves differently. But what if the client crashes during commit phase? i also cannot use optimistic in this scenario. Thanks, Binti -- View this message in context: http://apache-ignite-users.70518.x6.nabble.com/transaction-not-timing-out-tp5540p6557.html Sent from the Apache Ignite Users mailing list archive at Nabble.com.
Re: Ignite - Using - complex object with nested attributes and search
You can do this using ScanQuery by passing predicate to it. Predicate will
have all conditions you want to check like
person.name.contains(searchparameter) For JOINS, you have to use SqlQuery
but I havent tried that.
On Wed, Jul 27, 2016 at 6:50 AM, M Singh wrote:
> Hi:
>
> Can we insert a complex object into ignite cache and search it ?
>
> Eg ( psuedocode ):
> public class Person {
> int age,
> String name,
> List address,
> Tuple2 parents,
> }
>
> If it is possible, how can we annotated them ? Also, can these nested
> attributes be used in SQL joins ?
>
> Thanks
>
>
Re: How to use the logger in the BackupFilter?
Thanks, AG. The machine to group assignment may be change in our usage scenario, but it should be very rare. And if we can control its change rate every time, even if the machine group for the running Ignite node is different with its real new group which will take effective when the Ignite restarts, its impact should be under control. BTW, Plugin with org.apache.ignite.internal.GridComponent#collectDiscoveryData and org.apache.ignite.internal.GridComponent#onDiscoveryDataReceived can let one or few nodes control the change just like a center, right? Thanks, -Jason -- View this message in context: http://apache-ignite-users.70518.x6.nabble.com/How-to-use-the-logger-in-the-BackupFilter-tp6442p6559.html Sent from the Apache Ignite Users mailing list archive at Nabble.com.
