[GitHub] [ignite] nizhikov edited a comment on issue #6907: IGNITE-11723: IgniteSpark integration should support skipStore option for internal dataStreamer (IgniteRdd and Ignite DataFrame)
nizhikov edited a comment on issue #6907: IGNITE-11723: IgniteSpark integration should support skipStore option for internal dataStreamer (IgniteRdd and Ignite DataFrame) URL: https://github.com/apache/ignite/pull/6907#issuecomment-535187356 @zaleslaw Overall looks good. 1. Seems your IDE code format differ from Ignite one. There are some nits in the patch. Please, revert unnecessary changes. 2. Can we add the test for a new property? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [ignite] nizhikov commented on issue #6907: IGNITE-11723: IgniteSpark integration should support skipStore option for internal dataStreamer (IgniteRdd and Ignite DataFrame)
nizhikov commented on issue #6907: IGNITE-11723: IgniteSpark integration should support skipStore option for internal dataStreamer (IgniteRdd and Ignite DataFrame) URL: https://github.com/apache/ignite/pull/6907#issuecomment-535187356 @zaleslaw Overall looks good. 1. Seems your IDE code format changes differ from Ignite one. There is some nits in patch. Please, revert unnecessary changes. 2. Can we add the test for a new property? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [ignite] nizhikov commented on a change in pull request #6907: IGNITE-11723: IgniteSpark integration should support skipStore option for internal dataStreamer (IgniteRdd and Ignite DataFrame)
nizhikov commented on a change in pull request #6907: IGNITE-11723: IgniteSpark integration should support skipStore option for internal dataStreamer (IgniteRdd and Ignite DataFrame) URL: https://github.com/apache/ignite/pull/6907#discussion_r328311330 ## File path: modules/spark/src/main/scala/org/apache/ignite/spark/JavaIgniteRDD.scala ## @@ -82,20 +82,21 @@ class JavaIgniteRDD[K, V](override val rdd: IgniteRDD[K, V]) def saveValues[T](jrdd: JavaRDD[T], f: (T, IgniteContext) ⇒ V) = rdd.saveValues(JavaRDD.toRDD(jrdd), f) -def savePairs(jrdd: JavaPairRDD[K, V], overwrite: Boolean) = { +def savePairs(jrdd: JavaPairRDD[K, V], overwrite: Boolean, skipStore: Boolean) = { val rrdd: RDD[(K, V)] = JavaPairRDD.toRDD(jrdd) -rdd.savePairs(rrdd, overwrite) +rdd.savePairs(rrdd, overwrite, skipStore) } -def savePairs(jrdd: JavaPairRDD[K, V]) : Unit = savePairs(jrdd, overwrite = false) +def savePairs(jrdd: JavaPairRDD[K, V]): Unit = savePairs(jrdd, overwrite = false, skipStore = false) -def savePairs[T](jrdd: JavaRDD[T], f: (T, IgniteContext) ⇒ (K, V), overwrite: Boolean = false) = { -rdd.savePairs(JavaRDD.toRDD(jrdd), f, overwrite) +def savePairs[T](jrdd: JavaRDD[T], f: (T, IgniteContext) ⇒ (K, V), overwrite: Boolean = false, Review comment: Redundant NL This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [ignite] nizhikov commented on a change in pull request #6907: IGNITE-11723: IgniteSpark integration should support skipStore option for internal dataStreamer (IgniteRdd and Ignite DataFrame)
nizhikov commented on a change in pull request #6907: IGNITE-11723: IgniteSpark integration should support skipStore option for internal dataStreamer (IgniteRdd and Ignite DataFrame) URL: https://github.com/apache/ignite/pull/6907#discussion_r328311012 ## File path: modules/spark/src/main/scala/org/apache/ignite/spark/IgniteRDD.scala ## @@ -223,13 +222,14 @@ class IgniteRDD[K, V] ( } /** - * Saves values from the given key-value RDD into Ignite. - * - * @param rdd RDD instance to save values from. - * @param overwrite Boolean flag indicating whether the call on this method should overwrite existing - * values in Ignite cache. - */ -def savePairs(rdd: RDD[(K, V)], overwrite: Boolean = false) = { + * Saves values from the given key-value RDD into Ignite. Review comment: Why thins reformatting? Please, revert unnecessary format changes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [ignite] nizhikov commented on a change in pull request #6907: IGNITE-11723: IgniteSpark integration should support skipStore option for internal dataStreamer (IgniteRdd and Ignite DataFrame)
nizhikov commented on a change in pull request #6907: IGNITE-11723: IgniteSpark integration should support skipStore option for internal dataStreamer (IgniteRdd and Ignite DataFrame) URL: https://github.com/apache/ignite/pull/6907#discussion_r328311065 ## File path: modules/spark/src/main/scala/org/apache/ignite/spark/IgniteRDD.scala ## @@ -254,12 +255,13 @@ class IgniteRDD[K, V] ( /** * Saves values from the given RDD into Ignite. * - * @param rdd RDD instance to save values from. - * @param f Transformation function. - * @param overwrite Boolean flag indicating whether the call on this method should overwrite existing - * values in Ignite cache. + * @param rddRDD instance to save values from. Review comment: Why thins reformatting? Please, revert unnecessary format changes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [ignite] nizhikov commented on a change in pull request #6907: IGNITE-11723: IgniteSpark integration should support skipStore option for internal dataStreamer (IgniteRdd and Ignite DataFrame)
nizhikov commented on a change in pull request #6907: IGNITE-11723: IgniteSpark integration should support skipStore option for internal dataStreamer (IgniteRdd and Ignite DataFrame) URL: https://github.com/apache/ignite/pull/6907#discussion_r328310665 ## File path: modules/spark/src/main/scala/org/apache/ignite/spark/IgniteRDD.scala ## @@ -223,13 +222,14 @@ class IgniteRDD[K, V] ( } /** - * Saves values from the given key-value RDD into Ignite. - * - * @param rdd RDD instance to save values from. - * @param overwrite Boolean flag indicating whether the call on this method should overwrite existing - * values in Ignite cache. - */ -def savePairs(rdd: RDD[(K, V)], overwrite: Boolean = false) = { + * Saves values from the given key-value RDD into Ignite. Review comment: Seems you added space for existing comment. Please, remove it. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [ignite] nizhikov commented on a change in pull request #6907: IGNITE-11723: IgniteSpark integration should support skipStore option for internal dataStreamer (IgniteRdd and Ignite DataFrame)
nizhikov commented on a change in pull request #6907: IGNITE-11723: IgniteSpark integration should support skipStore option for internal dataStreamer (IgniteRdd and Ignite DataFrame) URL: https://github.com/apache/ignite/pull/6907#discussion_r328310665 ## File path: modules/spark/src/main/scala/org/apache/ignite/spark/IgniteRDD.scala ## @@ -223,13 +222,14 @@ class IgniteRDD[K, V] ( } /** - * Saves values from the given key-value RDD into Ignite. - * - * @param rdd RDD instance to save values from. - * @param overwrite Boolean flag indicating whether the call on this method should overwrite existing - * values in Ignite cache. - */ -def savePairs(rdd: RDD[(K, V)], overwrite: Boolean = false) = { + * Saves values from the given key-value RDD into Ignite. Review comment: Seems you added space for existing comment. Please, remove it. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [ignite] ptupitsyn opened a new pull request #6914: IGNITE-12234 .NET: Fix flaky tests
ptupitsyn opened a new pull request #6914: IGNITE-12234 .NET: Fix flaky tests URL: https://github.com/apache/ignite/pull/6914 * PersistenceTest.TestBaselineTopology * CancellationTest.TestTask (fix NPE when cancelling tasks) * CacheTest.TestAsyncCompletionOrder * DataStorageMetricsTest.TestDataStorageMetrics * MessagingTest.TestMessageDataTypes * ClientConnectionTest.TestClientDisposeWhileOperationsAreInProgress This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [ignite] isuntsov-gridgain opened a new pull request #6913: Added benchmarks IgniteSqlInsertIndexed* and IgniteSqlUpdateFiltered benchmarks
isuntsov-gridgain opened a new pull request #6913: Added benchmarks IgniteSqlInsertIndexed* and IgniteSqlUpdateFiltered benchmarks URL: https://github.com/apache/ignite/pull/6913 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [ignite] ptupitsyn opened a new pull request #6912: IGNITE-9638 .NET: Fix JVM thread leak - call DetachCurrentThread on CLR thread exit
ptupitsyn opened a new pull request #6912: IGNITE-9638 .NET: Fix JVM thread leak - call DetachCurrentThread on CLR thread exit URL: https://github.com/apache/ignite/pull/6912 * Use thread local storage destructor callback as a way to know when thread is about to exit * Tested on Windows (.NET Classic, .NET Core), Linux (.NET Core), macOS (.NET Core) * Additionally: do not call AttachCurrentThread in callbacks - we already have JniEnv pointer, use it This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [ignite] ptupitsyn closed pull request #6883: IGNITE-9638 .NET: Fix JVM thread leak - call DetachCurrentThread on CLR thread exit
ptupitsyn closed pull request #6883: IGNITE-9638 .NET: Fix JVM thread leak - call DetachCurrentThread on CLR thread exit URL: https://github.com/apache/ignite/pull/6883 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [ignite] SomeFire opened a new pull request #6911: IGNITE-12232 NPE while node join processing
SomeFire opened a new pull request #6911: IGNITE-12232 NPE while node join processing URL: https://github.com/apache/ignite/pull/6911 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [ignite] SomeFire opened a new pull request #6910: IGNITE-11094 fix test dependency
SomeFire opened a new pull request #6910: IGNITE-11094 fix test dependency URL: https://github.com/apache/ignite/pull/6910 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [ignite] zaleslaw opened a new pull request #6909: IGNITE-12054: Upgrade Spark module to 2.4
zaleslaw opened a new pull request #6909: IGNITE-12054: Upgrade Spark module to 2.4 URL: https://github.com/apache/ignite/pull/6909 It's just an initial draft without new module creation This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [ignite] agura opened a new pull request #6908: IGNITE-12231 Fixed logging RollbackRecords to WAL
agura opened a new pull request #6908: IGNITE-12231 Fixed logging RollbackRecords to WAL URL: https://github.com/apache/ignite/pull/6908 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [ignite] zaleslaw commented on issue #6907: IGNITE-11723: IgniteSpark integration should support skipStore option for internal dataStreamer (IgniteRdd and Ignite DataFrame)
zaleslaw commented on issue #6907: IGNITE-11723: IgniteSpark integration should support skipStore option for internal dataStreamer (IgniteRdd and Ignite DataFrame) URL: https://github.com/apache/ignite/pull/6907#issuecomment-535004271 @nizhikov Also, I've run on TC next build: RunALL, does it includes Spark component and exammples or I should run something else? Please, have a look during the review https://ci.ignite.apache.org/viewLog.html?buildId=4633586&buildTypeId=IgniteTests24Java8_RunAll This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [ignite] nizhikov commented on issue #6907: IGNITE-11723: IgniteSpark integration should support skipStore option for internal dataStreamer (IgniteRdd and Ignite DataFrame)
nizhikov commented on issue #6907: IGNITE-11723: IgniteSpark integration should support skipStore option for internal dataStreamer (IgniteRdd and Ignite DataFrame) URL: https://github.com/apache/ignite/pull/6907#issuecomment-535002460 @zaleslaw Thanks for the PR. I will do review, shortly. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [ignite] zaleslaw commented on issue #6907: IGNITE-11723: IgniteSpark integration should support skipStore option for internal dataStreamer (IgniteRdd and Ignite DataFrame)
zaleslaw commented on issue #6907: IGNITE-11723: IgniteSpark integration should support skipStore option for internal dataStreamer (IgniteRdd and Ignite DataFrame) URL: https://github.com/apache/ignite/pull/6907#issuecomment-534997955 Dear @nizhikov this is my first commit to the Spark component, please have a look and approve this change This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [ignite] zaleslaw opened a new pull request #6907: IGNITE-11723: IgniteSpark integration should support skipStore option for internal dataStreamer (IgniteRdd and Ignite DataFrame)
zaleslaw opened a new pull request #6907: IGNITE-11723: IgniteSpark integration should support skipStore option for internal dataStreamer (IgniteRdd and Ignite DataFrame) URL: https://github.com/apache/ignite/pull/6907 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [ignite] ingvard opened a new pull request #6906: IGNITE-12230 Partition eviction during cache stop / deactivation may …
ingvard opened a new pull request #6906: IGNITE-12230 Partition eviction during cache stop / deactivation may … URL: https://github.com/apache/ignite/pull/6906 …cause errors leading to node failure and storage corruption This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [ignite] aealeksandrov opened a new pull request #6905: Ignite 2.7.2 p9
aealeksandrov opened a new pull request #6905: Ignite 2.7.2 p9 URL: https://github.com/apache/ignite/pull/6905 For TC run only. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [ignite] nizhikov merged pull request #6898: IGNITE-12211: Client connections system view
nizhikov merged pull request #6898: IGNITE-12211: Client connections system view URL: https://github.com/apache/ignite/pull/6898 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [ignite] alex-plekhanov commented on a change in pull request #6898: IGNITE-12211: Client connections system view
alex-plekhanov commented on a change in pull request #6898: IGNITE-12211: Client connections system view URL: https://github.com/apache/ignite/pull/6898#discussion_r328026859 ## File path: modules/core/src/test/java/org/apache/ignite/internal/metric/JmxExporterSpiTest.java ## @@ -62,6 +69,9 @@ import static org.apache.ignite.internal.processors.metric.GridMetricManager.GC_CPU_LOAD_DESCRIPTION; import static org.apache.ignite.internal.processors.metric.GridMetricManager.SYS_METRICS; import static org.apache.ignite.internal.processors.metric.impl.MetricUtils.metricName; +import static org.apache.ignite.internal.processors.odbc.ClientListenerProcessor.CLI_CONN_SYS_VIEW; +import static org.apache.ignite.internal.processors.odbc.jdbc.JdbcConnectionContext.CURRENT_VER; +import static org.apache.ignite.internal.processors.platform.client.ClientConnectionContext.DEFAULT_VER; Review comment: No, the client from the same source can have another version This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [ignite] alex-plekhanov commented on a change in pull request #6898: IGNITE-12211: Client connections system view
alex-plekhanov commented on a change in pull request #6898: IGNITE-12211: Client connections system view URL: https://github.com/apache/ignite/pull/6898#discussion_r328026298 ## File path: modules/core/src/test/java/org/apache/ignite/internal/metric/JmxExporterSpiTest.java ## @@ -62,6 +70,9 @@ import static org.apache.ignite.internal.processors.metric.GridMetricManager.GC_CPU_LOAD_DESCRIPTION; import static org.apache.ignite.internal.processors.metric.GridMetricManager.SYS_METRICS; import static org.apache.ignite.internal.processors.metric.impl.MetricUtils.metricName; +import static org.apache.ignite.internal.processors.odbc.ClientListenerProcessor.CLI_CONN_SYS_VIEW; +import static org.apache.ignite.internal.processors.odbc.jdbc.JdbcConnectionContext.CURRENT_VER; +import static org.apache.ignite.internal.processors.platform.client.ClientConnectionContext.DEFAULT_VER; Review comment: Version still hasn't been fixed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [ignite] alex-plekhanov commented on a change in pull request #6898: IGNITE-12211: Client connections system view
alex-plekhanov commented on a change in pull request #6898: IGNITE-12211: Client connections system view URL: https://github.com/apache/ignite/pull/6898#discussion_r328026051 ## File path: modules/indexing/src/test/java/org/apache/ignite/internal/processors/cache/metric/SqlViewExporterSpiTest.java ## @@ -243,6 +249,29 @@ public void testServices() throws Exception { assertEquals(srvcCfg.getMaxPerNodeCount(), sysView.get(4)); } +/** */ +@Test +public void testClientsConnections() throws Exception { +String host = ignite.configuration().getClientConnectorConfiguration().getHost(); + +if (host == null) +host = ignite.configuration().getLocalHost(); + +int port = ignite.configuration().getClientConnectorConfiguration().getPort(); + +try (IgniteClient client = + Ignition.startClient(new ClientConfiguration().setAddresses(host + ":" + port))) { + Review comment: Redundant NL This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [ignite] alex-plekhanov commented on a change in pull request #6898: IGNITE-12211: Client connections system view
alex-plekhanov commented on a change in pull request #6898: IGNITE-12211: Client connections system view URL: https://github.com/apache/ignite/pull/6898#discussion_r328025941 ## File path: modules/indexing/src/test/java/org/apache/ignite/internal/processors/cache/metric/SqlViewExporterSpiTest.java ## @@ -243,6 +249,29 @@ public void testServices() throws Exception { assertEquals(srvcCfg.getMaxPerNodeCount(), sysView.get(4)); } +/** */ +@Test +public void testClientsConnections() throws Exception { +String host = ignite.configuration().getClientConnectorConfiguration().getHost(); + +if (host == null) +host = ignite.configuration().getLocalHost(); + +int port = ignite.configuration().getClientConnectorConfiguration().getPort(); + +try (IgniteClient client = + Ignition.startClient(new ClientConfiguration().setAddresses(host + ":" + port))) { + +try (Connection conn = + new IgniteJdbcThinDriver().connect("jdbc:ignite:thin://" + host, new Properties())) { + Review comment: Redundant NL This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services