Re: [PR] [GLUTEN-5438] feat: Dynamically sizing off-heap memory [incubator-gluten]

2024-05-19 Thread via GitHub
ulysses-you commented on code in PR #5439: URL: https://github.com/apache/incubator-gluten/pull/5439#discussion_r1606198195 ## gluten-core/src/main/scala/org/apache/gluten/GlutenPlugin.scala: ## @@ -164,20 +165,71 @@ private[gluten] class GlutenDriverPlugin extends

Re: [PR] [GLUTEN-5438] feat: Dynamically sizing off-heap memory [incubator-gluten]

2024-05-19 Thread via GitHub
Yohahaha commented on code in PR #5439: URL: https://github.com/apache/incubator-gluten/pull/5439#discussion_r1606179989 ## gluten-core/src/main/scala/org/apache/gluten/GlutenPlugin.scala: ## @@ -164,20 +165,71 @@ private[gluten] class GlutenDriverPlugin extends DriverPlugin

Re: [PR] [GLUTEN-5438] feat: Dynamically sizing off-heap memory [incubator-gluten]

2024-05-19 Thread via GitHub
ulysses-you commented on code in PR #5439: URL: https://github.com/apache/incubator-gluten/pull/5439#discussion_r1606162953 ## gluten-core/src/main/scala/org/apache/gluten/GlutenPlugin.scala: ## @@ -164,20 +165,71 @@ private[gluten] class GlutenDriverPlugin extends

Re: [PR] [GLUTEN-5438] feat: Dynamically sizing off-heap memory [incubator-gluten]

2024-05-17 Thread via GitHub
Yohahaha commented on code in PR #5439: URL: https://github.com/apache/incubator-gluten/pull/5439#discussion_r1604684880 ## gluten-core/src/main/scala/org/apache/gluten/GlutenPlugin.scala: ## @@ -164,20 +165,71 @@ private[gluten] class GlutenDriverPlugin extends DriverPlugin

Re: [PR] [GLUTEN-5438] feat: Dynamically sizing off-heap memory [incubator-gluten]

2024-05-16 Thread via GitHub
GlutenPerfBot commented on PR #5439: URL: https://github.com/apache/incubator-gluten/pull/5439#issuecomment-2116587518 = Performance report for TPCH SF2000 with Velox backend, for reference only query

Re: [PR] [GLUTEN-5438] feat: Dynamically sizing off-heap memory [incubator-gluten]

2024-05-16 Thread via GitHub
ulysses-you merged PR #5439: URL: https://github.com/apache/incubator-gluten/pull/5439 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [PR] [GLUTEN-5438] feat: Dynamically sizing off-heap memory [incubator-gluten]

2024-05-15 Thread via GitHub
ulysses-you commented on PR #5439: URL: https://github.com/apache/incubator-gluten/pull/5439#issuecomment-2113930583 I'm fine with this pr, looking forward some best practices, thank you @supermem613 @zhztheplayer -- This is an automated message from the Apache Git Service. To respond

Re: [PR] [GLUTEN-5438] feat: Dynamically sizing off-heap memory [incubator-gluten]

2024-05-15 Thread via GitHub
github-actions[bot] commented on PR #5439: URL: https://github.com/apache/incubator-gluten/pull/5439#issuecomment-2112587570 Run Gluten Clickhouse CI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] [GLUTEN-5438] feat: Dynamically sizing off-heap memory [incubator-gluten]

2024-05-15 Thread via GitHub
github-actions[bot] commented on PR #5439: URL: https://github.com/apache/incubator-gluten/pull/5439#issuecomment-2112586665 Run Gluten Clickhouse CI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] [GLUTEN-5438] feat: Dynamically sizing off-heap memory [incubator-gluten]

2024-05-15 Thread via GitHub
supermem613 commented on code in PR #5439: URL: https://github.com/apache/incubator-gluten/pull/5439#discussion_r1601668026 ## shims/common/src/main/scala/org/apache/gluten/GlutenConfig.scala: ## @@ -1821,4 +1832,42 @@ object GlutenConfig { .internal()

Re: [PR] [GLUTEN-5438] feat: Dynamically sizing off-heap memory [incubator-gluten]

2024-05-15 Thread via GitHub
supermem613 commented on code in PR #5439: URL: https://github.com/apache/incubator-gluten/pull/5439#discussion_r1601663164 ## gluten-core/src/main/java/org/apache/gluten/memory/memtarget/DynamicOffHeapSizingMemoryTarget.java: ## @@ -0,0 +1,95 @@ +/* + * Licensed to the Apache

Re: [PR] [GLUTEN-5438] feat: Dynamically sizing off-heap memory [incubator-gluten]

2024-05-15 Thread via GitHub
supermem613 commented on code in PR #5439: URL: https://github.com/apache/incubator-gluten/pull/5439#discussion_r1601661282 ## gluten-core/src/main/java/org/apache/gluten/memory/memtarget/DynamicOffHeapSizingMemoryTarget.java: ## @@ -0,0 +1,95 @@ +/* + * Licensed to the Apache

Re: [PR] [GLUTEN-5438] feat: Dynamically sizing off-heap memory [incubator-gluten]

2024-05-09 Thread via GitHub
ulysses-you commented on code in PR #5439: URL: https://github.com/apache/incubator-gluten/pull/5439#discussion_r1596207765 ## gluten-core/src/main/java/org/apache/gluten/memory/memtarget/DynamicOffHeapSizingMemoryTarget.java: ## @@ -0,0 +1,95 @@ +/* + * Licensed to the Apache

Re: [PR] [GLUTEN-5438] feat: Dynamically sizing off-heap memory [incubator-gluten]

2024-05-09 Thread via GitHub
ulysses-you commented on code in PR #5439: URL: https://github.com/apache/incubator-gluten/pull/5439#discussion_r1596205549 ## gluten-core/src/main/java/org/apache/gluten/memory/memtarget/DynamicOffHeapSizingMemoryTarget.java: ## @@ -0,0 +1,95 @@ +/* + * Licensed to the Apache

Re: [PR] [GLUTEN-5438] feat: Dynamically sizing off-heap memory [incubator-gluten]

2024-05-09 Thread via GitHub
ulysses-you commented on code in PR #5439: URL: https://github.com/apache/incubator-gluten/pull/5439#discussion_r1596202734 ## gluten-core/src/main/java/org/apache/gluten/memory/memtarget/DynamicOffHeapSizingMemoryTarget.java: ## @@ -0,0 +1,95 @@ +/* + * Licensed to the Apache

Re: [PR] [GLUTEN-5438] feat: Dynamically sizing off-heap memory [incubator-gluten]

2024-05-09 Thread via GitHub
ulysses-you commented on code in PR #5439: URL: https://github.com/apache/incubator-gluten/pull/5439#discussion_r1596197989 ## shims/common/src/main/scala/org/apache/gluten/GlutenConfig.scala: ## @@ -1821,4 +1832,42 @@ object GlutenConfig { .internal()

Re: [PR] [GLUTEN-5438] feat: Dynamically sizing off-heap memory [incubator-gluten]

2024-05-08 Thread via GitHub
github-actions[bot] commented on PR #5439: URL: https://github.com/apache/incubator-gluten/pull/5439#issuecomment-2100796228 Run Gluten Clickhouse CI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] [GLUTEN-5438] feat: Dynamically sizing off-heap memory [incubator-gluten]

2024-05-08 Thread via GitHub
zhztheplayer commented on code in PR #5439: URL: https://github.com/apache/incubator-gluten/pull/5439#discussion_r1593670874 ## gluten-core/src/main/java/org/apache/gluten/memory/memtarget/MemoryTargets.java: ## @@ -54,6 +54,13 @@ public static MemoryTarget newConsumer( }

Re: [PR] [GLUTEN-5438] feat: Dynamically sizing off-heap memory [incubator-gluten]

2024-05-06 Thread via GitHub
github-actions[bot] commented on PR #5439: URL: https://github.com/apache/incubator-gluten/pull/5439#issuecomment-2096723564 Run Gluten Clickhouse CI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] [GLUTEN-5438] feat: Dynamically sizing off-heap memory [incubator-gluten]

2024-05-06 Thread via GitHub
github-actions[bot] commented on PR #5439: URL: https://github.com/apache/incubator-gluten/pull/5439#issuecomment-2096720057 Run Gluten Clickhouse CI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] [GLUTEN-5438] feat: Dynamically sizing off-heap memory [incubator-gluten]

2024-05-06 Thread via GitHub
supermem613 commented on code in PR #5439: URL: https://github.com/apache/incubator-gluten/pull/5439#discussion_r1591437289 ## gluten-core/src/main/java/org/apache/gluten/memory/memtarget/spark/DynamicOffHeapSizingPolicyChecker.java: ## @@ -0,0 +1,71 @@ +/* + * Licensed to the

Re: [PR] [GLUTEN-5438] feat: Dynamically sizing off-heap memory [incubator-gluten]

2024-05-06 Thread via GitHub
supermem613 commented on code in PR #5439: URL: https://github.com/apache/incubator-gluten/pull/5439#discussion_r1591120073 ## gluten-core/src/main/java/org/apache/gluten/memory/memtarget/spark/DynamicOffHeapSizingPolicyChecker.java: ## @@ -0,0 +1,71 @@ +/* + * Licensed to the

Re: [PR] [GLUTEN-5438] feat: Dynamically sizing off-heap memory [incubator-gluten]

2024-05-05 Thread via GitHub
zhztheplayer commented on code in PR #5439: URL: https://github.com/apache/incubator-gluten/pull/5439#discussion_r1571716511 ## gluten-core/src/main/java/org/apache/gluten/memory/memtarget/spark/DynamicOffHeapSizingPolicyChecker.java: ## @@ -0,0 +1,71 @@ +/* + * Licensed to the

Re: [PR] [GLUTEN-5438] feat: Dynamically sizing off-heap memory [incubator-gluten]

2024-05-05 Thread via GitHub
ulysses-you commented on code in PR #5439: URL: https://github.com/apache/incubator-gluten/pull/5439#discussion_r1590476656 ## gluten-core/src/main/java/org/apache/gluten/memory/memtarget/spark/DynamicOffHeapSizingPolicyChecker.java: ## @@ -0,0 +1,71 @@ +/* + * Licensed to the

Re: [PR] [GLUTEN-5438] feat: Dynamically sizing off-heap memory [incubator-gluten]

2024-05-03 Thread via GitHub
github-actions[bot] commented on PR #5439: URL: https://github.com/apache/incubator-gluten/pull/5439#issuecomment-2093001799 Run Gluten Clickhouse CI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] [GLUTEN-5438] feat: Dynamically sizing off-heap memory [incubator-gluten]

2024-05-03 Thread via GitHub
supermem613 commented on code in PR #5439: URL: https://github.com/apache/incubator-gluten/pull/5439#discussion_r1589171084 ## gluten-core/src/main/java/org/apache/gluten/memory/memtarget/spark/DynamicOffHeapSizingPolicyChecker.java: ## @@ -0,0 +1,71 @@ +/* + * Licensed to the

Re: [PR] [GLUTEN-5438] feat: Dynamically sizing off-heap memory [incubator-gluten]

2024-05-03 Thread via GitHub
supermem613 commented on code in PR #5439: URL: https://github.com/apache/incubator-gluten/pull/5439#discussion_r1589168735 ## gluten-core/src/main/java/org/apache/gluten/memory/memtarget/spark/DynamicOffHeapSizingPolicyChecker.java: ## @@ -0,0 +1,71 @@ +/* + * Licensed to the

Re: [PR] [GLUTEN-5438] feat: Dynamically sizing off-heap memory [incubator-gluten]

2024-05-03 Thread via GitHub
supermem613 commented on code in PR #5439: URL: https://github.com/apache/incubator-gluten/pull/5439#discussion_r1589167853 ## gluten-core/src/main/java/org/apache/gluten/memory/memtarget/spark/DynamicOffHeapSizingPolicyChecker.java: ## @@ -0,0 +1,71 @@ +/* + * Licensed to the

Re: [PR] [GLUTEN-5438] feat: Dynamically sizing off-heap memory [incubator-gluten]

2024-04-26 Thread via GitHub
ulysses-you commented on code in PR #5439: URL: https://github.com/apache/incubator-gluten/pull/5439#discussion_r1580689141 ## gluten-core/src/main/java/org/apache/gluten/memory/memtarget/spark/DynamicOffHeapSizingPolicyChecker.java: ## @@ -0,0 +1,71 @@ +/* + * Licensed to the

Re: [PR] [GLUTEN-5438] feat: Dynamically sizing off-heap memory [incubator-gluten]

2024-04-23 Thread via GitHub
srinivasst commented on code in PR #5439: URL: https://github.com/apache/incubator-gluten/pull/5439#discussion_r1576767187 ## gluten-core/src/main/java/org/apache/gluten/memory/memtarget/spark/DynamicOffHeapSizingPolicyChecker.java: ## @@ -0,0 +1,71 @@ +/* + * Licensed to the

Re: [PR] [GLUTEN-5438] feat: Dynamically sizing off-heap memory [incubator-gluten]

2024-04-18 Thread via GitHub
zhztheplayer commented on code in PR #5439: URL: https://github.com/apache/incubator-gluten/pull/5439#discussion_r1571726146 ## gluten-core/src/main/java/org/apache/gluten/memory/memtarget/spark/DynamicOffHeapSizingPolicyChecker.java: ## @@ -0,0 +1,71 @@ +/* + * Licensed to the

Re: [PR] [GLUTEN-5438] feat: Dynamically sizing off-heap memory [incubator-gluten]

2024-04-18 Thread via GitHub
zhztheplayer commented on code in PR #5439: URL: https://github.com/apache/incubator-gluten/pull/5439#discussion_r1571725004 ## gluten-core/src/main/java/org/apache/gluten/memory/memtarget/spark/DynamicOffHeapSizingPolicyChecker.java: ## @@ -0,0 +1,71 @@ +/* + * Licensed to the

Re: [PR] [GLUTEN-5438] feat: Dynamically sizing off-heap memory [incubator-gluten]

2024-04-18 Thread via GitHub
zhztheplayer commented on code in PR #5439: URL: https://github.com/apache/incubator-gluten/pull/5439#discussion_r1571716511 ## gluten-core/src/main/java/org/apache/gluten/memory/memtarget/spark/DynamicOffHeapSizingPolicyChecker.java: ## @@ -0,0 +1,71 @@ +/* + * Licensed to the

Re: [PR] [GLUTEN-5438] feat: Dynamically sizing off-heap memory [incubator-gluten]

2024-04-18 Thread via GitHub
supermem613 commented on code in PR #5439: URL: https://github.com/apache/incubator-gluten/pull/5439#discussion_r1570923153 ## gluten-core/src/main/java/org/apache/gluten/memory/memtarget/spark/DynamicOffHeapSizingPolicyChecker.java: ## @@ -0,0 +1,71 @@ +/* + * Licensed to the

Re: [PR] [GLUTEN-5438] feat: Dynamically sizing off-heap memory [incubator-gluten]

2024-04-18 Thread via GitHub
supermem613 commented on code in PR #5439: URL: https://github.com/apache/incubator-gluten/pull/5439#discussion_r1570853800 ## gluten-core/src/main/java/org/apache/gluten/memory/memtarget/spark/DynamicOffHeapSizingPolicyChecker.java: ## @@ -0,0 +1,71 @@ +/* + * Licensed to the

Re: [PR] [GLUTEN-5438] feat: Dynamically sizing off-heap memory [incubator-gluten]

2024-04-18 Thread via GitHub
supermem613 commented on code in PR #5439: URL: https://github.com/apache/incubator-gluten/pull/5439#discussion_r1570828320 ## gluten-core/src/main/java/org/apache/gluten/memory/memtarget/spark/DynamicOffHeapSizingPolicyChecker.java: ## @@ -0,0 +1,71 @@ +/* + * Licensed to the

Re: [PR] [GLUTEN-5438] feat: Dynamically sizing off-heap memory [incubator-gluten]

2024-04-18 Thread via GitHub
supermem613 commented on code in PR #5439: URL: https://github.com/apache/incubator-gluten/pull/5439#discussion_r1570827513 ## gluten-core/src/main/java/org/apache/gluten/memory/memtarget/spark/DynamicOffHeapSizingPolicyChecker.java: ## @@ -0,0 +1,71 @@ +/* + * Licensed to the

Re: [PR] [GLUTEN-5438] feat: Dynamically sizing off-heap memory [incubator-gluten]

2024-04-17 Thread via GitHub
zhztheplayer commented on code in PR #5439: URL: https://github.com/apache/incubator-gluten/pull/5439#discussion_r1569852415 ## gluten-core/src/main/java/org/apache/gluten/memory/memtarget/spark/DynamicOffHeapSizingPolicyChecker.java: ## @@ -0,0 +1,71 @@ +/* + * Licensed to the

Re: [PR] [GLUTEN-5438] feat: Dynamically sizing off-heap memory [incubator-gluten]

2024-04-17 Thread via GitHub
zhztheplayer commented on code in PR #5439: URL: https://github.com/apache/incubator-gluten/pull/5439#discussion_r1569852415 ## gluten-core/src/main/java/org/apache/gluten/memory/memtarget/spark/DynamicOffHeapSizingPolicyChecker.java: ## @@ -0,0 +1,71 @@ +/* + * Licensed to the

Re: [PR] [GLUTEN-5438] feat: Dynamically sizing off-heap memory [incubator-gluten]

2024-04-17 Thread via GitHub
FelixYBW commented on PR #5439: URL: https://github.com/apache/incubator-gluten/pull/5439#issuecomment-2062904153 Thank you for your PR! @zhanglistar may do a test quickly -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

Re: [PR] [GLUTEN-5438] feat: Dynamically sizing off-heap memory [incubator-gluten]

2024-04-17 Thread via GitHub
github-actions[bot] commented on PR #5439: URL: https://github.com/apache/incubator-gluten/pull/5439#issuecomment-2061473106 https://github.com/apache/incubator-gluten/issues/5438 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to