kou commented on issue #50471:
URL: https://github.com/apache/arrow/issues/50471#issuecomment-4964105930
I don't object that we backport the patch by ourselves but I'm negative for
the approach because it may introduce another problem.
How about using system `malloc()` on non-main thread something like the
following?
```diff
diff --git a/cpp/src/arrow/memory_pool.cc b/cpp/src/arrow/memory_pool.cc
index 0b4843bec3..d00760291d 100644
--- a/cpp/src/arrow/memory_pool.cc
+++ b/cpp/src/arrow/memory_pool.cc
@@ -130,12 +130,40 @@ std::optional<MemoryPoolBackend> UserSelectedBackend()
{
return user_selected_backend;
}
+bool IsMainThread() {
+#ifdef __linux__
+ // The main thread's ID must equal to process ID on Linux.
+ return gettid() == getpid();
+#elif defined(__APPLE__) || defined(BSD)
+ // We can use pthread_main_np() on BSD family.
+ return pthread_main_np() == 1;
+#elif defined(_WIN32)
+ // Windows doesn't provide an API for this. We can implement a
+ // heuristic approach.
+ return true;
+#else
+ // Assume the current thread is the main thread on others.
+ return true;
+#endif
+}
+
MemoryPoolBackend DefaultBackend() {
auto backend = UserSelectedBackend();
if (backend.has_value()) {
return backend.value();
}
- struct SupportedBackend default_backend = SupportedBackends().front();
+ static struct SupportedBackend default_backend;
+ static std::once_flag once;
+ std::call_once(once, [&]() {
+ if (IsMainThread()) {
+ default_backend = SupportedBackends().front();
+ } else {
+ // Fallback to the system backend.
+ // We can remove this once mimalloc()'s thread related problem is
fixed.
+ // See also: https://github.com/microsoft/mimalloc/issues/1287
+ default_backend = SupportedBackends().back();
+ }
+ });
return default_backend.backend;
}
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
