chineking created ARROW-2879:
--------------------------------
Summary: Arrow plasma can only use a small part of specified
shared memory
Key: ARROW-2879
URL: https://issues.apache.org/jira/browse/ARROW-2879
Project: Apache Arrow
Issue Type: Bug
Components: Python
Reporter: chineking
Hi, thanks for the great job of arrow, it helps us a lot.
However, we encounter a problem when we were using plasma.
The sample code:
{code:python}
import numpy as np
import pyarrow as pa
import pyarrow.plasma as plasma
client = plasma.connect("/tmp/plasma", "", 0)
puts = []
nbytes = 0
while True:
a = np.ones((1000, 1000))
try:
oid = client.put(a)
puts.append(client.get(oid))
nbytes += a.nbytes
except pa.lib.PlasmaStoreFull:
print('use nbytes', nbytes)
break
{code}
We start a plasma store with 1G memory, but the nbytes output above is only
496000000, which cannot even reach half of the memory we specified.
I cannot figure out why plasma can only use such a small part of shared memory.
Could anybody help me? Thanks a lot.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
