[ https://issues.apache.org/jira/browse/ARROW-2879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16692184#comment-16692184 ]
Wes McKinney commented on ARROW-2879: ------------------------------------- Any update on this? > [Python] Arrow plasma can only use a small part of specified shared memory > -------------------------------------------------------------------------- > > Key: ARROW-2879 > URL: https://issues.apache.org/jira/browse/ARROW-2879 > Project: Apache Arrow > Issue Type: Bug > Components: Python > Reporter: chineking > Priority: Major > Fix For: 0.13.0 > > > Hi, thanks for the great job of arrow, it helps us a lot. > However, we encounter a problem when we were using plasma. > The sample code: > {code:python} > import numpy as np > import pyarrow as pa > import pyarrow.plasma as plasma > client = plasma.connect("/tmp/plasma", "", 0) > puts = [] > nbytes = 0 > while True: > a = np.ones((1000, 1000)) > try: > oid = client.put(a) > puts.append(client.get(oid)) > nbytes += a.nbytes > except pa.lib.PlasmaStoreFull: > print('use nbytes', nbytes) > break > {code} > We start a plasma store with 1G memory, but the nbytes output above is only > 496000000, which cannot even reach half of the memory we specified. > I cannot figure out why plasma can only use such a small part of shared > memory. Could anybody help me? Thanks a lot. -- This message was sent by Atlassian JIRA (v7.6.3#76005)