RE: Lua plugin memory limit on Linux.

[Chou, Peter]

Kit,

This is the only error message seen from diags.log. It did not cause the ATS to 
core dump.

[Jun 20 13:27:45.258] traffic_server NOTE: plugin.config loading ...
[Jun 20 13:27:45.259] traffic_server NOTE: loading plugin 
'/home/pebc/opt/ats-gh-master/libexec/trafficserver/tslua.so'
[Jun 20 13:28:14.337] traffic_server ERROR: [ts_lua_add_module] lua_pcall 
/home/pebc/opt/ats-gh-master/etc/trafficserver/test-memory.lua failed: not 
enough memory
[Jun 20 13:28:14.337] traffic_server ERROR: [ts_lua][TSPluginInit] 
ts_lua_add_module failed
[Jun 20 13:28:14.337] traffic_server NOTE: plugin.config finished loading

This is how much memory was allocated before the failure --

[Jun 20 13:28:14.323] traffic_server DIAG: (ts_lua) Total bytes allocated = 
2071552000 bytes
[Jun 20 13:28:14.323] traffic_server DIAG: (ts_lua) Garbage Collector reports = 
2024154.9023438 MB

Modified the Lua script as follows --

function __init__(argtb)
   count = 0
   t = {}
   max = 2100

   FH = io.open("/dev/urandom", "rb")

   for i = 1, max, 1 do
      t[i] = FH:read(1024000)
      count = count + string.len(t[1])
      if i > 2000 then
         ts.debug("Total bytes allocated = " .. count .. " bytes")
         ts.debug("Garbage Collector reports = " .. collectgarbage("count") .. 
" MB")
      end
   end

   io.close(FH)

end

Thanks,
Peter

-----Original Message-----
From: Shu Kit Chan <chanshu...@gmail.com> 
Sent: Wednesday, June 19, 2019 11:32 PM
To: dev <dev@trafficserver.apache.org>
Subject: Re: Lua plugin memory limit on Linux.

Thanks for the info.

That's good to know. Have you tried how much more you can allocate?
And when you are over the limit, what is the message you are getting?

Perhaps we can update the doc with this information.

Kit

On Wed, Jun 19, 2019 at 12:01 PM Chou, Peter <pbc...@labs.att.com> wrote:
>
> Kit,
>
> Did not hear back from you regarding the memory limit for the Lua plugin on 
> Linux.
>
> Previously, I had tested on the command line with the luajit executable, but 
> I have completed some testing with the ATS master branch.
>
> I am running with Ubuntu 18.04 LTS Server 64-bit which ships with 'LuaJIT 
> 2.1.0-beta3'.
>
> Using the following configuration, I got good results, able to allocated up 
> to 2GB memory.
>
> plugin.config -
>
> tslua.so --states=1 test-memory.lua
>
> test-memory.lua -
>
> function __init__(argtb)
>    count = 0
>    t = {}
>    max = 2000
>
>    FH = io.open("/dev/urandom", "rb")
>
>    for i = 1, max, 1 do
>       t[i] = FH:read(1024000)
>       count = count + string.len(t[1])
>    end
>
>    io.close(FH)
>
>    ts.debug("Total bytes allocated = " .. count .. " bytes")
>    ts.debug("Garbage Collector reports = " .. collectgarbage("count") .. " 
> MB")
> end
>
> Result -
>
> pebc@piplup2:~/opt/ats-gh-master/bin$ ./traffic_server -V
> Traffic Server 9.0.0 Jun 18 2019 14:16:03 piplup2
> traffic_server: using root directory '/home/pebc/opt/ats-gh-master/'
> Apache Traffic Server - traffic_server - 9.0.0 - (build # 061814 on Jun 18 
> 2019 at 14:16:03)
> pebc@piplup2:~/opt/ats-gh-master/bin$ ./traffic_server
> Traffic Server 9.0.0 Jun 18 2019 14:16:03 piplup2
> traffic_server: using root directory '/home/pebc/opt/ats-gh-master/'
> [Jun 19 11:54:42.681] traffic_server DIAG: (ts_lua) Total bytes allocated = 
> 2048000000 bytes
> [Jun 19 11:54:42.681] traffic_server DIAG: (ts_lua) Garbage Collector reports 
> = 2001149.2382813 MB
>
> Appreciate it if you could take a look and validate these results.
>
> Thanks,
> Peter