[jira] [Commented] (TS-3744) Crash (Seg Fault) when reenabling a VIO from a continuator which is different from the VIO's continuator.

2015-07-23 Thread Pavel Vazharov (JIRA)

[ 
https://issues.apache.org/jira/browse/TS-3744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14638775#comment-14638775
 ] 

Pavel Vazharov commented on TS-3744:


I see. I totally missed that I can call every continuator with TSContCall, not 
only the transform one.

Thank you for the help. Tested it, it works as expected. I suppose that I have 
to close the issue as invalid. It was just a misunderstanding on my site.

 Crash (Seg Fault) when reenabling a VIO from a continuator which is different 
 from the VIO's continuator.
 -

 Key: TS-3744
 URL: https://issues.apache.org/jira/browse/TS-3744
 Project: Traffic Server
  Issue Type: Bug
  Components: Plugins, TS API
Affects Versions: 5.3.0
Reporter: Pavel Vazharov
Assignee: Alan M. Carroll
 Fix For: 6.1.0


 Hi,
 I'm trying to create ATS plugin which uses the API for cache write 
 (TSCacheWrite, TSVConnWrite). For the write part, from a transformation, I'm 
 trying to stream the data to both the client and the cache in the same time. 
 The problem described below IMO can be summarized as - crash reenabling of 
 one VIO from a continuator which is different from the VIO's continuator.
 Here is the backtrace of the crash. 
 traffic_server: Segmentation fault (Address not mapped to object [0x28])
 traffic_server - STACK TRACE: 
 /usr/local/bin/traffic_server(_Z19crash_logger_invokeiP9siginfo_tPv+0x8e)[0x4ad13e]
 /lib/x86_64-linux-gnu/libpthread.so.0(+0x10340)[0x2b9092c2a340]
 /usr/local/bin/traffic_server(_ZN7CacheVC8reenableEP3VIO+0x28)[0x6db868]
 /home/freak82/ats/src/plugins/ccontent/ccontent.so(+0x29e5)[0x2b9096bce9e5]
 /home/freak82/ats/src/plugins/ccontent/ccontent.so(+0x3094)[0x2b9096bcf094]
 /usr/local/bin/traffic_server(_ZN7EThread13process_eventEP5Eventi+0x120)[0x767ea0]
 /usr/local/bin/traffic_server(_ZN7EThread7executeEv+0x81b)[0x768aab]
 /usr/local/bin/traffic_server(main+0xee6)[0x495436]
 /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf5)[0x2b909387dec5]
 /usr/local/bin/traffic_server[0x49ba6f]
 It's like the VIO mutex-thread_holding or the VIO object itself are in some 
 inappropriate state or invalid. The VIO has the same memory address as the 
 originally created one and it's continuator is not explicitly destroyed 
 (TSContDestroy). The associated buffer and reader are also alive.
 I'm not sure if the thing (writing in-parallel) that I'm trying to do is 
 possible with the current API, by design? Is it possible/allowed by design to 
 copy bytes to one VIO buffer and reenable the same VIO from another 
 continuator, not the same continuator as the one of the VIO.
 If it's possible am I doing something wrong or this is a bug?
 Basically, I'm trying to do it in the following way. The explanations skip 
 the error handling.
 1. On transformation start, on the first EVENT_IMMEDIATE from the upstream, 
 the code initializes the client stream (TSBuffer, TSBufferReader and TSVIO as 
 in the null-transform plugin) and then start the cache write (TSCacheWrite) 
 with a created and digested cache key (TSCacheKey).
 2. On EVENT_CACHE_OPEN_WRITE, the code initializes the cache stream 
 (TSBuffer, TSBufferReader and TSVIO) in the same way as the client stream, 
 but using the passed TSCont and TSVConn from the event data. So far, it works 
 as expected.
 3. Both continuator callbacks, for the transformation and for the cache 
 write, are handling events WRITE_READY and WRITE_COMPLETE. The transformation 
 callback also handles EVENT_IMMEDIATE to know when there is more data from 
 the upstream.
 I was thinking to mark every stream as ready when the corresponding callback 
 receives WRITE_READY, and when both streams are ready to copy the available 
 data from the upstream to them, then reenable the both streams and the 
 upstream. Then when there are new data available from the upstream, to copy 
 them again when the both streams becomes ready, etc, etc.
 Usually the first writes/copies and reenables are made from inside the 
 TSCacheWrite, because it's reentrant and generates WRITE_READY for the cache 
 continuator. These operations succeeds. The problem is that the plugin leads 
 to crash in the ATS when it tries to reeenable the cache VIO from inside the 
 transform continuator.
 I tried to pass whole data from the upstream to the client first, copying 
 (TSIOBufferCopy) in-paralles them to a temporary buffer, and initiate cache 
 write at the end of the transformation and then write the data from the 
 buffer to the cache VIO (similarly to the metalink plugin). This also works 
 as expected.
 Thanks,
 Pavel.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (TS-3744) Crash (Seg Fault) when reenabling a VIO from a continuator which is different from the VIO's continuator.

2015-07-22 Thread Pavel Vazharov (JIRA)

[ 
https://issues.apache.org/jira/browse/TS-3744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14636450#comment-14636450
 ] 

Pavel Vazharov commented on TS-3744:


Thank you for the response. 

May I ask something that I still don't understand in the suggested scheme?
Suppose we have a case when we receive huge amount of data in the 
transformation i.e. we expect to receive multiple events IMMEDIATE from the 
upstream/server VIO in the transformation. I'll try to present it in steps.
1. We setup the transformation and a buffer (with two independent readers - one 
for client and one for cache) to write the input data from the transformation.
2. We receive the first event IMMEDIATE from the upstream.
2.1. We copy the incoming data to our intermediate buffer marking them as 
consumed for the upstream reader. Last we do TSContCall to the upstream VIO 
with event VCONN_WRITE_READY in order to inform it that we have consumed the 
data and we wait for new data.
2.2. We setup the downstream/client VIO and copy the data from the intermediate 
buffer to the downstream buffer, mark the data as consumed for the client side 
intermediate reader, and last we reenable the downstream VIO in order to 
receive WRITE_READY event from it later.
2.3. We start cache write (TSCacheWrite). Usually we receive event 
CACHE_OPEN_WRITE inside the call, because it is reentrant. Here we setup the 
cache write VIO and copy the data from the intermediate buffer to the cache 
write buffer. Then we mark the data as consumed for the cache side intermediate 
reader. Last thing here is to reenable the cachestream VIO in order to receive 
WRITE_READY event from it later.
3. Let's say that we receive WRITE_READY event in the cache write continuator 
before we receive new event IMMEDIATE from the upstream i.e. the cache is ready 
for new data but we still don't have such from the server. As far as I tested 
it, this WRITE_READY event usually happens inside the TSCacheWrite call when we 
reenable the cache write VIO. We can't do anything here and we just exit from 
the event handler.
HERE STARTS THE PART THAT I DON'T UNDERSTAND. I'll skip the writing to the 
client because it's not important (IMO) and works as expected.
4. We receive new event IMMEDIATE from the upstream/server VIO.
4.1. We copy the new data to the intermediate buffer marking them as consumed 
for the upstream reader. Then we do TSContCall to the upstream VIO with event 
VCONN_WRITE_READY.
We know that the cache VIO is ready for new data. We copy the data to it's 
buffer using the cache itermediate reader. However, as far as I understand, we 
need to reenable the cache VIO after that, in order to inform it to consume the 
data and send us VCONN_WRITE_READY later. But the problem is that we are inside 
a different continuator and we can't reenable the cache VIO from inside it. 

I know that we can just copy the data to the intermediate buffer and stream 
them to the client simultaneously. Later, when all data from the upstream are 
received, we can do the cache write with the whole amount and this works as 
expected. I'm not sure how this will work for the case with huge amount of data 
from the server, because as far as I understand, it'll keep the whole data 
alive in the memory and this could be a problem.

Thanks again,
Pavel.

 Crash (Seg Fault) when reenabling a VIO from a continuator which is different 
 from the VIO's continuator.
 -

 Key: TS-3744
 URL: https://issues.apache.org/jira/browse/TS-3744
 Project: Traffic Server
  Issue Type: Bug
  Components: Plugins, TS API
Affects Versions: 5.3.0
Reporter: Pavel Vazharov
 Fix For: 6.1.0


 Hi,
 I'm trying to create ATS plugin which uses the API for cache write 
 (TSCacheWrite, TSVConnWrite). For the write part, from a transformation, I'm 
 trying to stream the data to both the client and the cache in the same time. 
 The problem described below IMO can be summarized as - crash reenabling of 
 one VIO from a continuator which is different from the VIO's continuator.
 Here is the backtrace of the crash. 
 traffic_server: Segmentation fault (Address not mapped to object [0x28])
 traffic_server - STACK TRACE: 
 /usr/local/bin/traffic_server(_Z19crash_logger_invokeiP9siginfo_tPv+0x8e)[0x4ad13e]
 /lib/x86_64-linux-gnu/libpthread.so.0(+0x10340)[0x2b9092c2a340]
 /usr/local/bin/traffic_server(_ZN7CacheVC8reenableEP3VIO+0x28)[0x6db868]
 /home/freak82/ats/src/plugins/ccontent/ccontent.so(+0x29e5)[0x2b9096bce9e5]
 /home/freak82/ats/src/plugins/ccontent/ccontent.so(+0x3094)[0x2b9096bcf094]
 /usr/local/bin/traffic_server(_ZN7EThread13process_eventEP5Eventi+0x120)[0x767ea0]
 /usr/local/bin/traffic_server(_ZN7EThread7executeEv+0x81b)[0x768aab]
 

[jira] [Commented] (TS-3744) Crash (Seg Fault) when reenabling a VIO from a continuator which is different from the VIO's continuator.

2015-07-21 Thread Alan M. Carroll (JIRA)

[ 
https://issues.apache.org/jira/browse/TS-3744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14636186#comment-14636186
 ] 

Alan M. Carroll commented on TS-3744:
-

It should be possible to stream to both cache and user agent, the internal 
design is intended to be able to do exactly that. It looks like you've written 
a transform plugin already. If you just want that transformed data written to 
the HTTP cache, that should just work.

In general a VIO will have a lock in order to manipulate any of the data 
associated with that VIO, including any buffers. Although the data in your back 
trace is limited, looking at the code I would guess the crash is from the 
assert {{ink_assert(avio-mutex-thread_holding);}} which indicates the VIO 
mutex is not locked. For this reason it is tricky to re-enable a VIO from the 
continuation of another VIO. You would need to get the lock for the other VIO. 
It may also be the case that the other VIO is not in a state where a reenable 
is useful.

I think the best approach is to handle each continuation independently, that is 
what the internal design expects. The data in the IO buffers is referenced 
counted and shared, so that if you write the same IOBuffer to two streams, 
there is still only one copy of the data. This means it is not any more 
efficient to try to write the exact same data to both output streams 
simultaneously. The is the purpose of a buffer reader {{TSIOBufferReader}}. You 
write data to the buffer and it is consumed by the readers. When all readers 
have consumed a particular chunk of data, it is discarded. I would recommend 
setting up a buffer and writing to it from the input data in the transform. The 
transform would then write the data to its downstream VIO when it gets 
WRITE_READY. At the same time, independently,  the cache continuation would 
have a reader and write from that when it got its own WRITE_READY. You could 
then clean up the shared IOBuffer (not reader) when the transaction finishes.

I would also note that the direct cache API supports raw data, normally HTTP 
requests will not find any object written to the cache by the API, a plugin 
would be required to do the look up and serve that content.

 Crash (Seg Fault) when reenabling a VIO from a continuator which is different 
 from the VIO's continuator.
 -

 Key: TS-3744
 URL: https://issues.apache.org/jira/browse/TS-3744
 Project: Traffic Server
  Issue Type: Bug
  Components: Plugins, TS API
Affects Versions: 5.3.0
Reporter: Pavel Vazharov
 Fix For: 6.1.0


 Hi,
 I'm trying to create ATS plugin which uses the API for cache write 
 (TSCacheWrite, TSVConnWrite). For the write part, from a transformation, I'm 
 trying to stream the data to both the client and the cache in the same time. 
 The problem described below IMO can be summarized as - crash reenabling of 
 one VIO from a continuator which is different from the VIO's continuator.
 Here is the backtrace of the crash. 
 traffic_server: Segmentation fault (Address not mapped to object [0x28])
 traffic_server - STACK TRACE: 
 /usr/local/bin/traffic_server(_Z19crash_logger_invokeiP9siginfo_tPv+0x8e)[0x4ad13e]
 /lib/x86_64-linux-gnu/libpthread.so.0(+0x10340)[0x2b9092c2a340]
 /usr/local/bin/traffic_server(_ZN7CacheVC8reenableEP3VIO+0x28)[0x6db868]
 /home/freak82/ats/src/plugins/ccontent/ccontent.so(+0x29e5)[0x2b9096bce9e5]
 /home/freak82/ats/src/plugins/ccontent/ccontent.so(+0x3094)[0x2b9096bcf094]
 /usr/local/bin/traffic_server(_ZN7EThread13process_eventEP5Eventi+0x120)[0x767ea0]
 /usr/local/bin/traffic_server(_ZN7EThread7executeEv+0x81b)[0x768aab]
 /usr/local/bin/traffic_server(main+0xee6)[0x495436]
 /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf5)[0x2b909387dec5]
 /usr/local/bin/traffic_server[0x49ba6f]
 It's like the VIO mutex-thread_holding or the VIO object itself are in some 
 inappropriate state or invalid. The VIO has the same memory address as the 
 originally created one and it's continuator is not explicitly destroyed 
 (TSContDestroy). The associated buffer and reader are also alive.
 I'm not sure if the thing (writing in-parallel) that I'm trying to do is 
 possible with the current API, by design? Is it possible/allowed by design to 
 copy bytes to one VIO buffer and reenable the same VIO from another 
 continuator, not the same continuator as the one of the VIO.
 If it's possible am I doing something wrong or this is a bug?
 Basically, I'm trying to do it in the following way. The explanations skip 
 the error handling.
 1. On transformation start, on the first EVENT_IMMEDIATE from the upstream, 
 the code initializes the client stream (TSBuffer, TSBufferReader and TSVIO as 
 in the null-transform plugin) and then start the cache write