[jira] [Updated] (TS-1075) Port range bottleneck in transparent proxy mode
[ https://issues.apache.org/jira/browse/TS-1075?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] B Wyatt updated TS-1075: Fix Version/s: (was: 3.1.4) 3.3.0 > Port range bottleneck in transparent proxy mode > --- > > Key: TS-1075 > URL: https://issues.apache.org/jira/browse/TS-1075 > Project: Traffic Server > Issue Type: Bug > Components: Core >Affects Versions: 3.0.1 > Environment: Centos 5.6, kernel 2.6.39.2 compiled with TPROXY support > ATS compiled as: ./configure --enable-tproxy >Reporter: Danny Shporer >Assignee: B Wyatt > Fix For: 3.3.0 > > Attachments: ports.patch > > > The Linux TPROXY stack only takes into account the local addresses when using > dynamic bind (bind without specifying a specific port). This limits the port > range to only the local range (around 30K by default and can be extended to > around 64K) - this together with the TIME-WAIT Linux method of releasing > ports causes a bottleneck). > One symptom of this is that traffic_cop cannot open a connection to the > server to monitor it (it gets error 99 - address already in use) and kills > it. > Another issue is when opening the connection to the server. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (TS-1163) Raw disks with more than (2^32)-1 sectors (usually 2TB) are not supported on linux
[ https://issues.apache.org/jira/browse/TS-1163?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] B Wyatt updated TS-1163: Fix Version/s: (was: 3.1.4) 3.3.0 > Raw disks with more than (2^32)-1 sectors (usually 2TB) are not supported on > linux > -- > > Key: TS-1163 > URL: https://issues.apache.org/jira/browse/TS-1163 > Project: Traffic Server > Issue Type: Bug > Components: Cache >Reporter: B Wyatt >Assignee: B Wyatt > Fix For: 3.3.0 > > Attachments: blkgetsize64.bwyatt.patch > > > Due to 32bit integers in both the trafficersever code and the ioctl used to > determine raw disk size, the number of sectors reported to the cache storage > system is bound to 0-0x. If a disk has 512 byte sectors and is > larger than 2TB it will report (num_sectors % 0x) * 512 bytes > avaliable. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (TS-1090) Configuration and plugin support for SO_MARK (on supporting platforms)
[ https://issues.apache.org/jira/browse/TS-1090?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] B Wyatt updated TS-1090: Fix Version/s: (was: 3.1.4) 3.3.0 > Configuration and plugin support for SO_MARK (on supporting platforms) > -- > > Key: TS-1090 > URL: https://issues.apache.org/jira/browse/TS-1090 > Project: Traffic Server > Issue Type: New Feature > Components: HTTP, Network, TS API >Reporter: B Wyatt >Assignee: B Wyatt > Fix For: 3.3.0 > > Attachments: so_mark.patch > > > SO_MARK allows for all packets sent out via a given socket to be pre-marked > (similar to the -j MARK target in iptables, or the fwmark semantic in ip > rules). This feature allows configuration to dictate SO_MARKs for accepting > sockets, cluster sockets and origin server sockets independently. > Additionally, plugins and internal systems can set per-transaction SO_MARKS > for outgoing packets. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (TS-1087) TSHttpTxnOutgoingAddrSet forward declaration does not match implementation
[
https://issues.apache.org/jira/browse/TS-1087?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
B Wyatt updated TS-1087:
Fix Version/s: (was: 3.1.4)
3.3.0
> TSHttpTxnOutgoingAddrSet forward declaration does not match implementation
> --
>
> Key: TS-1087
> URL: https://issues.apache.org/jira/browse/TS-1087
> Project: Traffic Server
> Issue Type: Bug
> Components: TS API
>Affects Versions: 3.1.0
>Reporter: B Wyatt
>Assignee: B Wyatt
>Priority: Trivial
> Fix For: 3.3.0
>
> Attachments: txn-outgoing-addr.patch
>
> Original Estimate: 1m
> Remaining Estimate: 1m
>
> ts.h.in lists the following declaration:
> {code}TSReturnCode TSHttpTxnOutgoingAddrSet(TSHttpTxn txnp, struct sockaddr
> const* addr);{code}
> However, the current implementation has this function sig:
> {code}tsapi TSReturnCode TSHttpTxnOutgoingAddrSet(TSHttpTxn txnp, struct
> sockaddr const* addr, socklen_t addrlen);{code}
> Trafficserver is unable to load plugins which use this function due to the
> unresolved symbol.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (TS-1163) Raw disks with more than (2^32)-1 sectors (usually 2TB) are not supported on linux
[ https://issues.apache.org/jira/browse/TS-1163?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] B Wyatt updated TS-1163: Attachment: blkgetsize64.bwyatt.patch It will still be a little bit while I dig myself out of my hole and begin directly committing fixes to the repo. Until then, I've attached a patch here that should work on codebases as old as June 2011 that provides support for larger disks via the BLKGETSIZE64 ioctl on linux. > Raw disks with more than (2^32)-1 sectors (usually 2TB) are not supported on > linux > -- > > Key: TS-1163 > URL: https://issues.apache.org/jira/browse/TS-1163 > Project: Traffic Server > Issue Type: Bug > Components: Cache >Reporter: B Wyatt >Assignee: B Wyatt > Attachments: blkgetsize64.bwyatt.patch > > > Due to 32bit integers in both the trafficersever code and the ioctl used to > determine raw disk size, the number of sectors reported to the cache storage > system is bound to 0-0x. If a disk has 512 byte sectors and is > larger than 2TB it will report (num_sectors % 0x) * 512 bytes > avaliable. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (TS-1090) Configuration and plugin support for SO_MARK (on supporting platforms)
[ https://issues.apache.org/jira/browse/TS-1090?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] B Wyatt updated TS-1090: Attachment: so_mark.patch This patch implements the SO_MARK feature for linux systems (that include SO_MARK in the sys/socket.h) > Configuration and plugin support for SO_MARK (on supporting platforms) > -- > > Key: TS-1090 > URL: https://issues.apache.org/jira/browse/TS-1090 > Project: Traffic Server > Issue Type: New Feature > Components: HTTP, Network, TS API >Reporter: B Wyatt > Attachments: so_mark.patch > > > SO_MARK allows for all packets sent out via a given socket to be pre-marked > (similar to the -j MARK target in iptables, or the fwmark semantic in ip > rules). This feature allows configuration to dictate SO_MARKs for accepting > sockets, cluster sockets and origin server sockets independently. > Additionally, plugins and internal systems can set per-transaction SO_MARKS > for outgoing packets. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (TS-1089) Allow Plugins to create transparent internal http connections
[ https://issues.apache.org/jira/browse/TS-1089?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] B Wyatt updated TS-1089: Attachment: trans-pluginvc.patch This is against a 3.1.0 rc, which means that it is not compatible with head. I am attaching it so that it can remind me or someone else to make the minor changes and submit it. > Allow Plugins to create transparent internal http connections > - > > Key: TS-1089 > URL: https://issues.apache.org/jira/browse/TS-1089 > Project: Traffic Server > Issue Type: New Feature > Components: TS API >Affects Versions: 3.1.0 > Environment: This feature was built on top of a 3.1.0 release > candidate. Unfortunately, it will superficially conflict with the IPv6 > changes (I'm including it in case someone needs to update it before I do) >Reporter: B Wyatt > Attachments: trans-pluginvc.patch > > > Plugins can create a fake connection HTTP connection into the proxy and > provide a logging address. This feature allows those connections to appear > to the rest of trafficsever as incoming transparent connections (such as the > ones accepted by tproxy enabled installations). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (TS-1088) Allow Per-transaction Transparency (TProxy) Override
[ https://issues.apache.org/jira/browse/TS-1088?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] B Wyatt updated TS-1088: Attachment: tsapi-allow-transparency.patch This patch adds a flag to transactions that can be poked from plugins. This flag is used to suppress the NetVCOptions::FOREIGN_ADDR binding when opening the origin server connection. Specifically assigned binds still work, only transparency is suppressed. > Allow Per-transaction Transparency (TProxy) Override > > > Key: TS-1088 > URL: https://issues.apache.org/jira/browse/TS-1088 > Project: Traffic Server > Issue Type: New Feature > Components: HTTP, TS API >Affects Versions: 3.1.0 > Environment: The feature was built on top of a 3.1.0 release > candidate, however only minor adjustments are needed for HEAD >Reporter: B Wyatt > Attachments: tsapi-allow-transparency.patch > > > Address level transparency (using TProxy) is propagated forward from the > incoming connection based on global configuration. This feature allows > internal and external systems that use the TS api to "disable" propagation on > a per transaction basis. The result is that the client<->proxy connection > transparently appears as a client<->origin server connection and the > proxy<->origin server connection is opaque. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (TS-1088) Allow Per-transaction Transparency (TProxy) Override
[ https://issues.apache.org/jira/browse/TS-1088?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] B Wyatt updated TS-1088: Description: Address level transparency (using TProxy) is propagated forward from the incoming connection based on global configuration. This feature allows internal and external systems that use the TS api to "disable" propagation on a per transaction basis. The result is that the client<=>proxy connection transparently appears as a client<=>origin server connection and the proxy<=>origin server connection is opaque.(was: Address level transparency (using TProxy) is propagated forward from the incoming connection based on global configuration. This feature allows internal and external systems that use the TS api to "disable" propagation on a per transaction basis. The result is that the client<->proxy connection transparently appears as a client<->origin server connection and the proxy<->origin server connection is opaque. ) > Allow Per-transaction Transparency (TProxy) Override > > > Key: TS-1088 > URL: https://issues.apache.org/jira/browse/TS-1088 > Project: Traffic Server > Issue Type: New Feature > Components: HTTP, TS API >Affects Versions: 3.1.0 > Environment: The feature was built on top of a 3.1.0 release > candidate, however only minor adjustments are needed for HEAD >Reporter: B Wyatt > Attachments: tsapi-allow-transparency.patch > > > Address level transparency (using TProxy) is propagated forward from the > incoming connection based on global configuration. This feature allows > internal and external systems that use the TS api to "disable" propagation on > a per transaction basis. The result is that the client<=>proxy connection > transparently appears as a client<=>origin server connection and the > proxy<=>origin server connection is opaque. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (TS-1087) TSHttpTxnOutgoingAddrSet forward declaration does not match implementation
[
https://issues.apache.org/jira/browse/TS-1087?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
B Wyatt updated TS-1087:
Attachment: txn-outgoing-addr.patch
This version of a "Fix" is small but requires any plugins that use the function
(there can't be any that are up-to-date and working) to change their source.
> TSHttpTxnOutgoingAddrSet forward declaration does not match implementation
> --
>
> Key: TS-1087
> URL: https://issues.apache.org/jira/browse/TS-1087
> Project: Traffic Server
> Issue Type: Bug
> Components: TS API
>Affects Versions: 3.1.0
>Reporter: B Wyatt
>Priority: Trivial
> Attachments: txn-outgoing-addr.patch
>
> Original Estimate: 1m
> Remaining Estimate: 1m
>
> ts.h.in lists the following declaration:
> {code}TSReturnCode TSHttpTxnOutgoingAddrSet(TSHttpTxn txnp, struct sockaddr
> const* addr);{code}
> However, the current implementation has this function sig:
> {code}tsapi TSReturnCode TSHttpTxnOutgoingAddrSet(TSHttpTxn txnp, struct
> sockaddr const* addr, socklen_t addrlen);{code}
> Trafficserver is unable to load plugins which use this function due to the
> unresolved symbol.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (TS-996) HTTPHdr::m_host goes stale if HdrHeap::evacuate_from_str_heaps is called
[ https://issues.apache.org/jira/browse/TS-996?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] B Wyatt updated TS-996: --- Attachment: m_host.v4.patch Added initialization of host_len in the case that it isn't in the URL or the Mime header. The function returns a NULL string and 0 length in that case. > HTTPHdr::m_host goes stale if HdrHeap::evacuate_from_str_heaps is called > > > Key: TS-996 > URL: https://issues.apache.org/jira/browse/TS-996 > Project: Traffic Server > Issue Type: Bug > Components: HTTP, MIME >Affects Versions: 3.1.0 >Reporter: B Wyatt >Assignee: Leif Hedstrom > Fix For: 3.1.2 > > Attachments: m_host.V2.patch, m_host.patch, m_host.v3.patch, > m_host.v4.patch > > > class HTTPHdr stores a copy of the string pointer from either the URLimpl or > the MIMEHdr for the host name in m_host. In both cases, these strings can be > moved to a new heap underneath the HTTPHdr. When this happens, the process > will, at best read stale memory and be fine and at worst read unmapped memory > and segfault. > Currently, HdrHeap::evacuate_from_str_heaps is called to coalesce multiple > heaps into a single heap. When this happens it will directly access the low > level objects via ::move_strings calls. These objects do not posses the > necessary information to inform parent objects about the change, nor does the > HdrHeap directly inform interested parties. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (TS-996) HTTPHdr::m_host goes stale if HdrHeap::evacuate_from_str_heaps is called
[ https://issues.apache.org/jira/browse/TS-996?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] B Wyatt updated TS-996: --- Attachment: m_host.v3.patch Attaching V3 which has no changes from V2 save that it has the header changes (which existed in the original patch but were erroneously missed in the V2 patch) I have been running this on a small handful of deployments for a few months, no major issues have cropped up. > HTTPHdr::m_host goes stale if HdrHeap::evacuate_from_str_heaps is called > > > Key: TS-996 > URL: https://issues.apache.org/jira/browse/TS-996 > Project: Traffic Server > Issue Type: Bug > Components: HTTP, MIME >Affects Versions: 3.1.0 >Reporter: B Wyatt >Assignee: Leif Hedstrom > Fix For: 3.1.2 > > Attachments: m_host.V2.patch, m_host.patch, m_host.v3.patch > > > class HTTPHdr stores a copy of the string pointer from either the URLimpl or > the MIMEHdr for the host name in m_host. In both cases, these strings can be > moved to a new heap underneath the HTTPHdr. When this happens, the process > will, at best read stale memory and be fine and at worst read unmapped memory > and segfault. > Currently, HdrHeap::evacuate_from_str_heaps is called to coalesce multiple > heaps into a single heap. When this happens it will directly access the low > level objects via ::move_strings calls. These objects do not posses the > necessary information to inform parent objects about the change, nor does the > HdrHeap directly inform interested parties. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (TS-949) key->volume hash table is not consistent when a disk is marked as bad or removed due to failure
[
https://issues.apache.org/jira/browse/TS-949?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
B Wyatt updated TS-949:
---
Attachment: explicit-pair.patch
I made a quick patch which converts the implied pairing of elements in the
rtable array into an explicit pair (it applies on top of TS-949-jp2.patch).
It is a non-functional change however, I thought it may make future
review/modification a little easier.
Feel free to toss it in the circular file, it won't hurt my feelings.
> key->volume hash table is not consistent when a disk is marked as bad or
> removed due to failure
> ---
>
> Key: TS-949
> URL: https://issues.apache.org/jira/browse/TS-949
> Project: Traffic Server
> Issue Type: Bug
> Components: Cache
>Affects Versions: 3.1.0
> Environment: Multi-volume cache with apparently faulty drives
>Reporter: B Wyatt
>Assignee: John Plevyak
> Fix For: 3.1.2
>
> Attachments: TS-949-jp-1.patch, TS-949-jp2.patch, TS949-BW-p1.patch,
> explicit-pair.patch
>
>
> The method for resolving collisions when distributing hash-table space to
> volumes for the object_key->volume hash table creates inconsistency when a
> disk is determined to be bad, or when a failed disk is removed from the
> volume.config.
> Background:
> The hash space is distributed by round robin draft where each volume "drafts"
> a random index in the hash table until the hash space is exhausted. The
> random order in which a given volume drafts hash table slots is consistent
> across reboot/crash/disk-failure, however when a volume attempts to draft a
> slot which has already been occupied, it skips to its next random pick and
> attempts to draft that slot until it finds an open slot. This ensures that
> the hash is partitioned evenly between volumes.
> The issue:
> Resolving slot contention breaks the consistency as it is dependent on the
> order that the volumes draft. When rebuilding the hash after disk failure or
> reboot with fewer drives, a volume may secure an index that was previously
> occupied by the dead-disk. In the old hash, the surviving volume would have
> selected another random index due to contention. If this index is taken, by
> the next draft round it will represent an inconsistent key->volume result.
> The effects of one inconsistency will then cascade as whichever volume
> occupies that index after removing a dead disk is now behind on its draft
> sequence as well.
> An Example:
> ||Disk||Draft Sequence||
> |A|1,4,7,5|
> |B|4,2,8,1|
> |C|3,7,5,2|
> Pre-failure Hash Table after 2 rounds of draft:
> |A|B|C|B|C|?|A|?|
> Post-failure of drive B Hash Table after 3 rounds of draft:
> |A|C|C|A|{color:red}A{color}|?|{color:red}C{color}|?|
> Two slots have become inconsistent and more will probably follow. These
> inconsistencies become objects stored in a volume but lost to the top level
> cache for open/lookup.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (TS-949) key->volume hash table is not consistent when a disk is marked as bad or removed due to failure
[
https://issues.apache.org/jira/browse/TS-949?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
B Wyatt updated TS-949:
---
Attachment: TS949-BW-p1.patch
Attaching a modified version of JP's patch that removes the unnecessary(?)
multiplication and division. Additionally, it uses the volume index mapping.
> key->volume hash table is not consistent when a disk is marked as bad or
> removed due to failure
> ---
>
> Key: TS-949
> URL: https://issues.apache.org/jira/browse/TS-949
> Project: Traffic Server
> Issue Type: Bug
> Components: Cache
>Affects Versions: 3.1.0
> Environment: Multi-volume cache with apparently faulty drives
>Reporter: B Wyatt
>Assignee: John Plevyak
> Fix For: 3.1.2
>
> Attachments: TS-949-jp-1.patch, TS949-BW-p1.patch
>
>
> The method for resolving collisions when distributing hash-table space to
> volumes for the object_key->volume hash table creates inconsistency when a
> disk is determined to be bad, or when a failed disk is removed from the
> volume.config.
> Background:
> The hash space is distributed by round robin draft where each volume "drafts"
> a random index in the hash table until the hash space is exhausted. The
> random order in which a given volume drafts hash table slots is consistent
> across reboot/crash/disk-failure, however when a volume attempts to draft a
> slot which has already been occupied, it skips to its next random pick and
> attempts to draft that slot until it finds an open slot. This ensures that
> the hash is partitioned evenly between volumes.
> The issue:
> Resolving slot contention breaks the consistency as it is dependent on the
> order that the volumes draft. When rebuilding the hash after disk failure or
> reboot with fewer drives, a volume may secure an index that was previously
> occupied by the dead-disk. In the old hash, the surviving volume would have
> selected another random index due to contention. If this index is taken, by
> the next draft round it will represent an inconsistent key->volume result.
> The effects of one inconsistency will then cascade as whichever volume
> occupies that index after removing a dead disk is now behind on its draft
> sequence as well.
> An Example:
> ||Disk||Draft Sequence||
> |A|1,4,7,5|
> |B|4,2,8,1|
> |C|3,7,5,2|
> Pre-failure Hash Table after 2 rounds of draft:
> |A|B|C|B|C|?|A|?|
> Post-failure of drive B Hash Table after 3 rounds of draft:
> |A|C|C|A|{color:red}A{color}|?|{color:red}C{color}|?|
> Two slots have become inconsistent and more will probably follow. These
> inconsistencies become objects stored in a volume but lost to the top level
> cache for open/lookup.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (TS-947) AIO Race condition on non NT systems
[
https://issues.apache.org/jira/browse/TS-947?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
B Wyatt updated TS-947:
---
Attachment: timed-wait-AIO.patch
Added JPlevyak's timed wait solution to the ticket, it is the current solution
in the source tree.
> AIO Race condition on non NT systems
>
>
> Key: TS-947
> URL: https://issues.apache.org/jira/browse/TS-947
> Project: Traffic Server
> Issue Type: Bug
> Components: Core
> Environment: stock build with static libts, running on a 4 core server
>Reporter: B Wyatt
>Assignee: John Plevyak
> Fix For: sometime
>
> Attachments: lock-safe-AIO.patch, timed-wait-AIO.patch
>
>
> Refer to code below. The timeslice starting when a consumer thread
> determines that the temp_list is empty (A) and ending when it releases the
> aio_mutex(C) is unsafe if the work queues are empty and it breaks loop
> execution at B. During this timeslice (A-C) the consumer holds the aio_mutex
> and as a result request producers enqueue items on the temporary atomic list
> (D). As a consumer in this state will wait for a signal on aio_cond to
> proceed before processing the temp_list again, any requests on the temp_list
> are effectively stalled until a future request produces this signal or
> manually processes the temp_list.
> In the case of cache volume initialization, there is no "future request" and
> the initialization sequence soft locks.
> {code:title=iocore/aio/AIO.cc(annotated)}
> void *
> aio_thread_main(void *arg)
> {
> ...
> ink_mutex_acquire(&my_aio_req->aio_mutex);
> for (;;) {
> do {
> current_req = my_aio_req;
> /* check if any pending requests on the atomic list */
> A>>> if (!INK_ATOMICLIST_EMPTY(my_aio_req->aio_temp_list))
> aio_move(my_aio_req);
> if (!(op = my_aio_req->aio_todo.pop()) && !(op =
> my_aio_req->http_aio_todo.pop()))
> B>>>break;
> ...
> <>
> ...
> } while (1);
> C>>>ink_cond_wait(&my_aio_req->aio_cond, &my_aio_req->aio_mutex);
> }
> ...
> }
> static void
> aio_queue_req(AIOCallbackInternal *op, int fromAPI = 0)
> {
> ...
> if (!ink_mutex_try_acquire(&req->aio_mutex)) {
> D>>>ink_atomiclist_push(&req->aio_temp_list, op);
> } else {
> /* check if any pending requests on the atomic list */
> if (!INK_ATOMICLIST_EMPTY(req->aio_temp_list))
> aio_move(req);
> /* now put the new request */
> aio_insert(op, req);
> ink_cond_signal(&req->aio_cond);
> ink_mutex_release(&req->aio_mutex);
> }
> ...
> }
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (TS-996) HTTPHdr::m_host goes stale if HdrHeap::evacuate_from_str_heaps is called
[ https://issues.apache.org/jira/browse/TS-996?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] B Wyatt updated TS-996: --- Attachment: m_host.V2.patch The attached patch is in error. It is missing a (very important) line of code that properly sets m_host_length in the case that the host is in the header fields and does _not_ have a port designation. The V2 patch addresses this > HTTPHdr::m_host goes stale if HdrHeap::evacuate_from_str_heaps is called > > > Key: TS-996 > URL: https://issues.apache.org/jira/browse/TS-996 > Project: Traffic Server > Issue Type: Bug > Components: HTTP, MIME >Affects Versions: 3.1.0 >Reporter: B Wyatt > Attachments: m_host.V2.patch, m_host.patch > > > class HTTPHdr stores a copy of the string pointer from either the URLimpl or > the MIMEHdr for the host name in m_host. In both cases, these strings can be > moved to a new heap underneath the HTTPHdr. When this happens, the process > will, at best read stale memory and be fine and at worst read unmapped memory > and segfault. > Currently, HdrHeap::evacuate_from_str_heaps is called to coalesce multiple > heaps into a single heap. When this happens it will directly access the low > level objects via ::move_strings calls. These objects do not posses the > necessary information to inform parent objects about the change, nor does the > HdrHeap directly inform interested parties. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (TS-996) HTTPHdr::m_host goes stale if HdrHeap::evacuate_from_str_heaps is called
[ https://issues.apache.org/jira/browse/TS-996?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] B Wyatt updated TS-996: --- Attachment: m_host.patch > HTTPHdr::m_host goes stale if HdrHeap::evacuate_from_str_heaps is called > > > Key: TS-996 > URL: https://issues.apache.org/jira/browse/TS-996 > Project: Traffic Server > Issue Type: Bug > Components: HTTP, MIME >Affects Versions: 3.1.0 >Reporter: B Wyatt > Attachments: m_host.patch > > > class HTTPHdr stores a copy of the string pointer from either the URLimpl or > the MIMEHdr for the host name in m_host. In both cases, these strings can be > moved to a new heap underneath the HTTPHdr. When this happens, the process > will, at best read stale memory and be fine and at worst read unmapped memory > and segfault. > Currently, HdrHeap::evacuate_from_str_heaps is called to coalesce multiple > heaps into a single heap. When this happens it will directly access the low > level objects via ::move_strings calls. These objects do not posses the > necessary information to inform parent objects about the change, nor does the > HdrHeap directly inform interested parties. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
